Title: Explaining Datasets in Words: Statistical Models with Natural Language Parameters
Venue: NeurIPS 2024
Accepted: yes
Keywords: language model; explainability; exploratory analysis; data science; explainable modeling
Abstract: To make sense of massive data, we often first fit simplified models and then interpret the parameters; for example, we cluster the text embeddings and then interpret the mean parameters of each cluster.
However, these parameters are often high-dimensional and hard to interpret.
To make model parameters directly interpretable, we introduce a family of statistical models---including clustering, time series, and classification models---parameterized by *natural language predicates*. 
For example, a cluster of text about COVID could be parameterized by the predicate ``*discusses COVID*''.
To learn these statistical models effectively, we develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and discretizes them by prompting language models (LMs).
Finally, we apply our framework to a wide range of problems: taxonomizing user chat dialogues, characterizing how they evolve across time, finding categories where one language model is better than the other, clustering math problems based on subareas, and explaining visual features in memorable images.
Our framework is highly versatile, applicable to both textual and visual domains, can be easily steered to focus on specific properties (e.g. subareas), and explains sophisticated concepts that classical methods (e.g. n-gram analysis) struggle to produce.

Explaining Datasets in Words:

Abstract

To make sense of massive data, we often first fit simplified models and then interpret the parameters; for example, we cluster the text embeddings and then interpret the mean parameters of each cluster. However, these parameters are often highdimensional and hard to interpret. To make model parameters directly interpretable, we introduce a family of statistical models-including clustering, time series, and classification models-parameterized by natural language predicates . For example, a cluster of text about COVID could be parameterized by the predicate ' discusses COVID '. To learn these statistical models effectively, we develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and discretizes them by prompting language models (LMs). Finally, we apply our framework to a wide range of problems: taxonomizing user chat dialogues, characterizing how they evolve across time, finding categories where one language model is better than the other, clustering math problems based on subareas, and explaining visual features in memorable images. Our framework is highly versatile, applicable to both textual and visual domains, can be easily steered to focus on specific properties (e.g. subareas), and explains sophisticated concepts that classical methods (e.g. n-gram analysis) struggle to produce. 2

1 Introduction

To analyze massive datasets, we often fit simplified statistical models and interpret the learned parameters. For example, to categorize a set of user queries, we might cluster their embeddings, look at samples from each cluster, and hopefully each cluster corresponds to an explainable category, e.g. ' asks about COVID symptoms ' or ' discusses the U.S. Election '. Unfortunately, each cluster might contain an uninterpretable group of queries, thus failing to explain the categories.

Such a failure is not an isolated incident: many models explain datasets by learning high dimensional parameters, but these parameters might require significant human effort to interpret. For example, BERTopic [18] learns uninterpretable cluster centers over high-dimensional neural embeddings. LDA [7], Dynamic Topic Modeling [6] (time series), and Naive Bayes (classification) learn weights over a large set of words/phrases, which do not directly explain abstract concepts [9, 52, 63]. We want model parameters that are more interpretable, since explaining datasets is important in machine learning [60], business [4], political discussion [47], and science [17, 34].

To make model parameters directly interpretable, we introduce a family of models where some of their parameters are represented as natural language predicates, which are inherently interpretable. Our core insight is that we can use a predicate to extract a 0/1 feature by checking whether it is true on a sample. For instance, given the predicate ϕ = ' discusses the U.S. Election ', its denotation J ϕ K is a binary function that evaluates to 1 on texts x discussing the U.S. Election and 0 otherwise:

J ϕ : 'discusses the U.S. Election' K ( x : 'Is Georgia a swinging state this year?' ) = 1 .

ϕ

wi

x

i

-1

…

ϕ

w

y

i

x

i

Cluster

descriptions

Cluster

assignment

Text samples

p

(

x

∣

ϕ

1

)

p

(

x

∣

ϕ

2

)

The gap between voters' belief is huge.

Is Georgia a swinging state?

Can I go jogging outdoor during lockdown?

I got COVID. What's fatality rate?

+1

x

t

-1

x

t

x

t

+1

…

ϕ

ϕ

1

ϕ

2

. . .

. . .

=

asks

about

the

U.S.

Election

discusses

pandemics

. . .

. . .

2018

2019

2020

2021

2022

(Year)

t

y

i

=

1

[

x

i

is

asked

by

]

w

= [10, -10]

Ted asks more about the

U.S. Election but not about

the pandemics

Learned

features

Weight evolving

across time

Sample at time

t

Learned

features

Class

labels

Sample

text

Learned

weights

Using these 0/1 feature values, we define a wide variety of models, including clustering, classification, and time series modeling, all parameterized by natural langauage predicates (Figure 1).

Learning these predicates ϕ requires optimizing them to maximize the log-likelihood of the data. This is challenging because ϕ are discrete and thus do not admit gradient-based optimization. We propose a general method to effectively optimize ϕ : we create a continuous relaxation ˜ ϕ of ϕ and optimize ˜ ϕ with gradient descent; then we prompt an LLM to explain the behavior of ˜ ϕ , thus converting it back to discrete predicates (Section 4). We repeat this process to iteratively improve performance.

To evaluate our optimization algorithm, we create statistical modeling problems where the optimal predicate parameters are known, so we can use them as the ground truth. We evaluated on three different statistical models (clustering, multilabel classification, and time series modeling, as illustrated in Figure 1) and used five different datasets (NYT articles, AG-News, DBPedia, Bills, and Wiki [40, 58, 32, 23]). We found that both continuous relaxation and iterative refinement improve performance; additionally, our model-agnostic algorithm matches the performance ( 2% increase in F1 score) of the previous algorithm specialized for explainable text clustering [53].

Finally, we show that our framework is highly versatile by applying it to a wide range of tasks: taxonomizing user chat dialogues [59], characterizing how they evolve, finding categories where one language model is better than the other, clustering math problems [21] based on their subareas, and explaining what visual features make an image memorable [24]. Our framework applies to both text and visual domains, can be easily steered to explain specific abstract properties, and explains complicated concepts that classical methods (e.g. n-gram regression/topic model) struggle to produce. Combining LLM's ability to generate explanations along with traditional statistical models' ability to process sophisticated data patterns, our framework holds the promise to help humans better understand the complex world.

2 Related Work

Statistical Modeling in Text. Statistical models based on n-gram features or neural embeddings are broadly used to analyze text datasets. For example, logistic regression or naïve Bayes models are frequently used to explain differences between text distributions [51]; Gaussian mixture models on pre-trained embeddings can create text clusters [2]; topic models can mine major topics across a large collection of documents [7] and across time [6]. However, since these models usually rely on high-dimensional parameters, they are difficult to interpret: for example, human studies from [9] show that the most probable words for a topic might not form a semantically coherent category. To interpret these models, prior works proposed to explain each topic or cluster by extracting candidate phrases either from the corpus or from Wikipedia [8, 49, 57]. Our work complements these approaches to explain models with natural language predicates, which are potentially more flexible.

Prompting Language Model to Explain Dataset Patterns. Our algorithm heavily relies on the ability of LLMs to explain distributional patterns in data when prompted with datasets [39, 46]. [61, 62, 15] have prompted LLMs to explain differences between two text distributions; [53, 38, 50, 27] prompted LLMs to generate topic descriptions over unstructured texts; [44, 22, 64] prompted LLMs to explain the function that maps from an input to an output; [45, 5] prompted LLMs to explain what inputs activate a direction in the neural embedding space. However, these works focused on individual applications or models in isolation; in contrast, our work creates a unifying framework to define and learn more complex models (e.g. time series) with natural language parameters.

Concept Bottleneck Models (CBM). CBMs aim to achieve explainability by learning a simple model over a set of interpretable features [26], and recent works have proposed to extract these features using natural language phrases/predicates [3, 55, 30, 12, 41]. While most of these works focus on classification tasks, our work formalizes a broad family of models-including clustering and time series -and proposes a model-agnostic algorithm to learn them. Additionally, these prior works focus on downstream task performance (e.g. classification accuracy), thus implicitly assuming that the model grounds the feature explanations in the same way as humans; in contrast, since our focus is on explanations, we focus on our algorithm's ability to recover ground truth explainable features.

We discuss more related work on discrete prompt optimization, exploratory analysis, and learning with latent language in Appendix A.

3 Mathematical Formulation

3.1 Predicate-Conditioned Distribution

In order to model text distributions with natural language parameters, we introduce a new family of distributions, predicate-conditioned distributions ; these distributions will serve as building blocks for the models introduced later, just like normal distributions are building blocks for many classical models like Gaussian Mixture or Kalman Filter. Predicate-conditioned distributions p are supported on the set X of all the text samples we observe from the dataset, and they are parameterized by (1) a list of K predicates ⃗ ϕ ∈ Φ K , and (2) real-valued weights w ∈ R K on those predicates. Formally,

We now explain how to (1) extract a feature vector from x using ⃗ ϕ , (2) linearly combine ⃗ ϕ by re-weighting with w , and (3) use the reweighted values to define p ( x | w, ⃗ ϕ ) .

Natural Language Parameters ⃗ ϕ . Each predicate ϕ ∈ Φ is a natural language string and its denotation J ϕ K : X →{ 0 , 1 } maps samples to their value under the predicate. For example, if ϕ = ' is sports-related ', then J ϕ K (' I love soccer. ') = 1 . Since a model typically requires multiple features to explain the data, we consider vectors ⃗ ϕ ∈ Φ K of K predicates, where now J ⃗ ϕ K maps X to { 0 , 1 } K :

To instantiate J · K computationally, we prompt a language model to check whether ϕ is true on the input x , following the practice from prior works [61, 62]. See Figure 2 (left) for the prompt we used.

Reweighting with w . Consider the following example:

Then w T J ⃗ ϕ K has a value of -5 · 1 + 3 · 0 = -5 for an English, non-sports related sample x . More generally, w T J ⃗ ϕ K ( x ) is larger for non-English sports-related samples.

Defining p ( x | ⃗ ϕ, w ) . According to Equation 1, p ( x | ⃗ ϕ, w ) is a distribution over X , all the text samples we observe, but it puts more weights on x with higher values of w T J ⃗ ϕ K ( x ) . Using the example w and ⃗ ϕ above, p ( x | ⃗ ϕ, w ) has higher probability on non-English sports-related texts.

Finally, we define U ( x ) as the uniform distribution over X for later use.

Discretization: Discretize ( ˜ ϕ )

Check whether the TEXT satisfies a PROPERTY.

Respond with Yes or No. When uncertain, output No. Now

complete the following

input: PROPERTY: is sports-related

TEXT: 'I lover soccer'

Output:

yes

Here is a corpus of text samples, sorted from the lowest to the highest score.

Sample 0. 'athlete demonstrated remarkable prowess.' (score: -0.2)

Sample 1. 'see the player?' (score: -0.3)

…

…

Sample 9. 'Wonderful painting …' (score: 0.4)

Please suggest predicates about the text samples that are more likely to

achieve higher scores.

Your responses are:

-

'has a topic of art'

-

'has a topic of sports'

-

….

Denotation:

ϕ

x

x

∼

U

(

x

)

↑

cos

(

e

x

,

˜

ϕ

)

↑

[[

ϕ

]](

x

)

The input variables illustrated in blue and the output of each prompt in

bold

3.2 Example Models Parameterized by Natural Language Predicates

We introduce three models parameterized by predicates: clustering, time series, and multi-label classification. For each model, we explain its input, the learned parameters ⃗ ϕ and w , the log-likelihood loss L , and its relation to classical models.

Clustering. This model aims to help humans explore a large corpus by creating clusters, each explained by a predicate. Such a model may help humans obtain a quick overview for a large set of machine learning inputs [60], policy discussions [47], or business reviews [4]. Given a set of text X , our model produces a set of K clusters, each parameterized by a learned predicate ϕ k ; for example, if the predicate is ' discusses the U.S. Election ', then the corresponding cluster is a uniform distribution over all samples in X that discuss the U.S. Election.

Similar to K-means clustering, each sample x is assigned to a unique cluster. We use a one-hot basis vector b x ∈ R K to indicate the cluster assignment of x , and set w x = τ · b x , where τ has a large value (e.g. 10). We maximize the total log-likelihood:

However, some samples might not belong to any cluster and thus have 0 probability; to prevent infinite loss, we add another 'background cluster' U ( x ) that is uniform over all samples in X ; therefore, each sample x can back off to this cluster and incur a loss of at most -log U ( x ) = log( | X | ) .

Time Series Modeling. This model aims to explain latent variations in texts that change across time; for example, finding that an increasing number of people 'search about flu symptons' ( ϕ ) can help us forecast a potential outbreak [16]. Formally, the input is a sequence of T text samples X = { x t } T t =1 . Our model produces K predicates ϕ k that capture the principle axes of variation in x across time. We model w 1 . . . w T as being drawn from a Brownian motion, i.e.,

where λ is a real-valued hyper-parameter that regularizes how fast w can change. The loss L is hence

Multiclass Classification with Learned Feature Predicates. This model aims to explain the decision boundary between groups of texts, e.g. explaining what features are more correlated with the fake news class [35] compared to other news, or explaining what activates a neuron [5]. Suppose there are C classes in total; the dataset is a set of samples x i each associated with a class y i ∈ [ C ] . Our model is hence a linear logistic regression model on the feature vectors extracted by ⃗ ϕ , i.e.

where W ∈ R C × K is the weight matrix for logistic regression.

4 Method

We can now learn the parameters for each model above by minimizing the loss function L . Formally,

However, optimizing ⃗ ϕ is challenging, since it is discrete and therefore cannot be directly optimized by gradient-based methods. To address this challenge, we develop a general optimization method, which we describe at a high level in Section 4.1, introduce its individual components in Section 4.2, and explain our full algorithm in Section 4.3.

4.1 High-Level Overview

Our framework pieces together three core functions that require minimal model-specific design:

OptW , which optimizes w .

OptRelaxedPhi , which optimizes a continuous relaxation ˜ ϕ k for each predicate ϕ k .

Discretize , which maps from continuous predicate ˜ ϕ k to a list of candidate predicates.

Using these three components, our overall method initializes the set of predicates by first optimizing w and ˜ ϕ using OptW and OptRelaxedPhi and then discretizing ˜ ϕ with Discretize . To further improve the loss, it then iteratively removes the least useful predicate, re-optimizes its continuous representation, and discretizes it back to a natural language predicate.

To provide more intuition for these three components, we explain what they should achieve in the context of clustering. OptW should optimize the 1-hot choice vectors w x by assigning each text sample to the cluster with maximum likelihood. OptRelaxedPhi should find a continuous cluster representation ˜ ϕ k similar to the sample embeddings assigned to this cluster, and Discretize generates candidate predicates that explain which samples' embeddings are similar to ˜ ϕ k . Next, we introduce these three components formally for general models with predicate parameters.

4.2 Three Components of our framework

OptW optimizes w while fixing the values of ⃗ ϕ . Formally, OptW ( ⃗ ϕ ) := argmin w L ( ⃗ ϕ, w ) .

This function needs to be designed by the user for every new model, but it is generally straightforward: in the clustering model, it corresponds to finding the cluster that assigns the highest probability for each sample; in classification, it corresponds to learning a logistic regression model; in the time series model, the loss is convex with respect to w and hence can be optimized via gradient descent.

For later use, we define the fitness of a list of predicates ⃗ ϕ as the negative loss after w is optimized:

Next, we discuss OptRelaxedPhi . The parameters ⃗ ϕ are discrete strings, so the loss function is not differentiable with respect to ⃗ ϕ . To address this, we approximate J ⃗ ϕ K ( x ) with the dot product of two continuous vectors, ˜ ϕ k · e x , where e x ∈ R d is a feature embedding of x normalized to unit length (e.g. the last-layer activations of some neural network), and ˜ ϕ k ∈ R d is a unit-length, continuous relaxation of ϕ k . Intuitively, if the optimal ϕ = ' is sports-related ' and x is a sports-related sample with J ϕ K ( x ) = 1 , then we hope that ˜ ϕ would correspond to the latent direction encoding the sports topic and it has high similarity with the embedding e x of x . Under this relaxation, L becomes differentiable with respect to ˜ ϕ k and can be optimized with gradient descent.

Formally, OptRelaxedPhi optimizes all continuous predicates ˜ ϕ 1 ...K given a fixed value of w :

We sometimes also use it to optimize a single continuous predicate ˜ ϕ k given a fixed w and all discrete predicate variables other than ϕ k (denoted as ϕ -k ):

Finally, Discretize converts ˜ ϕ k into a list of M discrete candidate predicates to update the variable ϕ k . Our goal is to find ϕ whose denotation is highly correlated with the dot product simulation ˜ ϕ k · e x .

To discretize ˜ ϕ k , we prompt a language model to generate several candidate predicates and then re-rank them. Concretely, we draw samples x ∼ U ( x ) 3 and sort them based on their dot product ˜ ϕ k · e x . We then prompt a language model with these sorted samples and ask it to generate candidate predicates that can explain what types of samples are more likely to appear later in the sorted list (Figure 2 bottom). To filter out unpromising predicates, we re-rank them based on the pearson-r correlations between J ϕ K and ˜ ϕ k · e x on U if w cannot be negative (e.g. clustering), and the absolute value of pearson-r correlation otherwise. We then keep the topM predicates.

4.3 Piecing the Three Components Together

Our algorithm has two stages: we first initialize all the predicate variables and then iteratively refine each of them. During initialization, we

randomly initialize continuous predicates ˜ ϕ to be the embedding of random samples from X

optimize L ( ˜ ϕ, w ) by alternatively optimizing w and all the continuous predicates ˜ ϕ with OptW and OptRelaxedPhi , and

set ϕ k as the first candidate from Discretize ( ˜ ϕ k )

During refinement, we repeat the following steps for S iterations:

find the least useful predicate ϕ k ; we define the usefulness of ϕ k as how much the fitness would decrease if we zero it out, i.e. -Fitness ( ⃗ ϕ -k , 0) .

optimize ˜ ϕ k using OptRelaxedPhi and choose the fittest predicate from Discretize ( ˜ ϕ k )

We include a formal description of our algorithm in Appendix Algorithm H.

5 Experiments

In this section, we benchmark our algorithm from Section 4; we later apply it to open-ended applications in Section 6. We run our algorithm on datasets where we know the ground truth predicates ⃗ ϕ and evaluate whether it can recover them. On five datasets and three statistical models, continuous relaxation and iterative refinement consistently improve performance. Our general method also matches a previous specialized method for explainable clustering [53].

5.1 Datasets

We design a suite of datasets for each of the three statistical models mentioned from Section 3.2. Each dataset has a set of reference predicates, and we evaluate our algorithm's ability to recover them.

Clustering. We consider five datasets, AGNews , DBPedia , NYT , Bills , and Wiki [40, 58, 32, 23]. The datasets have 4/14/9/21/15 topic classes each described in a predicate, and we sample 2048 examples from each for evaluation.

Multiclass Classification. We design a classification dataset with 5,000 articles and 20 classes; its goal is to evaluate a method's ability to recover the latent interpretable features useful for classification. Therefore, we design each class to be a set of articles that satisfy three predicates about its topic, location, and language; for example, one of the classes can be described by the predicates ' has a topic of sports ', ' is in Japan ', and ' is written in English '. We create this dataset by adapting the New York Times Articles dataset [40], where each article is associated with a topic and a location predicate; we then translate them into Spanish, French, and Deutsch. We consider in total 4 + 4 + 4 = 12 different predicates for each of the topic/location/language attributes and subsample 20 classes from all 4 × 4 × 4 = 64 combinations.

Time Series modeling. We synthesize a time series problem by further adapting the translated NYT dataset above. We set the total time T = 2048 and sample x 1 . . . x T according to the time series model in Section 3.2 to create the benchmark. We set ⃗ ϕ to be the 12 predicates mentioned above and the weight w · ,k for each predicate ϕ k to be a cosine function with a period of T to simulate how each attribute evolves throughout time. In addition, we included three simpler datasets where there is only variation on one attribute (i.e. varies only on one of topic/location/language). We name these four time series modeling all , topic , locat , and lang , respectively. See Appendix B for a more detailed explanation.

5.2 Metrics

To evaluate our algorithm, we match each learned predicate ˆ ϕ k with a reference ϕ ∗ k ′ , compute the F1-score and surface similarity for each pair, and then report the average across all pairs. To create the matching, we match ˆ ϕ k to the ϕ ∗ k ′ with the highest overlap (number of samples where both are true); formally, we define a bi-partite matching problem to match each predicate in ˆ ϕ with one in ϕ ∗ , define the weight of matching ϕ ∗ k ′ and ϕ ∗ k ′ to be their overlap, and then find the maximum weight matching via the Hungarian algorithm. We now explain the F1-score and surface similarity metric.

F1-score Similarity. We compute the F1-score of using ˆ ϕ ( x ) to predict ϕ ∗ ( x ) on X , the set of samples we observe. This is similar to the standard protocol for evaluating cluster quality [28].

Surface Form Similiarity . We can also directly evaluate the similarity between two predicates based on their string values, e.g. ' is about sports ' is similar in meaning to ' has a topic of sports ', a metric previously used by [62]. For a pair of predicates, we ask gpt-4 to evaluate whether they are similar in meaning, related, or irrelevant, with each option associated with a surface-similarity score of 1/0.5/0. We display the prompt in Figure 5 and example ratings in Table 1.

5.3 Experiments on Our Benchmark

We now use these metrics and datasets to evaluate the optimization algorithm proposed in Section 4 and run ablations to investigate whether continuous relaxation and iterative refinement are effective. Wewill first introduce the overall experimental setup, and then discuss individual takeaways supported by experimental results in each paragraph.

Experimental Setup. When running the algorithm, we generate candidate predicates in Discretize with gpt-3.5-turbo [37]; to perform the denotation operation J ϕ K ( x ) , we use flan-t5-xl [13]; we create the embedding for each sample x with the Instructor-xl model [48] and then normalize it with ℓ 2 norm. We set the number of candidates M returned by Discretize to be 5 and the number of optimization iteration S to be 10. To reduce noises due to randomness, we average the performance of five random seeds for each experiment.

Table 2 reports the results of clustering and Table 3 reports other results. For each dataset, we perform several ablation experiments and present the takeaways from these results.

Takeaway 0: Is our method better than naïvely prompting language model to generate predicates? How does our approach compare to a naïve baseline approach, which directly prompts the language model to generate predicates based on dataset samples? For this baseline, we repeatedly prompt a language model to generate more predicates until we obtain K predicates, compute their denotation, evaluate them using the metrics in Section 5.2, and report the performance in Table 2 and 5, the Prompting row. Across all entries, our approach significantly outperforms this baseline.

Takeaway 1: Relax + discretize is better than exploring randomly generated predicates. Our optimization algorithm explores the top-5 LLM-generated predicates that have the highest correlations with ˜ ϕ k · e x . Would choosing a random predicate be equally effective? To investigate this question, we experimented with a variant of our algorithm that randomly chooses five predicates without utilizing the continuous representation ˜ ϕ k ( No-Relax ). In Table 2 and 3, No-Relax underperforms our full algorithm ( Ours ) in all cases. In Appendix Figure 6, we plot the loss after each iteration averaged across all tasks, and we find that Ours converges much faster than No-Relax .

Takeaway 2: Iterative refinement improves the performance. We considered a variant of our algorithm that only discretizes the initial continuous representations and does not iteratively refine the predicates ( No-Refine ). In Table 2 and 3, No-Refine underperforms the full algorithm in all cases.

Takeaway 3: Our model-agnostic method is competitive with previous methods specialized for explainable clustering. We compare our method to GoalEx from [53], which designs a specialized method for explainable clustering based on integer linear programming. Even though our method is model-agnostic, it matches or outperforms GoalEx on four out of five datasets and improves F1 by 0 . 02 on average.

Takeaway 4: Our method accounts for information beyond the set of text samples (e.g. temporal correlations in the time series). We investigate this claim using the time series datasets, where we shuffle the text order and hence destroy the time-dependent information a model could use to extract informative predicates ( Shuffled ). Table 3 finds that Ours is better than Shuffled in all cases, indicating that our method does make use of temporal correlations.

Appendix D includes additional results: 1) compared to topic modeling and K-means, our method achieves similar or better performance while being explainable; 2) we ran ablations on the effect of neural embeddings and show that informative embeddings are crucial to good performance; 3) Takeaways 1, 2, and 4, are significant with p < 1% under paired t-tests.

Cluster 2: request information or

answers regarding a specific topic

Root

Cluster 1 format a description

of an image generation prompt;

Cluster 1.A: the user wants

to request graphic design

prompts for Midjourney AI

Cluster 2.A ask for

historical information

on a specific topic

Cluster 2.B request

technical issue

resolution

Topic 2: jane year

14 old game …

Root

Topic 1: ar prompt

description detailed

…

Topic 1.A: title generate

post short …

Topic 1.B: create

little sky similar …

Topic 2.A: 2023 scale

world video …

Ours

Topic Model

6 Open-Ended Applications

We apply our framework to a broad range of applications to show that it is highly versatile. Our framework can monitor data streams (Section 6.1), apply to the visual domain (Section F.1), and be easily steered to explain specific abstract properties (Section F.2). Across all applications, our framework is able to explain complex concepts that classical methods struggle to produce.

6.1 Running Our Models Out of the Box: Monitoring Complex Data Streams of LLM Usages

We apply our models from Section 3.2 to monitor complex data streams of LLM usages. In particular, we recursively apply our clustering model to taxonomize user queries into application categories, apply our time series model to characterize trends in use cases across time, and apply our classification model to find categories where one LLM is better than the other. Due to space constraints, we present the key results in the main paper and the full results in Appendix G.

Taxonomizing User Applications via Clustering. LLMs are general-purpose systems, and users might applyLLMs in ways unanticipated by the developers. If the developers can better understand how theLLMs are used, they could collect training data correspondingly, ban unforeseen harmful applications, or develop application-specific methods. However, the amount of user queries is too large for individual developers to process, so an automatically constructed taxonomy could be helpful.

We recursively apply our clustering model to user queries to the ChatGPT language model. We obtain the queries by extracting the first turns from the dialogues in WildChat [59], a corpus of 1M real-world user-ChatGPT dialogues. We use gpt-4o [36] to discretize and claude-3.5-sonnet [1] to compute denotations. We first generate K = 6 clusters on a subset of 2048 queries; then we generate K = 4 subclusters for each cluster with > 32 samples.

We present part of the taxonomy in Figure 3 (left) and contrast it with the taxonomy constructed by directly applying LDA recursively (right). Although some LDA topics are plausibly related to certain applications, they are still ambiguous; for example, it is unclear what topic 1 ' ar prompt description detailed ' means. After manually inspecting the samples associated with this topic, we found that they were related to the application of writing prompts for an image-generation model. In contrast, our framework can explain complicated concepts that are difficult to infer from individual words; for example, it generates ' requesting graphic design prompts ' for the above application, which is much clearer in its meaning when explained in natural language.

Characterizing Temporal Trends via Time Series Modeling. Understanding temporal trends in user queries can help forecast flu outbreaks [16], prevent self-reinforcing trends [19], or identify new application opportunities. We run our time series model on 1000 queries from WildChat with K = 4 to identify temporal trends in user applications, and report part of the results in Figure 4. Based on the blue curve, we find that an increasing number of users ' requests writing or content creation .... creating stories based on given prompts. '. This helps motivate systems like Coauthor [29] to assist with this use case.

Finding Categories where One Language Model is Better than the Other. One popular method to evaluateLLMs is crowd-sourcing: an evaluation platform (e.g. ChatBotArena [11]) / or a company (e.g. OpenAI) accepts prompts from users, shows users responses from two different LLM systems,

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

Percentile of turns sorted by time

0

.

1

0

.

2

0

.

3

0

.

4

Frequency

Trends on WildChat

φ

0

:

request information or translation in a different language;

i.e.

the query involves requesting content in

another language or translating text

φ

1

:

engage in academic or scholarly discussion;

i.e.

the query involves seeking information or

topics related to academic subjects or discussions

φ

2

:

request writing or content creation in a specific style or genre;

i.e.

the query involves asking for creating

stories or other written content based on given

prompts

φ

3

:

request general information or answers to factual questions;

i.e.

the query involves seeking answers to broad

or factual questions

and the users indicate which one they like better. The ranking among the LLM systems is then determined by Elo-rating, i.e. how often they win against each other.

However, aggregate Elo-rating omits subtle differences between LLM systems. For example, LLama3-70B achieved a similar rating as Claude-3-Opus, and the LLM community was excited that open-weight models were catching up. However, is LLama-3-70B similarly capable across all categories, or is it significantly better/worse under some categories? Such information is important for downstream developers, since some capabilities are more commercially valuable than others: e.g. a programmer usually does not care about LLM's capability to write jokes. We need a more fine-grained comparison.

We directly apply the classification model from our framework to solve this task. To understand the categories where LLama-3-70B is better/worse than Claude-3-Opus, we gather user queries x from the ChatBotArena maintainers (personal communication), set y = 1 if the LLama-3-70B's response to x is preferred and y = 0 otherwise. We set K = 3 .

Our model finds that LLama-3-70B is better when the query ' asks an open-ended or thoughtprovoking question ' but worse when it ' presents a technical question ' or ' contains code snippets '. These findings are corroborated by manual analysis by the ChatBotArena maintainers, who also found that Llama-3 is better at open-ended and creative tasks while worse at technical problems 4 . We hope that our model can automatically generate similar analysis in the future when a new LLM is released, thus saving researchers' efforts.

To summarize, our framework 1) enables us to define a time series model to explain temporal trends in natural language, and 2) outputs sophisticated explanations that LDA fails to generate. However, it is far from perfect: it is slow to compute denotations for all pairs of x and candidates ϕ since it involves many LLM API calls, and the predicates themselves are sometimes redundant. We describe these limitations and potential ways to improve them in Appendix G.

Due to space constraints, we present applications in explaining visual features to make images memorable to humans and clustering math problems based on subareas in Appendix F.1 and F.2.

7 Conclusion

In this work, we formalize a broad family of models parameterized by natural language predicates. We design a learning algorithm based on continuous relaxation and iterative refinement, both of them effective based on our ablation studies. Finally, we apply our framework to a wide range of applications, showing that it is highly versatile, practically useful, applicable to both text and vision domains, and explains sophisticated concepts that classical methods struggle to produce. We hope future works can make our method more computationally efficient and apply it to more realistic applications, thus assisting humans to discover and understand complex patterns in the world.

References

URL https://www.anthropic.com/news/claude-3-5-sonnet .

Roee Aharoni and Yoav Goldberg. Unsupervised domain clusters in pretrained language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 7747-7763, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/ v1/2020.acl-main.692. URL https://aclanthology.org/2020.acl-main.692 .

Jacob Andreas, Dan Klein, and Sergey Levine. Learning with latent language. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) , pages 2166-2179, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/ N18-1197. URL https://aclanthology.org/N18-1197 .

Kasturi Bhattacharjee, Rashmi Gangadharaiah, Kathleen McKeown, and Dan Roth. What do users care about? detecting actionable insights from user feedback. In Anastassia Loukina, Rashmi Gangadharaiah, and Bonan Min, editors, Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track , pages 239-246, Hybrid: Seattle, Washington + Online, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-industry.27. URL https://aclanthology.org/2022.naacl-industry.27 .

Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models. URL https://openaipublic. blob. core. windows. net/neuronexplainer/paper/index. html.(Date accessed: 14.05. 2023) , 2023.

David M Blei and John D Lafferty. Dynamic topic models. In Proceedings of the 23rd international conference on Machine learning , pages 113-120, 2006.

David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. Journal of machine Learning research , 3(Jan):993-1022, 2003.

David Carmel, Haggai Roitman, and Naama Zwerdling. Enhancing cluster labeling using wikipedia. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval , pages 139-146, 2009.

Jonathan Chang, Sean Gerrish, Chong Wang, Jordan Boyd-Graber, and David Blei. Reading tea leaves: How humans interpret topic models. Advances in neural information processing systems , 22, 2009.

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 , 2021.

Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E Gonzalez, et al. Chatbot arena: An open platform for evaluating llms by human preference. arXiv preprint arXiv:2403.04132 , 2024.

Mia Chiquier, Utkarsh Mall, and Carl Vondrick. Evolving interpretable visual classifiers with large language models. arXiv preprint arXiv:2404.09941 , 2024.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 , 2022.

Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. RLPrompt: Optimizing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 3369-3391, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.emnlp-main.222. URL https://aclanthology.org/2022.emnlp-main.222 .

Lisa Dunlap, Yuhui Zhang, Xiaohan Wang, Ruiqi Zhong, Trevor Darrell, Jacob Steinhardt, Joseph E Gonzalez, and Serena Yeung-Levy. Describing differences in image sets with natural language. arXiv preprint arXiv:2312.02974 , 2023.

Jeremy Ginsberg, Matthew H Mohebbi, Rajan S Patel, Lynnette Brammer, Mark S Smolinski, and Larry Brilliant. Detecting influenza epidemics using search engine query data. Nature , 457 (7232):1012-1014, 2009.

Thomas L Griffiths and Mark Steyvers. Finding scientific topics. Proceedings of the National academy of Sciences , 101(suppl_1):5228-5235, 2004.

Maarten Grootendorst. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794 , 2022.

Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning , pages 1929-1938. PMLR, 2018.

Md Mahadi Hassan, Alex Knipper, and Shubhra Kanti Karmaker Santu. Chatgpt as your personal data scientist. arXiv preprint arXiv:2305.13657 , 2023.

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874 , 2021.

Or Honovich, Uri Shaham, Samuel R. Bowman, and Omer Levy. Instruction induction: From few examples to natural language task descriptions. In Annual Meeting of the Association for Computational Linguistics , 2022. URL https://api.semanticscholar.org/CorpusID: 248986755 .

Alexander Miserlis Hoyle, Pranav Goel, Rupak Sarkar, and Philip Resnik. Are neural topic models broken? In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Findings of the Association for Computational Linguistics: EMNLP 2022 , pages 5321-5344, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/ v1/2022.findings-emnlp.390. URL https://aclanthology.org/2022.findings-emnlp. 390 .

Phillip Isola, Devi Parikh, Antonio Torralba, and Aude Oliva. Understanding the intrinsic memorability of images. Advances in neural information processing systems , 24, 2011.

Siddharth Karamcheti, Megha Srivastava, Percy Liang, and Dorsa Sadigh. Lila: Languageinformed latent actions. In Conference on Robot Learning , pages 1379-1390. PMLR, 2022.

Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference on machine learning , pages 5338-5348. PMLR, 2020.

Michelle S Lam, Janice Teoh, James Landay, Jeffrey Heer, and Michael S Bernstein. Concept induction: Analyzing unstructured text with high-level concepts using lloom. arXiv preprint arXiv:2404.12259 , 2024.

Tilman Lange, Volker Roth, Mikio L Braun, and Joachim M Buhmann. Stability-based validation of clustering solutions. Neural computation , 16(6):1299-1323, 2004.

Mina Lee, Percy Liang, and Qian Yang. Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI conference on human factors in computing systems , pages 1-19, 2022.

Josh Magnus Ludan, Qing Lyu, Yue Yang, Liam Dugan, Mark Yatskar, and Chris CallisonBurch. Interpretable-by-design text classification with iteratively generated concept bottleneck. arXiv preprint arXiv:2310.19660 , 2023.

Pingchuan Ma, Rui Ding, Shuai Wang, Shi Han, and Dongmei Zhang. Demonstration of insightpilot: An llm-empowered automated data exploration system. arXiv preprint arXiv:2304.00477 , 2023.

Stephen Merity, Nitish Shirish Keskar, and Richard Socher. Regularizing and optimizing LSTM language models. In International Conference on Learning Representations , 2018. URL https://openreview.net/forum?id=SyyGPP0TZ .

Jesse Mu, Percy Liang, and Noah Goodman. Shaping visual representations with language for few-shot classification. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 4823-4830, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/ 2020.acl-main.436. URL https://aclanthology.org/2020.acl-main.436 .

Dong Nguyen, Maria Liakata, Simon DeDeo, Jacob Eisenstein, David Mimno, Rebekah Tromble, and Jane Winters. How we do things with words: Analyzing text as social and cultural data. Frontiers in Artificial Intelligence , 3:62, 2020.

Pontus Nordberg, Joakim Kävrestad, and Marcus Nohlberg. Automatic detection of fake news. In 6th International Workshop on Socio-Technical Perspective in IS Development, virtual conference in Grenoble, France, June 8-9, 2020 , pages 168-179. CEUR-WS, 2020.

OpenAI. Gpt-4. https://platform.openai.com/docs/models/gpt-4 , 2023. Accessed: 2024-05-14.

OpenAI. Gpt-3.5 turbo. https://platform.openai.com/docs/models/gpt-3-5 , 2024. Accessed: 2024-05-14.

Chau Minh Pham, Alexander Hoyle, Simeng Sun, and Mohit Iyyer. Topicgpt: A prompt-based topic modeling framework. arXiv preprint arXiv:2311.01449 , 2023.

Linlu Qiu, Liwei Jiang, Ximing Lu, Melanie Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang, Yoon Kim, Yejin Choi, Nouha Dziri, and Xiang Ren. Phenomenal yet puzzling: Testing inductive reasoning capabilities of language models with hypothesis refinement. In The Twelfth International Conference on Learning Representations , 2024. URL https:// openreview.net/forum?id=bNt7oajl2a .

Evan Sandhaus. The new york times annotated corpus. Linguistic Data Consortium, Philadelphia , 6(12):e26752, 2008.

Simon Schrodi, Julian Schur, Max Argus, and Thomas Brox. Concept bottleneck models without predefined concepts. arXiv preprint arXiv:2407.03921 , 2024.

Pratyusha Sharma, Antonio Torralba, and Jacob Andreas. Skill induction and planning with latent language. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 1713-1726, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.120. URL https: //aclanthology.org/2022.acl-long.120 .

Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980 , 2020.

Chandan Singh, John X Morris, Jyoti Aneja, Alexander M Rush, and Jianfeng Gao. Explaining patterns in data with language models via interpretable autoprompting. arXiv preprint arXiv:2210.01848 , 2022.

Chandan Singh, Aliyah R Hsu, Richard Antonello, Shailee Jain, Alexander G Huth, Bin Yu, and Jianfeng Gao. Explaining black box text modules in natural language with language models. arXiv preprint arXiv:2305.09863 , 2023.

Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. Rethinking interpretability in the era of large language models. arXiv preprint arXiv:2402.01761 , 2024.

Christopher T Small, Ivan Vendrov, Esin Durmus, Hadjar Homaei, Elizabeth Barry, Julien Cornebise, Ted Suzman, Deep Ganguli, and Colin Megill. Opportunities and risks of llms for scalable deliberation with polis. arXiv preprint arXiv:2306.11932 , 2023.

Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. One embedder, any task: Instruction-finetuned text embeddings. 2022. URL https://arxiv.org/abs/2212.09741 .

Pucktada Treeratpituk and Jamie Callan. Automatically labeling hierarchical clusters. In Proceedings of the 2006 international conference on Digital government research , pages 167176, 2006.

Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence, Tongshuang Wu, and Graham Neubig. Large language models enable few-shot clustering. arXiv preprint arXiv:2307.00524 , 2023.

Sida I Wang and Christopher D Manning. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages 90-94, 2012.

Zihan Wang, Jingbo Shang, and Ruiqi Zhong. Goal-driven explainable clustering via language descriptions. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages 10626-10649, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023. emnlp-main.657. URL https://aclanthology.org/2023.emnlp-main.657 .

Zihan Wang, Jingbo Shang, and Ruiqi Zhong. Goal-driven explainable clustering via language descriptions. arXiv preprint arXiv:2305.13749 , 2023.

Lionel Wong, Jiayuan Mao, Pratyusha Sharma, Zachary S Siegel, Jiahai Feng, Noa Korneev, Joshua B Tenenbaum, and Jacob Andreas. Learning adaptive planning representations with natural language guidance. arXiv preprint arXiv:2312.08566 , 2023.

Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19187-19197, 2023.

Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. Large language models for automated open-domain scientific hypotheses discovery. arXiv preprint arXiv:2309.02726 , 2023.

Chao Zhang, Fangbo Tao, Xiusi Chen, Jiaming Shen, Meng Jiang, Brian Sadler, Michelle Vanni, and Jiawei Han. Taxogen: Unsupervised topic taxonomy construction by adaptive term embedding and clustering. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 2701-2709, 2018.

Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems , 28, 2015.

Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: 1m chatGPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview.net/forum?id=Bl8u7ZRlbM .

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems , 36, 2024.

Ruiqi Zhong, Charlie Snell, Dan Klein, and Jacob Steinhardt. Describing differences between text distributions with natural language. In International Conference on Machine Learning , pages 27099-27116. PMLR, 2022.

Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, and Jacob Steinhardt. Goal driven discovery of distributional differences via language descriptions. arXiv preprint arXiv:2302.14233 , 2023.

Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, and Jacob Steinhardt. Goal driven discovery of distributional differences via language descriptions. Advances in Neural Information Processing Systems , 36, 2024.

Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910 , 2022.

A More Related Work

LLMfor Exploratory Analysis. Due to its code generation capability [10], large language models have been used to automatically generate programs to analyze a dataset and generate reports from them [31, 20]. In comparison, our work focuses on generating natural language parameters to extract real-valued features from structured data.

Discrete Prompt Optimization. Many prior works optimized discrete prompts to improve the predictive performance [43, 14], and some recent works demonstrated that LLMs can optimize prompts to reach state-of-the-art accuracy [64, 56]. In comparison, we focus on optimizing discrete predicates to explain patterns rather than improve task performance.

Learning with Latent Language. [3] first proposed to learn in a hypothesis space of natural language strings to improve generalization, and later works in this area have focused on using natural language to guide the learning process to improve downstream task performance [33, 25, 42, 54]. In contrast, our work focuses on explaining datasets with natural language, rather than improving downstream task performance.

B Time Series Dataset

To sample texts from the All time series problem, we sample from the time series model described in Section 3.2; we set ⃗ ϕ to be all the 12 predicates, sort them first by attributes (e.g. topic/location/language) then alphabets, and we set the weight for the k th predicate to be a sin function with period T and evenly spaced phases, i.e.

As a result, the weight for each predicate has evenly spaced phases and would peak at different time period.

C Surface form similarity prompt

We include our prompt used to evaluate the surface form similarity between the predicted predicate ˆ ϕ k and the reference predicate ϕ ∗ k in Figure 5.

̂

Similarity

Is text_a and text_b similar in meaning?

respond with yes, related, or no.

Target:

text_a: has a topic of sports

text_b: is about sports

output:

yes

LM's

generation

ϕ

*

ϕ

{

D Additional Results on Our Benchmark

Our method is similar or better than classical methods such as topic modeling or K-means. We report the performance of K-means clustering and topic modeling under the clustering benchmark

in Table 4. on average, our method is close to K-means and significantly outperforms TopicModel under the F1 similarity metric, while generating natural language explanations for each cluster.

Takeaway 5: Using informative text embedding is crucial to performance. We used neural embeddings when optimizing the continuous representation of the predicates. Does our algorithm actually make use of the information in the feature embeddings? To investigate this question, we ran an ablation of using one-hot text embeddings rather than neural embeddings ( OneHot ), which do not encode any information about the similarity between text samples. We report the performance in Table 4 and 5; across all settings, using neural embeddings consistently outperforms OneHot .

To make sure that this takeaway is general and not specific to one embedding model, we run our method with another text embedding model, all-mpnet-base-v2 5 and report the performance as

Speed of optimization

6.1

Ours

No-relax

6.0

5.9

Loss

5.8

5.7

5.6

0

2

4

6

8

Iteration

the OtherEmb row. We find that using this neural embedding also outperforms OneHot in most cases, indicating that our conclusion is robust.

Takeaway 1,2,and 4 are statistically significant. To compare the performance between our method and each variant, we conduct a one-sided paired t-test on their performance (F1-similarity) on each dataset, where the performance on each dataset is the averaged performance across five runs. Takeaway 1, 2, 4 has a p -value of 5 × 10 -4 , 2 × 10 -4 , and 6 × 10 -3 , respectively.

E Detecting Self-Reinforcing Trends in Machine Learning System

Machine learning systems sometimes have unintended side effects and reinforcement themselves. [19] illustrated an example failure mode, where a group of users is discriminated against and thus leave a platform, causing a ML system to discriminate them further and hence drive them away.

As a concrete illustration, let us imagine a social platform Y where users post tweets and the platform will display the most engaging ones; suppose there are two groups of users, one conservative and one liberal, where both groups prefer more engaging tweets but also tweets that agree with their political stances. Y implements a recommender system, which trains a classifier to predict whether a tweet is likely to be preferred by a random user, and then the platform Y will promote these tweets. If the two groups of users are balanced, the optimal classifier will make Y promote tweets that are engaging and place little weights on the political slant.

However, if there are fewer liberal users, the classifier will be biased and Y will promote conservative tweets more and focus less on whether the tweet is engaging or not. The liberal users will find the promoted tweets less attractive, thus leaving the platform Y. As a result, fewer liberal users will stick to Y, thus making the classifier more biased.

Now we provide a proof-of-concept experiment to illustrate how our time series model can be applied to detect such a reinforcing trend. We first simulate the setup above and obtain the tweets promoted by platform Y across time, and then apply our time series model to extract temporal trends from these tweets. Suppose there are two groups of users, liberal and conservative. At t = 0 , the fraction of liberal users is λ 0 = 0 . 5 and is the same as that of conservative users. To simulate the setup above and obtain the tweets promoted by platform Y across time, we assume that at each time step t , we will sample 2,000 tweets, where each tweet is a 2D datapoint with the x -value a random integer from [-1, 1] indicating whether it is liberal, non-political, or conservative, and y -value a random integer from [-2, 2] indicating how engaging the tweet is. For each tweet, we obtain a label of y = 1 / 0 if the user likes a tweet, and the user's probability for liking a tweet is defined by σ ( ux +0 . 5 y ) , where u = 1 if the user is conservative and 0 otherwise. We then train a logistic regression classifier to predict whether a random user will like a tweet and the platform Y will promote the tweets with the top 20% score. Let the fraction of tweets non-liberal tweets be a t and non-conservative tweets be b t , then the fraction of liberal users for the next round will be determined by:

which models how the group size will increase/decrease depending on whether the platform promotes tweets that agree with their views. We run this process for T = 10 and gather all the 2D datapoints promoted by platform Y.

We then turn these two-dimensional datapoints into text samples x . We ask the gpt-4o to write a liberal, non-political, or conservative tweet based on the x -value; then we ask gpt-4o to make it more/less engaging based on the y -value. For example, for a 2D value of (-1, 2), we ask gpt-4o to write a liberal tweet and ask it to make it more engaging two times; if the value is (1, -2), we ask gpt-4o to write a conservative tweet and then ask it to make it less engaging two times.

We now have a list of tweets across time, and we directly apply our time series model with K = 3 to extract trends from them. Our time series model find that there is an increasing amount of tweets that ' expresses patriotic sentiments ' and ' champions specific policies ', but a decreasing amount ' poses a question to engage the audience '. These predicates exactly recover all the underlying trends, that the self-reinforcing effect make the tweets more conservative, less non-political, and less engaging.

Learned Predicates

Weights

Class 1

Memorable

Class 0

Not Memorable

……

"portrays a sense of tranquility; e.g. the image

captures a serene sunset over a calm lake, with

soft orange and pink hues in the sky'

'highlights specific emotions or expressions; for

example, the child has a curious expression as

they hold an old-style flip phone.'

uses a mix of colors and textures; for example,

the cluster of dark blue berries nestled among

vibrant red branches

+0.19

+0.43

-0.64

……

……

……

1.

involves algebraic manipulation;

2.

involves probability or combinatorics

3.

requires geometric reasoning;

4.

pertains to number theory;

5.

involves calculus or limits;

1.

asy, draw, axis, operatorname, tabular

2.

divisors, probability, many, letters, unique

3.

decimal, det, compute, evaluate, power

4.

solutions, roots, polynomial, solution, minimum

5.

hyperbola, corresponds, proportional, vertices, points

Ours

Classical Method

Directly explainable

? Vague

Unclear what this means

Maybe combinatorics

Probably algebra

Another algebra cluster?

Maybe geometry

F More Applications

F.1 Applying Our Classification Model to Images: Explaining Memorable Visual Features

Our framework is applicable to the vision domain since a natural language predicate ϕ can extract binary values from an image x . For example, for the rightmost image x in Figure 7 right, the predicate ' portrays a person ' evaluates to 1, i.e. J ϕ K ( x ) = 1 , while ' contains texts ' evaluates to 0.

We present an application of our classification model from Section 3.2 to images, which learns linear weights over a set of visual features described by natural language predicates. This model has also appeared in prior works: our model is equivalent to the language-based concept bottleneck model proposed by [55, 41]; additionally, when K = 1 and C = 2 , our model is equivalent to the VisDiff framework [15], which finds a single predicate to discriminate samples from two classes of images.

We apply our classification model to the LaMem dataset [24] to understand what visual features make an image more memorable, an interesting cognitive science question. We now define the samples x i and their class labels y i to run our classification model. In LaMem, each image is associated with a score of how memorable it is as measured by whether humans can remember seeing it in the past; to make implementation easier, we set x i to be the caption of the image and y i = 1 if x i has an above median score and y i = 0 otherwise. To fit our classification model, we set K = 6 , use gpt-4o as the discretizer, and use gpt-4o-mini to compute denotation.

Wepresent three learned predicates in Figure 7. We find that an image is less memorable if it ' portrays a sense of tranquility; e.g. the image captures a serene sunset over a calm lake, with soft orange and pink hues in the sky ... ', and more likely to be memorable if it ' highlights specific emotions or expressions; for example, the child has a curious expression ... '. These results are consistent with the previous manual analysis from [24], suggesting the validity of our results.

F.2 Explaining Abstract Properties via Easy Steering: Clustering Problems Based on Subarea

Can our framework explain more abstract aspects of a sample x : e.g. subarea, the type of knowledge required to solve a math problem x ? We show this is feasible by applying our model from Section 3.2 to cluster math problems and steering it to focus on explaining subareas. Meanwhile, classical methods struggle to explain abstract aspects.

We apply our clustering model from Section 3.2 to cluster the MATH dataset [21] based on subareas. The MATH dataset contains five labeled subareas 6 and we hope our model can recover all of them: Algebra , counting_and_probability , geometry , number_theory , and precalculus . To steer our clustering model to explain subareas, we simply prompt the discretizer LLM ' I want to cluster these math problems based on the type of skills required to solve them. ' We set K = 5 , using gpt-4o to discretize and gpt-4o-mini to compute denotation.

We present the outputs of our model on the left of Figure 8. With simple prompting, our model is successfully steered to cluster based on subareas and recovers all five labeled subareas from the MATH dataset. Note that our explanations can explain abstract properties that have no word overlap with the samples that match them: for example, the math problems that ' requires geometric reasoning ' (Figure 6 left 3) usually contain neither of the word 'geometric' or 'reasoning'.

We compare our method to a classical baseline that first clusters the samples and then explains each cluster with representative words. In this baseline, we first perform K-means clustering on the neural embeddings of x and assign each sample to a cluster; we then extract representative words by first running a unigram regression to predict whether a sample belongs to the cluster and then selecting words with the most positive weights. We present the word-based explanations on the right in Figure 8. Overall, significant guesswork is needed to interpret the meaning of each word-based cluster (e.g., it is unclear what cluster 1 represents in Figure 8 right), while the predicates generated by our algorithm are directly explainable. Our framework can be steered to explain more abstract aspects of a sample x and significantly improve over classical methods.

G Implementation Details on Open-Ended Applications

To obtain the best outputs, we discretize with gpt-4o and compute denotations with claude-3.5-sonnet . Since we aim to analyze user queries, we explicitly prompt gpt-4o to generate detailed predicates about use cases when discretizing continuous predicates. See 9 for the full prompt.

G.1 Taxonomizing User Applications.

Implementation Details We cluster 1024 dialogue with K = 6 and S = 5 We only cluster on a small set of dialogue turns because it is slow to compute denotations: 1) we in total explored around 100 predicates and this amounts to ∼ 100 × 1024 = 100K LLM API calls, and 2) we used language model API ( claude-3.5-sonnet ), rather than a local small language model ( google/flan-t5-xl ), to compute denotations, since this is the cheapest model that we feel confident that it can handle more sophisticated predicates.

Full Results. We present the full results in Figure 10 Overall, we find that our framework can generate sophisticated explanations that classical methods cannot generate. However, some cluster descriptions are significantly overlapping (e.g., category 0, 1, and 0.D); additionally, some sub-clusters are not indeed subsets of their parent categories (e.g., subcategory 2.D does not belong to category 2). Future work can improve the taxonomy by 1) deduplicating semantically similar descriptions or more heavily penalizing cluster overlaps, and 2) steer the predicate generation process so that the descriptions for the subclusters are indeed subsets of their parent descriptions.

G.2 Characterizing Temporal Trends.

Implementation Detail. We run our time series model on 1K dialogue turns with K = 4 and the number of iterations S to be 10 to identify temporal trends in user applications. We obtain the smoothed frequency curve by updating with the follow equation:

We obtain the shaded area in Figure 4 by shuffling x t and find the highest and lowest f values across 100 random runs.

Figure 9: A discretizer prompt that explicitly asks LLM to explain user applications. E.g., at the end of the prompt, we explicitly requested the predicates to start with ' the user wants to... '.

G.3 Advantages and Limitations

Overall, we find that our framework allows us to define sophisticated models (e.g. time series) and can output highly sophisticated predicates, which can include detailed explanations and examples. Therefore, when implemented perfectly, its utility has a much higher upperbound than classical methods such as n-gram Bayes/regression or topic models.

However, the comparison between our method and classical methods is only qualitative: we only eyeballed the outputs from our method and the classical methods in Section 6 and did not quantitatively measure how useful they are in practice. Therefore, even if our method does outperform classical methods such as topic model on our benchmark (Table 4), it might not directly translate to how useful it is in real-world applications. Additionally, we did not compare to modern taxonomy construction method such as [57], which involves a lot of task-specific engineering; our method is model-agnostic and was applied out-of-the-box to construct the taxonomy. Section 6 only shows that our method can generate more sophisticated natural language explanations, which presents a higher upperbound of what our method could potentially achieve.

In terms of the weakness of our method, our method is currently slow, as its performance highly depends on LLMs to compute denotations correctly, it outputs semantically similar predicates that add little information, and it is hard to control the predicates to satisfy certain properties (e.g. being a subset of a parent category). We look forward to future works that can address these problems and realize the full potential of this framework. For example, to remove similar predicates, one could prompt a language model to check the pairwise surface similarity between two predicates; to speed up inference, one can distill a smaller but much more efficient model specialized for computing denotations.

0 the user wants to request a descriptive screenplay writing; specifically, the query asks for the generation of screenplays including dialogues and deta ne from the first person perspective.'

0.B the user wants to request role-playing; specifically, the query involves engaging in a narrative role-playing scenario with predefined chara cters. For example,'We'll role play as characters from the walking dead game season 4,you'll provide answers for Clementine and I'll take the role of L uis...'

0.D the user wants to request content rewriting; specifically， the query asks for altering or reimagining existing content within a new context. For example,'Can you rewrite 1972's Ben as a Tales From Crypt movie, it goes the same way but write the intro and outro of the Crypt Keeper?

1 the user wants assistance with writing or editing text; specifically， the query involves tasks like composing positive feedback or rewriting content.F or example,'Please re-write this as a positive feedback for Stelina, praising her.'.

ample,Write dialogue from a scene from the animated teen Jane, where 14 year old Jane, Jane's 14 year old girlfriend Sam and Jane's 14 year old friend and neighbour Aaron Ling hanging out at school when Jane and Sam finds out Aaron has a crush.'

1.B the user wants to request dialogue writing for fictional characters; specifically, the query requests the creation of conversation between c

1.C the user wants to request a short response or reply composition; specifically, the query involves crafting a concise message to send in a di scussion. For example,'give me a response to *smiles* Yes, I'm doing well. How about you?'.

1.D the user wants to request content rewriting and optimization; specifically， the query involves improving the language, structure, or adding references to existing text. For example，'summarize this parts for method section in an article and rewrite it to improve and add some references'.

SSSSSSSSSSSSSSS

2.A the user wants to request a concise and clear response for a given message; specifically, the query involves crafting a short and direct rep 1y for use in a discussion. For example,'give me a response to Absolutely. Compassion, understanding, and respect...'.

2 the user wants to request feedback or evaluation; specifically, the query involves asking for re-writing or giving feedback on a piece of text.For exa mple,'Please re-write this as a positive feedback for Stelina, praising her.'

2.B the user wants to request for grammar and spelling correction; specifically, the query involves correcting grammatical and spelling errors i n a given text. For example,'( there was an assistance in our village about Stationary and Hygiene's for the students of (Nasozai Markazai Lesa )...'.

ressions.Forexample，‘使下列文字符合学术表达方式[textin Chinese].

ef summary. For example,'In point form, simply summarize key insights on barriers to participate in procurement from the below text at an 8th-grade read ing level and provide brief supporting quotes with ID numbers'.

SSSSSSSSSESESSS

3.A the user wants to request data manipulation or computation; specifically， the query involves operations like calculations,data processing,

3 the user wants to request technical code/script writing; specifically, the query demands the creation or modification of a script or code.For example, '逐行解释以下代码classMediaPipeFace：'

3.B the user wants to request debugging assistance; specifically, the query involves asking for help to identify and fix errors or issues in a g ivencode.Forexample，'帮我看看这段代码有啥错误'.

3.C the user wants to create or enhance UI/Ux components within web development; specifically，the query requires designing, styling,or improvi ng user interface elements. For example, 'can you do some nice font for it looking'?

scratch to achieve a specific functionality.For example，‘write a code for DES Encryption from scratch in python'.

4.B the user wants to request for creative content generation; specifically,the query involves asking for extensive and imaginative description s or stories about fictional characters or scenarios.For example,'write a detailed fusion scenario of two anime characters.'

4.D the user wants to request a comparison or fusion of different art styles or characters; specifically, the query involves merging distinct st yles or characters into a single concept. For example, 'Freedom planet and Dragon ball: Goku betrayed and join the Freedom planet with beerus, whis, zeno and his guards. part 1'.

4.C the user wants to request graphic design prompts for Midjourney AI; specifically, the query involves generating prompts for creating specifi Generative prompt writer...'.

5.D the user wants to request technical issue resolution; specifically, the query involves troubleshooting or solving a problem with software or hardware. For example,'how do I make a circle in CSS that sits on top of other elements and applies a gray scale filter to all of the elements intersec ting its area?

H A Formal Description of Our Algorithm

A formal description of our algorithm can be seen in Algorithm H.

I Additional Details of Our paper

I.1 Limitations of Our Framework and Our Experiments

As mentioned in Appendix G.3, our current system is slow, as its performance highly depends on the LLMs to compute denotations correctly, it outputs semantically similar predicates that add little information, and it is hard to control the predicates to satisfy certain properties (e.g. being a subset of a parent category). Our experiments are limited since it assumes that the datasets and statistical models we used are reflective of real world application. We made our best effort to gather text clustering datasets that are commonly used in the literature (e.g. from [52, 38]) and defined models that are plausibly useful for practitioners. Additionally, note that our evaluation on topic clustering is more comprehensive than the prior work [52] by including two new datasets ( Bills and Wiki );

Algorithm 1 A formal description of our algorithm. Argument : S is the number of steps we use to run our algorithm. Output : var_ ϕ 1 ...K is the list of K predicates that we maintain, optimize, and return at the end of the algorithm. ˆ w are other parameters.

We first optimize the relaxed continuous predicates and discretize them (Line 3-10), and then iteratively refines the predicates (Line 11-21). During iterative refinement, we first find the least useful predicate k (Line 12), then we only optimize the continuous representation of the least useful predicate while fixing other discrete predicates (Line 14 - 17); finally we discretize the k th predicate (Line 18, 19

additionally, we used the exact same hyper-parameter across all clustering tasks, while [52] changed the hyper-parameters for different datasets.

I.2 Cost of the Experiments

All of the experiments ran in Section 5 are estimed to cost at most 200 GPU hours on an A100 GPU with 40GB memory, and cost less than $20 of API credit for gpt-3.5-turbo . The experiments in Section 6 costs at most $50 of API inference credit, but we are constrained by rate limit.

I.3 Licenses for Existing Datasets

[40, 58, 32, 23]) AG-News [58] has unknown license, the DB-Pedia dataset is released under Creative Commons Attribution Share Alike 3.0, the NYT dataset is distributed by LDC under the LDC's generic non-member license, the Bills dataset [23] are considered public domain works, and the Wiki dataset is licensed under CC BY-SA 4.0.

I.4 License for the Assets Provided by Our Paper

Our code will be shared under CC BY-SA 4.0.

I.5 Broader Impacts

This paper presents work whose goal is to advance the field of Machine Learning. Our framework could potentially make machine learning systems more explainable, thus making them safer, more trustworthy and easily auditable. On the other hand, however, the learned predicates only reflect

correlation rather than causations learned from data, and hence requires careful interpretation. Given that the performance of our model-agnostic method is still far from perfect and it is unclear how human users would use them in real world applications, the algorithm presented in this paper should only be used for research and not deployed in practice.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our main contributions are

(a) defining a family of models parameterized by predicates (justified by Section 3.2)

(b) proposed an optimization algorithm based on iterative refinement and continuous relaxation, both of which improves the performance (justified by Section 5 Takeaway 1 and 2)

(c) our method achieves similar performance as the previous method specialized for explainable clustering (justified by Section 5 Takeaway 3), and

(d) our framework can generate sophisticated predicates (justified by Section 6).

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The practical limitations are hinted at the end of Section 6 and more comprehensively discussed in Appendix G and I.1.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best

judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We included code to reproduce our experiments in the supplementary material.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We included code to reproduce our experiments in the supplementary material.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We included code to reproduce our experiments in the supplementary material.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We ran statistical tests in Appendix D.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We have described the cost of our experiments in Appendix I.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have reviewed the NeurIPS Code of Ethics and observed anonymity requriements.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discussed them in Appendix I.5.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cited the existing datasets and mentioned their license in Appendix I.3.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We discuss the license for the new assets introduced by our paper in Appendix I.4.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a general framework to generate explanations of different text datasets, by parameterizing the data distribution with textual predicates. The textual predicates, along with their weights, can be viewed as an explanation of the data. Three different tasks are used as examples: clustering, time-series modeling, and multi-class classification. To learn the textual predicates and their associated weights, the authors first iteratively learn the predicates in the embedding space along with the weights, and then use a pre-trained language model to discretize the predicate embeddings into textual space. Experiments are conducted on three datasets that have ground-truth predicates. A qualitative experiment with user-chatgpt dialogues is also presented.\n\nSTRENGTHS:\n1. The proposed framework nicely generalizes from the classical Bayesian text analysis methods and makes them more relevant/useful in the LLM era.\n\n2. Comprehensive experiments are performed on four different datasets/tasks.\n\n3. The paper is well-written and easy to follow.\n\nWEAKNESSES:\n1. Missing pure LLM baselines. It is fairly common nowadays that LLMs are directly used to generate explanations given the data, which can be applied to any dataset and any task. More evidence is needed to show the advantages of the proposed method compared to direct prompting. For example, for the clustering task, what if you perform clustering first, and then prompt GPT4 to directly explain each cluster? \n\n2. The explanations generated are mostly topic descriptions, and the datasets used for qualitative results are quite simple. Can the proposed method generate more complex descriptions for more difficult datasets? For example, for a STEM problem-solving dataset (e.g. MATH), can the proposed method explain the dataset by the theorem used/subarea of the subject/similar solving strategy? \n\nIn summary, I like the way the proposed method combines the classical Bayesian method with LLM prompting, but I'm not totally convinced that the proposed method is significant as there is no direct comparison with LLM prompting, or evaluation on harder datasets. I'm willing to raise my score if my concerns are properly addressed.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to use \"natural language predicates\" to explain text datasets. Authors develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and discretizes them by prompting language models. With this method, the authors can characterize the dataset with a distribution over predicates. The authors apply it to taxonomize real-world user chat dialogs and show how the text data's features can evolve across time, demonstrating the potential of the method in understanding text data.\n\nSTRENGTHS:\n1. The paper proposes a new method to explain text datasets. There are a few new concepts in the paper, such as \"Predicate-Conditioned Distribution\" and the corresponding optimization algorithm that involve both continuous and discrete optimization.\n\n2. A practical application of the method is demonstrated on real-world user chat dialogs. \n\n3. The problem is well-formulated and backed by enough mathematical and empirical evidence.\n\nWEAKNESSES:\n1. The motivation of the paper is somewhat unclear. It's not well explained in the Introduction section why people want to \"Explain text datasets\". Is it for controlling the dataset quality for pretraining? Or tailor the domain of dataset for finetuning LM? Although the authors showcase the method on a dataset of real-world user-ChatGPT dialogues, the results look arbitrary and not very insightful. Only \"Taxonomizing User Applications via Clustering\" and \"Characterizing Temporal Trends via Time Series Modeling\" are still not fully demonstrating the usefulness of the method.\n\n2. The method should be compared with more baseline methods. It's unknown what's the benefit of predicate-conditioned distribution over other methods. The authors should also explain why the \"predicate\" is the key to explain text datasets, instead of other factors such as just verbs, nouns, text representations, etc.\n\n3. The datasets being tested are relatively small and not showing the scalability of the method.\n\nQUESTIONS:\n1. Can you provide more insights on the motivation of the paper? Why do people want to explain text datasets? What are the potential applications of the method apart from the two demonstrated in the paper?\n\n2. What is the benefit of using \"predicate\" to explain text datasets? How is the predicate better than other forms of explanations?\n\nLIMITATIONS:\nLimitations are being discussed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a a framework for a family of models parameterized by natural language predicates. This models allow for a language-based interpretation of text distribution. Such framework can easily instatiate models include clustering, time-series, and classification models. The authors develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and then discretizes them by prompting language models (LM). This approach is demonstrated to be effective in taxonomizing news dataset as well user chat dialogues while also characterizing their evolution over time.\n\nSTRENGTHS:\nI find the paper to have strengths along several axes:\n\n- Novelty: The general framework is, to the best of my knowledge a novel approach to describe text datasets in a way that is interpretable and scalable.\n\n- Model-Agnostic Algorithm: The algorithm developed to optimize the models proposes is versatile and can be applied to various types datasets under different approaches, namely classification, clustering and time-series analysis.\n\n- Extensive experiments and interesting ablations: The authors demonstrated the practical utility of the approach by applying it to real-world datasets (user-LLM chats), as well as news data for comparison purposes.\n\n- The results show improvements over the baseline and when this is not the case Bills and NYT dataset for clustering in table 2, the difference is either minimal or compensated with the generality of the proposed method.\n\nOverall the paper is well written and easy to follow. The results well presented and support the claim made in the paper. The limitation as well as weaknesses of the work are presented and discussed.\n\nWEAKNESSES:\nI did not find major weaknesses in this paper. \n\n- The main one would be in the addition computational and money costs for the proposed method compared to other classical approaches. However, in my opinion, the additional costs are counter balanced by performance and generality of the method.\n\n- Another weakness is the use of a single text embedding model. I do not think this addition experiment is necessary for accepting this paper, but I do think that testing the robustness of this method with various text embedding would make the presented claims stronger.\n\n- Additionally, given the experiments are averages across several seeds, I would report the variance of the results.\n\n----\n\nMINOR\n\n- Please adjust the citation format, in the text there's a number-based format but there are not numbers in the references\n\n- Very minor: then I tried to read table table 1 (pag 6), I was confused about what metric surface was. This was explained later in page 7. I would add a brief explanation of surface in the caption of the table, make a reference to the section when surface is explained or move the table closer the experiments section.\n\n- Typo in caption of table 2: relaxiation -> relaxation\n\n- line 277 general purpose -> general-purpose\n\n- line 648: you refer section G.1 from section G.1, did you mean section E.3?\n\n---\n\nRecomendation \n\nOverall I consider this a strong a paper and I vote in favor of its inclusion to NeurIPS\n\nQUESTIONS:\n- How are you selecting language predicates lengths? Given the NLG component in the LLM call, can predicate length evolve over time? Have you done an analysis of such change in your optimization process?\n\n- When you mentioned ranking by correlation scores: the correlation is computed between $[[\\phi]]$ and the dot product between a predicate relaxation and a sample embedding. Are you correlating continuous vs discrete values? Is $[[\\phi]]$ a binary value? Could you please clarify your correlations here?\n\n- In the no-refine case, am I reading the results correctly If I understood that you select random predicates and discretize after one full round of optimization (of $w$ and $\\phi$ them once? \n\n- In the no-relax case, you are still discretizing using a language model, is that correct? Would this just mean that the optimization process would take longer given that you are skipping the correlation-based selection criterion?\n\nLIMITATIONS:\nYes, in section 6, E.3 and G.1.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-20"}