Sparse Autoencoders for Hypothesis Generation
Authors: Rajiv Movva
Kenny Peng
Nikhil Garg
Jon Kleinberg
Emma Pierson
Venue: ICML 2025 poster (2025)
Keywords: interpretability
hypothesis generation
sparse autoencoders
computational social science
topic modeling
Primary area: applications->social_sciences
TL;DR: We propose HypotheSAEs, a sparse autoencoder-based method to hypothesize interpretable relationships between input texts and a target variable; we show our method performs well on several computational social science datasets.

Abstract: We describe HypotheSAEs, a general method to hypothesize interpretable relationships between text data (e.g., headlines) and a target variable (e.g., clicks). HypotheSAEs has three steps: (1) train a sparse autoencoder on text embeddings to produce interpretable features describing the data distribution, (2) select features that predict the target variable, and (3) generate a natural language interpretation of each feature (e.g., *mentions being surprised or shocked*) using an LLM. Each interpretation serves as a hypothesis about what predicts the target variable. Compared to baselines, our method better identifies reference hypotheses on synthetic datasets (at least +0.06 in F1) and produces more predictive hypotheses on real datasets (~twice as many significant findings), despite requiring 1-2 orders of magnitude less compute than recent LLM-based methods. HypotheSAEs also produces novel discoveries on two well-studied tasks: explaining partisan differences in Congressional speeches and identifying drivers of engagement with online headlines.

Sparse Autoencoders for Hypothesis Generation
Rajiv Movva 11
Kenny Peng 23
Nikhil Garg²
Jon Kleinberg? Emma Pierson'
Abstract
Automated approaches to answering these questions have the potential to significantly expand and accelerate scientific discovery (Ludwig & Mullainathan, 2023). Indeed, multiple lines of work-spanning decades of research have sought to extract text features that can predict a target variable (Blei et al., 2003; Monroe et al., 2009), Recent LLMd-based hypothesis generation methods are especially exciting since they operate directly at the level of -iterpraatee natural language concepts (Zhou et al., 2024; Batista & Ross, 2024). Concretely, hypothesis generation methods take dataset of texts linked to target variable (e.g., headlines and their engagement level) and hypothesize natural language concepts that predict the target variable (e.g., mentions being surprised or shocked or asks a question to the reader).
We describe HYPOTHESAES. a general method to hypothesize interpretable relationships between text data (e.g., headlines) and a target variable (e.g., clicks). HYPOTHES has three steps: (1) train a sparse autoencoder on text embeddings to produce interpretable features describing the data distribution, (2) select features that predict the target variable, and (3) generate a natural lan- guage interpretation of each feature (e.g., men- fions being surprised or shocked) using an L.M.. Each interpretation serves as a hypothesis about what predicts the target variable. Compared to baselines, our method better identifies reference hypotheses on synthetic datasets (at least +0.06 in FI) and produces more predictive hypotheses real datasets (~twice as many significant find- ings), despite requiring 1-2 orders of magnitude less compute than recent LLM- based methods. HYPOTHES also produces novel discoveries on two well-studied tasks: explaining partisan dif- ferences in Congressional speeches and identify- ing drivers of engagement with online headlines.
However, basic hurdles impede the full usage of LLMs for hypothesis generation. Consider two natural approaches. The first prompts an LLM with positive and negative ex- amples and asks it to hypothesize concepts that distinguish them (Zhou et al., 2024; Batista & Ross, 2024; Ludan et al., 2024). However, due to context window and reasoning limi- tations, we can only give LLMSs a few training examples at time, making this hard scale. A second approach aims to interpret model fine-tuned to predict the target variable (Zhong et al., 2024), leveraging the full training dataset. However, neurons are often hard to interpret (Elhage et al., 2022). In summary, efficient and scalable learning of statis- tical relationships requires numerical representations of text (e.g., neurons), but interpreting neurons is hard.
1. Introduction
Large language models (LLMs) show promise as a tool for hypothesis generation. Discovering relationships between text data and target variable is an important and funda- mental task with diverse applications in economics, political science, sociology, medicine, and business (Grimmer, 2010; Rathje et al., 2021; Sun et al., 2022; Gentzkow & Shapiro, 2010; Monroe et al., 2009; Nelson, 2020; Ranard et al., 2016; Ting et al., 2017). What features of a restaurant re- view predict a low rating? What features of a social media post predict whether it will go viral? What features of a patient's clinical notes predict if they will develop cancer?
Our first contribution is theoretical framework that con- nects model interpretation and hypothesis generation, clari- fying the challenge above. Proposition 3.1 gives a "triangle inequality" for hypothesis generation: if a neuron is predic- tive of the target variable, then a sufficiently high-fidelity interpretation of the neuron is also predictive. This result di- rectly motivates a general hypothesis generation procedure:
(Feature generation) Learn interpretable neurons (i.e., that tend fire in the presence of human concepts).
"Equal contribution 'UC Berkeley CComell University 'Corell Tech. Correspondence to: Rajiv Movva <mmvvv ammova@berkcclo..dum berkeley.edu>>, Kenny Peng <kennypeng @ co..
2. (Feature selection) Select neurons that predict the target variable.
Proceedings of the 12nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
3. (Feature interpretation) Generate fighffideiity natural language interpretations of these neurons.

---

Sparse Autocncoders for Hypothesis Generation
Given texts (e.g.. headlines) labeled with target veriable (e.g. clicks), nypothesize interpretable festures that predict target veriable.
1. Feature Generation
2. Feature Selection
3. Feature Interpretation
Train sparee autoenooder lext embeddings. Hidden neurons serve as interpretable features.
Find neurons with activalions prodict the larget variable
Find natural lenguege concept consiltes neuron activation
I active neurons per irput. Spersity enoourages interpretabiiit.
Sempled Texts for Interpretation
High-Aatvation Examples
"They Rescue
Dog
Text Input
Hiding
They
Suuppiss
"I Was Surprised At The Largest Buyer Elections My State"
"Over Years, Ago, Little
People
The World
Minutes
Low-Activation Examples
Predictive
Dare You Kick Those Mental
Pre-Trained Embedding Model
nourons
Training Wheols And
Bike Ride This Inspiring"
"The Cable Paskage Chese Might An Eleborate Ilusion'
Imput Layer
Output Layer
Hypothesis
Sparse Hidden Layer
'meniione being surprized shocked
Figure HYPOTHESAES is general method that outputs natural language hypotheses from text datasets.
Our second contribution is a method, HYPOTHESAES, that successfully implements this procedure. A primary chal- lenge is learning the interpretable neurons in step l-after all, a long line of work has established the challenge of inter- preting neural networks (Olah et al., 2017; Kim et al., 2018; Elhage et al., 2022). To overcome this, we train a sparse autoencoder (SAE; Makhzani & Frey (2014)) on pre-trained text embeddings from our dataset. Neurons in the SAE hidden layer are highly interpretable, while retaining the explanatory power of embeddings (Cunningham et al., 2023; Bricken et al., 2023; Templeton et al., 2024; Gao et al., 2024; O' O'Neill et al., 2024). Intuitively, this architecture leverages the fact that natural language is sparse; only a small fraction of all human concepts are expressed at a time. In step 2, we select neurons that are predictive of the target variable (e.g., using Lasso). In step 3, we automatically generate high-fidelity interpretations of these predictive neurons by prompting an LLM with texts that activate the neuron, build- ing on recent work in autointerpretability (Bills et al., 2023). These interpretations serve as hypotheses.
HYPOTHES produces many more significant hypothe- ses than three baseline methods. On three real-world tasks, 45/60 hypotheses generated by our method are significant, compared to at most 24 for the baselines. On two datasets, we identify new hypotheses which previous detailed analy- ses have not uncovered. Intuitively, by decoupling feature generation from the prediction task, HYPoTHESAES able to explore a wider range of hypotheses.
HYPOTHES AES is also more than 10x faster and cheaper than recent LLM baselines, with similar runtime and cost to BERTOPIC, standard embedding based topic model. Filtering hypotheses based on the predictiveness of neu- rons is significantly cheaper than directly evaluating natural language concepts, since all neuron activations of an input can be computed from a single forward pass of an SAE (in comparison to one L.M call per natural language feature).
2. Related Work
2.1. Hypothesis Generation
We evaluate HYPOTHESA AES against recent state-of-the- art methods. We consider three synthetic tasks, well as three real-world tasks of practical interest: hypothesizing the relationship between headlines and engagement, speech text and political party, and review text and rating. Example HYPOTHES AES hypotheses are given below:
Classical text analysis methods, such as comparing n- gram frequencies between groups (Fightin" Words; Monroe et al. (2009)) or topic modeling (LDA, Blei et al. (2003); BERTopic, Grootendorst (2022)) remain standard tools to discover relationships between text and target variables (Grimmer et al., 2022). A challenge with these methods is that their outputs lists of words or documents- are not immediately interpretable by humans (Chang et al., 2009). As a result, applying these methods for scientific discovery requires sufficient domain expertise to prespecify hypothe- ses and assess whether the data support them (Demszky et al., 2019; Gentzkow et al., 2016; Sun et al., 2022).
Task
Example Hypotheses (Abbreviated)
Headline
(+) mentions being surprised or shocked
Engagement
(-) asks a question directly the reader
(Rep.) discusses illegal immigration
Speaker Party
(Dem.) criticizes tax breaks for the wealthy
Restaurant
(+) mentions plans return to the restaurant
Rating
(-) mentions issues related to food safery
Recently, several LLM-based methods aim to automate inter-
2

---

Sparse Autocncoders for Hypothesis Generation
pretable concept discovery from text datasets. Grootendorst (2024) prompts L.MMs to generate cluster labels, Pham et al. (2024) use LLMs for end-to-end topic modeling in natu- ral language, and Lam et al. (2024) build an LLM- interface to discover topics guided by user input. Another literature uses LLMs to propose concepts in bottleneck mod- els (Koh et al., 2020): Ludan et al. (2024) prompt LLMs with a task description to generate predictive concepts, and Feng et al. (2024) use LLMs to provide prior distribution over concepts. Sun et al. (2024) and Zhong et al. (2024) also use LLMSs to propose concepts, guided by neurons from a supervised model. Finally, Zhou et al. (2024) and Batista & Ross (2024) focus specifically on hypothesis generation: LLMs propose natural language descriptions of predictive relationships, which can then be tested for generalization.
Define S(Z) analogously. ! We measure the fidelity of the interpretation 2 by
5(2 = 1- min frecall, precision]
(1)
min = 0j, Pr|Z = 0)
where precision is how often the neuron fires when the con- cept is present and recall is how often the concept is present when the neuron fires. Lower 5 implies higher fidelity. High recall and precision means the interpretation approximates the neuron's 's behavior, and is not overly narrow generie. Empirically, the denominator min(Pr(2 0j, Pr[Z 0}} is close to I (neurons and concepts activate infrequently), so maximizing fidelity is approximately maximizing the minimum of recall and precision.
Proposition 3.1 (A Triangle Inequality for Hypothesis Gen- eration). Suppose Y is supported on [0, 1]. Then
Unlike several of these methods, SlYPOTHESAEs decouples feature generation (unsupervised, using the SAE), selection (supervised), and interpretation (the only LLM-depended step). Comparing to baselines from each class of methods ($5.3), we illustrate quantitative and qualitative improve- ments over classical and fully-LL baselines.
5(Z) - Z).
Intuitively, the result shows that if 2 is a high-fidelity pre- dictor of Z, then it will also have a similar separation score. A full proof is given in Appendix F.2
2.2. Sparse Autoencoders
Notice that what we ultimately care about is S(2) (how predictive our actual natural language hypothesis is), but our procedure focuses primarily on generating, selecting, and then interpreting the neuron Z. Proposition 3.1 shows that as long as we can generate a sufficiently high-fidelity interpretation 2 of each neuron Z, then we can focus on finding predictive neurons as a way to ultimately find natural language hypotheses. In particular, the result shows that
A recent line of work demonstrates that sparse autoencoders (Makhzani & Frey, 2014) are an effective architecture to learn interpretable features from intermediate layers of large language models (Cunningham et al., 2023; Bricken al., 2023; Templeton et al., 2024; Bills et al., 2023). This litera- ture is primarily motivated by mechanistic interpretability: since the neurons in the hidden layer of sparse autoencoders correspond to interpretable features, they can be strategi- cally altered to steer the behavior of language models. In the present work, we instead apply sparse autoencoders to learn interpretable text features for the purpose of hypothesizing interpretable relationships between text data and target variable. We follow O'Neill et al. (2024), who show that SAE features can guide semantic search applications, in training a sparse autoencoder directly on text embeddings.
S(2) -((Z) + ( - 5()))
(3)
2 S(Z) - 8(2,2)
(4)
neuron
interpretation
predictiveness
fidelity
Equation (4) shows that to find natural language concept such that S(2) is large, it suffices to first identify Z such that S(Z) is high (i.c., find a predictive neuron) and then 2 such that 8(2, 2) small (i.e., find figh-fideiity interpretation).
3. Theoretical Framework
In this section, we derive theoretical result that shows how a language model with highly interpretable neurons can be used for hypothesis generation, as well as where performance loss arises in such a procedure.
This directly motivates the actual procedure employed in HyPOTHESAES. First, we learn interpretable neurons Z using an SAE. Second, we identify which neurons Z are
'Separation score is one way we evaluate hypotheses empirically. We suspect that analogs of Proposition 3.1 hold for alternative measures of predictiveness, like explained variance.
Let Z be an indicator for whether a neuron fires for a given text input (i.e., whether the activation exceeds a threshold). Let 2 be an indicator for whether the text contains speci- fied natural language concept. Let Y be the target variable. Define the predictiveness of 2 as the separation score
"There are main observations that enable us to show the result. First, we focus on [ = 1] Y if the false negative rate (FNR) is greater than the false discovery rate (FDR), this value is bounded by the FNR; otherwise, it is bounded by the FDR. Second, we show roughly that as long
1]] and show that
as Pr[2 = 11, Pr = 1] < Pr = Pr = 0). then | 2 = 0| - F 7 = 0] < | = 1] - E = 1]}-
S(2) = 1 2 = 1] - E = O]|-
3

---

Sparse Autocncoders for Hypothesis Generation
predictive. Third, we generate high-fidelity interpretations 7 of these neurons with L.M..
Empirically, the dimensions of 2 are often highly inter- pretable, corresponding human concepts. M sets the total number of concepts learned across the entire dataset, and & sets the number of concepts that can be used to reconstruct each instance. The output of this first step is an N x M activation matrix, ZSAB-
Equation (4) also gives us a lens by which to analyze Hy- POTHESAEs empirically. The result decomposes the perfor- mance of hypotheses 2 into two parts: the predictiveness of the neurons and the fidelity of the interpretations Z. For example, if the SAE neurons are similarly predictive to the text embeddings they encode, then most performance loss (compared to predicting directly from embeddings) comes from low-fidelity neuron interpretation. Imperfect neuron interpretations can be due to either neurons being funda- mentally uninterpretable (there doesn't 't exist a high-fidelity interpretation), or due to difficulties in actually finding the interpretation. These challenges arise in steps (training the SAE) and step (generating the interpretation) respectively. We perform such an empirical analysis in Appendix C.3.
4.2. Feature Selection: Target Prediction with SAE Neurons.
To select a predictive subset of SAE neurons, we fit an -rrrulllrree linear or logistic regression predicting the target variable Y from the activation matrix ZSAE (Tibshi- rani, 1996). Formally, for regression', optimize
1 lly - 2 + X119111 N
min ((3; B
where y is a length-N vector, 7SAE is an Nx M matrix, and ,3 is a length-M vector of feature coefficients for each SAE neuron. The Lı penalty produces sparse coefficient vectors B where some coefficients are exactly zero (indicating a dropped feature). To generate H hypotheses, we perform binary search to identify value of X which produces exactly H nonzero coefficients.
x
4. Methods
In the previous section, we established theoretically that we can generate hypotheses by learning interpretable neurons, identifying which neurons are predictive, and then gener- ating high-fidelity interpretations of the neurons. We now describe our method, HYPOTHESAES, which accomplishes these steps in practice.
4.3. Feature Interpretation: Labeling Neurons with LLMs.
Given a dataset of training examples ((T), yi) Je(N]: where I; is the input text and yi is the target variable annotation, HYPOTHES AES outputs H natural language concepts that serve as hypotheses. The goal is for these natural language concepts to predict the target variable.
In this step, we generate high-fidelity interpretations (i.e., natural language concepts) of the subset of predictive neu- rons. To generate interpretations of a neuron, we prompt an LLM with example texts from a range of neuron activations and instruct it to identify a natural language concept that present in high-activation texts and absent in low-activation texts. We generate multiple interpretations by running this procedure several times, and choose the best interpretation according to measure of fidelity we describe below.
4.1. Feature Generation: Training Sparse Autoencoders.
Let e; denote D-dimensional text embedding of x4. We train k-sparse autoencoder (Makhzani & Frey, 2014). k-sparse autoencoders have successfully generated inter- pretable features both when applied to intermediate layers of language models, as well as text embeddings (Gao et al., 2024; O' Neill et al., 2024). The SAE encodes a text em- bedding e; as follows (we use OpenAl's -ext-embedding-3- small, where D 1536, in our experiments):
We test a variety of approaches for generating interpretations inspired by recent literature on automated interpretability (Bills et al., 2023; Templeton al., 2024). In our final pro- cedure, we sample high- and low-activating texts from the neuron's 's positive activation distribution, with the sampling percentile bins given in Appendix B.2. (Compared to zero- activating texts, texts with a low but nonzero activation act as hard negatives, improving the generated concept's 's pre- cision.) Treating these as true positives and true negatives, respectively, we prompt an LLM to generate natural lan- guage concept that distinguishes 10 samples of each class, We then define an interpretation's 's fidelity as its Fl score in this prediction task, using an LLM to annotate 100 positive and 100 negative samples for the presence of the concept.
2. = ReLU(TopK(Wenclee - (pxx) + beno)),
E Wiec2: + baec:
where Upre E RD,, Wene E MxD bone RM M,M, Waec E IR DxM M bdec E RD,, and TopK sets all activations except the top le to zero. In a basic k-sparse autoencoder, the loss on one input is
L f= - 1
(7)
We follow Gao et al. (2024); O'Neill et al. (2024) in further adding an auxillary loss to avoid "dead latents." " We describe details and hyperparameters in Appendix B.
For classification analogously use an Legruullaree loss: C( A) N BCE(y, a (Z5AE,3)) X||/8|11, where BCE is the hinary cross-entropy loss and o()) is the sigmoid function.
4

---

Sparse Autocncoders for Hypothesis Generation
We use GPT-4o with temperature 0.7 to generate concepts and GPT--4o-mini with temperature 0 for concept annotation.
1K validation, and 4.4K heldout.
YELP (Yelp, 2024): Which features of Yelp restaurant reviews predict users' 1-5 star ratings? We use 200K reviews for training, 10K for validation, and 10K for heldout eval.
This approach produced predictive interpretations when compared to other prompting and evaluation schemes, though some other methods performed similarly (see Ap- pendix C for details). Importantly, we found experimentally that higher-fidelity interpretations according to our mea- sure have better predictive performance (Appendix C.I), justifying our approach selecting interpretations.
CONGRESS (Gentzkow & Shapiro, 2010): Which features of U.S. congressional speeches predict party affiliation? Speeches are from the 109th Congress (2005-07) with binary labels (Rep. or Dem.). Our split sizes are 1 114K training, 16K validation, and 12K heldout.
5. Experiments
5.1. Datasets
On each of these tasks, our goal is to identify interpretable natural language features that predict the target variable. The former two are studied in prior work on hypothesis generation (Zhou et al., 2024; Batista & Ross, 2024; Lu- dan al., 2024); the latter is longstanding application in computational social science (Grimmer al., 2021).
We evaluate on synthetic and real-world datasets, described below. Appendix provides more details on all datasets.
Synthetic datasets: recovering known, reference hy- potheses. Our synthetic evaluation is motivated by real- world settings in which there are multiple disjoint hypothe- ses we would like to discover. We therefore use two datasets from prior work on interpretable clustering (Pham et al., 2024; Zhong et al., 2024): WIKI and BILLS. We construct target variable that is positive if a text belongs one of pre-specified list of reference categories; these are the ground-truth "hypotheses" to recover. Both datasets are labeled by humans with granular hierarchical topics, such as Media and Drama: Television: The Simpsons in WIKI and Macroeconomics: Tax Code in BILLS.
5.2. Evaluation Metries
Synthetic datasets. On the synthetic datasets, we eval- uate how well we recover the reference hypotheses. We run HYPOTHESAES by setting H to the number of refer- ences (e.g., 5 for WIKI-5). We use GPT-4o-mini to annotate each inferred hypothesis on the heldout set, producing an Nueloou x H binary matrix of annotations. Following Zhong et al. (2024), we compute the optimal matching between ref- erence and inferred hypotheses via the Hungarian algorithm the H x H correlation matrix of reference vs. inferred hypothesis annotations, and we report three metrics:
We generate ground-trute -truth labels based on the most frequent granular topics: articles in any of the top-5 topics are labeled '1', and all others are labeled '0', This produces 5 reference hypotheses, e.g., the bill is about Macroeconomics: Tax Code is one of them. For WIKI, we also include more challenging variation of recovering the top-15 most frequent topics. WIKI contains 11,979 total articles (15% positive for WIKI-5; 35% for WIKI-15), and BILLSS contains 21, (24% positive). For both datasets, we reserve 2,000 items for validation (i.e., SAE hyperparameter selection) and 2,000 heldout items to evaluate hypotheses; we use the remaining items for SAE training and feature selection.
F1 Similarity: For each matched (reference, inferred) pair, we compute the F1-score hetween the presence of the ground-truth reference topic and the inferred hypothesis annotations. We report the mean across the H pairs.
For each matched pair, we prompt GPT-4o to assess whether the hypotheses are the same, related, or distinct, with corresponding scores of 1.0, 0.5, and 0.0. To improve stability, we sample 5 outputs at temperature 0.7 and average them. We report the mean value of this score across the H pairs. (TTbll shows examples; Figure 6 provides the prompt.)
Surface Similarity:
Real-world datasets: Generating hypotheses that predict a target variable. We apply SYPOTHESAES to three real- world datasets which have been studied in prior work:
Overall AUC: Using annotations from the H inferred hy- potheses, we fit a multivariate logit and compute AUC to evaluate predictive performance. (Recall that the ground- truth label is logical OR of the reference topics.)
HEADLINES (Matias al., 2021): Which features of digital news headlines predict user engagement? Each instance is pair of differently-phrased headlines for the same article; users on Upworthy.com were randomly shown one, and the task is to identify which headline re- ceived higher click-rate. We reserve a large heldout set to improve statistical power: our split sizes are 8.8K training,
The first and second metrics are taken directly from Zhong et al. (2024); they assess whether the inferred hypotheses quantitatively and qualitatively match the ground-truth. The third metric assesses overall prediction quality. Some hy- potheses may not match a reference but still be predictive,
5

---

Sparse Autocncoders for Hypothesis Generation
Source
Overall ALC
Sig
Significant Hypotheses:
Green: 1 clicks
Red: clicks
Sep.
AUC
mentions photography, images, visuol representations
0.26
0.51
evoke discomfort embarrussent
0.58
reference to being surprised, shocked, or unprepured focuses on fomale subject
personal journey
0.52
mentions heavily emphasizes video or video-related content
mentions a surprising or uncxpected action or ouicome
0.13
0.58
focuses on a male protagonia it or subject
0.11
0.53
HYPOTHESAES
0.693
contains pirases about looking or seeing something
0.09
0.52
mentions uniqueness ss exclusivity using superiative comparative language mentions sponse to confrontation with offensive behavior mentions politicians, government actions, politicul topics asks question directly to the reader addresses collective human responsibility
0.09
0.53
0.09
0.53
-0.10
0.52
0.12
0.53
-0.13
0.56
enviroumental that povitively impact communitt
ecological consequences
-0.15
BERTOPIC
0.619
headlines emphasize emotional reactions short, impaciful videos
0.15
0.59
has un element of surprise in its structure
0.14
0.57
frames the content within gender specific context
0.11
0.53
NLPARAM
0.660
includes inspiring transformation
-0.08
0.51
words puns
-0.09
0.54
poses shetorical question
-0.12
0.55
implies personal dory aneedote that invites curinxity uses strong emotional language evokes curiosity poses rhetorical question that disengage the reader
0.15
0.56
HYPOGENIC
0.646
3
0.12
0.58
-0.11
0.55
Table Hypotheses HEADLINES with p 0.05 after Bonferroni correction in multivariate regression. 'SSep. is the separation ($3): 2 = 1|- 2 0j. where positive scores predict higher click-rates. The 'AUC' column provides AUCs of individual hypotheses. 'Overall AUC' all 20 hypotheses, including insignificant ones. Hypotheses for all methods abbreviated for space.
so this metric rewards any interpretable hypotheses that contribute predictive value, even if they are not optimal.
using all of the hypotheses in a shared regression, report- ing AUC for classification and R² for regression.
Real datasets. On the real datasets, we generate 20 hy- potheses per method, and assess the hypothesis sets for two qualities: (1) breadth: how many distinct, predictive hy- potheses were identified? (2) predictiveness: how well do the hypotheses collectively explain the label? We again pute Nneldout H annotation matrix for each method's s hypotheses, and measure these constructs as follows:
Qualitative Annotation: We recruit three computational science researchers to rate each hypothesis for (1) help- fulness and (2) interpretability (details in App. E). These attributes are inspired by prior work (Lam et al., 2024).
5.3. Baselines
We compare HYPOTHESAES against recent, statc-of-the. art methods ($2). For fair comparison, we run all methods with the same embeddings and LLMs as our method, un- less stated otherwise: OpenAl's text-embedding-3-nie embeddings; GPT-4o for hypothesis proposal; and GPT-4o- mini for hypothesis annotation. We run each method to produce H hypotheses and evaluate all methods identically,
Breadth: We a multivariate logit (for binary labels) or OLS (continuous) regressing the target variable against all of the hypothesis annotations, and report the count of hy- potheses that have significant, nonzero coefficient'. This metric benefits distinct hypotheses and penalizes redun- dancy. Breadth is important since the utility of hypotheses is often determined by criteria beyond predictiveness. For con-
for
BERTOPIC (Grootendorst, 2022) is a neural topic model that produces topies by clustering text embeddings. To produce H hypotheses from BERTOPIC, texts are clustered into topics, and we fit L--reullarized model to predict Y from topic ID; like our method, we set the penalty such that there are exactly H nonzero coefficients. We assign each topic a natural language label by prompting an LLM with a sample of documents and top terms (prompt in Figure 8).
example, a hypothesis may be more useful nected to past thcories or suggests a new direction for investigation. More breadth increases the likelihood that a hypothesis satisfying these criteria is produced.
Overall Predictive Performance: Similar to the syn- thetic datasets, we report the overall prediction quality
We use a Bonferroni-correeted p-value threshold of 2.5e-3.
NI.PARAM (Zhong al., 2024) trains a neural network
6

---

Sparse Autocncoders for Hypothesis Generation
WIKT, H
WIK1, = 15
BILLS, H
HEADLINES
YELP
CONGRESS
METHOD
F1
SURE
AUC
F1
SURF.
AUC
F1
SURF.
AUC
CT.
AUC
CT.
CT.
AUC
HYPOTHESAES
0.58
0.70
0.84
0.47
0.84
0.67
0.78
0.89
15
0.69
15
0.76
15
0.70
BERTOPIC
0.48
0.48
0.77
0.40
0.43
0.79
0.65
0.88
0.85
0.62
12
0.65
10
0.64
NLPARAM
0.15
0.30
0,69
0.16
0.34
0.77
0.32
0.50
0.74
5
0.66
11
0.58
0.65
HYPOGENIC
0.15
0.20
0.60
0.12
0.39
0.64
0.25
0.30
0.70
3
0.65
12
0.78
5
0.68
Table HYPOTHESAES performs best recovering reference human-annotated topics WIKI-5, WIKI-15, and BILLS. We also evaluate on three real-world datasets: HEADLINES, YELP, and CONGRESS. "Ct." refers to the count of significant hypotheses, out of 20 candidates; HYPOTHES generates the most significant hypotheses.
with length-K bottleneck layer to predict Y from text embeddings, and then uses an LLM to propose labels cor- responding to each one of the bottleneck neurons' activa- tions. The hypotheses are iteratively refined; hypothesis is retained only if its annotations (computed on the entire dataset) improve the loss. We run NLPARAM for the default 10 iterations, with 3 candidate hypotheses per neuron.
ses either perfectly match or related to the references. On WIKI-5 (Table 4), we retrieve two hypotheses exactly; the other three are slightly too specific (HYPOTHES A AES infers a topic is about a New York State Route or highway, but the reference includes all Northeastern roads, not just New York) or broad (it infers historical battles but does not specify battles since 1800), but all inferred hypotheses are always related to a reference, i.e., has surface similarity > 0.5. No baselines meet this standard on WIKI-5: NL- PARAM and HYPOGENIC output 3/5 and 2/5 hypotheses that are unrelated to any of the reference topics, respectively. BERTOPIC performs well, but outputs redundant hypothe- ses about state highways and misses 'battles' entirely.
HYPOGENIC (Zhou et al., 2024) directly prompts lan- guage model to propose hypotheses given a task-specific prompt and labeled examples. They use a bandit reward function to score hypotheses and propose new ones based on incorrectly- labeled examples. We run HYPOGENIC with de- fault parameters; note that the method uses the same model for proposal and annotation, which defaults to GPT-4o-mini.
Our hypotheses also closely match ground-truth when used to annotate heldout data. The mean F1 score compares how closely annotations according an inferred hypothesis (e.g. is about Simpsons episode) match ground truth labels (whether the WTKI article belongs to the human-annotated Simpsons category). HYPOTHES beats all baselines on this metric, especially outperforming NLPARAM (+0.36, on average) and HYPOGENIC (+0.40). Finally, regressing the label using each method's annotations, HYPOTHES AES achieves higher AUC than all baselines. Overall, HYPOTHE-- is designed
Training set size. A key advantage of HYPoTHESAES is that feature generation and selection require only an SAE forward pass, which is cheap even for large datasets; the only L.M-dependent step is feature interpretation. BERTOPIC also scales well, as it only uses LLM inference for topic labeling. In contrast, NLPARAM and HYPOGENIC score and select hypotheses by annotating every training example, so their inference costs scale with training set size. Running either method on 200K Yelp reviews, for instance, would cost $550 and 78 hours (compared to .0$1 and 20 min- utes for HYPOTHESAES). To compare to the baselines as favorably as possible, we allow each method to use up to 50x the cost and runtime of ours. To fit this budget, we use up to 20,000 training samples for NLPARAM and Hy- POGENIC, which accommodates the full training sets WIKI, BILLS, and HEADLINES; we use random subsets for YELP and CONGRESS. Notably, this sample budget is sig- nificantly larger than either method uses in their publication experiments (4,748 for NLPARAM; 200 for HYPOGENIC), suggesting our comparisons are generous to the baselines.
S SAFS usually outperforms exactly for this topic inference setting-and substantially outperforms two sfate-of-the-- art LLM hypothesis generation methods.
6.2. World Datasets
Table shows quantitative metrics for the real-world datasets. Tables 5, and 6 show all significant hypotheses on HEADLINES, CONGRESS, and YELP respectively.
HYPOTHESA Es identifies more distinct hypotheses than other methods. Compared to baselines. AAPoTHESAES produces the most hypotheses which are significant in a multivariate regression. Importantly, this evaluation reflects successful generalization: the hypotheses are learned on the training and validation sets, and they remain significantly associated with the target variable when annotated by GPT- 4o-mini on the heldout set. Across the three datasets, 45 out of 60 candidate hypotheses are significant, substantially
6. Results
6.1. Synthetic Datasets
HYPOTHESA A ES recovers ground-truth hypotheses. Ta- ble 2 shows quantitative metrics: HYPOTHESAES beats all baselines on 8 of 9 dataset-metric pairs. As captured by sur- face similarity, most of HYPOTHESA AEs's inferred hypothe-
7

---

Sparse Autocncoders for Hypothesis Generation
ahead of the 24, 23, and 20 for NLPARAM, BERTOPIC, and HYPOGENIC respectively.
PCA: Same as Embedding. but we first use PCA to re-
duce the embedding dimension from 1536 down to 256.
When evaluating overall predictive performance, HYPOTHE- SAES beats baselines on 8 of 9 comparisons, demonstrating strongly predictive hypotheses. The exception is that HY- POGENIC achieves 2% higher It² on Yelp. However, HY- POGENIC's most predictive hypotheses, such as expresses disappointment with the overall dining experience, essen- tially restate the rating prediction task. While most LLMs can perform such a prediction, these hypotheses do not reveal specific insights about the task. In contrast, our hy- potheses- in Table 6 for YELP- tend to be more specific: we find that negative reviews mention food poisoning, rude and aggressive staff, or dishonest business practices.
Bottleneck: Inspired by bottleneck models (Yang et al., 2023), fit a neural network predict y using text embeddings as input and single hidden layer with 20 neurons. We generate hypotheses by interpreting the hid- den neurons (in the same way we interpret SAE ncurons).
We show results in Table 7. In addition to the count of significant hypotheses and overall predictiveness, we also report average neuron fidelity.
We establish two benefits of the SAE: the neuron interpreta- tions are (1) higher fidelity and (2) cover wider variety of non-redundant concepts. Regarding (I), interpretations of SAE neurons achieve a mean F1 of 0.84, compared to 0.64 for the best ablation (bottleneck). Regarding (2), we see im- provements to downstream metrics: the SAE produces many more significant hypotheses (45/60 across the three datasets, vs. 30/60 for the best ablation) and achieves slightly higher predictiveness. Qualitatively, the SAE neurons yield more specific interpretations, while the ablations tend to yield generic ones. Both benefits (1) and (2) are consistent with the SAE producing monosemantic neurons, which are easy to interpret and specific enough to avoid redundancy.
HYPOTHESAES outputs are rated as qualitatively helpful and interpretable. The results of our qualitative evaluation (Figure 2) confirm the above intuition: most of the predictive concepts generated by HYPOTHES AES are rated helpful (24 out of 30) and interpretable (29 out of 3 Several hypotheses from NLPARAM and HYPOGENIC, de- spite being predictive, do not pass these checks. The total count of hypotheses that are predictive, helpful, and inter- pretable generated by HYPOTHESAEs (23) far exceeds any baseline (BERTOPIC is second-bes with 7).
30
100%
6.4. Evaluating Hypothesis Novelty
Percent of Hypolheses
Hypolhesis Count
75%
Beyond quantitative comparisons against baselines, we eval- uate if HYPOTHESAE: AES discovers new hypotheses even on datasets that have been thoroughly analyzed prior work.
20-
HypotheSAEs BERTopic
50%
HypoGeniC
NLParam
25%
CONGRESS. A classic study from Gentzkow et al. (2016) identifies bigrams and trigrams in Congressional speeches that mark Republican (R) or Democrat (D) speakers. We compare to their work by computing counts, in each speech, of their full reported n-gram lists (150 R / run HYPOTHESA A to produce 100 hypotheses, of which 33 are significant. A regression using only n-gram counts achieves AUC 0.61, while including our 33 hypotheses in- creases AUC to 0.74. Notably, 28 out of 33 hypotheses re- main significantly predictive when controlling for n-grams, showing that our hypotheses add signal to classical methods.
Helptul
Interpretable
Helpful
Interpretable
Figure 2. HYPOTHES AES produces a larger number (left) and per- centage (right) of significant hypotheses that are rated helpful and interpretable. We report majority vote of 3 ratings per hypothesis.
150 D): we also
6.3. Non-SAE Ablations
To isolate the value of the SAE, we conduct ablation ex- periments. Because our method differs from baselines in multiple ways- e.g., our neuron interpretation procedure differs from that of NLPARAM-iL it is worth asking what gain the SAE specifically provides. To answer this question, we run the following ablations on the real-world datasets:
Beyond predictive performance, HYPOTHESAED AES offers two qualitative advantages. First, our hypotheses are inherently interpretable. For example, while Gentzkow et al. (2016) report that "oil companies" correlates with Democrats, requires context to interpret: are speakers referring to oil companies positively or negatively? In contrast, HYPOTHE- S SAE'' corresponding hypothesis is criticizes oil compu- nies or oil industry practices. Second, our hypotheses cap- ture nuanced patterns that n-grams cannot expresss, like criticizes government resource allocation, highlighting dis- parities between domestic needs and actions abroad.
We skip training an SAE and directly select 20 features, via Lasso, from text embeddings. The rest of our method is unchanged.
Embedding:
"To reduce annotator workload, our qualitative evaluation only included results from HEADLINES and CONGRESS, since these are more well-studied social science tasks than YEL.P.
8

---

Sparse Autocncoders for Hypothesis Generation
HEADLINES. To test our discoveries against more mod- methods, we compare Batista & Ross (2024), an LLM hypothesis generation study designed for HEADLINES and aimed at producing new insights for marketing researchers. Each of their 5 hypotheses is validated via human annotation. Using our heldout evaluation protocol, we compute anno- tations using their hypotheses, which achieve AUC 0.59 at predicting binary engagement; adding in HYPOTESAAEs's 15 significant hypotheses increases AUC 0.70. Crucially, all 15 of our hypotheses remain significant when control- ling for the prior discoveries in a shared regression.
Method
Time (min)
Tokens (M)
Cost
HYPOTHESAES
18.1
7.2
$1.29
SAE-NOVAL
7.9
0.1 0.0
$0.15
NLPARAM
904.9
2.8 414
$69.92
HYPOGENIC
575.4
0.0/
$11.02
BERTOPIC
3.0
0.1
$0.19
Table 3. Runtimes and costs for hypothesis generation on CONGRESS. We report millions of tokens for GPT-4o (concept generation) and GPT-4o-min miii (concept annotation) respectively as {# 40} / {# 4o-mini). Input and output token counts are summed (> (>99% are input tokens). SAE-NOVAL is our method without the fidelity validation step (i.e., for each neuron, we generate only label instead of choosing from three based on fidelity). Costs are in USD and based on OpenAl pricing as of 01/2025.
Several other prior works have also studied HEADLINES using classical dictionary- based methods (Robertson et al., 2023; Gligorić et al., 2023; Banerjee & Urminsky, 2024; Aubin Le Quéré & Matias, 2025). HYPOTHESAES qualita- tively improves on these findings by producing more specific hypotheses. For example, Robertson et al. (2023) find that negative words increase engagement, and positive words decrease it. HYPOTIESABE AES supports this finding, and also make it more precise: we find that situations that evoke discomfort or embarrassment and mentions of offensive behavior increase clicks, while initiatives that positively impact a community decreases them (Table 1). Banerjee & Urminsky (2024) find that words in "emotional intensity" dictionary increase engagement, which may be difficult to operationalize; our corresponding finding is that headlines referencing being surprised, shocked, or unprepared in- crease engagement. Our findings are more precise, and can therefore provide richer insights to researchers.
BERTOPIC is cheapest, and remains a good baseline for a essurce-limited analysis. However, HYPoTHESĀEs can also be made cheaper by skipping the non-essential label validation step: instead of generating 3 candidate labels per neuron and choosing the highest-fidelity one, we can simply generate 1 label per neuron and use it directly. This reduces cost to that of BERTOPIC, while still exceeding its performance; we describe this ablation in Appendix B.2.
7. Conclusion
We propose HYPOTHESAES,, a scalable method to generate interpretable hypotheses from black-box text representa- tions. On three synthetic and three real-world tasks, Hy- POTHES outperforms statc-of-thh-ar topic modeling and LLM baselines, at less than 10x the time and cost of the latter. Beyond the tasks we study, the method applies naturally to the many settings in the social sciences where large text corpus is labeled with a target variable of in- terest (Ziems et al., 2024; Bollen et al., 2011; Card et al., 2022; Breuer et al., 2025). In future work, an exciting direc- tion is extending our method to explain non-text datasets, like images (Dunlap et al., 2024) and proteins (Vig et al., 2021). as well as text datasets from specialized domains like healthcare (Hsu et al., 2023; Robitschek et al., 2025). More generally, there are many applications where we can accu- rately predict target variable, but producing new scientific insights remains difficult. Our work helps bridge this gap.
These results demonstrate that HYPOTHESAES, despite no datasct-specific design, uncovers novel hypotheses that advance prior work from domain spperts-suggestnng broad applicability across diverse settings.
6.5. Costs
In Table 3, we report the runtimes, LLM inference to- ken counts, and costs (at current OpenAl API pricing) for all methods on CONGRESS. The trends are similar for all datasets. Runtimes include training the SAEs on one NVIDIA A6000 GPU. Training is fast because our SAEs can afford to be small (~ features, compared to 10" in Gao et al. (2024)), since our focus is to learn domain-specific features rather than features of all language.
HYPOTHESAES is much more efficient than NLPARAM and HYPOGENIC. Despite the fact that these methods are trained on less data-200 examples vs. 1 114K for HYPOTHE- HYES-HYPOTHESAES is 330500x faster and 10-50x cheaper. This is because NLPARAM and HYPOGENIC score hypotheses by annotating every example in the dataset. As result, their LLM annotation requirements scale with both the number of hypotheses H and dataset size N, while HYPOTHESAES's scales only with H.
Software and Data
Code is available on GitHub a and can be installed via pip with pip install hypothesaes. A notebook to re- produce experimental results in the paper is available in the repository. All data used in the paper are available on HuggingFace. A demo to visualize all SAE neurons and how well they predict the target for HEADLINES. YELP, and CONGRESS is available at hypothesaes.org

---

Sparse Autocncoders for Hypothesis Generation
Acknowledgements
the stock market. Journal of Computational Science, (1):1-8, March 2011. ISSN 18777503.
Thanks to Rishi Jha, Hal Triedman, Anna Lyubarskaja, Ruiqi Zhong. Serina Chang, and Sammi Cheung for help- ful discussions. Thanks to Sidhika Balachandar, Divya Shanmugam, and Gabriel Agostini for qualitative annota- tion. Thanks to the ICML reviewers for helpful sugges- tions (specifically, to include SAE ablations and qualitative evaluation). Thanks to Neil Movva and OpenAl for LLM inference credits.
Breuer, A., Dietrich, B. Crespin, M. H.,, Butler, M., Pyrse,
J. A., and Imai, K. Using Al to Summarize US Presi- dential Campaign TV Advertisement Videos, 1952-2012, March 2025. Comment: 17 pages, 7 tables, 4 figures, and linked datasets.
Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2, 2023.
Funding: RM is supported by NSF DGE #2146752. NG is supported by NSF CAREER #2339427, and Cornell Tech Urban Tech Hub, Meta, and Amazon research awards. JK is supported by Simons Collaboration grant and grant from the Aachrthur Foundation. EP is supported by Google Re- search Scholar award, an A12050 Early Career Fellowship, NSF CAREER #2142419, a CIFAR Azricli Global scholar- ship, gift to the LinkedIn-Corndd Bowers CIS Strategic Partnership, the Survival and Flourishing Fund, Open Phi- lanthropy, and the Abby Joseph Cohen Faculty Fund.
Card, D., Chang, S., Becker, C., Mendelsohn, J., Voigt, R., Boustan, L., Abramitzky, R., and Jurafsky, D. Compu- tational analysis of 140 years of US political speeches reveals more positive but increasingly polarized framing of immigration. Proceedings of the National Academy of Sciences, 1 119(31)) 200:e22222521111 101 01,9, August 2022.
Chang, J., Gerrish, S., Wang, C., Boyd-grabes J., and Blei, D. Reading Tea Leaves: How Humans Interpret Topic Models. In Advances in Neural Information Processing Systems, volume 22, Curran Associates, Inc., 2009.
Impact Statement
Our work advances the field of Al-assisted hypothesis gen- eration, which we hope will drive progress the social and natural sciences. To the best of our knowledge, there are no particular negative social consequences imposed by our work compared to machine learning research in general.
Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L.. Sparse autoencoders find highly inter- pretable features in language models. arXiv preprint TrXiv:2399008000 2023.
References
Demszky, D., Garg, N., Voigt, R., Zou, J., Gentzkow, M., Shapiro, J., and Jurafsky, D. Analyzing Polar- ization in Social Media: Method and Application to Tweets on 21 Mass Shootings, April 2019. Com- ment: NAACL 2019; code and data available at https//ggiibbb.ommmmeenks//fiann
Aubin Le Quéré, M. and Matias, J. N. When curiosity gaps backfire: Effects of headline concreteness on information selection decisions. Scientific Reports, 15(1):994, January 2025. ISSN 2045-2322.
Banerjee, A. and Urminsky, O. The Language That Drives Engagement: A Systematic Large-scale Analysis of Head- line Experiments. Marketing Science, November 2024. ISSN 0732-2399.
Dunlap, L., Zhang, Y., Wang, X., Zhong, R., Darrell, T. Steinhardt, J., Gonzalez, J. E., and Yeung-Levy, S. De- scribing Differences in Image Sets with Natural Lan- guage, April 2024. Comment: CVPR 2024 Oral.
Elhage, N., Hume, T. Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., Grosse, R., McCandlish, S. Kaplan, J., Amodei, D., Wattenberg, M., and Olah, C. Toy Models of Superposition. September 2022. Comment: Also available https:/ttrassormmrr fireuisspub/20022oyymmom .mdel//index.htm..
Batista, R. M. and Ross, J. Words that Work: Using Lan- guage to Generate Hypotheses, July 2024.
Bills, S., Cammarata, N., Mossing, D., Tillman, H., Gao, L.,
Goh, G., Sutskever, I., Leike, J., Wu, J., and Saunders, W. Language models can explain neurons in language models. https: httpss://openaipublicc blob. core. windows. net//neuroon-eepllaaerre/eeer/ee
blob. html, 2023.
Feng. J., Kothari, A., Zier, L., Singh, C., and Tan, Y. S. Bayesian Concept Bottleneck Models with LLM Priors, October 2024.
Blei, D. M., Ng. A. Y., and Jordan, M. I. Latent dirichlet al- location. J. Mach. Learn. Res., 3(n01):993-1022, March 2003. ISSN 1532-4435.
Gao, L., Ia Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J. Scaling and evaluating sparse autoencoders, June 2024.
Bollen, J., Mao, H., and Zeng, X.-J. Twitter mood predicts
10

---

Sparse Autocncoders for Hypothesis Generation
Gentzkow, M. and Shapiro. M. What Drives Media Slant? Evidence From U.S. Daily Newspapers. Econometrica, 78(1):35-71, 2010. ISSN 1468-0262.
Lam, M. S., Teoh, J., Landay, J., Heer, J., and Bernstein,
M. S. Concept induction: Analyzing unstructured text with high-level concepts using Iloom, 2024. URL https: //doi org/10. 1145/361 3904. 3642830.
Gentzkow, M., Shapiro, J. M., and Taddy, M. Measur- ing Group Differences in Highh-Diensional Choices: Method and Application to Congressional Speech, July 2016.
Ludan, J. M.. Lyu, Q. Yang, Y., Dugan, L.,, Yatskar, M., and Callison- Burch, C. Interpretable-by- by-essgnn Text Under- standing with Iteratively Generated Concept Bottleneck, April 2024.
Gligorić, K., Lifchits, G., West, R., and Anderson, A. Lin- guistic effects on news headline success: Evidence from thousands online field experiments (Registered Re- port). PLOS ONE, ((3))e0281822 March 2023. ISSN 1932-6203.
Ludwig, J. and Mullainathan, S. Machine Learning Tool for Hypothesis Generation, March 2023.
Makhzani, A. and Frey, B. K-Sparse Autoencoders, March 2014.
Grimmer, A bayesian hierarchical topic model for politi- cal texts: Measuring expressed agendas in senate press releases. Political analysis, 18(1):1-35, 2010.
Matias, J. N., Munger, K., Le Quere, M. A., and Ebersole, C. The Upworthy Research Archive, a time series of
Grimmer, J., Roberts, M. E., and Stewart, B. M. Machine Learning for Social Science: An Agnostic Approach. Annual Review of Political Science, 24(Volume 24, 2021): 395-419, May 2021. ISSN 1094-2939, 1545-1577.
32,487 experiments U.S. media. Scientific Data, 8(1): 195, August 2021. ISSN 2052-4463.
Merity, S., Xiong, C., Bradbury. J., and Socher, R. Pointer Sentinel Mixture Models, September 2016.
Grimmer, J., Roberts, M. E., and Stewart, B. M. Text as Data: A New Framework for Machine Learning and the Social Sciences. Princeton University Press, Jamuary 2022. ISBN 778-0-691-22079955
Monroe, B. L.,, Colaresi, M. P., and Quinn, K. M. Fightin' Words: Lexical Feature Selection and Evaluation for Iden- tifying the Content of Political Conflict. Political Anal- ysis, 16(4):372-44) 403, August 2009. ISSN 1047-1987, 1476-4989.
Grootendorst, M. BERTopic: Neural topic modeling with class-based TF-IDF procedure, March 2022. Com- ment: BERTopic has python implementation, see http:///ttuubbooo//Maattttf/B//
Nelson, L. K. Computational grounded theory: A method- ological framework. Sociological Methods & Research, 49(1):3 2020.
Grootendorst, M. LLM & Generative Al BERTopic.
http://aarooggggiiooooooEEEogeee wttp::ttnnnnttttttedderreseennie 2024.
(tm ttvevvittee,, A., and Schubert, L. Feature Visual- ization. Distill, 2(11):10.238 11.....99///ssiiiii 5/distiil.00007, November 2017. ISSN 2476-0757.
Hoyle, A. M., Sarkar, R., Goel, P., and Resnik, P. Are Neural Topic Models Broken? In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.). Findings of the Association for Com- putational Linguistics: EMNLP 2022, pp. 5321-5344. Association for Computational Linguisties, December 2022.
O'Neill, C., Ye, C., Iyer, K., and Wu, J. F. Disentangling Dense Embeddings with Sparse Autoencoders, August 2024.
Pham, C. M., Hoyle, A., Sun, S., Resnik, P., and Iyyer, M. TopicGPT: A Prompt-based Topic Modeling Framework, April 2024.
Hsu, C.-C., Obermeyer, 7., and Tan, C. Clinical Notes Reveal Physician Fatigue, December 2023.
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., and Sayres, R. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activa- tion Vectors (TCAV). In Proceedings of the 35th Interna- tional Conference on Machine Learning, pp. 2668 2677.. PMLR, July 2018
Ranard, B. L.., Werner, R. M., Antanavicius, T., Schwartz, H. A., Smith, R. J., Meisel, Z. F., Asch, D. A., Ungar, L. H., and Merchant, R. M. Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care. Health Affairs, 35(4):697-705, 2016.
Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., and Liang, P. Concept Bottleneck Models, December 2020. Comment: Edited for clarity from the ICML 2020 version.
Rathje, S., Van Bavel, J. J., and Van Der Linden, S. Out- group animosity drives engagement on social media. Pro- ceedings of the National Academy of Sciences, 118(26): e20242921 18, 2021.

---

Sparse Autocncoders for Hypothesis Generation
Robertson, C. E., Pröllochs, N., Schwarzenegger, K., Pärnamets, P., Van Bavel, J. J., and Feuerriegel, S. Neg- ativity drives online news consumption. Nature Human Behaviour, 7(5):112-88) May 2023. ISSN 2397-3374.
Zhou, Y., Liu, H., Srivastava, T., Mei, H., and Tan, C. Hy- pothesis Generation with Large Language Models, Au- gust 2024.
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang. Z., and Yang. D. Can Large Language Models Transform Com- putational Social Science?, February 2024. Comment: To appear in "Computational Linguisties" (CL).
Robitschek, E., Bastani, A., Horwath, K., Sordean, S., Pletcher, M. J., Lai, J. C., Galletta, S., Ash, E., Ge, J., and Chen, Y. A large language model-based approach to quantifying the effects of social determinants in liver transplant decisions, January 2025. Comment: Spotlight Paper, ML4H 2024; Leonidas H. Berry Health Equity Re- search Award, ACG 2024; Plenary Presentation, AASLD 2024.
Sun, C.-E., Oikarinen, T., Ustun, B., and Weng, T.-W. Con- cept Bottleneck Large Language Models, December 2024. Comment: arXiv admin note: substantial text overlap with arXiv:2407.0453
Sun, M., Oliwa, T., Peek, M. E., and Tung, E. L. Negative Patient Descriptors: Documenting Racial Bias In The Electronic Health Record. Health Affairs, 4122)200--211
February 2022. ISSN 0278-2715.
Templeton, A., Conerly, T., Marcus, J., Lindsey. J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E,, Jones, A., et al. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. transformer circuits thread, 2024.
Tibshirani, R. Regression Shrinkage and Selection Via the Lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267-28) 288,, January 1996. ISSN 0035-9246.
Ting, P.-J. L., Chen, S.-L., Chen, H., and Fang, W.-C. Using big data and text analytics to understand how customer experiences posted on yelp. com impact the hospitality industry. Contemporary Management Research, 13(2), 2017.
Vig, J.. Madani, A., Varshney, L. R., Xiong, C., Socher, R., and Rajani, N. F. BERTology Meets Biology: Interpret- ing Attention in Protein Language Models, March 2021. Comment: To appear in ICLR 2021.
Yang, Y., Panagopoulou, A., Zhou, S., Jin, D., Callison- Burch, C., and Yatskar, M. Language a Bottle: Language Model Guided Concept Bottlenecks for In- terpretable Image Classification, April 2023. Comment: Published in CVPR 2023, 18 pages, 12 figures, 16 tables.
Yelp.
Yelp http://wwsieessseeeccccc/aaareee 2024.
Open Dataset.
Zhong, R., Wang, H., Klein, D., and Steinhardt, J. Explain- ing Datasets in Words: Statistical Models with Natural Language Parameters, September 2024.
12

---

Sparse Autocncoders for Hypothesis Generation
REFERENCE
INFERRED
SURFACE
Other music articles >> Porformers, groups, composers, and music-related people
about a rock band or a musical act
0.39
0.5
Earth science Tropical eyelones: Atlantic
hurricanes or tropical storms
Battles, exercises, and conflicts > Modern history (1800 to present)
about a Nistorical battle
0.36
0.5
Television The Simpsons episodes
about episodes of The Simpsons television show
0.92
Transport > Road infrastructure: Northeattem United States
is about a New York State Route
0.71
0.5
Table 4. Evaluating hypotheses inferred by HyPoTHESAEs against the reference topics on WIKI.
Source
Overall AUC
Sig Significant Hypotheses: Red: Republican
Bluc: 1 Democrat
Sep.
AUC
contains the phruse ask
0.30
0.53
hearings
Committees
mentions scheduling details about Senate votes or amendments
0.17
discusses illegal immigration and ossociated implications
0.51
mentions victories or successes in militory or political contexts mentions froedom or liberty
0.15
0.51
0.06
0.50
discusses economic growth growth rates
0.01
0.50
HYPOTRESAES
0.702
15 discusses government spending or budgetary issues
-0.09
0.54
mentions the need for reform proposes reforms mentions figures related to the civil rights critcizes government policies actions
-0.17
0.58
.0.22
0.51
.0.29
0.63
national debt raising debt limit or advantages wealthy
-0.40
-0.44
criticizes Republicon leadershap or policies
criticizes administration' handling of the Traq war
-0.45
0.52
Senote procedural requests and adjouronent schedules Requests
0.39
0.53
authorize committee meetings
0.53
procedunal diseussions scheduling of vates
discusses ommigration and border security mentions individuals with profersional ritles or roles debates on earmarks and federal spending processes
0.13
0.51
0.08
0.53
BERTOPIC
0.636
0.05
0.50
discusses the Darfor conflict and International intervention Congressional oversight of defouse contracting Irag discusses Asian Pacific Americon communities
-0.30
0.50
.0.33
0.50
-0.36
0.50
Critiques Republican majority leadership
-0.46
0.55
expresses strong patriotism
0.09
0.51
contains specific legislative references
0.08
0.53
mentions specific individuals organizations
-0.03
0.52
includes namerical data
-0.08
0.53
NLPARAM
0.650
addvesses critical national issue
.0.14
0.56
expresses strong support opposition
-0.15
0.58
advocates for specific groups employs emotional
-0.17
language
-0.23
focuves national security and immigration enforcement discusses social ismes such healthcure and civil rights
0.18
0.51
-0.22
0.58
HYPOGENIC
0.675
of government inefficiencies calls reform odvocates
-0.27
0.62
critiques government
emphasizes need for government accountability
Table CONGRESS hypotheses, each method, with p 0.05 Bonferroni correction a multivariate regression. The 'Sep.' column shows the separation score: Æ E 0), where positive scores predict Republican speech and negative scores predict Democratic speech. The 'AUC' column provides univariate AUCs for individual hypotheses. 'Overall AUC" uses all hypotheses, including insignificant ones.
13

---

Sparse Autocncoders for Hypothesis Generation
Souree
R
Sig
Significant Hypotheses: Green:
ratiig
Red: 4. rating
Scp.
R²
expresses extreme entlossoamm or excitement with phrases like 'holy fack', "can't
1.45
0.23
go back'. uses exclamation enfuuriastically consey positive sentiment
exclamation potrats
mentions anticipation or plans to return the restaurant or order again
1.20
0.17
explicitly states that food ieem or restaurant is the best, often wring superlative longuage
1.10
0.07
mediocrity or averageneis of food experience related food safety, hygiene, or improper food handling
HYPOTIESAES
0.755
13
mentions issues restourant staff communication or behavior
mentions dissatisfaction criticism of service, staff, or management mentions food poisoning or symptoms of foodborne illeess mentions rude aggressive behavior from restaurant staff mentions dissatisfaction with restauront policies, rules, restrictions criticizes the behavior or atiinde of staff complains about rude unprefessional behavior from staff management
2.32
0.02
-2.37
0.30
-2.37
0.53
-2440
0.50
-2.42
0.46
mentions dishomest anethical business practices the establistment describes specific instance poor customer
2.48
0.17
2.50
0.50
praises the combination of excellen food quality and friendly, attentive service describes deteiled experiences with food quelity, proprration and flavor munnces
1.61
0.33
0.46
0.03
crificizes the fresimess, quality, preparation sushi and fish dishes
-1.77
0.07
describes issues with
closures inaccwate operating hours
-1.77
0.10
complains about uncleanliness restaurontt criticizer quality authemicity of Chinese dishes criticizes Iullia dishes, particularly pasta souces complains about iszues with pizza orders
0.05
BERTOPIC
0.647
12
1.94
-2.08
0.16
criticizes breakfast food quality, portion sizes, and service experience complains about ismues with delivery orders
-2.15
0.37
-2.20
0.25
complains about poor service related to waiting times and staff behavior complains about rude or unprofexrional behavior from bartenders or bar stuff
-2.31
0.49
2.31
0.31
uses enthusiastic or excersively positive language describes exceptional food quality
1.90
0.47
1.65
describes positive recurring experience recommends the restaurant to others
1.45
0.26
1.29
0.18
uses exclamation marks to express entlusiasm
1.22
0.17
NLPARAM
0,583
II
mentions attentive or friendly staff
1.01
0.12
expresses enthusiamm for ambiance atmosphere
0,92
0.07
mentions thoughtful accommodation
preferences
0.79
describes generosity in portion
comments on portion size
long wait to receive service
highlights pleasant atmosphere enjoyable dining experience expresses willingness
2.40
0.66
the
others
2.13
0.55
expresses
retunn
the place
describes food delicious highlights specific dishes positive adjectives
notes issues with food quality, such as being coid or poorly cooked
-187
0.29
includes compleints about long wait times, for food or service
1194
0.30
HYPOGENIC
0.775
12 expresses di disappointment with the quelity of food compared expectations
2.17
0.49
mentions poor service unresponsive staff during the dining experience complains about poor customer service or rude staff complains about service unhelpfai staff expresses disappoietmmnt with the overall dining experience expresses di cipppinnnm with overall experience ambiance
-.36
0.51
2.41
0.50
-2.42
0.60
-2.42
0.67
2.43
0.67
Table YELP hypotheses, for each method, with p 0.05 after Bonferroni correction multivariate regression. 'Sep.' is the separation score: g 1| ž = 0), where positive scores predict higher restaurant ratings. The " column provides univariate R² values of individual hypotheses. 'Overall R2 uses all 20 hypotheses, including insignificant ones. Hypotheses for methods are abbreviated.
14

---

Sparse Autocncoders for Hypothesis Generation
Method
Total Count
Avg. Predictiveness
Avg. Fidelity
SAE
45 /
0.717
0.836
Embedding
30/ 60
0.706
0.545
PCA
23 / 60
0.697
0.547
Bottleneck
27 / 60
0.699
0.640
Table 7. Experiments testing ablations of the SAE on the three real-world datasets: HEADLINES, YELP, CONGRESS. Using the SAE yields many more significant hypotheses. slightly more predictive hypotheses, and much higher fidelity interpretations (iee. interpretations that better explain the underlying neuron activations). The count of significant hypotheses is aggregated across the three datasets. Predictiveness is averaged across the three datasets. Fidelity is averaged across the 60 selected neurons (3 datasets, 20 neurons selected per dataset).
A. Data Preprocessing
WIKL. We download the dataset processed by Pham et al. (2024); the data are derived from the WikiText corpus assembled by Merity et al. (2016). The articles are categorized by Wikipedia editors; each article has supercategory, category, subcategory (e.g., Media and Drama > Television > The Simpsons Episodes). We focus on recovering article subcategories, since these are the most specific and therefore most difficult to fully recover. After filtering out duplicates and infrequent (<100 articles) subtopics, the dataset contains 11,979 items spanning 69 subcategories. For the WIKI-5 dataset, we set the ground-trute topics to the most common subcategories. That is, we label all 1,771 articles (14.8%) in these subcategories as positives, while all other article subcategories are negatives. The W1KI-15 dataset is constructed similarly using the 15 most common subcategories (4,190 positives, 35.0%).
We also download BILLS from Pham et al. (2024); the dataset was originally assembled by Hoyle et al. (2022) using GovTrack The bills are from the 110th- 1 14th U.S. Congresses (Jan 2007 Jan 2017). Each bill has topic and subtopic. After filtering out very short bill texts (which may not have been properly transcribed), duplicate bill texts, bills with no subtopic, and bills with infrequent (<100) subtopics, there are 20,834 items spanning 70 subtopics. We label the 5,038 bills (24.2%) with the 5 most common subtopics as positives, and all others as negatives.
BILLS.
HEADLINES. We use the HEADLINES dataset as collected and released by the Upworthy Research Archive (Matias et al., 2021). All of the data is derived from web traffic to Upworthy.com, a digital media platform known for its attention- -raabbing articles. The dataset consists of thousands of A/B tests, where users were randomly shown one of several possible headline variations for the same underlying article. Each headline is linked to a corresponding number of impressions and clicks, allowing researchers to estimate the causal effect of headline text click-rate (clicks/ selicks/impeessiiii /impressions, also referred as clickthrough-rate or engagement).
To preprocess the data, we start by downloading all publicly available data from the Upworthy archive?, which includes clicks and impressions for 1 122,565 distinct headlines (we remove 28K rows from a problematic data range as recently reported by the dataset ) We then group headlines which have the same test.id (signifying that those headlines were randomized against one another). There are 14,128 such groups with at least two headlines. Following Zhou et al. (2024), we take the headlines with max and min click-rates in each group, and design binary classification task predict which headline in the pair received higher engagement. That is, we construct tuples (headline A, headline B, label), where headline A and headline B are randomly shuffled, and the label is 1 if headline A received higher click-rate than headline B.
Following Batista & Ross (2024), we take specific care to prevent train-holdout leakage on this dataset. Specifically, the same headline texts may be used in multiple randomized experiments (i.e., the same headline may occur in different test test_id's) We ensure that no identical headlines will appear in both the train split and the heldout split; we encourage future researchers to follow this practice, since splitting on test.id alone does not fully prevent headline leakage.
To handle the pairwise design, train the SAE the embeddings of all unique headlines in the training set, and then compute the activation matrices ZÁ and SAA for all of the headline A's and headline B's respectively. We select predictive
hhtps://upwortyy. http:://upwootty..mmmaaasssssso wwt.:/ynnaatmmtiicccmonoeeeeeeee
"htps://upworthy
15

---

Sparse Autocncoders for Hypothesis Generation
neurons by identifying columns in the matrix difference AZAAE ZSSA - which are predictive of y.
YELP. We download the Yelp Open Dataset', and we filter to the 4.72M reviews where the business contains the 'Restaurant" tag. From this, we randomly sample training, validation, and heldout sets (200K, 10K, and 10K respectively).
CONGRESS. We download all congressional speech transcripts from the 109th U.S. Congress (Gentzkow & Shapiro, 2010)"0, which met from January 2005 to January 2007. We filter speeches which are very short (less than 250 characters) and include only speeches from Republican or Democrat speakers; the dataset is roughly balanced (51.7% Republican). If speech is longer than ten sentences, we split it into ten sentence chunks, and exclude any chunks beyond the first five (to avoid over-representatioo specific speeches), For the heldout set, we restrict to one chunk per speech to ensure that our hypotheses generalize to a diverse corpus. Ultimately, we use 114K speech chunks for training, 16K for validation, and 12K for heldout eval. If a speech has multiple excerpts, we ensure that all of them are included in the same split to avoid leakage.
B. plppepaarameter
B.1. Training sparse autoencoders
We train k-sparse autoencoders, where each forward pass masks all neurons besides the top-k activating ones (Makhzani & Frey, 2014; Gao et al., 2024). The most influential hyperparameters, which we tune for each dataset, are M and k. We choose (M, k) by maximizing validation performance. Specifically, for each combination (M, k). we evaluate validation AUC (or R on YELP) after fitting an Lı-regularized predictor with exactly H < M nonzero coefficients. We constrain M and & to powers of two to tractably limit the search space.
These parameters, broadly, control the level of granularity of the concepts learned by each SAE neuron: M is the total number of concepts which can be learned across the entire dataset, and k is the number of concepts which can be used to represent each instance. For intuition, assume that with (M, k) = (16, 4) on YELP, the SAE learns a single neuron that fires when reviews mention price; with (M, k)) = (32, 4), the SAE may learn separate neurons for reviews which mention high prices vs. low prices (described as "feature splitting" in Bricken et al. (2023)); with (M, A) = (32, 8), the SAE may learn to represent more niche features altogether, such as the number of exclamation marks. In general, increasing either parameter yields more granular features, at the risk of producing some neurons that are uninterpretable or redundant.
In some cases we would like features at multiple levels of granularity: for example, we may want both the general feature "mentions prices" and the more specific feature "mentions that the cocktails were cheap during happy hour", but training a single large SAE (e.g. (1024, 32)) may only produce features as specific as the latter. A simple solution is to train multiple SAEs of different sizes, concatenate their activation matrices, and select features from any of them. We find that this approach is helpful (improves validation AUC) on the HEADLINES dataset. Ultimately, we use the following values:
WIKI-5: (32,4)
WIKI-15 & BILLS: (64, 4).
HEADLINES: (256, 8); (32, 4).
YELP: (1024, 32).
CONGRESS: (4096, 32).
Following prior work, we add several optimizations improve SAE training. Following best practices based on results from Bricken et al. (2023), we (i) tie the weights of the pre-encoder bias and the decoder bias, bpre = bdec: (ii) initialize 6pre to the geometric median of the training set; (iii) initialize Waec WT and (iv) force all decoder rows to have unit norm.
To minimize the number of "dead" latent neurons in the SAE-neurons which are never selected by the TopK filter we use an auxiliary loss (Gao et al., 2024; O' Neill et al., 2024). The auxiliary loss term provides nonzero gradients for up to
hhups://businesss.yy htntp:://tataassaafood... www./oooeoorrruu/oooereeeesssss text
16

---

Sparse Autocncoders for Hypothesis Generation
kaux neurons which were not active in the last several forward passes. We fit these neurons the residual reconstruction emror left by the TopK neurons. That is, let 2 zuxk consist of the activations of the top kaux dead neurons in 2;; then
Laxx
2e Auxk - (es E.)1
and the full loss, then, is L Topp + Wanx C
Hyperparameters for the auxiliary loss and other training hyperparameters are given below (identical for all experiments):
kaux: 2 k; that is, up to 2 dead neurons are used to fit the TopK reconstruction error.
Dead neuron threshold steps: 256. This is the number of consecutive steps for which a neuron was not selected by TopK to be considered "dead."
Auxilliary loss coefficient Wanx: 1/32. This parameter gives the weight of the auxiliary reconstruction's 's loss.
Batch size: 512; learning rate: 5e-4; gradient clipping threshold: 1.0.
Epochs: Up to 200, with early stopping after 5 epochs of validation loss not decreasing.
B.2. Interpreting neurons with LLMs
There are several parameters for neuron interpretation which we fix across experiments:
Interpretation model: GPT-4o (version 2024-11-20).
Temperature: 0.7.
Number of highly-activating examples: 10.
Number of weakly-activating examples: 10.
Maximum word count per example: 256 (examples longer than this are truncated).
Number of candidate interpretations: 3 (we choose the highest-fidelity interpretation out of candidates).
Number of samples to evaluate fidelity: 200.
In Appendix C, we describe experiments which justify these choices: they either improve fidelity have no effect compared alternate choices. We also provide the full interpretation prompt in Figure 5.
There is one hyperparameter which we adjust per dataset: the bin percentiles from which we sample highly-activating and weakly-activatiing examples for the neuron interpretation prompt. By default, highly-activating texts are sampled from the top decile of positive activations [90, 100), and weakly-activating texts are sampled from the bottom decile of positive activations [0, 10]. This works well general, especially for larger datasets (YELP, CONGRESS). For smaller datasets like WIKI, the [90, 100] bin produces interpretations which are too specific (e.g., "articles about Napoleon", even if the neuron fires on articles about European leaders more generally). Note that this requires a judgment call: what is "too specific" depends practitioner needs. For example, WIKI, choose bins such that the resulting interpretations roughly granular as the topics in the "!taser We ultimately choose the following percentile bins for highly-activating texts:
WIKI-5, W1KI-15, HEADLINES: [80, 100).
BILLS: [50, 100).
YELP, CONGRESS: [90, 100).
"Choosing this parameter requires similar decision-making to choosing minimum cluster size BERTOPIC.
17

---

Sparse Autocncoders for Hypothesis Generation
Choosing the number of candidate interpretations depends budget. In Appendix C we find that selecting the interpretation with highest fidelity from pool of candidates increases the resulting fidelity modestly, yet statistically significantly; we therefore use 3 candidate interpretations for all experiments. However, we also ablate this thap is, generating single candidate interpretation and using it directly, without computing fidelity. On CONGRESS (we did not test on the other datasets), the resulting hypotheses are of similar quality: 14/20 are significant, with AUC 0.70, compared to 15/20 significant and AUC 0.70 when using 3 candidate interpretations.. This variant of our method, SAE-NOVAL, costs as little as BERTOPIC (Table 3).
C. Supporting Experiments
Our final neuron interpretation procedure for our main experiments is described in $4, with hyperparameters provided in Appendix B.2. Here, we describe several experiments to justify our procedure. The section is organized as follows. In C.I, we show that our metric of interpretation fidelity-the the F1 score comparing concept annotations against neuron activations-is a useful selection metric to improve downstream hypothesis quality. In C.2, we test several different interpretation strategies and hyperparameters and measure their impacts on fidelity. In C.3, we analyze how much prediction signal is lost each stage of our method, shedding light on how future work can improve hypothesis quality.
C.1. Higher-fidelity interpretations improve hypotheses.
To test whether fidelity is useful metric to select from different neuron interpretations, we conduct the following experiment on the Yelp dataset:
1. Train an SAE (Step 1), and identify the top-20 predictive neurons via Lasso (Step 2).
2. Generate 10 candidate interpretations (at temperature 0.7, and with different random samples in the interpretation prompt) for each neuron using the hyperparameters in B.2 (Step 3).
3. Compute the fidelity for each candidate interpretation: that is, binarizing 100 examples in the top decile of the neuron's 's positive activations as positives, and 100 examples in the bottom decile of the neuron's positive activations as negatives, compute the Fl score between the GPT-4o-mini annotations according to the interpretation against the binarized neuron activations.
4. Store two sets of neuron interpretations: one set includes the interpretations with the best F1 score for each neuron (high-fidelity), and the other set includes the interpretations with the worst Fl score for each neuron (low-fidelity).
5. Compute annotations on the full heldout set (10K examples) for the high-fidelity hypotheses and the low-fidelity hypotheses, and compare how predictive they are as measured by R².
This procedure yields three interesting results. First, there is enough variation in the 10 candidate interpretations such that the best and worst Fl scores are substantially different: the median best F1 is 0.79, while the median worst F1 0.53, Second, find that the high-fidelity set is indeed more predictive: in multivariate OLS, the high-fidelity set achieves an R2 of 0.76, compared to 0.61 for the low-fidelity set.
Third, most importantly, using a neuron's 's Figure shows that, for 17 out of 20 neurons, moving from the low-fidelity interpretation to the high-fidelity interpretation increases the R2 of the resulting hypothesis (p-value, paired t-test: 0.002). For example, for one neuron, the high- interpretation is "mentions disputes or issues related to billing or refunds" (F1: 0.79), while the low-fideliyy interpretation more generically "mentions an attempt by the restaurant or management address or respond to a problem or complaint." The former hypothesis achieves heldout R² 0.10 with restaurant rating, while the latter is not significantly correlated with rating (R²² = 0.003). These experiments show that fidelity-as as measured via Fl score-is a useful heuristic to choose between multiple candidate neuron interpretations, when our goal is to produce predictive hypotheses.
interpretation results in more predictive hypothesis, on average.
Finally, following prior work (Bills et al., 2023), we also tested another metric of fidelity: the R² between the concept annotations and the neuron activations across 100 top-activating samples and 100 random samples. Choosing the top candidate interpretation based on this alternative fidelity metric did not produce hypotheses that were significantly more or less predictive than using our Fl fidelity metric.
iToo illustrate this point, we tested solely on CONGRESS; we expect similarly modest differences on other datasets.
18

---

Sparse Autocncoders for Hypothesis Generation
0.6
paired t-test:
p = 0.002
0.5
(hyypothesis, rating)
0.4
0.3
0.2
0.1
0.0
Low F1
High F1
Figure 3. For 17 of the top 20 neurons on YELP, a higher-fidelity neuron interpretation results in a more predictive hypothesis. Fidelity highly-activating and
measured by score between the concept annotations and neuron activations on sample weakly-activating examples. We generate 10 candidate interpretations for each neuron, and compare the interpretations with best- and worst-F1. Predictiveness is measured by computing the R² between concept annotations and Y (restaurant rating) heldout examples.
C.2. Effects of the neuron interpretation procedure on fidelity.
Having established that improving interpretation fidelity improves downstream predictiveness, we search across different hyperparameters for neuron interpretation maximize fidelity. Our final procedure generate 3 candidate interpretations for each neuron by prompting GPT-4o with 10 examples in the top decile of positive activations vs. 10 examples in the bottom decile of positive activations. We use the highest fidelity interpretation out of these 3. There are two sources of non-determinism: each candidate interpretation is generated with a different random seed to sample examples, and we use an LLM temperature 0.7.
To compare this strategy against other strategies- to compare strategy and strategy B more generally- compute interpretations, and their Fl scores, for the top 100 neurons on YELP under both strategies separately. We run t-test comparing the pairs of resulting F1 scores to test whether one strategy produces consistently higher Fl scores than the other.
We observe the following:
Number of candidate interpretations: Relative to baseline (generating 3 candidate interpretations and taking the one with highest F1 score), generating only single candidate interpretation is significantly worse (-0.04). Generating candidate interpretations improves significantly on 3 (+0.01 F1, paired t-test p 0.003), but the improvement is modest and results in slower, more expensive method. Therefore, we use only 3 candidate interpretations.
Top-random vs. high-weak sampling strategy: Prior work (e.g. Templeton al. (2024); Bills et al. (2023)) interprets SAE neurons by prompting with the top-activating texts and random texts. Relative our baseline (prompting with highly-activating texts, but not necessarily oopacciiatiing: and weakly-activating texts instead of random), this alternate approach has no significant effect on Fl score, though the interpretations tend to be more specific: that is, they have higher precision and lower recall. This is because a neuron's is absolute top-activating examples generally share more specific characteristic than examples sampled randomly from the top decile. Though neither strategy wins clearly in terms F1, we choose high-wee weak sampling for our experiments because, empirically, top-random produces hypotheses which can be too specific (thereby lowering predictiveness); this discussed further in B.2. Our code defaults to top-random for
19

---

Sparse Autocncoders for Hypothesis Generation
simplicity, but practitioners should treat sampling strategy tunable hyperparameter that depends on desired hypothesis granularity.
Continuous vs. binary prompt structure: Prior work (e.g. Zhong et al. (2024)) prompts the interpreter LLM with a sorted list of texts and their continuous activations. Relative baseline (showing separate lists of positive samples and negative samples), this significantly lowers Fl score (-0.06),
use the binary prompt structure.
Chain-of-thoughtt Prior work (e.g. O'Neill et al. (2024)) uses chain-of-thougge reasoning prompt interpret neurons, where the LLM is encouraged to iteratively discover a concept which all of the positive samples contain and none of the negative samples contain. Relative to baseline, we found that CoT has an insignificant but weakly negative effect (-0.03 mean F1), and also sometimes fails to output any interpretation. However, this is still a promising line for future work: it's possible that while GPT-4o does not reason well, better models or reasoning-speciff models (OpenAl o1, DeepSeek R1, etc.) will produce higher fidelity labels with chain-of- thhough....
Relative to baseline (prompting with 20 total samples, 10 highly-activating and 10 weakly-activatingg) prompting with total samples lowers average Fl score by 0.03. Prompting with 50 samples has no effect, so we use 20 favor shorter prompts.
Sample count:
Temperature: Relative to baseline (temperature 0.7), using a lower (0.0) or higher (1.0) temperature has an insignificant but weakly negative effect (-0.01 in mean FI), so we use 0.7.
C.3. Localizing losses in predictive performance to different steps of the method.
HYPOTHESABE starts from embeddings and ultimately produces hypotheses, after three steps. It is worth asking- how does predictive performance change at each step of our method? Answering this question points to possible improvements: if the SAE's S full hidden representation is much less predictive of restaurant ratings than the input text embeddings, perhaps we need a larger (more expressive) SAF, or more data to train the SAE's representations; meanwhile, if the largest performance loss comes from translating neurons into natural language hypotheses, perhaps we need a stronger LLM for interpretation.
Figure 4 displays this experiment. At each stage, we fit a logistic or linear regression on the training set to predict the labels on HEADLINES, YELP, and CONGRESS. The four stages are: (i) text embeddings (length-1536 vectors of floats); (ii) the full SAE activation matrix, ZSAB (length-288, 1024, and 4096 vectors of floats for the three tasks, respectively); (iii) the activations of the top-20 selected SAE neurons (length-20 vectors of floats); and (iv) the annotations from the interpreted hypotheses (length-20 vectors of binary 0/1 values).
0.90
0.85
0.80
AUC or R?
0.75
0.70
Headlines
0.65
Yelp
Congress
0.60
Embedding
Full SAE
Top-20 SAE Top-20 Hypotheses
Figure 4. Performance losses at each step of the HYPOTHESAES pipeline. The leftmost point shows the performance of training a supervised single-layer model to predict the outcome using text embeddings; the second point shows the supervised performance using the full SAE representation; the third point shows the supervised performance using the top-20 SAE neurons; the rightmost point shows the supervised performance using the top-20 hypothesis annotations after neuron interpretation.
The trends depend on the dataset. On HEADLINES, we find that the performance drops most at the first stage (-0.08 AUC): the full SAE is significantly less expressive than the original embeddings. This could suggest using a larger SAF, but
20

---

Sparse Autocncoders for Hypothesis Generation
empirically we find that increasing the SAE size does not improve AUC, perhaps because there is not enough data (~14K headlines) make use of more parameters. On YELP, we find that performance is mostly retained until neuron interpration, at which point R2 drops by 0.06. This suggests some combination of: our neuron interpretations are imperfect, GPT--4-mini is not annotating the hypotheses faithfully, or binary annotations are not expressive enough to capture the neuron activations (which are continuous). Finally, on CONGRESS, the only drop comes from using the top-20 neurons to predict the label instead of the full SAE representation (4096 neurons). This implies that 20 features aren't It expressive enough; or, more optimistically, that there are more than 20 distinct hypotheses to discover on this dataset (see $6.4). Remarkably, there are no losses in the interpretation step on the CONGRESS dataset, suggesting that on this dataset our interpretations are very high-fidelitty Ultimately, this analysis can be useful diagnostic tool to direct one's s efforts when trying improve hypothesis quality: losses from embedding + full SAE suggest that the dataset and/or the SAE are too small; losses from full SAE + top- H SAE suggest that H is too small; losses from top-H/ SAE = top- H hypotheses suggest that the interpretation step can be improved. Meanwhile, if the top-/1 hypotheses achieve similar AUC to the underlying embeddings, this implics that there may be no more "juice to squeeze" in the dataset.
D. Hyperparameters for Baseline Methods
BERTOPIC (Grootendorst, 2022). To run BERTOPIC we feed in the same OpenAl embeddings that use for PPOOHESSAE, and otherwise use default choices for the algorithm: UMAP for dimensionality reduction, HDBSCAN for clustering, and c-TF-IDF to compute the top words associated with each topic. We tune the minimum cluster size hyperparameter in BERTOPIC, which controls topic granularity. The default value is 10, which often produces topics which are too granular (c.g., relative to the underlying granularity of the subtopics on WIKI and BILLS). We test all values in {10, 20, 50, 100, 200, 5003. We choose this parameter by maximizing the validation performance of the Lı-reuularized predictor which uses H features; this is identical to how we choose hyperparameters for HYPOTHESAES. We use default values of all other hyperparameters, which is standard in prior work (e.g., Pham et al. (2024)). After running BERTOPIC, we label topics using the prompt Figure 8.
NLPARAM (Zhong et al., 2024). We run NLPARAM using the publicly available implementation 14, We set the number of candidate labels per bottleneck layer neuron to 3 (the same as our method), and we use the same text embeddings as our method (OpenAl's text-3 3-eee--ddng----meeeeeeeee small) Zhong et al. (2024) release two different prompts, one for "simple" hypotheses and one for "complex" hypotheses; following their work, we use the "simple" prompt for our 3 synthetic tasks and the "complex" prompt for our 3 real-world tasks. For all other hyperparameters, we use default values.
HYPOGENIC (Zhou al., 2024). We run HYPOGENIC using the publicly available implementation 15 We use the default parameters as defined in examples/gennration.pp Following the format in their example tasks, we write custom prompts in a consistent format describing each our of tasks. The method uses the same LLM for hypothesis generation and hypothesis scoring, and we run all experiments using GPT-4o-mini (the default model in their code repository, and also the model that we use for scoring hypotheses in HYPOTHESAES).
E. Qualitative Evaluation
To confirm that our hypotheses pass a "sniff test," we recruit three computational social science researchers perform a qualitative evaluation. These researchers are not authors on our study. We collect ratings for two attributes, following Lam et al. (2024): (1) helpfulness and (2) interpretability. We explain these to the raters as follows:
1. Helpful: Does this hypothesis help you understand what predicts the target variable? If you were studying this dataset, is this a hypothesis you would want to and could explore further? Rate if yes, O if no or only little.
2. Interpretable: When you read the hypothesis, is it clear what it means? Can you easily and unambiguously apply it to new examples? Rate 1 if yes, 0 if no or it would often be subjective.
We collect 3 annotations for each of the significant hypotheses outputted by any method on the HEADLINES and CONGRESS
is Sttggz/wiihh...oommteegeegllllls
21

---

Sparse Autocncoders for Hypothesis Generation
datasets. (We removed the YELP dataset reduce annotator load.) Hypotheses shown annotators in random order, i.e,, they grouped by method. We aggregate labels by taking the majority label.
F. Proofs
Proof of Proposition 3.1. Pab Pob = Pr(2 = a, Z = bj and yab == B|y 2 = a, 2 = b] for a, E {0, 13. Then
|| 2 1] V11P11 + y10P10
(8)
P11 P10
Z 2 = 1] 1111 + y01
(9)
P11 Pul
Therefore,
B|y B = -
=
311P11 + M1oP10
311P11 + MonPo1
(10)
P11
P10
P11 Po1
P11
P11
310P10
Y01Po1
V11
(11)
PL1 + P10
P11 pot
P11 + P10
P11 p01
Po1
P1o
310P10
Yon
911
(12)
Pi1 + Pol
Pı1 P10
Pı1 + Pio
P11 Pol
Since y1o < 1 and You 2 0, (12) is bounded from above by
P10
211
(13)
P11
Po1
P11 + P1o
P11 P10
Po1
P10
2 0, (13) is bounded from above by
Poi
P10
+
P10
Po1
(14)
P11
P01
P11
+ P10
P11 P10
P11 + P01
since V/ $ 1. Otherwise, (13) is bounded from above by
P10
(15)
P11 P1o
Also note that (12) bounded from below by
P11
P11
Po1
Po1
P10
Poi
V11
911
(16)
P11 + P1o
P11 + Poi
P11 + P10
P11 + Po1
P11 + P1o
Pi1
P10
If
Pot
P10 P11
< 0, (16) is bounded from below by
P
Poi
P10
Pot
P10
(17)
P11 + P01
PI1 + Pio
PI1 + P1o
P11 + P10
since V11 5 1. Otherwise, (16) is bounded from below by
Po1
(18)
P11 + Po1
Therefore,
E| 2 = E 1]| s max
Pio
Poi
(19)
P11 + P1o
P11 + P01
Analogously, we can bound 2 B 0| by
P10
Pot
max
(20)
Poo + P1o"
Poo + Pot
22

---

Sparse Autocncoders for Hypothesis Generation
Notice that
P10
P10
P11 + P10
(21)
Puu
P10
P11
P10
Poo + P10
Pr|2 1]
P10
(22)
P11 P10
Pr [ 0)
Similarly,
P01
P01
P11 + P01
(23)
Poo + PP1
Pii + Poi
poo + P01
Po1
Priz = 1]
(24)
P11
Pol
Pr[2- =0
Set := min(Pr[2 PrIZ = 0}}- Then Pr/ž 0). Pr[Z 0] 2 p and P 1
Pr[ = 1] Pr[Z 1|
Pr|Z = 0] Pr[ž v 1-p
(25)
It follows that
S(2) S(Z) = (B[Y) 1] E| 0)) (B|Y z 1] B| = 0))
(26)
most
[+ 1-p P
P10
Pui
P1o
Po1
max
max
(27)
P11 + P10
P11 + P01
P11 + P1o
P11 + P01
The result follows by noting that p min (Pr(2 = 0). Pr[Z 0}} and
1 recall, precision.
P11 PPa
P11
G. Prompts
23

---

Sparse Autocncoders for Hypothesis Generation
You machine learning researcher who has trained neural network text dataset. You trying understand what text features cause specific neuron the neural network to fire.
You are given two sets of SAMPLES: POSITIVE SAMPLES that strongly activate the neuron, and NEGATIVE SAMPLES from the same distribution that do not activate the neuron. Your goal is to identify feature that is present in the positive samples but absent in the negative samples.
Example features could be:
"uses multiple adjectives to describe colors"
"describes patient experiencing seizures or epilepsy"
"contains multiple angle-digit numbers"
POSITIVE: SAMPLES:
(positive samples}
NEGATIVE SAMPLES:
Diegative_samples
Rules about the feature you identify:
The feature should be objective, focusing concrete attributes rather than abstract concepts.
The feature should be present in the positive samples and absent in the negative samples. Do not output generic feature which
appears negative samples.
The feature should be as specific as possible, while still applying to of the positive samples. For example, of the positive samples mention Golden or Labrador retrievers, then the feature should be "mentions retriever dogs", not "mentions dogs" or "mentions Golden retrievers".
Do not output anything else. Your response should be formatted exactly as shown examples above. Please suggest exactly one description, starting with and surrounded by quotes ... Your response is:
Figure 5. The prompt interpret SAE neurons. The positive samples that strongly activate the neuron, while the negative samples are texts that weakly activate the neuron. The "example features" section can be edited to produce features in different format.
24

---

Sparse Autocncoders for Hypothesis Generation
Is text_a and text_b similar in meaning? Respond with yes, related, no.
Here are few examples.
Example 1:
lext.a: topic of protecting the environment
text.b: has a topic of environmental protection and sustainability output: yes
Example
text_a: has language German text h: has a language of Deutsch output: yes
Example
text.a: has a topic of the relation between political figures text_b: has topic of international diplomacy
output: related
Example
text.a: has a topic of the sports
text.b: has topic of sports team recruiting new members output: related
Example
text_a: has a named language of Korean
text h: uses archaic and poctic diction output:
Example 6:
text_a: has named language of Korean text_b: has named language of Japanese output: no
Example 7:
text_a: describes important 20th century historical event text. b: describes 20th century European politician
output: no
Target: text_a: [text_a}
text_h: [text_h)
output:
Figure The prompt we use to evaluate surface similarity between inferred hypotheses and reference topics. The prompt is lightly edited from Zhong al. (2024). The few-shot examples demonstrate three levels of similarity: equivalent meaning (yes), related concepts (related), and unrelated concepts (no).
25

---

Sparse Autocncoders for Hypothesis Generation
Check whether the TEXT satisfies PROPERTY. Respond with Yes or No with an explanation that discusses the evidence from the TEXT (at most sentence). When uncertain, output No.
Example
PROPERTY: "mentions natural scene." TEXT: "I love the way the sun sets in the evening."
Ouiput: Yes. "Sun sets" are clearly natural scenes.
Example
PROPERTY: "writes in 1st person perspective."
TEXT: "Jacob is smart." Output: No. This text is written in 3rd person perspective.
Example
PROPERTY:: "is better than group B."
TEXT: "I also need to buy chair."
Output: No. It unclear what the PROPERTY means (e.g., what does group B mean?) and doesn't seem related the text.
Example 4:
PROPERTY: "mentions that the breakfast is good on the airline."
TEXT: "The airline staff was really nice! Enjoyable flight."
Ouiput: No. Although the text appreciates the flight experience, it DOES NOT mention about the breakfast.
Example S:
PROPERTY: "appreciates the writing style of the author." TEXT: "The paper absolutely sucks because its underlying logic wrong. However, the presentation of the paper clear and the use language really impressive." Output: Yes. Although the text dislikes the paper, it says "the presentation of the paper is clear", so DOES like the writing style.
Example 6:
PROPERTY: "has formal style; specifically, the language in the text is relatively formal, complex and academic. For example, 'represent whom and which" TEXT: "investigates formation of nominalization"
Output: Yes. "formation" and "nominalization" are abstract and complex nouns.
Example
PROPERTY: "refers to historical dates; specifically, there are references to years or specific dates in the text. For example, 'Obama was born on August 4, 1961.""
TEXT: "A member of the Democratic Party, he was the first African-American president of the United States."
Output: No. The text does not mention date.
Now complete the following example Respond with Yes or No with an explanation that discusses the evidence from the TEXT. When uncertain, output No.
PROPERTY: (hypothesis) TEXT: [text] Output:
Figure 7. The prompt we use to compute hypothesis annotations with GPT-4o-mini to evaluate hypothesis quality. The prompt is lightly edited from Zhong et al. (2024).
26

---

Sparse Autocncoders for Hypothesis Generation
You machine learning researcher analyzing clusters from topic model. You need to identify the key distinguishing features of documents specific topic cluster.
Example features could be:
"uses multiple adjectives to describe colors"
"describes patient experiencing scizures or epilepsy"
"contains multiple angle-digit numbers"
These are the top keywords describing the topic: (eopic.keywords]
You will also see sample of documents belonging to this topic and a random sample of documents from other top- ies from the same dataset.
TOPIC DOCUMENTS (a sample of documents belonging to this topic):
cocuuendcummnts)
RANDOM DOCUMENTS (a sample of documents from other topics):
(random.dem documents)
Your goal is to create a short, specific label that captures what makes the topic documents different from the con- trasting documents.
The label should be:
1. As specific as possible, while still applying to all topic documents
2. Focused on concrete features rather than abstract concepts
3. Based characteristies present in the topic documents but not in the contrasting documents
Do not output anything clse. Your response should be a single topic label starting with and surrounded by quotes Your response is:
Figure The prompt distinguish the topic from others, and also with two sets of documents: 10 documents helonging to the target topic, and random sample of 10 documents from other topics. This labeling approach inspired by Grootendorst (2024), but we designed the prompt ourselves (intending it to be similar, where possible, to our neuron interpretation prompt).
generate interpretable topic labels for BERTOPIC. The L.M prompted with the keywords that
27


Reviews:
[{"review_id": "r1f7460bGF", "reviewer": "Reviewer_K1AB", "summary": "This paper proposes a method to generate hypotheses using SAEs. The first step of the method involves generating interpretable features by training SAEs on feature embeddings. The second step involves identifying which features are predictive for a task, using Lasso. Finally, the third step involves using LLMs to generate human interpretable natural language explanations for the identified predictive features. The experiments show that this method identifies better hypotheses than baselines, while requiring 1-2 orders of magnitude less compute.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful and positive review. We’re glad that you found the claims of the paper to be very clear and well supported.\n\nWe agree that a significant benefit of our method is that it reduces compute requirements by 1-2 orders of magnitude, and believe this will facilitate real-world applications. We would like to also emphasize what we see as an even more important benefit of the method: its performance in identifying predictive hypotheses. On synthetic tasks (where we know ground-truth hypotheses), the method outperforms all three baselines on 11/12 metrics. On real-world tasks, the method outperforms all three baselines on 5/6 metrics, generating 45 significant hypotheses out of 60 candidates, while the next best generates only 24.\n\nWe also appreciate your feedback about the phrase “triangle inequality.” Perhaps we could call it a “sufficient condition for hypothesis generation.” We would be curious for your thoughts on this phrase. An alternative could be to drop the phrase.\n\nWe also thank you for suggesting ablations, which we agree would clarify the importance of the SAE. On the three real-world datasets (Headlines, Yelp, Congress), we ran both ablations you mentioned—using embeddings directly and fitting a bottleneck—and a third suggested by R2, which selects features from an embedding PCA. We keep other steps of our method fixed.\n\n**Embedding**: Run Lasso to select 20 dimensions directly from embeddings. \n**PCA** (Suggested by R2): Transform embeddings to PCA basis (k=256, which explains ~80% variance), then select 20 dimensions using Lasso. \n**Bottleneck**: Fit a simple neural network which uses the embeddings as input, a hidden 20-neuron “bottleneck” layer, and predicts the target as output. Then interpret the 20 bottleneck neurons. (This is conceptually similar to NLParam, but simpler.)\n\n| Method     | Total Count | Average Predictiveness | Average Fidelity |\n|------------|-------------|------------------------|------------------|\n| SAE        | **45/60**   | **0.717**             | **0.836**        |\n| Embedding  | 30/60       | 0.706                 | 0.545            |\n| PCA        | 23/60       | 0.697                 | 0.547            |\n| Bottleneck | 27/60       | 0.699                 | 0.640            |\n\nMetrics are combined/averaged across the three datasets. Recall that **count** is the number of significant hypotheses in a multivariate regression; **predictiveness** is the AUC (or R^2 for Yelp) using all hypotheses together; **fidelity** is the F1 score for how well the interpretation matches the neuron activations.\n\nUsing the SAE produces many more hypotheses which are significant in a multivariate regression (45/60 across the three datasets for SAE vs. 30/60 for the next best baseline); slightly higher predictiveness; and much higher interpretation fidelity. This is consistent with our qualitative finding that SAE neurons fire on more specific, distinct concepts than embeddings. Specificity permits the high-fidelity interpretations, and distinctiveness results in the wide breadth of hypotheses. While these baselines perform reasonably well in terms of predictiveness, for downstream utility, predictiveness is only a prerequisite: we ultimately want hypotheses that are non-trivial and non-redundant (as is discussed in Sec 6.2).\n\nWe believe the results of these baselines clarify the value of the SAE in our setting, and we are excited to include them in the updated version of the paper; thank you for suggesting them!\n\nWe believe that we have now addressed the only weakness you raised (comparison to additional baselines) and were hoping, in light of this, that you would be willing to raise your score. If not, please let us know if there are further concerns we could address or experiments  we could conduct. Thank you again for your suggestions, which have strengthened the paper!", "claims_and_evidence": "The claims made by this submission are very clear and are well-supported by evidence.", "methods_and_evaluation_criteria": "The methods and evaluation criteria make sense for the problem and application at hand.", "theoretical_claims": "I did not formally check the correctness of the theoretical claims.", "experimental_designs_or_analyses": "I did not find any issues with the soundness or validity of the experimental designs.", "relation_to_broader_scientific_literature": "One of the key contributions of the paper is proposing a compute-efficient method to perform hypothesis discovery using LLMs. While some previous methods (Zhou et al, \"Hypothesis Generation with Large Language Model\", 2024) entirely rely on LLMs to propose hypotheses for relationships between two target variables, and other methods (Zhong et al, \"Explaining Datasets in Words: Statistical Models with Natural\nLanguage Parameters\", 2024) require large amounts of compute to achieve this task; this paper performs the same task with about 1-2 orders of magnitude reduction in compute. \n\nAnother contribution is using SAEs in a systematic manner to identify hypothesis. While most work in the SAE literature focusses on improving or measuring the fidelity of the representations, this paper makes use of imperfect interpretations to generate hypothesis; while being able to reason about the corresponding fidelity of the hypotheses generated.", "essential_references_not_discussed": "None that comes to mind.", "other_strengths_and_weaknesses": "Strengths:\n\n+ This paper proposes a systematic framework to generate and identify hypotheses using SAEs / Lasso, along with LLMs in the loop. The paper is overall well-written and its goals are clearly stated. The biggest advantage of this method is that it seems to use significantly less compute than the baselines; making this a useful contribution to the field.\n\nWeaknesses:\n\n- **Missing non-SAE baseline**: The proposed method consists of three independent steps; the first involves training an SAE, the second involves identifying suitable features via a Lasso regressor. Finally, an auto-interpretation step is proposed to interpret the identified features in terms of natural language explanations. It would be great to have an ablation analysis that tests the utility of step 1, i.e., using SAEs features. For example, \n   - why not use trained bottleneck features like NLParam?, or \n   - directly use Lasso on top of the pre-trained language embedding features, skipping the SAE step altogether?  \nThese ablations would help assess the importance of the SAE.", "other_comments_or_suggestions": "As a suggestion, the phrase \"triangle inequality\" may not be appropriate for the result in Proposition 3.1; as such a result requires three quantities in a shared space (like a field or a vector space) and corresponding notions of distance between them. While the proposed theory is useful, using the term \"triangle inequality\" might lead to confusion, and I suggest that the authors reconsider its usage in the paper.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "69anKWnf95", "reviewer": "Reviewer_vX7C", "summary": "This paper addresses the task of using LLMs to take a labeled text dataset and propose natural language hypotheses predicting those labels. The performance of hypotheses is measured by having an LLM evaluate each prompt according to the explanations, producing a boolean vector. Using this vector, a linear regression is learned to predict whether the output is true or false, and this is scored in several ways - recovering known hypotheses on synthetic datasets, getting many statistically significant hypotheses, having high AUC, etc.\n\nThe authors implement several existing baselines. Their key contribution is a new method based on sparse autoencoders on text embeddings. They use the OpenAI text embedding API and train a narrow sparse autoencoder with approximately a thousand units. They then identify which units are predictive of the labels using L1 regularised logistic regression and select the most predictive ones.\n\nThey then use auto-interp on the top units to generate 3 hypotheses (GPT-4o) score them (GPT-4o mini) and then validate the predictive power of the hypotheses on a held-out validation set, and with a Bonferroni correction for multiple hypothesis testing.\n\nThey find that this significantly outperforms existing (largely LLM-based) baselines across most metrics. Finally, they present several qualitative examples of the explanations produced. These explanations broadly seem reasonable and interpretable, are often more specific than prior methods would give, and on some well-studied datasets give what the authors consider to be novel hypotheses.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful and positive review. We’re especially glad to read that as an interpretability researcher, you found our work to be “highly significant because it is the most compelling example I have yet seen of sparse autoencoders beating baselines on a real task that people have actually tried to solve with other methods.” Indeed, we agree that recent literature is mixed on what value SAEs contribute, and we see the method as highlighting an area in which they have a real advantage: the ability (as you note) to find interpretable and *unexpected* patterns in data. We will further highlight this contribution in the paper.\n\nWe also thank you for suggesting simpler baselines, which we agree would clarify the importance of the SAE. We ran several of them on the three real-world datasets (Headlines, Yelp, Congress). Besides the feature generation step, the baselines are identical to our method.\n\n**Embedding**: Run Lasso to select 20 dimensions directly from embeddings. \n**PCA**: Transform embeddings to PCA basis (k=256, which explains ~80% variance), then select 20 dimensions using Lasso. \n**Bottleneck** (suggested by R1): Fit a simple neural network which uses the embeddings as input, a hidden 20-neuron “bottleneck” layer, and predicts the target as output. Then interpret the 20 bottleneck neurons. (This is conceptually similar to NLParam, but simpler.)\n\n| Method     | Total Count | Average Predictiveness | Average Fidelity |\n|------------|-------------|------------------------|------------------|\n| SAE        | **45/60**   | **0.717**             | **0.836**        |\n| Embedding  | 30/60       | 0.706                 | 0.545            |\n| PCA        | 23/60       | 0.697                 | 0.547            |\n| Bottleneck | 27/60       | 0.699                 | 0.640            |\n\nMetrics are combined/averaged across the three datasets. Recall that **count** is the number of significant hypotheses in a multivariate regression; **predictiveness** is the AUC (or R^2 for Yelp) using all hypotheses together; **fidelity** is the F1 score for how well the interpretation matches the neuron activations.\n\nUsing the SAE produces many more hypotheses which are significant in a multivariate regression (45/60 across the three datasets for SAE vs. 30/60 for the next best baseline); slightly higher predictiveness; and much higher interpretation fidelity. This is consistent with our qualitative finding that SAE neurons fire on more specific, distinct concepts than embeddings. Specificity permits the high-fidelity interpretations, and distinctiveness results in the wide breadth of hypotheses. While these baselines perform reasonably well in terms of predictiveness, for downstream utility, predictiveness is only a prerequisite: we ultimately want hypotheses that are non-trivial and non-redundant (as is discussed in Sec 6.2).\n\nWe believe the results of these baselines clarify the value of the SAE in our setting, and we are excited to include them in the updated version of the paper.\n\nRegarding your suggestion to fit an n-gram/bag-of-words model: in the paper, we perform a related analysis for the Congress dataset, using n-gram results from Gentzkow et al. (a seminal, widely-cited analysis). They fit an L1 regression using bigrams and trigrams, and report 600 predictive ones. Controlling for counts of the n-grams on this list, we find that our hypotheses improve AUC from 0.61 to 0.74 and 28 out of 33 remain significant. This suggests that n-grams aren’t sufficient to cover our hypotheses. Qualitatively, we find hypotheses which we think n-grams can’t easily capture, like “criticizes the allocation of resources or priorities by the government, particularly highlighting disparities between domestic needs (e.g., education, healthcare, energy) and actions abroad (e.g., Iraq)”.\n\nThank you also for the pointer to Matryoshka SAEs. We’ve been excited about these as well, and plan to add them to our codebase and try them. We agree they may reduce the need to stitch together SAEs of different sizes.\n\nWe agree that working with domain experts would be a valuable direction to increase confidence in our results. In the last week, we took some initial steps here (more detail in the response to R3): we (1) confirmed that two domain experts did not know of specific work producing several of our findings on the headlines dataset (the researchers also noted that some of these findings support theories in psychology) (2) conducted a human eval for whether our hypotheses are helpful and interpretable, which produced strongly positive results (https://imgur.com/a/qw6bt3s).\n\nWe share your desire to make our code easy to work with. In fact, we’ve released a public codebase with a pip-installable package, though unfortunately we are not allowed to link to it (even anonymously).\n\nThank you again for your suggestions, and we’re really happy to see that you found the paper to make a significant contribution to interpretability research.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Seems reasonable, though I am not very aware of the right baselines and implementation for this task", "theoretical_claims": "Briefly", "experimental_designs_or_analyses": "The HypotheSAEs method is sound", "relation_to_broader_scientific_literature": "Sparse autoencoders have been a major focus of the mechanistic interpretability field over the past year. A major open question is whether sparse autoencoders are practically useful on downstream tasks in a way that beats baselines, and there have been many recent negative results. In my opinion, as an interpretability researcher, this paper is highly significant because it is the most compelling example I have yet seen of sparse autoencoders beating baselines on a real task that people have actually tried to solve with other methods. Furthermore, this kind of exploratory analysis and hypothesis generation is exactly the kind of thing where I would expect sparse autoencoders to add significant value, as they are able to surface unexpected hypotheses.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "As discussed above, this paper is a compelling example of sparse autoencoders (SAEs) beating baselines on a real task. This is the best example I have seen on an extremely important question, and for this reason, I consider this work a strong accept.\n\nNot only are SAEs substantially cheaper than the existing modern baselines, but they also performed substantially better. Enough qualitative examples were provided that I am reasonably satisfied that real hypotheses are being surfaced rather than spurious findings. Similarly, it was able to recover the correct hypotheses on the synthetic tasks.\n\nThe statistical analysis and effort put into baselines seemed of fairly high quality, though I found it difficult to judge the details here, as I am not familiar with this task under these baselines.", "other_comments_or_suggestions": "A few things could improve the paper further:\n\nI would love if you could take a dataset where your method seems to generate novel hypotheses, like the Congress one, and show them to a domain expert. Have them evaluate whether these are interesting and novel, and whether they seem plausible. If they say yes, I am much more impressed by your results.\n\nIt would be worthwhile to try simpler baselines, such as:\n   - Using the PCA basis rather than the SAE basis\n   - Decision trees\n   - Logistic regression with L1 on bag of words\n   - Frequency analysis of words or bigrams\n   - Verifying that these don't work well\n\nI'm concerned that with any complex baseline, it could be easy to overcomplicate things or misimplement them. We might have the wrong hyperparameters, and it's good to try a range of styles, just in case.\n\nI'd also be excited to see whether [matryoshka SAEs](https://www.lesswrong.com/posts/rKM9b6B2LqwSB5ToN/learning-multi-level-features-with-matryoshka-saes) help here. You mentioned in the appendix that sometimes you train several SAEs of different sizes, and matryoshka is designed to remove the need for that by allowing concepts in a range of different granularities to be learned in the same SAE. I would be keen to see the authors open source this code and make it easy for other people to work with.", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JGRD1i0WXf", "reviewer": "Reviewer_WP4k", "summary": "The paper presents HYPOTHESAES, a three-step method (SAE-based feature generation, feature selection, and LLM-based feature interpretation) for hypothesis generation that identifies interpretable relationships between text data and a target variable. The approach leverages sparse autoencoders (SAEs) to learn meaningful, human-interpretable features, which are then used to generate hypotheses through feature selection and large language model (LLM)-based interpretation.\n\nKey Contributions\n\n1.\tTheoretical Framework: The authors establish a formal framework for hypothesis generation, introducing a \"triangle inequality\" that relates the predictiveness of learned features to their interpretability.\n2.\tAlgorithm - HYPOTHESAES: A three-step approach:\nFeature Generation: Train a sparse autoencoder on text embeddings to create interpretable neurons.\nFeature Selection: Identify predictive neurons using an L1-regularized regression.\nFeature Interpretation: Use an LLM to label neurons with natural language descriptions, forming hypotheses.\n\nMain Findings\n\nImproved Hypothesis Generation: HYPOTHESAES outperforms recent LLM-based hypothesis generation methods and traditional topic modeling approaches.\nEfficiency: Requires 1-2 orders of magnitude less computation than LLM-driven baselines.\nInterpretability: The sparse autoencoder structure ensures that identified features align with human-interpretable concepts.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed review and suggestions. We're glad you found our method original, clearly presented, theoretically grounded, and computationally efficient, with the performance gains convincingly supported.\n\nTo respond to your questions and suggestions:\n\n(1) “ ‘Broad applicability beyond text-based tasks claim’ is overstated”. \nCould you clarify which part of the paper you’re referring to (we couldn’t find this quote directly)? We agree that we do not provide evidence that the method generalizes beyond text-based tasks; we would be happy to revise specific areas where you thought this was implied.\n\n(2) “What if we directly use pretrained LLMs [...] as one of the baselines?” \nWe included a baseline, HypoGeniC, which uses pretrained LLMs directly. This method performed worse across 14 of 15 quantitative comparisons and was ~10x more expensive and ~30x slower, owing to the need to make many LLM calls to select candidate hypotheses. If you have specific suggestions for other ways to use pretrained LLMs, please let us know!\n\n(3) “Novelty claim could benefit from additional validation, especially through expert evaluation.” Thank you; based on this suggestion, we reached out to two experts to assess the novelty of our findings on the Headlines dataset (Table 1 of the paper). In particular, we asked them whether the three hypotheses most negatively associated with engagement—\"addressing collective human responsibility or action\", \"mentioning environmental issues or ecological consequences\", \"mentioning actions or initiatives that positively impact a community\"—were novel or reported previously. They did not provide papers with these specific findings; for example, one expert noted, “I don't know of any papers specifically looking at [the hypotheses you mention]”. Both experts pointed us to theories that are broadly supported by these findings: e.g., Robertson et al. (2023) find that negativity drives news consumption, which is consistent with the third hypothesis.\n\nIn light of your and R4’s comments, we also conducted a human eval where we asked three computational social science researchers (not involved with the paper) to evaluate all significant hypotheses on the Headlines and Congress datasets. We followed prior HCI work (Lam et al. 2024, “Concept Induction”) and asked them to annotate for “Helpful” and “Interpretable” hypotheses. We use the median of the three ratings. HypotheSAEs substantially outperforms baselines in terms of raw counts and percentages: 24/30 (80%) are rated helpful, and 29/30 (97%) are interpretable:\n\n[Hypothesis Human Eval - Imgur](https://imgur.com/a/qw6bt3s)\n\nWe hope these new findings increase your confidence that the results from HypotheSAEs are (1) novel, as per our correspondence with experts; and (2) helpful and interpretable, as per our human evals.\n\n\n(4) You asked about “the necessity of the specific theoretical bound (proposition 3.1) in practical performance.” We agree the theoretical bound is not strictly necessary for practical performance, but rather serves as a broader motivation for the procedure, as you note. It also provides us a way to conceptually decompose hypothesis generation performance into (1) neuron predictiveness, and (2) interpretation fidelity. C.3 explores this empirically.\n\n\n(5) You asked about “the difference between ... the paper and more complex scientific hypothesis generation scenario in the papers below.” Thank you for these references, which we will include in an additional related work paragraph titled “Automated Idea Generation”. We agree that these literatures are both in service of helping researchers conduct science, but they address different tasks:\n\nOur work is focused on solving the problem: “given a dataset of texts and a target variable, what are the human-understandable concepts that predict the target?” In contrast, the literature you mention focuses on: “given a corpus of prior scientific papers, can we propose promising research ideas?” Practically, the former task emerges when a researcher knows what they are studying and have collected data, but they need tools to make sense of the data. The latter task emerges when a researcher is trying to decide *what* to study based on prior literature. Methodologically, the former task involves methods like clustering & interpretability, while the latter involves methods like prompting & RAG. We think these two literatures are distinct, but complementary; for example, a political scientist might use an idea generation method to decide to study partisan differences in social media posts, and then, after collecting a dataset, use HypotheSAEs to propose data-driven hypotheses for further validation.\n\n(6) More formal language; appendix table of contents: Thank you! We will make these edits. \n\nGiven the strengths you highlight, and our experiments to address your comments, would you consider raising your score? If not, do you have further questions?", "claims_and_evidence": "Most claims are convincingly supported, particularly those regarding performance gains, computational efficiency, and the effectiveness of SAEs.\n\nNovelty claim could benefit from additional validation, especially through expert evaluation. While the method identifies hypotheses that were not explicitly found in prior studies, the paper does not provide direct human validation of the novelty and importance of these hypotheses.\n\n“Broad applicability beyond text-based tasks claim” is overstated. The method is only tested on text-based hypothesis generation tasks. There is no evidence that it generalizes to: Healthcare (e.g., clinical note analysis) Biology (e.g., scientific literature mining) The claim would need domain-specific evaluations before being fully credible.", "methods_and_evaluation_criteria": "Yes, they do.", "theoretical_claims": "The theoretical framework (Triangle Inequality for Hypothesis Generation) is a fundamental contribution but it’s weakly supported: The theoretical proposition (Proposition 3.1) is an interesting insight, but: The empirical validation is indirect—while the model works well, the necessity of this specific theoretical bound in practical performance is unclear.", "experimental_designs_or_analyses": "The evaluation metrics, statistical tests, and experimental setups are generally valid and well-designed. However, a key limitation is the lack of human evaluation to assess the novelty of the generated hypotheses.", "relation_to_broader_scientific_literature": "This paper synthesizes ideas from hypothesis generation, sparse representation learning, and interpretability research to create a scalable and structured approach for discovering meaningful insights from text data.\nUnique contributions to the broader literature include:\n1. Combines sparse autoencoders with LLM interpretation for efficient hypothesis generation.\n2. Establishes a theoretical link between interpretability and predictiveness in machine learning models.\n3. Demonstrates real-world applications that extend findings in political science, marketing, and behavioral research.", "essential_references_not_discussed": "It's better for the authors to discuss the difference between the hypothesis generation scenario in the paper and more complex scientific hypothesis generation scenario in the papers below:\n\nCan LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers arXiv preprint arXiv:2409.04109\n\nLearning to generate novel scientific directions with contextualized literature-based discovery arXiv preprint arXiv:2305.14259\n\nScimon: Scientific inspiration machines optimized for novelty arXiv preprint arXiv:2305.14259\n\nResearchagent: Iterative research idea generation over scientific literature with large language models arXiv preprint arXiv:2404.07738", "other_strengths_and_weaknesses": "Strengths\n1. Originality\n(a) Novel combination of sparse autoencoders and hypothesis generation. The paper introduces a creative synthesis of ideas from sparse representation learning, interpretability research, and hypothesis generation. Prior work has focused on either interpretable feature extraction (SAEs) or LLM-driven hypothesis discovery, but this paper combines the two, making hypothesis generation more scalable and efficient. The triangle inequality formulation (Proposition 3.1) provides a new theoretical perspective on the trade-off between feature predictiveness and interpretability.\n(b) Breaks from fully LLM-driven approaches. Recent LLM-based hypothesis generation methods (e.g., HYPOGENIC, NLPARAM) require high computational cost, limiting their scalability. By decoupling feature learning from LLM inference, HYPOTHESAES provides a more practical and resource-efficient solution.\n\n2. Clarity\n(a) Well-structured and clearly written. The paper provides a step-by-step explanation of the method (Figure 1) and offers intuitive interpretations of the results. The supplementary material includes detailed experimental settings and theoretical justifications, ensuring reproducibility.\n(b) Strong theoretical grounding. Proposition 3.1 is well-motivated, and the proofs in the appendix provide a rigorous foundation for the method’s effectiveness.\n\nWeaknesses\nIssues listed in the above sections.", "other_comments_or_suggestions": "The introduction of the datasets should be in more formal language. For example, around line 263, 268, 274 etc., the sentences can be modified to be:\n\nWe utilize 200k reviews for training, 10k for validation, and 10k for held-out evaluation.\n\n\nAs the appendix is long, it would be better to have a table of contents to better organize everything and add a brief paragraph about all the prompt templates, etc.", "questions_for_authors": "1. What if we directly use pretrained LLMs to perform the same task and use them as one of the baselines? It seems pretrained LLMs can also perform the task and offer some self-explanations. I am curious about the performance and the cost.\n\n2. How do you define the novelty of a generated hypothesis? While the method identifies hypotheses that were not explicitly found in prior studies, the paper does not provide direct human validation of the novelty and importance of these hypotheses.\n\n3. Can you provide any supporting evidence that the method can be applied to more complicated real-world hypothesis generation in other fields like healthcare? Or can you discuss the difference between complicated scientific hypothesis generation using LLMs in the papers I mentioned above and the proposed method?\n\n4. It would be better if the authors could add more discussion about the necessity of the specific theoretical bound (proposition 3.1) in practical performance.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1kYN6s75L5", "reviewer": "Reviewer_ENNZ", "summary": "This paper proposes a hypothesize generation method by training a sparse auto encoder to find text examples trigger the same neuron which interpreted as containing the same human concepts, and then leverage on LLM to generate the interpretation of the neuron from these examples. The paper performs experiments on both synthesized datasets and real-world datasets, and show results that the method has better accuracy and efficiency than the SOTA.\n\n## update after rebuttal\nThe authors have addressed my major concerns and questions, and I increased my rating to accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed, positive review. We are glad you find the claims convincing, the paper well-presented, and the method efficient. \n\nTo reply to your questions: \n\n(1) “The method proposed here is also subjected to the context window limit[...]” Thanks for pointing this out; we will clarify in the paper. HypotheSAEs doesn't run into context window limit issues in practice because it applies LLMs only to *interpret* SAE neurons, for which the LLM requires only a few examples (we test this in C.2). In contrast, prior methods use LLMs to *learn* the interpretable features in the first place, which requires reasoning over many examples.\n\n(2) “it claims the dimension of z_i to be [...] highly interpretable [...] without any supporting evidence” \nThanks for this point. Supporting references are in Sec 1+2.2, and we will include them here as well. However, this sentence is meant to motivate using SAEs, rather than to prove this claim; our experiments on real datasets demonstrate it holds in our setting. One quantitative metric (see response to R2) is that interpreting SAE neurons yields much higher fidelity vs. interpreting embeddings directly (F1: 0.84 vs. 0.54).\n\n(3) “Interpreting each single neuron of the activation matrix to be a nature language concept is problematic” Thanks; we agree this is a key challenge, which is why we conduct extensive experiments to maximize fidelity to the underlying neuron (Appendix C). Empirically, Figure 3 demonstrates that the loss in predictive performance due to using discrete concepts to approximate the neurons is quite small on average (-2.4%). \n\n (4) “How stable are the interpretations when the examples are changed” Thanks for this question. We ran a new experiment to measure stability. For 100 random neurons on the Yelp dataset, we generated 3 interpretations with different random seeds, and computed text embeddings of all interpretations. The mean pairwise cosine similarity of two interpretations generated for the same neuron is high: 0.84 (vs. 0.34 for a pair of interpretations from two different neurons). This is despite two sources of randomness—sampling different examples and an LLM temperature of 0.7—showing that the underlying concept learned by each neuron is durable. Here is a set of interpretations for a random neuron (can share more upon request):\n\nNeuron 50 (stability: 0.76): \n['discusses the seasoning of food, either praising it as perfect or criticizing it as lacking', \n'mentions seasoning or lack of seasoning in the food', \n'mentions seasoning or lack thereof in the context of food flavor']\n\n\n(5) “Interpreting the neurons with LLM requires per dataset manual efforts to tune the bin percentiles”; “How sensitive is the method to different sparsity levels (k in TopK)”.\nThanks for these questions about hyperparameters. We ran some experiments and found that results are not particularly sensitive; they work well with defaults:\n\nHeadlines, using default bin [90, 100] instead of [80, 100] as in paper: AUC 0.70, 11/20 significant (still beats all baselines)\nYelp, using default k=8 instead of k=32 as in paper: R^2 0.77, 14/20 significant (~identical to original)\n\nWe provide hyperparameter guidance for practitioners in Appendix B, with more detail in the Python package we released publicly (which, unfortunately, we aren't permitted to link here). \n\n(6)  “Lack of human evaluation for the interpretation quality of the LLMs and the generated hypothesize” \nThank you; in light of this comment, we conducted a qualitative human eval where we asked three computational social science researchers (not involved with the paper) to evaluate all significant hypotheses on the Headlines and Congress datasets. We followed prior HCI work (Lam et al. 2024, “Concept Induction”) and asked them to annotate for “Helpful” and “Interpretable” hypotheses. We use the median of the three ratings. HypotheSAEs substantially outperforms baselines in terms of raw counts and percentages: 24/30 (80%) are rated helpful, and 29/30 (97%) are interpretable.\n\nResults plot: https://imgur.com/a/qw6bt3s\n\nWe also spoke to domain experts about novelty; see our reply to R3. We hope these findings increase your confidence that our hypotheses are high-quality.\n\n(7) “Does the H [...] equal the dimension M”\nNo: M is the total number of SAE neurons, from which we select H predictive neurons to interpret. We choose it based on the validation AUC of how well the SAE neurons predict *y* (see B.1 + our repo). \n\n(8) “Did you observe any hallucinations from the LLM when labeling neurons?” ~ We did not observe hallucinations in the usual sense, but some interpretations did not describe the neuron well, which is why we generate multiple interpretations and choose the highest fidelity one. In practice, our results are not very sensitive to this step (see B.2).\n\nGiven the strengths you highlight, and our experiments to address your comments, would you consider raising your score? If not, do you have further questions?", "claims_and_evidence": "The claims in this paper are overall convincing, with some claims overstated or need more supports:\n1. In introduction, the paper claims it \"clarifies the challenge\" of context window and reasoning limitations in LLM-based approach for hypothesize generation, however, the method proposed here is also subjected to the context window limit when generating interpretations for each neuron.\n2. In section 4.1., it claims the dimension of z_i to be empirically \"often highly interpretable, corresponding to human concepts\", but without any supporting evidence or reference. The validity of this statement is critical for the entire work and deserve more attention.", "methods_and_evaluation_criteria": "For the method, using sparse auto encoder to find text examples that may carry the same concept makes much sense, but interpreting each single neuron of the activation matrix to be a nature language concept is problematic: neurons are discrete, but language concepts are not. \n\nFurthermore, there is no convergence analysis on the concepts (labels) of neurons that interpreted by LLM. Since LLM has to interpret a neuron base on small number of examples that activate the target neuron, it should provide more insight to show how stable are the the interpretations when the examples are changed, or even just change the temperature.", "theoretical_claims": "Not.", "experimental_designs_or_analyses": "Yes, checked the datasets and metrics.", "relation_to_broader_scientific_literature": "It improves LLM-only hypothesize generation approaches by using sparse autoencoder to generate interpretable feature representation (SAE neurons) instead of directly prompting an LLM.", "essential_references_not_discussed": "Not in my awareness.", "other_strengths_and_weaknesses": "Strengths:\n- The paper is well structured and presented.\n- The method has good computational efficiency.\n\nWeakness:\n- Interpreting the neurons with LLM requires per dataset manual efforts to tune the bin percentiles from which to sample highly-activating\nand weakly-activating examples, making the method less generalizable.\n- Lack of human evaluation for the interpretation quality of the LLMs and the generated hypothesize.", "other_comments_or_suggestions": "in equation (6), the `z` should be `z_i`", "questions_for_authors": "- Does the H from \"HYPOTHESAES outputs H natural language concepts (line 188)\" equal the dimension M of the activation matrix, Z_SAE? And how do you determine M?\n- Did you observe any hallucinations from the LLM when labeling neurons? If so, how did you address this?\n- How sensitive is the method to different sparsity levels (k in TopK activation)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper shows how to generate interpretable relationships between text data and target variables leveraging sparse-autoencoders. It compares favorably against state-of-the-art methods, while using less compute. 

The reviewers generally agree that this work is a meaningful contribution. One of the reviewers points out that the scope of the method may be limited to text data, and the revised version of the paper should reflect this.
Decision: Accept (poster)