Goal Driven Discovery of Distributional Differences via Language Descriptions
Authors: Ruiqi Zhong
Peter Zhang
Steve Li
Jinwoo Ahn
Dan Klein
Jacob Steinhardt
Venue: NeurIPS 2023 poster (2023)
Keywords: large language model
prompting
exploratory text analysis
TL;DR: We introduced a new task, D5, that discovers differences between text corpora in a goal driven way.

Abstract: Exploring large corpora can generate useful discoveries but is time-consuming for humans.
    We formulate a new task, D5, that automatically discovers differences between two large corpora in a goal-driven way. 
    The task input is a problem comprising a user-specified research goal (“*comparing the side effects of drug A and drug*”) and a corpus pair (two large collections of patients' self-reported reactions after taking each drug). 
    The output is a goal-related description (discovery) of how these corpora differ (patients taking drug A “*mention feelings of paranoia*” more often).
    We build a D5 system, and to quantitatively evaluate its performance, we 1) build a diagnostic benchmark, SynD5, to test whether it can recover known differences between two synthetic corpora, and 2) contribute a meta-dataset, OpenD5, aggregating 675 open-ended problems ranging across business, social sciences, humanities, machine learning, and health.
    With both synthetic and real datasets, we confirm that language models can leverage the user-specified goals to propose more relevant candidate discoveries, and they sometimes produce discoveries previously unknown to the authors, including demographic differences in discussion topics, political stances in speech, insights in commercial reviews, and error patterns in NLP models.
    Finally, we discuss the limitations of the current D5 system, which discovers correlation rather than causation and has the potential to reinforce societal biases when misused; therefore, practitioners should treat the outputs of our system with caution.

Goal Driven Discovery of Distributional Differences via Language Descriptions
Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, Jacob Steinhardt
Peter
Abstract
Exploring large corpora can generate useful discoveries but is time-consuming for humans. We formulate a new task, D5, that automatically discovers differences between two large corpora in a goal-driven way. The task input is a problem comprising user-specified exploration goal ("comparing the side effects of drug A and drug B²) and corpus pair (collections patients' self-reported reactions after taking each drug). The output is goal-relevant description (discovery) of how these corpora differ (patients taking drug A "mention feelings of paranoia" more often). We build a D5 system, and quantitatively evaluate its performance, we I) build a diagnostic benchmark, SYND5, to test whether it can recover known differ- ences between two synthetic corpora, and 2) contribute meta-dataset, OPEND5, aggregating 675 open-ended problems ranging across business, social sciences, humanities, machine learning, and health. With both synthetic and real datasets, we confirm that language models can leverage user-specified goals to propose more relevant candidate discoveries, and they sometimes produce discoveries previously unknown to the authors, including demographic differences in discussion topics, political stances in speech, insights in commercial reviews, and error patterns in NLP models. Finally, we discuss the limitations of our D5 system, which discovers correlation rather than causation and potentially reinforces biases when misused; therefore, practitioners should treat the outputs of our system with caution.
Introduction
Exploring large corpora and generating discoveries from them can be ad hoc and laborious. For example, to compare the side effects of drug A and drug B, doctors might inspect two large corpora of patients' self-reported reactions after taking each drug; based on ad hoc insights, they hypothesize that patients taking drug more often "mentions feelings of paranoia" and then validate this hypothesis by laboriously inspecting the two corpora. Since machines can automatically process large amount of texts, we might hope MI. systems to facilitate exploratory analyses like the one above.
However, an ML task requires a unified input-output space and evaluation metric so that it can be automated, benchmarked, learned, and analyzed. To this end, formalize one type of exploratory analysis problem as a natural language generation task: goal driven discovery of differences between text distributions via language descriptions (DS). As shown in Figure I the input to the D5 task is "problem" comprising description of a user-specified exploration goal (understanding side effects) and a corpus pair (text samples from the distributions of self-reported reactions after taking each drug). The output is "discovery" represented as natural language predicate ("mentions feelings of paranoia"). We evaluate a discovery with two criteria (Section 3): (1) validity: should describe true difference (Zhong et al. 2022): and (2) relevance to the goal (McGarry 2005
Unniversity of California, Berkeley, EECS Department. Email: muigizzhon@berrkleeee
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Corpus
Corpus B
Even little sound dries me crazy
Coughing for two months
Exploration
Feelig to worried to focus
-Felt sleepy today
AK more samples omitted brevity]
["44K more samples omiited brevity]
split
Corpus Pair
Input Problem
My family complaints I'm too irritated
Slowly recovering.
Validation
["4K more samples omitted for brevity]
["4K more samples omnitted brevity)
split
The original dataset includes natient's self-reported reactions after taking drug. The two corpora generated based on what drug the patient has taken. Samples from Corpus A include self-reported reactions after taking drug while samples Corpus include self-reported reactions taking B. doctor. goal understand side effects of drue
Exploration Goal
Output
Corpus A has more samples that "mention feelings of paranoia"
Discovery
Figure 1: Each problem in OPENDS contains 1) corpus pair, which has 17K samples on average and is partitioned into two halves called "exploration split" and "validation split", and 2) a natural language description of the exploration goal, which also contains information about how the corpus pair was collected. A D5 system takes the goal and the exploration split as inputs and generates valid and relevant discoveries in natural language outputs. The underlined texts in the exploration goal vary across problems, while the rest are templates.
Since D5 is open-ended and aims at discovering unknowns, the most popular benchmark practice- comparing system-gent generated outputs with buman-writted references on a test set-is infeasible. We therefore design two evaluation strategies.
Diagnostic:
we synthesized a dataset DS problems with known solutions, SYND5, to diagnose whether D5 system can recover known differences between two synthetic corpora.
This strategy is cheap and automated but might not reflect user utility in real applications. Open-ended: collected dataset, OPEND5, by aggregating 675 open-ended D5 problems ranging across business, social sciences, humanitics, health, and machine learning (Figure 2 comprising 4.4 million text samples in total across problem corpora. We then manually evaluated a subset of the output discoveries. This strategy is subjective and expensive, but useful for obtaining qualitative insights more realistic applications.
These two strategies allow us to quantitatively evaluate and compare D5 systems. For example, we compared 1) the system from Zhong et al. (2022) designed to describe corpus-level differences without goals, and 2) goal-conditioned variant that we develop in Section [ We found language models successfully use the specified goal: the goal-conditions variant is correct 12% more often on SYND5, and produces relevant candidate discoveries %1% more often on OPEND5.
We envision OPENDS to be a growing, diverse repository of open-ended D5 problems. They will not only help us evaluate D5 systems more reliably, but also allow the following operations:
Facilitate exploratory analysis. Every time we build a better D5 system, we can apply it to a repository of open problems and send the discoveries to researchers who posed them. We show this paradigm is plausible by using our system to automatically produce useful discoveries on OPEND5 (Section 6.1.. including insights from commercial reviews, temporal and demographic differences in discussion topics, political stances and stereotypes speeches, differences in lyric styles, and error patterns NLP systems. We anticipate future systems produce more discoveries.
Analyze the limitations our evaluation. Using concrete examples from OPENDS, we show that our current evaluation metries do not encourage diverse findings, do not always produce causal conclusions, and cannot evaluate discoveries involving heavy expert knowledge (Section 6.2). More help identify more limitations, which inform areas for future improvement.
D5 problems
Train better D5 systems. Like other ML tasks, we can train a system once we have a dataset. We describe a self- supervised learning algorithm that uses repository of problems (without reference solutions) to train LMs propose more valid hypotheses (Section 4.3). As proof-of-concept we show that can make LMs better describe the differences between small groups of text samples.

---

To conclude. we show that D5 can be quantitatively evaluated, automated, analyzed, and learned. Like other ML tasks, it would benefit from a more diverse, authentic, and larger dataset. We hope future works can gather feedback from domain experts and curate an ever-larger dataset of D5 problems, thus accelerating exploratory analyses and facilitating scientific discoveries.
Datasets: SYND5 and OPEND5
We first introduce how each input problem is formatted. Then we discuss 1) how synthesized SYND5, which is used for automatic diagnostic evaluation, and 2) how we collected OPEND5, which used to investigate the practical value of D5 systems in open-ended applications.
2.1
Task Format
Each D5 problem represented by a corpus pair (Corpus A/B) and a description of the exploration goal. For example, Corpus A/B might be self-reported reactions after taking drug A/B, and the goal description would be "comparing the side effects of drug A and drug B". The desired output is valid and relevant discoveries in the form of natural language predicates (Figure D e.g. Corpus A has more samples that "mentions feelings of paranoia".
SYND5, a Diagnostic Benchmark with Reference Solutions and an Automatic Metric
To automatically diagnose a D5 system, we synthesized S SYND5, dataset of DS problem with reference solutions. To synthesize Corpus and Corpus for each input problem, we used language model (I.M) to generate two corpora that simultaneously differ on two dimensions, one of which is goal-relevant and one of which a distractor. For instance, suppose the goal is to "understand how Corpus A differs from Corpus B in terms of topic". Then we would synthesize an example where Corpus A is more sports-related while B is more art-related (goal-relevant: varying topic), while additionally Corpus A is in English while B is in French (distractor: varying language). The reference solution is the difference on the goal-relevant dimension, e.g. "is sports-related".
In more detail, to synthesize example in SYND5, we first picked one goal-relevant and one distractor dimension from the set (topic. genre, language]. and sampled a value for each corpus and dimension (e.g. Corpus A: [sports, English]: Corpus B: [art, French]). We then synthesized Corpus A/B such that all its samples are in English/French (i.e. completely different on the distractor dimension) while V percent of them are sports- moorts-related/arr--elaae where we varied V from 0.6 to 1. Since the distractor difference is more salient, SYND5 penalizes DS systems that ignore the goal and output the incorrect distractor difference "is in English", We synthesized 300 problems in total to create SYND5; see Appendix 9 for a detailed description of the pipeline.
To compute D5 system's 's accuracy, we prompted Claude-vis v1. (Bai et al. 2022b) to judge how often the output discovery semantically equivalent to the reference. We construct the prompt by using 6 pairs of predicates with the labels of "equivalent", "similar", or "irrelevant" as few- shot examples, and ask Claude-v1. to judge whether the output discovery and the reference are "equivalent". As a result, we can automatically diagnose D5 system. See Appendix [2] for the prompt for equivalence judgement and Appendix [0 for two robustness checks, which (a) consider "similar" discoveries to be correct as well, and (b) use other LMs for equivalence judgement.
2.3 OPEND5, Realistic Open-Endes Dataset without Reference Solutions
To evaluate D5 system's utility under realistic applications, we also gathered OPENDS, realistic dataset of 675 open-ended D5 problems. These problems range across business, social sciences, humanities, health, and machine learning; see Figure 2 for few examples. To build OPENDS, two of the authors performed an extensive literature review on problems that could potentially benefit from our system, e.g., reading survey papers (Nguyen al., (022)) and courses on computational social sciences, and skimming through the ACL proceedings from the past decade and datasets from Kaggle that have an NLP tag; we then annotated the exploration goals, scraped//enerated the
"Our code is released at https:://gite sstp::///gttuub..oo//uuggg woom//uiiiizzoteetee released at htttps:://// //gi: interpret its outputs with caution and not use to fully automate scientific discoveries.
and our code to download OPENDS httpppeete//pppll Given the limitations of our system, practitioners should

---

Domain
Example Datasets
How the Corpus Pairs are Generated
Corpus
Corpus B
Business problems
Commercial
Airline
1 class passenger reviews
Economy passenger reviews
Reviews
Product Reviews
Reviews that give 10 stars
Reviews that give O star
VC startups
Successful startup descriptions
Falled startup descriptions
Finance
Headlines
headlines when rises
headlines
278 Social Sciences problems
Politics
Administration policy
Admin policy from Trump
Admin policy from Obama
News
Reuters headlines
Headlines from 2014
Headlines from 2015
Craiglist Negotiations
Dialogue from successes Lies
Dialogue from faillres
Language
Diplomacy Dialogues
Honest statements
Happy moments
relfore happy moments from females
Self reported happy moments from males
Sociology
Rate My Professor
Reviews female lecturers
Reviews male lecturers
169 Humanities problems
Arts
Music lyrics
Drake rap lyrics Essays that received full score
Kanye rap lyrics
Education
Student essays
Essays with only partial credit
10 Health problems
note
diagnosed with preumania
Patients not diagnosed with pneumonia
131 Machine Learning problems
NU- distribution shift
Samples from SNU
Samples from MNLI
DGP spurious correlation individual questions with label "paraphrase"
Individual questions with label "nom-paraphrese"
Machine Learning
LM's output
Generations from UM
Generations from another
Inputs error analysis
Inputs where model is correct
Inputs where one model is wrong
WikiText - clustering
Samples from ane cluster
Samples not from a cluster
Figure 2: OPENDS contains 675 problems. See citations in Appendix 23
corresponding corpora, and post-processee them over nine months (see complete list of citations in Appendix 23). As shown in Figure each goal describes the original dataset, how the two given corpora are generated, who is using the system, and what property of the two corpora does the user want to understand. Each OPENDS problem is reviewed by at least two of the authors to reduce grammatical mistakes and ambiguous interpretations of the goal.
Each corpus contains around 17K text samples on average, and OPEND5 in total comprises 4.4 million distinct text samples. We use 50% of each corpus as the "exploration" split and 50% as the "validation" split. The system can only access the exploration split, while the validation split is reserved for the evaluators to validate the discovery. A validation split prevents overfitting the discoveries to the given samples and is analogous to the train-test split in machine learning.
Since we hope to build systems that can tackle challenging open-ended problems, we did not avoid cases where we do not know the ground truth answer. This is different from standard benchmarking practices, where humans can provide a reference solution to evaluate an Al system. However, even though we do not know the ground truth, once a system produces a discovery, we can still evaluate We present our evaluation metrics in the next section.
3 Evaluation Metrics for Open-Ended D5 problems
For the goal of comparing the side effects of drug and drug B, how do we evaluate a system- generated discovery that Corpus A "mention feelings of paranoia" more often? First, it needs to be valid, such that indeed more samples from Corpus A satisfy this predicate, which can be evaluated (approximately) objectively. Second, it needs to be relevant to the goal of understanding side effects, which depends on the user's subjective judgement. We define validity and relevance below.
Validity. Similar to Zhong et al. (002)). we require an output discovery / to be a truth predicate on a text sample. For example, if "mentions about family", then true on the string 21 = "My daughter loves me" and false on the string = "I'm going to school". Define T(h, x) E [0, 1] as "the certainty that is true on z", e.g., T(h, x1)= and T(h, 22)) z We approximate T(h, x) by asking three Turkers how certain they are and averaging their responses (see Appendix [1 for details).
Let Dval and 'Dyal denote the validation sets for Corpus A and B. We define the "validity" V
(h) == Ex-Dy (T(h, x)] - Ea (T(,
(I)

---

Computing (h) is expensive since it requires human annotations T(h, 2)) on a set of text samples even to evaluate single discovery h. In practice, we do not have the budget to compute (h) on the entire validation split; therefore, we approximate this quantity by randomly sampling from Corpus A and Corpus B. We use these samples to compute an empirical estimate of V, as well as a p-value for the null hypothesis that V < 0 using a one-sided t-test.
Relevance. A discovery may be irrelevant even if = 1. For example, if the goal is to understand the writing style differences between higher-scoring essays (Corpus A) and lower-scoring ones (Corpus B). the discovery that Corpus A "achieves higher scores" has high validity score by definition but irrelevant to the goal of understanding stylistic differences.
Therefore, we designed procedure evaluate relevance, where human or language model evaluators would score each discovery with 2/D/O. The evaluators used the rubric below, which illustrates the meaning each score with the essay example above:
2, relevant; e.g. the discovery "write in first person" is directly related the writing style.
①. indirectly relevant; e.g. the discovery "use the word "T"", is not exactly a writing style, but can still inform the relevant underlying principle of "write in first person".
@. irrelevant; e.g. the discovery "argue for abortion" is unrelated to the writing style.
To minimize biases while comparing two systems, the evaluators are blind to which system generates which discoveries.
To conclude, an ideal discovery would have a high V value with small p-value and achieve ratings of ( in relevance. In the next section, we will build a D5 system that addresses these criteria by first proposing goal-relevant candidate discoveries (hypotheses) and then automatically validate them.
Other metrics. We also explored two other subjective metrics, novelty (how difficult it is to generate the discovery) and significance (how beneficial it is to learn about the discovery). Due to space limit, we present their rubrics and related results in Appendix 14
4 Methods: Building a D5 System
We describe our D5 system, which maps from a corpus pair and an exploration goal to a set of natural language predicates. Our system is inspired by a two-stage model of how humans discover patterns in data: creatively brainstorming hypotheses and then rigorously validating them on the data (Ludwig & Mullainathan, 2022). Analogously, we first propose hypotheses conditioned on the exploration goal and a subset of samples from the corpus pair (Section 4.1). We then use a language model to approximately compute the validity of each hypothesis, and output the valid ones as the final discoveries (Section (.2). Our system closely mirrors that ofong et al. (2022). except that we leverage the goal to propose more relevant hypotheses. Finally, we present self-supervised learning algorithm to improve an LM's ability to propose more valid hypotheses (Section 4.3): however, due to API access constraint, we cannot apply it to fine-tune gpt-3, we provide proof of concept experiment on F1an-T5 (Chung et al. (022))
4.1 Hypothesis Proposer
We prompt gpt-3 (Ouyang al. 2022) propose hypotheses. Denoting the exploration split of Corpus A/B as D we construct the prompt by concatenating a few random samples from DAP and DeP, the exploration goal, and an instruction to output a list of hypotheses. Figure] (left) depicts an example of the resulting prompt, together with typical language model output.
Since the entire corpus pair might not fit into one prompt, we construct multiple prompts with different sets of samples so that gpt-3 can "see" many different samples possible in our pipeline. We continue sampling hypotheses with different prompts until obtaining a set of 60 hypotheses, which we call Hini. Appendix [5 includes more details on selecting the sets of samples for different prompts.
4.2 Hypothesis Validator
Many hypotheses in Hii have low validity: they are not more often true on DA than on DB (i.e. (h) 0). To automatically filter them out, use language model T"' to simulate the Turkers'
5

---

Proposer prompt
Validator prompt
Gituup Willoww
theee whather the TEXT
PROPERTY Reppard The When
Samples em from the two corpora
samples not inck uood for brants
under raats output
Proidient tatah Sand Alriion
Now compless eto follcwe cuampic input: PROPPRTYY montions sports
Mhhhammad.
camples incluced for
now
TEXT Serens Willowe UGA sunputt
(some sentences truncated for brevity...
Explaration Goal
The gioal detooet includes news summertes. The
proeented based
schen setre puntished isted Samplen from Group include news from 2007, samples from Group includie from 2008 ausurrala at understana what topica
Pr(NextWorr "Yes"] 99%
Please ras a lut typotheses Iseperated by bullet points "-") how dasapornts from Grous A diter from these from Group B. Each topothoils should be formatted us centence fragment Hort are thres oomples
Check whetler TEXT sottflle FROPENTY Respord with Yes No. When uncertain, output No.
Formatting Instructions
"talks about polities such as presidenti election."
"contalles Insutting binguage for mmmigronts"
Now complete the following eaamppee inpun: PRIPP ATC mentions sports beam
*uses dou bile regation, Le, using two negations sentence."
mecruting member
nased on the nann senteace groups (A and Bi) from the above, more sentencees in Group
TEXI: Eaypt's President Abdel futtah shi and Saudl Araba''s
Language Model Outputs
"mentiora about academic ae welatoos, such twachers or itudents"
"menttons a sports beam mecuuting new mamber"
cutput:
"mentions about W/ lst of hypotheses not unddd for breaity
Pr(NestWord "Yes"] " 5%
Figure 3: All underlined content in the prompt differs across problems, while the other content in the prompt is templated. Left: proposer prompt. The generated hypotheses are in blue. All content with colored background excluded for brevity. For the baseline of not using the exploration goal, we removed the "exploration goal" block from the prompt. Right: the validator prompt.
judgement T and hence approximate the validity score V(h) with the function V'(h), defined as (hh == 1x (T,, x)] Ey-pyy (T' xr).
To compute 7", we ask Flan-T5 whether x satisfies h with the prompt shown in Figure (right). To better simulate Turker's judgment, we collected additional Turker annotations fine-tune FLAN-T5 (see Appendix [6] for details about the data collection process). We then obtain a significance value p' by performing t-test compare the mean value of zh),,) on the exploration split of Corpus A to that of Corpus B, rule out the hypotheses with p' greater than 0.001, and output the remainder as discoveries. Finally, we obtain additional discoveries by repeating the same process but asking our system to propose and validate hypotheses about Corpus B rather than Corpus A. Appendix Figure visualizes our entire pipeline and Appendix Oppendix(ddiccuue discusses the computational resources we used.
4.3
Supe-Supe Learning with Open-Ended Problems: A Proof of Concept
Since D5 problems are open-ended, future systems could potentially produce discoveries with higher validity scores than any known discovery. Therefore, design self-supervised learning algorithm to improve an LM's ability to propose more valid hypotheses, using the principle that it is easier to validate a discovery than to generate one.
Algorithm. Suppose we are given a set of problems for training and an initial language model mg- Our goal is to automatically generate set of prompt-completion pairs to fine-tune mo that it can propose hypotheses that are more valid. To generate a prompt, we randomly sample a problem and create proposer prompt following the procedure in Section 4.1 To generate the desired completion given a prompt, we sample multiple hypotheses from Minin approximate their V' score on the samples in the proposer prompt with the same language model Mini (Section 4.2 2 and select the highest
scoring hypothesis. Finally, we use the prompt-completion pairs to fine-tune mag-
A Proof of Concept Experiment. Since we cannot fine-tune text tonxt-davinci-O00 experiment with Flan-T5-xx (Chung et al., 2022), an open-sourced instruction-tune model that might only work well for easier "minieproblems" As proof of concept, we tested the above self-supervised learning algorithm on the task of describing groups of four samples, where each group comes from text cluster.
only
We computed both the automated "self-evaluation" validity score V and the "true" validity score V according to Turker evaluation for evaluation. After self-training, V" improves substantially from 0.22 to 0.37, and the V improves from 0.07 to 001 with a p-value of 0.02. This result provides preliminary evidence that self- self-training could be applied to a large set of problems to improve the

---

text-davinci-003 w/ goal
wo/ goal
gpt-4 w/ validator
w/ goal
wo/ goal
w/ validator
12%
2%
27%
15%
wo/ validator
4%
1%
wo/ validator
8%
5%
Table 1: The accuracy on SYND5 using different proposers, with/without incorporating goals, and with/without using validators. Using the validator, the goals, and gpt-4 leads to better results.
Hypothesis Relevance
!
!
average
Using the goal Not using the goal
79%
9%
12%
1.68
52%
16%
32%
1.20
Table 2: How often the hypotheses proposed by text-davinci-003 are rated by the authors as 2/C/ in terms of relevance (Section 3). Overall, using the goal significantly increases relevance.
validity of the hypotheses; we expect future validators to simulate human judgments better, hence decreasing the approximated gap of improvement between V and V". We discuss more training and evaluation detail in Appendix 20
5 Quantitative Evaluation on SYND5 and OPEND5
We show that both SYND5 and OPENDS can be used to quantitatively evaluate D5 systems. Since SYND5 is automatic, we used compare broad range of DS systems and studied the contributions of three different factors: the quality of the proposer model (gpt-4 vs. text-daving -003), the use of validator, and the use of goal. We then further investigated the effect of using goals under realistic applications through human evaluation on OPEND5.
Automatically comparing different variants with SYNDS. As mentioned above, ablated factors, resulting in 2 variants. We compared 1) using sext-davinci-00s vs. gpt-4 as the hypothesis proposer; 2) using the validator to compute V for each hypothesis and outputting the highest-scoring hypothesis, vs. not using the validator and outputting a random hypothesis; and 3) using the goal vs. replacing it with "/ want understand how Corpus A is different from Corpus B.". We then automatically calculated the accuracy for each variant as described in Section 2.2
We report the results in Table [ We find that using the validator and the goals significantly improve the performance, and gpt-4 outperforms sext--davinc--ie 1 % under t-test). We conducted two additional robustness checks in the Appendix [O] (a) using text-davinci -003 instead of Claude .1.3 to judge predicate equivalence, and (b) considering discoveries semantically similar the references also to be correct; conclusions do not change.
with goals and the validator (p <
Finally, to improve the accessibility of our research, we ran the same experiments using gpt-3.5- 50-rbo and flan-t5-xxl our proposer, and report the results in Appendix Table 6 To show that our conclusions are general and not only apply to synthetically generated texts, we additionally constructed
extension of SYND5 with human- written texts by adapting the NYT dataset from Wang et al. (2023), where each text sample a New York Times article with a topic and location label: the topic dimension has 9 different values (e.g., politics, arts) and the location dimension has 10 different values (French, Italy); we then followed the same procedure described in Section 2.2 to create this extension of SYND5, and report systems' performance in Appendix Table
In all experiments, using the validator and the goal improves the performance.
Investigating whether using goals improves relevance OPEND5. We then investigated whether text texi-davinci can leverage the goals to propose more relevant hypotheses on more realistic applications in OPENDS. We sampled 100 problems from OPEND5 with distinct goals and randomly sampled 2 hypotheses from text text-davinci- in 400 hypotheses evaluate. Three authors then rated their relevance based on the rubric in Section 3 while being blinded about which hypotheses were generated with the goal. Our main paper tocuses presenting the evaluations performed by ourselves, since crowdworkers might be noisy and untrustworthy (Veselovsky et al. 2023 Suhr et al., (021).
hith/without using goals (see Figure 3], resulting
We report the results in Table Since this evaluation is subjective, the inter-annotator agreement is only moderate (Kappa=0.56) however, we can still robustly conclude that text-davinci-0003 can leverage goals to propose hypotheses with higher average relevance rating, since this conclusion can
7

---

be independently reproduced by every individual evaluator with < 1 To make sure that the same conclusion can be robustly reproduced by external non-authors, we also evaluated the relevance of the hypotheses with Amazon Mechanical Turks, gpt-3. 5- 5-turbo, Claude-vi 3, and gpt-4. We report the results Appendix Table [ and found that our conclusion robustly holds under five different types of evaluators, including expert authors, external crowdworkers, and language models from different companies with different levels of capabilities.
Finally, we conducted similar experiments for the novelty and significance metrics in Appendix L4 and found that they both benefit from using goals as well. In the next section, we present example discoveries on OPEND5 to qualitatively understand what D5 system can achieve.
6 Qualitatively Analyzing Discoveries and Limitations with OPEND5
To understand the utility and the limitation of a D5 system, we ran it on OPEND5, set of realistic D5 problems, and analyze the output discoveries qualitatively.
6.1 Producing Discoveries on OPENDS and Analyzing Them
We ran our D5 system on OPEND5, producing 3296 discoveries in total. However, we do not have enough budget to validate every finding, since estimating V is expensive (Section 3 Therefore, from the remaining 3296 discoveries, we manually selected 21 discoveries that 1) achieve relevance score of 22. 2) are representative of potential use cases, 3) do not require expert knowledge for Turkers to judge, and 4) are likely to achieve small p-value with fewer than 200 samples from D
We then estimated their validity based on the procedure described in Section 3 by using fewer than 200 samples from the validation split and calculated the p-values, which cost ~$100 in total on MTuur. Since we are testing multiple discoveries and each of them can be statistically significant merely due chance, we keep 13 discoveries with V that significantly non-zero with p-value below 7%, a threshold determined by the Benjamini Hochberg's 's procedure with a false discovery rate of 10%. In other words, <10% of the discoveries presented are false discoveries in expectation.
We detail 5 of the 13 discoveries in this section, with the remainder in Appendix [8] For each discovery, report its automated validity score V", the estimated true validity score V. and their respective p values in Table 3
Understanding political stances and stereotypes in speeches. When comparing presidential speeches on immigrants from Obama to those from Trump, the former "argues for a path forward to promote the fair and just treatment of immigrants", while the latter more frequently "refers to illegal immigrants as criminals".
Analyzing errors in NLP systems. We fine-tuned a pair of models on two different natural language inference datasets, (a) MNLI and (b) SNLI. To understand their patterns of errors, we defined Corpus A to be the subset of MNLI where a is right and b is wrong, and Corpus B to be where b is right and is wrong. We found that the latter more often "has an informal tone, such as slang or colloquial speechi". One possible explanation is that MNLI contains more different genres and hence more informal speeches, causing the former model perform better on these examples.
Output discovery
V
p
V
p'
"argues for path forward to promote the fair.."
0.16
1.26e-04
0.35
2.01e-73
"refers to illegal immigrants as criminals"
0.09
6.17e-03
0.19
3.17e-38
"has an informal tone, such as slang or colloqu..."
0.08
2.35e-03
0.24
1.46e-35
"mentions lack of legroom"
0.16
1.15e-03
0.38
1.34e-45
"mentions children or family"
0.08
1.00e-05
0.11
8,05e-09
Table 3: A subset of discoveries presented in Section 6. and their associated estimated validity score V. validity score approximated by model V", and their respective p-values p (p') for the null hypothesis that V(V") < O under a t-test. We present the full set of 13 discoveries in Table 10
8

---

Analyzing airline customer reviews. We compared the concerns in reviews of the airline Air Canada v.s. its subsidiary, Air Canada Rogue. which is considered a low-price wing of Air Canada. The latter more often "mentions lack of legroom".
Analyzing gender differences in self-reported happy moments. Compared self-reported happy moments written by males, those by females "mentions children or family" more often. Caution: misinterpreting this correlation as causation could reinforce societal biases (Section 6.2).
Due to space constraints, we list more examples on analyzing distribution shifts, text clusters, lyric styles, and news headlines in Appendix [8 and their associated V and V values in Appendix Table I0, Across these discoveries, the approximated validity score V has a 71% spearman rank correlation with human rating V (66% for Pearson correlation), thus providing informative yet unreliable signals to practitioners about their validity. We hope that V' can better approximate V values in the future as the quality of the validators improve. Finally, future works collect more open problems, allowing D5 systems to produce more impactful discoveries.
6.2 Concrete Examples in OPENDS Inform Limitations of D5 Evaluation
We discuss limitations of D5 evaluation in this section using concrete examples from OPENDS.
Our metrics do not evaluate diversity. There are often multiple valid and relevant discoveries, and our system ideally should generate all of them. For example, when comparing low-rating and high-rating reviews to understand what stands out to customers, both "mentions the hidden fees and poor customer service at the airport" and "mentions the airline charging extra for carry-on items" could be valid discoveries. Our current evaluation does not reward diverse discoveries, and the current system sometimes repeats discovery using similar paraphrases, e.g., "mentions the rude and unprofessional arritude of the staff" and "mentions the staff being rude and unhelpful". Future evaluation metries can take diversity into account.
Interpreting discoveries requires domain experts. We used Turkers' judgment when computing T(h, x) judge the validity of discovery. However, many discoveries require expert knowledge to interpret properly. For example, it requires medical training to reliably judge whether a self-reported drug-use experience satisfies "mentions psychedelics, such as LSD and shrooms."
Correlation A causation. Our metrics currently do not evaluate whether the discovery is causally related to how the corpus pair was generated. For example, when comparing self-reported happy moments from females and males, even if the former corpus has more samples that "mention children and familly", it does not necessarily imply family plays more important role in inter inter-personal relations for females; an alternative hypothesis is that females might mention people in general more often than males do, hence leading to the observation that they mention family more often. Spurious correlations could also sneak into our validity evaluation: for example, if the Turkers implicitly associate female activities family-related Greenwald & Banaji (1995), then we might falsely make this discovery due to evaluator biases. Future metrics should also consider plausible alternative hypotheses to evaluate causality and control the potential biases from the human evaluators. Additionally, we should treat the discovery from D5 with caution to prevent automating and amplifying societal biases.
We discuss other limitations, such restricting the discovery to be single predicate, the biases in authors' qualitative evaluation, and the incomprchensiveness of OPEND5 in Appendix 19
7 Related Work and Discussion
Inductive Reasoning with NLP Models. Recent works show that language models are capable of inductive reasoning under restricted settings, discovering patterns from set of text data points and describing them with language (Honovich et al. 2022). Yang et al. (2022) use this capability to induce natural language rules with the format of "if then ... Zhou et al. (2022) and Ye et al. (2022) use this capability to improve zero/few-she shot accuracy by inferring the most likely instruction using input-output example(s) of the target task. Zhong al. (2022) and and (inggh et al. (2022) use this capability to discover patterns in datasets, and we improve by building an automatic benchmark and a dataset of open-ended problems and require the discovery to be relevant.
MI. models can also perform inductive reasoning in other modalities, such as vision. Hernande7
et al. (2021) describes visual features that activate neuron; Zhu al. (2022) describes distribution

---

shifts between the training distribution and the test distribution for images; and Eyuboglu et al. (2022) describes errors made by vision models. We hope future models can perform inductive reasoning in other modalities, such sound [Aghajanyan et al. 2023 physical senses (Thomason et al. 2016).
Exploratory Analysis and Automated Discovery. It is not new to automatically discover patterns by learning from empirical data. To list a few classical methods, linear regression analyzes the effect of each real-valued feature by interpreting the learned weights (Draper & Smith 1998); n-gram models can extract discriminative phrases, thus yielding insights about corpus-leve differences Manning Schutze, 1999;; ; topic models (Blei et al. 2003) can extract major topical variations across documents, where each topic is represented as a distribution over words; small decision trees can extract interpretable if-then statements (Letham
al.) 2013); and an entity embedding model learned on existing relations between entities can predict unseen relations (Socher et al. 2013). In comparison, D5 produces discoveries in the form of natural language predicates, which are interpretable and can express abstract concepts; additionally, it is more directed at the goal, while machine learning classifiers like naïve bayes or linear regression will pick up any discriminative features: Appendix offerrs more comprehensive discussion using examples from SYND5. Given the respective strength of D5 and traditional exploratory methods, we envision D5 to serve as a complementary method to traditional methods.
Epistemology. While the process of validating a hypothesis is cell-formulated, it is much less well- understood how to automatically generate hypotheses and decide what discoveries are meaningful (Shapere 1964 Heckman & Singer 2017). Related works this area have been sparse, among which
McGarry 2005) sketches high-level principles for evaluating knowledge discoveries and Ludwig & Mullainathan (2022) proposes to crowd-source hypotheses from MTurk workers. We coneur with the perspective of Polanyi et al. (2000) that meaningfulness of hypothesis cannot be explicitly verbalized with simple logic but is dependent on implicit community norms; therefore, the process proposing hypotheses should be learned from empirical data (e.g. pre-training, self-training, or human feedback) rather than deduced from a priori analysis of concepts (Quine) 1969). We hope contributions from other domains can provide more empirical data on what discoveries are meaningful, hence guiding our system to produce more important discoveries.
Acknowledgement
We thank Xiaochuang Han and Sam Bowman for their early discussions on this project. We thank Cathy Chen, Erik Jones, Jessy Lin, Alex Pan, Chenglei Si, Xi Ye, and Tianyi Zhang for their helpful feedback the paper draft. We thank OpenAl and Anthropic for providing model access.
References
The hewlett foundation:
Automated essay scoring, 2012.
URL
competitions/ asap-
The hewlett foundation: Short answer scoring, 2013. URL huttp:///kggglle...//mmttttttttp asap-sas
Ad observer. https://adobserveerorggg 2021. Accessed: 2022-12-30.
Armen Aghajanyan, Lili Yu, Alexis Conneau, Wei-Ning Hsu, Karen Hambardzumyan, Susan Zhang, Stephen Roller, Naman Goyal, Omer Levy, and Luke Zettlemoyer. Scaling laws for generative mixed-modal language models. arXiv preprint arXiv:2301.0377 033728 2023.
Roce Aharoni and Yoav Goldberg. Unsupervised domain clusters in pretrained language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7747-7763, Online, July 2020. Association for Computational Linguistics. doi: 10. 10.18653/v1/2020. acl-main.692. URL
org. acl main 692
Mohammad Alali, Shaayan Syed, Mohammed Alsayed, Smit Patel, and Hemanth Bodala. Justice: A benchmark dataset for supreme court's judgment prediction. arXiv preprint / 2021.
10

---

Akari Asai, Sara Evensen, Behzad Golshan, Alon Halevy, Vivian Li, Andrei Lopatenko, Daniela Stepanov, Yoshihiko Suhara, Wang-Chiew Tan, and Yinzhan Xu. Happydb: A corpus of 100,000 crowdsourced happy moments. arXiv preprint arXiv: 100:18011000 087.007766 2018
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 2022a.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint
2022b.
Miriam Barnum and James Lo. Is the npt unraveling? evidence from text analysis of review conference statements. Journal of Peace Research, 57(6):740-751, 2020.
Alexander Baturo, Niheer Dasandi, and Slava J Mikhaylov. Understanding state preferences with text as data: Introducing the un general debate corpus. Research & 4 12)1212311711117711111, 2017.
Akshay Bhalotia. Yc company scraper. httpg:///htaaao.oonaaaaaaaaaaaap scraper 2022.
Steven Bird, Ewan Klein, and Edward Loper. Natural language processing with Python: analyzing text with the natural language toolkit. O'Reeill Media, Inc.", "oc..." 2009.
David M Blei, Andrew Y Ng. and Michael I Jordan. Latent dirichlet allocation. Journal of machine Learning research, 3(Jn)):933-1022, 2003.
Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. The snli corpus. 2015.
Divy Bramhecha. Poetry Foundation Poems, 2019. URL https: htop::///wwww.. http://wwwwaagllccooooo htt.agggll..o///tttteeeeeeeeeeee http:://peetry-oooudaiio------- -pomss.
Dallas Card, Serina Chang, Chris Becker, Julia Mendelsohn, Rob Voigt, Leah Boustan, Ran
Abramitzky, and Dan Jurafsky. Replication code and data for "Computational analysis of 140 years of US political speeches reveals more positive but increasingly polarized framing of immigration" 2022.
[dataset].
www/dallascar//iiiiiiiioaaoaaaaa wwwp:/aaaadddnimirraimiition mmmigration-sspecleeee
Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. Neural legal judgment prediction in
english. arXiv preprint arXiv: 1906.0209, 2019.
Sihao Chen, Daniel Khashabi, Wenpeng Yin, Chris Callison-Burch, and Dan Roth. Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims. In Proc. of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2019, URL http:://coc/ org/
pdff
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-fintunne language models. arXiv preprint arXiv:22 20. 1416, 2022.
Christopher Clark, Kenton Lee, Ming-Wor Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv: /905. 10044, 2019.
Norman R Draper and Harry Smith. Applied regression analysis, volume 326. John Wiley & Sons, 1998.
Sabri Eyuboglu. Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Ré. Domino: Discovering systematic errors with cross- modal embeddings. arXiv preprint arXiv:2203. 14960, 2022.
Yujia Gao, Jinu Jang, and Diyi Yang. Understanding the usage of online media for parenting from infancy to preschool at scale. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1-12, 2021.
11

---

Anthony G Greenwald and Mahzarin R Banaji. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review, 102(1):4, 1995.
Ivan Habernal and Iryna Gurevych. Which argument is more convincing? Analyzing and predicting convincingness of Web arguments using bidirectional LSTM. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1589-1599, Berlin, Germany, 2016. Association for Computational Linguistics. URL http: aclweb org brg/anthology// 1tt.hooooooy11---1155151 PP6-1150
Kevin Hartman. Advertisement Transcripts from Various Industries, 2019. URL https::///ttinurr com/5x36dwdx
He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. Decoupling strategy and generation in negotiation dialogues, 2018
Jibo He. Big Data Set from RateMyyProfesso.ccomm for Professors' Teaching Evaluation, 2020. URL
Samuel He. Goodbye world: using natural language processing to identify suicidal posts, 2021. URL
James J Heckman and Burton Singer. Abducting economics. American Economic Review, 107(5): 298-302, 2017.
Evan Hernandez. Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, and Jacob Andreas. Natural language descriptions of deep visual features. In International Conference on Learning Representations, 2021.
Or Honovich, Uri Shaham, Samuel Bowman, and Omer Levy. Instruction induction: From few examples to natural language task descriptions. arXiv preprint arXiv:2205. 2022.
Nabil Hossain, John Krumm, and Michael Gamon. ppresident vows cut< taxes> hair": Dataset and analysis of creative text editing for humorous headlines. arXiv preprint 17X1::1906.002744 2019.
Kaggle. TMDB 5000 Movie Dataset, 2018. URL httppp///wr..tgggllee...//aaaaap
Rohit Kulkarni. Million News Headlines, 2018. URL http:://6oi.oog/111790000 SYBGZL
Rohit Kulkamni. The Examiner - Clickbait Catalog, 2020a. URL fettps://www.kappt
Rohit Kulkarni. Urban Dictionary Words And Definitions, 2020b. URL htttps:/wwww.kggl
-dtaset
Rohit Kulkami.. India News Headlines Dataset, 2022. htlLhttps://wwwt congglee.cm/dttssttss/
therohk/ind hawrro///indiaahhaannnee---e--es -nessds------eeeetsassaeeee
Benjamin Letham, Cynthia Rudin, Tyler H. McCormick, and David Madigan. Interpretable classifiers
using rules and bayesian analysis: Building better stroke prediction model. The Annals of
Applied Statistics, 9(3), Sep 2015. ISSN 1932-6157. doi: 10.1214/15-a0as888. URL http org/ 10. 111/111-1AASSS
Derek Lim and Austin R Benson. Expertise and dynamics within crowdsourced musical knowledge curation: A case study of the genius platform. In ICWSM, pp. 373-384, 2021.
Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. Wanli: Worker and ai collaboration for natural language inference dataset creation, January 2022. /ttpp:////a/ org/
2201.0595
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint 19Xiv:1907.11992 2019.
12

---

Zhi Liu. Reuter Souter__0__tt 50_50 Data Set, 2011. URL Reuter_50
wcw.
Jens Ludwig and Sendhil Mullainathan. Algorithmic behavioral science: Machine learning as a tool for scientific discovery. Chicago Booth Research Paper, (22-15), 2022.
Christopher Manning and Hinrich Schutze. Foundations of statistical natural language processing. MIT press. 1999.
Ken McGarry. A survey of interestingness measures for knowledge discovery. The knowledge engineering review, 20(1):39-61 2005.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture
models, 2016.
Natan Mish. Federal Reserve Governors Speeches 1996 2020, 2020. URL https://tinur com /omm332279a6
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. In ACL, 2022.
Rishabh Misra and Prahal Arora. Sarcasm detection using hybrid neural network. arXiv preprint arXiv:1908.07411 2019.
Rishabh Misra and Jigyasa Grover. Sculpting Data for ML.: The first act of Machine Learning. 01 2021. ISBN 9798585463570.
Nuno Moniz and LuâeTMis Torgo Multi-source social feedback of online news feeds. CoRR, [Web Link], 2018.
Mickaël Mouillé. Kickstarter Projects, 2017. URL
www.kkaggge..dddaaaaaaee. /atasessss 200. 201612..011
Dong Nguyen, Maria Liakata, Simon DeDeo, Jacob Eisenstein, David Mimno, Rebekah Tromble, and Jane Winters. How do things with words: Analyzing text as social and cultural data. Frontiers in Artificial Intelligence, 3:62, 2020.
Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine poeegrained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference natural language processing (EMNLP-IJCNLP), pp. 1888197,10 2019.
Elle O' O'Brien. iterative/aita_dateee dataset: Praw rescrape of entire dataset, February 2020. URL https 7/doi org/ 5281/zenodo. 3677563.
Long Ouyang. Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203: 2203.02/55, 2022.
Verónica Pérez-Rosas and Rada Mihalcea. Experiments in open domain deception detection. In Proceedings of the 2015 conference on empirical methods in natural language processing, pp. 1120-1125, 2015.
Verónica Pérez-Rosas, Mohamed Abouelenien, Rada Mihalcea, and Mihai Burzo. Deception detection using real-life trial data. In Proceedings of the 2015 ACM International Conference on Multimodal Interaction, pp. 59-66,666 2015.
Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. Automatic detection of fake news. arXiv preprint arXiv:1708.07100 2017.
Denis Peskov, Benny Cheng, Ahmed Elgohary, Joe Barrow, Cristian Sinescu-Niculescu- and Jordan Goyd-Graber. It takes two to lie: One to lie and one to listen. In Association for Computational Linguistics, 2020.
13

---

John P. Pestian, Chris Brew, Pawel Matykiewicz. DJ Hovermale, Neil Johnson, K. Bretonnel Cohen, and Wlodzislaw Duch. A shared task involving multi- classification of clinical free text. In Biological, translational, and clinical language processing, pp. 97-104, Prague, Czech Republic. June 2007. Association for Computational Linguisties. URI. https://aclanttooogggoooo W07-1013
Michael Polanyi, John Ziman, and Steve Fuller. The republic of science: its political and economic theory minerva, i (1)(1662), 54-73. Minerva, 38( (3(1)) 2000.
Ilan Price, Jordan Gifford-Moore, Jory Fleming, Saul Musker, Maayan Roichman, Guillaume Sylvain, Nithum Thain, Lucas Dixon, and Jeffrey Sorensen. Six attributes of unhealthy conversation. arXiv preprint ar arXiv:2010.074 2020.
Demand Progress.
Statements of Administration Policy, 2022.
URL
https
//github. cow Srgpr//teeeettttttttttoeoeeeaaaa statements- of wtt.-aamwiiiiittttiiaaooo.....
PrompiCloud. U.S. Technology Jobs on Dice.com, 2017. URL https://wwwt
htgp:::///w....llee.... kaggle com/
jobs on- oo-oonnccoooe dicecom
WVO Quine. Naturalistic epistemology. Ontological relativity and other essays, pp. 69-90, 1969.
Quora. Quora Question
Pairs, 2017.
URL httpp://www.kagee
quora- - wetee-uuusiiiiiiiieeee pairs
Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable questions for squad. arXiv preprint 1rXiv: 03822, 2018.
Justin Robischon. Wikipedia Movie Plots, 2019.
Allen Roush and Arvind Balaji. DebateSum: A large-scale argument mining and summarization dataset. In Proceedings of the 7th Workshop on Argument Mining, pp. 1-7, Online, December 2020. Association for Computational Linguistics. URL hotphh:t/////// ftttp:://alaanthology .rgge.o.rrg/// org/2020 argmining-1.1
Dudley Shapere. The structure of scientific revolutions. The Philosophical Review, 73(3):383-394, 1964.
Chandan Singh, John X Morris, Jyoti Aneja, Alexander M Rush, and Jianfeng Gao. Explain- ing patterns in data with language models via interpretable autoprompting. arXiv preprint arXiv:2210.01888, 2022.
Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. Reasoning with neural tensor networks for knowledge base completion. Advances in neural information processing systems, 26, 2013.
Alane Suhr, Clara Vania, Nikita Nangia. Maarten Sap, Mark Yatskar, Samuel Bowman, and Yoav Artzi Crowdsourcing beyond annotation: Case studies in benchmark data collection. In Proceedings of the 202/ Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts, pp. 6, 2021.
J Sun. Daily News for Stock Market Prediction, 2017. URL https: / /ttp://wwwwwke. /w/p///.kkn kaggle....... com/ datasets/aaron//un/ stocknewss
Jesse Thomason, Jivko Sinapov, Maxwell Svetlik, Peter Stone, and Raymond Mooney. Learning multi-modal grounded linguistic semantics by playing" i spy". In IJCAI, pp. 3477-3483, 2016.
Andrew Thompson. All the News 1.0, 2019. URI. https::/componentse one/ /etedtttsss//////
all-the- wttee-arrtcteeddttteeeeeee
Elsbeth Turcan and Kathleen McKeown. Dreaddit: reddit dataset for stress analysis social media. arXiv preprint 11000:19,, 2019.
14

---

Udacity. Armenian Online Job Postings, 2017. URL httpp::/www..een www.glle..ommdaaaeeeeeee udacity/armeniar online- job- pott-psstingss
Veniamin Veselovsky, Manoel Horta Ribeiro, and Robert West. Artificial artificial artificial intelli- gence: Crowd workers widely use large language models for text production tasks. arXiv preprint arXiv:2306.07899, 2023.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabsshi, Yeganch Kordi, Amirreza Mirzaci, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, al. Super-nnaturalinstuutionss Generalization via declarative instructions on 1600+ nlp tasks. URL https://arxii. org/abs/2204.07705 2022.
Zihan Wang, Jingbo Shang, and Ruiqi Zhong. Goal-driven explainable clustering via language descriptions. arXiv preprint arXiv:2305. 13749, 2023.
Orion Weller and Kevin Seppi. The rJokes dataset: large scale humor collection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. Marseille, France, May 2020. European Language Resources Association. ISBN 979-10-95546 34-44 URL https //aclanthology. org/
org/0020.00 2020. 1rec- 1.2c-1.753
Adina Williams, Nikita Nangia, and Samuel R Bowman. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv: 2017.
Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, and Furu Wei. Language models inductive reasoners. arXiv preprint arXiv:2212.109233 2022.
Seonghyeon Ye, Doyoung Kim, Joel Jang, Joongbo Shin, and Minjoon Seo. Guess the instruction! making language models stronger zero-shot learners. arXiv preprint arXiv:2210.02969, 2022.
Ruiqi Zhong, Charlie Snell, Dan Klein, and Jacob Steinhardt. Describing differences between text distributions with natural language. In International Conference on Machine Learning, pp. 27099-27116. PMLR, 2022.
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910. 2022.
Zhiying Zhu, Weixin Liang, and James Zou. Gselip: A framework for explaining distribution shifts in natural language. arXiv preprint arXiv:2206. / 5007, 2022.
15

---

8 Cost for Running the Experiments
For each problem, we ran the proposer for 10 times on average; assuming each prompt to at most 4000 tokens, we spent around $2.4 for each problem on OpenAI APIS if we use gpt-4 and text-davinci
and the cost would decrease $0.8 if we use gpt- 5 turbo. Notice that these estimates are computed based on the prices of 05/14/2023, and we expect the price to further decrease in the future. We ran the Flan-T5 based validator for hours on 80G A100 GPUs.
The total amount of computational resources spent in this research paper is around $2,500 in terms of OpenAI API and 3,000 hours of compute on A A100 GPU with 80G memory.
Generation Process of SYND5
The high-level description is in Section 2.2 Here we discuss the procedure that generated SYND5.
We consider three dimensions of differences: topic, genre, and style. For each, we generated 14/9/7 values, e.g., "celebrity love stories" and spprtss team recruiting athletes for the topic attribute, "rap lyrics" and "screen play" for the style attribute, and "French" and "Spanish" for the language attribute. We then used GPT-4 and the Claude API synthesize 54K text samples, where for each text sample we sampled topic, genre, and style randomly, e.g. "Write a rap about sports team recruiting athletes in French". synthesize random SYND5 problem, we randomly sampled distractor dimension (e.g. language) and target dimension (e.g. topic). and for each dimension we sampled two random values (e.g. English and French for language, sports and for topic).
For each problem, we sampled 10 texts for corpus such that all of them satisfy one sampled value for the distractor dimension (e.g. corpus A is entirely in English), and 10 texts for corpus B for to satsify the other distractor dimension (e.g. corpus B is entirely in French). Then we set V fraction of corpus A satisfy the reference target attribute, e.g. "is sports-related". and fraction of corpus B to satisfy the other value for the target dimension (e.g. "is art-related"). We chose V uniformly at random from [0.6, 0.8, 1]. Finally, we provide example hypotheses from the target dimension other than the target dimension values for Corpus A and Corpus B, and we chose k from [0. 2] uniformly random. We then sampled 300 D5 problems in total from this distribution.
10 Robustness Checks for Results on SYND5
Table 4 shows the accuracy of different systems using text-davinci-003 the judge for semantic equivalence. Table shows the accuracy of different systems if we consider outputs semantically similar to the reference be correct. Across all setups, found that the conclusion reached in Section 5 still holds under these robustness checks.
text-davinci-003 w/ goal
wo/ goal
gpt-4 w/ validator
w/ goal
wo/ goal
w/ validator wo/ validator
6%
1 1%
23%
9%
3%
0%
wo/ validator
6%
2%
Table 4: Same Table as except that we use text-davinc-00) instead Claude-vi. vindde-vi.3. to judge similarity. Using the validator, the goals, and gpt-4 leads to better results.
text-davinci-003 w/ goal
wo/ goal
gpt-4
w/ goal
wo/ goal
w/ validator wo/ validator
46%
23%
w/ validator
53%
43%
24%
16%
wo/ validator
24%
24%
Table 5: Same Table as equivalent, to the reference. Using the validator, the goals, and gpt-4 leads to better results.
except that we calculate how often the output is similar, rather than
To improve the accessibility of our research, we ran the same experiments with gpt-3. 3.5........... t-tubo and flan-t5-xxl,, and report the results Appendix Table To show that our conclusions are general and not only apply synthetically generated texts, we additionally constructed an extension of SYND5 with human-written written texts by adapting the NYT dataset from Wang et al.) (2023). where each text sample is New York Times article with topic and a location label: the topic dimension
16

---

has 9 different values (e.g., politics, arts) and the location dimension has 10 different values (French,
Italy); we then followed the same procedure described Section 2.2 to create this extension of
SYND5, and report our systems' performance in Appendix Table Under all experimental setups,
using the validator and the goal improves the performance.
w/ g, w/ V
wo/ w/ v
wo/ g. w/v
wo/ wo /v
flan-t5-xx xx1
0.05
0.03
0.02
0.01
gpt-3. 5-trbo gpt-4
0.27
0.10
0.08
0.03
0.27
0.15
0.08
0.05
Table Similar Table we used gpt-3.5-tte 5-turbo and flan-t5-xx) as the proposer to tackle the SynD5 dataset, and report the performance with/without using the goal (g), and with/without using the vadliator (v). We find that using the goal and the validator significantly improves the performance, and open-sourced models lag significantly behind. Additionally, gpt-4 does not significantly outperform 1pt-335-turee
w/ g, w/ v
wo/ g, w/ wo/ g, w/v
wo/ g, /v 0.10
5---3.5-turae gpt-4
0.61
0.24
0.22
0.55
0.28
0.22
0.16
Table 7: We created an extension of SynD5 by adapting dataset of New York Times articles with two dimensions: topic and locations, each with 9 and 10 values. We then used gpt-3. 3. 5-turbo and gpt-4 as the proposer, and found the same conclusion: using the goal and a validator improves the performance. Additionally, gpt-4 does not significantly outperform gpt-3. 3ppt-..5ttum 5-turbo.
11 Computing Turker Judgement
Scoring. To estimate T(h, x) with Turker's rating, where is a truth predicate of text sample æ, the Turker needs to read h and r and then choose among six options: "Certainly Yes", "Likely Yes". "Neutral", "Likely No", "Certainly No". and "Confusing/Cannot be decided." For each (h, æ) pair, we collect responses from three Turkers. To compute the average across them, we collect a list of scores using the following rule: each "Certainly Yes" would receive score of 1.00, "Likely Yes" 0.75, "Neutral" 0.50, "Likely No" 0.25, "Certainly No" 0.00, and "Confusing/Canno be decided." receive two scores of 0.50. We then take the average over all the scores we collected from the Turkers for one l and response to drag the average rating towards neutral and it has larger effect than choosing "Neutral".
Confusing/Cannee be decided." receives two scores of 0.50 because we want such
Payment. We adjust the payment for each HIT task based on the number of words they need to read. We pay them approximately 0.001 cent per word, and using the conservative estimate that adults read about 200 words per minute, we pay them around $12 per hour. We spent in total around $5K on this HIT task.
Qualification. We only recruited Turkers who are located in the U.S. Additionally, we designed qualification test with questions; the questions designed to be easy to answer long they have read our instructions below, and we only accepted turkers who made mistakes on at most one questions.
Annotation Instruction. We show our annotation instruction below. We only show examples of choosing "Certainly Yes", "Certainly No", and "Confusing" to encourage the Turkers not choose neutral ratings. Additionally, we explicitly tried to address Halo effect - where the text does not satisfy predicate h but satisfies a predicate h' that is highly correlated with h. For example, for the text sample x = "Really love the flight!! does not satisfy the predicate "mentions that the breakfeast good on the plane", even though it satisfies highly correlated predicate ht' "likes the flight."
11.1 Instructions
Below are the same instructions we have shown you during the qualification. Thanks for visiting this page and refresh your memory about the instruction!
17

---

Instruction: In this task, you will check whether a TEXT satisfies PROPERTY
Example 1
Property: mentions a natural scene.
Text: love the way the sets in the evening.
A) Certainly Yes.
Likely Yes.
C) Neutral.
D) Likely No.
E) Certainly No.
Confusing/Cannot decided.
Answer. A. set is nature-related; if you feel bit ambivalent, B is also acceptable.
Example 2
Property: writes in 1st person perspective. Text: Makima is cute.
A) Certainly Yes.
B) Likely Yes.
C) Neutral.
D) Likely No.
E) Certainly No.
F) Confusing/Cannot be decided.
Answer. E. This text is undoubtedly written in the 3rd person perspetive, so E.
Example 3
Property: better than group B. Text: also need to buy chair.
A) Certainly Yes.
B) Likely Yes.
C) Neutral.
D) Likely No.
E) Certainly No.
F) Confusing/Cannot be decided.
Answer: F. is unelear what the hypothesis mean (e.g., what does group B mean?) and doesn't 't seem related to the text. So F.
Example 4
Property: mentions that the breakfast is good on the airline.
Text: The airline staff was really nice! Enjoyable flight.
A) Certainly Yes.
B) Likely Yes.
C) Neutral.
D) Likely No.
E) Certainly No.
F) Confusing/Cannot be decided.
18

---

Answer. E. Although the text appreciates the flight experience, it DOES NOT mention about the breakfast. So the answer is E.
Example 5
Property: appreciates the writing style of the author.
Text: The paper absolutely sucks because its underlying logic is wrong. However, the presentation of the paper is clear and the of language really impressive.
A) Certainly Yes.
B) Likely Yes.
C) Neutral.
D) Likely No.
E) Certainly No.
F) Confusing/Cannot be decided.
Answer. A. Although the text dislikes the paper, it DOES like the writing style. So the answer is A.
12 Prompt to Judge Predicate Similarity
We prompt Claude v1.3 [Bai et al. 2022b) to judge whether the predicated predicate is similar to the reference. We consider response that leads a"yes" to be correct when require the discovery to be semantically equivalent to the reference, and consider a response that leads to a "yes" or "related" be correct when we require the discovery be semantically similar the reference.
Is text_a and text_b similar in meaning? respond with yes, related, or no.
Here are a few examples.
Example 1:
text a: has a topic of protecting the environment text has a topic of environmental protection and sustainability
output: yes
Example 2:
text_at has a language of German text texi_b: has a language of Deutsch outpul: yes
Example 3:
text a: has a topic of the relation between political figures text b: has topic of international diplomacy output: related
Example 4:
has a topic of the sports text b. has a topic of sports team recruiting new members output: related
Example 5:
text_at has a named language of Korean text text_b: b: uses archaic and poetic diction output: no
Example 6:
text texi_a: a: has a named language of Korean text b: has a named language of Japanese output: no
19

---

Target:
text a: (predicate]
text b: (reference)
outpu:""
13 Relevance Rating with External Non-Authors
To make sure that the conclusion that "using goal in the context improve hypotheses relevance" can be robustly reproduced by external non-authors, we also evaluated the relevance of the hypotheses with Amazon Mechanical Turks, in Table 8 and found that the conclusion still robustly holds.
Claude-v1.. 3, and GPT-4. We report the results
Relevance Rater
w goal
w /o goal
p-value
spearmanr
Authors
1.68
1,20
x100
1.00
Turkers
1,56
1.44
4 x1002
0.10
gpt-3. 3...bbrbbo
1.05
0,94
5 x1102
0.19
claude-v1.3
1.18
0.92
2 x1003
0.30
gpt-4
1.49
1.12
x100
0.45
Table 8: We rated the relevance in the same way Table 5 However, in this table we obtained the ratings not from the authors, but from four different evaluator types: Turkers, 3.-trbbrrrrb, claude-v1.3 and gpt-4. For each evaluator type, we calculate (1) the average rating of the candidate discovery when goal is (not) present in the proposers' prompt, (2) the p-value that the average rating when goal present is higher under a t-test, and (3) the spearman rank correlation between its rating and the authors' rating. We find that the y-value is smaller than 0.05 in all cases, indicating that our conclusion is robust; additionally, more capable models has a higher correlation with the authors.
14 Meaningfulness: Relevance, Novelty, and Significance
Not every valid discovery meaningful. For example, if the goal is understand the topical differences between news from 2008 (Corpus A) and news from 2007 (Corpus B), the discovery that Corpus A "contains news from 2008" is completely valid by definition but meaningless, since provides only trivial information and is irrelevant to the goal of understanding topical differences.
McGarry 2005) surveyed a list of desirable properties for discovery, and condensed them into three submetrics to rate how meaningful a discovery is based on the exploration goal: 1) relevance, 2) novelty, and 3) significance. We evaluate these independently of validity and assume that the discovery is already valid. For example, the discovery that "something can travel faster than light" is meaningful if true, even though it highly implausible.
We rate each submetric with O, D. or ④, where higher better. We show the evaluation instructions below and present our rating on text devitddvvnci--.. -003 proposed hypotheses.
14.1 Evaluation Instructions
How relevant the discovery to the goal. For example, suppose we were a student comparing essays rated as convincing vs. not convincing to figure out what writing style is convincing.
Relevance.
Then:
The discovery "write in first person" is directly related to the writing style, so rate it 2
The discovery "use the word "I"" is not exactly writing style, but can still inform the relevant underlying principle "write first person", so we rate it ①
The discovery "argue for abortion" does not tell about the underlying writing style, so we rate it (.
Novelty, The difficulty of generating the discovery, e.g. can we think of the discovery minutes with the goal but without looking at the corpora? For example, suppose were an airline manager
20

---

trying to find improvements to the flight experience, and we were comparing negative reviews vs. positive reviews. Then:
The discovery "contain more negative language" is almost certain for negative reviews, so we rate it !..
The discovery "complain about the crew members" not entirely novel, but is not tautologi- cally true and hence requires confirmation, so rate it !.
The discovery "mention a language barrier with the crew members" is specific and hard to think of without looking at the data,
rate it ②
Note that our evaluation is "blinded to the samples": we still consider discovery novel long it is hard to think of before looking at the corpora, even if it might be easy to think of after looking at the corpora. For example, the physical law that F ma is easy observe if we have collected and plotted the data on acceleration, mass, and force; however, it might be difficult to think of before we see any such data, so we consider it novel.
Significance. Given the exploration goal, how beneficial is it to learn the discovery for the first time? For example, suppose we were an Amazon retailer trying to figure out what customers like and dislike about my product based on negative reviews and positive reviews. Then:
The discovery "accuses the team pushing out a bad product" is not significant since cannot direct the retailer to improve the product, so rate it !
The discovery "asks for a more durable product" gives some hints about how to improve the product, but isn't sufficiently helpful its own, rate it I
The discovery "says the wrench is missing" can lead to concrete actions for improvement, so we rate it 2
14.2
Goal Leads to More Mcaningful Hypotheses
with-goal
no-goal
kappa
spearmanr
of avg
worst p of ind
Relevance 1.68
1.20
0.56
0.71
1 X 10-10 -10
1 x 10-8
Novelty 1.56
1.24
0.97
0.37
0.50
5 5 10-6
4 x 10-2
Significance
1.05
0.46
0.64
2 2 10
2 x 10-7
Table 9: Left. For each metric, we report the average rating on hypotheses generated with or without using the exploration goal, and find that the former performs better. Middle. The anter-annotator agreement rate averaged across pairs of author evaluators, measured by Kappa and Spearman rank coefficient; we find substantial correlations between evaluators across all these subjective metries, with relevance >> significance >> novelty. Right. We compute the p-values for the null hypothesis that "with-goal and no-goal result the same performance". The p of avg column reports the p-values after we average the ratings from all evaluators, while the "worst of ind" column takes the max of all p- values based on ratings of individual evaluators. Overall, the conclusions are statistically significant and they can be robustly reproduced across individual evaluators.
Compared to Zhong et al. (2022). we added the exploration goal to our prompt when generating hypotheses. Does this improve the quality of the proposed hypotheses? To investigate this, we sampled 100 problems from OPENDS with distinct exploration goals and randomly sampled 2 hypotheses from GPT-3 with and without using exploration goal (see Figure 3)), resulting in 400 hypotheses to evaluate. Three authors then rated their meaningfulness based on the three metrics defined in Section while being blinded about which hypotheses were generated with the exploration goal.
The results are shown in Table We found that, when prompted with the exploration goal, GPT-3 on average proposes more relevant, novel, and significant hypotheses; additionally, it proposes hypothe- ses with ratings higher than @ 31% //1 %/28% more often in terms of redevance/ovellyyiiggiiicc Since this is subjective evaluation, the Kappa inter-annotator agreement is only moderate, ranging from 0.37 to 0.56. However, we can still robustly conclude that the model can propose more mean- ingful hypotheses when conditioned on the goal: we calculate the p- y-values for the null hypothesis that with-goal and no-goal have equal performance, and we find p- values to be highly significant and robust across evaluators, for all three submetries.
21

---

15 Full Pipeline of the Proposer
We present the full details of how we generated the hypotheses with the language model. The process roughly contains four stages: 1) obtaining representative samples for each corpus. 2) sampling hypotheses from GPT-3, 3) rewriting hypotheses, and 4) optionally plugging in example hypotheses.
Obtaining representative samples. This step is the same as Zhong et al. (2022), and we borrow the related text from that paper for the reader's convenience. Since D and D might overlap significantly, random samples from D and D might not be representative and informative enough for GPT-3 to notice the differences between the two distributions. Therefore, we choose samples that are representative of their differences. To find those samples, we fine-tune RoBERTa-L eet al. (2019) to predict whether each sample comes from Corpus A or Corpus and keep the top-p percentile samples with the highest confidence. Next, take samples from the top-p percentile to prompt GPT-3.
Liu
Selecting samples to prompt GPT-3. We randomly select S =25 samples from the top-5 percentile from Corpus A and Corpus B to prompt GPT-3 to propose the hypotheses, using the template shown in Figure left. We require the length of the prompt be at most 3,200 GPT-3 tokens (the max window size for GPT-3 text-davinci-00s. is 4096) and gradually decrease the number of samples S in the prompt until the prompt length is less than 3,200; additionally, we truncate each text samples to at most 256 GPT-3 tokens. Finally, to prevent GPT-3 from proposing hypotheses that reflect simple lexical correlations that can be detected with unigram models, e.g., "uses the word "hey" more often." " we incrementally construct the subset of samples for Corpus A and Corpus B such that at any time of the construction, no single word can appear 0.255 times more often one corpus than the other. We repeat the same process for the top-20 and 100 percentile until we obtain 60 hypotheses.
Rewriting hypotheses with GPT-3. As mentioned Section 6.2 the hypotheses generated by GPT-3 are frequently statements about the corpus, while the validator requires the hypothesis to be a predicate on individual text samples. For example, when comparing definitions that people like from (UrbanDcctioaarryy com to other definitions, the hypothesis that the former "is more likely to include slang or colloquial terms." is a statement about a collection of text samples, rather than a predicate
individual sample. T(h, x)) is undefined in this case, since it does not make sense to check whether a single text sample is more likely to include slang. Ideally, we want to detect these comparison statements and automatically remove the comparatives, e.g., rewrite it "includes slang or colloquial terms.".
To detect and remove the comparatives from the hypotheses, we tag the part of speech for each word in the hypotheses using the NLTK package (Bird et al.) 2009) and check whether any tag is JJR or RBR. If hypothesis indeed contain theses tags, we prompt GPT-3 rewrite the hypothesis. We show an example prompt in Figure 4
Plugging in example hypotheses (optionally). We can also add few problem-specinn example hypotheses to the prompt to elicit more relevant hypotheses, and we do so by adding them to the "formatting instruction" part in the prompt used to propose hypotheses Figure 3 In OPEND5, provided example hypotheses for each problem to steer our system to generate more meaningful discoveries; we produced the example hypotheses by prompting GPT-3 to generate few hypotheses and selecting the meaningful ones from them.
For the reported discoveries in Section 6. we confirmed that they are unambiguously different from our provided hypotheses; otherwise, the system might have produced the discoveries by copying the provided hypotheses. We did not use the example hypotheses in Section 5 to test GPT-3's zero-shot understanding of the goal.
16
Collecting Data to Fine-tune the Validator
Here we provide high-level description of how the data was collected. For each problem in OPEND5, we used our proposer to produce list of hypotheses. We automatically judged each hypothesis on subset of samples from the research split using GPT-3 dexi-davinci-002 (Ouyang et al. (002). Flan-T55
(Chung et al. 2022), and model trained with RLHF from Bai et al. (022a). We created the input
22

---

Remove the comperetives. Remove mention Group A or Bf they appeer.
longer sentences from group
Output: contain sentences
Inouu: eupports DACA more strongly
Output: supports DACA strongly
Trigut: uses the hashtag #Alll IvesMattte more often
Output: uses hashtag MAlltieaMatter
Trggu: more likely to contain grammatical errors Output: contain grammatical errors
those
Output: sounds hapay
Inout: more likaly nalude slang or colloquial terms Output/Ineludes colloquial terms
Figure 4: The prompt to remove comparatives from a hypotheses.
distribution for training by combining and equally weighting the following 3 2 4 distributions: the subset of (h, x) pairs that GPT-3/Flan-TSSRRLLR. answer. We then collected averaged turker ratings for in total 3138 (h, pairs and used them to fine-tune Flan-Tr T5 to create the validator (Chung et al. 2022).
considers Yes or No to be the most likely
To test cross problem generalization capability of our D5 system, whenever we applied our D5 system problem in OPENDS Section 6.1] we used a validator that is NOT fine-tuned on the (h, pairs from this problem. We achieved this by keeping track of which problem each (h, æ) pair comes from and split all the (h, x) pairs into three folds based on the problems; whenever we applied our D5 system to problem, used the validator trained the two folds that do not contain this problem.
Samples from Corpus
hypothesis;
hypathesss
LM
100%
Corpus DIff Sound?
Corpus
LM
hypothesis,
hypothesis,
Samale
Menonle.XIiRe AaameeeIAAAaaammma.aa A
sypothesis
LM
0%
hypothesis) 100%
Plyotthesis:
90%
100%
0%
90%
0%
hypothesiss
Sample Carpus
hypothesis 10%
15%
-5%
No
Problem Context
hypothesis,
[Other hypotheses not included for brovity]
Propose hypotheses based on the problem context and samples from Corpus and Corpus
Judge hypothesis, on all individual samples from Corpus Corpus
Compare how often each hypothesis is Corpus compared Corpus
Figure 5: A sketch of the baseline method. The description can be seen in Section 4 and the actual prompts can seen in Figure
17
What Discoveries Did we Choose to Present
Our system in total produces 3296 discoveries on OPENDS. However, we do not have enough budget validate every finding, since estimating is expensive (Section ??). Therefore, from the remaining 3296 discoveries, we manually selected 21 discoveries that 1) the authors think are relevant enough, 2) are representative of potential use cases. 3) do not require expert knowledge for Turkers to judge, and 4) are likely to achieve a small p-value with fewer than 200 samples from D and Dyal. We then estimated their validity based on the procedure described in Section ?? by using fewer than 200 samples from the validation split and calculated the p-values Since we are testing multiple discoveries and each of them be statistically significant merely due to chance, keep 13 discoveries with V that are significantly non-zero with p-value below 7%, a threshold determined by the Benjamini Hochberg's 's procedure with a false discovery of 10%. In other words, fewer than 10% of the discoveries presented are false discoveries in expectation.
'e determined the number of samples s.t. V can achieve p-value of 0.005. Estimating V for these discoveries costs $$5000.
23

---

discovery
V
P
V"
P
argues for path forward to promote the fair ...
0.16
1 1.26e-04
0.35
2.01c-73
refers to illegal immigrants as criminals
0.09
6.17e-03
0.19
3.17e-38
has an informal tone, such as slang or colloqu...
0.08
2.35e-03
0.24
1.46e-35
mentions lack of legroom mentions children or family
0.16
1.15e-03
0.38
1.34e-45
0.08
1.00e-05
0.11
8.05e-09
Uses language that is positive uplifting
0.12
2.12e-03
0.24
4.18e-59
references violence or aggression involves physical activity, such as walking, p...
0,06
9.87e-03
0.17
4.25e-26
0.13
4.92e-03
0.37
7.07e-101
contains keywords related to business, finance...
0.08
2.89e-02
0.35
1.45e-95
mention disasters and crimes, such as plane ac...
0.03
7.03e-02
0.09
4.61e-06
discusses coronavirus-related topics references pop culture, such as movies, books....
0.21
1.01e-04
0.27
9.19c-78
0.21
2.67e-04
0.58
2.09e-30
uses vivid imagery and metaphors to convey a f...
0.09
2.47e-02
0.45
5.04c-64
Table 10: The full table of discoveries, along with their V, V', p. and scores.
18 More Example Discoveries on OPEND5
Analyzing errors NLP systems. We considered the task of perspectrum classification (Chen al. 2019), which has the following instruction: "given perspective and claim, classify whether the given perspective supports or undermines the claim. If the perspective could possibly convince someone with different view, is supporting, otherwise it is undermining." We considered two few- shot learning systems: GPT-3 Instruct Curie (Ouyang et al. 2022) and Tk-Instruct-1 1B (Wang et al., 2022). We focused on the perspectives where the ground truth label is undermining, and compare the following two corpora: Corpus A - the set of perspectives where Curie correctly classifies the input undermining but Tk-11B is wrong, and Corpus B the set where TK-11B is correct while Curie is wrong. We found that Corpus B more often "Uses language that is positive or uplifting" (V z 0.12, AUCROC =0.67). One possible explanation that Curie made many mistakes by misinterpreting undermining as label for negative sentiment rather than a logical relation between the claim and the perspective.
Comparing lyrics from different eras. Compared to lyrics from the 70s, those from the 80s more often "references violence or aggression" (V 0.06, AUCROC z 0.58)
Describing distribution shift. We compared the premises from the SNLI dataset and MNLI dataset, and the former "involves physical activity, such as walking, playing, climbing, or biking" (V Rs 0.13, AUC-ROC 2o0.64). One possible explanation is that SNLI is based on image captions.
Comparing discussion topics between bots and human users. We compared the topical differences between tweets identified written by bots vs. human users on Twitter, and our system finds that the bots more often "contains keywords related to business, finance or trading" (V ~ 0.08, AUC-ROC 0.61). One possible explanation that bots are frequently used to generate finance-related scams.
Identifying temporal differences news headlines. We compared headlines published by ABC news across different years. Compared 2014, headlines from 2010 "mention disasters and crimes, such as plane accidents and assaulis" more often (V ss 0.03, AUCROC Rs 0.53). Compared to year 2019, year 2020 more often "discusses coronavirus-related topics" (V RJ 0.21, AUCROC 0.65).
Describing text clusters. We present two example descriptions for text clusters. One from Wikipedia: "references pop culture, such as movies. books, and television shows." (V zo 0.21, AUC-ROC se 0.73); one from PoctryFoundation.com "uses vivid imagery and metaphors to convey a feeling" (V x 0.09, AUC-ROC 2..55).
19 Limitations and Future Work
We still face many challenges in building broadly useful system. We describe technical challenges that machine learning researchers can tackle in Appendix 19.1 require domain experts in Appendix 19.2
19. and organizational challenges that
24

---

19.1 Engineering Challenges
Hypotheses about the corpora might not be appropriate predicates on individual samples. When comparing highly rated definitions from UrbanDictionary com to others, our system generates the hypothesis that the former "is more likely to include slang colloquial terms." This is statement about a collection of text samples, but the validator requires the hypothesis h to be a predicate on individual text samples x. To address this, we used GPT-3 automatically remove comparatives from the hypotheses, e.g. rewriting the hypothesis above to "include slang or colloquial terms."
However, some versions of this problem were harder to remove. For example, when comparing reviews from American Airlines (AA) flights and Delta Airlines to understand which aspects each airline are doing better/worse, the proposer generated the hypothesis "mentions American Airlines' staff being unfriendly and unhelpful". Interpreted literally, this hypothesis can only be true on the corpus of AA reviews, since it presupposes the review to be about AA. The correct predicate for use on individual samples should instead be "mentions staff being unfriendly and unhelpful" (without the words "American Airlines""). Therefore, future systems should explicitly convert corpus-level statements to their corresponding correct predicates, and the metrics should evaluate whether the validity of the predicates implies the corpus-level statements.
Beyond truth predicates. Our work requires the discovery to be a truth predicate that maps a text sample to truth value. However, scientific discoveries can be arbitrary natural language expressions: extending to more flexible expressions requires a significant redesign of our system and evaluation framework. Some more feasible near-term extensions include 1) allowing natural language expressions that map from text samples to real values, e.g., "how polite the sentence is compared to other samples from the corpora" or 2) using additional logical forms to combine individual truth predicates; e.g., learn a shallow and interpretable decision tree where each split point is a natural language predicate.
Beyond corpus-level differences. Our work focuses on describing corpus-level differences and validates a discovery by comparing how often it is true on each corpus. Future work can consider other ways to validate a discovery: for example, suppose each text sample is associated with a continuous target variable, we can validate whether a discovery is more likely true if the target variable is large.
Investigating sensitivity towards prompt format. In this paper we hand-crafted the prompt for the proposer and manually annotated the exploration goals on our own for OPEND5. However, due to budget limitation, we have not investigated how sensitive is our D5 system towards prompt formatting and paraphrasing, or whether the performance could have been improved with better prompts. Future works can investigate more in this research direction.
Clarifying a discovery. Some discoveries seem to have clear meanings on the surface, but they become ambiguous when judge them individual text samples. For example, judging whether text sample h = "mentions people" seems like an unambiguous task a priori; however, it is unclear whether it true on the sample "I woke up this morning." ", since the "people" in h a plural form, while x only mentions one person "/". Future work can use a language model to automatically clarify the meaning of a hypothesis and make it more specific, e.g., rewrite "mentions one or more humans."
judging
Correlation & causation. Like other tools that rely correlations to analyze patterns in data (e.g.. linear regression), our system cannot establish causal relations either. For example, when comparing self-reported happy moments from females and males, even if the former corpus has more samples that "mention children and family", it does not necessarily imply family plays a more important role in inter-personal relations for females; an alternative hypothesis is that females might mention any other people more often than males, hence leading to the observation that they mention family more often. Future work can use language models propose what control hypothesis test.
Decreasing the cost of validation. As alluded to in Section 3 estimating V is extremely expensive as it requires lot of human labor. Future work can consider an importance sampling procedure that uses f as a proposer to improve the sample efficiency of estimating V.
Training a better proposer. We developed self-supervised learning algorithm to propose more valid hypotheses. However, it does not take into account the meaningfulness metric, and it is unclear how to manage its trade-offs with validity they exist. We look forward to future works that can train a better proposer with minimal supervision as possible.
25

---

Combining Meaningfulness and Validity Metrics. To simplify evaluation, we assumed meaningful- ness to be independent of the magnitude validity V. Such an assumption allows us to directly evaluate hypotheses that are not necessarily valid but is also limiting for evaluating the final discoveries: for example, for that 2008 "discuss economy" more often than 2007, it would be way more significant V = 0.99 compared to V = 0.0000001. Future works can propose better metrics that do not assume that validity and meaningfulness are independent.
Extending to Non-English Language OPENDS is currently annotated with English goals and most of the corpora are in English. Future work can consider extending this to other languages.
19.2 Organizational Challenges
As discussed in PPoaany et al. (2000), it requires implicit community norms rather than explicit deductive logic to decide what counts as good research results; to guide our system to produce truly important discoveries, our system needs feedback from researchers who work in the domain of interest. However, except for machine learning, the authors do not have research expertise in most of the domains listed in Figure 2 concrete directions below.
We look forward to future contributions from other domains and list
What problems to solve? We generated the problems in OPENDS by reading relevant papers and guessing what domain experts might care about. However, our guesses can be inaccurate. Future works can directly gather problems from domain experts to reflect the actual usage of our system.
How to interpret discovery? We asked for Turker's judgment compute T(h, æ)) However, many hypotheses require expert knowledge to interpret properly. For example, only law experts can reliably judge whether a contract x satisfies the predicate h "contains a license grant that is irrevocable." Domain experts are needed to evaluate the validity of discovery and supervise the validator.
What discoveries are meaningful? Our work developed the evaluation instructions approximately evaluate what hypotheses are meaningful. However, just as no one can become an outstanding peer reviewer simply by reading the review guideline, we do not consider it feasible to provide a gold evaluation simply by reading our instructions. Whether a discovery is meaningful depends heavily on implicit community norms, and we hope domain experts can provide better evaluation and training signals for our system.
20 Self-Supervised Learning with Open-Ended Problems: A Proof of Concept
Since the problems in OPEND5 are open-ended, our system could potentially produce discoveries with higher validity scores than our current system. Therefore, design a self-supervised learning algorithm to improve an LM's ability to propose more valid hypotheses, using the principle that is easier to validate discovery than to generate one.
Algorithm. Suppose we are given a set of problems for training and an initial language model Mint- Our goal is to automatically generate a set of prompt-complltton pairs to fine-tune Minit that it can propose hypotheses that are more valid. To generate a prompr, we randomly sample a problem and create proposer prompt following the procedure in Section 4.1 To generate the desired completion given a prompt, we sample multiple hypotheses from Mimnt approximate their V' score on the samples in the proposer prompt with the same language model Mini (Section 4.2) and select the highest scoring hypothesis. Finally, we use the prompt-completion pairs to fine-tune Minit-
However, since we cannot fine-tune instruction-tune GPT-3, we can only experiment with Flan-T5 (Chung et al. 2022), an open-sourced instruction tuned model that might only work well for casier "mini-problems" " As proof of concept, we tested our algorithms for describing groups of four samples, where each group comes from a text cluster. As an overly simplified example, we will give the LM the prompt "Group A: I. dog 2. cat 3. pig 4. cow. Group B: 1. phone 2. laptop 3. desk 4. cup" as an input and the LM can output "mentions an animal" as hypothesis.
Data. We created 33 corpora by merging all corpora in OPENDS with the same domain, and automatically generated 4503 text clusters using RoBERTa embeddings (Aharoni & Goldberg 2020). We focused on clustering because it can automatically generate large amount of semantically coherent groups of samples. To create pair of four samples, we randomly sampled a corpus, sampled two clusters within that corpus, and took four random samples from each cluster. To test
26

---

cross-corpus generalization, we reserved 28 of the 33 corpora to create mini tiiiieeoolems for evaluation, using the rest for training. We used Flan-T5 (Chung et al. 2022) as Minit and sampled hypotheses with a temperature of 0.8. For training, we sampled 30,000 mini-problems and selected the best of eight hypotheses generated by Timi as the target completion; for evaluation, we sampled 200 mini-problems to calculate V with Turkers and 1500 mini-problent to calculate V' automatically.
Results. We evaluated randomly sampled hypotheses from the language model before and after self-supervised training. The automated "self-evaluation" validity score V' improves substantially from 0.22 to 0.37, and the "true" validity score V according to Turker evaluation improves from 0.07 to 0.10, with p-value of 0.02. This result provides preliminary evidence that our algorithm (or similar variants) could be applied we expect future validators to simulate human judgments better, hence decreasing the approximated gap of improvement between V and V
large set of problems improve the validity of the hypotheses:
21 Comparing D5 to Naïve Bayes
We qualitatively compare the discovery generated by our D5 system to the top-5 unigram features extracted by Naive Bayes, a traditional exploratory analysis method. The Naive Bayes method is effective when the target difference can be saliently reflected by individual words. For example, "yo" implies a rap genre, "dic" implies a language of Deutsch, and ["rank", "higher", "univeristy") hints at the topic of "college ranking changes". Additionally, compared to black-box neural networks, such a method is fully interpretable.
In comparison, D5 can directly generate a semantically coherent description for the target difference, saving users' time to guess the underlying correlation by inspecting the top unigram features. Addi- tionally, it can capture differences that are hard to detect at a word level; for example, "the genre of biblical scripture" is mainly reflected in its sentence structure rather than individual words. Finally, D5 only describes goal-related differences, while Naive Bayes picks up on any discriminative feature; for example, when identifying the topical differences between a English and a Deutsch corpus, Naive Bayes fails catastrophically and only picks up common determiners such as "the" or "die" instead of topic words, since they are the most useful feature at telling which sample comes from which corpus. Given the respective strength of D5 and traditional exploratory methods, we envision D5 to serve as a complementary method to traditional methods.
22 Annotation Interface to Collect Human-Generated Hypotheses
(This section describes an interesting research direction we did not have time fully pursue.)
Task. To fine-tune the language model to propose better hypotheses and perform validation more accurately, we also designed an interface to collect human annotations carlier in the project. In this annotation task, the annotators see five text samples from each of the two corpora; they then write one or many natural language predicate(s) that describe how samples from the two groups and choose which text samples satisfy each predicate the annotator has written. Since it is challenging for humans to identify systematic differences between even groups of five sentences, we made the task easier for them by
different
we chose the representative samples from each corpus form the two groups of samples, similar to the process in Section [5] and
we highlighted subspan of the text samples that are informative for how the two corpora differ. For example, if Corpus A is sports related while Corpus B is entertainment related, we hope to highlight sports-related words like "basketball". To automatically identify the fine-tuned RoBERTa to classify whether sample comes from
text spans to highlight, Corpus A and Corpus B, used the SHAP library to calculate how much each text span influences the classifier's decision, and highlighted the text spans based on the influence.
A screenshot of the annotation interface can be seen in Figure 6
Preliminary Results We performed initial experiments on text clusters formed on the wikitext-2 dataset (Merity et al.] (016). We asked the authors to write hypotheses for 30-50 samples and then compare the results with GPT-3 generated hypotheses. We found that human annotators were able to
27

---

1) hypotheses
2) Select most representative samples a written hypothesis
3) Commit hypotheses and show SHAP highlights
SHAP highlight view
4) Write any additional hypotheses after seeing highlights
Figure 6: A detailed screenshot of our annotation interface.
write 2-4 valid hypotheses per pair of text groups, while GPT-3 text-davinci-003 was able to generate 4-6. Out of the valid generated hypotheses, approximately third were variations another valid hypothesis. The number of times humans were able to write hypothesis that GPT-3 was unable to generate was around third of the samples, while GPT-3 was able to generate novel hypothesis humans have not thought about before in nearly every single text corpora. Given that GPT-3 is close our author's 's ability to write hypotheses, we estimated that we would not be able fine-tune T5 propose better hypotheses with human annotations, and hence gave up on this research direction.
23 Datasets
Many of our datasets come from the following sources: the Computational Models Social Mcaning class from Columbia Universityhnttp://owww. columbia. www.e//aaaaaaiiiiieeeaeaa
the ACL Anthology /atps::/accantthoogge org, and Kaggle datasets with an NLP tag. https
//www //www.kgght kaggle. com
abc-headlines. We collect headlines published by ABC news, an American news company from
Kulkarni (2018). ABC. headlines are directly downloaded from Harvard Dataverse. The year is extracted from the publication date field. Samples are constructed from the headline text. The data downloadable from http:: //doi. org/ 10. 7910/D9M/11BBCC SYBGZL with license CCO 1.0.
28

---

ad-transeripts. We collect ad scripts from variety of industries from Hartman 2019). Ad transcripts are directly downloaded from Kaggle. The top eight industries by frequency are selected. Newlines are replaced with spaces. The dataset is downlondable from https: www. kaggle com/ datasets seenssttskeviinee www.ieartaaaaaanaaa..aaaa wattaaartman/ddvveiiseeeeeeee advertissmeettt f with license CCO Public Domain.
admin-statements. We collect statements of administration policy from American pres- idents from Progress (2022). Administration statements are extracted from a collec- tion hosted on GitHub. Extraneous symbols are removed and samples are split by paragraph. The dataset is downloadable from https: com/unitedstates) statements of administration www.ooo-ttlccctttttttmeetttee of- pewniniistratiinnpolicc and origin files have a Creative Commons Attribution 3.0 License.
dataset released by the Allen Institute for AT from Mishra et al. (2022). Natural instruction tasks are directly downloaded without modification. The dataset is released under an Apache-2.0 license.
aiz-naturralinstruutiio We collect Baaming-from-instee
airline-reviews. We collect reviews of airlines collected from the review website Skytrax. Airline reviews for airlines, airports, and seats are downloaded from public GitHub repository. Names of aircraft, airlines, countries, and traveler types are standardized. Ratings of 1, 4, or 5 on a scale of 5, and 1, 5, 8, or 10 scale of 10 kept. This dataset can be downloaded via https: thub com/
aita. We collect posts the "Am I The Asshole" Subreddit, online forum people ask others whether they were in the wrong from O'Brien| 2020). Posts from reAslTheesshole are downloaded from a praw scrape of Reddit. Topic areas are chosen based on common themes in posts and coarsely defined based on manual keywords. Each post can belong to multiple topic areas. The dataset can be downloaded at hhttps://dii. org/ /0. 1.055 5281/zenodo 3677563
all-the-news. We collect news articles collected from various outlets between 2015 and 2017 from Thompson (2019). News articles are downloaded directly from the Components website. The titles are used as text samples. The The dataset can be downloaded at hhtttts:/ http:::///mmoomeettte onw.one//ee..ee datasets/ all- thews-eneww-- articles- dataset).
amazon-reviews. We collect Amazon reviews collected from various product categories from Ni et al. (2019). Amazon reviews are downloaded from a 2018 crawl of the website. The first 100,000 review texts are treated as the text sample. The dataset can be downloaded at hhtps //nijianma .acaamoo..gtuuut github. io/ camaazo/inndes html
armenian-jobs. We collect job postings in Armenia from (dacityy (2017). The Armenian job postings dataset downloaded from a snapshot on GitHub. Different IT jobs are manually coded and time intervals are defined in order balance sample availability. The dataset be downloaded at https://www http//wwww..g.....e/ com/ datasets/udacian arrenianm oone-ennoneeeoee jol-innn-joo--osinnga-- postings
boolq. We collect reading comprehension dataset of yes/no questions from Clark et al. (019))
Boolean questions are downloaded directly as is. The dataset can be downloaded at https://
github.comm. wotom//gogle-eeee reseererrrsecccch---ee waw.earrra-aaaaattttooooee---aae -owetggeesnnnnnssseiooonn with license CC-SA-3.0.
clickbat-headdlinss We collect headlines across time from the Examiner, clickbait news site from Kulkarni (2020a). The Examiner headlines are directly downloaded from Kaggle. The year is ex- tracted from the publication date field. Samples are constructed from the headline text. The dataset can be downloaded at /https://wit /wwww/www.. kaggle. cww coomddatsets/////oeooo ttterok///h//h//// examine- thee- eiteeexxminerr with license CCO: public domain.
convincing-arguments We collect arguments on a variety of topics annotated for convincingness from Habernal & Gurevych| (2066). Annotated arguments are downloaded from the GitHub repository. Arguments are sorted by rank. The bottom 400 are treated as "unconvincing", the top 200 are treated as "convincing"", and the next 200 are treated as "somewhat convincing." " The dataset can be downloaded at bthttps://githhi com Wwt.Laa/aa12011-cooniiiii htwp:/00--.....i.iiiiiaaaaaaaaaa with license CC-BY 4.0.
craigstist-negooiattons We collect dialogue from Craigslist negotiations, an online seller platform from He et al.] (2018). Craigslist negotiations are downloaded from Huggingface. Sequences which contained a "quit" intention or "reject" intention categorized as failures; those which contained an "accept" intention are categorized as successes. The mid mid-price is defined as the mean
29

---

price of the items sold. Within each category, the items are sorted by mid-price The top half is treated as high-price and the bottom half is treated as low-price. This dataset can be downloaded at Deapppeeeeeeaaeeeummmmm with MIT license.
debate. We collect evidence compiled for American competitive policy debate, published online by debate camps from Roush & Balaji (2020). The train split is downloaded from Huggingface. For each sample, we use the abstract as the text. Arguments are categorized by type, debate camp of origin, and topic/specific argument. For topics, we use domain knowledge to list relevant keywords for each topic and include any sample with file name that includes any keyword. A single sample can belong to multiple topics. This dataset can be downloaded at datasets httaao/s/illlssscccpppopppeaaaae
with MIT license.
dice-jobs. We collect American technology job postings on dice.com from PromptCloud (2017). Job postings are downloaded from Kaggle. Posts from the six most popular companies are categorized by company. We remove miscellaneous characters and blank descriptions. We additionally apply our splitting procedure reduce description length. This dataset can be downloaded at https:// / www www.kaggle.. com httpdddataatttttppataiieeeeeeeee wwwpuud/////uuutuueaaeee wws.seehoooogy---eee jobsy-jobs-aae on diccicccco under CC BY-SA 4.0.
We collect dialogue from games of Diplomacy, which involves deception from Peskov et al. (2020). Diplomacy dialogues are downloaded from GitHub (all splits). The data are ASCII encoded and newlines are removed. Each message and label is treated as a sample. This un-
diplomacy-
dataset can be downloaded at htthtps: hattps://huggiingffaset co/ data//tssssst//eeeeaeaeaaaaeee http:/dpppooaadeeeeeeeeee der unknown license.
echr-decisions. We collect facts of cases heard before the European Court of Human Rights from Chalkidis et al.] (2019). Decisions are downloaded from public archive. A random sample of 500 decisions is selected from the files. The samples with any violated articles are categorized as "violation," " while the rest are categorized as "no violation." This dataset can be downloaded at https://papersvithcod.e. com/ dataset/ under unknown license.
essay-scoring. We collect essays from students from ess (012)). Essays are downloaded from a GitHub repository. Only essays from set 5 are considered. Essays with a score of at least 3 are categorized as good essays, while essays with score less than 3 are bad essays. This dataset can be downloaded at https::/www/ kaggle. maaagle.com////// c/sppp/ssspppppeee aes under unknown license.
fake-news. We collect fake and legitimate news from Pérez- al. (0017). Fake news articles are downloaded from the author's website. Full articles are treated as text snippets. This dataset can be downloaded at httht:t///// http:://web.eee eecs. umi ch. edu/"mihalcen" ceahaalllllleeee hww////wwwooooo.ooooooooooomslsd cem.halllla//dmooaaa under CC-BY-4.0.
htme:FaeeNews
fome-speeches. We collect Federal Open Market Committee (FOMC) speeches from 1996- 2020, which describe Federal Reserve policy from Mish (2020). Fed speeches are down- loaded from Kaggle. The macro indicator data merged in the year and month. Full speech text is split by paragraph and categorized by speaker, year, and macroeconomic in- dicator. This dataset can be downloaded at https: http://wwwwww.gll.oeee cow //ddddattt/ttaaamm..... rederal-reserve- wowerross--ppeeeeeeeeeeeeeeeeeee
2020 under unknown license.
genius-lyrics. We collect lyrics collected from Genius.com com before 2020 from Lim & Benson 20021) Genius lyries are downloaded from Google Drive. The lyrics are merged with song metadata and treated as samples. We categorize lyrics by hand-sclecting popular artists, common genres, time peri- ods, and view counts (over 1M views is high, 500k- IM is medium). This dataset can be downloaded at htttts://www.cst
edu/ watpearr////eee/ae//aaaaaa/aaaea http///abbbaaaaaaaeeeeeea
expertise/ under unknown license.
happy-moments. We collect repfrrep happy moments and demographic characteristics from Asai et al. (2018). The HappyDB dataset is downloaded from the official GitHub repository. Demographic data is cleaned and merged into happy moments. Happy moment descriptions are treated as samples and are categorized by type of happy moment, country of origin, and other demographic features. This dataset can be downloaded at (tttttt://// /ttte::///tttttt wwt..o/og//omaaaaaaea aawpopppp/a//a under unknown license.
huff- huff-post-hees post-headlines. We collect headlines from the news outlet Huffington Post from Misra & Arora (2019) and Misra & Grover 2021). Huffington Post headlines are downloaded from Kaggle. The
30

---

short description of each article is treated as a sample and tokenized at the sentence level. This dataset can be downloaded at htttps:/rishaabhiiis. github..hhu.... bocuuuiioopuuuu wwtppuliiattons////////o under CC-BY-4.0.
immigration-speect
We collect collect congressional and presidential speeches that mention immigration from 1880 to the present from Card et al. (2022). Immigration speeches downloaded from the replication package. The speech text is preprocessed to remove extraneous spaces. We engineer features corresponding to time periods, well-known speakers, other significant time periods, the racial group under discussion, and the geographic area within the United States. This dataset can be downloaded at htttts///githut:
releasees
kickstarter. We collect names of startups on kickstarter.com from Mouillé (2017). We down- load 2018 crawl from Kickstarter from Kaggle. The project name treated as the text sample. This dataset can be downloaded at https: www. kaggle. com/ datasets/kemtt /amic (ecesaarttrr-pojeetttttelelttttd
201612..01111..... csv under CC BY-NC-SA 4.0.
microedit-humor. We collect funny sentences generated by making one-word edits to normal statements from Hossain et al.] (2019). The Microedit dataset is downloaded from the author's website. We make the relevant edit to each text sample and treat the edited text sample as the data point. We bin the mean annotator grade into 4 and denote each unfunny, neutral, funny, and very funny, respectively. This dataset can be downloaded at humi croedit)
con/dataset/
mnli. We collect a collection sentence pairs annotated with textual entailment information from range of genres from Williams et al. (017). The MNLI corpus is downloaded from the official website. We treat the premise and hypothesis as text samples. This dataset can be downloaded from https://cims nyu. stth://sssoowam////e// htt//mmulnnniiiii tinli/ most of which are under the OANC license.
monster-jobs. We collect American job postings on monster.com. Jobs on Monster.com are down- loaded from Kaggle. Job descriptions are treated as samples and split at the paragraph and sentence level. We keep and categorize jobs from seventeen large cities. This dataset can be downloaded from https://wwst hwwp://wwwwkggg......./ com///leccmm//ee dataasts//rompttllouui oudd:/uuuuuuuuuuuu jus-jobs-- on ca-monstercom under CC BY-SA 4.0.
movie-tmdb. We collect movie plot summaries from TMDB from Kaggle (018). TMDB movie overviews are downloaded from Kaggle. We keep only English movies and bin popularity by deciles. The top decile is considered "hits," the 70-80th percentiles are considered "average," and the 30-40th percentiles are considered "bad." This dataset can be downloaded from https:///wwtt https///www.kgllleee matiiee-etaaaa221111
com/datasets//tiin teaeettsttmmd//mdmmbbbbbe
movie-wiki.
We collect movie plot summaries collected from Wikipedia from Robischon (011)).
Wikipedia movie summaries are downloaded from Kaggle. This dataset can be downloaded from https://www http://wwww.ggg..eeee kaggle. cwrpagggee...mddeeee com/datasets. jttp::/sssooooiiooooooooooooooe wwwpsshhoowwiiiieiee http://iiiiiioooiiiiiiaeee SA 4.0.
under CC BY-
We collect news headlines posted on social media platforms from Moniz & Torgo (2018). Headlines are downloaded from reproduction package. The headline and title text are cleaned, and the title is treated as the text sample. The 100 most positive and nega- tive or popular and unpopular articles on each topic are used as distributions. This dataset can be downloaded from htttpp://archive.iice
news-popularity.
We collect training examples from various natural language inference (NLI) datasets from Liu al. (2022). NLI benchmarks are downloaded from public collection on Google Drive. We examine the premise and hypothesis separately as samples. This dataset can be downloaded from httpt:tpts///ggi
nli-benchmarks.
com/ aaoommaaie www.waffleeswaaaa.mm...
npt-conferences. We collect Kon-Proliferation Nuclear Weapons (NPT) conference transcripts from Barnum & Lo (2020). NPT conference notes are extracted from the accompanying replication package. Text is split by paragraph, and only paragraphs longer than 50 characters are preserved. Text split into three time ranges: pre-2008, 2012-2012, and p0st-2012. This dataset can be downloaded sagepub. com hamp//di//full htt//diiffuuulol 10. 1 11770002333329966222
from httts::/
open-deception. We collect arbitrary lies and truths from any domain generated by crowdworkers from Seree--Rosa & Mihalccal (2015). Open domain lies are downloaded from the public dataset
31

---

and lie texts are split into lies and truths. This dataset can be downloaded from https::/wbtbbee.. eecs umich.edu/"miinn. edt/"mihalcca//mmoe dww.ea/ddomoodasssoooomoo wwwhhh#mmm@@mi Seeml/0@eeeeeptii.o.
open-review. We collect submissions to ICLR, a machine learning conference from 2018 to 2021. Open review abstracts are accessed via the openreview API. We query for abstracts from the 2018- 2021 ICLR blind submissions. Abstracts are classified based on rating: >= 7 ("great"), 5-6 ("good"), and <= ("bad"".. This dataset can be downloaded from woamm/ttss////em httpps://openrevie net/
parenting-subreddit. We collect posts from various parenting-related subreddits, which are text- based forums the site Reddit from et al. (202I). Posts from various subreddits are downloaded from the paper's 's GitHub repository. We clean the text and split the posts according to the topic(s) each post is tagged with. This dataset can be downloaded from httts:///itthbb www.sooToSSS...... SSLL-NNM NLP/ Parenting OnlineUsage
poetry. We collect poems from PoetryyFoundation.omm from Bramheeha (2019). Poems are downloaded from 2019 scrape of the PoetryFoundation website from Kaggle. The text is cleaned and split according to subject tags and authorship. This dataset can be downloaded from http::ttp/// : //tw://www// http//wwwwggge.cccmo
under GNU
Affero General Public License.
political-ads. We collect political ads observed by Facebook users from pol (202)) Ads are downloaded from the Ad Observer website, which maintains an aggregate of all collected ads. We extract targeting metadata from the targeting field and define splits according to age, gender, location, interests, time, and political lean. This dataset can be downloaded from https: fttt::://adobsevvve org/ad-database.
qqp. We collect questions from Quora.com from Quora 2017).
rate-my-prof. We collect reviews of lecturers from MiteMyyrrofssor.com from He 2020). We MateMMProfeesor
reviews from an online repo. We clean the text and guess the gender of the reviewed lecturer from the first name using the gender-guesser package. Due to data availability, we consider only male and female names. To improve the quality of the classification, remove any posts which pronouns from the opposing sex (e.g. "him"). This dataset can be downloaded from ht t/st:p//aa/...... mendeley. com com
download a sample
under CC BY 4.0.
radiology-diagnoss. We collect impressions and medical histories of radiology patients from Pestian (ct al. (0077). Radiology diagnoses are downloaded from a GitHub copy of the original task dataset. We parse the metadata to retrieve the diagnostic code, decision type, impression, and patient history. Referencing the associated ICD codes, we convert codes to colloquial diagnoses (e.g. 786.2 denotes cough). We treat the histories and impressions as samples and split them according to diagnosis and level of consensus.
reddit- We collect jokes posted on the Reddit forum r/Jokes, message board for sharing jokes from Weller & Seppil (2020). Jokes are downloaded from the dev and test splits of the dataset. We clean the text and split the dataset according to whether they are labeled as funny. This dataset under Reddit License and Terms of Service, and users must follow the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.
can be downloaded from https: //g tiggtb....... com/ grrionw/rookesie
reddit-stress. We collect stress-related posts on Reddit from Turcan & McKeown (011)). We split the post text based on which subreddit they are posted on (related to PTSD, anxiety. or stress generally). Reddit posts are downloaded from https: github. com/ com/gill gilggillli an850413 Insight_ Strees_Analysiss, and we recommend following the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.
reuters-authorship We collect articles from various Reuters authors from Liu 2011 The articles are split according to the author. Reuters articles are downloaded from the UCI repository https
//archive. ics wws.uci......................... www.wu.i.u........... http////aa/e///a/aeeaaaaaaaaeee BeeaaaaseeeeReeeeeeeeeeeeee _5__50
We generated several riddles. The 3000 most common English words are manually copied from website. Words with between 5 and 8 characters are kept. We create two popular riddles. First, we split words based on whether they have a duplicate character. We exclude any words with multiple "doubles" more than 2 of any character. Second, split words based on whether they have the letter T.
riddles.
32

---

scotus-cases. We collect facts from cases heard by the Supreme Court of the United States (SCOTUS) from Alali et al. (02I]. Supreme Court cases are downloaded from a GitHub repository. We identify state/federal parties by manually defining keywords. We split based on the winning party, the identity of each party, and the type of decision. We then define several time periods and relevant political eras and split decisions accordingly. Finally, we split according to the ruling's 's policy area and how it changes over time. The dataset can be downloaded from httpp:://paperrsitthhhee wwt.tttwwwwoooooooeeeeoooottttee com/paper/ www.bbeennnccaaatceeeeeee
supr court under CC-BY-SA.
shorttansserr-sorring We collect short answers from students from sho (2013). Short answers are downloaded from a GitHub mirror of the dataset. We consider only responses to essay set 1. The two
scores are averaged and binned into good (>= 2.5), medium (1.5-2.5), and bad (<1.5). The dataset can be downloaded from htttps///wwt kaggle. com/ C/ /saap-pps00 sas
snli. We collect collection of sentence pairs annotated with textual entailment information from images from Bowman et al. (2015) The dataset can be downloaded from https: //npp.///...//p stanf ord edu/projeect//nnli/ under CC BY-SA 4.0.
squad-v2. We collect reading comprehension questions crowdsourced from Wikipedia articles from Rajpurkar al. (2018). The dataset can be downloaded from https://rajjuraat wwapinaaa.gggtuutt github..h....... io/ exp under CC BY-SA 4.0.
stock-news. We collect top news headlines on Reddit, an online message board from Sun 2017). Headlines are downloaded from a GitHub mirror. We clean the text and divide the samples based on whether the DOW rose or fell that day. The dataset can be downloaded from htttt::/tps//t thub. com/ ShravanChi intha/ Stock Market- prrs--preiccc cenddictionuuuin
daily- news headlinedee under Reddit License and Terms of Service and users must follow the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.
suicide-notes. We collect posts from Watch and r/depression, two forums on Reddit from (202T). The post title and body are combined form the text samples. Samples are split based on whether they were posted in a nelatee Subreddit. The dataset can be downloaded from a github: https://gi thub. com con/hesamuell /ttp//ggogeeeeeeeeeeooeeeeeeeeee _worldd under Reddit License and Terms of Service, and users must follow the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.
times-india- headanns. We collect headlines from Times of India news from Kulkarni (022)) Headlines are downloaded from Dataverse mirror. We use the first 1000 headlines in each year as samples. The dataset can be downloaded from htttps://www http://wwwwkgglll. kaggle. com dwwaaats//tteeeeeeeeeeee india-headl ines- -ww.eadd--ne-----------ae -awa-dataset under CCO Public Domain.
trial-deception. We collect testimonies from witnesses in real trials from Pérez- et al. (0055).
Trial testimonies are downloaded from the author's website. The testimonies divided based on whether they are considered truthful. The dataset can be downloaded from https: //t::///eee //webwww..... eecs html #RealLiieeeeeettiii
un-debates. We collect speeches from debates at the United Nations from Baturo et al. (017)) Debate transcripts are downloaded from the Dataverse reproduction package. Samples are divided based on the country and year of the snippet. First, we isolate samples from Russia, China, and the United States and specify 3 time periods of interest. Next, we divide all samples by the decade. Finally, we create distributions for 19 countries of interest. The dataset can be downloaded from https://doi. org/ 10.
JXBBY under CCO 1.0.
unbeaalthy-conversstions We collect expert-annotatee unhealthy conversations from Price al. (2020). Conversation transcripts are downloaded from the official GitHub repository. For each anno- tated attribute, we split the dataset based on whether that form of unhealthy conversation is present in the sample. The dataset can be downloaded from https: tttpbbbttttttt/t////////ggeeb.tt www.tttw..oooo.eeeeooooeooeeteet cone/reatveeaiii........ unbeallthy-conveeaatii
under CC BY-NC-SA 4.0.
urban-dictionary. We collect definitions from ErbaaDictionary.com a crowdsourced English dictionary from Kulkarn (2020b). Urban Dictionary entries are downloaded from Kaggle. Definitions are split into groups representing the top 1, 5, and 10 percent of definitions ranked by both upvotes and downvotes; we sample 10,000 from each and create a control distribution by randomly sampling 10,000 definitions from all entries. The dataset can be downloaded from httpp://wwwt http:://www.kagee kaggle com/ wom/therohk/iee wuraannuuura--aaee www.iiiaayywworrrecoo dataset under CCO Public Domain.
33

---

wikitext. We collect text snippets from Wikipedia from Merity et al. (2016). The Wikipedia snippets loaded from HuggingFace. We remove any samples that are empty or start with '=' (which represent headings); samples are tokenized at the sentence level and used for clustering. The dataset can be downloaded from https::/hhugiiiggaaaee co/ /wwtatttt/wwieteeettttteeeeeeeee under CC BY-SA 3.0.
yc-startups. We collect descriptions of companies that were part of the Combinator startup incubator from Bhalotia (2022). YCombinator company descriptions are downloaded from 2022 scrape on GitHub. Only companies with long descriptions are preserved. Companies are split according founder characteristics, year, "top company" designation, operating status, and loca- tion. The dataset can be downloaded from https: www. kaggle com/ datasets/ beeaaaeeteebbbbeeer.... benhamner y cycombinator- companies.
34


Reviews:
[{"review_id": "NajN3lhhhW", "reviewer": "Reviewer_cy6L", "summary": "This work formulates a new task called D5, proposes a language model (LM) based approach to generate desired outputs in the context of this proposed task, and contributes a new dataset of open-ended problems to highlight how their LM-based approach can facilitate exploration of differences between two large English text corpora, with respect to a stated goal. The goals (for exploring differences) are stated in natural language, and the output 'discovery' of the differences is also generated in natural language. \n\nThe work shows how discoveries generated by their D5 system can be evaluated on both a synthetic diagnostic dataset (where ground truth differences are known and recovery of these differences can be directly tested), and on the open-ended OpenD5 dataset. The evaluation in the latter case depends on one somewhat objective criterion of validity, and then relevance to the user-stated discovery goal is evaluated manually using subjective human assessments, where humans are either the authors themselves or crowd workers (not domain experts). This work as a whole seeks to demonstrate the potential of LMs to facilitate human or subject matter exploration of hypotheses for differences in two corpora, and these discovered hypotheses are meaningful towards a user-stated goal. ", "questions": "1. Is it safe to say that at least for the paper as it stands, “discovering unknowns” is not something that has been evaluated or judged? Is there a robust plan to enable this system to reliably help domain experts discover true unknowns? \n\n2. What can we expect about the robustness of the system to variations in the phrasing and framing of the stated exploration goal by different non-author users (see the first point in the weaknesses section)? Also, from an end-user as well as a research perspective, are any experiments conducted to test what must be included in the exploration goal or other non-templatic parts of the prompt, what should be constraints on phrasing, if the order in which various things (about the corpus, goal, etc.) things matters? Can we expect that different phrasings of the same underlying exploration goal will affect the validity or relevance of the model output discovery?\n\n3. How was the prompt template designed, and do we know if a different template could yield better results?\n\n4. As noted above in the third point in the weaknesses section, even within the scope of identified limitations, it does not seem to be easily identifiable for an end user when correlation might get framed as causation or when bias reinforcement might happen, and therefore, can the current system be used with those potent limitations and harms as a possibility? In other words, given these known limitations and harms, can we currently handle them effectively, or if not, should D5 usage be held until those limitations are overcome or at least mitigated effectively?\n\n5. Section 6.1 is an important assessment of OpenD5, however a lot of details do not seem to be provided which hampers the review but also the clarity a reader needs. The manual selection process of the 21 discoveries is not clear: how was the ‘representativeness’ of potential use cases judged? Also, how were criteria 3 and 4 actually assessed (lines 254-255)? Apart from this section, some other key details and justification behind certain decisions are not provided which are also important, these are noted below.\n\n\nSuggestions (acknowledging possible space limitations for author response, questions above are separated from some questions below – addressing the below questions can improve the paper): \n\n- With respect to lines 49-50, goal-relevance being higher is expected (and good that it does happen) since one model has goal-conditioning and the other was not designed to handle stated goals; but it is interesting that validity or correctness is higher too – analysis of what invalid differences are now not generated in the proposed model or what valid differences were missed in baseline model would be interesting (unless already done, in which case a clearer pointer to this analysis should be included). \n\n- For the synthetic diagnostic test, it seems like the testing would benefit from either an additional or alternative approach to a synthetic test bed where the two corpora are not generated by language models (since systems being tested in this work are also language models, which calls into the question the strength of the proposed synthetic benchmark). Given that the properties for goal-relevant and distractor dimensions are around topic, genre, and language, a straightforward synthetic benchmark is using parallel corpora and then running actual topic models on them to generate and divide documents/texts by topic using the outputs of topic models: in this way, the corpora being compared remain real-world human-authored collections while the distraction vs goal-relevant test can still be conducted. OpenD5, while being a valuable real-world dataset contribution relevant to the task, does not test this distractor element present in SynD5. So something like the proposed alternative diagnostic with a) corpora authored by humans, and b) a known distractor element should provide a robust synthetic testbed arguably necessary to gain insight into the proposed framework for using LLMs.\n\n- Line 98: cannot just stay Claude-v1.3 – what actually is this, exactly, needs to be specified.\n\n- What courses on computational social science (line 110-111) were used? A list should ideally be provided in the appendix.\n\n- Line 112: what is the inter-annotator agreement for the exploration goals?\n\n- Line 179: why 60?\n\n- For the hypothesis validator (lines 183-184), how good are language models in terms of simulating Turkers' validity judgement? Is this a valid proxy for this hypothesis validation step?\n\n- Especially for synd5, it seems like there was an opportunity to test with a variety of open source open access models which is especially important since we should operate under the assumption that many users will not have accept to GPT-4. \n\n- Line 238: It actually is important that non-authors rate relevance in order to get a more robust assessment of relevant improvement when using goals on OpenD5.\n\n- Typo in line 294: the closing quotation marker should not be the beginning of a new line.\n\n- Line 329 in related work section: topic models deserve a mention since they are used for the discovery of human interpretable latent concepts, and in fact relevant to one of the experiments conducted in this very work where authors use the topic as one of the dimensions varying between two input corpora samples (see the suggestion above around a more robust synthetic test). \n", "limitations": "While some key limitations are already acknowledged in this work, there does not seem to be a robust discussion on managing and mitigating these in real-world usage of the system. With things like correlation being potentially framed or interpreted as causality, for example, when can users expect such things to happen – without any markers or indications for a system output, it is hard not to be skeptical of any discovery generated by the proposed system. In other words, the key question that seems to be underexplored here: if it is not easily identifiable for an end user when correlation might get framed as causation or bias reinforcement is happening, should the system be used with those potent limitations and harms as a possibility, and given these known limitations and harms, if we do not know if they can be handled effectively or not, should D5 usage be held until those limitations are overcome or at least mitigated effectively? \n\nAcknowledgment is an important system, but adequately addressing limitations needs more work, and there are also at least two more really important broader considerations not discussed in this paper that need to be addressed: \n\n1. The work seems to operate in the English language alone, at least in terms of substantive testing and application in the paper itself. There is no discussion about theoretical generalizability to other languages and expectations around that, and at least there should be an acknowledgment of the lack of empirical generalizability of findings about the system to other languages. The languages in which the synthetic and contributed real-world dataset operates should be made clear by the end of the introduction itself. \n\n2. Some important broader impacts/limitations are not being discussed and there should be more discussion in the paper about who gets to use the proposed system and potential double-use scenarios. For example, given that GPT-4 leads to better performance than the open-source alternative, if using this system for effective discovery relies on OpenAI API access and compute resources discussed in Section 8, many scientists and other end-users will probably not be able to use this method of exploration effectively, advantaging some communities over others — leading to many ramifications such as expanding existing inequities in access and resources. How can open-sourced tools be leveraged here, and can smaller models requiring fewer resources be incorporated effectively? This unequal opportunity question also ties to the previous point about language -- doctors, for example, can be operating in different cultural contexts using different languages to make their notes and we do not know if their experiences be used effectively for hypothesis generation same as those operating with English. Low-resource languages are also a potential limitation and concern that warrants discussion.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is well written and is clear in its definition of the proposed task as well as the objectives of the work. The goals carry real-world significance, as a valid and reliable method to enable exploration of goal-relevant differences between two corpora can help scientists and domain experts in generating useful hypotheses and potentially help generate insights that would have otherwise been too laborious or just not that obvious via a manual assessment (especially when the two corpora are big in size). The work takes a concrete step towards these goals of enabling human exploration and discovery. While exploratory differences between text corpora have been studied in NLP before, adding a goal, expressed in natural language, to this process of discovering differences and enabling systems to take this goal into account is a novel modification to this kind of exploration task. The OpenD5 dataset is a valuable data contribution if released and documented properly. Proposed evaluation criteria and robustness checks also serve as a key framework for designing similar tasks and making sense of the goals of the proposed system and task. ", "weaknesses": "- While proposed criteria and robustness checks are valid and sensible, when the goal is facilitating human exploration and providing a system and framework to end-users, it is important that people other than the authors create the prompts or fill in the non-fixed portions of the prompt template because it is not clear if even slight differences in how the ‘open-ended’ parts of the prompt text (and its framing) could affect the generated hypotheses and discoveries. It is understandable that recruiting people to also generate prompts and assess discoveries for the same goal and input corpora but with prompts written by different people is difficult, especially if a well-powered human experiment is to be conducted. However, it is critical to study the effect of how the exploration goal is phrased — which includes paraphrases. While the validity and relevance to criteria are necessary and well-motivated eval criteria, arguably another really important criterion here (since the system is intended to help human exploration) is stability (which can map to intra-coder reliability): is system output robust to tiny, non-meaningful changes in the stated exploration goal? \nIn fact, this can be seen as an important component of system reliability – if we cannot expect consistently similar or the same discoveries for stated exploration goals or prompts that carry the same meaning and logically entail the same output, then the system is clearly not reliable for use by humans. Additionally, this robustness or stability is important to establish, given literature in NLP, for example, on adversarial robustness where minor perturbations in text (such as in text style which should be irrelevant to the task) could 'break' the NLP system, and more. See Qi et al (2021) for an example with text style transfer, Zhang et al (2020) for a relevant survey, and Wallace et al (2019) for some useful pointers on adversarial triggers in NLP. \nIn addition to this important robustness criterion, it is also generally important to study the effects and properties of the language of the exploration goal. The following questions can be used to create a more tested system of prompt design: What must be included? What are the constraints on phrasing? Does the order of specifying things matter? Is there a guide that should be followed when constructing the exploration goal for the most 'valid' and 'relevant' results? Do different phrasings of the same underlying exploration goal affect the validity or relevance of the model output discovery? \nEven for the template portions of the prompt (which are not meant to be modified), it is not clear how this template was constructed and what happens when these templates are changed. Theoretically, could a different template achieve better results? Without such an analysis, both the usage of the system and the generalizability of the research work are negatively affected. \n\n- One of the stated aims of this work and the proposed system is facilitating the discovery of unknowns or the generation of novel, previously unknown insights. However, including what has been presented in section 6.1, it is not clear at all if this has actually been established even using the OpenD5 dataset. With only the authors judging the discoveries and no subject matter experts involved, such a claim cannot be evaluated or it needs to be reframed and scoped out appropriately. Novel to the authors does not seem to represent a significant advance and does not seem to represent a step towards “discovering unknowns” stated in line 35. However, this is a critical potential contribution of this work and not taking concrete steps to establish this potential negatively affects the significance of the work. Also, in lines 284-285, the authors acknowledge that the signal about validity is unreliable, and since validity is critical when it comes to hypotheses generation and exploratory differences, it is unclear if the proposed framework is ready for uptake and if it could provide potential value to practitioners as things stand. \n\n- The explicit acknowledgment and some discussion on some of the limitations are important and add to this work, however, the limitations that have been acknowledged are of great importance and represent harm. The exploratory analyses aspect and the aim of discovering the unknown make the harm of bias reinforcement even more critical, yet no path to mitigating these harms has been provided. The biases in authors’ subjective evaluations are also similarly really important here and it is not clear why at least other humans or recruited crowd workers cannot help in all the key human judgement-based evaluations, though the cost is certainly an understandable factor. Not incorporating domain experts and their knowledge in assessing discoveries is also a big limitation since the potential for this system to generate novel insights and find unknowns remains unclear. With the authors rightfully cautioning users about bias amplification and reinforcement towards the end of this paper, with no pathway to mitigate or understand when such cases arise, it is questionable whether the system should be used at all. There are also at least two other key limitations (discussed below) that also need to be addressed or at least comprehensively discussed in this work. \n\nIn summary, the key weaknesses of the work as it currently stands include a) system reliability for usage by end-users is not yet fully established, with more robustness checks on prompt construction and framing needed; b) while stated as a goal, the step taken in this work does not seem to demonstrate potential for generating novel insights or discovering unknowns, and evaluation signal for the validity of the discoveries that do get generated also seems to be unreliable; and c) limitations, both the ones explicitly acknowledged and the ones that are not, represent real-world harms, and the mitigation of these harms has not been tackled and as things stand, these harms seem to outweigh any benefits of the proposed system and framework. \n\nReferences:\nFanchao Qi, Yangyi Chen, Xurui Zhang, Mukai Li, Zhiyuan Liu, and Maosong Sun. 2021. Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4569–4580, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.\n\nZhang, W. E., Sheng, Q. Z., Alhazmi, A., & Li, C. (2020). Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Transactions on Intelligent Systems and Technology (TIST), 11(3), 1-41.\n\nEric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal Adversarial Triggers for Attacking and Analyzing NLP. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2153–2162, Hong Kong, China. Association for Computational Linguistics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\nThanks for your thorough review!\n\nThe weaknesses raised centered around (a) the societal impact of D5, (b) the robustness and reproducibility of our conclusions, and (c) our evaluation on novelty. We discuss (a) by clarifying the main goal of this submission, and discuss (c) by providing additional discussion\n\nWe addressed (b) by adding three experiments following your recommendations:\n- We rated relevance with external non-authors (e.g crowdworkers)\n- To address concerns that some experiments depend on inaccessible models, we evaluated more accessible models (gpt-3.5 and flan-t5).\n- We constructed an extension of SynD5 using human-written texts.\n\n> Re: “Should D5 usage be held until limitations are mitigated?”\n\nWe agree with this recommendation. Our main contribution is a new task, datasets, and a **baseline system** meant for research use, but NOT a system meant for productions.\n\n> Re: “Rating relevance with non-authors to obtain robust assessment” (new experiment)\n\nWe chose to evaluate relevance ourselves, as crowdworkers might be untrustworthy [5,6]. However, we agree that external ratings would make our conclusion more convincing. Therefore, we used Amazon MTurks to test the conclusion that “prompting with goals leads to more relevant hypotheses'' (Table 2). Our conclusion is still significant, with a p-value of 0.04 calculated in the same way as in L243. Our conclusion also robustly holds when we rated relevance with three LM APIs: GPT-3.5-turbo, Claude-v1.3, and GPT-4. See Table 1 in the response pdf. \n\n> Re: “Test with open source open access models.” (new experiment)\n\nSince some users might not have GPT-4 access, we followed your recommendations and ran a new experiment with gpt-3.5-turbo and flan-t5-xxl as the proposer on SynD5. Overall, 1) gpt-3.5-turbo delivers stronger performance than text-davinci-003, and 2) flan-t5-xxl is still far from the state of the art. These experiments require only 1 A100 GPU with 40 GB memory, or < $5 OpenAI API credit without any approval from OpenAI required. See Table 2 in the response pdf. \n\nWe additionally trained a distilled 3B parameter validator that is similar in quality as flan-t5-xxl. We plan to release it so that people with a 24GB memory GPU can run our system.\n\n> Re: “A synthetic test bed where the two corpora are not generated by language models.” (new experiment)\n\nWe constructed an extension of SynD5 containing human-written texts by adapting the NYT dataset from [9], where each text sample is a New York Times article with a topic and a location label. The topic dimension has 9 different values (e.g., politics, arts) and the location dimension has 10 different values (French, Italy). We then constructed the SynD5 extension by following the same procedure in Section 2.2. \n\nWe then evaluated several D5 systems on it. Similar to Table 2, we found that 1) LMs can leverage the goals effectively, 2) using a validator improves the performance. See Table 3 in the response pdf for more details. \n\n> Re: “Discovering unknowns” is not something that has been evaluated or judged?”\n\nWe evaluated novelty in the following ways:\n- Proxy evaluation: we evaluated the novelty of the candidate discoveries to our best knowledge about the domain experts (results in Appendix 13.2). While we are not real users, the authors collectively have degrees in Econ, public policy, Linguistics, and Computer Science, which covers most of the scope of OpenD5 problems. Proxy evaluations are common practices in machine learning research, where the authors and the crowdworkers, rather than real users, evaluate the output of a machine learning system (e.g., asking crowdworkers rather than professional writers to evaluate creative story generation)[1]. \n- Our judgment on novelty as real users with NLP expertise: We are experts in NLP research and are qualified to make novelty judgments for NLP-related OpenD5 problems. We and our colleagues (non-authors) consider the discoveries for “Analyzing errors in NLP systems” and “Describing distribution shifts” in (Section 6.1) and Appendix 17 to be novel. \n\nIndeed, the ideal evaluation is to 1) gather domain experts and 2) ask for their novelty rating. However, this is organizationally challenging (Appendix 18.2), and we do not consider this ideal evaluation feasible within the first paper written on this task. \n\nIn the future, we hope this paper can attract enough attention from the community of interest and we are actively reaching out to potential real users. So far we have contacted the authors of [2, 3, 4]. All of them are surprised by the discoveries generated by our system; some of them found our D5-generated discoveries novel and are continuing working with us to explore how to use the D5 system properly.\n\n> Re: “It is critical to study the effect of how the exploration goal is phrased” and whether our system is robust to different templates\n\nWe agree that our system might be sensitive to prompt templates; however, it is common for authors to manually write the prompts themselves [7,8]. As prompt robustness is an underlying issue for all NLP evaluations, evaluating and decreasing prompt sensitivity is generally considered its own research question, rather than intrinsic to each dataset.\n\nAgain, thanks for your review, and we hope to hear back from you!\n\n[1] DOC: Improving Long Story Coherence With Detailed Outline Control\n\n[2] Accurate measures of vaccination and concerns of vaccine holdouts from web search logs\n\n[3] Human Heuristics for AI-Generated Language Are Flawed\n\n[4] Understanding the Usage of Online Media for Parenting from Infancy to Preschool At Scale \n\n[5] Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks\n\n[6] Crowdsourcing Beyond Annotation: Case Studies in Benchmark Data Collection\n\n[7] Language Models as Knowledge Bases?\n\n[8] Finetuned Language Models Are Zero-Shot Learners\n\n[9] Goal-Driven Explainable Clustering via Language Descriptions\n"}, {"review_id": "mYZtfwoE92", "reviewer": "Reviewer_Eiwe", "summary": "The authors formulate a new task, D5, that automatically discovers differences between two large corpora in a goal-driven way. The task input is a problem comprising a user-specified research goal (“comparing the side effects of drug A and drug”) and a corpus pair (two large collections of patients' self-reported reactions after taking each drug). The output is a goal-related description (discovery) of how these corpora differ (patients taking drug A “mention feelings of paranoia” more often). \nIn addition to the dataset the authors also build a D5 system, and to quantitatively evaluate its performance, \nFinally, the authors discuss the limitations of the current D5 system, which discovers correlation rather than causation and has the potential to reinforce societal biases when misused.\n\n", "questions": "Above.", "limitations": "Yes, the authors state limitations as well as possible biases.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors propose new tasks and datasets that are interesting and intriguing, which is considered a contribution to the community. The idea of generating discoveries with controllable goal is a valid and useful task.\n2. Overall the task looks well-formulated, and the dataset is covering multiple data domains.\n3. The writing and presentation is of high quality. The paper is generally easy to follow and understand.\n4. The qualitative analysis part is very well done and complete.", "weaknesses": "1. More discussion could be done regarding the uneven distribution of data examples in different domains. For example only 10 from Health versus more than 200 in social sciences.\n2. The authors designed a set of scores for evaluation metrics, and it would be better if there could be some sort of relevance study between the subjective scores with a set of human scorers.\n3. Although using closed large language models are exciting because they generally provides better results, it would be nice to also add in any results with open-source models or smaller models, both for the sake of reproducibility (since OpenAI will deprecate the old APIs soon) and also to show the community where the current open models is at for your proposed tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for appreciating our work! \n\n> (**New experiment**) “Would be nice to also add in any results with open-source models or smaller models”.\n \nSince some users might not have GPT-4 access and text-davinci-003 will deprecate soon, we followed your recommendations and ran a new experiment with gpt-3.5-turbo and flan-t5-xxl as the proposer on SynD5. Overall, 1) gpt-3.5-turbo delivers stronger performance than text-davinci-003, and 2) flan-t5-xxl is still far from the state of the art. These experiments require only 1 A100 GPU with 40 GB memory, or < $5 OpenAI API credit without any approval from OpenAI required. See Table 2 in the response pdf. \n\nWe additionally trained a distilled 3B parameter validator that is similar in quality as flan-t5-xxl. We plan to release it so that people with a 24GB memory GPU can run our system.\n\n> \"It would be better if there could be some sort of relevance study between the subjective scores with a set of human scorers\"\n\nHere we present two types of correlations: 1) for the same metric, how well does the evaluator correlate with each other, and 2) how does different metric correlate with each other\n\nHow well do evaluators correlate with each other: for each metric, we present the average correlation (Kappa or Spearman rank correlation) between evaluators in Table 6. We found that 1) evaluators’ judgements highly correlate with each other, 2) the correlation is the highest for the relevance metric, followed by significance and novelty.\n\nCorrelation between different metrics: for each pair of authors, we compute the pairwise correlation between each metric, e.g., what’s the correlation between evaluator 2’s relevance rating and evaluator 1’s novelty rating. The metrics are substantially correlated, but still sufficiently different: for example, to predict the relevance rating of evaluator 1, it is more informative to use evaluator 1’s relevance rating rather than novelty rating.  See Table 4 in the response pdf for more details. \n\n> \"Regarding the uneven distribution of data examples in different domains … for example only 10 from Health versus more than 200 in social sciences\"\n\nFor many health applications, the data is not publicly available due to privacy concerns. Meanwhile, many social science problems rely on online comments or public speeches, many of which are available online without re-distribution constraints. \n"}, {"review_id": "mLdJCl7RF6", "reviewer": "Reviewer_EmG8", "summary": "The paper introduces a new machine learning task - D5 - to model and discover differences between two large corpora by providing a task input as an exploration goal. The paper also introduces a synthetic dataset, SynD5,  for evaluation of D5, as well as OPEND5, a dataset containing 675 problems that D5 can be used to train on. ", "questions": "- I wasn't clear what was the purpose of the distractor (mentioned in 2.1)\n- I’m not sure I understand the process described in 4.3 where a discovery’s validity is meant to improve by fine-tuning a LM on discoveries with validity scores obtained from an LM. If we can’t trust the validity of the LM to begin with, why would we trust a model fine tuned on it?", "limitations": "I've mentioned a couple limitations in the paragraphs above. I believe the authors have adequately managed to address the limitations in the paper. ", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The task tackled by the paper - generating discoveries based on corpora is impactful \n- The paper provides in-depth description of their methodology and datasets, as well as how they were built\n- The proposed framework is innovative", "weaknesses": "- OPEND5, the dataset that was built specifically for D5, has been collected manually by the authors, which could introduce biases. \n- As the authors themselves mentioned, the extracted hypotheses can be merely correlations in the corpora and not necessary causations. However, D5 is built in such a way that it would treat them as 'discoveries'. Given the nuance around the types of things the model would extract, having clearer, more robust measures and datasets for evaluation of such a system should be in place before releasing it. Validity scores being quite low support this as well. \n- The paper selects 21 out of 3296 examples to showcase as discoveries, potentially showing only cherry-picked examples. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review!\n\n> “OPEND5 has been collected manually by the authors, which could introduce biases”\n\nWe would like to note that for practical purposes, it is common practice in machine learning research for authors to manually aggregate datasets and annotate prompts themselves [2,3,4], especially for tasks that require significant background knowledge and do not have enough naturally occurring data online. We believe that our dataset meets the scientific standard as long as we are transparent and “present in-depth description of our datasets” (as your review has acknowledged). We take the stance of [2], that our performance might be improved with better prompts and hence represents a lower bound of what current systems can achieve. Future work can collect more authentic datasets to facilitate research in this area.\n\n> “Having clearer, more robust measures and datasets for evaluation of such a system should be in place before releasing it.”\n\nWe would like to clarify that:\n\n- The main contribution of our paper is a new task, datasets, and a **baseline** system meant for **research** use, but NOT a robust system meant for broad adoption. \n- Our system is meant to be used in an assistive setting, where the users hold the ultimate responsibility to interpret the outputs of the system properly. \n\nOur submission only serves as a preliminary step towards a broadly applicable D5 system, and we agree that our system is limited in many ways. Therefore, we tried to document as many failure modes and limitations as possible and hope that future works can address them (Section 6.2 and Appendix 18). In the updated version, we will explicitly mention that our system is only meant for research use and should be used in an assistive setting.\n\n> D5 is limited because “correlation != causation”\n\nAs mentioned above, our system is meant to be used in an assistive setting. Therefore, the users hold the responsibility to interpret D5 discoveries as correlations instead of causations, just as users of linear regression should be responsible for not interpreting the coefficients as causal. \n\n> \"I wasn't clear what was the purpose of the distractor (mentioned in 2.1)\"\n\nCorpora A and B will have two differences, one is the correct goal-related difference and the other is the distractor difference. If a system fails to use the goal, it might output the distractor difference. As a result, we can penalize systems that are not goal-driven. (L95)\n\n> “If we can’t trust the validity of the LM to begin with, why would we trust a model fine tuned on it?”\n\nThe validator is trained to approximate human judgment (L187), and model-approximated human judgment is broadly used in the literature, e.g., RLHF[1]. Finally, we confirmed the success of our algorithm with human evaluators (L212), who are more trustworthy sources.\n\n> “The paper selects 21 out of 3296 examples to showcase as discoveries, potentially showing only cherry-picked examples.”\n\nWe stated in Section 6.1 that this section is meant to be a qualitative evaluation that gives readers better insights about our current D5 system, and we agree that it can be potentially biased by the authors’ choice. However, our paper also includes a lot of quantitative evaluations, which are presented in Section 5. \n\n[1] Learning to summarize from human feedback\n\n[2] Language Models as Knowledge Bases?\n\n[3] Fine-tuned Language Models Are Zero-Shot Learners\n\n[4] Evaluating Large Language Models Trained on Code\n"}, {"review_id": "1JEXcx67Qt", "reviewer": "Reviewer_sfxq", "summary": "This paper proposes a goal-directed discovery and description of differences between text distributions (D5). Given two corpora and a research goal, the task is to generate a discovery in a natural language predicate. The paper creates datasets SynD5 and OpenD5, which can be used to evaluate  validity, relevance, novelty, and significance. \n", "questions": "- As raised in the Weaknesses above.\n- Q. In 2.1 Task Format, is the goal description given for the task (and thus is included in the OpenD5 dataset)? \n", "limitations": "The paper discusses the limitations. For ethical concerns, I didn’t check the licenses of all the datasets (whether they can be redistributed after modification)\n", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper tackles an interesting task and formulates research discovery as an NLP task. \n- The paper creates new SynD5 and OpenD5 datasets from a variety of problems. \n", "weaknesses": "- #1 The task is not well-formed or well-motivated. \n- #2 The dataset quality is not ensured. As stated in LL119-123, OpenD5 does not ensure that there is ground-trugh and thus is not clear hwo reliable it is as a benchmark. \n- #3 The paper is generally well-written but the key concept and main messages are hard to parse. \n\nFirst of all, it is a great effort to compile hundreds of datasets into a single benchmark but I’m afraid that I’m not very clear what we can conclude after seeing method A perform better than method B on this dataset. After reading the paper, I wasn’t able to answer a question “What functionality assessment can this test bed be used for?” by myself. \n\nLooking at examples in Figure 2, the OpenD5 data split looks synthetic and some of them rely on the ground-truth labels for the original task. (e.g., 10-star reviews and 0-star reviews.) In other words, how the original corpus was split into Corpus A and B is essential to answer the research question. Thus, if the split is given, the problem is partially solved already. \n\nIn my understanding, the task essentially asks if the model can figure out how those two corpora were created from the original corpus with additional context information as the research goal. \n\nFor Machine Learning problems, I believe analysis on each pair could be one research paper (e.g., the difference between SNLI and MNLI; LM’s output differences etc.) Then, again I’m not very sure what we can learn from evaluation results on the OpenD5 benchmark. \n\nTo me, the problem the paper is tackling is rather automating Exploratory Data Analysis using LLMs (like the paper below. To be clear, it’s considered concurrent work and the paper doesn’t have to compare with it.)\n\nPingchuan Ma, Rui Ding, Shuai Wang, Shi Han, Dongmei Zhang, Demonstration of InsightPilot: An LLM-Empowered Automated Data Exploration System, https://arxiv.org/abs/2304.00477\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review!\n\n> \"The task is not well-formed or well-motivated …  the task essentially asks if the model can figure out how those two corpora were created from the original corpus with additional context information as the research goal.\"\n\nJust to clarify, this task is not about recovering how the two corpora were generated – quite the contrary, we actually tell the D5 system how the two corpora were generated (Figure 1 Samples from Corpus A include self-reported …). Instead, we want a D5 system to explain goal-related differences. For example, given Corpus A= 1-star reviews vs. Corpus B=10-star review and a goal of “understanding what makes the customer unhappy”, the output discovery “Corpus A is 1-star” is not relevant, and we hope to generate something like “Corpus A mentions delivery issues more often” (L141). \n\n> “What functionality assessment can this test bed be used for?”\n\nThe benchmarks evaluate a D5 system’s ability to generate a goal-related natural language explanation for how two text corpora are different. This is different from Ma et. al, which operates on tabular data. \n\n> “Task Format, is the goal description given for the task (and thus is included in the OpenD5 dataset)?”\n\nThe goal descriptions are also given in OpenD5.\n\n> \"OpenD5 does not ensure that there is ground-truth and thus is not clear how reliable it is as a benchmark\"\n\nEven though we do not have ground truth available, once a system produces a discovery, we can evaluate them with the metrics in Section 3. This is more reliable than assuming a fixed ground truth, as there could be multiple relevant discoveries and a single discovery could be phrased in multiple ways (similar to how in machine translation, assessing similarity to a “ground truth” translation is less reliable than assessing the quality of a translation directly). \n\nWe also included a diagnostic evaluation, SynD5, for automatic evaluation that does have a single specified ground truth for each example.\n\n> License concerns for OpenD5 \n\nThe licenses for all 58 data sources are documented in Appendix 22. If there are any that violate a license, please let us know so that we can remove them.\n"}, {"review_id": "Zk2e2Eld3w", "reviewer": "Reviewer_vHRK", "summary": "This paper presents a new task called D5 (goal Driven Discovery of Differences between text Distributions via language Descriptions)\n) where the objective is to identify and describe meaningful differences between two given text corpora. Authors created two datasets to evaluate their D5 task based: SYND5 and OPEND5, based on validity and relevance. ", "questions": "The above weakness are also my questions. \n\nEvaluation Metrics: \nThe paper primarily reviews the task based on the metrics of validity and relevance, but also mentions evaluating subjective metrics like novelty and significance. Could the authors explain why validity and relevance were prioritized in the evaluation process? And can the authors provide a brief overview or key insights from the evaluations of novelty and significance, given that they are confined to the appendix due to space constraints?\n\nRelated work: \nThe author propose their contribution as the novel D5 task but also mentioned they mirrored Zhong et al. (2022)'s study. Could the author explain what is their novel contribution compared with Zhong et al's work? ", "limitations": "The authors have indeed discussed several limitations of their work which they couldn't address at this stage. ", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The creation of OPEND5 and SYND5 datasets for training and evaluation shows originality.", "weaknesses": "Task justification and applicability: \nThe paper introduces a new task D5  (goal Driven Discovery of Differences between text Distributions via language Descriptions). However, the need for such a task is not clearly articulated. The authors do not provide sufficient examples or scenarios where this task could be beneficial, making it difficult for the reader to fully appreciate the significance of this work. It would be beneficial if the authors could provide concrete real-world applications or problems where this task could contribute to a solution or add value. If the main usage is as the example shown in Figure 3, the users can use GPT to generate answers themselves nowadays.  \n\nInterpretation of discoveries: \nThe paper acknowledges that the interpretation of the discovered differences is challenging and often requires domain experts. However, the study doesn't suggest concrete ways to address this problem, which could limit the practical application of the D5 system. \n\nReliance on Turkers' judgments: \nThe paper relies heavily on the judgments of Mechanical Turkers to evaluate the validity of discoveries. This approach might introduce another layer of bias or inaccuracies due to the potential lack of expertise among Turkers as authors mentioned in the section 6.2.  \n\nMissing a conclusion section\n\nMinor suggestion: the right edge of Figure 2 is cut off which need to adjustment. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> \"The authors do not provide sufficient examples or scenarios where this task could be beneficial, making it difficult for the reader to fully appreciate the significance of this work.\"\n\nThe goal of the task is to explain how two corpora are different in a goal-driven way, and we provided examples of applications in the abstract, intro, Figure 1, and Section 6.1. Here is an example from the abstract: “The task input is a problem comprising a user-specified exploration goal (comparing the side effects of drug A and drug B”) and a corpus pair (collections of patients’ self-reported reactions after taking each drug). The output is a goal-relevant description (discovery) of how these corpora differ (patients taking drug A “mention feelings of paranoia” more often).” Such an automated system would be useful because exploring large corpora can generate useful discoveries but is time-consuming for humans.\n\n> “If the main usage is as the example shown in Figure 3 (naive prompting), the users can use GPT to generate answers themselves nowadays.”\n\nNaively prompting GPT with the goal alone would not work because the output needs to be grounded into the two given corpora, and using a validator is crucial to the performance. In Table 1 right gpt-4,w/g column, we find that prompting the language model (proposer) to generate an answer without validation has significantly lower performance (27% -> 8%). Intuitively, validator is crucial because 1) the language model proposer is not reliable to generate the correct answer within a single answer, and 2) while we cannot feed the entire corpus into the proposer due to context length limitations, we can feed individual samples from the entire corpus to the validator to filter out hypothesis. \n\nL298: Interpretation of D5’s output might require domain experts, thus limiting the impact of our application. \n\nWe would like to clarify that our system is meant to be used in an assistive setting, where the users hold the ultimate responsibility to interpret the outputs of the system properly. Therefore, in practice we expect that domain expert users themselves to be able to interpret D5’s outputs in the domain of interest, or be calibrated on whether they can interpret the outputs.\n\nWhy was novelty and significance evaluation presented in the Appendix? \n\nIndeed, we are constrained in space, and we prioritized presenting validity and relevance in the main paper because they are more objective in their definition. As you have recommended, we will include insights about evaluating novelty and significance in the main paper for the updated version. \n\n> \"Could the author explain what is their novel contribution compared with Zhong et al's work?\"\n\nCompared to Zhong et al.,\n- We collected a large dataset, OpenD5, so that we can study D5 in a data-driven approach. This allows us to identify more promising use cases and identify more limitations (Section 6).\n- We included an exploration goal in the task input, which can potentially lead to more useful output discoveries\n\n> “Missing a conclusion section”\n\nConclusion section typically contextualizes results, discusses limitations, and looks forward. In our submission, sections 6 and 7 do this in detail. We consider it only a stylistic difference that we called our final sections “discussion and related work”. \n"}]

Author rebuttal:
Thanks to all the reviewers for their comments! We are glad that most reviewers agree that the task is well-motivated and potentially impactful, the paper is well-written, and OpenD5 is a valuable resource for the community.

Many of the weaknesses brought up by the reviewers were 1) limitations of the current system, and 2) whether the conclusion drawn from our paper is robust and reproducible by the broader community. We address 1) by clarifying the contribution of our paper and address 2) by **adding 3 experiments following the reviewers’ recommendations**.

> **Clarifying Our Contribution**

Many of the weaknesses brought up by the reviewers were in fact limitations that our submission has already discussed. We hypothesize that there might be a mismatch in expectation between us and the reviewers, so we would like to clarify that:

- The main contribution of our paper is a new task, datasets, and a **baseline** system meant for **research** use, but **NOT** a system meant for broad adoption. 
- Our system is meant to be **assistive**, where the users hold the ultimate responsibility to use the outputs of the system properly. 

Given that this is the first paper written on this task, we SHOULD expect the current system to have many limitations; in fact, one goal of collecting OpenD5 is to actively look for as many limitations as possible in a data-driven way (L59). Since these limitations might have never surfaced without our invested effort, **we consider our extended discussion about the limitations as our strength rather than weakness** (Section 6.2 and Appendix 18), and we hope that our discussion can open up future research that can address them.

> **New Experiments**

We followed the reviewers recommendations and ran three additional experiments (see our uploaded pdf for more detail): 
- We rated relevance with external non-authors (e.g., human crowdworkers). Our conclusion about relevance robustly holds under 5 different types of evaluators.
- To address concerns that some of our experiments depend on models that are not broadly accessible, we evaluated more accessible models (gpt-3.5 and flan-t5) on SynD5. Now the experiments on SynD5 can be run with only a 24GB memory GPU and < $5 OpenAI API credit without any approval required.
- We constructed a version of SynD5 using human-written texts instead of model-written texts. We found that our conclusions in Table 1 unchanged.

> (New experiment 1) Rating relevance with non-authors

In our submission, we chose to evaluate relevance ourselves: even though it is an easy and common practice to recruit crowdworkers for evaluation, they might be noisy and untrustworthy [1,2]. To deliver the most reliable results, we chose to evaluate as many outputs as possible manually ourselves and ensured unbiasedness by being blind to how outputs were generated.

However, we agree that external relevance rating would make our conclusion more convincing. Therefore, we ran a new experiment and validated the conclusion that “prompting with goals leads to more relevant hypotheses'' (Table 2) by using Amazon Mechanical Turks. We still find our conclusion to be significant, with a p-value of 0.04 calculated in the same way described in L243. We ran additional robustness checks and our conclusion still holds by rating relevance with three language model APIs: GPT-3.5-turbo, Claude-v1.3, and GPT-4. Therefore, our conclusion on relevance robustly holds under 5 different types of evaluators, including expert authors, external crowdworkers, and language models from different companies with different levels of capabilities. See Table 1 in the response pdf for more details. 

> (New experiment 2) Performance of Open Sourced Models on SynD5

To address concerns that some of our experiments depend on models that are not broadly accessible, we additionally evaluated gpt-3.5-turbo and flan-t5-xxl as the proposer on SynD5. Overall, we find that 1) gpt-3.5-turbo delivers stronger performance than text-davinci-003, and 2) flan-t5-xxl is still far from the state of the art. These experiments require only 1 A100 GPU with 40 GB memory, or < $5 OpenAI API credit.  See Table 2 in the response pdf for more details. 

(We did not experiment with the LLaMa family since the instruction-tuned version is not yet available to us.)

> (New experiment 3) SynD5 with Human-Written Text

To address the concern that the text in SynD5 is model-generated, we constructed an extension of SynD5 with human-written texts by adapting the NYT dataset from [3], where each text sample is a New York Times article with a topic and a location label. The topic dimension has 9 different values (e.g., politics, arts) and the location dimension has 10 different values (French, Italy). We then followed the same procedure described in Section 2.2 to create this SynD5 extension.

We then evaluated several D5 systems on this dataset. Similar to the findings in Table 2, we found that 1) language models can leverage the goals effectively and 2) using a validator improves the performance. See Table 3 in the response pdf for more details. 

[1] Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks

[2] Crowdsourcing Beyond Annotation: Case Studies in Benchmark Data Collection

[3] Goal-Driven Explainable Clustering via Language Descriptions


Meta-review: This paper proposes an interesting new task, D5, and presents a dataset for the task. D5 is comparing two corpora and seeing how they differ for a specific goal. While some reviewers thought the motivation for this task was not clear, I believe the motivation is definitely there, and perhaps the authors can try to revise the paper to make that even clearer. Some other weaknesses were pointed out in the reviews, but they were mostly addressed well by the authors.

There was an ethics issue, mainly for how one would/could use the system, and whether bias and/or hallucination would present problems. The authors addressed the issue well, and I encourage the authors to revise the paper acordingly.

Overall, this is a good paper tackling an important and interesting problem.
Decision: Accept (poster)