Title: Goal Driven Discovery of Distributional Differences via Language Descriptions
Venue: NeurIPS 2023
Accepted: yes
Keywords: large language model

prompting

exploratory text analysis
Abstract: Exploring large corpora can generate useful discoveries but is time-consuming for humans.
    We formulate a new task, D5, that automatically discovers differences between two large corpora in a goal-driven way. 
    The task input is a problem comprising a user-specified research goal (“*comparing the side effects of drug A and drug*”) and a corpus pair (two large collections of patients' self-reported reactions after taking each drug). 
    The output is a goal-related description (discovery) of how these corpora differ (patients taking drug A “*mention feelings of paranoia*” more often).
    We build a D5 system, and to quantitatively evaluate its performance, we 1) build a diagnostic benchmark, SynD5, to test whether it can recover known differences between two synthetic corpora, and 2) contribute a meta-dataset, OpenD5, aggregating 675 open-ended problems ranging across business, social sciences, humanities, machine learning, and health.
    With both synthetic and real datasets, we confirm that language models can leverage the user-specified goals to propose more relevant candidate discoveries, and they sometimes produce discoveries previously unknown to the authors, including demographic differences in discussion topics, political stances in speech, insights in commercial reviews, and error patterns in NLP models.
    Finally, we discuss the limitations of the current D5 system, which discovers correlation rather than causation and has the potential to reinforce societal biases when misused; therefore, practitioners should treat the outputs of our system with caution.

Goal Driven Discovery of Distributional Differences via Language Descriptions

Abstract

Exploring large corpora can generate useful discoveries but is time-consuming for humans. We formulate a new task, D5, that automatically discovers differences between two large corpora in a goal-driven way. The task input is a problem comprising a user-specified exploration goal (' comparing the side effects of drug A and drug B ') and a corpus pair (collections of patients' self-reported reactions after taking each drug). The output is a goal-relevant description (discovery) of how these corpora differ (patients taking drug A ' mention feelings of paranoia ' more often). We build a D5 system, and to quantitatively evaluate its performance, we 1) build a diagnostic benchmark, SYND5, to test whether it can recover known differences between two synthetic corpora, and 2) contribute a meta-dataset, OPEND5, aggregating 675 open-ended problems ranging across business, social sciences, humanities, machine learning, and health. With both synthetic and real datasets, we confirm that language models can leverage user-specified goals to propose more relevant candidate discoveries, and they sometimes produce discoveries previously unknown to the authors, including demographic differences in discussion topics, political stances in speech, insights in commercial reviews, and error patterns in NLP models. Finally, we discuss the limitations of our D5 system, which discovers correlation rather than causation and potentially reinforces biases when misused; therefore, practitioners should treat the outputs of our system with caution.

1 Introduction

Exploring large corpora and generating discoveries from them can be ad hoc and laborious. For example, to compare the side effects of drug A and drug B, doctors might inspect two large corpora of patients' self-reported reactions after taking each drug; based on ad hoc insights, they hypothesize that patients taking drug A more often ' mentions feelings of paranoia ', and then validate this hypothesis by laboriously inspecting the two corpora. Since machines can automatically process a large amount of texts, we might hope for ML systems to facilitate exploratory analyses like the one above.

However, an ML task requires a unified input-output space and evaluation metric so that it can be automated, benchmarked, learned, and analyzed. To this end, we formalize one type of exploratory analysis problem as a natural language generation task: goal d riven d iscovery of d ifferences between text d istributions via language d escriptions (D5). As shown in Figure 1, the input to the D5 task is a 'problem' comprising a description of a user-specified exploration goal (understanding side effects) and a corpus pair (text samples from the distributions of self-reported reactions after taking each drug). The output is a 'discovery' represented as a natural language predicate (' mentions feelings of paranoia '). We evaluate a discovery with two criteria (Section 3): (1) validity: it should describe a true difference (Zhong et al., 2022); and (2) relevance to the goal (McGarry, 2005).

Explora(on

Goal

The original dataset includes pa(ent's self-reported reac(ons a9er taking a drug. The two corpora are

generated based on what drug the pa(ent has taken. Samples from Corpus A include self-reported

reac(ons a9er taking drug A, while samples from Corpus B include self-reported reac(ons a9er taking

drug B. I am a doctor. My goal is to understand the side effects of drug A.

Corpus

Pair

-Coughing for two months

-Felt sleepy today

-[~4K more samples omiNed for brevity]

-Slowly recovering.

-[~4K more samples omiNed for brevity]

-Even liNle sound dries me crazy

- Feelig to worried to focus

-[~4K more samples omiNed for brevity]

-My family complaints I'm too irritated

-[~4K more samples omiNed for brevity]

Corpus A

Corpus B

Explora(on

split

Valida(on

split

Output

Discovery

Corpus A has more samples that

'

mention feelings of paranoia

'

Input

Problem

Since D5 is open-ended and aims at discovering unknowns, the most popular benchmark practicecomparing system-generated outputs with human-written references on a test set-is infeasible. We therefore design two evaluation strategies.

Diagnostic : we synthesized a dataset of D5 problems with known solutions, SYND5, to diagnose whether a D5 system can recover known differences between two synthetic corpora. This strategy is cheap and automated but might not reflect user utility in real applications.

Open-ended : we collected a dataset, OPEND5, by aggregating 675 open-ended D5 problems ranging across business, social sciences, humanities, health, and machine learning (Figure 2), comprising 4.4 million text samples in total across problem corpora. We then manually evaluated a subset of the output discoveries. This strategy is subjective and expensive, but useful for obtaining qualitative insights on more realistic applications.

These two strategies allow us to quantitatively evaluate and compare D5 systems. For example, we compared 1) the system from Zhong et al. (2022) designed to describe corpus-level differences without goals, and 2) a goal-conditioned variant that we develop in Section 4. We found language models successfully use the specified goal: the goal-conditioned variant is correct 12% more often on SYND5, and it produces relevant candidate discoveries 31% more often on OPEND5.

We envision OPEND5 to be a growing, diverse repository of open-ended D5 problems. They will not only help us evaluate D5 systems more reliably, but also allow the following operations:

Facilitate exploratory analysis. Every time we build a better D5 system, we can apply it to a repository of open problems and send the discoveries to researchers who posed them. We show this paradigm is plausible by using our system to automatically produce useful discoveries on OPEND5 (Section 6.1), including insights from commercial reviews, temporal and demographic differences in discussion topics, political stances and stereotypes in speeches, differences in lyric styles, and error patterns in NLP systems. We anticipate future systems to produce more discoveries.

Analyze the limitations of our evaluation. Using concrete examples from OPEND5, we show that our current evaluation metrics do not encourage diverse findings, do not always produce causal conclusions, and cannot evaluate discoveries involving heavy expert knowledge (Section 6.2). More D5 problems can help us identify more limitations, which inform areas for future improvement.

Train better D5 systems. Like other ML tasks, we can train a system once we have a dataset. We describe a self-supervised learning algorithm that uses a repository of problems (without reference solutions) to train LMs to propose more valid hypotheses (Section 4.3). As a proof-of-concept, we show that it can make LMs better describe the differences between small groups of text samples.

To conclude, we show that D5 can be quantitatively evaluated, automated, analyzed, and learned. Like other ML tasks, it would benefit from a more diverse, authentic, and larger dataset. We hope future works can gather feedback from domain experts and curate an ever-larger dataset of D5 problems, thus accelerating exploratory analyses and facilitating scientific discoveries. 2

2 Datasets: SYND5 and OPEND5

We first introduce how each input problem is formatted. Then we discuss 1) how we synthesized SYND5, which is used for automatic diagnostic evaluation, and 2) how we collected OPEND5, which is used to investigate the practical value of D5 systems in open-ended applications.

2.1 Task Format

Each D5 problem is represented by a corpus pair (Corpus A/B) and a description of the exploration goal. For example, Corpus A/B might be self-reported reactions after taking drug A/B, and the goal description would be ' comparing the side effects of drug A and drug B '. The desired output is valid and relevant discoveries in the form of natural language predicates (Figure 1), e.g. Corpus A has more samples that ' mentions feelings of paranoia '.

2.2 SYND5, a Diagnostic Benchmark with Reference Solutions and an Automatic Metric

To automatically diagnose a D5 system, we synthesized SYND5, a dataset of D5 problem with reference solutions. To synthesize Corpus A and Corpus B for each input problem, we used a language model (LM) to generate two corpora that simultaneously differ on two dimensions, one of which is goal-relevant and one of which is a distractor. For instance, suppose the goal is to ' understand how Corpus A differs from Corpus B in terms of topic '. Then we would synthesize an example where Corpus A is more sports-related while B is more art-related (goal-relevant: varying topic), while additionally Corpus A is in English while B is in French (distractor: varying language). The reference solution is the difference on the goal-relevant dimension, e.g. ' is sports-related '.

In more detail, to synthesize an example in SYND5, we first picked one goal-relevant and one distractor dimension from the set { topic , genre , language } , and sampled a value for each corpus and dimension (e.g. Corpus A: [sports, English]; Corpus B: [art, French]). We then synthesized Corpus A/B such that all its samples are in English/French (i.e. completely different on the distractor dimension) while V percent of them are sports-related/art-related, where we varied V from 0.6 to 1. Since the distractor difference is more salient, SYND5 penalizes D5 systems that ignore the goal and output the incorrect distractor difference ' is in English '. We synthesized 300 problems in total to create SYND5; see Appendix 9 for a detailed description of the pipeline.

To compute a D5 system's accuracy, we prompted Claude-v1.3 (Bai et al., 2022b) to judge how often the output discovery is semantically equivalent to the reference. We construct the prompt by using 6 pairs of predicates with the labels of 'equivalent', 'similar', or 'irrelevant' as fewshot examples, and ask Claude-v1.3 to judge whether the output discovery and the reference are 'equivalent'. As a result, we can automatically diagnose a D5 system. See Appendix 12 for the prompt for equivalence judgement and Appendix 10 for two robustness checks, which (a) consider 'similar' discoveries to be correct as well, and (b) use other LMs for equivalence judgement.

2.3 OPEND5, a Realistic Open-Ended Dataset without Reference Solutions

To evaluate a D5 system's utility under realistic applications, we also gathered OPEND5, a realistic dataset of 675 open-ended D5 problems. These problems range across business, social sciences, humanities, health, and machine learning; see Figure 2 for a few examples. To build OPEND5, two of the authors performed an extensive literature review on problems that could potentially benefit from our system, e.g., reading survey papers (Nguyen et al., 2020) and courses on computational social sciences, and skimming through the ACL proceedings from the past decade and datasets from Kaggle that have an NLP tag; we then annotated the exploration goals, scraped/generated the

corresponding corpora, and post-processed them over nine months (see complete list of citations in Appendix 23). As shown in Figure 1, each goal describes the original dataset, how the two given corpora are generated, who is using the system, and what property of the two corpora does the user want to understand. Each OPEND5 problem is reviewed by at least two of the authors to reduce grammatical mistakes and ambiguous interpretations of the goal.

Each corpus contains around 17K text samples on average, and OPEND5 in total comprises 4.4 million distinct text samples. We use 50% of each corpus as the 'exploration' split and 50% as the 'validation' split. The system can only access the exploration split, while the validation split is reserved for the evaluators to validate the discovery. A validation split prevents overfitting the discoveries to the given samples and is analogous to the train-test split in machine learning.

Since we hope to build systems that can tackle challenging open-ended problems, we did not avoid cases where we do not know the ground truth answer. This is different from standard benchmarking practices, where humans can provide a reference solution to evaluate an AI system. However, even though we do not know the ground truth, once a system produces a discovery, we can still evaluate it. We present our evaluation metrics in the next section.

3 Evaluation Metrics for Open-Ended D5 problems

For the goal of comparing the side effects of drug A and drug B, how do we evaluate a systemgenerated discovery that Corpus A ' mention feelings of paranoia ' more often? First, it needs to be valid, such that indeed more samples from Corpus A satisfy this predicate, which can be evaluated (approximately) objectively. Second, it needs to be relevant to the goal of understanding side effects, which depends on the user's subjective judgement. We define validity and relevance below.

Validity. Similar to Zhong et al. (2022), we require an output discovery h to be a truth predicate on a text sample. For example, if h = ' mentions about family ', then h is true on the string x 1 = ' My daughter loves me ' and false on the string x 2 = ' I'm going to school '. Define T ( h, x ) ∈ [0 , 1] as 'the certainty that h is true on x ', e.g., T ( h, x 1 ) ≈ 1 and T ( h, x 2 ) ≈ 0 . We approximate T ( h, x ) by asking three Turkers how certain they are and averaging their responses (see Appendix 11 for details).

Let D val A and D val B denote the validation sets for Corpus A and B. We define the 'validity' V as

Computing V ( h ) is expensive since it requires human annotations T ( h, x ) on a set of text samples even to evaluate a single discovery h . In practice, we do not have the budget to compute V ( h ) on the entire validation split; therefore, we approximate this quantity by randomly sampling from Corpus A and Corpus B . We use these samples to compute an empirical estimate of V , as well as a p -value for the null hypothesis that V ≤ 0 using a one-sided t-test.

Relevance. Adiscovery may be irrelevant even if V = 1 . For example, if the goal is to understand the writing style differences between higher-scoring essays (Corpus A) and lower-scoring ones (Corpus B), the discovery that Corpus A ' achieves higher scores ' has high validity score by definition but irrelevant to the goal of understanding stylistic differences.

Therefore, we designed a procedure to evaluate relevance, where human or language model evaluators would score each discovery with 2 ⃝ / 1 ⃝ / 0 ⃝ . The evaluators used the rubric below, which illustrates the meaning of each score with the essay example above:

2 ⃝ , relevant; e.g. the discovery ' write in first person ' is directly related to the writing style.

1 ⃝ , indirectly relevant; e.g. the discovery ' use the word 'I' ', is not exactly a writing style, but can still inform the relevant underlying principle of ' write in first person '.

0 ⃝ , irrelevant; e.g. the discovery ' argue for abortion ' is unrelated to the writing style.

To minimize biases while comparing two systems, the evaluators are blind to which system generates which discoveries.

To conclude, an ideal discovery would have a high V value with a small p -value and achieve ratings of 2 ⃝ in relevance. In the next section, we will build a D5 system that addresses these criteria by first proposing goal-relevant candidate discoveries (hypotheses) and then automatically validate them.

Other metrics. We also explored two other subjective metrics, novelty (how difficult it is to generate the discovery) and significance (how beneficial it is to learn about the discovery). Due to space limit, we present their rubrics and related results in Appendix 14.

4 Methods: Building a D5 System

We describe our D5 system, which maps from a corpus pair and an exploration goal to a set of natural language predicates. Our system is inspired by a two-stage model of how humans discover patterns in data: creatively brainstorming hypotheses and then rigorously validating them on the data (Ludwig & Mullainathan, 2022). Analogously, we first propose hypotheses conditioned on the exploration goal and a subset of samples from the corpus pair (Section 4.1). We then use a language model to approximately compute the validity of each hypothesis, and output the valid ones as the final discoveries (Section 4.2). Our system closely mirrors that of Zhong et al. (2022), except that we leverage the goal to propose more relevant hypotheses. Finally, we present a self-supervised learning algorithm to improve an LM's ability to propose more valid hypotheses (Section 4.3); however, due to API access constraint, we cannot apply it to fine-tune gpt-3 , so we provide a proof of concept experiment on Flan-T5 (Chung et al., 2022).

4.1 Hypothesis Proposer

We prompt gpt-3 (Ouyang et al., 2022) to propose hypotheses. Denoting the exploration split of Corpus A/B as D exp A / D exp B , we construct the prompt by concatenating a few random samples from D exp A and D exp B , the exploration goal, and an instruction to output a list of hypotheses. Figure 3 (left) depicts an example of the resulting prompt, together with a typical language model output.

Since the entire corpus pair might not fit into one prompt, we construct multiple prompts with different sets of samples so that gpt-3 can 'see' as many different samples as possible in our pipeline. We continue sampling hypotheses with different prompts until obtaining a set of 60 hypotheses, which we call H init. Appendix 15 includes more details on selecting the sets of samples for different prompts.

4.2 Hypothesis Validator

Many hypotheses in H init have low validity: they are not more often true on D A than on D B (i.e. V ( h ) ≤ 0 ). To automatically filter them out, we use a language model T ′ to simulate the Turkers'

Group A: The Manchester United soccer squad welcomes rising star Juan Silva, …

Group A: As Serena Willows joins the UCLA women's tennis roster,

Group A:

Group B: Egypt's President Abdel Fa?ah el-Sisi and Saudi Arabia's …

Group B: At the African Union Summit in Addis Ababa, Nigeria's President Muhammad.

Group B:

(some of the sentences are truncated for brevity) …

The original dataset includes news summaries. The two corpora are generated based on

when they were published. Samples from Group A

include news from 2007

, while samples

from Group B

include news from 2008

. I am a journalist trying to understand what topics

are popular across years.

Please write a list of hypotheses  (separated by bullet points "-") of how datapoints from

Group A differ from those from Group B. Each hypothesis should be formaMed as a

sentence fragment. Here are three examples.

- "talks about poli;cs, such as presiden;al elec;on.'

- "contains insul;ng language for immigrants."

- "uses double nega;on, i.e., using two nega;ons in a sentence."

Based on the two sentence groups (A and B) from the above, more sentences in Group A ...

- '

menLons a sports team recruiLng a new member'

- 'menLons about academic relaLons, such as teachers or students'

- 'menLons about

Samples from

the two corpora

Explora;on

Goal

Formaqng

Instruc;ons

Language

Check whether the TEXT sa;sfies a

PROPERTY. Respond with Yes or No. When

uncertain, output No.

Now complete the following example -

input: PROPERTY:

menLons a sports team

recruiLng a new member

TEXT:

women's tennis roster

output:

Proposer prompt

Check whether the TEXT sa;sfies a

PROPERTY. Respond with Yes or No. When

uncertain, output No.

Now complete the following example -

input: PROPERTY:

menLons a sports team

recruiLng a new member

TEXT:

and Saudi Arabia's

.

output:

Validator prompt

Pr[NextWord = 'Yes'] = 99%

Pr[NextWord = 'Yes'] = 5%

// a list of hypotheses not included for brevity

// 20 samples not included for brevity

// 20 samples not included for brevity

judgement T and hence approximate the validity score V ( h ) with the function V ′ ( h ) , defined as

To compute T ′ , we ask Flan-T5 whether x satisfies h with the prompt shown in Figure 3 (right). To better simulate Turker's judgment, we collected additional Turker annotations to fine-tune FLAN-T5 (see Appendix 16 for details about the data collection process). We then obtain a significance value p ′ by performing a t-test to compare the mean value of V ′ ( h, x ) on the exploration split of Corpus A to that of Corpus B , rule out the hypotheses with p ′ greater than 0.001, and output the remainder as discoveries. Finally, we obtain additional discoveries by repeating the same process but asking our system to propose and validate hypotheses about Corpus B rather than Corpus A . Appendix Figure 5 visualizes our entire pipeline and Appendix 8 discusses the computational resources we used.

4.3 Self-Supervised Learning with Open-Ended Problems: A Proof of Concept

Since D5 problems are open-ended, future systems could potentially produce discoveries with higher validity scores than any known discovery. Therefore, we design a self-supervised learning algorithm to improve an LM's ability to propose more valid hypotheses, using the principle that it is easier to validate a discovery than to generate one .

Algorithm. Suppose we are given a set of problems for training and an initial language model m 0. Our goal is to automatically generate a set of prompt -completion pairs to fine-tune m 0 so that it can propose hypotheses that are more valid. To generate a prompt , we randomly sample a problem and create a proposer prompt following the procedure in Section 4.1. To generate the desired completion given a prompt, we sample multiple hypotheses from m init, approximate their V ′ score on the samples in the proposer prompt with the same language model m init (Section 4.2), and select the highest scoring hypothesis. Finally, we use the prompt-completion pairs to fine-tune m 0.

A Proof of Concept Experiment. Since we cannot fine-tune text-davinci-003 , we can only experiment with Flan-T5-xxl (Chung et al., 2022), an open-sourced instruction-tuned model that might only work well for easier 'mini-problems'. As a proof of concept, we tested the above self-supervised learning algorithm on the task of describing groups of four samples, where each group comes from a text cluster.

We computed both the automated 'self-evaluation' validity score V ′ and the 'true' validity score V according to Turker evaluation for evaluation. After self-training, V ′ improves substantially from 0.22 to 0.37, and the V improves from 0.07 to 0.10, with a p -value of 0.02. This result provides preliminary evidence that self-training could be applied to a large set of problems to improve the

validity of the hypotheses; we expect future validators to simulate human judgments better, hence decreasing the approximated gap of improvement between V and V ′ . We discuss more training and evaluation detail in Appendix 20.

5 Quantitative Evaluation on SYND5 and OPEND5

We show that both SYND5 and OPEND5 can be used to quantitatively evaluate D5 systems. Since SYND5 is automatic, we used it to compare a broad range of D5 systems and studied the contributions of three different factors: the quality of the proposer model ( gpt-4 vs. text-davinci-003 ), the use of a validator, and the use of a goal. We then further investigated the effect of using goals under realistic applications through human evaluation on OPEND5.

Automatically comparing different variants with SYND5. As mentioned above, we ablated 3 factors, resulting in 2 3 = 8 variants. We compared 1) using text-davinci-003 vs. gpt-4 as the hypothesis proposer; 2) using the validator to compute V ′ for each hypothesis and outputting the highest-scoring hypothesis, vs. not using the validator and outputting a random hypothesis; and 3) using the goal vs. replacing it with ' I want to understand how Corpus A is different from Corpus B. '. We then automatically calculated the accuracy for each variant as described in Section 2.2.

We report the results in Table 1. We find that using the validator and the goals significantly improve the performance, and gpt-4 outperforms text-davinci-003 with goals and the validator ( p < 1% under a t-test). We conducted two additional robustness checks in the Appendix 10: (a) using text-davinci-003 instead of Claude-v1.3 to judge predicate equivalence, and (b) considering discoveries semantically similar to the references also to be correct; our conclusions do not change.

Finally, to improve the accessibility of our research, we ran the same experiments using gpt-3.5-turbo and flan-t5-xxl as our proposer, and report the results in Appendix Table 6. To show that our conclusions are general and not only apply to synthetically generated texts, we additionally constructed an extension of SYND5 with human-written texts by adapting the NYT dataset from Wang et al. (2023), where each text sample is a New York Times article with a topic and a location label: the topic dimension has 9 different values (e.g., politics, arts) and the location dimension has 10 different values (French, Italy); we then followed the same procedure described in Section 2.2 to create this extension of SYND5, and report our systems' performance in Appendix Table 7. In all experiments, using the validator and the goal improves the performance.

Investigating whether using goals improves relevance on OPEND5. We then investigated whether text-davinci-003 can leverage the goals to propose more relevant hypotheses on more realistic applications in OPEND5. We sampled 100 problems from OPEND5 with distinct goals and randomly sampled 2 hypotheses from text-davinci-003 with/without using goals (see Figure 3), resulting in 400 hypotheses to evaluate. Three authors then rated their relevance based on the rubric in Section 3, while being blinded about which hypotheses were generated with the goal. Our main paper focuses on presenting the evaluations performed by ourselves, since crowdworkers might be noisy and untrustworthy (Veselovsky et al., 2023; Suhr et al., 2021).

We report the results in Table 5. Since this evaluation is subjective, the inter-annotator agreement is only moderate (Kappa= 0 . 56 ); however, we can still robustly conclude that text-davinci-003 can leverage goals to propose hypotheses with higher average relevance rating, since this conclusion can

be independently reproduced by every individual evaluator with p < 10 -8 . To make sure that the same conclusion can be robustly reproduced by external non-authors, we also evaluated the relevance of the hypotheses with Amazon Mechanical Turks, gpt-3.5-turbo , Claude-v1.3 , and gpt-4 . We report the results in Appendix Table 8 and found that our conclusion robustly holds under five different types of evaluators, including expert authors, external crowdworkers, and language models from different companies with different levels of capabilities.

Finally, we conducted similar experiments for the novelty and significance metrics in Appendix 14 and found that they both benefit from using goals as well. In the next section, we present example discoveries on OPEND5 to qualitatively understand what a D5 system can achieve.

6 Qualitatively Analyzing Discoveries and Limitations with OPEND5

To understand the utility and the limitation of a D5 system, we ran it on OPEND5, a set of realistic D5 problems, and analyze the output discoveries qualitatively.

6.1 Producing Discoveries on OPEND5 and Analyzing Them

We ran our D5 system on OPEND5, producing 3296 discoveries in total. However, we do not have enough budget to validate every finding, since estimating V is expensive (Section 3). Therefore, from the remaining 3296 discoveries, we manually selected 21 discoveries that 1) achieve a relevance score of 2 ⃝ , 2) are representative of potential use cases, 3) do not require expert knowledge for Turkers to judge, and 4) are likely to achieve a small p -value with fewer than 200 samples from D val .

We then estimated their validity based on the procedure described in Section 3 by using fewer than 200 samples from the validation split and calculated the p -values, which cost us ∼ $1500 in total on MTurk. Since we are testing multiple discoveries and each of them can be statistically significant merely due to chance, we keep 13 discoveries with V that are significantly non-zero with p -value below 7%, a threshold determined by the Benjamini Hochberg's procedure with a false discovery rate of 10%. In other words, <10% of the discoveries presented are false discoveries in expectation.

We detail 5 of the 13 discoveries in this section, with the remainder in Appendix 18. For each discovery, we report its automated validity score V ′ , the estimated true validity score V , and their respective p values in Table 3.

Understanding political stances and stereotypes in speeches. When comparing presidential speeches on immigrants from Obama to those from Trump, the former ' argues for a path forward to promote the fair and just treatment of immigrants ', while the latter more frequently ' refers to illegal immigrants as criminals '.

Analyzing errors in NLP systems. We fine-tuned a pair of models on two different natural language inference datasets, (a) MNLI and (b) SNLI. To understand their patterns of errors, we defined Corpus A to be the subset of MNLI where a is right and b is wrong, and Corpus B to be where b is right and a is wrong. We found that the latter more often ' has an informal tone, such as slang or colloquial speech '. One possible explanation is that MNLI contains more different genres and hence more informal speeches, causing the former model to perform better on these examples.

Table 3: A subset of discoveries presented in Section 6.1 and their associated estimated validity score V , validity score approximated by a model V ′ , and their respective p -values p ( p ′ ) for the null hypothesis that V ( V ′ ) < 0 under a t-test. We present the full set of 13 discoveries in Table 10.

Analyzing airline customer reviews. We compared the concerns in reviews of the airline Air Canada v.s. its subsidiary, Air Canada Rogue, which is considered a low-price wing of Air Canada. The latter more often ' mentions lack of legroom '.

Analyzing gender differences in self-reported happy moments. Compared to self-reported happy moments written by males, those by females ' mentions children or family ' more often. Caution: misinterpreting this correlation as causation could reinforce societal biases (Section 6.2).

Due to space constraints, we list more examples on analyzing distribution shifts, text clusters, lyric styles, and news headlines in Appendix 18 and their associated V and V ′ values in Appendix Table 10. Across these discoveries, the approximated validity score V ′ has a 71% spearman rank correlation with human rating V (66% for Pearson correlation), thus providing informative yet unreliable signals to practitioners about their validity. We hope that V ′ can better approximate V values in the future as the quality of the validators improve. Finally, future works can collect more open problems, allowing D5 systems to produce more impactful discoveries.

6.2 Concrete Examples in OPEND5 Inform Limitations of D5 Evaluation

We discuss limitations of D5 evaluation in this section using concrete examples from OPEND5.

Our metrics do not evaluate diversity. There are often multiple valid and relevant discoveries, and our system ideally should generate all of them. For example, when comparing low-rating and high-rating reviews to understand what stands out to customers, both ' mentions the hidden fees and poor customer service at the airport ' and ' mentions the airline charging extra for carry-on items ' could be valid discoveries. Our current evaluation does not reward diverse discoveries, and the current system sometimes repeats a discovery using similar paraphrases, e.g., ' mentions the rude and unprofessional attitude of the staff ' and ' mentions the staff being rude and unhelpful '. Future evaluation metrics can take diversity into account.

Interpreting discoveries requires domain experts. We used Turkers' judgment when computing T ( h, x ) to judge the validity of a discovery. However, many discoveries require expert knowledge to interpret properly. For example, it requires medical training to reliably judge whether a self-reported drug-use experience satisfies ' mentions psychedelics, such as LSD and shrooms. '

̸

Correlation = causation. Our metrics currently do not evaluate whether the discovery is causally related to how the corpus pair was generated. For example, when comparing self-reported happy moments from females and males, even if the former corpus has more samples that ' mention children and family ', it does not necessarily imply family plays a more important role in inter-personal relations for females; an alternative hypothesis is that females might mention people in general more often than males do, hence leading to the observation that they mention family more often. Spurious correlations could also sneak into our validity evaluation: for example, if the Turkers implicitly associate female activities as family-related Greenwald & Banaji (1995), then we might falsely make this discovery due to evaluator biases. Future metrics should also consider plausible alternative hypotheses to evaluate causality and control the potential biases from the human evaluators. Additionally, we should treat the discovery from D5 with caution to prevent automating and amplifying societal biases.

We discuss other limitations, such as restricting the discovery to be a single predicate, the biases in authors' qualitative evaluation, and the incomprehensiveness of OPEND5 in Appendix 19.

7 Related Work and Discussion

Inductive Reasoning with NLP Models. Recent works show that language models are capable of inductive reasoning under restricted settings, discovering patterns from a set of text data points and describing them with language (Honovich et al., 2022). Yang et al. (2022) use this capability to induce natural language rules with the format of ' if . . . then . . . '. Zhou et al. (2022) and Ye et al. (2022) use this capability to improve zero/few-shot accuracy by inferring the most likely instruction using input-output example(s) of the target task. Zhong et al. (2022) and Singh et al. (2022) use this capability to discover patterns in datasets, and we improve by building an automatic benchmark and a dataset of open-ended problems and require the discovery to be relevant.

ML models can also perform inductive reasoning in other modalities, such as vision. Hernandez et al. (2021) describes visual features that activate a neuron; Zhu et al. (2022) describes distribution

shifts between the training distribution and the test distribution for images; and Eyuboglu et al. (2022) describes errors made by vision models. We hope future models can perform inductive reasoning in other modalities, such as sound (Aghajanyan et al., 2023) or physical senses (Thomason et al., 2016).

Exploratory Analysis and Automated Discovery. It is not new to automatically discover patterns by learning from empirical data. To list a few classical methods, linear regression analyzes the effect of each real-valued feature by interpreting the learned weights (Draper & Smith, 1998); n-gram models can extract discriminative phrases, thus yielding insights about corpus-level differences (Manning & Schutze, 1999); topic models (Blei et al., 2003) can extract major topical variations across documents, where each topic is represented as a distribution over words; small decision trees can extract interpretable if-then statements (Letham et al., 2015); and an entity embedding model learned on existing relations between entities can predict unseen relations (Socher et al., 2013). In comparison, D5 produces discoveries in the form of natural language predicates, which are interpretable and can express abstract concepts; additionally, it is more directed at the goal, while machine learning classifiers like naïve bayes or linear regression will pick up any discriminative features: Appendix 21 offers a more comprehensive discussion using examples from SYND5. Given the respective strength of D5 and traditional exploratory methods, we envision D5 to serve as a complementary method to traditional methods.

Epistemology. While the process of validating a hypothesis is well-formulated, it is much less wellunderstood how to automatically generate hypotheses and decide what discoveries are meaningful (Shapere, 1964; Heckman & Singer, 2017). Related works in this area have been sparse, among which McGarry (2005) sketches high-level principles for evaluating knowledge discoveries and Ludwig & Mullainathan (2022) proposes to crowd-source hypotheses from MTurk workers. We concur with the perspective of Polanyi et al. (2000) that meaningfulness of a hypothesis cannot be explicitly verbalized with simple logic but is dependent on implicit community norms; therefore, the process of proposing hypotheses should be learned from empirical data (e.g. pre-training, self-training, or human feedback) rather than deduced from a priori analysis of concepts (Quine, 1969). We hope contributions from other domains can provide more empirical data on what discoveries are meaningful, hence guiding our system to produce more important discoveries.

References

The hewlett foundation: Automated essay scoring, 2012. URL https://kaggle.com/ competitions/asap-aes .

The hewlett foundation: Short answer scoring, 2013. URL https://kaggle.com/competitions/ asap-sas .

Ad observer. https://adobserver.org/, 2021. Accessed: 2022-12-30.

Armen Aghajanyan, Lili Yu, Alexis Conneau, Wei-Ning Hsu, Karen Hambardzumyan, Susan Zhang, Stephen Roller, Naman Goyal, Omer Levy, and Luke Zettlemoyer. Scaling laws for generative mixed-modal language models. arXiv preprint arXiv:2301.03728 , 2023.

Roee Aharoni and Yoav Goldberg. Unsupervised domain clusters in pretrained language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pp. 7747-7763, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020. acl-main.692. URL https://aclanthology.org/2020.acl-main.692 .

Mohammad Alali, Shaayan Syed, Mohammed Alsayed, Smit Patel, and Hemanth Bodala. Justice: A benchmark dataset for supreme court's judgment prediction. arXiv preprint arXiv:2112.03414 , 2021.

Akari Asai, Sara Evensen, Behzad Golshan, Alon Halevy, Vivian Li, Andrei Lopatenko, Daniela Stepanov, Yoshihiko Suhara, Wang-Chiew Tan, and Yinzhan Xu. Happydb: A corpus of 100,000 crowdsourced happy moments. arXiv preprint arXiv:1801.07746 , 2018.

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 , 2022a.

Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073 , 2022b.

Miriam Barnum and James Lo. Is the npt unraveling? evidence from text analysis of review conference statements. Journal of Peace Research , 57(6):740-751, 2020.

Alexander Baturo, Niheer Dasandi, and Slava J Mikhaylov. Understanding state preferences with text as data: Introducing the un general debate corpus. Research & Politics , 4(2):2053168017712821, 2017.

Akshay Bhalotia. Yc company scraper. https://github.com/akshaybhalotia/yc_company_ scraper , 2022.

Steven Bird, Ewan Klein, and Edward Loper. Natural language processing with Python: analyzing text with the natural language toolkit . " O'Reilly Media, Inc.", 2009.

David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. Journal of machine Learning research , 3(Jan):993-1022, 2003.

Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. The snli corpus. 2015.

Divy Bramhecha. Poetry Foundation Poems, 2019. URL https://www.kaggle.com/datasets/ tgdivy/poetry-foundation-poems .

Dallas Card, Serina Chang, Chris Becker, Julia Mendelsohn, Rob Voigt, Leah Boustan, Ran Abramitzky, and Dan Jurafsky. Replication code and data for 'Computational analysis of 140 years of US political speeches reveals more positive but increasingly polarized framing of immigration' [dataset]. https://github.com/dallascard/us-immigration-speeches/ , 2022.

Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. Neural legal judgment prediction in english. arXiv preprint arXiv:1906.02059 , 2019.

Sihao Chen, Daniel Khashabi, Wenpeng Yin, Chris Callison-Burch, and Dan Roth. Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims. In Proc. of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) , 2019. URL http://cogcomp.org/papers/CKYCR19.pdf .

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 , 2022.

Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044 , 2019.

Norman R Draper and Harry Smith. Applied regression analysis , volume 326. John Wiley & Sons, 1998.

Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Ré. Domino: Discovering systematic errors with crossmodal embeddings. arXiv preprint arXiv:2203.14960 , 2022.

Yujia Gao, Jinu Jang, and Diyi Yang. Understanding the usage of online media for parenting from infancy to preschool at scale. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems , pp. 1-12, 2021.

Anthony G Greenwald and Mahzarin R Banaji. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review , 102(1):4, 1995.

Ivan Habernal and Iryna Gurevych. Which argument is more convincing? Analyzing and predicting convincingness of Web arguments using bidirectional LSTM. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 1589-1599, Berlin, Germany, 2016. Association for Computational Linguistics. URL http://www.aclweb. org/anthology/P16-1150 .

Kevin Hartman. Advertisement Transcripts from Various Industries, 2019. URL https://tinyurl. com/5w36dwdx .

He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. Decoupling strategy and generation in negotiation dialogues, 2018.

Jibo He. Big Data Set from RateMyProfessor.com for Professors' Teaching Evaluation, 2020. URL https://data.mendeley.com/datasets/fvtfjyvw7d/2 .

Samuel He. Goodbye world: using natural language processing to identify suicidal posts, 2021. URL https://github.com/hesamuel/goodbye_world .

James J Heckman and Burton Singer. Abducting economics. American Economic Review , 107(5): 298-302, 2017.

Evan Hernandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, and Jacob Andreas. Natural language descriptions of deep visual features. In International Conference on Learning Representations , 2021.

Or Honovich, Uri Shaham, Samuel R Bowman, and Omer Levy. Instruction induction: From few examples to natural language task descriptions. arXiv preprint arXiv:2205.10782 , 2022.

Nabil Hossain, John Krumm, and Michael Gamon. " president vows to cut< taxes> hair": Dataset and analysis of creative text editing for humorous headlines. arXiv preprint arXiv:1906.00274 , 2019.

Kaggle. TMDB 5000 Movie Dataset, 2018. URL https://www.kaggle.com/datasets/tmdb/ tmdb-movie-metadata .

Rohit Kulkarni. A Million News Headlines, 2018. URL https://doi.org/10.7910/DVN/ SYBGZL .

Rohit Kulkarni. The Examiner - Spam Clickbait Catalog, 2020a. URL https://www.kaggle.com/ datasets/therohk/examine-the-examiner .

Rohit Kulkarni. Urban Dictionary Words And Definitions, 2020b. URL https://www.kaggle. com/datasets/therohk/urban-dictionary-words-dataset .

Rohit Kulkarni. India News Headlines Dataset, 2022. URL https://www.kaggle.com/datasets/ therohk/india-headlines-news-dataset .

Benjamin Letham, Cynthia Rudin, Tyler H. McCormick, and David Madigan. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics , 9(3), Sep 2015. ISSN 1932-6157. doi: 10.1214/15-aoas848. URL http: //dx.doi.org/10.1214/15-AOAS848 .

Derek Lim and Austin R Benson. Expertise and dynamics within crowdsourced musical knowledge curation: A case study of the genius platform. In ICWSM , pp. 373-384, 2021.

Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. Wanli: Worker and ai collaboration for natural language inference dataset creation, January 2022. URL https://arxiv.org/pdf/ 2201.05955 .

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019.

Zhi Liu. Reuter_50_50 Data Set, 2011. URL https://archive.ics.uci.edu/ml/datasets/ Reuter_50_50 .

Jens Ludwig and Sendhil Mullainathan. Algorithmic behavioral science: Machine learning as a tool for scientific discovery. Chicago Booth Research Paper , (22-15), 2022.

Christopher Manning and Hinrich Schutze. Foundations of statistical natural language processing . MIT press, 1999.

Ken McGarry. A survey of interestingness measures for knowledge discovery. The knowledge engineering review , 20(1):39-61, 2005.

Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016.

Natan Mish. Federal Reserve Governors Speeches 1996 - 2020, 2020. URL https://tinyurl. com/3j2e79a6 .

Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. In ACL , 2022.

Rishabh Misra and Prahal Arora. Sarcasm detection using hybrid neural network. arXiv preprint arXiv:1908.07414 , 2019.

Rishabh Misra and Jigyasa Grover. Sculpting Data for ML: The first act of Machine Learning . 01 2021. ISBN 9798585463570.

Nuno Moniz and LuâC™is Torgo. Multi-source social feedback of online news feeds. CoRR , [Web Link], 2018.

Mickaël Mouillé. Kickstarter Projects, 2017. URL https://www.kaggle.com/datasets/ kemical/kickstarter-projects?select=ks-projects-201612.csv .

Dong Nguyen, Maria Liakata, Simon DeDeo, Jacob Eisenstein, David Mimno, Rebekah Tromble, and Jane Winters. How we do things with words: Analyzing text as social and cultural data. Frontiers in Artificial Intelligence , 3:62, 2020.

Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pp. 188-197, 2019.

Elle O'Brien. iterative/aita_dataset: Praw rescrape of entire dataset, February 2020. URL https: //doi.org/10.5281/zenodo.3677563 .

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155 , 2022.

Verónica Pérez-Rosas and Rada Mihalcea. Experiments in open domain deception detection. In Proceedings of the 2015 conference on empirical methods in natural language processing , pp. 1120-1125, 2015.

Verónica Pérez-Rosas, Mohamed Abouelenien, Rada Mihalcea, and Mihai Burzo. Deception detection using real-life trial data. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction , pp. 59-66, 2015.

Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. Automatic detection of fake news. arXiv preprint arXiv:1708.07104 , 2017.

Denis Peskov, Benny Cheng, Ahmed Elgohary, Joe Barrow, Cristian Danescu-Niculescu-Mizil, and Jordan Boyd-Graber. It takes two to lie: One to lie and one to listen. In Association for Computational Linguistics , 2020.

John P. Pestian, Chris Brew, Pawel Matykiewicz, DJ Hovermale, Neil Johnson, K. Bretonnel Cohen, and Wlodzislaw Duch. A shared task involving multi-label classification of clinical free text. In Biological, translational, and clinical language processing , pp. 97-104, Prague, Czech Republic, June 2007. Association for Computational Linguistics. URL https://aclanthology.org/ W07-1013 .

Michael Polanyi, John Ziman, and Steve Fuller. The republic of science: its political and economic theory minerva, i (1)(1962), 54-73. Minerva , 38(1):1-32, 2000.

Ilan Price, Jordan Gifford-Moore, Jory Fleming, Saul Musker, Maayan Roichman, Guillaume Sylvain, Nithum Thain, Lucas Dixon, and Jeffrey Sorensen. Six attributes of unhealthy conversation. arXiv preprint arXiv:2010.07410 , 2020.

Demand Progress. Statements of Administration Policy, 2022. URL https: //github.com/unitedstates/statements-of-administration-policy# statements-of-administration-policy .

PromptCloud. U.S. Technology Jobs on Dice.com, 2017. URL https://www.kaggle.com/ datasets/PromptCloudHQ/us-technology-jobs-on-dicecom .

WVOQuine. Naturalistic epistemology. Ontological relativity and other essays , pp. 69-90, 1969.

Quora. Quora Question Pairs, 2017. URL https://www.kaggle.com/c/ quora-question-pairs .

Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable questions for squad. arXiv preprint arXiv:1806.03822 , 2018.

Justin Robischon. Wikipedia Movie Plots, 2019. URL https://www.kaggle.com/datasets/ jrobischon/wikipedia-movie-plots .

Allen Roush and Arvind Balaji. DebateSum: A large-scale argument mining and summarization dataset. In Proceedings of the 7th Workshop on Argument Mining , pp. 1-7, Online, December 2020. Association for Computational Linguistics. URL https://aclanthology.org/2020. argmining-1.1 .

Dudley Shapere. The structure of scientific revolutions. The Philosophical Review , 73(3):383-394, 1964.

Chandan Singh, John X Morris, Jyoti Aneja, Alexander M Rush, and Jianfeng Gao. Explaining patterns in data with language models via interpretable autoprompting. arXiv preprint arXiv:2210.01848 , 2022.

Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. Reasoning with neural tensor networks for knowledge base completion. Advances in neural information processing systems , 26, 2013.

Alane Suhr, Clara Vania, Nikita Nangia, Maarten Sap, Mark Yatskar, Samuel Bowman, and Yoav Artzi. Crowdsourcing beyond annotation: Case studies in benchmark data collection. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts , pp. 1-6, 2021.

J Sun. Daily News for Stock Market Prediction, 2017. URL https://www.kaggle.com/ datasets/aaron7sun/stocknews .

Jesse Thomason, Jivko Sinapov, Maxwell Svetlik, Peter Stone, and Raymond J Mooney. Learning multi-modal grounded linguistic semantics by playing" i spy". In IJCAI , pp. 3477-3483, 2016.

Andrew Thompson. All the News 1.0, 2019. URL https://components.one/datasets/ all-the-news-articles-dataset .

Elsbeth Turcan and Kathleen McKeown. Dreaddit: A reddit dataset for stress analysis in social media. arXiv preprint arXiv:1911.00133 , 2019.

Udacity. Armenian Online Job Postings, 2017. URL https://www.kaggle.com/datasets/ udacity/armenian-online-job-postings .

Veniamin Veselovsky, Manoel Horta Ribeiro, and Robert West. Artificial artificial artificial intelligence: Crowd workers widely use large language models for text production tasks. arXiv preprint arXiv:2306.07899 , 2023.

Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. URL https://arxiv. org/abs/2204.07705 , 2022.

Zihan Wang, Jingbo Shang, and Ruiqi Zhong. Goal-driven explainable clustering via language descriptions. arXiv preprint arXiv:2305.13749 , 2023.

Orion Weller and Kevin Seppi. The rJokes dataset: a large scale humor collection. In Proceedings of the Twelfth Language Resources and Evaluation Conference , pp. 6136-6141, Marseille, France, May 2020. European Language Resources Association. ISBN 979-10-95546-34-4. URL https: //aclanthology.org/2020.lrec-1.753 .

Adina Williams, Nikita Nangia, and Samuel R Bowman. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426 , 2017.

Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, and Furu Wei. Language models as inductive reasoners. arXiv preprint arXiv:2212.10923 , 2022.

Seonghyeon Ye, Doyoung Kim, Joel Jang, Joongbo Shin, and Minjoon Seo. Guess the instruction! making language models stronger zero-shot learners. arXiv preprint arXiv:2210.02969 , 2022.

Ruiqi Zhong, Charlie Snell, Dan Klein, and Jacob Steinhardt. Describing differences between text distributions with natural language. In International Conference on Machine Learning , pp. 27099-27116. PMLR, 2022.

Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910 , 2022.

Zhiying Zhu, Weixin Liang, and James Zou. Gsclip: A framework for explaining distribution shifts in natural language. arXiv preprint arXiv:2206.15007 , 2022.

8 Cost for Running the Experiments

For each problem, we ran the proposer for 10 times on average; assuming each prompt to be at most 4000 tokens, we spent around $2.4 for each problem on OpenAI APIs if we use gpt-4 and text-davinci-003 , and the cost would decrease to $0.8 if we use gpt-3.5-turbo . Notice that these estimates are computed based on the prices as of 05/14/2023, and we expect the price to further decrease in the future. We ran the Flan-T5 based validator for ˜ 2 hours on 1 80G A100 GPUs.

The total amount of computational resources spent in this research paper is around $2,500 in terms of OpenAI API and 3,000 hours of compute on A100 GPU with 80G memory.

9 Generation Process of SYND5

The high-level description is in Section 2.2. Here we discuss the procedure that generated SYND5.

We consider three dimensions of differences: topic, genre, and style. For each, we generated 14/9/7 values, e.g., ' celebrity love stories ' and ' sports team recruiting athletes for the topic attribute, ' rap lyrics ' and ' screen play ' for the style attribute, and ' French ' and ' Spanish ' for the language attribute. We then used GPT-4 and the Claude API to synthesize 54K text samples, where for each text sample we sampled a topic, genre, and style randomly, e.g. ' Write a rap about a sports team recruiting athletes in French '. To synthesize a random SYND5 problem, we randomly sampled a distractor dimension (e.g. language) and a target dimension (e.g. topic), and for each dimension we sampled two random values (e.g. English and French for language, sports and art for topic).

For each problem, we sampled 10 texts for corpus A such that all of them satisfy one sampled value for the distractor dimension (e.g. corpus A is entirely in English), and 10 texts for corpus B for to satsify the other distractor dimension (e.g. corpus B is entirely in French). Then we set V fraction of corpus A to satisfy the reference target attribute, e.g. 'is sports-related', and f fraction of corpus B to satisfy the other value for the target dimension (e.g. 'is art-related'). We chose V uniformly at random from [0.6, 0.8, 1]. Finally, we provide k example hypotheses from the target dimension other than the target dimension values for Corpus A and Corpus B, and we chose k from [0, 2] uniformly at random. We then sampled 300 D5 problems in total from this distribution.

10 Robustness Checks for Results on SYND5

Table 4 shows the accuracy of different systems using text-davinci-003 as the judge for semantic equivalence. Table 5 shows the accuracy of different systems if we consider outputs semantically similar to the reference to be correct. Across all setups, we found that the conclusion reached in Section 5 still holds under these robustness checks.

To improve the accessibility of our research, we ran the same experiments with gpt-3.5-turbo and flan-t5-xxl , and report the results in Appendix Table 6. To show that our conclusions are general and not only apply to synthetically generated texts, we additionally constructed an extension of SYND5 with human-written texts by adapting the NYT dataset from Wang et al. (2023), where each text sample is a New York Times article with a topic and a location label: the topic dimension

has 9 different values (e.g., politics, arts) and the location dimension has 10 different values (French, Italy); we then followed the same procedure described in Section 2.2 to create this extension of SYND5, and report our systems' performance in Appendix Table 7. Under all experimental setups, using the validator and the goal improves the performance.

11 Computing Turker Judgement

Scoring. To estimate T ( h, x ) with Turker's rating, where h is a truth predicate of a text sample x , the Turker needs to read h and x and then choose among six options: 'Certainly Yes', 'Likely Yes', 'Neutral', 'Likely No', 'Certainly No', and 'Confusing/Cannot be decided.' For each ( h, x ) pair, we collect responses from three Turkers. To compute the average across them, we collect a list of scores using the following rule: each 'Certainly Yes' would receive a score of 1.00, 'Likely Yes' 0.75, 'Neutral' 0.50, 'Likely No' 0.25, 'Certainly No' 0.00, and 'Confusing/Cannot be decided.' receive two scores of 0.50. We then take the average over all the scores we collected from the Turkers for one h and x . 'Confusing/Cannot be decided.' receives two scores of 0.50 because we want such a response to drag the average rating towards neutral and it has a larger effect than choosing 'Neutral'.

Payment. We adjust the payment for each HIT task based on the number of words they need to read. We pay them approximately 0.001 cent per word, and using the conservative estimate that adults read about 200 words per minute, we pay them around $12 per hour. We spent in total around $5K on this HIT task.

Qualification. We only recruited Turkers who are located in the U.S. Additionally, we designed qualification test with 8 questions; the questions are designed to be easy to answer as long as they have read our instructions below, and we only accepted turkers who made mistakes on at most one questions.

Annotation Instruction. We show our annotation instruction below. We only show examples of choosing 'Certainly Yes', 'Certainly No', and 'Confusing' to encourage the Turkers not to choose neutral ratings. Additionally, we explicitly tried to address Halo effect - where the text does not satisfy a predicate h but satisfies a predicate h ′ that is highly correlated with h . For example, for the text sample x = ' Really love the flight!! ' does not satisfy the predicate h = ' mentions that the breakfeast is good on the plane ', even though it satisfies a highly correlated predicate h ′ = ' likes the flight. '

11.1 Instructions

Below are the same instructions we have shown you during the qualification. Thanks for visiting this page and refresh your memory about the instruction!

Instruction : In this task, you will check whether a TEXT satisfies a PROPERTY

Example 1

Property : mentions a natural scene.

Text : I love the way the sun sets in the evening.

A) Certainly Yes.

B) Likely Yes.

C) Neutral.

D) Likely No.

E) Certainly No.

F) Confusing/Cannot be decided.

Answer. A. sun set is nature-related; if you feel a bit ambivalent, B is also acceptable.

Example 2

Property : writes in a 1st person perspective.

Text : Makima is cute.

A) Certainly Yes.

B) Likely Yes.

C) Neutral.

D) Likely No.

E) Certainly No.

F) Confusing/Cannot be decided.

Answer. E. This text is undoubtedly written in the 3rd person perspetive, so E.

Example 3

Property : is better than group B.

Text : I also need to buy a chair.

A) Certainly Yes.

B) Likely Yes.

C) Neutral.

D) Likely No.

E) Certainly No.

F) Confusing/Cannot be decided.

Answer. F. It is unclear what the hypothesis mean (e.g., what does group B mean?) and doesn't seem related to the text. So F.

Example 4

Property : mentions that the breakfast is good on the airline.

Text

: The airline staff was really nice! Enjoyable flight.

A) Certainly Yes.

B) Likely Yes.

C) Neutral.

D) Likely No.

E) Certainly No.

F) Confusing/Cannot be decided.

Answer. E. Although the text appreciates the flight experience, it DOES NOT mention about the breakfast. So the answer is E.

Example 5

Property : appreciates the writing style of the author.

Text : The paper absolutely sucks because its underlying logic is wrong. However, the presentation of the paper is clear and the use of language is really impressive.

A) Certainly Yes.

B) Likely Yes.

C) Neutral.

D) Likely No.

E) Certainly No.

F) Confusing/Cannot be decided.

Answer. A. Although the text dislikes the paper, it DOES like the writing style. So the answer is A.

12 Prompt to Judge Predicate Similarity

We prompt Claude v1.3 (Bai et al., 2022b) to judge whether the predicated predicate is similar to the reference. We consider a response that leads to a'yes' to be correct when we require the discovery to be semantically equivalent to the reference, and consider a response that leads to a 'yes' or 'related' to be correct when we require the discovery to be semantically similar to the reference.

' Is text_a and text_b similar in meaning? respond with yes, related, or no.

Target:

text_a: {

predicate }

text_b: {

reference }

output: '

13 Relevance Rating with External Non-Authors

To make sure that the conclusion that 'using goal in the context can improve hypotheses relevance' can be robustly reproduced by external non-authors, we also evaluated the relevance of the hypotheses with Amazon Mechanical Turks, GPT-3.5-turbo , Claude-v1.3 , and GPT-4 . We report the results in Table 8 and found that the conclusion still robustly holds.

14 Meaningfulness: Relevance, Novelty, and Significance

Not every valid discovery is meaningful. For example, if the goal is to understand the topical differences between news from 2008 (Corpus A) and news from 2007 (Corpus B), the discovery that Corpus A ' contains news from 2008 ' is completely valid by definition but meaningless, since it provides only trivial information and is irrelevant to the goal of understanding topical differences.

McGarry (2005) surveyed a list of desirable properties for discovery, and we condensed them into three submetrics to rate how meaningful a discovery is based on the exploration goal: 1) relevance, 2) novelty, and 3) significance. We evaluate these independently of validity and assume that the discovery is already valid. For example, the discovery that 'something can travel faster than light' is meaningful if true, even though it is highly implausible.

We rate each submetric with 0 ⃝ , 1 ⃝ , or 2 ⃝ , where higher is better. We show the evaluation instructions below and present our rating on text-davinci-003 proposed hypotheses.

14.1 Evaluation Instructions

Relevance. How relevant the discovery is to the goal. For example, suppose we were a student comparing essays rated as convincing vs. not convincing to figure out what writing style is convincing. Then:

The discovery ' write in first person ' is directly related to the writing style, so we rate it 2 ⃝ .

The discovery ' use the word 'I' ', is not exactly a writing style, but can still inform the relevant underlying principle of ' write in first person ', so we rate it 1 ⃝ .

The discovery ' argue for abortion ' does not tell us about the underlying writing style, so we rate it 0 ⃝ .

Novelty. The difficulty of generating the discovery, e.g. can we think of the discovery in 5 minutes with the goal but without looking at the corpora? For example, suppose we were an airline manager

trying to find improvements to the flight experience, and we were comparing negative reviews vs. positive reviews. Then:

The discovery ' contain more negative language ' is almost certain for negative reviews, so we rate it 0 ⃝ .

The discovery ' complain about the crew members ' is not entirely novel, but is not tautologically true and hence requires confirmation, so we rate it 1 ⃝ .

The discovery ' mention a language barrier with the crew members ' is specific and hard to think of without looking at the data, so we rate it 2 ⃝ .

Note that our evaluation is 'blinded to the samples': we still consider a discovery novel as long as it is hard to think of before looking at the corpora, even if it might be easy to think of after looking at the corpora. For example, the physical law that F = ma is easy to observe if we have collected and plotted the data on acceleration, mass, and force; however, it might be difficult to think of before we see any such data, so we consider it novel.

Significance. Given the exploration goal, how beneficial is it to learn the discovery for the first time? For example, suppose we were an Amazon retailer trying to figure out what customers like and dislike about my product based on negative reviews and positive reviews. Then:

The discovery ' accuses the team pushing out a bad product ' is not significant since it cannot direct the retailer to improve the product, so we rate it 0 ⃝ .

The discovery ' asks for a more durable product ' gives some hints about how to improve the product, but isn't sufficiently helpful on its own, so we rate it 1 ⃝ .

The discovery ' says the wrench is missing ' can lead to concrete actions for improvement, so we rate it 2 ⃝ .

14.2 Goal Leads to More Meaningful Hypotheses

Table 9: Left. For each metric, we report the average rating on hypotheses generated with or without using the exploration goal, and find that the former performs better. Middle. The inter-annotator agreement rate averaged across pairs of author evaluators, measured by Kappa and Spearman rank coefficient; we find substantial correlations between evaluators across all these subjective metrics, with relevance > significance > novelty. Right. We compute the p -values for the null hypothesis that 'with-goal and no-goal result in the same performance'. The p of avg column reports the p -values after we average the ratings from all evaluators, while the 'worst p of ind' column takes the max of all p -values based on ratings of individual evaluators. Overall, the conclusions are statistically significant and they can be robustly reproduced across individual evaluators.

Compared to Zhong et al. (2022), we added the exploration goal to our prompt when generating hypotheses. Does this improve the quality of the proposed hypotheses? To investigate this, we sampled 100 problems from OPEND5 with distinct exploration goals and randomly sampled 2 hypotheses from GPT-3 with and without using exploration goal (see Figure 3), resulting in 400 hypotheses to evaluate. Three authors then rated their meaningfulness based on the three metrics defined in Section 3, while being blinded about which hypotheses were generated with the exploration goal.

The results are shown in Table 9. We found that, when prompted with the exploration goal, GPT-3 on average proposes more relevant, novel, and significant hypotheses; additionally, it proposes hypotheses with ratings higher than 0 ⃝ 31%/21%/28% more often in terms of relevance/novelty/significance. Since this is a subjective evaluation, the Kappa inter-annotator agreement is only moderate, ranging from 0.37 to 0.56. However, we can still robustly conclude that the model can propose more meaningful hypotheses when conditioned on the goal: we calculate the p -values for the null hypothesis that with-goal and no-goal have equal performance, and we find p -values to be highly significant and robust across evaluators, for all three submetrics.

15 Full Pipeline of the Proposer

We present the full details of how we generated the hypotheses with the language model. The process roughly contains four stages: 1) obtaining representative samples for each corpus, 2) sampling hypotheses from GPT-3, 3) rewriting hypotheses, and 4) optionally plugging in example hypotheses.

Obtaining representative samples. This step is the same as Zhong et al. (2022), and we borrow the related text from that paper for the reader's convenience. Since D res A and D res B might overlap significantly, random samples from D res A and D res B might not be representative and informative enough for GPT-3 to notice the differences between the two distributions. Therefore, we choose samples that are representative of their differences. To find those samples, we fine-tune RoBERTa-Large Liu et al. (2019) to predict whether each sample comes from Corpus A or Corpus B and keep the top-p percentile samples with the highest confidence. Next, we take samples from the topp percentile to prompt GPT-3.

Selecting samples to prompt GPT-3. We randomly select S = 25 samples from the top-5 percentile from Corpus A and Corpus B to prompt GPT-3 to propose the hypotheses, using the template shown in Figure 3 left. We require the length of the prompt to be at most 3,200 GPT-3 tokens (the max window size for GPT-3 text-davinci-003 is 4096) and gradually decrease the number of samples S in the prompt until the prompt length is less than 3,200; additionally, we truncate each text samples to at most 256 GPT-3 tokens. Finally, to prevent GPT-3 from proposing hypotheses that reflect simple lexical correlations that can be detected with unigram models, e.g., ' uses the word 'hey' more often. ', we incrementally construct the subset of samples for Corpus A and Corpus B such that at any time of the construction, no single word can appear 0 . 25 S times more often in one corpus than the other. We repeat the same process for the top-20 and top-100 percentile until we obtain 60 hypotheses.

Rewriting hypotheses with GPT-3. As mentioned in Section 6.2, the hypotheses generated by GPT-3 are frequently statements about the corpus, while the validator requires the hypothesis to be a predicate on individual text samples. For example, when comparing definitions that people like from UrbanDictionary.com to other definitions, the hypothesis that the former ' is more likely to include slang or colloquial terms. ' is a statement about a collection of text samples, rather than a predicate on an individual sample. T ( h, x ) is undefined in this case, since it does not make sense to check whether a single text sample is more likely to include slang. Ideally, we want to detect these comparison statements and automatically remove the comparatives, e.g., rewrite it to ' includes slang or colloquial terms. '.

To detect and remove the comparatives from the hypotheses, we tag the part of speech for each word in the hypotheses using the NLTK package (Bird et al., 2009) and check whether any tag is JJR or RBR . If a hypothesis indeed contain theses tags, we prompt GPT-3 to rewrite the hypothesis. We show an example prompt in Figure 4.

Plugging in example hypotheses (optionally). We can also add a few problem-specific example hypotheses to the prompt to elicit more relevant hypotheses, and we do so by adding them to the 'formatting instruction' part in the prompt used to propose hypotheses Figure 3. In OPEND5, we provided example hypotheses for each problem to steer our system to generate more meaningful discoveries; we produced the example hypotheses by prompting GPT-3 to generate a few hypotheses and selecting the meaningful ones from them.

For the reported discoveries in Section 6.1, we confirmed that they are unambiguously different from our provided hypotheses; otherwise, the system might have produced the discoveries by copying the provided hypotheses. We did not use the example hypotheses in Section 5 to test GPT-3's zero-shot understanding of the goal.

16 Collecting Data to Fine-tune the Validator

Here we provide a high-level description of how the data was collected. For each problem in OPEND5, we used our proposer to produce a list of hypotheses. We automatically judged each hypothesis on a subset of samples from the research split using GPT-3 text-davinci-002 (Ouyang et al., 2022), Flan-T5 (Chung et al., 2022), and a model trained with RLHF from Bai et al. (2022a). We created the input

Removethecomparatives.RemovementionofGroupAorBiftheyappear

Input:containlongersentencesthanthosefromgroupB Output:containlongsentences

Input:supportsDACA more strongly Output:supportsDACAstrongly

Input:usesthehashtag#AllLivesMattermoreoften Output:uses thehashtag #AllLivesMatter

Input:ismorelikelytocontaingrammaticalerrors Output:containgrammaticalerrors

Input:soundshappierthanthosefromGroupB Output:soundshappy

Input:ismorelikelytoincludeslangorcolloquial terms Output:includesslangorcolloquialterms

distribution for training by combining and equally weighting the following 3 × 2 = 4 distributions: the subset of ( h, x ) pairs that GPT-3/Flan-T5/'RLHF' considers Yes or No to be the most likely answer. We then collected averaged turker ratings for in total 3138 ( h, x ) pairs and used them to fine-tune Flan-T5 to create the validator (Chung et al., 2022).

To test cross problem generalization capability of our D5 system, whenever we applied our D5 system to a problem in OPEND5 in Section 6.1, we used a validator that is NOT fine-tuned on the ( h, x ) pairs from this problem. We achieved this by keeping track of which problem each ( h, x ) pair comes from and split all the ( h, x ) pairs into three folds based on the problems; whenever we applied our D5 system to a problem, we used the validator trained on the two folds that do not contain this problem.

Samples from Corpus A

+

Samples from Corpus B

+

- hypothesis1

- hypothesis2

- hypothesis3

- hypothesis4

- hypothesis5

LM

hypothesis1

+

Sample X from Corpus A

LM

100%

hypothesis1

+

Sample Y from Corpus B

LM

0%

Judge

hypothesis1

on all individual

samples from Corpus A and Corpus B

……

Compare how oJen each hypothesis is

true on Corpus A compared to Corpus B

Propose hypotheses based on the problem context

and  some samples from Corpus A and Corpus B

Corpus A Corpus B Diff Sound?

hypothesis1

90%

0%

90%

Yes

hypothesis2

100%

100%

0%

No

10%

15%

-5%

No

[Other hypotheses not included for brevity]

17 What Discoveries Did we Choose to Present

Our system in total produces 3296 discoveries on OPEND5. However, we do not have enough budget to validate every finding, since estimating V is expensive (Section ?? ). Therefore, from the remaining 3296 discoveries, we manually selected 21 discoveries that 1) the authors think are relevant enough, 2) are representative of potential use cases, 3) do not require expert knowledge for Turkers to judge, and 4) are likely to achieve a small p -value with fewer than 200 samples from D val A and D val B . We then estimated their validity based on the procedure described in Section ?? by using fewer than 200 samples from the validation split and calculated the p -values. 3 Since we are testing multiple discoveries and each of them can be statistically significant merely due to chance, we keep 13 discoveries with V that are significantly non-zero with p -value below 7%, a threshold determined by the Benjamini Hochberg's procedure with a false discovery rate of 10%. In other words, fewer than 10% of the discoveries presented are false discoveries in expectation.

18 More Example Discoveries on OPEND5

Analyzing errors in NLP systems. We considered the task of perspectrum classification (Chen et al., 2019), which has the following instruction: 'given a perspective and a claim, classify whether the given perspective supports or undermines the claim. If the perspective could possibly convince someone with different view, it is supporting, otherwise it is undermining.' We considered two fewshot learning systems: GPT-3 Instruct Curie (Ouyang et al., 2022) and Tk-Instruct-11B (Wang et al., 2022). We focused on the perspectives where the ground truth label is undermining, and compare the following two corpora: Corpus A - the set of perspectives where Curie correctly classifies the input as undermining but Tk-11B is wrong, and Corpus B - the set where TK-11B is correct while Curie is wrong. We found that Corpus B more often ' Uses language that is positive or uplifting ' ( V ≈ 0 . 12 , AUCROC ≈ 0.67). One possible explanation is that Curie made many mistakes by misinterpreting undermining as a label for negative sentiment rather than a logical relation between the claim and the perspective.

Comparing lyrics from different eras. Compared to lyrics from the 70s, those from the 80s more often ' references violence or aggression ' ( V ≈ 0 . 06 , AUCROC ≈ 0.58).

Describing distribution shift. We compared the premises from the SNLI dataset and MNLI dataset, and the former ' involves physical activity, such as walking, playing, climbing, or biking ' ( V ≈ 0 . 13 , AUC-ROC ≈ 0.64). One possible explanation is that SNLI is based on image captions.

Comparing discussion topics between bots and human users. We compared the topical differences between tweets identified as written by bots vs. human users on Twitter, and our system finds that the bots more often ' contains keywords related to business, finance or trading ' ( V ≈ 0 . 08 , AUC-ROC ≈ 0.61). One possible explanation is that bots are frequently used to generate finance-related scams.

Identifying temporal differences in news headlines. We compared headlines published by ABC news across different years. Compared to 2014, headlines from 2010 ' mention disasters and crimes, such as plane accidents and assaults ' more often ( V ≈ 0 . 03 , AUCROC ≈ 0.53). Compared to year 2019, year 2020 more often ' discusses coronavirus-related topics ' ( V ≈ 0 . 21 , AUCROC ≈ 0.65).

Describing text clusters. We present two example descriptions for text clusters. One from Wikipedia: ' references pop culture, such as movies, books, and television shows .' ( V ≈ 0 . 21 , AUC-ROC ≈ 0.73); one from PoetryFoundation.com: ' uses vivid imagery and metaphors to convey a feeling ' ( V ≈ 0 . 09 , AUC-ROC ≈ 0.65).

19 Limitations and Future Work

We still face many challenges in building a broadly useful system. We describe technical challenges that machine learning researchers can tackle in Appendix 19.1 and organizational challenges that require domain experts in Appendix 19.2.

19.1 Engineering Challenges

Hypotheses about the corpora might not be appropriate predicates on individual samples. When comparing highly rated definitions from UrbanDictionary.com to others, our system generates the hypothesis that the former ' is more likely to include slang or colloquial terms. ' This is a statement about a collection of text samples, but the validator requires the hypothesis h to be a predicate on individual text samples x . To address this, we used GPT-3 to automatically remove comparatives from the hypotheses, e.g. rewriting the hypothesis above to ' include slang or colloquial terms .'

However, some versions of this problem were harder to remove. For example, when comparing reviews from American Airlines (AA) flights and Delta Airlines to understand which aspects of each airline are doing better/worse, the proposer generated the hypothesis ' mentions American Airlines' staff being unfriendly and unhelpful '. Interpreted literally, this hypothesis can only be true on the corpus of AA reviews, since it presupposes the review to be about AA. The correct predicate for use on individual samples should instead be ' mentions staff being unfriendly and unhelpful ' (without the words ' American Airlines ''). Therefore, future systems should explicitly convert corpus-level statements to their corresponding correct predicates, and the metrics should evaluate whether the validity of the predicates implies the corpus-level statements.

Beyond truth predicates. Our work requires the discovery to be a truth predicate that maps a text sample to a truth value. However, scientific discoveries can be arbitrary natural language expressions; extending to more flexible expressions requires a significant redesign of our system and evaluation framework. Some more feasible near-term extensions include 1) allowing natural language expressions that map from text samples to real values, e.g., 'how polite the sentence is compared to other samples from the corpora' or 2) using additional logical forms to combine individual truth predicates; e.g., learn a shallow and interpretable decision tree where each split point is a natural language predicate.

Beyond corpus-level differences. Our work focuses on describing corpus-level differences and validates a discovery by comparing how often it is true on each corpus. Future work can consider other ways to validate a discovery: for example, suppose each text sample is associated with a continuous target variable, we can validate whether a discovery is more likely true if the target variable is large.

Investigating sensitivity towards prompt format. In this paper we hand-crafted the prompt for the proposer and manually annotated the exploration goals on our own for OPEND5. However, due to budget limitation, we have not investigated how sensitive is our D5 system towards prompt formatting and paraphrasing, or whether the performance could have been improved with better prompts. Future works can investigate more in this research direction.

Clarifying a discovery. Some discoveries seem to have clear meanings on the surface, but they become ambiguous when we judge them on individual text samples. For example, judging whether a text sample h = ' mentions people ' seems like an unambiguous task a priori; however, it is unclear whether it is true on the sample x = ' I woke up this morning. ', since the ' people ' in h is a plural form, while x only mentions one person ' I '. Future work can use a language model to automatically clarify the meaning of a hypothesis and make it more specific, e.g., rewrite h as ' mentions one or more humans. '

̸

Correlation = causation. Like other tools that rely on correlations to analyze patterns in data (e.g., linear regression), our system cannot establish causal relations either. For example, when comparing self-reported happy moments from females and males, even if the former corpus has more samples that ' mention children and family ', it does not necessarily imply family plays a more important role in inter-personal relations for females; an alternative hypothesis is that females might mention any other people more often than males, hence leading to the observation that they mention family more often. Future work can use language models to propose what control hypothesis to test.

Decreasing the cost of validation. As alluded to in Section 3, estimating V is extremely expensive as it requires a lot of human labor. Future work can consider an importance sampling procedure that uses ˆ T as a proposer to improve the sample efficiency of estimating V .

Training a better proposer. We developed a self-supervised learning algorithm to propose more valid hypotheses. However, it does not take into account the meaningfulness metric, and it is unclear how to manage its trade-offs with validity if they exist. We look forward to future works that can train a better proposer with as minimal supervision as possible.

Combining Meaningfulness and Validity Metrics. To simplify evaluation, we assumed meaningfulness to be independent of the magnitude validity V . Such an assumption allows us to directly evaluate hypotheses that are not necessarily valid but is also limiting for evaluating the final discoveries: for example, for that 2008 ' discuss economy ' more often than 2007, it would be way more significant if V = 0 . 99 compared to V = 0 . 0000001 . Future works can propose better metrics that do not assume that validity and meaningfulness are independent.

Extending to Non-English Language OPEND5 is currently annotated with English goals and most of the corpora are in English. Future work can consider extending this to other languages.

19.2 Organizational Challenges

As discussed in Polanyi et al. (2000), it requires implicit community norms rather than explicit deductive logic to decide what counts as good research results; to guide our system to produce truly important discoveries, our system needs feedback from researchers who work in the domain of interest. However, except for machine learning, the authors do not have research expertise in most of the domains listed in Figure 2. We look forward to future contributions from other domains and list concrete directions below.

What problems to solve? We generated the problems in OPEND5 by reading relevant papers and guessing what domain experts might care about. However, our guesses can be inaccurate. Future works can directly gather problems from domain experts to reflect the actual usage of our system.

How to interpret a discovery? We asked for Turker's judgment to compute T ( h, x ) . However, many hypotheses require expert knowledge to interpret properly. For example, only law experts can reliably judge whether a contract x satisfies the predicate h ' contains a license grant that is irrevocable .' Domain experts are needed to evaluate the validity of a discovery and supervise the validator.

What discoveries are meaningful? Our work developed the evaluation instructions to approximately evaluate what hypotheses are meaningful. However, just as no one can become an outstanding peer reviewer simply by reading the review guideline, we do not consider it feasible to provide a gold evaluation simply by reading our instructions. Whether a discovery is meaningful depends heavily on implicit community norms, and we hope domain experts can provide better evaluation and training signals for our system.

20 Self-Supervised Learning with Open-Ended Problems: A Proof of Concept

Since the problems in OPEND5 are open-ended, our system could potentially produce discoveries with higher validity scores than our current system. Therefore, we design a self-supervised learning algorithm to improve an LM's ability to propose more valid hypotheses, using the principle that it is easier to validate a discovery than to generate one .

Algorithm. Suppose we are given a set of problems for training and an initial language model m init . Our goal is to automatically generate a set of prompt -completion pairs to fine-tune m init so that it can propose hypotheses that are more valid. To generate a prompt , we randomly sample a problem and create a proposer prompt following the procedure in Section 4.1. To generate the desired completion given a prompt, we sample multiple hypotheses from m init, approximate their V ′ score on the samples in the proposer prompt with the same language model m init (Section 4.2), and select the highest scoring hypothesis. Finally, we use the prompt-completion pairs to fine-tune m init .

However, since we cannot fine-tune instruction-tuned GPT-3, we can only experiment with Flan-T5 (Chung et al., 2022), an open-sourced instruction-tuned model that might only work well for easier 'mini-problems'. As a proof of concept, we tested our algorithms for describing groups of four samples, where each group comes from a text cluster. As an overly simplified example, we will give the LM the prompt ' Group A: 1. dog 2. cat 3. pig 4. cow. Group B: 1. phone 2. laptop 3. desk 4. cup ' as an input and the LM can output ' mentions an animal ' as a hypothesis.

Data. We created 33 corpora by merging all corpora in OPEND5 with the same domain, and automatically generated 4503 text clusters using RoBERTa embeddings (Aharoni & Goldberg, 2020). We focused on clustering because it can automatically generate a large amount of semantically coherent groups of samples. To create a pair of four samples, we randomly sampled a corpus, sampled two clusters within that corpus, and took four random samples from each cluster. To test

cross-corpus generalization, we reserved 28 of the 33 corpora to create mini-problems for evaluation, using the rest for training. We used Flan-T5 (Chung et al., 2022) as m init and sampled hypotheses with a temperature of 0.8. For training, we sampled 30,000 mini-problems and selected the best of eight hypotheses generated by m init as the target completion; for evaluation, we sampled 200 mini-problems to calculate V with Turkers and 1500 mini-problems to calculate V ′ automatically.

Results. We evaluated randomly sampled hypotheses from the language model before and after self-supervised training. The automated 'self-evaluation' validity score V ′ improves substantially from 0.22 to 0.37, and the 'true' validity score V according to Turker evaluation improves from 0.07 to 0.10, with a p -value of 0.02. This result provides preliminary evidence that our algorithm (or similar variants) could be applied to a large set of problems to improve the validity of the hypotheses; we expect future validators to simulate human judgments better, hence decreasing the approximated gap of improvement between V and V ′ .

21 Comparing D5 to Naïve Bayes

We qualitatively compare the discovery generated by our D5 system to the top-5 unigram features extracted by Naive Bayes, a traditional exploratory analysis method. The Naive Bayes method is effective when the target difference can be saliently reflected by individual words. For example, ' yo ' implies a rap genre, ' die ' implies a language of Deutsch, and [' rank ', ' higher ', ' univeristy '] hints at the topic of ' college ranking changes '. Additionally, compared to black-box neural networks, such a method is fully interpretable.

In comparison, D5 can directly generate a semantically coherent description for the target difference, saving users' time to guess the underlying correlation by inspecting the top unigram features. Additionally, it can capture differences that are hard to detect at a word level; for example, ' the genre of biblical scripture ' is mainly reflected in its sentence structure rather than individual words. Finally, D5 only describes goal-related differences, while Naïve Bayes picks up on any discriminative feature; for example, when identifying the topical differences between a English and a Deutsch corpus, Naïve Bayes fails catastrophically and only picks up common determiners such as ' the ' or ' die ' instead of topic words, since they are the most useful feature at telling which sample comes from which corpus. Given the respective strength of D5 and traditional exploratory methods, we envision D5 to serve as a complementary method to traditional methods.

22 Annotation Interface to Collect Human-Generated Hypotheses

(This section describes an interesting research direction we did not have time to fully pursue.)

Task. To fine-tune the language model to propose better hypotheses and perform validation more accurately, we also designed an interface to collect human annotations earlier in the project. In this annotation task, the annotators see five text samples from each of the two corpora; they then write one or many natural language predicate(s) that describe how samples from the two groups are different and choose which text samples satisfy each predicate the annotator has written. Since it is challenging for humans to identify systematic differences between even groups of five sentences, we made the task easier for them by

we chose the representative samples from each corpus to form the two groups of samples, similar to the process in Section 15, and

we highlighted subspan of the text samples that are informative for how the two corpora differ. For example, if Corpus A is sports related while Corpus B is entertainment related, we hope to highlight sports-related words like 'basketball'. To automatically identify the text spans to highlight, we fine-tuned RoBERTa to classify whether a sample comes from Corpus A and Corpus B, used the SHAP library to calculate how much each text span influences the classifier's decision, and highlighted the text spans based on the influence.

A screenshot of the annotation interface can be seen in Figure 6.

Preliminary Results We performed initial experiments on text clusters formed on the wikitext-2 dataset (Merity et al., 2016). We asked the authors to write hypotheses for 30-50 samples and then compare the results with GPT-3 generated hypotheses. We found that human annotators were able to

1) Write hypotheses

2) Select most representative

samples for a written hypothesis

3) Commit hypotheses and

show SHAP highlights

SHAP highlight view

4) Write any additional

hypotheses after seeing

highlights

GroupA

Group B

口

aticket."

QuayorQuay Brothersheyare influential stopmotion

animators.Theyare also therecipients ofthe199 Drama

American identical twin brothers betterknown as the Brothers

play *The Chairs".

DeskAward for OutstandingSet Designfortheirwork onthe

口

wanhcawdwwilefe tt

losses and selthe comeny.

cottagesandfambulldng.Athe201Cesusthe

Landican () isa hamlet on the Wiral Peninsula, England it s

situated onthe outskirts of Birkenhead,near to Woodchurch

and the M53 motorway. Landican consists of a smallgroup of

community had a population of only 20.

theprblfneteastht

The authors conclude that the factors that explain variation in

varilation n the probablty of surviving.

psychedelicrockbndKingGizrd&tezad Wizard.was

ATOsnthUdStsdlRecing

Murder of the Universe is the tenth studio album by Australian

asedon23June2017byFlightessRecndsnustll

the United dom. It is the second of five albums set to be

released in2017.

口

The agency’srole is to develop and maintaln a strategic plan

that is responsive to the changing needs of the agency and the

nation.

d

for the UniversityofTeen

leoot ago puno pn au u sa

mlericanfootballrunningback ieNheDenverBroncosof

Aseauqpapsempu

口

ofthemsberetdasdicetrcntinuus

basicquestion is whether the final stage of development

口

abbrevlaedAlphCaCasthbrghtsine

Canopus ().also designated Alpha Carinae（a Carinae,

variable.

southerm contellation of Carina and the second-brightest

magnitude is 0.74, and it has an absolute magnitude of

-5.71.

Selecting top 3 and bottom 3 for hypothesis: 1

Hypothesis format

Each sentence from Group B

Talks about a famous person or a famous

Add additional hypothesis

Show Instructions

Commit Hypotheses and show highlight

Refresh Question

Submit

GroupA

Group B

Top

Bottom

Top

Sottom

口

口

StephenandTimothyQuay（bomJune 17.1947eAmerican

identicalwinbrothersbeterkownastheBrothersQuayor

are also the recipients ofthe 19s Drama DeskAward for

be forced tocutour

cottagesandfarmbdingActhe2001Censusthecmunity

LandicanletnWlPlaEnand

and the M53 motoway Landicanconsists of a small group of

situated ntheoutskirts fBirkeneadeartoWoodchuch

had a population of only 20.

iariacionin

authors con

nclude that the

Dyingare

actors that explain variationi

Hmeas

those that explain

口

Unitedngottsond ofbsse

psychedelicrockbandKingGizad&theizardWzards

Murderof the Universeisthe tenthstudo

edn3Jue2017byFigessecdsiusl

RecordsntheUiedStaesndHevenlyecrdingsnte

ibumbyAustralian

LTot u paseaj

thatisrepnstehanneedstheagecande

nation,

rolestodevelopandmantainsrtegclan

口

theUieTenddbaa

ChfsittdruntF

NationlFbalLgueHplayeolefoblor

Americanfootballrunningbackforthe DerverBroncosofthe

Canopus (1,also designated Alpha Carinae (α Carinae,

abbreviatedAlphaCarCahebrigststarine

inthniyusCa

southecnlanfCinantcndbriest

0.74, and it has an absolute magnitude of 5.71.

Selecting top 3and bottom 3 forhypothesis:1

1.Hypothesis format

Hypothesis

Each sentence from Group B

Talks about a famous person or a famous

New Hypotheses for highlight:

Hypothesis fomat

Hypothesis

Compared to sentences from Group B, each sentence from Group A

Enter yourhypothesis here.

Add additional hypothesis

Show Instructions

Commit Hypothe

RefreshQuestion

Submit

write 2-4 valid hypotheses per pair of text groups, while GPT-3 text-davinci-003 was able to generate 4-6. Out of the valid generated hypotheses, approximately a third were variations on another valid hypothesis. The number of times humans were able to write a hypothesis that GPT-3 was unable to generate was around a third of the samples, while GPT-3 was able to generate a novel hypothesis humans have not thought about before in nearly every single text corpora. Given that GPT-3 is close to our author's ability to write hypotheses, we estimated that we would not be able to fine-tune T5 to propose better hypotheses with human annotations, and hence gave up on this research direction.

23 Datasets

Many of our datasets come from the following sources: the Computational Models of Social Meaning class from Columbia University http://www1.cs.columbia.edu/~smara/teaching/S18/ , the ACL Anthology https://aclanthology.org , , and Kaggle datasets with an NLP tag. https: //www.kaggle.com abc-headlines . We collect headlines published by ABC news, an American news company from Kulkarni (2018). ABC headlines are directly downloaded from Harvard Dataverse. The year is extracted from the publication date field. Samples are constructed from the headline text. The data is downloadable from https://doi.org/10.7910/DVN/SYBGZL with license CC0 1.0.

ad-transcripts . We collect ad scripts from a variety of industries from Hartman (2019). Ad transcripts are directly downloaded from Kaggle. The top eight industries by frequency are selected. Newlines are replaced with spaces. The dataset is downloadable from https://www.kaggle.com/ datasets/kevinhartman0/advertisement-transcripts-from-various-industries with license CC0 Public Domain.

admin-statements . We collect statements of administration policy from American presidents from Progress (2022). Administration statements are extracted from a collection hosted on GitHub. Extraneous symbols are removed and samples are split by paragraph. The dataset is downloadable from https://github.com/unitedstates/ statements-of-administration-policy#statements-of-administration-policy and origin files have a Creative Commons Attribution 3.0 License.

ai2-natural-instruction . We collect a learning-from-instructions dataset released by the Allen Institute for AI from Mishra et al. (2022). Natural instruction tasks are directly downloaded without modification. The dataset is released under an Apache-2.0 license.

airline-reviews . We collect reviews of airlines collected from the review website Skytrax. Airline reviews for airlines, airports, and seats are downloaded from a public GitHub repository. Names of aircraft, airlines, countries, and traveler types are standardized. Ratings of 1, 4, or 5 on a scale of 5, and 1, 5, 8, or 10 on a scale of 10 are kept. This dataset can be downloaded via https://github.com/quankiquanki/skytrax-reviews-dataset .

aita . We collect posts on the 'Am I The Asshole' Subreddit, an online forum people ask others whether they were in the wrong from O'Brien (2020). Posts from r/AmITheAsshole are downloaded from a praw scrape of Reddit. Topic areas are chosen based on common themes in posts and coarsely defined based on manual keywords. Each post can belong to multiple topic areas. The dataset can be downloaded at https://doi.org/10.5281/zenodo.3677563 .

all-the-news . We collect news articles collected from various outlets between 2015 and 2017 from Thompson (2019). News articles are downloaded directly from the Components website. The titles are used as text samples.The dataset can be downloaded at https://components.one/datasets/ all-the-news-articles-dataset .

amazon-reviews . We collect Amazon reviews collected from various product categories from Ni et al. (2019). Amazon reviews are downloaded from a 2018 crawl of the website. The first 100,000 review texts are treated as the text sample. The dataset can be downloaded at https: //nijianmo.github.io/amazon/index.html .

armenian-jobs . We collect job postings in Armenia from Udacity (2017). The Armenian job postings dataset is downloaded from a snapshot on GitHub. Different IT jobs are manually coded and time intervals are defined in order to balance sample availability. The dataset can be downloaded at https://www.kaggle.com/datasets/udacity/armenian-online-job-postings .

boolq . We collect a reading comprehension dataset of yes/no questions from Clark et al. (2019). Boolean questions are downloaded directly as is. The dataset can be downloaded at https:// github.com/google-research-datasets/boolean-questions with license CC-SA-3.0.

clickbait-headlines . We collect headlines across time from the Examiner, a clickbait news site from Kulkarni (2020a). The Examiner headlines are directly downloaded from Kaggle. The year is extracted from the publication date field. Samples are constructed from the headline text. The dataset can be downloaded at https://www.kaggle.com/datasets/therohk/examine-the-examiner , with license CC0: public domain.

convincing-arguments . We collect arguments on a variety of topics annotated for convincingness from Habernal & Gurevych (2016). Annotated arguments are downloaded from the GitHub repository. Arguments are sorted by rank. The bottom 400 are treated as 'unconvincing', the top 200 are treated as 'convincing', and the next 200 are treated as 'somewhat convincing.' The dataset can be downloaded at https://github.com/UKPLab/acl2016-convincing-arguments , with license CC-BY 4.0.

craigslist-negotiations . We collect dialogue from Craigslist negotiations, an online seller platform from He et al. (2018). Craigslist negotiations are downloaded from Huggingface. Sequences which contained a 'quit' intention or 'reject' intention are categorized as failures; those which contained an 'accept' intention are categorized as successes. The mid-price is defined as the mean

price of the items sold. Within each category, the items are sorted by mid-price. The top half is treated as high-price and the bottom half is treated as low-price. This dataset can be downloaded at https://huggingface.co/datasets/Hellisotherpeople/DebateSum with MIT license.

debate . We collect evidence compiled for American competitive policy debate, published online by debate camps from Roush & Balaji (2020). The train split is downloaded from Huggingface. For each sample, we use the abstract as the text. Arguments are categorized by type, debate camp of origin, and topic/specific argument. For topics, we use domain knowledge to list relevant keywords for each topic and include any sample with a file name that includes any keyword. A single sample can belong to multiple topics. This dataset can be downloaded at https://huggingface.co/ datasets/Hellisotherpeople/DebateSum with MIT license.

dice-jobs . We collect American technology job postings on dice.com from PromptCloud (2017). Job postings are downloaded from Kaggle. Posts from the six most popular companies are categorized by company. We remove miscellaneous characters and blank descriptions. We additionally apply our splitting procedure to reduce description length. This dataset can be downloaded at https:// www.kaggle.com/datasets/PromptCloudHQ/us-technology-jobs-on-dicecom under CC BY-SA 4.0 .

diplomacy-deception . We collect dialogue from games of Diplomacy, which involves deception from Peskov et al. (2020). Diplomacy dialogues are downloaded from GitHub (all splits). The data are ASCII encoded and newlines are removed. Each message and label is treated as a sample. This dataset can be downloaded at https://huggingface.co/datasets/diplomacy_detection under unknown license.

echr-decisions . We collect facts of cases heard before the European Court of Human Rights from Chalkidis et al. (2019). Decisions are downloaded from a public archive. A random sample of 500 decisions is selected from the files. The samples with any violated articles are categorized as 'violation,' while the rest are categorized as 'no violation.' This dataset can be downloaded at https://paperswithcode.com/dataset/echr under unknown license.

essay-scoring . We collect essays from students from ess (2012). Essays are downloaded from a GitHub repository. Only essays from set 5 are considered. Essays with a score of at least 3 are categorized as good essays, while essays with a score less than 3 are bad essays. This dataset can be downloaded at https://www.kaggle.com/c/asap-aes under unknown license.

fake-news . We collect fake and legitimate news from Pérez-Rosas et al. (2017). Fake news articles are downloaded from the author's website. Full articles are treated as text snippets. This dataset can be downloaded at http://web.eecs.umich.edu/~mihalcea/downloads.html#FakeNews under CC-BY-4.0.

fomc-speeches . We collect Federal Open Market Committee (FOMC) speeches from 19962020, which describe Federal Reserve policy from Mish (2020). Fed speeches are downloaded from Kaggle. The macro indicator data are merged in on the year and month. Full speech text is split by paragraph and categorized by speaker, year, and macroeconomic indicator. This dataset can be downloaded at https://www.kaggle.com/datasets/natanm/ federal-reserve-governors-speeches-1996-2020 under unknown license.

genius-lyrics . We collect lyrics collected from Genius.com before 2020 from Lim & Benson (2021). Genius lyrics are downloaded from Google Drive. The lyrics are merged with song metadata and treated as samples. We categorize lyrics by hand-selecting popular artists, common genres, time periods, and view counts (over 1M views is high, 500k-1M is medium). This dataset can be downloaded at https://www.cs.cornell.edu/~arb/data/genius-expertise/ under unknown license.

happy-moments . Wecollect self-reported happy moments and demographic characteristics from Asai et al. (2018). The HappyDB dataset is downloaded from the official GitHub repository. Demographic data is cleaned and merged into happy moments. Happy moment descriptions are treated as samples and are categorized by type of happy moment, country of origin, and other demographic features. This dataset can be downloaded at https://github.com/megagonlabs/HappyDB under unknown license.

huff-post-headlines . We collect headlines from the news outlet Huffington Post from Misra & Arora (2019) and Misra & Grover (2021). Huffington Post headlines are downloaded from Kaggle. The

short description of each article is treated as a sample and tokenized at the sentence level. This dataset can be downloaded at https://rishabhmisra.github.io/publications/ under CC-BY-4.0.

immigration-speeches . We collect congressional and presidential speeches that mention immigration from 1880 to the present from Card et al. (2022). Immigration speeches are downloaded from the replication package. The speech text is preprocessed to remove extraneous spaces. We engineer features corresponding to time periods, well-known speakers, other significant time periods, the racial group under discussion, and the geographic area within the United States. This dataset can be downloaded at https://github.com/dallascard/us-immigration-speeches/releases .

kickstarter . We collect names of startups on kickstarter.com from Mouillé (2017). We download a 2018 crawl from Kickstarter from Kaggle. The project name is treated as the text sample. This dataset can be downloaded at https://www.kaggle.com/datasets/kemical/ kickstarter-projects?select=ks-projects-201612.csv under CC BY-NC-SA 4.0.

microedit-humor . We collect funny sentences generated by making one-word edits to normal statements from Hossain et al. (2019). The Microedit dataset is downloaded from the author's website. We make the relevant edit to each text sample and treat the edited text sample as the data point. We bin the mean annotator grade into 4 and denote each as unfunny, neutral, funny, and very funny, respectively. This dataset can be downloaded at https://paperswithcode.com/dataset/ humicroedit .

mnli . We collect a collection of sentence pairs annotated with textual entailment information from a range of genres from Williams et al. (2017). The MNLI corpus is downloaded from the official website. We treat the premise and hypothesis as text samples. This dataset can be downloaded from https://cims.nyu.edu/~sbowman/multinli/ , most of which are under the OANC license.

monster-jobs . We collect American job postings on monster.com. Jobs on Monster.com are downloaded from Kaggle. Job descriptions are treated as samples and split at the paragraph and sentence level. We keep and categorize jobs from seventeen large cities. This dataset can be downloaded from https://www.kaggle.com/datasets/PromptCloudHQ/us-jobs-on-monstercom under CC BY-SA 4.0 .

movie-tmdb . We collect movie plot summaries from TMDB from Kaggle (2018). TMDB movie overviews are downloaded from Kaggle. We keep only English movies and bin popularity by deciles. The top decile is considered 'hits,' the 70-80th percentiles are considered 'average,' and the 30-40th percentiles are considered 'bad.' This dataset can be downloaded from https://www.kaggle. com/datasets/tmdb/tmdb-movie-metadata21 .

movie-wiki . We collect movie plot summaries collected from Wikipedia from Robischon (2019). Wikipedia movie summaries are downloaded from Kaggle. This dataset can be downloaded from https://www.kaggle.com/datasets/jrobischon/wikipedia-movie-plots under CC BYSA 4.0.

news-popularity . We collect news headlines posted on social media platforms from Moniz & Torgo (2018). Headlines are downloaded from a reproduction package. The headline and title text are cleaned, and the title is treated as the text sample. The 100 most positive and negative or popular and unpopular articles on each topic are used as distributions. This dataset can be downloaded from https://archive.ics.uci.edu/ml/datasets/News+Popularity+in+ Multiple+Social+Media+Platforms .

nli-benchmarks . We collect training examples from various natural language inference (NLI) datasets from Liu et al. (2022). NLI benchmarks are downloaded from a public collection on Google Drive. We examine the premise and hypothesis separately as samples. This dataset can be downloaded from https://github.com/alisawuffles/wanli .

npt-conferences . We collect Non-Proliferation of Nuclear Weapons (NPT) conference transcripts from Barnum & Lo (2020). NPT conference notes are extracted from the accompanying replication package. Text is split by paragraph, and only paragraphs longer than 50 characters are preserved. Text is split into three time ranges: pre-2008, 2008-2012, and post-2012. This dataset can be downloaded from https://journals.sagepub.com/doi/full/10.1177/0022343320960523 .

open-deception . We collect arbitrary lies and truths from any domain generated by crowdworkers from Pérez-Rosas & Mihalcea (2015). Open domain lies are downloaded from the public dataset

and lie texts are split into lies and truths. This dataset can be downloaded from https://web.eecs. umich.edu/~mihalcea/downloads.html#OpenDeception .

open-review . We collect submissions to ICLR, a machine learning conference from 2018 to 2021. Open review abstracts are accessed via the openreview API. We query for abstracts from the 20182021 ICLR blind submissions. Abstracts are classified based on rating: > = 7 ('great'), 5-6 ('good'), and < = 4 ('bad'). This dataset can be downloaded from https://openreview.net/ .

parenting-subreddits . We collect posts from various parenting-related subreddits, which are textbased forums on the site Reddit from Gao et al. (2021). Posts from various subreddits are downloaded from the paper's GitHub repository. We clean the text and split the posts according to the topic(s) each post is tagged with. This dataset can be downloaded from https://github.com/SALT-NLP/ Parenting_OnlineUsage .

poetry . We collect poems from PoetryFoundation.com from Bramhecha (2019). Poems are downloaded from a 2019 scrape of the PoetryFoundation website from Kaggle. The text is cleaned and split according to subject tags and authorship. This dataset can be downloaded from https://www.kaggle.com/datasets/tgdivy/poetry-foundation-poems under GNU Affero General Public License.

political-ads . We collect political ads observed by Facebook users from pol (2021). Ads are downloaded from the Ad Observer website, which maintains an aggregate of all collected ads. We extract targeting metadata from the targeting field and define splits according to age, gender, location, interests, time, and political lean. This dataset can be downloaded from https://adobserver. org/ad-database/ .

qqp . We collect questions from Quora.com from Quora (2017).

rate-my-prof . We collect reviews of lecturers from RateMyProfessor.com from He (2020). We download a sample of RateMyProfessor.com reviews from an online repo. We clean the text and guess the gender of the reviewed lecturer from the first name using the gender-guesser package. Due to data availability, we consider only male and female names. To improve the quality of the classification, we remove any posts which use pronouns from the opposing sex (e.g. 'him'). This dataset can be downloaded from https://data.mendeley.com/datasets/fvtfjyvw7d/2 under CC BY 4.0 .

radiology-diagnosis . We collect impressions and medical histories of radiology patients from Pestian et al. (2007). Radiology diagnoses are downloaded from a GitHub copy of the original task dataset. We parse the metadata to retrieve the diagnostic code, decision type, impression, and patient history. Referencing the associated ICD codes, we convert codes to colloquial diagnoses (e.g. 786.2 denotes cough). We treat the histories and impressions as samples and split them according to diagnosis and level of consensus.

reddit-humor . We collect jokes posted on the Reddit forum r/Jokes, a message board for sharing jokes from Weller & Seppi (2020). Jokes are downloaded from the dev and test splits of the dataset. We clean the text and split the dataset according to whether they are labeled as funny. This dataset can be downloaded from https://github.com/orionw/rJokesData under Reddit License and Terms of Service, and users must follow the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.

reddit-stress . We collect stress-related posts on Reddit from Turcan & McKeown (2019). We split the post text based on which subreddit they are posted on (related to PTSD, anxiety, or stress generally). Reddit posts are downloaded from https://github.com/gillian850413/Insight_ Stress_Analysis , and we recommend following the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.

reuters-authorship . We collect articles from various Reuters authors from Liu (2011). The articles are split according to the author. Reuters articles are downloaded from the UCI repository https: //archive.ics.uci.edu/ml/datasets/Reuter_50_50 .

riddles . We generated several riddles. The 3000 most common English words are manually copied from a website. Words with between 5 and 8 characters are kept. We create two popular riddles. First, we split words based on whether they have a duplicate character. We exclude any words with multiple 'doubles' or more than 2 of any character. Second, we split words based on whether they have the letter T.

scotus-cases . We collect facts from cases heard by the Supreme Court of the United States (SCOTUS) from Alali et al. (2021). Supreme Court cases are downloaded from a GitHub repository. We identify state/federal parties by manually defining keywords. We split based on the winning party, the identity of each party, and the type of decision. We then define several time periods and relevant political eras and split decisions accordingly. Finally, we split according to the ruling's policy area and how it changes over time. The dataset can be downloaded from https://paperswithcode.com/paper/ justice-a-benchmark-dataset-for-supreme-court under CC-BY-SA.

short-answer-scoring . We collect short answers from students from sho (2013). Short answers are downloaded from a GitHub mirror of the dataset. We consider only responses to essay set 1. The two scores are averaged and binned into good ( > = 2.5), medium (1.5-2.5), and bad ( < 1.5). The dataset can be downloaded from https://www.kaggle.com/c/asap-sas .

snli . We collect a collection of sentence pairs annotated with textual entailment information from images from Bowman et al. (2015). The dataset can be downloaded from https://nlp.stanford. edu/projects/snli/ under CC BY-SA 4.0.

squad-v2 . We collect reading comprehension questions crowdsourced from Wikipedia articles from Rajpurkar et al. (2018). The dataset can be downloaded from https://rajpurkar.github.io/ SQuAD-explorer/ under CC BY-SA 4.0.

stock-news . We collect top news headlines on Reddit, an online message board from Sun (2017). Headlines are downloaded from a GitHub mirror. We clean the text and divide the samples based on whether the DOW rose or fell that day. The dataset can be downloaded from https://github.com/ ShravanChintha/Stock-Market-prediction-using-daily-news-headlines under Reddit License and Terms of Service, and users must follow the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.

suicide-notes . We collect posts from r/SuicideWatch and r/depression, two forums on Reddit fromHe (2021). The post title and body are combined to form the text samples. Samples are split based on whether they were posted in a suicide-related Subreddit. The dataset can be downloaded from a github: https://github.com/hesamuel/goodbye_world , under Reddit License and Terms of Service, and users must follow the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.

times-india-headlines . We collect headlines from Times of India news from Kulkarni (2022). Headlines are downloaded from a Dataverse mirror. We use the first 1000 headlines in each year as samples. The dataset can be downloaded from https://www.kaggle.com/datasets/therohk/ india-headlines-news-dataset under CC0 Public Domain.

trial-deception . We collect testimonies from witnesses in real trials from Pérez-Rosas et al. (2015). Trial testimonies are downloaded from the author's website. The testimonies are divided based on whether they are considered truthful. The dataset can be downloaded from https://web.eecs. umich.edu/~mihalcea/downloads.html#RealLifeDeception .

un-debates . We collect speeches from debates at the United Nations from Baturo et al. (2017). Debate transcripts are downloaded from the Dataverse reproduction package. Samples are divided based on the country and year of the snippet. First, we isolate samples from Russia, China, and the United States and specify 3 time periods of interest. Next, we divide all samples by the decade. Finally, we create distributions for 19 countries of interest. The dataset can be downloaded from https://doi.org/10.7910/DVN/0TJX8Y under CC0 1.0 .

unhealthy-conversations . We collect expert-annotated unhealthy conversations from Price et al. (2020). Conversation transcripts are downloaded from the official GitHub repository. For each annotated attribute, we split the dataset based on whether that form of unhealthy conversation is present in the sample. The dataset can be downloaded from https://github.com/conversationai/ unhealthy-conversations under CC BY-NC-SA 4.0.

urban-dictionary . We collect definitions from UrbanDictionary.com, a crowdsourced English dictionary from Kulkarni (2020b). Urban Dictionary entries are downloaded from Kaggle. Definitions are split into groups representing the top 1, 5, and 10 percent of definitions ranked by both upvotes and downvotes; we sample 10,000 from each and create a control distribution by randomly sampling 10,000 definitions from all entries. The dataset can be downloaded from https://www.kaggle. com/therohk/urban-dictionary-words-dataset under CC0 Public Domain.

wikitext . We collect text snippets from Wikipedia from Merity et al. (2016). The Wikipedia snippets are loaded from HuggingFace. We remove any samples that are empty or start with '=' (which represent headings); samples are tokenized at the sentence level and used for clustering. The dataset can be downloaded from https://huggingface.co/datasets/wikitext under CC BY-SA 3.0.

yc -startups . We collect descriptions of companies that were part of the Y Combinator startup incubator from Bhalotia (2022). YCombinator company descriptions are downloaded from a 2022 scrape on GitHub. Only companies with long descriptions are preserved. Companies are split according to founder characteristics, year, 'top company' designation, operating status, and location. The dataset can be downloaded from https://www.kaggle.com/datasets/benhamner/ y-combinator-companies .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new task called D5 (goal Driven Discovery of Differences between text Distributions via language Descriptions)\n) where the objective is to identify and describe meaningful differences between two given text corpora. Authors created two datasets to evaluate their D5 task based: SYND5 and OPEND5, based on validity and relevance.\n\nSTRENGTHS:\nThe creation of OPEND5 and SYND5 datasets for training and evaluation shows originality.\n\nWEAKNESSES:\nTask justification and applicability: \nThe paper introduces a new task D5  (goal Driven Discovery of Differences between text Distributions via language Descriptions). However, the need for such a task is not clearly articulated. The authors do not provide sufficient examples or scenarios where this task could be beneficial, making it difficult for the reader to fully appreciate the significance of this work. It would be beneficial if the authors could provide concrete real-world applications or problems where this task could contribute to a solution or add value. If the main usage is as the example shown in Figure 3, the users can use GPT to generate answers themselves nowadays.  \n\nInterpretation of discoveries: \nThe paper acknowledges that the interpretation of the discovered differences is challenging and often requires domain experts. However, the study doesn't suggest concrete ways to address this problem, which could limit the practical application of the D5 system. \n\nReliance on Turkers' judgments: \nThe paper relies heavily on the judgments of Mechanical Turkers to evaluate the validity of discoveries. This approach might introduce another layer of bias or inaccuracies due to the potential lack of expertise among Turkers as authors mentioned in the section 6.2.  \n\nMissing a conclusion section\n\nMinor suggestion: the right edge of Figure 2 is cut off which need to adjustment.\n\nQUESTIONS:\nThe above weakness are also my questions. \n\nEvaluation Metrics: \nThe paper primarily reviews the task based on the metrics of validity and relevance, but also mentions evaluating subjective metrics like novelty and significance. Could the authors explain why validity and relevance were prioritized in the evaluation process? And can the authors provide a brief overview or key insights from the evaluations of novelty and significance, given that they are confined to the appendix due to space constraints?\n\nRelated work: \nThe author propose their contribution as the novel D5 task but also mentioned they mirrored Zhong et al. (2022)'s study. Could the author explain what is their novel contribution compared with Zhong et al's work?\n\nLIMITATIONS:\nThe authors have indeed discussed several limitations of their work which they couldn't address at this stage.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a goal-directed discovery and description of differences between text distributions (D5). Given two corpora and a research goal, the task is to generate a discovery in a natural language predicate. The paper creates datasets SynD5 and OpenD5, which can be used to evaluate  validity, relevance, novelty, and significance.\n\nSTRENGTHS:\n- The paper tackles an interesting task and formulates research discovery as an NLP task. \n- The paper creates new SynD5 and OpenD5 datasets from a variety of problems.\n\nWEAKNESSES:\n- #1 The task is not well-formed or well-motivated. \n- #2 The dataset quality is not ensured. As stated in LL119-123, OpenD5 does not ensure that there is ground-trugh and thus is not clear hwo reliable it is as a benchmark. \n- #3 The paper is generally well-written but the key concept and main messages are hard to parse. \n\nFirst of all, it is a great effort to compile hundreds of datasets into a single benchmark but I’m afraid that I’m not very clear what we can conclude after seeing method A perform better than method B on this dataset. After reading the paper, I wasn’t able to answer a question “What functionality assessment can this test bed be used for?” by myself. \n\nLooking at examples in Figure 2, the OpenD5 data split looks synthetic and some of them rely on the ground-truth labels for the original task. (e.g., 10-star reviews and 0-star reviews.) In other words, how the original corpus was split into Corpus A and B is essential to answer the research question. Thus, if the split is given, the problem is partially solved already. \n\nIn my understanding, the task essentially asks if the model can figure out how those two corpora were created from the original corpus with additional context information as the research goal. \n\nFor Machine Learning problems, I believe analysis on each pair could be one research paper (e.g., the difference between SNLI and MNLI; LM’s output differences etc.) Then, again I’m not very sure what we can learn from evaluation results on the OpenD5 benchmark. \n\nTo me, the problem the paper is tackling is rather automating Exploratory Data Analysis using LLMs (like the paper below. To be clear, it’s considered concurrent work and the paper doesn’t have to compare with it.)\n\nPingchuan Ma, Rui Ding, Shuai Wang, Shi Han, Dongmei Zhang, Demonstration of InsightPilot: An LLM-Empowered Automated Data Exploration System, https://arxiv.org/abs/2304.00477\n\nQUESTIONS:\n- As raised in the Weaknesses above.\n- Q. In 2.1 Task Format, is the goal description given for the task (and thus is included in the OpenD5 dataset)?\n\nLIMITATIONS:\nThe paper discusses the limitations. For ethical concerns, I didn’t check the licenses of all the datasets (whether they can be redistributed after modification)", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new machine learning task - D5 - to model and discover differences between two large corpora by providing a task input as an exploration goal. The paper also introduces a synthetic dataset, SynD5,  for evaluation of D5, as well as OPEND5, a dataset containing 675 problems that D5 can be used to train on.\n\nSTRENGTHS:\n- The task tackled by the paper - generating discoveries based on corpora is impactful \n- The paper provides in-depth description of their methodology and datasets, as well as how they were built\n- The proposed framework is innovative\n\nWEAKNESSES:\n- OPEND5, the dataset that was built specifically for D5, has been collected manually by the authors, which could introduce biases. \n- As the authors themselves mentioned, the extracted hypotheses can be merely correlations in the corpora and not necessary causations. However, D5 is built in such a way that it would treat them as 'discoveries'. Given the nuance around the types of things the model would extract, having clearer, more robust measures and datasets for evaluation of such a system should be in place before releasing it. Validity scores being quite low support this as well. \n- The paper selects 21 out of 3296 examples to showcase as discoveries, potentially showing only cherry-picked examples.\n\nQUESTIONS:\n- I wasn't clear what was the purpose of the distractor (mentioned in 2.1)\n- I’m not sure I understand the process described in 4.3 where a discovery’s validity is meant to improve by fine-tuning a LM on discoveries with validity scores obtained from an LM. If we can’t trust the validity of the LM to begin with, why would we trust a model fine tuned on it?\n\nLIMITATIONS:\nI've mentioned a couple limitations in the paragraphs above. I believe the authors have adequately managed to address the limitations in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors formulate a new task, D5, that automatically discovers differences between two large corpora in a goal-driven way. The task input is a problem comprising a user-specified research goal (“comparing the side effects of drug A and drug”) and a corpus pair (two large collections of patients' self-reported reactions after taking each drug). The output is a goal-related description (discovery) of how these corpora differ (patients taking drug A “mention feelings of paranoia” more often). \nIn addition to the dataset the authors also build a D5 system, and to quantitatively evaluate its performance, \nFinally, the authors discuss the limitations of the current D5 system, which discovers correlation rather than causation and has the potential to reinforce societal biases when misused.\n\nSTRENGTHS:\n1. The authors propose new tasks and datasets that are interesting and intriguing, which is considered a contribution to the community. The idea of generating discoveries with controllable goal is a valid and useful task.\n2. Overall the task looks well-formulated, and the dataset is covering multiple data domains.\n3. The writing and presentation is of high quality. The paper is generally easy to follow and understand.\n4. The qualitative analysis part is very well done and complete.\n\nWEAKNESSES:\n1. More discussion could be done regarding the uneven distribution of data examples in different domains. For example only 10 from Health versus more than 200 in social sciences.\n2. The authors designed a set of scores for evaluation metrics, and it would be better if there could be some sort of relevance study between the subjective scores with a set of human scorers.\n3. Although using closed large language models are exciting because they generally provides better results, it would be nice to also add in any results with open-source models or smaller models, both for the sake of reproducibility (since OpenAI will deprecate the old APIs soon) and also to show the community where the current open models is at for your proposed tasks.\n\nQUESTIONS:\nAbove.\n\nLIMITATIONS:\nYes, the authors state limitations as well as possible biases.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work formulates a new task called D5, proposes a language model (LM) based approach to generate desired outputs in the context of this proposed task, and contributes a new dataset of open-ended problems to highlight how their LM-based approach can facilitate exploration of differences between two large English text corpora, with respect to a stated goal. The goals (for exploring differences) are stated in natural language, and the output 'discovery' of the differences is also generated in natural language. \n\nThe work shows how discoveries generated by their D5 system can be evaluated on both a synthetic diagnostic dataset (where ground truth differences are known and recovery of these differences can be directly tested), and on the open-ended OpenD5 dataset. The evaluation in the latter case depends on one somewhat objective criterion of validity, and then relevance to the user-stated discovery goal is evaluated manually using subjective human assessments, where humans are either the authors themselves or crowd workers (not domain experts). This work as a whole seeks to demonstrate the potential of LMs to facilitate human or subject matter exploration of hypotheses for differences in two corpora, and these discovered hypotheses are meaningful towards a user-stated goal.\n\nSTRENGTHS:\nThe paper is well written and is clear in its definition of the proposed task as well as the objectives of the work. The goals carry real-world significance, as a valid and reliable method to enable exploration of goal-relevant differences between two corpora can help scientists and domain experts in generating useful hypotheses and potentially help generate insights that would have otherwise been too laborious or just not that obvious via a manual assessment (especially when the two corpora are big in size). The work takes a concrete step towards these goals of enabling human exploration and discovery. While exploratory differences between text corpora have been studied in NLP before, adding a goal, expressed in natural language, to this process of discovering differences and enabling systems to take this goal into account is a novel modification to this kind of exploration task. The OpenD5 dataset is a valuable data contribution if released and documented properly. Proposed evaluation criteria and robustness checks also serve as a key framework for designing similar tasks and making sense of the goals of the proposed system and task.\n\nWEAKNESSES:\n- While proposed criteria and robustness checks are valid and sensible, when the goal is facilitating human exploration and providing a system and framework to end-users, it is important that people other than the authors create the prompts or fill in the non-fixed portions of the prompt template because it is not clear if even slight differences in how the ‘open-ended’ parts of the prompt text (and its framing) could affect the generated hypotheses and discoveries. It is understandable that recruiting people to also generate prompts and assess discoveries for the same goal and input corpora but with prompts written by different people is difficult, especially if a well-powered human experiment is to be conducted. However, it is critical to study the effect of how the exploration goal is phrased — which includes paraphrases. While the validity and relevance to criteria are necessary and well-motivated eval criteria, arguably another really important criterion here (since the system is intended to help human exploration) is stability (which can map to intra-coder reliability): is system output robust to tiny, non-meaningful changes in the stated exploration goal? \nIn fact, this can be seen as an important component of system reliability – if we cannot expect consistently similar or the same discoveries for stated exploration goals or prompts that carry the same meaning and logically entail the same output, then the system is clearly not reliable for use by humans. Additionally, this robustness or stability is important to establish, given literature in NLP, for example, on adversarial robustness where minor perturbations in text (such as in text style which should be irrelevant to the task) could 'break' the NLP system, and more. See Qi et al (2021) for an example with text style transfer, Zhang et al (2020) for a relevant survey, and Wallace et al (2019) for some useful pointers on adversarial triggers in NLP. \nIn addition to this important robustness criterion, it is also generally important to study the effects and properties of the language of the exploration goal. The following questions can be used to create a more tested system of prompt design: What must be included? What are the constraints on phrasing? Does the order of specifying things matter? Is there a guide that should be followed when constructing the exploration goal for the most 'valid' and 'relevant' results? Do different phrasings of the same underlying exploration goal affect the validity or relevance of the model output discovery? \nEven for the template portions of the prompt (which are not meant to be modified), it is not clear how this template was constructed and what happens when these templates are changed. Theoretically, could a different template achieve better results? Without such an analysis, both the usage of the system and the generalizability of the research work are negatively affected. \n\n- One of the stated aims of this work and the proposed system is facilitating the discovery of unknowns or the generation of novel, previously unknown insights. However, including what has been presented in section 6.1, it is not clear at all if this has actually been established even using the OpenD5 dataset. With only the authors judging the discoveries and no subject matter experts involved, such a claim cannot be evaluated or it needs to be reframed and scoped out appropriately. Novel to the authors does not seem to represent a significant advance and does not seem to represent a step towards “discovering unknowns” stated in line 35. However, this is a critical potential contribution of this work and not taking concrete steps to establish this potential negatively affects the significance of the work. Also, in lines 284-285, the authors acknowledge that the signal about validity is unreliable, and since validity is critical when it comes to hypotheses generation and exploratory differences, it is unclear if the proposed framework is ready for uptake and if it could provide potential value to practitioners as things stand. \n\n- The explicit acknowledgment and some discussion on some of the limitations are important and add to this work, however, the limitations that have been acknowledged are of great importance and represent harm. The exploratory analyses aspect and the aim of discovering the unknown make the harm of bias reinforcement even more critical, yet no path to mitigating these harms has been provided. The biases in authors’ subjective evaluations are also similarly really important here and it is not clear why at least other humans or recruited crowd workers cannot help in all the key human judgement-based evaluations, though the cost is certainly an understandable factor. Not incorporating domain experts and their knowledge in assessing discoveries is also a big limitation since the potential for this system to generate novel insights and find unknowns remains unclear. With the authors rightfully cautioning users about bias amplification and reinforcement towards the end of this paper, with no pathway to mitigate or understand when such cases arise, it is questionable whether the system should be used at all. There are also at least two other key limitations (discussed below) that also need to be addressed or at least comprehensively discussed in this work. \n\nIn summary, the key weaknesses of the work as it currently stands include a) system reliability for usage by end-users is not yet fully established, with more robustness checks on prompt construction and framing needed; b) while stated as a goal, the step taken in this work does not seem to demonstrate potential for generating novel insights or discovering unknowns, and evaluation signal for the validity of the discoveries that do get generated also seems to be unreliable; and c) limitations, both the ones explicitly acknowledged and the ones that are not, represent real-world harms, and the mitigation of these harms has not been tackled and as things stand, these harms seem to outweigh any benefits of the proposed system and framework. \n\nReferences:\nFanchao Qi, Yangyi Chen, Xurui Zhang, Mukai Li, Zhiyuan Liu, and Maosong Sun. 2021. Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4569–4580, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.\n\nZhang, W. E., Sheng, Q. Z., Alhazmi, A., & Li, C. (2020). Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Transactions on Intelligent Systems and Technology (TIST), 11(3), 1-41.\n\nEric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal Adversarial Triggers for Attacking and Analyzing NLP. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2153–2162, Hong Kong, China. Association for Computational Linguistics.\n\nQUESTIONS:\n1. Is it safe to say that at least for the paper as it stands, “discovering unknowns” is not something that has been evaluated or judged? Is there a robust plan to enable this system to reliably help domain experts discover true unknowns? \n\n2. What can we expect about the robustness of the system to variations in the phrasing and framing of the stated exploration goal by different non-author users (see the first point in the weaknesses section)? Also, from an end-user as well as a research perspective, are any experiments conducted to test what must be included in the exploration goal or other non-templatic parts of the prompt, what should be constraints on phrasing, if the order in which various things (about the corpus, goal, etc.) things matters? Can we expect that different phrasings of the same underlying exploration goal will affect the validity or relevance of the model output discovery?\n\n3. How was the prompt template designed, and do we know if a different template could yield better results?\n\n4. As noted above in the third point in the weaknesses section, even within the scope of identified limitations, it does not seem to be easily identifiable for an end user when correlation might get framed as causation or when bias reinforcement might happen, and therefore, can the current system be used with those potent limitations and harms as a possibility? In other words, given these known limitations and harms, can we currently handle them effectively, or if not, should D5 usage be held until those limitations are overcome or at least mitigated effectively?\n\n5. Section 6.1 is an important assessment of OpenD5, however a lot of details do not seem to be provided which hampers the review but also the clarity a reader needs. The manual selection process of the 21 discoveries is not clear: how was the ‘representativeness’ of potential use cases judged? Also, how were criteria 3 and 4 actually assessed (lines 254-255)? Apart from this section, some other key details and justification behind certain decisions are not provided which are also important, these are noted below.\n\n\nSuggestions (acknowledging possible space limitations for author response, questions above are separated from some questions below – addressing the below questions can improve the paper): \n\n- With respect to lines 49-50, goal-relevance being higher is expected (and good that it does happen) since one model has goal-conditioning and the other was not designed to handle stated goals; but it is interesting that validity or correctness is higher too – analysis of what invalid differences are now not generated in the proposed model or what valid differences were missed in baseline model would be interesting (unless already done, in which case a clearer pointer to this analysis should be included). \n\n- For the synthetic diagnostic test, it seems like the testing would benefit from either an additional or alternative approach to a synthetic test bed where the two corpora are not generated by language models (since systems being tested in this work are also language models, which calls into the question the strength of the proposed synthetic benchmark). Given that the properties for goal-relevant and distractor dimensions are around topic, genre, and language, a straightforward synthetic benchmark is using parallel corpora and then running actual topic models on them to generate and divide documents/texts by topic using the outputs of topic models: in this way, the corpora being compared remain real-world human-authored collections while the distraction vs goal-relevant test can still be conducted. OpenD5, while being a valuable real-world dataset contribution relevant to the task, does not test this distractor element present in SynD5. So something like the proposed alternative diagnostic with a) corpora authored by humans, and b) a known distractor element should provide a robust synthetic testbed arguably necessary to gain insight into the proposed framework for using LLMs.\n\n- Line 98: cannot just stay Claude-v1.3 – what actually is this, exactly, needs to be specified.\n\n- What courses on computational social science (line 110-111) were used? A list should ideally be provided in the appendix.\n\n- Line 112: what is the inter-annotator agreement for the exploration goals?\n\n- Line 179: why 60?\n\n- For the hypothesis validator (lines 183-184), how good are language models in terms of simulating Turkers' validity judgement? Is this a valid proxy for this hypothesis validation step?\n\n- Especially for synd5, it seems like there was an opportunity to test with a variety of open source open access models which is especially important since we should operate under the assumption that many users will not have accept to GPT-4. \n\n- Line 238: It actually is important that non-authors rate relevance in order to get a more robust assessment of relevant improvement when using goals on OpenD5.\n\n- Typo in line 294: the closing quotation marker should not be the beginning of a new line.\n\n- Line 329 in related work section: topic models deserve a mention since they are used for the discovery of human interpretable latent concepts, and in fact relevant to one of the experiments conducted in this very work where authors use the topic as one of the dimensions varying between two input corpora samples (see the suggestion above around a more robust synthetic test).\n\nLIMITATIONS:\nWhile some key limitations are already acknowledged in this work, there does not seem to be a robust discussion on managing and mitigating these in real-world usage of the system. With things like correlation being potentially framed or interpreted as causality, for example, when can users expect such things to happen – without any markers or indications for a system output, it is hard not to be skeptical of any discovery generated by the proposed system. In other words, the key question that seems to be underexplored here: if it is not easily identifiable for an end user when correlation might get framed as causation or bias reinforcement is happening, should the system be used with those potent limitations and harms as a possibility, and given these known limitations and harms, if we do not know if they can be handled effectively or not, should D5 usage be held until those limitations are overcome or at least mitigated effectively? \n\nAcknowledgment is an important system, but adequately addressing limitations needs more work, and there are also at least two more really important broader considerations not discussed in this paper that need to be addressed: \n\n1. The work seems to operate in the English language alone, at least in terms of substantive testing and application in the paper itself. There is no discussion about theoretical generalizability to other languages and expectations around that, and at least there should be an acknowledgment of the lack of empirical generalizability of findings about the system to other languages. The languages in which the synthetic and contributed real-world dataset operates should be made clear by the end of the introduction itself. \n\n2. Some important broader impacts/limitations are not being discussed and there should be more discussion in the paper about who gets to use the proposed system and potential double-use scenarios. For example, given that GPT-4 leads to better performance than the open-source alternative, if using this system for effective discovery relies on OpenAI API access and compute resources discussed in Section 8, many scientists and other end-users will probably not be able to use this method of exploration effectively, advantaging some communities over others — leading to many ramifications such as expanding existing inequities in access and resources. How can open-sourced tools be leveraged here, and can smaller models requiring fewer resources be incorporated effectively? This unequal opportunity question also ties to the previous point about language -- doctors, for example, can be operating in different cultural contexts using different languages to make their notes and we do not know if their experiences be used effectively for hypothesis generation same as those operating with English. Low-resource languages are also a potential limitation and concern that warrants discussion.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-21"}