SlowGuess/NeurIPS_2023_OCR huggingface dataset:
<sup>3</sup>We determined the number of samples s.t.  $V'$  can achieve a  $p$ -value of 0.005. Estimating  $V$  for these discoveries costs  $\sim$ 1500.
discovery	V	p	V'	p'
argues for a path forward to promote the fair ...	0.16	1.26e-04	0.35	2.01e-73
refers to illegal immigrants as criminals	0.09	6.17e-03	0.19	3.17e-38
has an informal tone, such as slang or colloqu...	0.08	2.35e-03	0.24	1.46e-35
mentions lack of legroom	0.16	1.15e-03	0.38	1.34e-45
mentions children or family	0.08	1.00e-05	0.11	8.05e-09
Uses language that is positive or uplifting	0.12	2.12e-03	0.24	4.18e-59
references violence or aggression	0.06	9.87e-03	0.17	4.25e-26
involves physical activity, such as walking, p...	0.13	4.92e-03	0.37	7.07e-101
contains keywords related to business, finance...	0.08	2.89e-02	0.35	1.45e-95
mention disasters and crimes, such as plane accidents...	0.03	7.03e-02	0.09	4.61e-06
discusses coronavirus-related topics	0.21	1.01e-04	0.27	9.19e-78
references pop culture, such as movies, books,...	0.21	2.67e-04	0.58	2.09e-30
uses vivid imagery and metaphors to convey a f...	0.09	2.47e-02	0.45	5.04e-64
Table 10: The full table of discoveries, along with their  $V$ ,  $V'$ ,  $p$ , and  $p'$  scores.
18 More Example Discoveries on OPEND5
Analyzing errors in NLP systems. We considered the task of perspectrum classification (Chen et al., 2019), which has the following instruction: "given a perspective and a claim, classify whether the given perspective supports or undermines the claim. If the perspective could possibly convince someone with different view, it is supporting, otherwise it is undermining." We considered two few-shot learning systems: GPT-3 Instruct Curie (Ouyang et al., 2022) and Tk-Instruct-11B (Wang et al., 2022). We focused on the perspectives where the ground truth label is undermining, and compare the following two corpora: Corpus A – the set of perspectives where Curie correctly classifies the input as undermining but Tk-11B is wrong, and Corpus B – the set where TK-11B is correct while Curie is wrong. We found that Corpus B more often "Uses language that is positive or uplifting" ( $V \approx 0.12$ , AUCROC ≈0.67). One possible explanation is that Curie made many mistakes by misinterpreting undermining as a label for negative sentiment rather than a logical relation between the claim and the perspective.
Comparing lyrics from different eras. Compared to lyrics from the 70s, those from the 80s more often "references violence or aggression" ( $V \approx 0.06$ , AUCROC  $\approx 0.58$ ).
Describing distribution shift. We compared the premises from the SNLI dataset and MNLI dataset, and the former "involves physical activity, such as walking, playing, climbing, or biking" ( $V \approx 0.13$ , AUC-ROC ≈0.64). One possible explanation is that SNLI is based on image captions.
Comparing discussion topics between bots and human users. We compared the topical differences between tweets identified as written by bots vs. human users on Twitter, and our system finds that the bots more often "contains keywords related to business, finance or trading" ( $V \approx 0.08$ , AUC-ROC  $\approx 0.61$ ). One possible explanation is that bots are frequently used to generate finance-related scams.
Identifying temporal differences in news headlines. We compared headlines published by ABC news across different years. Compared to 2014, headlines from 2010 "mention disasters and crimes, such as plane accidents and assaults" more often ( $V \approx 0.03$ , AUCROC  $\approx 0.53$ ). Compared to year 2019, year 2020 more often "discusses coronavirus-related topics" ( $V \approx 0.21$ , AUCROC  $\approx 0.65$ ).
Describing text clusters. We present two example descriptions for text clusters. One from Wikipedia: "references pop culture, such as movies, books, and television shows." ( $V \approx 0.21$ , AUC-ROC ≈ 0.73); one from PoetryFoundation.com: "uses vivid imagery and metaphors to convey a feeling" ( $V \approx 0.09$ , AUC-ROC ≈ 0.65).
19 Limitations and Future Work
We still face many challenges in building a broadly useful system. We describe technical challenges that machine learning researchers can tackle in Appendix 19.1 and organizational challenges that require domain experts in Appendix 19.2.
19.1 Engineering Challenges
Hypotheses about the corpora might not be appropriate predicates on individual samples. When comparing highly rated definitions from UrbanDictionary.com to others, our system generates the hypothesis that the former "is more likely to include slang or colloquial terms." This is a statement about a collection of text samples, but the validator requires the hypothesis  $h$  to be a predicate on individual text samples  $x$ . To address this, we used GPT-3 to automatically remove comparatives from the hypotheses, e.g. rewriting the hypothesis above to "include slang or colloquial terms."
However, some versions of this problem were harder to remove. For example, when comparing reviews from American Airlines (AA) flights and Delta Airlines to understand which aspects of each airline are doing better/worse, the proposer generated the hypothesis "mentions American Airlines" staff being unfriendly and unhelpful". Interpreted literally, this hypothesis can only be true on the corpus of AA reviews, since it presupposes the review to be about AA. The correct predicate for use on individual samples should instead be "mentions staff being unfriendly and unhelpful" (without the words "American Airlines"). Therefore, future systems should explicitly convert corpus-level statements to their corresponding correct predicates, and the metrics should evaluate whether the validity of the predicates implies the corpus-level statements.
Beyond truth predicates. Our work requires the discovery to be a truth predicate that maps a text sample to a truth value. However, scientific discoveries can be arbitrary natural language expressions; extending to more flexible expressions requires a significant redesign of our system and evaluation framework. Some more feasible near-term extensions include 1) allowing natural language expressions that map from text samples to real values, e.g., "how polite the sentence is compared to other samples from the corpora" or 2) using additional logical forms to combine individual truth predicates; e.g., learn a shallow and interpretable decision tree where each split point is a natural language predicate.
Beyond corpus-level differences. Our work focuses on describing corpus-level differences and validates a discovery by comparing how often it is true on each corpus. Future work can consider other ways to validate a discovery: for example, suppose each text sample is associated with a continuous target variable, we can validate whether a discovery is more likely true if the target variable is large.
Investigating sensitivity towards prompt format. In this paper we hand-crafted the prompt for the proposer and manually annotated the exploration goals on our own for OPEND5. However, due to budget limitation, we have not investigated how sensitive is our D5 system towards prompt formatting and paraphrasing, or whether the performance could have been improved with better prompts. Future works can investigate more in this research direction.
Clarifying a discovery. Some discoveries seem to have clear meanings on the surface, but they become ambiguous when we judge them on individual text samples. For example, judging whether a text sample  $h =$  "mentions people" seems like an unambiguous task a priori; however, it is unclear whether it is true on the sample  $x =$  "I woke up this morning.", since the "people" in  $h$  is a plural form, while  $x$  only mentions one person "I". Future work can use a language model to automatically clarify the meaning of a hypothesis and make it more specific, e.g., rewrite  $h$  as "mentions one or more humans."
Correlation  $\neq$  causation. Like other tools that rely on correlations to analyze patterns in data (e.g., linear regression), our system cannot establish causal relations either. For example, when comparing self-reported happy moments from females and males, even if the former corpus has more samples that "mention children and family", it does not necessarily imply family plays a more important role in inter-personal relations for females; an alternative hypothesis is that females might mention any other people more often than males, hence leading to the observation that they mention family more often. Future work can use language models to propose what control hypothesis to test.
Decreasing the cost of validation. As alluded to in Section 3, estimating  $V$  is extremely expensive as it requires a lot of human labor. Future work can consider an importance sampling procedure that uses  $\hat{T}$  as a proposer to improve the sample efficiency of estimating  $V$ .
Training a better proposer. We developed a self-supervised learning algorithm to propose more valid hypotheses. However, it does not take into account the meaningfulness metric, and it is unclear how to manage its trade-offs with validity if they exist. We look forward to future works that can train a better proposer with as minimal supervision as possible.
Combining Meaningfulness and Validity Metrics. To simplify evaluation, we assumed meaningfulness to be independent of the magnitude validity  $V$ . Such an assumption allows us to directly evaluate hypotheses that are not necessarily valid but is also limiting for evaluating the final discoveries: for example, for that 2008 "discuss economy" more often than 2007, it would be way more significant if  $V = 0.99$  compared to  $V = 0.0000001$ . Future works can propose better metrics that do not assume that validity and meaningfulness are independent.
Extending to Non-English Language OPEND5 is currently annotated with English goals and most of the corpora are in English. Future work can consider extending this to other languages.
19.2 Organizational Challenges
As discussed in Polanyi et al. (2000), it requires implicit community norms rather than explicit deductive logic to decide what counts as good research results; to guide our system to produce truly important discoveries, our system needs feedback from researchers who work in the domain of interest. However, except for machine learning, the authors do not have research expertise in most of the domains listed in Figure 2. We look forward to future contributions from other domains and list concrete directions below.
What problems to solve? We generated the problems in OPEND5 by reading relevant papers and guessing what domain experts might care about. However, our guesses can be inaccurate. Future works can directly gather problems from domain experts to reflect the actual usage of our system.
How to interpret a discovery? We asked for Turker's judgment to compute  $T(h, x)$ . However, many hypotheses require expert knowledge to interpret properly. For example, only law experts can reliably judge whether a contract  $x$  satisfies the predicate  $h$  "contains a license grant that is irrevocable." Domain experts are needed to evaluate the validity of a discovery and supervise the validator.
What discoveries are meaningful? Our work developed the evaluation instructions to approximately evaluate what hypotheses are meaningful. However, just as no one can become an outstanding peer reviewer simply by reading the review guideline, we do not consider it feasible to provide a gold evaluation simply by reading our instructions. Whether a discovery is meaningful depends heavily on implicit community norms, and we hope domain experts can provide better evaluation and training signals for our system.
20 Self-Supervised Learning with Open-Ended Problems: A Proof of Concept
Since the problems in OPEND5 are open-ended, our system could potentially produce discoveries with higher validity scores than our current system. Therefore, we design a self-supervised learning algorithm to improve an LM's ability to propose more valid hypotheses, using the principle that it is easier to validate a discovery than to generate one.
Algorithm. Suppose we are given a set of problems for training and an initial language model  $m_{\mathrm{init}}$ . Our goal is to automatically generate a set of prompt-completion pairs to fine-tune  $m_{\mathrm{init}}$  so that it can propose hypotheses that are more valid. To generate a prompt, we randomly sample a problem and create a proposer prompt following the procedure in Section 4.1. To generate the desired completion given a prompt, we sample multiple hypotheses from  $m_{\mathrm{init}}$ , approximate their  $V'$  score on the samples in the proposer prompt with the same language model  $m_{\mathrm{init}}$  (Section 4.2), and select the highest scoring hypothesis. Finally, we use the prompt-completion pairs to fine-tune  $m_{\mathrm{init}}$ .
However, since we cannot fine-tune instruction-tuned GPT-3, we can only experiment with Flan-T5 (Chung et al., 2022), an open-sourced instruction-tuned model that might only work well for easier "mini-problems". As a proof of concept, we tested our algorithms for describing groups of four samples, where each group comes from a text cluster. As an overly simplified example, we will give the LM the prompt "Group A: 1. dog 2. cat 3. pig 4. cow. Group B: 1. phone 2. laptop 3. desk 4. cup" as an input and the LM can output "mentions an animal" as a hypothesis.
Data. We created 33 corpora by merging all corpora in OPEND5 with the same domain, and automatically generated 4503 text clusters using RoBERTa embeddings (Aharoni & Goldberg, 2020). We focused on clustering because it can automatically generate a large amount of semantically coherent groups of samples. To create a pair of four samples, we randomly sampled a corpus, sampled two clusters within that corpus, and took four random samples from each cluster. To test
cross-corpus generalization, we reserved 28 of the 33 corpora to create mini-problems for evaluation, using the rest for training. We used Flan-T5 (Chung et al., 2022) as  $m_{\mathrm{init}}$  and sampled hypotheses with a temperature of 0.8. For training, we sampled 30,000 mini-problems and selected the best of eight hypotheses generated by  $m_{\mathrm{init}}$  as the target completion; for evaluation, we sampled 200 mini-problems to calculate  $V$  with Turkers and 1500 mini-problems to calculate  $V'$  automatically.
Results. We evaluated randomly sampled hypotheses from the language model before and after self-supervised training. The automated "self-evaluation" validity score  $V'$  improves substantially from 0.22 to 0.37, and the "true" validity score  $V$  according to Turker evaluation improves from 0.07 to 0.10, with a  $p$ -value of 0.02. This result provides preliminary evidence that our algorithm (or similar variants) could be applied to a large set of problems to improve the validity of the hypotheses; we expect future validation to simulate human judgments better, hence decreasing the approximated gap of improvement between  $V$  and  $V'$ .
21 Comparing D5 to Naïve Bayes
We qualitatively compare the discovery generated by our D5 system to the top-5 unigram features extracted by Naive Bayes, a traditional exploratory analysis method. The Naive Bayes method is effective when the target difference can be saliently reflected by individual words. For example, "yo" implies a rap genre, "die" implies a language of Deutsch, and ["rank", "higher", "univeristy"] hints at the topic of "college ranking changes". Additionally, compared to black-box neural networks, such a method is fully interpretable.
In comparison, D5 can directly generate a semantically coherent description for the target difference, saving users' time to guess the underlying correlation by inspecting the top unigram features. Additionally, it can capture differences that are hard to detect at a word level; for example, "the genre of biblical scripture" is mainly reflected in its sentence structure rather than individual words. Finally, D5 only describes goal-related differences, while Naïve Bayes picks up on any discriminative feature; for example, when identifying the topical differences between a English and a Deutsch corpus, Naïve Bayes fails catastrophically and only picks up common determiners such as "the" or "die" instead of topic words, since they are the most useful feature at telling which sample comes from which corpus. Given the respective strength of D5 and traditional exploratory methods, we envision D5 to serve as a complementary method to traditional methods.
22 Annotation Interface to Collect Human-Generated Hypotheses
(This section describes an interesting research direction we did not have time to fully pursue.)
Task. To fine-tune the language model to propose better hypotheses and perform validation more accurately, we also designed an interface to collect human annotations earlier in the project. In this annotation task, the annotators see five text samples from each of the two corpora; they then write one or many natural language predicate(s) that describe how samples from the two groups are different and choose which text samples satisfy each predicate the annotator has written. Since it is challenging for humans to identify systematic differences between even groups of five sentences, we made the task easier for them by
- we chose the representative samples from each corpus to form the two groups of samples, similar to the process in Section 15, and
- we highlighted subspan of the text samples that are informative for how the two corpora differ. For example, if Corpus A is sports related while Corpus B is entertainment related, we hope to highlight sports-related words like "basketball". To automatically identify the text spans to highlight, we fine-tuned RoBERTa to classify whether a sample comes from Corpus A and Corpus B, used the SHAP library to calculate how much each text span influences the classifier's decision, and highlighted the text spans based on the influence.
A screenshot of the annotation interface can be seen in Figure 6.
Preliminary Results We performed initial experiments on text clusters formed on the wikitext-2 dataset (Merit et al., 2016). We asked the authors to write hypotheses for 30-50 samples and then compare the results with GPT-3 generated hypotheses. We found that human annotators were able to
Figure 6: A detailed screenshot of our annotation interface.
write 2-4 valid hypotheses per pair of text groups, while GPT-3 text-davinci-003 was able to generate 4-6. Out of the valid generated hypotheses, approximately a third were variations on another valid hypothesis. The number of times humans were able to write a hypothesis that GPT-3 was unable to generate was around a third of the samples, while GPT-3 was able to generate a novel hypothesis humans have not thought about before in nearly every single text corpora. Given that GPT-3 is close to our author's ability to write hypotheses, we estimated that we would not be able to fine-tune T5 to propose better hypotheses with human annotations, and hence gave up on this research direction.
23 Datasets
Many of our datasets come from the following sources: the Computational Models of Social Meaning class from Columbia Universityhttp://www1.cs.columbia.edu/\~smara/teaching/S18/, the ACL Anthology https://aclanthology.org, , and Kaggle datasets with an NLP tag. https: //www.kaggle.com
abc-headlines. We collect headlines published by ABC news, an American news company from Kulkarni (2018). ABC headlines are directly downloaded from Harvard Dataverse. The year is extracted from the publication date field. Samples are constructed from the headline text. The data is downloadable from https://doi.org/10.7910/DVN/SYBGZL with license CC0 1.0.
ad-transcripts. We collect ad scripts from a variety of industries from Hartman (2019). Ad transcripts are directly downloaded from Kaggle. The top eight industries by frequency are selected. Newlines are replaced with spaces. The dataset is downloadable from https://www.kaggle.com/datasets/kevinhartman0/advertisement-transcripts-from- various-industries with license CC0 Public Domain.
admin-statements. We collect statements of administration policy from American presidents from Progress (2022). Administration statements are extracted from a collection hosted on GitHub. Extraneous symbols are removed and samples are split by paragraph. The dataset is downloadable from https://github.com/unitedstates/statements-of-administration-policy#statements-of-administration-policy and origin files have a Creative Commons Attribution 3.0 License.
ai2-natural-instruction. We collect a learning-from-instructions dataset released by the Allen Institute for AI from Mishra et al. (2022). Natural instruction tasks are directly downloaded without modification. The dataset is released under an Apache-2.0 license.
airline-reviews. We collect reviews of airlines collected from the review website Skytrax. Airline reviews for airlines, airports, and seats are downloaded from a public GitHub repository. Names of aircraft, airlines, countries, and traveler types are standardized. Ratings of 1, 4, or 5 on a scale of 5, and 1, 5, 8, or 10 on a scale of 10 are kept. This dataset can be downloaded via https://github.com/quankiquanki/skytrax-reviews-dataset.
aita. We collect posts on the "Am I The Asshole" Subreddit, an online forum people ask others whether they were in the wrong from O'Brien (2020). Posts from r/AmITheAsshole are downloaded from a praw scrape of Reddit. Topic areas are chosen based on common themes in posts and coarsely defined based on manual keywords. Each post can belong to multiple topic areas. The dataset can be downloaded at https://doi.org/10.5281/zenodo.3677563.
all-the-news. We collect news articles collected from various outlets between 2015 and 2017 from Thompson (2019). News articles are downloaded directly from the Components website. The titles are used as text samples. The dataset can be downloaded at https://components.one/datasets/all-the-news-articles-dataset.
amazon-reviews. We collect Amazon reviews collected from various product categories from Ni et al. (2019). Amazon reviews are downloaded from a 2018 crawl of the website. The first 100,000 review texts are treated as the text sample. The dataset can be downloaded at https://nijianmo.github.io/amazon/index.html.
armenian-jobs. We collect job postings in Armenia from Udacity (2017). The Armenian job postings dataset is downloaded from a snapshot on GitHub. Different IT jobs are manually coded and time intervals are defined in order to balance sample availability. The dataset can be downloaded at https://www.kaggle.com/datasets/udacity/armenian-online-job-postings.
boolq. We collect a reading comprehension dataset of yes/no questions from Clark et al. (2019). Boolean questions are downloaded directly as is. The dataset can be downloaded at https://github.com/google-research-datasets/boolean-questions with license CC-SA-3.0.
clickbait-headlines. We collect headlines across time from the Examiner, a clickbait news site from Kulkarni (2020a). The Examiner headlines are directly downloaded from Kaggle. The year is extracted from the publication date field. Samples are constructed from the headline text. The dataset can be downloaded at https://www.kaggle.com/datasets/therohk/examine-the-examiner, with license CC0: public domain.
convincing-arguments. We collect arguments on a variety of topics annotated for convincingness from Habernal & Gurevych (2016). Annotated arguments are downloaded from the GitHub repository. Arguments are sorted by rank. The bottom 400 are treated as "unconvincing", the top 200 are treated as "convincing", and the next 200 are treated as "somewhat convincing." The dataset can be downloaded at https://github.com/UKPLab/acl2016-convincing-arguments, with license CC-BY 4.0.
craiglist-negotiations. We collect dialogue from Craigslist negotiations, an online seller platform from He et al. (2018). Craigslist negotiations are downloaded from Huggingface. Sequences which contained a "quit" intention or "reject" intention are categorized as failures; those which contained an "accept" intention are categorized as successes. The mid-price is defined as the mean