SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Finally, to improve the accessibility of our research, we ran the same experiments using gpt-3.5-turbo and f1an-t5-xxl as our proposer, and report the results in Appendix Table 6. To show that our conclusions are general and not only apply to synthetically generated texts, we additionally constructed an extension of SYND5 with human-written texts by adapting the NYT dataset from Wang et al. (2023), where each text sample is a New York Times article with a topic and a location label: the topic dimension has 9 different values (e.g., politics, arts) and the location dimension has 10 different values (French, Italy); we then followed the same procedure described in Section 2.2 to create this extension of SYND5, and report our systems' performance in Appendix Table 7. In all experiments, using the validator and the goal improves the performance.
Investigating whether using goals improves relevance on OPEND5. We then investigated whether text-davinci-003 can leverage the goals to propose more relevant hypotheses on more realistic applications in OPEND5. We sampled 100 problems from OPEND5 with distinct goals and randomly sampled 2 hypotheses from text-davinci-003 with/without using goals (see Figure 3), resulting in 400 hypotheses to evaluate. Three authors then rated their relevance based on the rubric in Section 3, while being blinded about which hypotheses were generated with the goal. Our main paper focuses on presenting the evaluations performed by ourselves, since crowdworkers might be noisy and untrustworthy (Veselovsky et al., 2023; Suhr et al., 2021).
We report the results in Table 5. Since this evaluation is subjective, the inter-annotator agreement is only moderate (Kappa=0.56); however, we can still robustly conclude that text-davinci-003 can leverage goals to propose hypotheses with higher average relevance rating, since this conclusion can
be independently reproduced by every individual evaluator with  $p < 10^{-8}$ . To make sure that the same conclusion can be robustly reproduced by external non-authors, we also evaluated the relevance of the hypotheses with Amazon Mechanical Turks, gpt-3.5-turbo, Claude-v1.3, and gpt-4. We report the results in Appendix Table 8 and found that our conclusion robustly holds under five different types of evaluators, including expert authors, external crowdworkers, and language models from different companies with different levels of capabilities.
Finally, we conducted similar experiments for the novelty and significance metrics in Appendix 14 and found that they both benefit from using goals as well. In the next section, we present example discoveries on OPEND5 to qualitatively understand what a D5 system can achieve.
6 Qualitatively Analyzing Discoveries and Limitations with OPEND5
To understand the utility and the limitation of a D5 system, we ran it on OPEND5, a set of realistic D5 problems, and analyze the output discoveries qualitatively.
6.1 Producing Discoveries on OPEND5 and Analyzing Them
We ran our D5 system on OPEND5, producing 3296 discoveries in total. However, we do not have enough budget to validate every finding, since estimating  $V$  is expensive (Section 3). Therefore, from the remaining 3296 discoveries, we manually selected 21 discoveries that 1) achieve a relevance score of ②, 2) are representative of potential use cases, 3) do not require expert knowledge for Turkers to judge, and 4) are likely to achieve a small  $p$ -value with fewer than 200 samples from  $\mathcal{D}^{\mathrm{val}}$ .
We then estimated their validity based on the procedure described in Section 3 by using fewer than 200 samples from the validation split and calculated the  $p$ -values, which cost us  $\sim$ 1500 in total on MTurk. Since we are testing multiple discoveries and each of them can be statistically significant merely due to chance, we keep 13 discoveries with  $V$  that are significantly non-zero with  $p$ -value below  $7\%$ , a threshold determined by the Benjamini Hochberg's procedure with a false discovery rate of  $10\%$ . In other words,  $< 10\%$  of the discoveries presented are false discoveries in expectation.
We detail 5 of the 13 discoveries in this section, with the remainder in Appendix 18. For each discovery, we report its automated validity score  $V'$ , the estimated true validity score  $V$ , and their respective  $p$  values in Table 3.
Understanding political stances and stereotypes in speeches. When comparing presidential speeches on immigrants from Obama to those from Trump, the former "argues for a path forward to promote the fair and just treatment of immigrants", while the latter more frequently "refers to illegal immigrants as criminals".
Analyzing errors in NLP systems. We fine-tuned a pair of models on two different natural language inference datasets, (a) MNLI and (b) SNLI. To understand their patterns of errors, we defined Corpus A to be the subset of MNLI where a is right and b is wrong, and Corpus B to be where b is right and a is wrong. We found that the latter more often "has an informal tone, such as slang or colloquial speech". One possible explanation is that MNLI contains more different genres and hence more informal speeches, causing the former model to perform better on these examples.
Output discovery	V	p	V'	p'
“argues for a path forward to promote the fair ...”	0.16	1.26e-04	0.35	2.01e-73
“refers to illegal immigrants as criminals”	0.09	6.17e-03	0.19	3.17e-38
“has an informal tone, such as slang or colloqu...”	0.08	2.35e-03	0.24	1.46e-35
“mentions lack of legroom”	0.16	1.15e-03	0.38	1.34e-45
“mentions children or family”	0.08	1.00e-05	0.11	8.05e-09
Table 3: A subset of discoveries presented in Section 6.1 and their associated estimated validity score  $V$ , validity score approximated by a model  $V'$ , and their respective  $p$ -values  $p(p')$  for the null hypothesis that  $V(V') < 0$  under a t-test. We present the full set of 13 discoveries in Table 10.
Analyzing airline customer reviews. We compared the concerns in reviews of the airline Air Canada v.s. its subsidiary, Air Canada Rogue, which is considered a low-price wing of Air Canada. The latter more often "mentions lack of legroom".
Analyzing gender differences in self-reported happy moments. Compared to self-reported happy moments written by males, those by females "mentions children or family" more often. Caution: misinterpreting this correlation as causation could reinforce societal biases (Section 6.2).
Due to space constraints, we list more examples on analyzing distribution shifts, text clusters, lyric styles, and news headlines in Appendix 18 and their associated  $V$  and  $V'$  values in Appendix Table 10. Across these discoveries, the approximated validity score  $V'$  has a  $71\%$  spearman rank correlation with human rating  $V$  (66% for Pearson correlation), thus providing informative yet unreliable signals to practitioners about their validity. We hope that  $V'$  can better approximate  $V$  values in the future as the quality of the annotators improve. Finally, future works can collect more open problems, allowing D5 systems to produce more impactful discoveries.
6.2 Concrete Examples in OPEND5 Inform Limitations of D5 Evaluation
We discuss limitations of D5 evaluation in this section using concrete examples from OPEND5.
Our metrics do not evaluate diversity. There are often multiple valid and relevant discoveries, and our system ideally should generate all of them. For example, when comparing low-rating and high-rating reviews to understand what stands out to customers, both "mentions the hidden fees and poor customer service at the airport" and "mentions the airline charging extra for carry-on items" could be valid discoveries. Our current evaluation does not reward diverse discoveries, and the current system sometimes repeats a discovery using similar paraphrases, e.g., "mentions the rude and unprofessional attitude of the staff" and "mentions the staff being rude and unhelpful". Future evaluation metrics can take diversity into account.
Interpreting discoveries requires domain experts. We used Turkers' judgment when computing  $T(h, x)$  to judge the validity of a discovery. However, many discoveries require expert knowledge to interpret properly. For example, it requires medical training to reliably judge whether a self-reported drug-use experience satisfies "mentions psychedelics, such as LSD and shrooms."
Correlation  $\neq$  causation. Our metrics currently do not evaluate whether the discovery is causally related to how the corpus pair was generated. For example, when comparing self-reported happy moments from females and males, even if the former corpus has more samples that "mention children and family", it does not necessarily imply family plays a more important role in inter-personal relations for females; an alternative hypothesis is that females might mention people in general more often than males do, hence leading to the observation that they mention family more often. Spurious correlations could also sneak into our validity evaluation: for example, if the Turkers implicitly associate female activities as family-related Greenwald & Banaji (1995), then we might falsely make this discovery due to evaluator biases. Future metrics should also consider plausible alternative hypotheses to evaluate causality and control the potential biases from the human evaluators. Additionally, we should treat the discovery from D5 with caution to prevent automating and amplifying societal biases.
We discuss other limitations, such as restricting the discovery to be a single predicate, the biases in authors' qualitative evaluation, and the incomprehensiveness of OPEND5 in Appendix 19.
7 Related Work and Discussion
Inductive Reasoning with NLP Models. Recent works show that language models are capable of inductive reasoning under restricted settings, discovering patterns from a set of text data points and describing them with language (Honovich et al., 2022). Yang et al. (2022) use this capability to induce natural language rules with the format of "if...then..." Zhou et al. (2022) and Ye et al. (2022) use this capability to improve zero/few-shot accuracy by inferring the most likely instruction using input-output example(s) of the target task. Zhong et al. (2022) and Singh et al. (2022) use this capability to discover patterns in datasets, and we improve by building an automatic benchmark and a dataset of open-ended problems and require the discovery to be relevant.
ML models can also perform inductive reasoning in other modalities, such as vision. Hernandez et al. (2021) describes visual features that activate a neuron; Zhu et al. (2022) describes distribution
shifts between the training distribution and the test distribution for images; and Eyuboglu et al. (2022) describes errors made by vision models. We hope future models can perform inductive reasoning in other modalities, such as sound (Aghajanyan et al., 2023) or physical senses (Thomason et al., 2016).
Exploratory Analysis and Automated Discovery. It is not new to automatically discover patterns by learning from empirical data. To list a few classical methods, linear regression analyzes the effect of each real-valued feature by interpreting the learned weights (Draper & Smith, 1998); n-gram models can extract discriminative phrases, thus yielding insights about corpus-level differences (Manning & Schutze, 1999); topic models (Blei et al., 2003) can extract major topical variations across documents, where each topic is represented as a distribution over words; small decision trees can extract interpretable if-then statements (Letham et al., 2015); and an entity embedding model learned on existing relations between entities can predict unseen relations (Socher et al., 2013). In comparison, D5 produces discoveries in the form of natural language predicates, which are interpretable and can express abstract concepts; additionally, it is more directed at the goal, while machine learning classifiers like naive bayes or linear regression will pick up any discriminative features: Appendix 21 offers a more comprehensive discussion using examples from SYND5. Given the respective strength of D5 and traditional exploratory methods, we envision D5 to serve as a complementary method to traditional methods.
Epistemology. While the process of validating a hypothesis is well-formulated, it is much less well-understood how to automatically generate hypotheses and decide what discoveries are meaningful (Shapere, 1964; Heckman & Singer, 2017). Related works in this area have been sparse, among which McGarry (2005) sketches high-level principles for evaluating knowledge discoveries and Ludwig & Mullainathan (2022) proposes to crowd-source hypotheses from MTurk workers. We concur with the perspective of Polanyi et al. (2000) that meaningfulness of a hypothesis cannot be explicitly verbalized with simple logic but is dependent on implicit community norms; therefore, the process of proposing hypotheses should be learned from empirical data (e.g. pre-training, self-training, or human feedback) rather than deduced from a priori analysis of concepts (Quine, 1969). We hope contributions from other domains can provide more empirical data on what discoveries are meaningful, hence guiding our system to produce more important discoveries.
We thank Xiaochuang Han and Sam Bowman for their early discussions on this project. We thank Cathy Chen, Erik Jones, Jessy Lin, Alex Pan, Chenglei Si, Xi Ye, and Tianyi Zhang for their helpful feedback on the paper draft. We thank OpenAI and Anthropic for providing model access.
The hewlett foundation: Automated essay scoring, 2012. URL https://kaggle.com/competitions/asap-aes.
The hewlett foundation: Short answer scoring, 2013. URL https://kaggle.com/competitions/asap-sas.
Ad observer. https://adobserver.org/, 2021. Accessed: 2022-12-30.
Armen Aghajanyan, Lili Yu, Alexis Conneau, Wei-Ning Hsu, Karen Hambardzumyan, Susan Zhang, Stephen Roller, Naman Goyal, Omer Levy, and Luke Zettlemoyer. Scaling laws for generative mixed-modal language models. arXiv preprint arXiv:2301.03728, 2023.
Roee Aharoni and Yoav Goldberg. Unsupervised domain clusters in pretrained language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7747-7763, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.692. URL https://aclanthology.org/2020.acl-main.692.
Mohammad Alali, Shaayan Syed, Mohammed Alsayed, Smit Patel, and Hemanth Bodala. Justice: A benchmark dataset for supreme court's judgment prediction. arXiv preprint arXiv:2112.03414, 2021.
Zhi Liu. Reuter_50_50 Data Set, 2011. URL https://archive.ics.uci.edu/ml/datasets/Reuter_50_50.
Jens Ludwig and Sendhil Mullainathan. Algorithmic behavioral science: Machine learning as a tool for scientific discovery. Chicago Booth Research Paper, (22-15), 2022.
Christopher Manning and Hinrich Schutze. Foundations of statistical natural language processing. MIT press, 1999.
Ken McGarry. A survey of interestingness measures for knowledge discovery. The knowledge engineering review, 20(1):39-61, 2005.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016.
Natan Mish. Federal Reserve Governors Speeches 1996 - 2020, 2020. URL https://tinyurl.com/3j2e79a6.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. In ACL, 2022.
Rishabh Misra and Prahal Arora. Sarcasm detection using hybrid neural network. arXiv preprint arXiv:1908.07414, 2019.
Rishabh Misra and Jigyasa Grover. *Sculpting Data for ML: The first act of Machine Learning*. 01 2021. ISBN 9798585463570.
Nuno Moniz and Luâ°TMis Torgo. Multi-source social feedback of online news feeds. CoRR, [Web Link], 2018.
Mickaël Mouillé. *Kickstarter Projects*, 2017. URL https://www.kaggle.com/datasets/ kemical/kickstarter-projects?select=ks-projects-201612.csv.
Dong Nguyen, Maria Liakata, Simon DeDeo, Jacob Eisenstein, David Mimno, Rebekah Tromble, and Jane Winters. How we do things with words: Analyzing text as social and cultural data. Frontiers in Artificial Intelligence, 3:62, 2020.
Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp. 188-197, 2019.
Elle O'Brien. iterative/aita_dataset: Praw rescrape of entire dataset, February 2020. URL https://doi.org/10.5281/zenodo.3677563.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155, 2022.
Verónica Pérez-Rosas and Rada Mihalcea. Experiments in open domain deception detection. In Proceedings of the 2015 conference on empirical methods in natural language processing, pp. 1120-1125, 2015.
Verónica Pérez-Rosas, Mohamed Abouelenien, Rada Mihalcea, and Mihai Burzo. Deception detection using real-life trial data. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 59-66, 2015.
Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. Automatic detection of fake news. arXiv preprint arXiv:1708.07104, 2017.
Denis Peskov, Benny Cheng, Ahmed Elgohary, Joe Barrow, Cristian Danescu-Niculescu-Mizil, and Jordan Boyd-Graber. It takes two to lie: One to lie and one to listen. In Association for Computational Linguistics, 2020.
John P. Pestian, Chris Brew, Pawel Matykiewicz, DJ Hovermale, Neil Johnson, K. Bretonnel Cohen, and Wlodzislaw Duch. A shared task involving multi-label classification of clinical free text. In Biological, translational, and clinical language processing, pp. 97-104, Prague, Czech Republic, June 2007. Association for Computational Linguistics. URL https://aclanthology.org/W07-1013.
Michael Polanyi, John Ziman, and Steve Fuller. The republic of science: its political and economic theory minerva, i (1)(1962), 54-73. Minerva, 38(1):1-32, 2000.
Ilan Price, Jordan Gifford-Moore, Jory Fleming, Saul Musker, Maayan Roichman, Guillaume Sylvain, Nithum Thain, Lucas Dixon, and Jeffrey Sorensen. Six attributes of unhealthy conversation. arXiv preprint arXiv:2010.07410, 2020.
Demand Progress. Statements of Administration Policy, 2022. URL https://github.com/unitedstates/statements-of-administration-policy#statements-of-administration-policy.
PromptCloud. U.S. Technology Jobs on Dice.com, 2017. URL https://www.kaggle.com/datasets/PromptCloudHQ/us-technology-jobs-on-dicecom.
WVO Quine. Naturalistic epistemology. Ontological relativity and other essays, pp. 69-90, 1969.
Quora. Quora Question Pairs, 2017. URL https://www.kaggle.com/c/quora-question-pairs.
Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable questions for squad. arXiv preprint arXiv:1806.03822, 2018.
Justin Robischon. Wikipedia Movie Plots, 2019. URL https://www.kaggle.com/datasets/jrobischon/wikipedia-movie-plots.
Allen Roush and Arvind Balaji. DebateSum: A large-scale argument mining and summarization dataset. In Proceedings of the 7th Workshop on Argument Mining, pp. 1-7, Online, December 2020. Association for Computational Linguistics. URL https://aclanthology.org/2020辩论-1.1.
Dudley Shapere. The structure of scientific revolutions. The Philosophical Review, 73(3):383-394, 1964.
Chandan Singh, John X Morris, Jyoti Aneja, Alexander M Rush, and Jianfeng Gao. Explaining patterns in data with language models via interpretable autoprompting. arXiv preprint arXiv:2210.01848, 2022.
Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. Reasoning with neural tensor networks for knowledge base completion. Advances in neural information processing systems, 26, 2013.
Alane Suhr, Clara Vania, Nikita Nangia, Maarten Sap, Mark Yatskar, Samuel Bowman, and Yoav Artzi. Crowdsourcing beyond annotation: Case studies in benchmark data collection. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts, pp. 1-6, 2021.
J Sun. Daily News for Stock Market Prediction, 2017. URL https://www.kaggle.com/datasets/aaron7sun/stocknews.
Jesse Thomason, Jivko Sinapov, Maxwell Svetlik, Peter Stone, and Raymond J Mooney. Learning multi-modal grounded linguistic semantics by playing "i spy". In *IJCAI*, pp. 3477-3483, 2016.
Andrew Thompson. All the News 1.0, 2019. URL https://components.one/datasets/all-the-news-articles-dataset.
Elsbeth Turcan and Kathleen McKeown. Dreaddit: A reddit dataset for stress analysis in social media. arXiv preprint arXiv:1911.00133, 2019.
Udacity. Armenian Online Job Postings, 2017. URL https://www.kaggle.com/datasets/udacity/armenian-online-job-postings.
Veniamin Veselovsky, Manoel Horta Ribeiro, and Robert West. Artificial artificial artificial intelligence: Crowd workers widely use large language models for text production tasks. arXiv preprint arXiv:2306.07899, 2023.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. URL https://arxiv.org/abs/2204.07705, 2022.
Zihan Wang, Jingbo Shang, and Ruiqi Zhong. Goal-driven explainable clustering via language descriptions. arXiv preprint arXiv:2305.13749, 2023.
Orion Weller and Kevin Seppi. The rJokes dataset: a large scale humor collection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 6136-6141, Marseille, France, May 2020. European Language Resources Association. ISBN 979-10-95546-34-4. URL https://aclanthology.org/2020.lrec-1.753.
Adina Williams, Nikita Nangia, and Samuel R Bowman. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426, 2017.
Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, and Furu Wei. Language models as inductive reasoners. arXiv preprint arXiv:2212.10923, 2022.
Seonghyeon Ye, Doyoung Kim, Joel Jang, Joongbo Shin, and Minjoon Seo. Guess the instruction! making language models stronger zero-shot learners. arXiv preprint arXiv:2210.02969, 2022.
Ruiqi Zhong, Charlie Snell, Dan Klein, and Jacob Steinhardt. Describing differences between text distributions with natural language. In International Conference on Machine Learning, pp. 27099-27116. PMLR, 2022.
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910, 2022.
Zhiying Zhu, Weixin Liang, and James Zou. Gsclip: A framework for explaining distribution shifts in natural language. arXiv preprint arXiv:2206.15007, 2022.
8 Cost for Running the Experiments
For each problem, we ran the proposer for 10 times on average; assuming each prompt to be at most 4000 tokens, we spent around $2.4 for each problem on OpenAI APIs if we use gpt-4 and text-davinci-003, and the cost would decrease to $0.8 if we use gpt-3.5-turbo. Notice that these estimates are computed based on the prices as of 05/14/2023, and we expect the price to further decrease in the future. We ran the Flan-T5 based validator for 2 hours on 180G A100 GPUs.
The total amount of computational resources spent in this research paper is around $2,500 in terms of OpenAI API and 3,000 hours of compute on A100 GPU with 80G memory.
9 Generation Process of SYND5
The high-level description is in Section 2.2. Here we discuss the procedure that generated SYND5.