The Case for Scalable, Data-Driven Theory: A Paradigm for Scientific Progress in NLP (2023)

Abstract: I propose a paradigm for scientific progress in NLP centered around developing scalable, data-driven theories of linguistic structure. The idea is to collect data in tightly scoped, carefully defined ways which allow for exhaustive annotation of behavioral phenomena of interest, and then use machine learning to construct explanatory theories of these phenomena which can form building blocks for intelligible AI systems. After laying some conceptual groundwork, I describe several investigations into data-driven theories of shallow semantic structure using Question-Answer driven Semantic Role Labeling (QA-SRL), a schema for annotating verbal predicate-argument relations using highly constrained question-answer pairs. While this only scratches the surface of the complex language behaviors of interest in AI, I outline principles for data collection and theoretical modeling which can inform future scientific progress. This note summarizes and draws heavily on my PhD thesis.

## introduction formal representations of linguistic structure and meaning have long guided our understanding of how to build nlp systems, e.g., in the traditional nlp pipeline @xcite . however, this approach has always had limitations: together with the effectiveness of deep learning, these challenges led to the proliferation of end-toend neural network models which directly perform tasks without intermediate formal representations of linguistic structure @xcite @xcite . this trend continues with language model assistants like @xcite and claude @xcite which can perform a wide range of tasks. however, these systems are still not robust, often reporting false or biased answers @xcite and making false claims about their own reasoning @xcite . ensuring ai systems' robustness requires us to precisely characterize and control their generalization behaviors. to this end, formal theories, e.g., of linguistic structure, common sense, reasoning, and world knowledge, provide frameworks for evaluation. they inform the design and construction of challenge sets @xcite @xcite , measures of systematicity @xcite , behavioral tests @xcite , and probing experiments @xcite . as these theories allow us to characterize generalization behaviors we desire, they will likely play a pivotal role in the design and training of trustworthy systems. so core improvements in formal theories of aspects of intelligent behavior may yield boons for both the construction and evaluation of nlp systems. but the question remains of how to achieve this: decades of work on semantic ontologies @xcite , commonsense knowledge bases @xcite , and formal reasoning systems @xcite have largely been superseded in nlp by deep learning and language models. theory-driven approaches in ai have been so disappointing that @xcite famously argues that intelligence and the world are simply too complex for us to capture with domain theories, and we should instead focus on general-purpose learning systems that can capture this intrinsic complexity from data. however, i believe this is too pessimistic, giving up on the intelligibility of ai systems that is provided by accurate theories of their behavior, which is necessary for verifying their safety and usefulness in high-risk, high capability settings @xcite . instead, the deep learning era presents an opportunity to rethink how we develop theories of language behavior. in particular, i propose scalable, data-driven theory as a paradigm to address the shortcomings mentioned at the beginning of this article: resolving or sidestepping theoretical questions, producing representations with broad coverage, and applying them effectively in downstream tasks. inspired by pragmatist epistemology @xcite , this approach avoids requiring the linguist or theoretician to specify the entire theory by hand, instead integrating machine learning in a judicious way which allows for the scalable, automated induction of formal theoretical constructs (e.g., ontologies) which are grounded in task-relevant linguistic behaviors. ## pragmatist principles for scientific progress @xcite describes the history of computational linguistics on a pendulum, swinging between rationalist (theory-driven) and empiricist (datadriven) paradigms every 20 years. church lists the "swings" as follows (with my comments): why is this? sutton may say it's because the world is too complex: the rationalist theoretician carefully formalizing the problems at hand has no hope of capturing the world's intricacies in a manually-crafted theory, though a system implementing that theory can be understood and controlled. the empiricist tinkerer, on the other hand, can build a system that mostly works by trial, error, patching and fastening; so they win on empirical benchmarks. however, the resulting system is too complex to fully understand or control, and generalizes in unpredictable ways. an odd feature of the rationalism/empiricism dichotomy is that neither epistemology accurately describes the pursuit of science in most fields. in fields like physics, chemistry, and biology, theoretical and experimental approaches are not in conflict; rather, they synergize and inform each other, as theories are continually updated to align with new experimental data. to make sense of this, we can turn to an epistemology inspired by how people actually operate in the world: pragmatism. pragmatism is an epistemological framework which conceptualizes knowing in terms of the actions that the knowledge licenses, i.e., by the predictions that follow from that knowledge. prominent pragmatists include charles sanders peirce and @xcite . like empiricism, pragmatism embraces experience as the primary source of knowledge. but unlike empiricists, pragmatists such as james embrace formal and linguistic categories as comprising the content of knowledge, on the basis of their usefulness in making predictions and licensing actions @xcite . unlike in rationalism, the pragmatist search for truth is not a search for one true theory which fundamentally describes the world, but for an everexpanding set of theoretical tools and concepts that can be picked up and put down according to the needs of the knower. in pithy terms, a pragmatist might agree with the statistical aphorism that that "all models are wrong; some are useful" @xcite . pragmatists such as @xcite claim that this perspective more accurately describes human behavior with respect to knowledge (and indeed, the pursuit of science) than prior epistemologies. combining the core ideology of pragmatism with observations from computational linguistics, we can derive two guiding principles for the development of theories that may have prospective use in nlp: decouple data from theory (section 2.1), and make data reflect use (section 2.2). ## scalable, data-driven theory the principles in section 2 imply a general framework for building useful theories, which i call datadriven theory: first, annotate data in a theoreticallyminimal way, scoped carefully to reflect specific phenomena that we want to explain; then, automatically induce theories to explain those phenomena using computational methods like machine learning. but how does this method scale in practice? even if the resulting theories are high-quality, requiring annotated data limits their scope to orders of magnitude less than what is leveraged by standard pretrained models @xcite @xcite . black-box data simulators this is where blackbox models may actually be able to help. even if they are uninterpretable on their own, their high accuracy and data efficiency means they can be used as data simulators, generating phenomenological data -potentially at a level of granularity or exhaustivity unobtainable from humans -which can be fed into another, more interpretable algorithm to distill a theory from it. this is the approach we take in @xcite , de-scribed in section 5: we first train a black-box model to generate qa-srl questions, where each role is labeled with only a single question in the training data. then we decode full question distributions from this model, and induce an ontology of semantic roles by clustering arguments based on the overlap of their question distributions. while this work required a large training set of qa-srl annotations @xcite , it may now be possible to do such experiments without largescale human data annotation at all, thanks to recent advances in instruction following by language models @xcite . it may seem like the use of a black-box model as a data simulator begs the question: if our concern is that the black-box model isn't learning the underlying function we hope it is, then doesn't using it to simulate data risk leading us to a theory of the wrong function? well, yes -but the theory lets us do something about it. examining the "wrong" parts of the resulting theory (e.g., induced semantic roles that don't match what we intuitively expect, or that lead to downstream predictions we think are wrong), and their connection to the training data, will identify one of the following: all of these are positive outcomes for scientific progress. see @xcite for an in-depth analysis of this kind. ## data: scoping language behaviors the first step to developing theories of linguistic structure in an empirical, data-driven way is to carefully choose the data. to guide this, i propose four principles of scientific data for nlp: principles 1 and 2 instantiate section 2.1's recommendation to decouple data from theory, while principles 2, 3 and 4 help make it tractable to develop broad-coverage, comprehensible theories from this data. the final requirement is that the data reflect relevant downstream use cases (section 2.2), which in our case means it should encode phenomena representing the intended behavior of ai systems performing language tasks. foot_1 i focus on a key strategy to meet these requirements: annotating natural language with natural language question-answer pairs. question answering has long been used as a general-purpose format for testing language comprehension or executing practical language tasks @xcite @xcite , as nearly any task can be phrased as a question and questions which test a reader's comprehension of a text need not require specialized linguistic or theoretical expertise to answer. the downside of this great generality is that data annotation tends to be highly under-constrained and unsystematic @xcite , so we must judiciously constrain the space of question-answer pairs we use in accordance with the four principles. this work is focused on annotations of shallow semantic structure: syntax, semantic roles, and other predicate-argument structure relations expressed in text. @xcite pioneered the use of question-answer pairs as a proxy for such structure in question-answer driven semantic role labeling (qa-srl), a framework for annotating english verbal predicate-argument relations using simple, highly constrained question-answer pairs. in the rest of this section, i will describe three data annotation projects which explored variations of this approach, illustrating some of the basic tensions between the four principles. i want to eat the pizza you put on the table subj xcomp subj obj obj subj prep pobj subj verb obj prep xcomp you put the pizza on the table i want to eat the pizza i eat the pizza dependency question answer @xmath0: question-answer pair generation for humanin-the-loop parsing @xcite . we use the predicted ccg category of each verb to generate the questions, which are in in one-to-one relation with syntactic dependencies in the sentence. this one-to-one assumption was ultimately too strong, as workers answer these questions according to semantics and not just syntax. ## theory: from language, structure in this section, i will describe two projects which show how qa-srl can be used to build a datadriven theory which is directly applicable in downstream tasks.