Translational NLP : A New Paradigm and General Principles for Natural Language Processing Research (2021)

Abstract: Natural language processing (NLP) research combines the study of universal principles, through basic science, with applied science targeting specific use cases and settings. However, the process of exchange between basic NLP and applications is often assumed to emerge naturally, resulting in many innovations going unapplied and many important questions left unstudied. We describe a new paradigm of Translational NLP, which aims to structure and facilitate the processes by which basic and applied NLP research inform one another. Translational NLP thus presents a third research paradigm, focused on understanding the challenges posed by application needs and how these challenges can drive innovation in basic science and technology design. We show that many significant advances in NLP research have emerged from the intersection of basic principles with application needs, and present a conceptual framework outlining the stakeholders and key questions in translational research. Our framework provides a roadmap for developing Translational NLP as a dedicated research area, and identifies general translational principles to facilitate exchange between basic and applied research.

## introduction natural language processing (nlp) lies at the intersection of basic science and applied technologies. however, translating innovations in basic nlp methods to successful applications remains a difficult task in which failure points often appear late in the development process, delaying or preventing potential impact in research and industry. application challenges range widely, from changes in data distributions @xcite to computational bottlenecks @xcite and integration with domain expertise @xcite . when unanticipated, such challenges can be fatal to applications of new nlp methodologies, leaving exciting innovations with minimal practical impact. meanwhile, real-world applications may rely on regular expressions @xcite or unigram frequencies @xcite when more sophisticated methods would yield deeper insight. when successful translations of basic nlp insights into practical applied technologies do occur, the factors contributing to this success are rarely analyzed, limiting our ability to learn how to enable the next project and the next technology. we argue for a third kind of nlp research, which we call translational nlp. translational nlp research aims to understand why one translation succeeds while another fails, and to develop general, reusable processes to facilitate more (and easier) translation between basic nlp advances and realworld application settings. much nlp research already includes translational insights, but often considers them properties of a specific application rather than generalizable findings that can advance the field. this paper illustrates why general principles of the translational process enhance mutual exchange between linguistic inquiry, model development, and application research (illustrated in figure 1 ), and are key drivers of nlp advances. we present a conceptual framework for translational nlp, with specific elements of the translational process that are key to successful applications, each of which presents distinct areas for research. our framework provides a concrete path for designing use-inspired basic research so that research products can effectively be turned into practical technologies, and provides the tools to understand why a technology translation succeeds or fails. a translational perspective further enables factorizing "grand challenge" research questions into clearly-defined pieces, producing intermediate results and driving new basic research questions. our paper makes the following contributions: ## the translational nlp framework we present a conceptual framework for translational nlp, to formalize shared principles describing how basic and applied research interact to cre-ate nlp solutions. our framework codifies fundamental variables in this process, providing a roadmap for negotiating the design of methodological innovations with an eye towards potential applications. although it is certainly not the case that every basic research advance must be tied to a downstream application need, designing foundational technologies for potential application from the beginning produces more robust technologies that are easier to transfer to practical settings, increasing the impact of basic research. by defining common variables, our framework also provides a structure for aligning application needs to basic technologies, helping to identify potential failure points and new research needs early for faster adoption of basic nlp advances. our framework has two components: ## case study: nlp for disability review we illustrate our translational nlp framework using our recent line of research on developing nlp tools to assist us social security administration (ssa) officials in reviewing applications for disability benefits @xcite . the goal of this effort was to help identify relevant pieces of medical evidence for making decisions about disability benefits, analyzing vast quantities of medical records collected during the review process. the stakeholders in this setting included: nlp researchers (interested in developing generalizable methods); subject matter experts in disability and rehabilitation; and ssa end users (limited computing, large data but strictly controlled, overall priorities of efficiency and accuracy). the translational nlp checklist for this setting is shown in while these studies do not systematically explore evaluation, interpretation, or application engineering, they illustrate how the characteristics of one application setting can lead to a line of translational nlp research with broader implications. several further challenges of this application area remain unstudied: for example, representing and modeling the complex timelines of persons with chronic health conditions and intermittent health care and adapting nlp systems to highly variable medical language from practitioners and patients around the us. these present intriguing challenges for basic nlp research that can inform many other applications beyond this case study. of course, these studies are far from the only examples of translational nlp research. many studies tackle translational questions, from domain adaptation (shifts in data characteristics) and lowresource learning (limited available resources), and the growing nlp literature in domain-specific venues such as medical research, law, finance, and more involves all aspects of the translational process. rather, this case study is simply one illustration of how an explicitly translational perspective in study design can identify and connect broad opportunities for contributions to nlp research. information need overall goal: improve disability benefits review process by highlighting relevant information formal representation: spans of evidence, with attributes for activity type and level of limitation data characteristics medical records and administrative forms from usa, mostly english task paradigms information extraction (spans), information retrieval (documents), span classification (activity and limitations) available resources minimal knowledge sources for function and disability, no public corpora; us government computing systems; high throughput requirements (thousands of records/day) nlp technologies low-latency, low-compute sequence models; rule-based systems evaluation standard metrics (f-1, accuracy). information retrieval metrics reported for use case prototypes. interpretation interpretation needs primarily around human decision-making; nlp tools highlight and organize information in context. no ml interpretability reported in published results. application engineering open-source implementations using standardized frameworks for preprocessing. no data exchange reported. ## discussion our paradigm of translational nlp defines and gives structure to a valuable area of research not explicitly represented in the acl community. we note that translational research is not meant to replace either basic or applied research, nor do we intend to say that all basic nlp studies must be tied to specific application needs. rather we aim to highlight the value of studying the processes of turning basic innovations into successful applications. these processes, from scaling model computation to redesigning tools to meet changing application needs, can inform new research in model design, domain adaptation, etc., and can help us understand why some tools succeed in application while others fail. in addition to helping more innovations successfully translate, the principles outlined in this paper can be of use to basic and applied nlp researchers as well as translational ones, in identifying common variables and concerns to connect new work to the broader community. translational research is equally at home in industry and academia, and already occurring in both. while resource disparities between industrial and academic research increasingly push large-scale modeling efforts out of reach of academic teams, a translational lens can help to identify rich areas of knowledge-driven study that do not require exascale data or computing resources. the general principles and interdisciplinary nature of translational research make it a natural fit for public knowledgedriven academic settings, while its applicability to commercial needs is highly relevant to industry. our framework provides a starting point for the translational process, which will evolve differently for every project. the specifics of different applications will expand our initial questions in different ways (e.g., "data characteristics" may involve multimodal data, or different language styles), and the dynamics of collaborations will shift answers over time (e.g., a change in evaluation criteria may motivate different model training approaches). our checklist provides a minimal set of common questions, and can function as a touchstone for discussions throughout the research process, but it can and should be tailored to the nature of each project. our framework is itself a preliminary characterization of translational nlp research, and will evolve over time as the field continues to develop. ## conclusion we have outlined a new model of nlp research, translational nlp, which aims to bridge the gap between basic and applied nlp research with generalizable principles, tools, and processes. we identified key types of stakeholders in nlp applications and how they inform the translational process, and presented a checklist of common variables and translational principles to consider in basic, translational, or applied nlp research. the translational framework reflects the central role that integrating basic and applied research has played in the development of the nlp field, and is illustrated by both the broad successes of machine translation, speech processing, and web search, as well as many individual studies in the acl community and beyond. under award number t15 lm007059, and national science foundation grant 1822831.