A Data-Centric Framework for Composable NLP Workflows (2020)

Abstract: Empirical natural language processing (NLP) systems in application domains (e.g., healthcare, finance, education) involve interoperation among multiple components, ranging from data ingestion, human annotation, to text retrieval, analysis, generation, and visualization. We establish a unified open-source framework to support fast development of such sophisticated NLP workflows in a composable manner. The framework introduces a uniform data representation to encode heterogeneous results by a wide range of NLP tasks. It offers a large repository of processors for NLP tasks, visualization, and annotation, which can be easily assembled with full interoperability under the unified representation. The highly extensible framework allows plugging in custom processors from external off-the-shelf NLP and deep learning libraries. The whole framework is delivered through two modularized yet integratable open-source projects, namely Forte (for workflow infrastructure and NLP function processors) and Stave (for user interaction, visualization, and annotation).

## introduction natural language processing (nlp) techniques are playing an increasingly central role in industrial applications. a real-world nlp system involves a wide range of nlp tasks that interoperate with each other and interact with users to accomplish complex workflows. for example, in an assistive medical system for diagnosis (figure 4 ), diverse text analysis tasks (e.g., named entity recognition, relation extraction, entity coreference) are performed to extract key information (e.g., symptoms, treatment history) from clinical notes and link to knowledge bases; a medical practitioner could select any extracted entity to retrieve similar past cases for reference; text generation techniques are used to produce summaries from diverse sources. to develop domain-specific nlp systems fast, it is highly desirable to have a unified open-source framework that supports: (1) seamless integration and interoperation across nlp functions ranging from text analysis to retrieval to generation; (2) rich user interaction for data visualization and annotation; (3) extensible plug-ins for customized components; and (4) highly reusable components. a wealth of nlp toolkits exist ( §4), such as spacy @xcite , dkpro @xcite , corenlp @xcite , for pipelining multiple nlp functions; brat @xcite and yedda @xcite for annotating certain types of data. none of them have addressed all the desiderata uniformly. combining them for a complete workflow requires non-trivial effort and expertise (e.g., ad-hoc gluing code), posing challenges for maintenance and upgrading. we introduce a new unified framework to support complex nlp workflows that involve text data ingestion, analysis, retrieval, generation, visualization, and annotation. the framework provides an infrastructure to simply plug in arbitrary nlp functions and offers pre-built and reusable components to build desired workflows. importantly, the framework is designed to be extensible, allowing users to write custom components (e.g., specialized annotation interfaces) or wrap other existing libraries (e.g., @xcite easily. the framework's design is founded on a datacentric perspective. we design a universal text data representation that can encode diverse input/output formats of various nlp tasks uniformly. each component ("processor") in the workflow fetches relevant parts of data as inputs, and passes its results to subsequent processors by adding the results to workflow example tensorflow/pytorch/texar … spacy/allennlp/stanza/nltk ... infrastructure universal data representation data representation markup management data apis markup type enforcement auto batching io and serialization training control vocabulary handling workflow control … … … ner tokenizer sentence segmenter pos tagger dependency parser elastic search indexer/retriever chunker sentiment analyzer semantic role labeler machine translation gpt2 text generator bert reranker bert indexer/retriever table-to-text generator relation extractor entity coreferencer text embedder entity linker multi document viewer … plugin system single document viewer x add edit delete relation extractor entity linker text embedder ner data flows in a unified format retrieval x analysis text generation x q a b c d ? x knowledge graph viewer processors figure 1: stack of the data-centric framework for nlp workflows, including workflow infrastructure, and processors for nlp tasks and interactions (e.g., visualization, annotation). different processors are composed together with the infrastructure apis to form an arbitrary complex workflow. the example workflow transforms an unstructured text corpus into a knowledge graph through a series of nlp functions. the data flow (figure 1 ). in this way, different processors are properly decoupled, and each is implemented with a uniform interface without the need of accommodating other processors. visualization and annotation are also abstracted as standalone components based on the data representation. we demonstrate two case studies on using the framework to build a sophisticated assistive medical workflow and a neural-symbolic hybrid chatbot. ## data-centric nlp framework figure 1 shows the stack of the framework, consisting of several major parts: (1) we first introduce the underlying infrastructure ( §2.1), in particular, a universal representation scheme for heterogeneous nlp data. the highly-organized unified representation plays a key role in supporting composable nlp workflows, which differentiates our framework from prominent toolkits such as corenlp @xcite , spacy @xcite , and allennlp @xcite . we then introduce a range of functionalities that enable the convenient use of the symbolic data/features in neural modeling, which are not available in traditional nlp workflow toolkits such as dkpro @xcite . ## related work the framework shares some characteristics with uima @xcite backed systems, such as dkpro @xcite , cleartk @xcite and ctakes @xcite . there are nlp toolboxes like nltk @xcite and allennlp @xcite , glu-onnlp @xcite , nlp pipelines like stanford corenlp @xcite , spacy (honnibal and montani, 2017), and illinois curator @xcite . as in §2.2, our system develops a convenient scaffold and provides a rich set of utilities to reconcile the benefits of symbolic data system, neural modeling, and human interaction, making it suit for building complex workflows. compared to open-source text annotation toolkits, such as protégé knowtator @xcite , brat @xcite , anafora @xcite , gate @xcite , and yedda @xcite , our system provides a more flexible experience with customizable plug-ins, extendable data types, and full-fledged nlp support. the prodigy tool by spacy is not open-source and supports only pre-defined annotation tasks like ner. ## conclusions and future work we present a data-centric framework for building complex nlp workflows with heterogeneous mod-ules. we will continue to improve the framework on other advanced functionalities, such as multitask learning, joint inference, data augmentation, and provide a broader arsenal of processors to help build better nlp solutions and other data science workflows. we also plan to further facilitate workflow development by providing more flexible and robust data management processors.