BENTO : A Visual Platform for Building Clinical NLP Pipelines Based on C oda L ab (2020)

Abstract: CodaLab is an open-source web-based platform for collaborative computational research. Although CodaLab has gained popularity in the research community, its interface has limited support for creating reusable tools that can be easily applied to new datasets and composed into pipelines. In clinical domain, natural language processing (NLP) on medical notes generally involves multiple steps, like tokenization, named entity recognition, etc. Since these steps require different tools which are usually scattered in different publications, it is not easy for researchers to use them to process their own datasets. In this paper, we present BENTO, a workflow management platform with a graphic user interface (GUI) that is built on top of CodaLab, to facilitate the process of building clinical NLP pipelines. BENTO comes with a number of clinical NLP tools that have been pre-trained using medical notes and expert annotations and can be readily used for various clinical NLP tasks. It also allows researchers and developers to create their custom tools (e.g., pre-trained NLP models) and use them in a controlled and reproducible way. In addition, the GUI interface enables researchers with limited computer background to compose tools into NLP pipelines and then apply the pipelines on their own datasets in a “what you see is what you get” (WYSIWYG) way. Although BENTO is designed for clinical NLP applications, the underlying architecture is flexible to be tailored to any other domains.

## introduction with the machine learning research going deep, computational models are becoming increasingly large with intensive hyper-parameters tuning, making the research difficult to reproduce. to tackle 1 codalab.org figure 1 : the architecture of bento. the bento back end stores the description files of various tools (e.g., pre-trained nlp models), processes static contents of the application and handles compilation of the user-defined pipelines. the codalab back end stores the datasets (bundles) and executes computational jobs. the two back end servers are brought behind a single domain name using a reverse proxy server. building pipelines is essential for the research of certain domains. take the medical informatics research as an example, a complete nlp analysis on medical notes often involves multiple steps like tokenization, de-identification @xcite , entity recognition @xcite @xcite and normalization @xcite @xcite , relation extraction @xcite , etc. since these steps require different tools and these tools are usually scattered in different publications, it is far from trivial to leverage these tools on new datasets even though the authors have released the source code. therefore, we developed a user-friendly workflow management platform, biomedical nlp toolkits (bento), to facilitate the process of building and applying of clinical nlp pipelines. the architecture of bento is illustrated in figure 1 . bento has three main components. the web interface is supported by two back ends brought together by a reverse-proxy server. the codalab back end stores the datasets and executes computational jobs. the bento back end serves tool information and transforms user-defined pipelines to codalab commands. the advantages of such architecture are two-fold. first, it is flexible to use codalab as the back end for adding custom tools (e.g., pre-trained nlp models) and processing data in a controlled and reproducible way. all the tools are containerized with docker 2 , which makes the platform to keep a unified interface to manage the models and not need to maintain different operating environment for different models. second, the web interface makes it easier for users to construct nlp pipelines through editing flowcharts and then apply the pipelines to their data. the web-based architecture also makes the platform widely accessible without complex installation and configuration. in this paper, we also show the examples of using bento to integrate several clinical nlp applications such as hypoglycemia detection @xcite and adverse drug event extraction @xcite , and build pipelines based on these tools. bento helps build nlp pipelines, which is a promising system to accelerate the medical informatics research. in the medical domain, nile @xcite ) is a java package which includes rule based nlp methods for information extraction from medical notes. apache ctakes (apache ctakes, 2018) and clamp @xcite are two clinical nlp systems with pipeline-based architecture in the uima framework. both systems have a graphical user interface, allowing users to build pipelines from build-in uima components. however, the uima framework has a steep learning curve. it is also not widely used in the machine-learning-based nlp research. furthermore, most nlp applications are often released as command line programs. therefore, it is hard to extend applications that use the uima framework with new models. in contrast, tools on our bento platform are based on command line programs and users can easily define their own tools with little restriction. ## system description bento mainly comprises three parts: a front-end web application, a bento back end server and a codalab back end. as shown in figure 1 , bento has a web-based user interface, from which users can upload data, edit tools, submit jobs and perform various other operations. the bento back end is a web server that is mainly used for storing the tools, including the user-defined ones, so they can be accessed in different sessions. the codalab back end is used for execution of each computational job. when a tool is being executed, bento will generate a series of codalab commands based on the tool information and the input bundles. the outputs of the tool are the run bundles generated from those commands which can be passed on to the down-stream tools and inspected by the users on the codalab interface. ## tools integrated in bento in this section, we list the tools that have already been integrated to our platform, including: we provide examples and instructions to use these tools on the demo page of our platform. for convenience, these tools all take plain text files as inputs and have the pre-processing and tokenization components built-in. in the future, we will integrate stand-alone components dedicated for preprocessing and tokenization to bento which can be shared by different application tools. we also plan to incorporate more nlp tools developed by our group @xcite @xcite . ## conclusion in this paper, we have described the design of the workflow management platform bento. to the best of our knowledge, bento represents the first web-based workflow management platform for nlp research. using bento, researchers can make use of existing tools or define their own tools. computational pipelines can be configured through a web-based user-interface and then automatically executed on codalab. bento includes a number of clinical nlp tools to facilitate the process of ehr notes. a demo of our platform is available at bio-nlp.org/bentodemo/.