CLPT : A Universal Annotation Scheme and Toolkit for Clinical Language Processing (2022)

Abstract: With the abundance of natural language processing (NLP) frameworks and toolkits being used in the clinical arena, a new challenge has arisen - how do technologists collaborate across several projects in an easy way? Private sector companies are usually not willing to share their work due to intellectual property rights and profit-bearing decisions. Therefore, the annotation schemes and toolkits that they use are rarely shared with the wider community. We present the clinical language pipeline toolkit (CLPT) and its corresponding annotation scheme called the CLAO (Clinical Language Annotation Object) with the aim of creating a way to share research results and other efforts through a software solution. The CLAO is a unified annotation scheme for clinical technology processing (CTP) projects that forms part of the CLPT and is more reliable than previous standards such as UIMA, BioC, and cTakes for annotation searches, insertions, and deletions. Additionally, it offers a standardized object that can be exchanged through an API that the authors release publicly for CTP project inclusion.

## introduction with the resurgence of deep learning and neural networks, the interest in using a clinical language framework for classifying clinical text in a digital manner has been heightened in recent years. several workshops and shared tasks @xcite @xcite have focused on the state-ofthe-art approaches and the amount of private enterprises offering clinical solutions backed by machine learning technologies has increased drastically @xcite . nonetheless, a recent study @xcite shows that systems like uima @xcite , clamp @xcite , and ctakes @xcite , despite their age and typical technology stack (java), are still a standard for clinical language text classification and there are only a few publicly available clinical language frameworks or standardized annotation schemes that provide easy ways to share results and other pertinent information with organizations, private or public. we propose a modern standardized framework that supports collaboration on clinical language research. here we present the clinical language pipeline toolkit (clpt), a framework developed with python designed with software development principles. the clpt enables researchers and entities to share their project results easily and supports research to be conducted in a fast and reproducible way. the unified annotation scheme for the clpt is called the clinical language annotation object (clao). the clao is more reliable for annotation searches, insertions, and deletions than previous standards (e.g. uima @xcite , ctakes @xcite and @xcite ). 1 additionally, the clao can be easily shared and integrated due to its standardization which makes it accessible through an application programming interface (api). to illustrate the aforementioned concepts which will improve clinical technology processing (ctp) collaboration, we introduce five novel ideas and contributions in this article: in the following sections, we first go through related work on annotation and natural language processing (nlp) tools in section 2. in section 3.1, we then describe in detail the clao scheme. next, in section 3.2, we cover the four clpt modules for creating a typical ctp pipeline. lastly, we conclude with the availability and future work. ## related work several clinical text processing toolkits and annotations schemes have been introduced in the past but none of them provide the same functionality and efficiency as the clao and clpt. some widely used nlp tools for clinical text processing include the clinical text analysis and knowledge extraction system (ctakes) @xcite , bioc @xcite , brat rapid annotation tool (brat) @xcite , general architecture for text engineering (gate) @xcite , metamap @xcite , metamap lite @xcite , clinical language annotation, modeling, and processing (clamp) @xcite and scispacy @xcite . brat @xcite is a web-based annotation tool for defining entities and creating annotations. annotations created by brat are stored in a standoff format. since brat xml output is similar to clpt output, it can be easily adapted to clao by creating an adapted script, unlike outputs from ctakes or uima which are cas files that are serialized using java-style notation. though the clpt implements a similar approach of storing the annotation in a clao object, the clao's annotation scheme supports faster annotation insertion, deletion, and searching by implementing b-tree for indexing (see 3.1 for details). gate @xcite , clamp @xcite provide multiple tools which can be used for language processing tasks, annotating corpora, and performing evaluation. yet, all three of them are either based on java or c++. additionally, gate @xcite and clamp @xcite depend on a framework called the unstructured information management architecture (uima) @xcite . clpt makes similar offerings as the three aforementioned frameworks but it uses python which makes it easier to integrate with other modern deep-learning nlp frameworks such as tensorflow @xcite , medspacy @xcite and pytorch @xcite . the national library of medicine foot_1 presented a framework called metamap @xcite for mapping biomedical text to unified medical language system (umls) concepts. others @xcite ) have found metamap difficult for building machine learning models and hard to predict long entities due to its dictionary lookup method @xcite . previous research @xcite argues that neither metamap nor clamp incorporate deep learning models directly. we believe that the clao and clpt address several downfalls by creating an easy-to-use annotation scheme along with the targeted focus on deep learning. we consider the work on scispacy @xcite similar to ours because it was developed in python and takes into account recent classification techniques in deep learning. however, to our knowledge, scispacy @xcite does not support some of the default features found in the clpt, such as a shareable annotation file that can be serialized to disk and efficient entity lookups as are offered in the clao. 3 methods ## concluding remarks and future work we have introduced a novel and efficient toolkit for creating ctp pipelines with several new contributions. the thought has been to make a centralized format for exchanging information amongst entities, albeit academic or private. this will allow entities to compare and contrast results by comparing claos that adhere to a standardized guideline. we contribute this to the public community as a way to use a more updated framework for modern ctp techniques. it is our thought that the clpt can increase productivity and the exchange of information. the current implementation of the clpt and the clao is in its infancy; the plan is to develop more functionality such as multi-modal inputs, the creation of a knowledge graph, and improved evaluation methods. additionally, we plan on extending the current implementation which performs classification using public machine learning models and heuristics by training models with the clpt. once those models have been trained, we also plan on adding the capability for fine-tuning those models for several clinical tasks able to handle diverse nlp problems like seminal work @xcite has done.