In modern electronic medical records (EMR) much of the clinically important data -signs and symptoms, symptom severity, disease status, etc. -are not provided in structured data fields, but rather are encoded in clinician generated narrative text. Natural language processing (NLP) provides a means of "unlocking" this important data source for applications in clinical decision support, quality assurance, and public health. This chapter provides an overview of representative NLP systems in biomedicine based on a unified architectural view. A general architecture in an NLP system consists of two main components: background knowledge that includes biomedical knowledge resources and a framework that integrates NLP tools to process text. Systems differ in both components, which we will review briefly. Additionally, challenges facing current research efforts in biomedical NLP include the paucity of large, publicly available annotated corpora, although initiatives that facilitate data sharing, system evaluation, and collaborative work between researchers in clinical NLP are starting to emerge. In modern electronic medical records (EMR) most of the clinically important data -signs and symptoms, symptom severity, disease status, etc. -is not provided in structured data fields, but are rather encoded in clinician-generated narrative text. Natural language processing (NLP) provides a means of "unlocking" this important data source, converting unstructured text to structured, actionable data for use in applications for clinical decision support, quality assurance, and public health surveillance. There are currently many NLP systems that have been 2 successfully applied to biomedical text. It is not our goal to review all of them in this chapter, but rather to provide an overview of how the field evolved from producing monolithic software built on platforms that were available at the time they were developed to contemporary component-based systems built on top of general frameworks. More importantly, the performance of these systems is tightly associated with their "ingredients" (i.e., modules that are used to form its background knowledge), and how these modules are combined on top of the general framework. We highlight certain systems based on their landmark status as well as on the diversity of components and frameworks they are based on. [7] . The review in this chapter differs from previous work in that it emphasizes the historical development of landmark clinical NLP systems, and presents each system in light of a unified system architecture. We consider that each NLP system in biomedicine contains two main components: biomedical background knowledge and a framework that integrates NLP tools. In the rest of this paper, we will first outline our model architecture for NLP systems in biomedicine, before going on to review and summarize representative NLP systems, starting with an early NLP system, LSP-MLP, and closing our discussion with the presentation of a more recent system, cTAKES. Finally, we will discuss challenges as well as trends in the development of current and future biomedical NLP systems. Materials We start from a discussion by Friedman and Elhadad [8] in which NLP and its various components are illustrated, as reproduced in Fig. 1 . NLP aspects can be classified into two parts in the figure: the left part contains trained corpora, domain model, domain knowledge, and linguistic knowledge; the right part contains methods, tools, systems, and applications. From the viewpoint of system architecture, we consider a general architecture in which an NLP system contains two main components: background knowledge, which corresponds to the left part of the figure, and a framework that integrates NLP tools and modules, which corresponds to the right part of the figure. Our view of a general architecture is depicted in Fig. 2 . Below we describe the two main components and their roles in biomedical NLP systems. The rectangles on the left side represent background knowledge, and the components on the right side represent the framework (i.e., algorithms and tools). Background knowledge and framework are the main components of an NLP system. A general architecture of a clinical NLP system contains two main components: background knowledge and framework. Background contains ontologies, a domain model, domain knowledge, and trained corpora. Framework includes a low-level processor for tasks such as tokenization and part-of-speech tagging. A high-level processor is used for tasks such as named entity recognition and relation extraction. Tasks or modules in the framework can be dependent or independent and are organized sequentially or hierarchically.  As mentioned in the introduction, biomedical knowledge is an important component in building clinical NLP systems. Domain knowledge and linguistic knowledge are key elements. Earlier systems such as LSP-MLP built their own medical vocabulary and tools due to the lack of easily available resources at that time. The creation of the Unified Medical Language System (UMLS), which began development in 1986, substantially benefited clinical NLP systems. The UMLS contains three main components: the Metathesarus, the Semantic Network, and the SPECIALIST lexicon. For practical purposes, the UMLS can be considered to be like an ontology of biomedical concepts and their relations. Below we briefly summarize each component of the UMLS. The UMLS's Metathesarus currently contains over 1 million biomedical concepts and 5 millions concept names originating from over 150 controlled vocabularies in the biomedical sciences, such as ICD-10, MeSH, SNOMED CT, and RxNorm. The UMLS Semantic Network provides a consistent categorization of all concepts represented in the UMLS Metathesaurus. It reduces the complexity of the Metathesaurus by grouping concepts according to semantic types. Currently it contains 135 broad categories and 54 relationships among categories. For example, the category "Disease or Syndrome" has a relationship "associated_with" with the category "Finding", and the category "Hormone" has a relationship "Affects" with the category "Disease or Syndrome" in the semantic network. The UMLS SPECIALIST lexicon contains syntactic, morphological, and spelling information for biomedical terms [9] . Currently it contains over 200,000 terms and is used by the UMLS lexical tools for NLP tasks. Background knowledge also includes domain models and trained corpora, which are used to deal with specific domains such as radiology reports, pathology reports, and discharge summaries. Annotated corpora are manually marked-up by human annotators and used to train machine learning linguistic classifiers, as well as to evaluate rule-based systems. There are two main approaches for building NLP tools. The first is rule-based, which mainly uses dictionary look-up and rules. The second uses a machine learning approach that relies on annotated corpora to train learning algorithms. Early systems often used rule-based approach since they were relatively easy to design and implement. Currently, with the development of robust statistical 6 machine learning methods and an increasing number of annotated corpora, many clinical NLP systems have moved away from relying exclusively on rule-based methods, although there is still a high cost in generating new annotated training data, which are still required to account for differences in tasks, types of documents, as well as their provenance. As shown in many clinical NLP challenges, machine learning methods often achieve better results than rule-based methods. However, rule-based methods are somewhat easier to customize and adapt to a new domain. Most contemporary NLP systems are hybrid, i.e., built from a combination of rule-based and machine learning methods [8] . Fig. 2 shows how NLP tools can be integrated into a pipeline built on top of a particular framework. By framework we mean a software platform for the control and management of components such as loading, unloading, and handling components of the pipeline. Components within a framework can be embedded and linked together or used as plug-ins. For NLP systems in biomedicine, the framework can be divided into two levels: low-level and high-level processors. Low-level processors perform foundational tasks in NLP such as sentence boundary detection, section tagging, part-of-speech tagging, and noun phrase chunking. High-level processors perform semantic level processing such as named entity recognition (e.g., diseases/disorders, sign/symptoms, medications), relation extraction, and timeline extraction.  In order to give a unified view of system architecture, we selected representative NLP systems in this review. We selected systems due to their historical importance and influence in the biomedical NLP community. We first chose two widely influential landmark clinical NLP systems: LSP-MLP and MedLEE. LSP-MLP is a pioneering project and has greatly influenced subsequent NLP systems, and MedLEE is a system that is currently widely used in clinical NLP communities. We then selected a specific-purpose system called SymText, which was designed for radiology report processing. SymTex began development in the 1990s and is still in active use today. We also briefly review MetaMap, a widely used tool in the biomedical NLP community. We chose two systems based on GATE and UIMA: HITEx, and cTAKES, respectively. Summaries of characteristic features of the clinical NLP systems reviewed in this chapter are presented in Table 1. 14 http://uima.apache.org/doc-uima-why.  Methods Systems The Linguistic String Project (LSP) 17 was developed in 1965 at New York University by Sager et al. [1, 12] . It is one of the earliest research and development project in computer processing of natural language. The development of LSP was based on the linguistic theory of Zellig Harris: linguistic string theory, transformation analysis, and sublanguage grammar [13] [14] [15] . It mainly focused on medical language processing, including the sublanguage of clinical reporting, radiograph reports, and hospital discharge summaries. The LSP approach used a parsing program to identify the syntactic relations among words in a sentence. The project strongly influenced subsequent clinical NLP projects. The LSP's system was called the Medical Language Processor (MLP). The core component of MLP is a parser. The authors first developed a general NLP parser for the general English language domain, including English grammar and lexicon, and then they extended the system to the sublanguage of biomedicine by adding a medical lexicon and corresponding grammar. Below we summarize the main components of MLP. • Lexicons: MLP developed lexicons for both general English language and medical knowledge. In the lexicon, each word has an associated part-of-speech and grammatical and medical "attributes" called subclasses. The lexicon has 60 possible verb objects and 50 medical sub-classes. It also had lists of pre-defined prepositions, abbreviations, and doses. These attributes are used throughout the processing to guide the parsing and to resolve ambiguities. The parser proceeds from left to right through the sentence, and top to bottom through the BNF definitions. Once the parser associates a terminal symbol of the parse tree, the attributes of the word can be tested by a restriction, for example, the agreement of subject and verb. The following steps are involved in the processing of text: • Selection passes or rejects a parse based on subtrees. • Transformation decomposes sentences into their basic canonical sentences. • Regularization connects basic canonical sentences by conjunctions. • Information format maps the syntactic parse trees into medical information structures. MLP considers 11 information structures related to patients such as patients, family, medication, treatments, lab/test, etc. Finally, the output is written into two formats: tab delimited and XML format. 18 http://mlp-xml.sourceforge.net/ 11 Fig. 3 . Architecture of MedLEE, where the background knowledge contains components for the lexicon, grammar, mappings, and coding tables. The low-level processor is a preprocessor and the high-level processor consists of modules for parsing, error recovery, phrase regularization, and encoding. LSP-MLP was used for processing clinical narratives in English, and it was also extended into other languages such as French, German, and Dutch [1] . It has been used to map clinical text into SNOMED codes [17, 18] . LSP-MLP was designed for information retrieval from clinical text, hence there were no reports evaluating mapping. The performance in information retrieval tasks indicated 92.5% recall and 98.6% precision [18] . With its complete structures, LSP-MLP provided an early successful example for the development of subsequent NLP systems.
Studies on processing clinical texts in other alphabetic languages other than English, such as French, German, and Dutch, have been carried out as well, mapping clinical descriptions to Systematized Nomenclature of Medicine codes REF .
Natural Language Processing in Biomedicine: A Unified System Architecture Overview
Methods in molecular biology
Methods in molecular biology
Computer Science Medicine