L ight T ag: Text Annotation Platform (2021)

Abstract: Text annotation tools assume that their user’s goal is to create a labeled corpus. However,users view annotation as a necessary evil on the way to deliver business value through NLP.Thus an annotation tool should optimize for the throughput of the global NLP process, not only the productivity of individual annotators. LightTag is a text annotation tool designed and built on that principle. This paper shares our design rationale, data modeling choices, and user interface decisions then illustrates how those choices serve the full NLP lifecycle.

## introduction building supervised learning models is like operating a manufacturing plant. raw materials(data) need to be refined and processed(annotated) as a precursor to final assembly. some manufacturing plants rely on a supply chain (outsource annotation) while others are vertically integrated (annotate in house). according to the theory of constraints @xcite , a manufacturing process should optimize the global throughput and not any individual sub-process . lighttag is a text annotation tool built on the premise of global optimization by addressing annotator as well as project managers and data scientists who manage the work and enforce production quality. lighttag is a commercial offering with an unlimited free tier for academic use foot_0 . lighttag is unique not only in philosophical outlook but also in it's technical implementation and user interface choices, which we share in this paper. the remainder of this article is structured as follows. section 2 describes prior art. section 3 analyzes requirements and user personas to derive lighttag's goal. section 4 describes novel user facing features. section 5 highlights lighttag's data model and it's implications. we conclude with a number of case studies from industry and academia. the following generation of annotation tools were the first to leverage the browser as a user interface platform and include the brandeis annotation tool @xcite , gate teamware (cunningham and bontcheva, 2011), brat @xcite and @xcite . these also leveraged a client-server architecture to enable multi-user annotation projects and server side automation. the recent trends and ubiquity of nlp, along with improved web development frameworks and simplified delivery mechanisms, have inspired a new generation of tools which cater to data scientists as opposed to academics and emphasize ergonomics. this generation of tools, of which lighttag is a contemporary, include the open source docanno @xcite as well as the commercial prodi.gy @xcite which focuses on annotator productivity via active learning, and tagtog @xcite which optimizes for bio-medical annotation. lighttag's generation of annotation tools offer roughly the same set of capabilities as the previous generation, that of webanno, inception and brat. yet the current generation of tools enjoys a measure of commercial success, despite established and free alternatives. we posit that the cur-rent generation of tools has a stronger focus on user experience, ease of use and integration with the end users goals and systems. thus, despite the similar feature sets between the two generations, we offer the commercial success of lighttag and it's contemporaries as proof of innovation that satisfies previously unmet needs. ## goals and design in designing lighttag, we relied on the manufacturing metaphor mentioned above and identified three user personas and five broad needs that need to be served to optimize the overall "nlp process" as opposed to the local-maxima of individual annotator. we assume that the end user's goal is to solve a business problem with nlp and that text annotation is a bottleneck in that process @xcite . we distinguish between the rate at which labeled data is produced, and the rate at which labeled data propagates through the end user's nlp process and optimize for the latter. ## user interface and user facing features in this section, we present user interface decisions and user-facing features that are, to our knowledge, unique to lighttag. ## backend and data model lighttag's focus on project management and quality assurance requires a rich data management structure. lighttag's backend is a relational database using postgres and makes heavy use of relational design theory @xcite . in this section, we provide an overview of lighttag's data model and elaborate on useful implications.