UMR -Writer 2.0: Incorporating a New Keyboard Interface and Workflow into UMR -Writer (2023)

Abstract: UMR-Writer is a web-based tool for annotating semantic graphs with the Uniform Meaning Representation (UMR) scheme. UMR is a graph-based semantic representation that can be applied cross-linguistically for deep semantic analysis of texts. In this work, we implemented a new keyboard interface in UMR-Writer 2.0, which is a powerful addition to the original mouse interface, supporting faster annotation for more experienced annotators. The new interface also addresses issues with the original mouse interface. Additionally, we demonstrate an efficient workflow for annotation project management in UMR-Writer 2.0, which has been applied to many projects.

## introduction umr-writer @xcite is a web-based application used for annotating uniform meaning representation (umr). umr is a graph-based, cross-linguistically applicable semantic representation designed to support interpretable natural language applications that require deep semantic analysis @xcite . it captures the meaning of natural language sentences and documents in a structured, human-and machine-readable format (figure a1 shows a complete umr graph). umr is an extension of abstract meaning representation (amr, @xcite and enriches the amr semantic scheme to cover additional linguistic categories such as aspect @xcite , and scope @xcite in sentencelevel annotation. umr also supports documentlevel annotation for temporal relations @xcite , modality @xcite , and coreference @xcite . moreover, umr is also a universal multi-language semantic scheme that can be used to annotate low-resource languages such as arapaho, kukama, and secoya, etc @xcite . as graphs, umr can be serialized into triples (parent concept node, relation, child concept node). parent and child nodes can be abstract concepts, lexicalized concepts, or attribute values. relations can be roles or other types of semantic relations. umr-writer originally has a click-based interface for annotators to construct umr triples at the sentence level. an example is shown in figure 1 , which requires five steps for annotating the concept "free" in the sentence "edmund pope tasted freedom today for the first time in more than eight months". annotators could 1) select a parent concept node "taste" by clicking the node; 2) select the child concept node "free" by selecting a span from the raw text; 3) look up senses by clicking the "lexicalized concept" box. then, by hovering the cursor, annotators could 4) view the frame information and choose the correct concept sense, and finally 5) choose the correct relation (here, "arg1", proto-patient) from the corresponding drop-down menus. this approach creates several issues during annotation. firstly, many annotators have extensive experience in annotating amr with the amr editor @xcite . it uses a keyboard interface to annotate amr graphs by entering editing commands. therefore, annotators who are accustomed to the amr editor may prefer to keep the keyboard interface instead of learning how to annotate in a click-based interface from scratch. secondly, the multiple complicated drop-down menus in the click-based interface often confuse and overwhelm annotators. annotators need to move the mouse back and forth between multiple drop-down menus and the sentence itself in order to add just one node to the graph, in addition to simultaneously paying attention to the sentence-level umr graph, as shown in figure 1 . this impacts the annotation efficiency and quality. finally, some concepts are non-sequential in some languages, and it is tricky to select multiple non-sequential spans with a mouse. to address these issues, we implemented a keyboard interface in umr-writer 2.0 ( §3). the new interface was developed using flask, javascript/jquery, html/css, and postgresql. it is specifically designed for sentence-level annotation and coexists with the original click-based interface, allowing annotators to choose their preferred approach. the interface for document-level annotation remains unchanged. besides the annotation procedure, in terms of the workflow set-up, users also reported that managing annotation data for multiple corpora becomes difficult with the increasing size of the annotation. thus this paper also introduces an efficient workflow for project management ( §4), and other features for umr-writer 2.0 foot_0 . ## related tools amr editor is an easily accessible web-based annotation tool for amr with comprehensive functionalities @xcite . it is a commandbased tool where annotators can enter short editing commands to annotate amr graphs. besides the basic function of building amr graphs, it offers many useful features such as copy and paste of partial graphs, searching, and administrative support. many features of the keyboard interface in this paper are inspired by the amr editor. however, the amr editor does not support document-level annotation and languages other than english. there are other annotation tools available, such as anafora @xcite and brat @xcite . anafora is a webbased text annotation tool that is lightweight, flexible, easy to use, and capable of annotating with a variety of schemas. brat offers visualization for annotators to intuitively figure out the relations across text annotations. however, neither of these annotation tools is compatible with the umr scheme and annotation requirements because they cannot annotate the concepts in the form of word lemmas, concatenated words, or abstract concepts that do not correspond to any specific word tokens in the source text. like anafora and brat, umr-writer can be modified to extend its usage to other graph-based formalisms besides umr in theory, making it a versatile annotation tool. these modifications include customizing the relations and concept types to meet the requirements of various annotation tasks. ## the keyboard interface of umr-writer 2.0 we first overview the layout of the new keyboard interface, then introduce the annotation methods and related functionalities. ## annotation as projects the annotation process can become messy and disorganized if an annotator works on multiple corpora. to address this issue, we have introduced the "project" concept in umr-writer 2.0. each project folder contains two sub-folders for storing completed annotations submitted by annotators: the first sub-folder is called "quality control" (qc), which stores the final version of each annotation file, and the second sub-folder is called "double annotated" (da), which preserves multiple copies of the same file annotated by different annotators. to manage the annotation projects effectively, we have created an administrative permission hierarchy. the hierarchy of administrative permissions and the descriptions for each are shown in figure 3. the permission level decreases sequentially from left to right. the permission on the left side by default has all permissions on the right side. thus, users with the "annotate" permission have the "view" permission. similarly, users with the "edit" permission also have both the "view" and the "annotate" permissions, and so on. same-level permission can be issued to multiple users except for "admin", which belongs to the owner of the project only. moreover, we have established an efficient workflow for each project: members with the "annotate" permission or higher can check out files and independently edit annotations without impacting other members in the project who have checked out the same files. furthermore, umr-writer 2.0 provides a search functionality that allows users to search for annotations based on strings, concepts, or triples. users can also specify whose annotations they want to query by entering user names. each user has the option to choose the visibility of their project using a slider bar. the annotations in the project are publicly searchable by any user if the slider bar is checked. annotation managers can leverage the search functionality to check annotations for beginner annotators during the training process. ## conclusion umr-writer is a significant annotation tool for uniform meaning representation. this paper introduces a new keyboard interface that constructs umr graphs by entering index-based commands. this increases efficiency and guarantees higher accuracy of annotations. the new interface also solves the existing issues within the original clickbased interface for the tool such as non-sequential tokens as concepts and variable inconsistencies across languages. the keyboard interface is especially welcomed by annotators who annotated with the amr editor. moreover, we introduce an annotation project workflow that can manage annotation projects efficiently.