## 3 Results
### Implementation Process and Results  
Using Spring Boot and Vaadin 14 allowed for a fast implementation of a first prototype. The development of the prototype focused on the most important features of the application. At the end of the third development cycle, all \"must\" requirements were implemented. The application was then deployed to a virtual machine hosted by University Hospital Bonn equipped with four vCPUs and 16 GB RAM, of which CoAT required less than 1 GB. Based on the technologies chosen for implementation, we expect CoAT to run with sufficient performance and scalability on any typical hospital IT infrastructure. Users with the annotator role can annotate data tagged with their location using the annotation interface (Figure 2). In the top half of the user interface (A), the actual annotation is performed. At the very top, the data to be annotated is displayed (A1). Each uploaded data item is described by an object identifier, an internal identifier--which typically identifies both the source system and data item--and a description. In the three boxes below (A2), codes from the terminologies SNOMED CT, LOINC, and ATC can be added as annotations. Further down is a line of buttons (A3) to use the cross-mapping feature and to save the annotations. Figure 1: Data annotation workflow. Administrator, annotator, and verifier interact through the workflow management features of CoAT to produce quality assured annotations. Administrators start the annotation process by importing medical metadata into CoAT. Annotators can then search for fitting codes in standardized terminologies or use collaborative features to annotate the data. After that verifiers can either reject annotations and send them back to the annotation step for corrections or accept them. Accepted annotations can be exported by administrators for further usage, ending the annotation process. The bottom half of the annotation, UI (B), is used to search the terminologies for fitting codes and is further divided into a left and a right part. On the left (B1), terminologies can be searched by typing in search terms or in case of hierarchical terminologies like SNOMED CT, by navigating through the tree structure defined by the hierarchy visualized using the Vaadin Tree Grid widget. Users can click on any search result to obtain detailed information from the terminology, which appears on the right (B2). On the very bottom is a button to directly add the selected search result as an annotation. Figure 3 shows the collaborative search feature located in the bottom part of the annotation page. On the left, we can see that searching for \"aspirin\" in SNOMED CT yielded the result \"Aspirin therapy (procedure)\", which has been used to annotate some data once before (Figure 3, B3) on the Bonn site as well as in various contexts in the MDM portal. After clicking on the search result, additional information about the search result from the source terminology as well as information about its prior usage in annotation is shown. The table on the very right (Figure 3, B4) shows which data items in which location were annotated using the selected search result. In this case, the search result was used to annotate the data item \"Medication Aspirin 500 mg Tablet\" at the University Hospital Bonn. Figure 3: Detailed view of collaborative search feature. Searching terminologies yields terminology concepts with the number of times they were used as annotations (**B3**) as results. Selecting a single search result shows what local codes were annotated using this terminology concept including MDM portal integration (**B4**). Figure 2: Annotation page. The top part of the UI (**A**) shows a local code (**A1**) that can be annotated by adding terminology codes to their respective boxes (**A2**) and using the cross-mapping feature (**A3**). The bottom part (**B**) displays the terminology search (**B1**) and detailed information on specific search results (**B2**).