Corpus-Based Multilingual Event-type Ontology: Annotation Tools and Principles (2023)

Abstract: In the course of building a multilingual Event-type Ontology resource called SynSemClass, it was necessary to provide the maintainers and the annotators with a set of tools to facilitate their job, achieve data format consistency, and in general obtain high-quality data. We have adapted a previously existing tool (Urešová et al., 2018b), developed to assist the work in capturing bilingual synonymy. This tool needed to be both substantially expanded with some new features and fundamentally changed in the context of developing the resource for more languages, which necessarily is to be done in parallel. We are thus presenting here the tool, the new data structure design which had to change at the same time, and the associated workflow.

## introduction this paper describes the tools and the associated annotation process used for building up a corpusbased multilingual event-type ontology, called synsemclass @xcite . since the overall premise is based on working from data ("bottom-up", see esp. @xcite ), the work starts from a parallel corpus (at least between english and the given language being processed/annotated). similarly, the ontology classes are also built from the language side: there is no predefined ontology. the words included in the classes are translational counterparts and as such can be considered synonyms (and we will refer to them in such a way with all the caveats connected with such simplification). the language of original texts is english, the verb synonyms captured in the ontology are, at the moment, only english, czech, german, and spanish. in order to allow for independent annotation of different languages, it was necessary to develop guidelines for an annotation procedure applicable to many languages, design a workflow of the annotation for new languages, and create a configurable annotation tool for adding such new languages. this involved, among other things, designing a general configuration, including e.g., urls for external linking of language resources, a stand-off annotation scheme (each language needs to be annotated separately by, presumably, teams scattered all over the world), and an editor capable of working with just the relevant language(s). here, we describe the principles of restructuring and reformatting the dataset to accommodate multilinguality as well as the description of the capabilities of the extended new tool. ## related work general editors over databases used for editing lexical resources are not suitable due to the amount of customization and overhead needed for the complex structure of synsemclass ontology, as argued already by @xcite . specific tools for building lexicons have been built and/or used since at least the 1980s, as described e.g., in @xcite . lexicon creator is suitable for working with pre-extracted wordlists. lexicon builder is a web service @xcite for compiling custom lexicons from bioportal ontologies. cobalt editor @xcite has been used for historical texts and lexica. dicet @xcite is aimed at lexical graphs (this one is closest to the needs of synsemclass annotation). a broad overview and a brief description of some available editors and environments that can be used for the building of ontologies is provided for example by @xcite . others are, e.g., apollo 1 , ontostudio 2 , protégé 3 , swoop 4 and top-braid composer free edition foot_4 . also relevant for our work are the general linguistic linked open data (llod) editors, but the synsemclass data are still to be (re)defined as llod. 6 synsemclass is linked to a number of existing resources having their own specific editors so we tested also the suitability of their editors for our purposes but we found them not readily adaptable to the synsemclass annotation scheme, since it requires more tasks to be covered than e.g., framenet editor @xcite or propbank frameset editor @xcite can provide. foot_6 ## starting point as described in @xcite and especially in @xcite , the previous version of the synsemclass ontology was aimed at building the core, bilingual event-type ontology. it has been done in a specific situation -when advanced, manually annotated resources existed for both czech and english, which had (indeed) been used to get an efficient workflow and accurate, richly annotated resource. the complexity of the definition of the then-called czengclass resource -with its syntactic-semantic mappings of valency slots to the newly developed semantic roles (associated with every class), linking to 9 external resources, and examples from a parallel corpushas led to the development of the syned annotation tool with its functions tailored to the resources at hand. while the the semantic roles resemble framenet @xcite ) "frame elements", and sometimes borrow their names from there, it should be pointed out that there is one fundamental difference: the semantic roles used in synsem-class aim at being defined across the ontology and not per class (as they would be if we follow the "per frame" approach used in framenet). in addition, the existence of the parallel treebank (the prague czech-english dependency treebank, @xcite ) with its rich annotation scheme, exactly matching the task at hand in that it contained the necessary sense distinctions as recorded in the valency frames of the czech and english valency lexicons, was taken advantage of in the design. the associated workflow was then very ef-ficient, including complete double annotation and adjudication to arrive at high-quality resource. the resources used come from the following datasets: the result of the previous efforts to create czengclass and subsequently extend it as synsem-class is publicly available as a dataset 13 and for browsing as a web interface and service. 14 this latest version contains 883 classes; 63 of them already contain german verbs (while still being added using the original workflow and editor). of the original 67,401 class member candidates, approx. 8,000 class members remained in this synsemclass version, i.e., approx. 3,595 english, 464 german, and 4,110 czech class members. 15 adding german (and then spanish) brought many new issues that needed to be addressed, and eventually it led to the development of the new data structure, editor and workflow that we are presenting in this paper. we summarize briefly the design of the lexicon (fig. 1 ) and the main points of its composition and structure. 16 each class in synsemclass is assigned a common set of semantic roles, called a "roleset", figure 1 : example entry in synsemclass ("complain-stěžovat si") indicating the prototypical meaning of the given class. a roleset contains the core "situational participants" labelled as "semantic roles" common for all the multilingual class members (the individual multilingual verb senses) in one class. each class in synsemclass is viewed as a substitute for an ontology unit, similar to the treatment of wordnet synsets. class members are (for the time being) verbs (in different languages). it is essential that these verbs are sense-distinguished; more precisely, each "class member" is meant to be a verb sense. these senses must be predefined (or defined on-thefly and assigned a particular sense id). for czech end english, they have been taken from the existing valency lexicons pdt-vallex and engvallex, where the individual valency frames are already sense-disambiguated (and ids assigned to them). class members are linked to both internal and external resources. the czech class members are linked to the following czech valency lexicons: pdt-vallex, czengvallex (both internal, linked by means of their id's), and vallex (external). 17 the english class members are linked to the internal lexicons engvallex and czengvallex and to external sources, i.e., framenet, verbnet, prop-bank, ontonotes groups and wordnet. ## towards multilingual lexicon design the work on adding german and now spanish @xcite made it clear that a refactorization of both the data structure of synsemclass and corresponding changes in the syned and the associated workflow are necessary in order to be able to concurrently work on more languages. we are testing the new, more "universal" approach on spanish. in the future, we would like to engage external teams (anyone who wants to contribute their language). in such a case, the design, datasets, suggested workflow and tools provided must be easy to use and understandable. however, this has not changed the original idea nor the overall design of synsemclass -it is still an event-type ontology with classes as the main units representing event-type concepts, associated with a fixed set of semantic roles, and the class members are word senses representing the expression of that eventtype concept in a particular language. this schema will not change even if other parts of speech (nouns, adjectives, etc.) are added. but still, some assumptions providing a basis for the creation of the original structure and tools have to be scaled down. for example, a parallel (deeply and richly annotated) corpus exists only for a handful of language pairs and only some of them have associated valency (or predicate-argument, or wordsense-disambiguated, or similar) lexicons linked to the corpus. word (or even just verb) senses are sometimes available (e.g., in multilingual word-nets), but generally not in referred to as such from an annotated corpus. each language has a different set of available semantic lexicons to which the class members in that particular language can be linked. ## conclusions and future work we have demonstrated a gradual approach to adapting a data specification and an annotation tool and the associated workflow to a multi-language, or in other words, partly language independent model, allowing concurrent annotation by independent teams working on the individual languages. each new version of all the components is based on a practical experience with the previous version. while inspired very much by the ud approach to adding, maintaining, validating and publishing new datasets and languages. however, work on a multilingual ontology has one substantial difference: the amount of data that are language independent, but which are heavily being linked to from the language-dependent parts is much larger and will be changing often. the experimental version of syned described herein is now being used for adding spanish @xcite . 27 both the spanish data and the editor will be publicly released as version 5.0 of synsemclass. part of future work -once the components are in place -will be to open the development to the community, interested in similar resources, and also to develop tools that would allow possible (semi-)automatic "conversions" of those resources to the synsemclass set. the web application that shows the then-current version of synsemclass is in place. 28 it is automatically generated from the dataset. however, at the moment it lacks more advanced search features that would serve possible more complex research tasks on this resource -another future work item.