L atvian W ord N et (2023)

Abstract: This paper describes the recently developed Latvian WordNet and the main linguistic principles used in its development. The inventory of words and senses is based on the Te̅zaurs.lv online dictionary, restructuring the senses of the most frequently used words based on corpus evidence. The semantic linking methodology adapts Princeton WordNet principles to fit the Latvian language usage and existing linguistic tradition. The semantic links include hyponymy, meronymy, antonymy, similarity, conceptual connection and gradation. We also measure inter-annotator agreement for different types of semantic links. The dataset consists of 7609 words linked in 6515 synsets. 1266 of these words are considered fully completed as they have all the outgoing semantic links annotated, corpus examples assigned for each sense, as well as links to the English Princeton WordNet formed. The data is available to the public on Te̅zaurs.lv as an addition to the general dictionary data, and is also published as a downloadable dataset.

## introduction a wordnet @xcite is a lexico-semantic resource, which links an inventory of senses in synonym sets and other semantic relations, making it a valuable resource for nlp applications that can benefit from a formal structure of language semantics and relationships between specific word meanings. over the last three years we have been developing the first wordnet for latvian, which is finally being formally released. we have chosen to form this resource based on corpus evidence and existing latvian lexical resources, similar to the approach taken by plwordnet @xcite and pol-net @xcite , instead of extending or translating word senses of some existing resource from other languages, such as the english princeton wordnet. the key tasks for forming latvian wordnet were reviewing the sense inventory of the most frequently used latvian words based on corpus evidence, annotating corpus examples to specific word senses, determining the members of synsets (synonym sets) and annotating outgoing semantic links, as well as later forming interlingual links to the english princeton wordnet where applicable. the annotation work was performed with a custom lexicographic tool used for tēzaurs.lv online dictionary, as described in @xcite . the resulting manually curated resource consists of 7609 words linked in 6515 synsets. in addition we have an ongoing manual review of automatically obtained candidate links to princeton wordnet @xcite . the consistency of semantic links was evaluated in an inter-annotator agreement experiment with three annotators on a limited subset of this data. the following section describes the linguistic principles used in the development of latvian wordnet, followed by a discussion of semantic links and an evaluation of their inter-annotator agreement in sections 3 and 4 respectively. after that, the article discusses the process of linking latvian synsets to the princeton wordnet in section 5 and the evaluation of these links in section 6. the concluding part consists of a discussion of the public availability of the resource in section 7 and conclusions and future work in section 8. ## linguistic principles of latvian wordnet the decision was made to develop latvian word-net based on the inventory of latvian word senses instead of adapting semantic hierarchy and relations from another language. it was also decided to build semantic relations between synsets from bottom up, allowing the hierarchy of word senses to grow and develop on its own. all word sense relations are made between synsets. to choose the initial set of senses to work with, a list of 2000 most frequently used words was created based on the balanced corpus of modern latvian @xcite . the list was then revised, leaving only words from four main word classes -nouns (except proper nouns), verbs, adjectives and adverbs. the resulting list of words and their senses are the core of latvian wordnet. the senses of these words were taken from from the explanatory latvian dictionary tezaurs.lv @xcite , after which an additional sense revision was carried out, as the first attempts of semantic linking showed many outdated word senses, as well as inconsistent sense granularity. we chose to look for corpus evidence if the senses are still currently relevant, whether any new senses have appeared or whether specific uses of a word demonstrate the validity of word sense distinction, in a manner similar to how sense distinctions and definitions were done in estonian wordnet @xcite . the sense revision was primarily based on data from the balanced corpus of modern latvian, however, for rare meanings that are only used in colloquial language or other specific language genres we looked for additional corpus evidence from several different corpora from latvian national corpora collection @xcite . the specific principles of distinguishing word senses were developed for the convenience of both annotators and target users of the dictionary @xcite . given that the most frequently used words are also often polysemous, the lexicographic work of processing them proved time-consuming, but also resulted in a thorough, high-quality inventory for the core wordnet. regarding other linguistic principles, the semantic relations of latvian wordnet are usually annotated between synsets of the same word class, with only rare, well-argumented exceptions when such a link is allowed between the senses of different word classes. for example, participles are considered as verb forms but can also be related to synsets of adjectives. additionally, some meanings of definite adjectives can be linked to synsets of nouns. such cases are often characteristic of partial word conversion, when a separate form of a word begins to perform the function of another word class and therefore has a separate meaning while still belonging to the same word entry in a dictionary. word class boundaries are a separate research issue that was not addressed within the scope of our task. ## semantic links in latvian wordnet the most common and better studied semantic relations traditionally included in wordnets of various languages are synonymy, antonymy, hyponymy and meronymy @xcite in addition to these four major relations, latvian wordnet is enriched with gradation relations which are not included in most wordnets, an exception being e.g. plwordnet @xcite . the basic unit of a wordnet is a synset. the opinions of language users about the synonymy of certain senses may differ, so the following set of criteria is used to determine a set of synonyms. firstly, dictionary definitions, namely, semantic features are compared: if most of them match, the senses are considered synonymous. secondly, a substitution criterion is applied: if the words are interchangeable in most contexts, the senses are considered synonymous. it should be noted that a synset may include both neutral and expressive meanings. the fact that the subtler semantic distinctions among the elements of a synset are beyond the scope of description might be considered one of the most serious shortcomings of wordnets @xcite . in latvian wordnet, this is compensated by the representation of data in tezaurs.lv which includes the definitions and stylistic nuances of the specific sense for each word, not a single definition for the whole synset. one of the sources used in annotating synset relations was an existing latvian synonym dictionary @xcite , however, its application was limited as it lists synonyms on a word (not sense) level and includes many words that are related but not strictly synonymous. as the degree of synonymy between senses may be different, latvian wordnet also includes a similarity link for senses which do not fall under the category of a full synonym. firstly, the similarity link is established between senses if the semantic differences are too significant to be considered synonymous, e.g. the synset (diskusija, pārrunas) 'discussion, treatment, discourse' is considered as similar to the synset (apspriede, sanāksme, sēde, sapulce, konference, saruna) 'meeting, group meeting'. secondly, the similarity link is estbalished between words which cannot be substituted for each other in the context due to grammatical peculiarities, e.g. the sense of the word konteksts 'circumstance, setting' is characteristic only to the locative case form and can not be substituted with the locative forms of seemingly synonymous (apstāklis, situācija, stāvoklis) 'situation, state of affairs'. thirdly, verbs with distributional differences are also considered similar, e.g. the transitive spēlēt 'to play' can not be substituted with the intransitive rotal , āties 'to play' despite their semantic closeness. hyponymy is mainly observed in nouns and verbs. the hyponymy between verbs is widespread @xcite , and wordnets tend to include a special subtype of verb hyponymy, namely, troponymy @xcite , in which case the hypernym denotes a more general action or process whereas the hyponyms differ in the manner of how the action or process happens or is carried out. since the concept of troponymy is not known in latvian linguistics so far, the latvian wordnet does not differentiate any subtypes of hyponymy. hyponymic relations are not established between adjectives and adverbs. meronymy is characteristic mainly of nouns especially those having a concrete meaning. in some cases, meronymy borders on hyponymy. this type of semantic relation can be applied mainly to physical objects, as well as to other more abstract ones, such as institutional units, e. g. the meronyms of (uzn , ēmums) 'enterprise' are (filiāle, nodal , a) 'subsidiary company'. antonymy is a relationship between semantic opposites. however, there are several subtypes of opposition and not all of them are considered antonymic. a prototypical group of antonyms consists of words denoting gradable notions @xcite , e.g. for the synset (brangs, dižens, dižs, ievērojams, liels, pamatīgs, prāvs) 'large, big, great' the antonym is (mačs, mazs) 'small, little'. in latvian wordnet, a wide understanding of antonymy is adopted, including other types of opposites as well. they are, firstly, complementaries, e. g. (klātbūtne, klātiene) 'presence' vs. (trūkums) 'absence', secondly, reversives, e.g. (ārā) 'outside' vs. (iekšā) 'inside', thirdly, converses, e.g. (pārdot) 'to sell' vs. (pirkt) 'to buy'. other words that are often contrasted in language use are also considered antonyms, e.g. (praktisks) 'practical' vs. (teorētisks) 'theoretical' and (sekas) 'effect' vs. (cēlonis) 'cause'. words and synsets in one gradation set express different values of the same attribute. the relation of gradation is mainly seen between adjectives, however, it also occasionally occurs in nouns and verbs. in gradation sets, other semantic links may exist as well, e.g., if the gradable values cover the whole scale, antonymic relations are also included. on the other hand, gradation sets of verbs and nouns may include hyponymy, e.g. the word līt 'to rain' has a series of semantically linked verbs denoting raining of various intensity, which can also be considered types of raining and, thus, hyponyms. in the future, it is planned to develop a system of simultaneous marking for gradation and hyponymy where necessary. in addition to the semantic links mentioned above, we also annotate conceptual connections (as "see also"), as a category for words that are semantically related, but not by any of the mentioned semantic relations. ## evaluation of semantic linking in order to assess how consistently the linking principles developed during the project are applied, a three-person inter-annotator agreement (iaa) evaluation was conducted on 15 words (5 nouns, 5 verbs and 5 adjectives) with 85 senses altogether. adverbs were excluded from the experiment as they are poorly represented in the dictionary due to the lexicographic tradition. the words were chosen from the core list of the most frequently used words by selecting words with a moderate number of senses (2-6 superordinate senses and possible subsenses). revision of the sense inventory was not included in the scope of this experiment, so it was ascertained beforehand that the words selected from the dictionary already had comparatively suitable senses for wordnet linking. the experiment was carried out in three stages. we are using fleiss' kappa measurement to judge inter-annotator agreement between multiple annotators. it is interesting to note that most evaluations of wordnet quality in literature only rarely (e.g @xcite ) attempt to make such estimates for the semantic links within the wordnet, the results of the first stage (see r1 part of the results of the second round where annotators got a pre-made list of potential sense are given in r2 part of the level of agreement on adjective links is lower than the agreement on noun and verb links, which indicates that the methodology of marking adjectival links should be further expanded and clarified. when looking at separate link types, a precise agreement also appeared in antonyms and gradation sets, suggesting that when such candidates are presented, the semantic relation is recognized. the results of the third round were also used for making the mcnemar's test, resulting in a pvalue of 2.51 × 10 -23 indicating that consultations made statistically significant changes to the data. in 15% of the discussed cases disagreements still remained even after consultations. from this it can be concluded that the linguists' seminars organized regularly during the project to solve various labeling and annotation dilemmas for specific words are notably beneficial for the creation of a more consistent system. at the same time, it can be seen that even after a unified theoretical base, a developed methodology and regular discussions, there are cases when annotators have differing opinions. some of the cases of disagreement are as follows. firstly, there were varying opinions as to whether the synset (vebkamera) 'webcam' is a hyponym of synset (kamera) 'photografic or video camera', considering that a webcam carries out an additional function of transmitting an image instantly, which a regular camera does not. this raised speculations about whether a webcam is a new type of camera or they both are types of some more general meaning of camera that is not represented in the dictionary. secondly, there were discussions regarding the synsets (inspekcija) 'inspection' and (apskate) 'examination'. opinions differed as to whether they are members of the same synset or whether the 'inspection' includes 'examination', but 'examination' can exist without 'inspection'. both of the given examples show a different understanding of the importance of one seme to distinguish a new meaning or a new semantic relation. the difference of opinion also occurred in situations where the linguist feels a close semantic connection between the senses, but is unable to define it in the currently available relation set, or in moments, when each linguist indicated a different type of relation, although most likely none of the currently available relations fully corresponds to it in its general sense. the synset (tonis) 'tone -a quality of a given color that differs slightly from another color' and synset (krāsa) 'color' serves as an illustrative example for this. one linguist suggested that color consists of various tones and therefore a meronymy/holonymy link could be used; at the same time, another linguist believed that tone is an attribute of color and therefore the appropriate link type is "see also". it should also be noted that none of the linguists suggested a relation to a hierarchy in this case, although that is exactly the type of link used in princeton wordnet between these synsets. in order to obtain a gold standard, it may be necessary to assign an authoritative linguist who will determine the final opinion in such cases. the qualitative analysis of the data gave sufficient grounds for the additional conclusion that link formation can successfully highlight cases, when sense revision is necessary during the process of annotation. there were cases when it was agreed that the reason for disagreement was the vague definition of certain word senses, which, in turn, complicated the possibility if agreement, as there was too much space for interpretation. in short, the experiment has demonstrated the complexity of the given problem, but also provides an opportunity to evaluate the consistency of annotated data. a more detailed analysis of separate semantic link types is planned in future, to further improve our methodology. ## linking latvian wordnet to princeton wordnet as a part of the project, latvian wordnet to princeton wordnet sense mapping is carried out to identify english equivalents for latvian word meanings. currently, only a manual mapping has been implemented for the 2000 most frequently used latvian words. however, the manually generated data are being used to develop and train the algorithm for automated sense linking, which will be carried out for a significantly broader scope of word meanings. the version that the latvian word meanings are presently being mapped to is princeton wordnet 3.0. currently, the project implements wordnet to wordnet interlinking on the level of synsets, as opposed to linking individual word senses as seen, for example, in plwordnet @xcite . such choice of approach is motivated by the need to primarily secure a foundation of optimal interlingual equivalence based on meaning, that would later potentially serve as a basis for more intricate equivalence structures based on stylistic register, dialect, gender and other aspects, which can be linked sense to sense. the project's main theoretical base for creating interlingual links and word sense equivalence is taken from translation theories that offer various perspectives on equivalence (e.g. natural vs. directional) @xcite @xcite , to better understand the poten-tial asymmetry between two or more languages. thus, not only full or direct equivalence is taken into account, but also such types as functional, formal, stylistic, situational and semantic equivalence @xcite . this provides additional context for each decision to minimise inconsistency or artificially rigid or symmetrical interlingual structures. the current process of interlinking is facilitated by automatic suggestions of possible equivalents for each word, based on bilingual dictionaries and machine translation. this feature is integrated in the editing tool, but the linguist may also freely choose and select other english word meanings if the suggestions do not seem to fit the specific meaning in latvian. therefore, both automated and manual methods are already combined in this step of the process. so far, 3139 interlingual links of various types have been created between latvian wordnet and priceton wordnet. however, during the early stages of wordnet to wordnet linking it was concluded that direct links alone cannot fully convey the various cases of interlingual hyponymy, namely, cases when a synset in the source language conveys a broader or narrower scope of meanings than its closest equivalent in the target language. consequently, three types of interlingual links were created, enabling the editors to mark a latvian synset as a full equivalent, as well as being broader or narrower than its english counterpart. if an equivalent synset can be identified, links of narrower or wider meanings are not allowed. if an equivalent synset can not be identified, multiple links of narrower and wider meanings are allowed. full equivalence may be seen in the latvian synset (jautājums, prasījums, vaicājums) and the princeton wordnet synset (question, interrogation, interrogative, interrogative sentence): the meanings describe a sufficiently similar concept with the same level of semantisation. this type of direct link is the most often used -it constitutes 1891 of all interlingual links. but, for example, considering the latvian synset (pārmest), roughly translated as 'reprimand' or 'reprove', it can be concluded that there is no single equivalent for it in the princeton wordnet; instead, several, broader synsets, such as (reproach, upbraid) and (admonish, reprove, reproof) are linked to it through interlingual hyponymy links, each denoting a part of its full, comparatively broader range of meanings. there are currently 545 such links. conversely, there are also certain cases, when princeton wordnet synsets have a broader set of meanings than their latvian counterparts. for example, the synset (sibling), which includes both brothers and sisters, does not have a direct equivalent in latvian foot_0 . therefore two separate hyponymy links need to be made with the more specific (bālelin , š, bālin , š, brālis) 'brother', and (māsa) 'sister' to convey the full meaning of the concept of a sibling. 703 such links have been created in latvian wordnet so far. interlingual hyponymy links not only help in the previously described cases, but also in linking cultural realia to more general meanings in the other language. thus, the data that would otherwise be left unmarked can be involved in forming the interlingual hierarchies between latvian wordnet and princeton wordnet. a notably problematic aspect in the formation of interlingual links are word meaning definitions, which in some cases have become outdated over the course of time or have been left unnecessary broad or narrow. for example, princeton wordnet lists only the general meaning of 'dispute' (disagreement), without separating the meaning of a legal dispute, which exists in latvian wordnet. similar cases have been observed in latvian wordnet, especially in instances when meaning definitions list two aspects separated by a semicolon. such ambiguous cases automatically involve selective use of annotators' personal knowledge or additional research to discern the true level of meaning equivalence; such cases are discussed in greater detail during the weekly project linguist seminars to reach the most objective solution. so far, distinguishing three types of interlingual links has proved useful to bridge the gaps and differences between latvian and english. it is expected, that this approach will also facilitate the future aspirations of incorporating latvian wordnet into open multilingual wordnet @xcite , as a working mechanism will already be established to deal with any potential inconsistencies or language differences. ## evaluating interlingual links to evaluate our process of automatic interlingual link creation, another iaa experiment was car-ried out. in the experiment, annotators evaluated the machine-translated suggestions, taking into account the opinions of three annotators. the proposed links were separated in the following four categories: the iaa experiment was performed using words from common vocabulary with only one sense in latvian (including homonyms), excluding regional words, slang etc. there are up to five possible candidates of english equivalents offered by the system which the annotators can choose from. three linguists annotated 684 instances in total. on 272 corresponding outputs all annotators agreed that the proposed interlingual link should be approved, and in 94 cases all annotators decided that the suggested links definitely do not match the latvian meaning. in 57% cases annotators fully agree, and out of all the automatically provided candidate links 40% are undisputed interlingual matches. in cases when all three annotators chose to select the "wider/narrower meaning" option, several links were proposed. for example, apnikums (a mental state when a person is bored and tired of everything) had four suggested links: (boredom, ennui, tedium), (depression), (fatigue, weariness, tiredness), (tediousness, tedium, tiresomeness). all of suggested links are somehow semantically connected to apnikums, but none of them corresponds completely. from this it can be concluded that the automated system has already noted the absence of complete equivalence in this case. the main reason of annotators' disagreement with automatic suggestions was the occasional inability of mt to correctly interpret the meaning of derived words. for example, apgaismniecība 'enlightenment' (derived from gaisma 'light') had the automatic mt suggestion of "lighting" (the craft of providing artificial light). another reason for disagreement was based on grammatical differences between latvian and english, especially in the use of genitive case. in latvian, a noun in genitive case is often used to name a quality, taking the place of adjective. for example, inflexible genitive noun apl , veida (derived form aplis 'circle') is used only in this (genitive) case and implies quality (circular, round). because it is a noun, mt suggests a link to the noun synset (circle, round). differences in word meaning definitions between wordnets may occur for seemingly similar concepts. in that case answers between annotators may vary. for instance, apašs 'apache' is defined in latvian a "a french gangster", whereas princeton wordnet suggests that it is "a parisian gangster". two annotators considered this as a direct link, one viewed this as wider/narrower case. thus, the annotators had to look at each case individually and decide whether to base their decision on their knowledge of the subject or to stick to the given definitions, leading to the conclusion that the result in this case cannot be completely objective. it also brings to attention the difference which even a minimal manual control can make in automatically created data. disagreement based on annotators' personal opinion frequently appeared on words that name state, condition, sensation and other abstract concepts. these differences are mainly based on annotators' personal understanding of the concept in latvian. personal opinion also may vary on how we perceive translation quality and which semantic differences are essential when choosing between direct, wider/narrower or no link. for example the latvian meaning asthma "a fit of loss of breath, shortness of breath" and the english synset (asthma, asthma attack, bronchial asthma) "respiratory disorder characterized by wheezing; usually of allergic origin" has a different answer from each annotator: 1 "corresponds", 1 "wider/narrower" and 1 "needs more information". the iaa results for interlingual links not only have helped reinforce the importance of multiple link types, but also aided in the future the development of clearer strategies and criteria for annotating ambiguous, more complicated meanings. ## publishing results the main access point for this resource to the general public is through the tēzaurs.lv (https:// tezaurs.lv) online dictionary, which is widely used in latvia. however, for the purposes of the research community we also publish this data in various formats and in multiple repositories. latvian wordnet is developed and maintained in the tezaurs.lv lexicographic platform with a postgresql database custom data structure, which then can be exported in multiple widely recognised data formats. currently we provide an open multilingual wordnet compatible lmf xml foot_1 export for the wordnet data, and a more detailed tei 5 (text encoding initiative) dictionary chapter xml foot_2 which contains both tezaurs.lv dictionary data and latvian wordnet synsets and links. the tei format also contains information about gradation sets, which is not available in lmf due to format restrictions. all the latest version data (including a full database dump) are available on the project homepage foot_3 , where we also provide a list of latvian wordnet core words. the tei xml dataset is also regularly published in the clarin-lv repository foot_4 @xcite . our intent is to publish lmf export both via clarin-lv and omw infrastructure. we do quarterly releases for all our dictionary and wordnet data. ## conclusions and future work to summarize, we are happy to present the first major release of latvian wordnet, providing a manually curated resource of a reasonable size, based on latvian corpus evidence and linguistic tradition that can be a solid basis for future research work. the current latvian wordnet consists of 7609 words linked in 6515 synsets, out of which 1266 synsets are considered completed as they have all the outgoing semantic links annotated, corpus examples assigned for at least one word in the synset, as well as links to the english princeton wordnet formed, and the remainder being less frequently used words that have been joined by outgoing semantic links from the 'core' synsets. 70826 corpus examples were linked to specific word senses and subsenses. this information is available to pub-lic as an integrated part of the tēzaurs.lv online dictionary and has received positive user feedback regarding its usefulness. from the perspective of linguistic principles, we are satisfied with our choice to form a wordnet from scratch. even if bootstrapping from english resources would have taken less effort, our experience with linking to the princeton wordnet has indicated many interlingual differences, which through automatic means would have imposed an artificial, english-derived structure upon this resource. it is interesting to note that sense granularity is still an issue open for debate among the annotators, with no clear consensus despite the fact that it was one of the primary drivers for restructuring the existing sense inventory and a key part of the methodology discussion over the last three years. developing an adequate sense inventory takes a large amount of time and effort compared to forming synonym sets and other semantic links. our approach of word sense selection based on corpus evidence has also resulted in a large quantity of corpus examples aligned to the specific word senses, which forms a useful dataset for training word sense disambiguation systems @xcite . ongoing future work in this direction is annotating a gold standard text -the first two chapters of the little prince -with specific word senses from latvian wordnet. the results of our inter-annotator agreement experiments for semantic links within latvian word-net indicate the difficulty and the subjective nature of semantic linking. a relevant observation is that providing automatically generated candidates improves the linking coverage, as annotators often agree that the link should be made if they are aware of the option, but might not come up with the related word on their own. it seems that when annotators are provided a large quantity of proposed candidates, more links are made but inter-annotator agreement decreases as annotators are forced to make a choice about words they did not consider themselves. it also indicates that annotator discussions improve consistency, so the differences apparently involve also a different understanding of methodology, not a fundamental disagreement about the discussed words. in 57% cases annotators fully agree, and out of all the automatically provided candidate links 40% are undisputed interlingual matches. for latvian-english links we observe 57% exact match iaa between all three annotators, with some disagreement whether a certain sense is the same or broader in one of the languages. we observe less agreement over abstract concepts, as their perception seems to be more subjective, and it is difficult to decide on the most appropriate interlingual link. in general, the generation of automatically provided candidates were very helpful in rapidly creating links, as the 40% of candidates were clearly proper links, but they do need manual review. for further improvement of latvian wordnet the planned future tasks involve adding links for word derivation, extending the automatic link candidate derivation also for intra-language semantic links based on existing word definitions and language models from large corpora, and also continuing the manual review of proposed latvian-english links which could then enable a transfer of semantic relations from princeton wordnet to latvian wordnet. it would be interesting to apply this resource for cross-lingual research on semantic alignment and differences between latvian and lithuanian wordnet @xcite , as well as going beyond current semantic links to word derivation and etymology. continued extension of the manually developed latvian wordnet is also an obvious direction of future work, but is highly contingent on funding opportunities. we are also considering a specific project to integrate idiomatic expressions and other multiword entities in the latvian wordnet.