Evaluating the W ordnet and C o R o L a-based Word Embedding Vectors for R omanian as Resources in the Task of Microworlds Lexicon Expansion (2019)

Abstract: Within a larger frame of facilitating human-robot interaction, we present here the creation of a core vocabulary to be learned by a robot. It is extracted from two tokenised and lemmatized scenarios pertaining to two imagined microworlds in which the robot is supposed to play an assistive role. We also evaluate two resources for their utility for expanding this vocabulary so as to better cope with the robot’s communication needs. The language under study is Romanian and the resources used are the Romanian wordnet and word embedding vectors extracted from the large representative corpus of contemporary Romanian, CoRoLa. The evaluation is made for two situations: one in which the words are not semantically disambiguated before expanding the lexicon, and another one in which they are disambiguated with senses from the Romanian wordnet. The appropriateness of each resource is discussed.

## introduction the work presented in this paper was carried out in the broader frame of the robin foot_0 project, whose aim is to develop systems and services for using robots in various contexts occasioned by the emerging digital society we live in. focused on different types of robots, from those specialized in assisting elderly people to software robots dedicated to autonomous or semi-autonomous car-driving, robin has a sub-component that deals with the essential function of human-robot language communication in romanian (robin-dialog foot_1 ). the prototype system for verbal inter-action with the robot was restricted to several microworlds (see section 3) and the romanian language resources and tools that are under development at the moment are specifically targeted at describing and serving these microworlds. as a result, the robot should be able to communicate successfully with the human users on topics concerning the specified microworlds and to perform some tasks designated to it, all these activities involving spoken romanian. the robot used in this project is pepper, created by softbank robotics foot_2 and designed to be massproduced and to become an important actor, improving human everyday life by assisting in different activities. therefore, pepper was intended to receive widespread acceptance in society and its shape, size, look and behavior were customized to emulate sociability @xcite . a system able to ensure the dialog between a robot and a human user combines different modules dedicated to automatic speech recognition (asr) (translating the human's vocal message into text), natural language processing (nlp) with its tasks of analysis and synthesis, a dialog management (dm) system and automatic speech generation from text (text-to-speech, tts) @xcite . except for the dm module, all the other components are language dependent, thus they need training on romanian data and use (at run-time) romanian acoustic and language models and a romanian lexicon enhanced with information about stress, syllabification and phonetic transcription. in the context of the robin-dialog project, the acoustic and language models could benefit from all the available bimodal training data (see @xcite for the description of the speech component of the reference corpus of the contemporary romanian language -corola), but tailoring the system to the specific microworlds is necessary for preventing semantic ambiguities and misleading. this can be done by designing a wide enough lexicon to cover various ways of expressing the semantic content possible in the targeted microworlds but limited to the semantic fields of interest (e.g., avoiding out-ofcontext senses for polysemous words). the process of constructing a lexicon -balancing all the needs of the dialog system modules in this specific context -is the focus of this paper. ## related work one of the important steps in human-robot language communication is addressing the problem of creating exhaustive lexicons on different topics, so as to enable the robot to process different ways of expressing the same topic. wordnet is one of the main resources used for the enrichment of different domain specific vocabularies. @xcite presented a semi-automatic approach used to disambiguate the senses present in wordnet in order to enrich the vocabulary for ontology concepts in the domain of amphibians. other important resources used in expanding the lexicons are the word embeddings vectors extracted from different corpora. the main hypothesis on which the current models of semantic word representations are based is that words occurring in similar contexts have similar meanings @xcite . moreover, such representations, most of the times, get closer to human intuitions @xcite . therefore, pretrained word embeddings vectors are used to a wide variety of nlp tasks, including vocabulary expansion. for example, @xcite and @xcite demonstrated that word embeddings are able to capture synonyms and analogies. @xcite used synonym and antonym information extracted from thesauri together with distributional information obtained from large scale unlabelled data in order to train word embeddings to capture antonyms. we are not aware of any work in which the results obtained by using these two resources to be evaluated and this is one of our aims in this paper. in what follows we define a microworld (section 3), describe the extraction of the lexicon from the screenplays based on two microworlds (section 4), we explain how we have expanded this lexicon using the romanian wordnet and corola-based word embeddings (section 5), then we analyze the results obtained and discuss their relevance (section 6) before concluding the paper. ## designing microworlds we define a microworld as an extremely reduced universe that is confined to a well-delimited space, is anchored in time, contains a finite set of objects, is populated by some people and the robot, among which verbal exchanges occur. these exchanges are on topics connected to the microworld. these people know how to collaborate with the robot, while the robot is meant to learn how to collaborate with the people. the learning phase of the robot needs to cover the following topics: the space topology, recognizing the people in the microworld, understanding natural language and reacting to it, which presupposes the ability to formulate an oral response to a human's command or to execute the command within the microworld. for the present paper, we focus on two microworlds imagined for the interaction with the robot, which is attributed an assistive role: a private home and a research laboratory. in the former, the robot will help people to take care of themselves: undergo some measurements of relevance for their condition (e.g., measure the blood glucose), communicate the value of a certain measurement at a specific time, keep track of them during a longer period of time, display their evolution for a specified period of time, remind people what medication to take, when to do it and even where the medication is, etc. in the latter microworld, the robot will be the host for visitors of the laboratory, greeting, welcoming known people, introducing itself to the new visitors. it will also transmit verbal messages from one person to another, provided that they are present in the laboratory. after designing the microworlds, a first preparatory step in the process of teaching the robot to interact with people is the creation of a screenplay for each microworld, with verbal interactions and actions. our focus here is the former, namely the possible dialogues in natural language (romanian) between people and the robot. a set of possible actions the robot could do in each microworld was identified and possible topics for verbal exchanges corresponding to them were created. this is to be understood at a conceptual level, while all the possible ways of expressing these topics are registered as their lexicalized forms. the robot must be able to understand them all, that is why it has to be taught a large vocabulary and numerous syntactic structures. for example, the following ways of asking the robot if it knows the person called george were identified in romanian: " îl s ¸tii pe george?" / "s ¸tii cine e/este george?" / " îl cunos ¸ti pe george?". the human-robot communication is confined to the entities and possible activities in the respective microworlds. the robot will never initiate the dialogue. it is there to help the human by answering a question or carrying out a task. the robot is not to be understood as a repository of world knowledge. it can only answer questions about the entities in the respective microworld (such as "este george în sala 306?" (is george in room 306?), iff george is known to the robot, i.e. the latter was trained to recognize john's face, and the robot knows where room 306 is in the space whose topology it was taught). the human will give the robot as much information as necessary for performing the task, will formulate it concisely, clearly, avoiding obscurity and ambiguity. consequently, the vocabulary used in such dialogues cannot be conceived as specific to a domain. terms specific to a domain, such as "blood pressure", "blood glucose", etc. may occur in the microworld with pepper playing an assistive role in a private home. however, they are terms that have penetrated the general language, are familiar to every speaker, thus their domain specificity being drastically reduced. ## extracting the lexicon based on the screenplays mentioned in section 3, an initial list of lemmas was created. the screenplays serve as a corpus that was processed with the teprolin platform (ion, 2018), using the ttl module to normalize, sentence split, tokenize, pos-tag and lemmatize the data. then, a list of all the unique lemmas in this annotated corpus was extracted, to serve as a starting point for the enhancement process described in the next sections. we treat differently the content words and the function words: we teach the robot the limited list of all the function words in romanian, but we want to control the (virtually unlimited) set of content words the robot has to deal with, to stay in the discourse microworlds. therefore, we set apart a list of 190 content lemmas to work with in our experiments. for the final form of the extended robin lex-icon (containing a comprehensive list of lemmas that need to be represented in our resource), we added: ## expanding the lexicon the lexicon extracted from screenplays, as presented in section 4, was expanded with the purpose of enhancing it with words capable of capturing the lexical and syntactic varieties of the language. in order to extend the lexicon, two resources were used: the romanian wordnet (rown) @xcite and precalculated word embeddings vectors based on the corola corpus @xcite . from the initial lexicon we chose only those lemmas that occur both in rown and in corola. we call this subset l and it contains 178 content lemmas. the difference between the whole set of content lemmas extracted from the screenplays and l is represented by foreign words (e.g. cool), proper nouns (e.g. george), and several content words not implemented in rown (e.g. the adverb românes ¸te "in a romanian way"). we ran another experiment in which we semantically disambiguated the words in l. for six of them, no sense implemented in rown is the one with which the respective words are used in the screenplays. consequently, we obtained a smaller set of 172 disambiguated words, which we call l (l ⊂ l). ## analysis of the words extracted from the two resources the aim of this analysis is to discuss the relevance of the words extracted using the resources described in section 5 above for the task of extending the lexicon coverage for the two screenplays. a word is considered relevant if one can imagine a sentence that could fit the screenplays either for rephrasing an existing sentence or for completing the screenplay with further exchanges. both resources face the challenge of overgeneration: words tend to have more senses in corpora, while in wordnets they occur with many if not all their senses. however, in the screenplays they are mostly used with one of their senses, as the microworld could be thought of as a closed, limited domain. having the expansion of l as a purpose, we discuss the results obtained without semantically disambiguating the words in the initial lexicon and then the results obtained after semantically disambiguating them (l ) . for the sake of clarity, let: l lemma 1 : rownlemma 1,1 , ..., rownlemma 1,i , ... l lemma 2 : rownlemma 2,1 , ..., rownlemma 2,j , ... ... l lemma n : rownlemma n,1 , ..., rownlemma n,k , ... l lemma 1 : welemma 1,11 , ..., welemma 1,10 l lemma 2 : welemma 2,1 , ..., welemma 2,10 , ... l lemma n : welemma n,1 , ..., welemma n,10 -similarly, b be the set of n lemmas from l that were identified in corola together with the set of lemmas extracted from the word-embedding vectors. we applied the following set operations to the two resources in order to find: in what follows we discuss the results of these set operations. ## conclusions the experiments presented here prove the adequacy of rown and corola-based word embeddings for expanding a lexicon so as to ensure a wider lexical and syntactic coverage, meant to ensure the ability of a robot to understand humans in specific microworlds. we worked with a list of 178 nondisambiguated initial lemmas (l) and with a list of 172 disambiguated initial lemmas (l') and we obtained a number of 1,694 unique lemmas (a ∩ b) and, respectively, a number of 1,287 unique lemmas (a ∩ b ), extracted from rown and corola. the amount of validation work is substantially decreased in the disambiguated setting (even with the supplementary disambiguation costs) and, while such a solution is preferable in similar tasks, the loss in interesting, valid extracted words corresponding to different senses of the lemmas has to be taken into account. a solution would be to accept more senses for a specific lemma in the disambiguation phase, when the human validator considers it necessary. words identified as related by the two resources are most probably good candidates, while in the disambiguated setting the probability of their usefulness is close to 100%. as far as the contribution of different relations in wordnet is concerned, the way in which the task was formulated seems to have determined the acceptance of mainly synonyms (even if in a larger sense than that accepted by the wordnet projects), antonyms and words derived from the