Fast developing of a Natural Language Interface for a P ortuguese W ord N et: Leveraging on Sentence Embeddings (2019)

Abstract: We describe how a natural language interface can be developed for a wordnet with a small set of handcrafted templates, leveraging on sentence embeddings. The proposed approach does not use rules for parsing natural language queries but experiments showed that the embeddings model is tolerant enough for correctly predicting relation types that do not match known patterns exactly. It was tested with OpenWordNet-PT, for which this method may provide an alternative interface, with benefits also on the curation process.

## introduction a natural way of interacting with computational systems or knowledge bases is to use the same language we use for interacting with other humans. however, due to all the complex phenomena of natural language, most systems rely on browsing, keyword-based search interfaces or their combination. this is simpler at the technical level and avoids having to deal with natural language understanding issues. the previous phenomena include ambiguity and language variability and are the reason why matching natural language with formal queries is not a trivial task. to overcome this challenge, we investigate how a model of sentence similarity can be exploited by a natural language interface (nli) for a wordnet. our approach is tested in openwordnet-pt @xcite (own-pt), probably the most active portuguese wordnet @xcite . the development of this system, dubbed nelio, requires only a small set of handcrafted templates for each query to be covered. instantiating those templates with arguments from own-pt results in a large set of sentences, used for training a doc2vec @xcite model. the latter is a variation of the popular word2vec @xcite but, besides learning dense vector representations of words, it learns a representation for documents (sentences, in our case), based on the words used and on a document label. such a model can be used e.g., for predicting the most suitable label for an unseen document. in this work, we rely on the trained doc2vec model for predicting the relation type that a natural language query is asking for. we then use this information for querying own-pt and retrieving suitable answers. this process is fast enough and avoids writing a set of rules for parsing natural language queries. besides providing a more natural way of interacting with own-pt, nelio turns out to be an alternative way of exploring own-pt and reveal flaws that, otherwise, would not be easy to spot. the remainder of this paper briefly overviews own-pt, describes the development of nelio, reports on performed experiments, including a systematic evaluation of the model in this context, and, before concluding, overviews related work. own-pt is freely available in rdf/owl. its data can be retrieved via a sparql endpoint, but it can also be explored through its own web interface foot_0 or through the interface of the open multilingual wordnet @xcite . as previously suggested @xcite , a visual interface helps to discover interesting issues to work on. the research presented here is related to lessons previously learned. ## related work traditional automatic question answering (qa) follows an information retrieval perspective @xcite . queries are typically natural language questions (nlqs) and answers are retrieved from a collection of written documents. but the development of natural language interfaces (nlis) for databases has also been a research topic for a long time @xcite . here, the primary challenge involves translating nlqs to formal queries made to a database. knowledge-based qa systems are a specific case of the previous. several nlis for ontologies -e.g., querix @xcite , panto @xcite , freya @xcite -translate nlqs to sparql with a set of rules on the result of syntactically parsing nlqs, possibly using pwn for synonym expansion. a similar approach @xcite may be based on sparql templates, to be filled with entities and predicates identified in the nlq. other systems rely on domain-independent semantic parsers that learn how to map nlqs to predicates in a large knowledge base, based on question-answer pairs. sempre @xcite maps words to predicates and then combines the predicates to the final logical form. another possibility @xcite is to parse utterances for producing an underspecified logical form, before mapping lexical predicates to the target ontology predicates. the previous systems were assessed while resorting to freebase for answering nlqs. yet, as opposing to freebase or dbpedia, wordnets have a much smaller number of predicates. so, it could be worth exploring how semantic parsers could be adapted for our work. once translated to sparql, generally to a subset of this language, expressiveness is limited. to avoid this, squall @xcite is a controlled natural language for querying and updating rdf datasets. nouns and intransitive verbs are used as classes; relation nouns and transitive verbs as properties; and proper nouns as resources. syntactic and semantic analysis is implemented as a montague grammar, an approach that would work for querying a wordnet, considering the simplicity of its rdf model. on the other hand, squall requires that end-users comply with its controlled syntax, and know the rdf vocabulary. an alternative approach @xcite ) learns low-dimensional embeddings of words and entities, respectively in questions and relation types of freebase. this way, representations of questions and of their corresponding answers are close to each other in the joint embedding space. more recent works @xcite rely on neural networks for translating nlqs to formal queries, thus avoiding domain-specific grammars or rules. ## conclusion we have described how we can leverage on sentence embeddings in the development of a nli for a wordnet. the proposed procedure was applied to own-pt with some success. when trained in a subset with at least 20% of the possible questions, generated with a small set of templates, and tested with the remaining questions, accuracies were higher than 91%, when using the first pre-diction, or 97%, when trying with the first three predictions, in case the previous did not return an answer. this simple experiment confirmed that the proposed approach works well with the doc2vec model for predicting the correct relation type. despite the positive results, this experiment revealed that the system is confused by similar relations, for which the templates share vocabulary, namely the three types of meronymy. the problem can be minimised by considering the top-3 predictions, but others, such as merging the three relations, can be analysed in the future. still, this was a limited experiment, where known limitations of the system had a low impact. this includes questions with vocabulary not covered by the templates, or questions that do not end with the fixed word. the former can be minimised by adding alternative templates. the second is due to a simplification that works for many cases, but fails for some, as in the question 'quais frutas existem?' (what fruits exist?), where the target word is frutas. the previous question has to be made like 'quais os tipos de fruta?'. in the future, we will devise more general ways of extracting the target argument from the question, e.g., having in mind that, among the words/expressions in the question, it should be the least frequent in the dataset; or maybe training an automatic sequence labeller for identifying the target argument in the context of a question. in the latter case, training data should also include templates that do not end with the target argument. other possible directions for future work include: (i) presenting the answers according to the senses they apply to, because context is not enough for disambiguation (currently, there is an option for considering only the first sense); (ii) adding alternative types of question e.g., what is the relation between x and y ? or is y related to x ?, to be answered, respectively, with the name of a relation between x and y in own-pt, or yes/no, depending on the existence of such a relation; (iii) exploring recent models for representing sentence meaning, learned from natural language inference data @xcite , though available data in portuguese @xcite may not be enough. despite its limitations, nelio was already helpful for finding issues in own-pt that need to be fixed. it showed flaws such as inconsistencies in the capitalization (e.g., salmão, pardal), presence of underscores instead of spaces (e.g., animal doméstico), or plural instead of singular form (e.g., epidemias, montanhas), not to mention actual errors (e.g, danc ¸ar entails andar, in english, dancing entails walking). a mid-term goal is to make nelio available from a web interface. in the meantime, its source code is available online, at https://github. com/hgoliv/nli_openwordnet-pt . although, so far, the proposed approach was only used as a nli for a wordnet, in principle, a similar approach could be used in the development of a nli for any knowledge base represented as a-relatedto-b triples.