Sõnajaht: Definition Embeddings and Semantic Search for Reverse Dictionary Creation (2024)

Abstract: We present an information retrieval based reverse dictionary system using modern pre-trained language models and approximate nearest neighbors search algorithms. The proposed approach is applied to an existing Estonian language lexicon resource, Sõnaveeb (word web), with the purpose of enhancing and enriching it by introducing cross-lingual reverse dictionary functionality powered by semantic search. The performance of the system is evaluated using both an existing labeled English dataset of words and definitions that is extended to contain also Estonian and Russian translations, and a novel unlabeled evaluation approach that extracts the evaluation data from the lexicon resource itself using synonymy relations. Evaluation results indicate that the information retrieval based semantic search approach without any model training is feasible, producing median rank of 1 in the monolingual setting and median rank of 2 in the cross-lingual setting using the unlabeled evaluation approach, with models trained for cross-lingual retrieval and including Estonian in their training data showing superior performance in our particular task.

## introduction a reverse dictionary (see examples in early approaches to building reverse dictionary systems were based on information retrieval (ir) techniques reliant on exact term matching: both user inputs and candidate collections were represented using sets of keywords or sparse term-based vector representations @xcite . such representations are very limited in their ability to represent the compositional meaning of sentences due to texts being represented as simple collections of discrete terms. more recent works on reverse dictionary focused on training models to reconstruct word embeddings @xcite or on fine-tuning pretrained transformers @xcite @xcite . the main limitation of these approaches is that they require labeled data to train predictive models and, as such, are not easily generalizable to new settings or languages. while the earlier ir-based approaches were limited by the expressive power of sparse text vectors and term-based text representations, dense sentence representations of modern pre-trained transformerbased language models make these problems obsolete and provide suitable representations for implementing semantic search functionality @xcite . when applied to lexicographical data, semantic search may be leveraged to create a reverse dictionary system. word definitions encoded by a pre-trained language model represent the search index, which is then queried with the encoded representation of the user's input (definition or description of a concept). in this work, we develop an ir-based reverse query: "tugev emotsionaalne füüsiline või vaimne külgetõmme kellegi suhtes" translation: "a strong emotional feeling of physical or mental attraction towards somebody" rank word translation 1 armastama to love 3 armastaja lover 4 armastus love 6 armupalang love fervour 9 armunud in love query: "группа людей, таких как матьотец и дети, которые все родственники" translation: "a group of people like a mother, father and children who are all related" rank word translation 1 abielu marriage 2 asurkond population 27 kollektiiv group 46 noorpere young family 51 pere family query: "when you tell other people that something is very good and the right choice" rank word translation 1 austama to respect 13 jaatama to agree 44 meelitama to convince 76 soovitama to recommend 98 ülistama to praise the words used for querying may also be in an inflected form. however, the current system does not support approximate search-the user has to spell the words precisely. search over definitions is not currently supported in any capacity. the system we propose is based on word definitions: every word in the sõnaveeb has at least one distinct sense, and each sense has at least one definition. any given definition in the sõnaveeb can be linked back to its corresponding word and to a specific sense of that word. we encode the definitions using a pre-trained language model and then store these definitions in a vector database. then, when a user inputs their description of a desired meaning, it is encoded with the same language model to be used as a query. the approximate nearest neighbor search is then used to query the vector database to return definitions linked to corresponding words. although all components of this system-dense sentence representations, similaritybased search, and approximate nearest neighborsare well-known, their combination to build a reverse dictionary functionality has, according to our knowledge, not been studied previously. for evaluating our reverse dictionary system, we introduce a novel unlabeled evaluation approach that relies on the word relation structure present in the sõnaveeb dictionary itself. additionally, we utilize and extend the labeled english dataset of words and definitions introduced by @xcite by translating the target words to estonian, as well as introducing definitions in estonian and russian in addition to english. to summarize, our contributions in this work are as follows: we make the code and data available on github foot_2 and huggingface hub foot_3 , respectively. ## related work the approaches used to address the reverse dictionary problem can be divided into two-predictionbased methods and information retrieval (ir) based methods. both approaches assume a dictionary dataset but use it differently-while predictionbased methods use the data for training a predictive model, ir-based methods require access to a dictionary during inference. ## methodology our approach to the reverse dictionary problem is, similarly to @xcite and @xcite , based on information retrieval techniques. we assume the presence of a lexicon of words with their definitions. the system treats individual definitions as candidates in the search database, compares the user input to all candidates, and outputs the most relevant results based on cosine similarity. when a search database has a significant number of entries, a brute-force all-to-all comparison becomes computationally infeasible. thus, we adopt the approximate nearest neighbors algorithm that reduces the computational complexity of vector search. we chose the qdrant vector database 5 that implements the hierarchical navigable small world (hnsw) approximate nearest neighbors algorithm @xcite . in our experiments, we estimate the nearest neighbors search to be approximately 60 times faster than the brute-force search. the vectors are stored in the database together with some additional metadata, such as the language of the definition and both word and definition identifiers for the ease of later filtering. to store supplementary information, such as definitions themselves and synonymy relations, we opted to use sqlite for our simple implementation. ## results we expect that the output of a good reverse dictionary system would contain as many relevant items as possible and they would be as close to the top as possible. meanwhile, the user's interest seems unlikely to be strictly limited to a single item. therefore, although we show all measures described in section 3.5, we consider mean average precision (map) as the main measure because it rewards situations where relevant items are grouped at the top. ## conclusion we proposed an ir-based reverse dictionary system leveraging pre-trained transformer-based language models for semantic search and evaluated it thoroughly on an estonian language dictionary resource, the sõnaveeb. unlike the prediction-based approaches that have been mainly focused on in re-cent works, the ir-based approach does not require training any specialized models. furthermore, it can be easily adapted to work in the cross-lingual setting. the evaluations using the unlabeled evaluation procedure based on the synonym relations of the target dictionary and a small multilingual dataset showed that language models trained for cross-lingual retrieval are optimal for our use case. we expect that the proposed approach to building and validating reverse dictionary systems is reusable, approachable, and generalizable due to its simplicity, thus facilitating the development and improvement of language resources for lessrepresented languages with a focus on language learning, documentation, and preservation. in future work, we would like to develop a user interface to perform human evaluation of our proposed system. the ultimate goal is to integrate the best-performing model into the existing sõnaveeb language portal to enrich this resource by making the search and navigation more accessible and to support language learners via cross-lingual search.