Qamosy at A rabic Reverse Dictionary shared task: Semi Decoder Architecture for Reverse Dictionary with SBERT Encoder (2023)

Abstract: A reverse dictionary takes a descriptive phrase of a particular concept and returns words with definitions that align with that phrase. While many reverse dictionaries cater to languages such as English and are readily available online or have been developed by researchers, there is a notable lack of similar resources for the Arabic language. This paper describes our participation in the Arabic Reverse Dictionary shared task. Our proposed method consists of two main steps: First, we convert word definitions into multidimensional vectors. Then, we train these encoded vectors using the Semi-Decoder model for our target task. Our system secured 2nd place based on the Rank metric for both embeddings (Electra and Sgns).

## introduction a reverse dictionary takes a phrase describing a specific concept as input and provides words whose definitions match that entered phrase. in contrast, a regular or common (forward) dictionary function contains word-to-meaning or definition mappings, which represents a useful solution for readers when encountering unfamiliar words in a text. for example, a forward dictionary would tell the user that ( -he felt compassion for him) means ( -have mercy on him) whereas a reverse dictionary allows the user to input the phrase ( have mercy on him) and would likely produce the word( -he felt compassion for him) along with other words having similar meanings as the output. reverse dictionaries offer significant practical value; primarily, they are highly effective in resolving the tip-of-the-tongue phenomenon which we encounter every day; people have difficulties finding the precise word to convey their thoughts, despite being on their tongue tip. as a result, they use phrases to explain the word or the concept. this challenge could stem from memory retrieval issues or a limited understanding of a particular language. such a predicament is widespread, especially when someone is endeavoring to learn a new language and has a restricted number of vocabulary words or people who write frequently and seek a word that precisely matches their intended thought or expression. regarding natural language processing (nlp), reverse dictionaries serve various purposes. one of these is assessing sentence representation quality. additionally, they prove advantageous in tasks related to text-to-entity mapping, such as question answering and information retrieval. moreover, reverse dictionaries consider not only the individual meanings of words but also how those meanings change when combined. many words have synonyms, making determining the exact match for a given definition difficult. for instance, the input "to come together" could correspond to various options like "meet," "gather," "assemble," and more. consequently, reverse dictionaries offer several potential word options rather than one possible word. numerous reverse dictionaries have been available online or have been created by researchers catering to different languages like english, japanese, turkish, french, and persian. however, a noticeable absence of equivalent resources can be observed in the arabic language. this shortage could stem from the lack of appropriate or substantial datasets containing words and their respective definitions which entails significant efforts in collecting and structuring language data. this paper, outlines our contribution to the arabic reverse dictionary shared task. our approach involves two phases: first, transforming word definitions into multidimensional vectors, and then training these vectors with the simi-decoder model for the intended task. ## methodology in this section, we will start describing the dataset used for our work. then, we will explain our approach, divided into two primary steps. the first is to represent or encode the inputs (the definitions of the words) as multidimensional vectors. the second stage is to train the encoded inputs using a simi-decoder model for our downstream task. ## result we began our initial experiments with the development dataset. afterward, we selected the most efficient method to produce predictions for the test dataset. our evaluation relied on the official evaluation metric supported by the event organizers: mean square error (mse), cosine similarity, and rank, which evaluates the model's ability to order predictions in relation to actual values. the primary evaluation metric is rank. if models yield similar results based on this metric, the mean squared error (mse) is then employed as a secondary measure. in cases where further differentiation is required, cosine similarity serves as the tertiary metric. to clarify, we experimented with seven different architecture setups to train the model, taking into account the number of embedding layers and dropoff. we chose the best model on electra (model no. 3) to apply them to sgns, in addition to the baseline model (model no. 1). the concept behind our approach is modifying the standard encoder-decoder architecture by truncating its latter section, which we have called the 'semi-decoder'. due to the extensive scale of our model, an epoch range of 100-300 was inadequate for training. when the epochs exceeded 2000, overfitting issues emerged with our test data. this observation led us to conclude that the optimal epoch range is between 1000 and 2000. specifically, the 2000-epoch mark resulted in a 'semi-overfitting' sit- ## conclusion our methodology encompasses two fundamental stages. initially, we encode the word definitions, translating them into multidimensional vector representations. subsequently, we subject these encoded vectors to training via the simi-decoder model to address our designated task. our system secured a 2nd place based on rank metric for both embeddings (electra and sgns). future work could involve collecting more data for training or validation, or providing the service online for public access. improving our model's performance might be achieved by adopting the bert or transformer model for training, known for efficient parallel processing and capturing longterm dependencies.