ASOS at KSAA - CAD 2024: One Embedding is All You Need for Your Dictionary (2024)

Abstract: Semantic search tasks have grown extremely fast following the advancements in large language models, including the Reverse Dictionary and Word Sense Disambiguation in Arabic. This paper describes our participation in the Contemporary Arabic Dictionary Shared Task. We propose two models that achieved first place in both tasks. We conducted comprehensive experiments on the latest five multilingual sentence transformers and the Arabic BERT model for semantic embedding extraction. We achieved a ranking score of 0.06 for the reverse dictionary task, which is double than last year’s winner. We had an accuracy score of 0.268 for the Word Sense Disambiguation task.

## introduction a reverse dictionary (rd) takes a phrase that describes a specific concept as input and provides words whose definitions match the entered phrase. this is different from the traditional function of a forward dictionary, which maps words to their meanings or definitions, helping users understand unfamiliar terms. for example, a forward dictionary would tell the user that ( ) means ( -things are superimposed on top of each other) whereas an rd allows the user to input the phrase ( things are superimposed on top of each other) and would likely produce the word ( ) along with other words having similar meanings as the output. while numerous rds exist for languages like english, japanese, and french, arabic has seen limited development in this area, often due to the absence of substantial datasets with word definitions. rds are extremely helpful in overcoming the tipof-the-tongue phenomenon, where people struggle to find the right word to express their thoughts. additionally, rds are essential in applications of nlp such as evaluating sentence representations, mapping text to specific entities, answering questions, and retrieving information. it is crucial to understand the precise meanings of words in context. this understanding is improved by word sense disambiguation (wsd), which helps to identify the most relevant meaning of a word in a given context, thus enhancing the effectiveness and accuracy of reverse dictionaries in handling complex queries. for example, the phrase "to come together" could correspond to various options like "meet," "gather," "assemble," and more. each of these words, while synonyms, may be more appropriate in different contexts, highlighting the importance of wsd. wsd plays a significant role in improving the accuracy of rd systems by helping to determine which word meanings are most relevant based on contextual cues. this paper describes our participation in the contemporary arabic dictionary shared task, organized by the king salman global academy for arabic language (ksaa-cad). the task aims to tackle challenges in the arabic linguistic domain, focusing on two main areas: rd and wsd. in this study, we extend the methodologies previously developed in our work @xcite , utilizing an expanded model further to enhance the performance and capabilities of our approach. by participating in both tasks, we aim to contribute valuable insights and advancements to the fields of rd and wsd, particularly within the context of the arabic language, where such resources have historically been scarce. ## related work previously, researchers have tackled the rd problem using a traditional approach called semantic analysis, utilizing wordnet @xcite . they employed semantic similarity measurements to determine how similar two words are, using distance-based similarity measures to establish connections between the term and the input words in a graph @xcite . recently, many researchers have been utilizing embedding techniques alongside advanced neural networks to enhance the generation of reverse dictionaries. pilehvar (2019) implemented a combination of bidirectional long short-term memory (bilstm) and cascade forward neural network (cfnn) to improve the performance of neural rds. to assess whether a proposed neural network framework is universally effective across all languages, bendahman et al. (2022) employed sequential models with a variety of neural networks, including embedding networks, denser networks, and long short-term memory (lstm) networks. for arabic rd, al-matham et al. (2023) proposed the first shared task focused on developing rd systems for the arabic language. this encompassed two subtasks: arabic rd and crosslingual rd. the aim was to enable users to locate words based on their meanings or definitions in both arabic and english. @xcite , proposed a method that converts word definitions into multidimensional vectors and trains them using the semidecoder model. the proposed system secured the 2nd place based on the rank metric for target embeddings (electra and sgns). @xcite proposed a method utilizing an ensemble of fine-tuned arabic bert-based models to predict the word embedding for a given definition. the final representation is obtained by averaging the output embeddings from each model within the ensemble. in studies on the wsd task, @xcite introduced continuous sense comprehension (consec), a novel approach designed to address limitations in traditional supervised neural wsd systems. in conventional systems, each word is disambiguated independently without considering the explicit senses assigned to nearby words. focusing on eight languages, consec introduced a feedback loop strategy that conditions the disambiguation process on both the context and possible meanings of a word, as well as the senses assigned to adjacent words. @xcite proposed the development of a dataset for arabic wsd, utilizing an ensemble learning architecture that combined multiple bert models. the ensemble model aimed to address word ambiguity effectively and enhance classification performance by integrating various models, each equipped with unique feature sets. these feature sets included part-of-speech (pos) information, word frequency counts, and weak supervision annotations, which contributed valuable contextual information to the disambiguation process. @xcite tackled the task of word sense induction (wsi), a similar nlp task that plays a pivotal role in understanding and interpreting the nuanced meanings of language. their proposed approach for arabic wsi seeked to group sentences based on their semantic similarity. this approach involved a two-stage clustering method. in the first stage, sentences were encoded using bert or distilbert to generate sentence embeddings. in the second stage, clustering algorithms like k-means and hierarchical agglomerative clustering were used to partition sentences based on their semantic similarity. the model specifically employed the k-means algorithm to cluster the sentence embedding vectors. ## methodology in this section we will describe the used methodology that was used in the paper by starting of by the description of the dataset and our approach to expand subset of it, then the explanation about the used model and how we expanded our approach in our past paper @xcite , and for the second task wsd we will mention it in end of each section. ## result in this section we will describe our results from training the model mentioned in section 3.2.1 but before that we will mention an experiment that we did while working on the task. ## conclusion in this paper, we conducted a comprehensive experimental analysis on sentence transformers and arbertv2 to create semantic embeddings for rd and wsd tasks, training a fixed neural network with four hidden layers. our models achieved first place in both tasks, with a rank score of 0.06 for the rd task and an accuracy score of 0.268 for the wsd task. additionally, we performed a dataset analysis to develop a linguistic, logical methodology for building structured definitions for both tasks. for future work, we plan to explore multiple methods for generating embeddings from the ar-bertv2 model and aim to build a larger and more comprehensive dataset to enhance our understanding of task semantics.