S omali Information Retrieval Corpus: Bridging the Gap between Query Translation and Dedicated Language Resources (2023)

Abstract: Despite the growing use of the Somali language in various online domains, research on Somali language information retrieval remains limited and primarily relies on query translation due to the lack of a dedicated corpus. To address this problem, we collaborated with language experts and natural language processing (NLP) researchers to create an annotated corpus for Somali information retrieval. This corpus comprises 2335 documents collected from various well-known online sites, such as hiiraan online, dhacdo net, and Somali poetry books. We explain how the corpus was constructed, and develop a Somali language information retrieval system using a pseudo-relevance feedback (PRF) query expansion technique on the corpus. Note that collecting such a data set for the low-resourced Somali language can help overcome NLP barriers, such as the lack of electronically available data sets. Which, if available, can enable the development of various NLP tools and applications such as question-answering and text classification. It also provides researchers with a valuable resource for investigating and developing new techniques and approaches for Somali.

## introduction information retrieval (ir) studies help design important techniques for extracting information from a large collection of data @xcite . there are several applications of ir in which search engine is the most known application. however, attempts made to develop such information retrieval applications for the somali language are limited and primarily rely on query translation @xcite @xcite . cross-language information retrieval has certain limitations, such as finding the appropriate translation of the source language query @xcite . therefore, building the somali information retrieval corpus will significantly contribute to the existing literature. this corpus will assist researchers in conducting further research on it or using it in other areas of nlp. mainly, it can be used for text classification-related tasks and question-answering research purposes. so far, existing research on the somali language has not provided a monolingual corpus for this language. the absence of essential digital resources is the key obstacle to advancing language technologies in various low-resourced languages @xcite . thus, building electronic data sets such as ir system evaluation corpus provides valuable support to the scientific community. corpus as an electronic data set contributes extensively to the study of the language in many ways, such as syntax, semantics, and discourse @xcite ). an annotated corpus is a powerful tool for the evaluation and optimization of ir systems @xcite . in this paper, we created an ir corpus for the somali language. the corpus is annotated manually by preparing a group of queries and relevant judgments used to evaluate the corpus based on the requirements in the text retrieval conference (trec) foot_0 . after the corpus was built, we evaluated it using the pseudo-relevance feedback (prf) query expansion technique. this type of query expansion technique operates on the assumption that it retrieves first an initial set of top-ranked documents. then, the most frequent words from the top-ranked documents are added to the previous query terms to modify the query, in our case, the top 5 terms. the newly formulated query terms are then submitted to the system to retrieve the relevant documents. the proposed approach for somali information retrieval has shown a good performance. contributions: the main contributions of this paper are three-fold: ## related work an essential tool for ir is the corpus or test collection, as introduced in @xcite . by constructing a corpus, various algorithms can measure the system's effectiveness. an idea offered by @xcite , also strongly reveals the trend toward open data for scientific openness, @xcite proposed a new test collection for adhoc retrieval, addressing issues such as techniques used and challenging topics. furthermore, to create a corpus that accurately reflects real user behavior, @xcite found systematic judgment errors by comparing labels assigned by different individuals. @xcite investigated whether annotators followed guidelines, revealing potential cases of partiality. additionally, @xcite demonstrated the use of a thesaurus for query expansion, where synonymous words are used to expand query terms. this approach is similar to a dictionary but lacks examples. multilingual query translation, as experimented @xcite @xcite for information retrieval in low-resource languages like somali, offers a promising corridor. furthermore, @xcite explained how ontology can enhance query expansion, better than keyword-based systems. the sequential modules in an ontology-based retrieval system build on one another, as each module's output informs the next. a technique proposed by @xcite ) also entails diversifying the prf retrieval process, in order to get more relevant documents and reduce overlooking user attention. however, prf can be sensitive and ignores user attention, possibly causing some issues @xcite introduced query modification using prf, combining term expansion techniques to select important words for user queries. this study stated that the number of query terms doesn't matter, but sometimes can divert user expectations. in summary, however, there are various studies on both corpus construction and ir techniques. this study marks the first of its type for the somali language ir evaluation. ## corpus in this section of the paper, we briefly defined the construction process of the corpus and how we have annotated it. we also discussed the sources of the data collected. this section also introduces the query generation process and the technique applied to test the quality of the corpus. ## results for ranking the retrieved documents based on their relevance to the query, we used the best match 25 (bm25) ranking model @xcite . the bm25 matching and ranking model sets values for parameters such as term frequencies, document frequency, and field length, etc. one reason to use the bm25 model is that it is good in accuracy and fairness compared to other ranking models. thus, utilizing this model, we have summarised the results in ## conclusion and future work in this paper, we have introduced the creation of the first publicly available ir evaluation corpus for the somali language. the corpus contains 2335 documents collected from various domains such as education, culture, etc. we have also provided a new list of somali stop words and removed any stop word from the corpus using the created list. we then evaluated the corpus using the prf query expansion technique, using well-established metrics like p , r, and f . the prf technique exhibited its efficiency on the created corpus, achieving a recall of 92% for r@20. in comparison, the traditional tf-idf ranking model scored lower compared to prf and scored a recall of 69% for r@20. this shows that the prf technique outperforms tf-idf in this corpus.