Somali Information Retrieval Corpus: Bridging the Gap between Query Translation and Dedicated Language Resources
Authors: Abdisalam Mahamed Badel
Ting Zhong
Wenxin Tai
Fan Zhou
Venue: EMNLP 2023 Main (2023)
Keywords: Somali language
low-resource
information retrieval

Abstract: Despite the growing use of the Somali language in various online domains, research on Somali language information retrieval remains limited and primarily relies on query translation due to the lack of a dedicated corpus. To address this problem, we collaborated with language experts and natural language processing (NLP) researchers to create an annotated corpus for Somali information retrieval. This corpus comprises 2335 documents collected from various well-known online sites, such as hiiraan online, dhacdo net, and Somali poetry books. We explain how the corpus was constructed, and develop a Somali language information retrieval system using a pseudo-relevance feedback (PRF) query expansion technique on the corpus. Note that collecting such a data set for the low-resourced Somali language can help overcome NLP barriers, such as the lack of electronically available data sets. Which, if available, can enable the development of various NLP tools and applications such as question-answering and text classification. It also provides researchers with a valuable resource for investigating and developing new techniques and approaches for Somali.

Somali Information Retrieval Corpus: Bridging the Gap between Query Translation and Dedicated Language Resources
Abdisalam Mahamed Badel', Ting Zhong', Wenxin Tai Fan Zhou UUiveesitty of Electronic Science and Technology of China, Chengdu, Sichuan, China Kashi Institute of Electronics and Information Industry, Kashi, Xinjiang, China fricane111 @ gmail.com, zhongting @ uestc.edu.cn, wxtai @outlook.com,, fan.zhou @ uestc.edu.cn
Abstract
et al., 2016). Therefore, building the Somali in- formation retrieval corpus will significantly con- tribute to the existing literature. This corpus will assist researchers in conducting further research on it or using it in other areas of NLP. Mainly, it can be used for text classification--elltee tasks and question-ansseriigg research purposes. So far, existing research on the Somali language has not provided monolingual corpus for this language. The absence of essential digital resources the key obstacle to advancing language technologies in various low-resourced languages (Babirye et al., 2022). Thus, building electronic data sets such as IR system evaluation corpus provides valuable support to the scientific community. Corpus an electronic data set contributes extensively to the study of the language in many ways, such syn- tax, semantics, and discourse (Rayson et al., 2006). An annotated corpus a powerful tool for the eval- uation and optimization of IR systems (Kazai et al., 2012).
Despite the growing of the Somali lan- guage in various online domains, research on Somali language information retrieval remains limited and primarily relies on query transla- tion due to the lack of a dedicated corpus. To address this problem, we collaborated with lan- guage experts and natural language processing (NLP) researchers to create an annotated corpus for Somali information retrieval. This corpus comprises 2335 documents collected from var- ious well- welll-nowwn online sites, such hiiraan online, dhacdo net, and Somali poetry books. We explain how the corpus was constructed, and develop Somali language information retrieval system using pseudo-relevance feed- back (PRF) query expansion technique on the corpus. Note that collecting such data for the low-resourced Somali language can help overcome NLP barriers, such as the lack of elec- tronically available data sets. Which, if avail- able, can enable the development of various NL.P tools and applications such as question- answering and text classification. It also pro- vides researchers with a valuable resource for investigating and developing new techniques and approaches for Somali.
In this paper, we created an IR corpus for the Somali language. The corpus is annotated manu- ally by preparing group of queries and relevant judgments used evaluate the corpus based on the requirements in the Text Retrieval Conference (TREC) After the corpus was built, we evaluated using the pseudo-relevance feedback (PRF) query expansion technique. This type query expansion technique operates on the assumption that re- trieves first an initial set of top-ranked documents. Then, the most frequent words from the top-ranked documents are added to the previous query terms to modify the query, in our case, the top 5 terms. The newly formulated query terms are then submitted the system to retrieve the relevant documents. The proposed approach for Somali information retrieval has shown a good performance.
1 INTRODUCTION
Information retrieval (IR) studies help design im- portant techniques for extracting information from a large collection of data (Bhavadharani et al., 2019). There are several applications of IR in which search engine is the most known applica- tion. However, attempts made to develop such information retrieval applications for the Somali language are limited and primarily rely on query translation (Karakos et al., 2020; Boschee et al., 2019; Ogundepo et al., 2022).
Cross- languga information retrieval has certain limitations, such as finding the appropriate trans- lation of the source language query (Bhattacharya
Contributions: The main contributions of this pa- per are three-fold:
Corresponding author: @xtai@outlook.com
wtwp://tttttttttoooooooooooootee html

---

We have created corpus for Somali IR system evaluation.
Table Somali IR evaluation corpus statistics.
Number of sentences:
16450
We proposed PRF-based technique to evaluate the corpus.
Number of words:
578216
Number of documents:
2335
Average sentences per document:
7.0449
We identified a list of Somali stop words and used them as part of the research.
Average words per document:
247.6269
Average sentence length:
33.1494
Number of queries:
16
2 RELATED WORK
Average words per query:
5.56
An essential tool for IR is the corpus or test col- lection, as introduced in (Scholer et al., 2016). By mea-
3 CORPUS
constructing corpus, various algorithms sure the system's 's effectiveness. An idea offered by (Kato al.. 2021), also strongly reveals the trend toward open data for scientific openness, (Kato et al., 2021) proposed new test collection for ad- hoc retrieval, addressing issues such as techniques used and challenging topics. Furthermore, to create a corpus that accurately reflects real user behavior, (Kazai et al., 2012) found systematic judgment er- by comparing labels assigned by different in- dividuals. (Kazai al., 2012) investigated whether annotators followed guidelines, revealing potential cases of partiality. Additionally, (Mahamed Badel, 2020) demonstrated the use thesaurus for query expansion, where synonymous words are used to expand query terms. This approach is similar to a dictionary but lacks examples. Multilingual query translation, experimented (Karakos et al., 2020; Boschee et al., 90; Ogundepo al., 2022) for in- formation retrieval in low-resource languages like Somali, offers promising corridor. Furthermore, (Vijayalakshmi and Dixit, 2019) explained how on- tology can enhance query expansion, better than keyword- based systems. The sequential modules in an ontology-baue based retrieval system build on one another, as each module's 's output informs the next. A technique proposed by (Yan and Gao, 2018) also entails diversifying the PRF retrieval process. in order to get more relevant documents and reduce overlooking user attention. However, PRF can be sensitive and ignores user attention, possibly caus- ing some issues (Alhamed, 2019) introduced query modification using PRF, combining term expan- sion techniques to select important words for user queries. This study stated that the number of query terms doesn't 't matter, but sometimes can divert user expectations. In summary, however, there are var- ious studies on both corpus construction and IR techniques. This study marks the first of its type for the Somali language IR evaluation.
In this section of the paper, we briefly defined the construction process of the corpus and how we have annotated it. We also discussed the sources of the data collected. This section also introduces the query generation process and the technique applied to test the quality of the corpus.
3.1 Somali Language
The Somali language is a part of the Cushitic lan- guages that form a branch of the Afro-Asiatic fam- ily 2. The language is spoken Africa, especially the Horn Africa (Mohamed, 2013). The Somali language, as a member of the Cushitic family, is the second most spoken language in the Cushitic languages. Its writing system consists of 22 conso- nants, including glottal stop. The Somali alphabet uses all alphabets in English except three (Z. V, and P). Symbols in Somali are (b, t. j. x, kh, d, r, s, sh, dh, c, g. f. q. k, m, n, w, h, y). addition this (Mohamed, 2013), the Somali language is an agglutinative language in terms of morphological formations.
3.2 Query Generation
During the corpus preparation, the first task we conducted was generating the queries. Like the task in (Esmaili et al., 2014), we have followed the standard test collection guidelines of TREC. Therefore, here focus three parts: documents, set of queries, and relevance judgment (Right answers).
Thus, to generate the queries or topics we di- rectly interviewed a group of native speakers to forward their information needs based on subjects.
We have also asked them provide descrip- tion of what the topies they searching about. Based on set of guidelines that we gave them
hhtnp://nalrr.inddaaa...o indiani hnt.iianw.ee////////ccc//////o/a

---

Table 2:
Sample queries.
consist of domains such as sport, politics, culture. education, etc.
Query
English Meaning
fatahaada wabiga shabeelle
Shabelle River floods
3.4 Relevance judgment
suuqa jigjiga ee gublay
The burned Jigjiga mall situation in tigray region
xaalada deeguanku tigreegu
The decision that query is relevant to a document in this study is made by annotators, who are lan- guage experts and NLP researchers. In accordance with the TREC test collection guidelines, a query can be regarded as relevant to document if the query is found in any portion of the document. Fur- thermore, document is considered relevant if it either contains the query terms or owns partial rel- evance to the query. In line with the guidelines, a document may also be considered relevant even if lacks the exact query words, as long as includes synonyms of the query terms.
to follow. Then, NLP researchers and linguists formulated the queries based on the information needed and the description provided by the infor- mation seekers. The guidelines set the length of the queries to be more than one word. In addition, the queries were also allocated be from differ- ent categories of sources so that we could have a rich corpus. Moreover, the relevance of the queries was verified based on the documents in the corpus as discussed in section 3.4. After the information need of users is found and experts formulated the queries, we have shown sample relevant queries in Ta 2.
Nature of the queries used: We have used several sets of queries to evaluate the effectiveness of the corpus. Our queries range from keyword- based queries longer more descriptive ones. We have also given consideration the complexity of the queries using both simple and more complex queries. The queries span various topics in the corpus scope consisting of different user needs.
3.3 Collecting the Documents
We have collected the corpus by following the ap- proach used by most test collection studies such as (Haddow and Kirefu, 2020). Based the queries generated in section 3.2, we manually collected news articles from several online Somali language posting sites, such as hiiraan online and dhacdo net news articles. Despite the manual collection of documents from hirraan online and dhacdo net, we have also manually collected text from cultural and history books that are publicly available. One of such books is available at 3. Because there can be some rights, we have asked permission access these sites, as shown appendix B and C. The reason we chose target sources to be these sources that in these sites and books, standard and error free So- mali words are found. The corpus annotation and cleaning are done by four researchers and two lin- guistics. The document is normalized by removing HTML tags, mentions, and languages other than Somali. A total of 2335 documents remain after we cleaned the documents. Figure 1 shows the size of the collected data among the different sources. Moreover, Table shows the statistics of the corpus such as the number of sentences, number of words, etc.
3.5 Labeling the documents and queries
Throughout the corpus annotation process, we as- signed labels to both queries and documents. In total, we selected 16 queries topics to judge against this corpus. These queries are denoted as Q-1, Q-2, Q-3, and so on up Q-16, with the Q denoting query terms. We also gave labels to the document ranging from Som-0000 to Som-2335. Furthermore, besides giving labels to queries and documents, we created a set of judgment doc- uments, that correspond to each query. The ex- act number of dlocuments, evaluated for every query are follows: Queries Q-1 through Q-10 were assigned with 10 documents each, while Q-11 has 7, Q- 12 has 5, Q-13 has 8, Q-14 has 5, Q-15 has 8, and Q-16 was judged against 11 documents. This makes 1 144 documents for the total relevant documents to the total queries.
3.6 Evaluating the Corpus
To evaluate the quality of the corpus, we have ap- plied the pscudo-relevance feedback technique. In this approach, the top documents trieved first, and from the top k documents (Docs), top m terms (top terms in our case) are selected as expansion terms. A new query is refined (re- formulated), and the refined query is given back
Documents collected: The annotated docu- ments, are collected from various categories and
sawammdd-aahuulle-xssaaa//// saaaaaadi-ssaylee maxamed--abbuulle-xassn

---

The total Retrieved is the total number of the documents retrieved in response to the queries. The total Relevant is the number of documents that are relevant for a set of queries.
Corpus Sources
13%
Dhacdo
hilrson online news
books
3.7 Experiment set up
70%
In order to conduct the experiment, we have not used any framework, that implements the retrieval system. Rather we have applied custom indexing system which is a Python- basod tool. Our imple- mented program reads text documents from speci- fied directory, tokenizes the content into words, and then creates an inverted index for uni-grams from the content of those documents. The inverted index is essentially a data structure that associates each term with the documents in which it appears and the frequency of its occurrence within those docu- ments. The indexing process in this study consists of creating inverted index, calculating term fre- quency (TF) and document frequency (DF), sorting and writing token information, etc.
Figure 1: Sources of the corpus.
the system to find more matching documents. The final documents are displayed to the user with the use of the refined query terms. PRF improves the retrieval performance by assuming that the h highest-ranking documents in the first retrieval are relevant, and it selects query expansion terms from them and refines the query (Elvina and Mandala, 2020). The PRF technique does not need any inter- ference from the user because the expansion terms are automatically added to the query. The queries, as well as the documents, are preprocessed initially. As we have discussed in section 3.7 of the paper Figure 2 shows how the Somali language informa- tion retrieval using PRF works. In collaboration with linguistics, language stop words. This stop- word identification was the first attempt, far as we know. We then used this list of stop words and removed any word that matched with the stop words from the corpus during retrieval. These stop words are eliminated, from the text by reading a list of stop words in a text file and finding matches for them from the corpus.
In Figure 2 we clearly show how we conducted this PRF query expansion retrieval process. More- over, the development environment that we have used is the Pycharm editor.
have identified list of Somali
4 RESULTS
For ranking the retrieved documents based on their relevance to the query, we used the best match 25 (BM25) ranking model (Zhao et al., 2018). The BM25 matching and ranking model sets values for parameters such term frequencies, document frequency, and field length, etc.
One reason to use the BM25 model is that it is good in accuracy and fairness compared to other ranking models. Thus, utilizing this model, we have summarised the results in Table 3. The re- sults investigated with P@ 10, R@ 10, F@ 10, and P@20. 20, @ @220, F@20 20. Which literally means the performance of the system when the top 10 most relevant documents to the query are retrieved and when the top 20 most relevant documents to (1) the query are retrieved. The proposed PRF ap- proach is compared with TF-IDF to see the effect (2) of the PRF on document retrieval over TF-IDF As result, the PRF technique outperformed the TF- IDF technique. By achieving an f-measure of 72% for F@ 10 and 57% for F@20 while, the TF-IDF scored an f-measure of 54% for F@ 10 and 44% for F@ Finally, shows that, in all cases, the PRF technique produced a good performance. Figure
After we have cleaned the document, tokenized it, and removed the stop words, have created an inverted index. Lastly, documents are retrieved based on the new query, which is reformulated after the top words are added to the initial query terms. In section 3.7, we implemented the following for- mulas to measure the performance.
P Retrieved Relevant
total Retrieved
retrieved Relevant
R =
total Relevant
F 2*PR
P+R
Where P is the precision, is the recall, and is the f-measure. The Retrieved Relevant is the document retrieved and relevant to the queries.

---

Query
Initial query
Somalii Test Collection
inverted indices
Matching
Query Preprocessing
Module
Does Preprocessing
Result Process module
Refined query
Module
Top Documents
Query Refinement
Index
Top terms
Final Result
Figure 2: Retrieval process.
PRF and scored recall of 69% for R @0.0 This shows that the PRF technique outperforms TE-IDF in this corpus.
TF-IDF vs. PRF Values.
Future work the future, we recommend the development of a fully functional stemmer for the Somali language, recognizing its significant usage in various NLP applications, including IR and ma- chine learning. We would also like to extend this corpus to create standard IR evaluation corpus for the Somali language.
0.4
P@10
P@20
R@10
R@20
F@10
F@20
PRF
TF-IDF
Figure 3: Comparison between PRF and TF-IDE
Limitations
Table 3: Results.
We encountered a few limitations including a word in the Somali language may be written in different forms, for example, the Somali word ra' iisul (Prime minister in English), may also be written raysulwassare (Prime minister in English). Another limitation is that there fully functional stemmer for the Somali language, and this became limitation of our study.
Model 10
P@20
R@10
R@20
FØ
FØ
TF-IDF:
0.512
0.319
0.565
0.689
0.537
0.436
PRF:
0.687
0.415
0.758
0.917
0.721
0.572
3 shows the comparison between the PRF and the TF-IDF ranking models.
5 CONCLUSION AND FUTURE WORK
Ethics Statement
This paper is original work of the authors, it is not published in another place. It is also not rently being considered for publication in another place. The results reported are true and not falsified. All sources are properly cited and all researchers and co-authors actively contributed to this research.
this paper, we have introduced the creation of the first publicly available IR evaluation corpus for the Somali language. The corpus contains 2335 documents collected from various domains such as education, culture, etc. We have also provided a new list of Somali stop words and removed any stop word from the corpus using the created list. We then evaluated the corpus using the PRF query pansion technique, using well-established metrics like P. R, and F. The PRF technique exhibited its efficiency on the created corpus, achieving recall 92% for 2020 In comparison, the traditional TF-IDF ranking model scored lower compared to
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant 62072077 and Grant 621 62176033; part by the Natu- ral Science Foundation Sichuan Province, China, under Grant 2022NSFSC0505; and in part by the

---

Kashgar Science and Technology Bureau under Grant KS2023025. Hiiraan online, dhacdo net, and the Somali education bureau are the main source providers. We extend our heartfelt appreci- ation to our esteemed language expert colleagues and fellow NLP researchers for their valuable con- tributions to this work. Additionally, we thank the three anonymous reviewers for their constructive comments.
Damianos Karakos, Rabih Zbib, William Hartmann. Richard Schwartz, and John Makhoul. 2020. Refor- mulating Information Retrieval from Speech and Text as a Detection Problem. Proceedings of the work- shop on Cruss-Lnggage Search and Summarization of Text and Speech (CLSSTS2020), (May):38-44 -3-43.
Makoto P. Kato, Hiroaki Ohshima, Ying Hsang Liu, and Hsin Liang Chen. 2021. A Test Collection for Ad-hoc Dataset Retrieval. SIGIR 2021 Prrocedding of the 44th International ACM SIGIR Conference Research and Development Information Retrieval, pages 2450-2456.
References
Gabriella Kazai, Nick Craswell, Famine Yilmaz, and S. M.M. Tahaghoghi. 2012. An analysis of system- atic judging errors information retrieval. ACM International Conference Proceeding Series, pages 105-114.
Wafaa Alhamed. 2019. Query Modification Based on Relevance Feedback. 2019 2nd International Confer- ence on Engineering Technology and its Applications. IICETA 2019, pages 185-190.
Abdisalam Mahamed Badel. 2020. Somali Language Information Retrieval Using Query Expansion.
C Babirye, Sataatumba-Nabbndee and 2022. Build- ing Text and Speech Datasets for Low Resourced Languages: A Case of Languages in East Africa. pages 1-11.
Ahmed Mohamed. 2013. An overview of the interface between aspects of Somali phonology and morphol- ogy: a resource guide for esl teachers.
Paheli Bhattacharya, Pawan Goyal, and Sudeshna Sarkar. 2016. Query Translation for Cross-Language Information Retrieval using Multilingual Word Clus- ters. Proceedings of the 6th Workshop on South and Southeast Asian Natural Lanuagg Processing (WS- SANLP2016), pages 162.
Odunayo Ogundepo, Xinyu Zhang, Shuo Sun, Kevin Duh, and Jimmy Lin. 2022. AfriiIIRMatrix: Information Retrieval for
Enabling Cross- African Languages. Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, EMNLP 2022, pages 8721-8728.
M. Bhavadharani, M. P. Ramkumar, and G. S.R. Emil Selvan. 2019. Information Retrieval in Search En- gines Using Pseudo Relevance Feedback Mechanism. Proceedings International Conference on Vision Towards Emerging Trends in Communication and Networking, ViTECoN 2019, pages 1-5.
Paul Rayson, James Walkerdine, William H. Fletcher, and Adam Kilgarriff. 2006. Annotated web as corpus. pages 27-33.
Falk Scholer, Diane Kelly, and Ben Carterette, 2016. Information retrieval evaluation using test collections. Information Retrieval Journal, 19(3)225-229,
Elizabeth Boschee, Joel Barry, Jayadev Billa, Marjorie Freedman, Thamme Gowda, Constantine Lignos, Chester Palen-Michel, Michael Pust, Banriskhem K. Khonglah, Srikanth Madikeri, Jonathan May, and Scout Miller. 2019. Saral: A Low-resource cross- lingual domain-focused information retrieval system for effective rapid document triage. ACL 2019- 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of System Demonstrations, pages 19 24.
H. C. Vijayalakshmi and Bhavana S. Dixit. 2019. In- formation Retrieval in Kannada using Ontology. CSITSS 2019 2019 4th International Conference on Computational Systems and Information Technol- ogy for Sustainable Solution, Proceedings.
Rong Yan and Guanglai Gao. 2018. Pseudo-based rele-
vance analysis for information retrieval. Proceedings Intermational Conference on Tools with Artificial Intelligence, ICTAI, 2017 -ovem:
Elvina and Rila Mandala. 2020. Improving effective- ness information retrieval system using pseudo irrele- vance feedback. In 2020 IEEE International Confer- ence Sustainable Engineering and Creative Com- puting (ICSECC), pages 463-468.
Futao Zhao, Zhong Yao, Biao Xu, and Pengfei Tang. 2018. Exploring faimnss and accuracy of retrieval models. In 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD) pages 986-992.
Kyumars Sheykh Esmaili, Shahin Salavati, and Anwita- man Datta. 2014. Towards Kurdish information re- trieval. ACM Transactions Asian Language Infor- mation Processing, 13(2).
Barry Haddow and Faheem Kirefu. 2020. PMIndia - Collection of Parallel Corpora of Languages of India.

---

A
Sample Somali Stop words
Table 4:
Stop words.
cen
oay
kaasi
ayuu
uuna
S11
ab
loo
sidi
iska
kii
hore
isagoo
ay
kiina
sidee
sababtoo
hadii
weeye
uusan siisaa
bal
ayuu
kuwee
cid
inaga
kali
ayna
waxaan
kee
aanay
ah
sidii
iska
ahayn
kii
ay
kiina
suu
naga
laakiín
iyo
kuwaa
wax
aynu
kuu
aam
kale
waxa
ayay
anaga
miyuu
ahaa
kaa
B Dhacdo net text permission
Ogolaansho (Permission)
External
Warbaahinta Dhacdo
to me
XA
English View original message
Ase wb, greetings later brother Abdisalam Mohamed You are authorized to use The rights information the words mentioned
Thank you Event Managementt
Abdisalam Mahamed btt.diiallimmmammeeeeeoeo to Warbaahinta Mahadssani watal. Warbeahints Dhacdo way mahadsantahay.
Figure Translated email of Dhacdo net.
C Hiiraan online text permission
Hirrann Online
Somall
Englas loh
original message
you your contect request
are aumonoed use the takaan posts
hope you Finen Online anicle les the wayy you merrioredin in your seticde
compleling stuckes
Hizan cdminisration online
Abdisalam Manamed ccnrissie maberedgooouddduo
Hilssn
Figure 5: Translated email of Hiiraan online.


Reviews:
[{"review_id": "V33QQ4VD3q", "reviewer": "Reviewer_gWGd", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed review and thoughtful feedback. In the next lines of text, we focus on answering some questions.\n***\n\n> Q1: The corpus is crawled from online resources, and it will be made available. But the question of text copyrights is not stated at all. If the authors have not obtained the necessary authorizations, the main result will be illegal. This must absolutely be cleared up before publication.\n\nAnswer: It's crucial to highlight that our process of compiling the corpus was extremely thorough, taking into account the usage terms of the websites we accessed.  We collected the text from various websites and annotated only those we had permission to use. Therefore we would like to add the legal authorization to the final paper appendix. \n***\n>Q2: The number of documents manually judged to define the sets of valid documents for each query is not clear. Similarly, the indexing and matching tools are not explicit.\n\nAnswer: Yes, in the submitted version of the paper we have not added such information, we regret not adding such a piece of important information, here is the answer to your question:\n \n 1). The number of documents manually judged to define the sets of valid documents for each query(Q) are discussed as:\n     Q-1.....Q-10 (Q-1 up to Q-10) has 10 valid documents each, Q-11 has 7 valid documents, Q-12 has 5 valid documents, Q-13 has 8 valid documents, Q-14 has 5 valid documents, Q-15 has 8 valid documents and Q-16 has 11 valid documents.\n\n2). The indexing Process we have applied:\n\nTo create the indexing we have utilized a custom indexing system using a Python-based tool, our program reads text documents from a specified directory, tokenizes the content into words, and then creates an inverted index for unigrams (single words) from the content of those documents. \n\nOur main indexing process involves:\n\n* An inverted Index is created \n*  Term Frequency (TF) and Document Frequency (DF) are calculated.\n*  We sort and write to files.\n*  Tokens information is then written.\n\n3). The matching and scoring tool adopted in this paper is the BM25 (best match 25), which is a probabilistic algorithm.\n***\n>Q3: 099-101: sentence meaning is hard to get More generally, Section 2 is hard to follow as there is a lack of connections between its sentences. I am myself, not a native English speaker, but I have the feeling that this paper could be largely improved using a proofreading by a native.\n\nAnswer: We agree with you that our paper contains many grammatical errors. We would like to revise Section 2 and produce a well-organized section in the final version. \n\nWe hope, that we have at least attempted to give answers to your questions.\n***", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Creating linguistic resources for low resource languages is a very important topic, particularly for languages from families with few high resources languages.\nThe work presented seems to be correctly made using classical methods, reproducing techniques initially created for English. This will allow to compare future results with state of the art.", "reasons_to_reject": "The corpus is crawled from online resources, and it will be made available. But the question of text copyrights is not stated at all. If the authors have not obtained the necessary authorizations, the main result will be illegal. This must absolutely be cleared up before publication. From authors answer, this will be done for final version.\n\nThe number of documents manually judged to define the sets of valid documents for each query is not clear. Similarly, the indexing and matching tools are not explicit. Authors answer clear up these questions. I Suggest for future works to use an indexing engine like Elastic Search or another to avoid having to reproduce everything.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This short paper presents the creation of a TREC-like Information Retrieval corpus for the Somali language. It presents the collection of queries from users, the crawling of a corpus and the generation of reference document lists from the corpus for each query. A secondary result is the first list of stop words for Somali.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "099-101: sentence meaning is hard to get\nMore generally, Section 2 is hard to follow as there is a lack of connections between its sentences.\n\nI am myself not a native English speaker, but I have the feeling that this paper could be largely improved using a proofreading by a native."}}, {"review_id": "eOm515GFlL", "reviewer": "Reviewer_PkYX", "summary": "", "questions": "Can you please provide the data mentioned in the previous section? If authors are able to provide them it would improve the overall Soundness of the paper, and I would reconsider its Soundness score.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your helpful feedback on the review of our paper. We would like to provide answers to specific questions below. \n***\n\n> Q1: The paper still needs work in some areas. It lacks certain details that are important when presenting a new resource. I would suggest authors provide with complete numbers, as the total number of queries in the dataset, and perhaps some statistics, such as the average length of documents and queries.\n\nAnswer: Right, the statistics are important information about a paper presenting a new resource. Therefore, below we have discussed the above question and listed the statistics of the corpus.\n\n * The number of queries in the data set is a total of **16** queries.\n*  Number of sentences: **19320**\n* Number of words: **578170**\n*  Number of documents: **2312**\n*  Average sentences per document: **8.356401384083044**\n*  Average sentence length: **29.927388469102578**\n*  Average words per document: **250.0735294117647**\n* Number of queries: **16**\n*  Average words per query: **4.9375** \n\n****\n> Q2: The Figures with results are too fine-grained. While they show how the proposed baseline has some improvement over tf-idf, it is done at the sample level for just 10 queries. I think there's a need for a table with results over the total dataset, by providing some relevant metric such as Recall@10 or similar.\n\nAnswer:  To answer the above question we have computed precision(P), recall(R) and f-measure(F) for the queries across the entire dataset, The performance is tested by retrieving the top 10 and the top 20 most relevant documents(k) to the queries, we have discussed precision@10, recall@10, precision@20, recall@20, f-measure@10 and f-measure@20 as:\n\n\n| metric    | **TF-IDF**|             &emsp;   &emsp;                        **Pseudo relevance feedback (PRF) approach** |      \n| --------  |  ---------- | -----------------------------------------------------------------------------------------------------|               \n| P@10  | 0.506       |   &emsp;      &emsp;   &emsp;                                                  0.69                                    |\n| R@10  | 0.562        |    &emsp;        &emsp;  &emsp;                                                   0.77                            |\n| F@10   | 0.532 |      &emsp;      &emsp;    &emsp;                                                       0.73                     |\n| P@20   | 0.31  |       &emsp;        &emsp;   &emsp;                                                0.41                |\n| R@20   |  0.69 |        &emsp;         &emsp;   &emsp;                                              0.92             |\n| F@20    | 0.43 |         &emsp;        &emsp;    &emsp;                                                 0.57               |\n---------------------------------------------------------------------------------------------------------------------------------\n\nTo calculate the above metrics we have used the following formulas for precision, recall, and f-measure:\n\n**P** = Retrieved relevant/total Retrieved\n\n**R** = Retrieved Relevant/total Relevant\n\n**F**= (2*PR)/P+R\n         \nWhere R is the recall, P is the precision and F is the F-measure. \n\n***\n> Q3: The current version of the paper needs improvement on the use of English. I suggest to share the paper with an English native speaker to check it, and if that is not possible, consider using a tool like Grammarly to correct it.\n\nAnswer: We understand that the current version of the paper has various grammatical errors, so we take this comment and improve the writing style of the final version of the paper.\n****\nWe believe that we have tried to address and give answers to your comments.\n****", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Somali is spoken by more than 20 million people, however resources are scarce. A new IR dataset is a small but important contribution to closing the gap with other languages. As comparison, languages such as Swedish, Finish or Bulgarian are spoken by far less speakers but have many more systems and resources available and have stemmers.\n\n- The dataset was created by native speakers and linguists with knowledge of the language.\n\n- They provide a new list of stop words for Somali.", "reasons_to_reject": "- The paper still needs work in some areas. It lacks certain details that are important when presenting a new resource. I would suggest authors provide with complete numbers, as the total number of queries in the dataset, and perhaps some statistics, such as the average length of documents and queries. The Figures with results are too fine-grained. While they show how the proposed baseline has some improvement over tf-idf, it is done at the sample level for just 10 queries. I think there's a need for a table with results over the total dataset, by providing some relevant metric such as Recall@10 or similar.", "questions_to_authors": "Can you please provide the data mentioned in the previous section? If authors are able to provide them it would improve the overall Soundness of the paper, and I would reconsider its Soundness score.", "paper_topic_and_main_contributions": "The paper presents a new dataset for Information Retrieval in Somali. The authors created a corpus of 2K documents and a set of queries. Then documents were labeled as relevant or not by annotators.\n\nThe annotation and data collection was done by native speakers and linguists, and is relevant due to the lack of NLP resources for Somali. The authors also point out the lack of proper stemmer for Somali and created a list of stop words.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "The current version of the paper needs improvement on the use of English. I suggest to share the paper with an English native speaker to check it, and if that is not possible, consider using a tool like Grammarly to correct it."}}, {"review_id": "sNSi1TiP8Q", "reviewer": "Reviewer_ReAG", "summary": "", "questions": "A. Please, justify the authors' position in relation to the reviewer's\nitem B in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but it would be necessary to review the new\nversion to issue a definitive opinion.\n\nB. Please, justify the authors' position in relation to the reviewer's\nitem C in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and provide all the details\nnecessary to reproduce the tagging process.\n\nC. Please, justify the authors' position in relation to the reviewer's\nitem D in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and also clarify how the concept\nof \"relevance\" has been formalized.\n\nD. Please, justify the authors' position in relation to the reviewer's\nitem E in the previous section \"Reasons to reject\".\n\nThe answer is confusing. Does not provide relevant information.\n\nA. Please, justify the authors' position in relation to the reviewer's\nitem B in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but it would be necessary to review the new\nversion to issue a definitive opinion.\n\nB. Please, justify the authors' position in relation to the reviewer's\nitem C in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and provide all the details\nnecessary to reproduce the tagging process.\n\nC. Please, justify the authors' position in relation to the reviewer's\nitem D in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and also clarify how the concept\nof \"relevance\" has been formalized.\n\nD. Please, justify the authors' position in relation to the reviewer's\nitem E in the previous section \"Reasons to reject\".\n\nThe answer is confusing. Does not provide relevant information.\n\nA. Please, justify the authors' position in relation to the reviewer's\nitem B in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but it would be necessary to review the new\nversion to issue a definitive opinion.\n\nB. Please, justify the authors' position in relation to the reviewer's\nitem C in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and provide all the details\nnecessary to reproduce the tagging process.\n\nC. Please, justify the authors' position in relation to the reviewer's\nitem D in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and also clarify how the concept\nof \"relevance\" has been formalized.\n\nD. Please, justify the authors' position in relation to the reviewer's\nitem E in the previous section \"Reasons to reject\".\n\nThe answer is confusing. Does not provide relevant information.\n\nE. Section 5 cannot really be considered as a set of conclusions on the\nquality of the elaborated corpus, but rather a simple summary of the\nproposal. In this same line, the authors also fail to include a real\ndiscussion of the practical results.\n\nThe answer is adequate.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very happy with your constructive comments those we think may help us enhance the writing style of our paper, We would like to give answers to some of the questions from the review the questions are numbered as A, B, C, D, E, Missing References, Typos Grammar Style And Presentation Improvements.\n***\n> A. The writing style is very sloppy and includes, in addition to typos, not a few grammatical constructions that seem out of context. The perceived sensation is that sentences have been accumulated, without paying too much attention to how they are linked together to form a comprehensible text.\n\nAnswer: We completely agree with you, that there are many styling problems in the paper. We will rewrite the whole text and submit a new version, which makes sense in the final, If our paper is accepted, the final version of the paper will be different than the current one, following the comments from the review.\n***\n\n> B.  The only information on the size of the corpus is the number of documents (2300), but we know nothing about the number of sentences, the number of words or the average size of the sentences. The information about the subject matter is also very scarce. Also we don't know anything about the set of labels used.\n\nAnswer: The corpus statistics, such as the number of sentences, the number of words, or the average size of the sentences is given as:\n* Number of sentences: **19320**\n* Number of words: **578170**\n* Number of documents: **2312**\n* Average sentences per document: **8.356401384083044**\n* Average sentence length: **29.927388469102578**\n* Average words per document: **250.0735294117647**\n* Number of queries: **16**\n* Average words per query: **4.9375**\n\n  **The information about the subject matter**: If your question is about the domain of the documents collected, it consists of domains such as sport, politics, culture, and education.\n\n    **Labels used**: The queries in this corpus are labeled as **Q-1**........**Q-16** and the documents are labeled as **Som-0000**.........**Som-2312**, we  hope that we have understood your concern. \n***\n\n>  C. The measures applied on the IR system used to evaluate the quality of the corpus have some shortcomings. So, why the authors do not use any of the standard measures, such as precision, recall, F-measure, ... ?\n\nAnswer: You are right, during the evaluation process of an IR system, the mentioned metrics make great sense. We would like to provide the  precision(P), recall(R) and F-measure(F) of the corpus by retrieving the top 10 and the top 20 most relevant documents to each query, the results are shown as:\n\n| metric    | **TF-IDF**|   &emsp;  &emsp;                                   **Pseudo relevance feedback (PRF) approach** |      \n| -------- | ----------|-----------------------------------------------------------------------------------------------------|               \n| P@10 | 0.506      |    &emsp;  &emsp;  &emsp;                                                              0.69                   |\n| R@10 | 0.562     |      &emsp;  &emsp;  &emsp;                                                        0.77                 |\n| F@10  | 0.532 |       &emsp;  &emsp;   &emsp;                                                           0.73                 |\n| P@20  | 0.31  |        &emsp;  &emsp;  &emsp;                                                      0.41                |\n| R@20  |  0.69 |       &emsp;  &emsp;   &emsp;                                                       0.92             |\n| F@20   | 0.43 |       &emsp;  &emsp;  &emsp;                                                           0.57               |\n---------------------------------------------------------------------------------------------------------------------------------\n\nTo calculate the above metrics we have used the following formulas for precision, recall, and f-measure:\n\n**P** = Retrieved relevant/total Retrieved\n\n**R** = retrieved relevant/total relevant\n\n**F**= (2*PR)/P+R\n         \nWhere R is the recall, P is the precision and F is the F-measure. \n***\n> D. Also, in relation to this evaluation process, no data is provided regarding the nature of the queries used.\n\nAnswer: The nature of the queries that we have used in the evaluation of this corpus, span different categories such as retrieving information about prominent players, culture, historical sites, songs, and educational information, for example, Q-6: wabiga shabeelle (Shabelle river).\n***\n> E. Section 5 cannot really be considered as a set of conclusions on the quality of the elaborated corpus, but rather a simple summary of the proposal. In this same line, the authors also fail to include a real discussion of the practical results.\n\nAnswer: We would like to express our keenness to re-arrange the conclusion part of the paper improve it, and add some practical aspects of the corpus.\n***\n> Missing References: The authors repeatedly refer to the existence of corpora for the Somali language that is based on query translation, but no references are provided.\n\nAnswer: We are too concerned about not citing referenced authors in the places you mentioned, but we were referencing (Karakos2020, Boschee2019) in the related work of the paper. \n***\n> Typos Grammar Style And Presentation Improvements: As already mentioned, the wording is very sloppy and requires a thorough revision. For example, Section 2 of the paper includes:\n\nAnswer: Indeed, we see the errors you figured out in the paper, section 2, and also other parts of the paper. The final version of the paper will not look like this if it is accepted at the conference. We will do a profound proofreading and improve any error you figured out in the paper.\n***\n We appreciate your questions and hope these responses address your concerns.\n****", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Attempts to retrieve information in the Somali language are probably scarce, so\nthis work should contribute to alleviate this situation.", "reasons_to_reject": "A. The writing style is very sloppy and includes, in addition to typos,\nnot a few grammatical constructions that seem out of context. The\nperceived sensation is that sentences have been accumulated, without\npaying too much attention to how they are linked together to form a\ncomprehensible text.\n\nB. The only information on the size of the corpus is the number of\ndocuments (2300), but we know nothing about the number of sentences,\nthe number of words or the average size of the sentences. The\ninformation about the subject matter is also very scarce. Also \nwe don't know anything about the set of labels used.\n\nC. The measures applied on the IR system used to evaluate the quality of\nthe corpus have some shortcomings. So, why the authors do not use\nany of the standard measures, such as precision, recall, F-measure, ... ?\n\nD. Also, in relation with this evaluation process, no data is provided\nregarding the nature of the queries used.\n\nE. Section 5 cannot really be considered as a set of conclusions on the\nquality of the elaborated corpus, but rather a simple summary of the\nproposal. In this same line, the authors also fail to include a real\ndiscussion of the practical results.", "questions_to_authors": "A. Please, justify the authors' position in relation to the reviewer's\nitem B in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but it would be necessary to review the new\nversion to issue a definitive opinion.\n\nB. Please, justify the authors' position in relation to the reviewer's\nitem C in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and provide all the details\nnecessary to reproduce the tagging process.\n\nC. Please, justify the authors' position in relation to the reviewer's\nitem D in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and also clarify how the concept\nof \"relevance\" has been formalized.\n\nD. Please, justify the authors' position in relation to the reviewer's\nitem E in the previous section \"Reasons to reject\".\n\nThe answer is confusing. Does not provide relevant information.\n\nA. Please, justify the authors' position in relation to the reviewer's\nitem B in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but it would be necessary to review the new\nversion to issue a definitive opinion.\n\nB. Please, justify the authors' position in relation to the reviewer's\nitem C in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and provide all the details\nnecessary to reproduce the tagging process.\n\nC. Please, justify the authors' position in relation to the reviewer's\nitem D in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and also clarify how the concept\nof \"relevance\" has been formalized.\n\nD. Please, justify the authors' position in relation to the reviewer's\nitem E in the previous section \"Reasons to reject\".\n\nThe answer is confusing. Does not provide relevant information.\n\nA. Please, justify the authors' position in relation to the reviewer's\nitem B in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but it would be necessary to review the new\nversion to issue a definitive opinion.\n\nB. Please, justify the authors' position in relation to the reviewer's\nitem C in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and provide all the details\nnecessary to reproduce the tagging process.\n\nC. Please, justify the authors' position in relation to the reviewer's\nitem D in the previous section \"Reasons to reject\".\n\nThe answer is adequate, but the authors should include that\ninformation in the revised version, and also clarify how the concept\nof \"relevance\" has been formalized.\n\nD. Please, justify the authors' position in relation to the reviewer's\nitem E in the previous section \"Reasons to reject\".\n\nThe answer is confusing. Does not provide relevant information.\n\nE. Section 5 cannot really be considered as a set of conclusions on the\nquality of the elaborated corpus, but rather a simple summary of the\nproposal. In this same line, the authors also fail to include a real\ndiscussion of the practical results.\n\nThe answer is adequate.", "paper_topic_and_main_contributions": "The authors present an (manually) annotated corpus for Somali language. The\ncontributions are in the field of the new data resources for\nlow-resource languages (Somali). The development of an information\nretrieval system on this corpus has only a referential value in\nrelation to its quality.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "The authors repeatedly refer to the existence of corpora for the\nSomali language that are based on query translation, but no references\nare provided."}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "As already mentioned, the wording is very sloppy and requires a\nthorough revision. For example, Section 2 of the paper includes:\n\nLine 095: \"By conducting deep analysis authors\" --> \"By conducting deep analysis, authors\"\n\nLine 097: \"What techniques have been used, what topics were hard are some of the issues studied in this paper.\" -->\n     \t  \"What techniques have been used and what topics were hard are some of the issues studied in this paper.\"\n\nLine 099: This sentence seems incomplete:\n\n\"To construct a corpus for information retrieval system that reflects real users (Kazai et al., 2012).\" \n\nLine 108: \"Thesaurus lists words which are synonyms, it is similar to dictionary but lacks examples and definitions.\" --> \n\n     \t  \"Thesaurus lists words which are synonyms, they are similar to dictionary but lacks examples and definitions.\" \n\nLine 103: AS --> As\n\nLine 117: \"Ontology can also improve query expansion as shown by (Vijayalakshmi and Dixit, 2019).\" -->\n     \t  \"Ontologies can also improve query expansion as shown by (Vijayalakshmi and Dixit, 2019).\"\n\nLine 121: ontology based --> ontology-based\n\nLine 124: This sentence seems incomplete:\n\n\"Novel approach that diversifies the pseudo relevance feed-back documents (Yan and Gao, 2018).\"\n\nAll of these defects extend (to a lesser extent) to the rest of the\nSections. So, for example:\n\nLine 057: \"Annotated corpus is powerful tool for evaluation\"  --> \"Annotated corpus is a powerful tool for evaluation\"\n\nLine 058: information retrieval --> IR\n\nLine 060: information retrieval --> IR\n\nLine 064: \"Text Retrieval Conference(TREC)\" --> \"Text Retrieval Conference (TREC)\" \n\nLine 174: \"In the guideline length\" --> \"In the guideline, length\"\n\nLine 194: Judges --> judges\n\nLine 216: \"we have Applied\" --> \"we have applied\"\n\nLine 219: documents(Docs) --> documents (Docs)\n\nLine 230: \"The Queries as well as the documents are Preprocessed initially\" --> \"The queries as well as the documents are preprocessed initially\"\n\nLine 238: Missing punctuation marks in this sentence ?\n\n\"Stop words are eliminated from the text by reading a list of stop-\nwords in a text file any word which matches with the stop-words listed\nin the file is then removed.\"\n\nLine 242: \"After the document is cleaned it is\" --> \"After the document is cleaned, it is\"\n\nLine 244: \"reformulated(refined)\" --> \"reformulated (refined)\"\n\nLine 248: \"documents(docs)\" --> \"documents (docs)\"\n\nLine 250: MB25 --> BM25\n\nLine 265: \"In addition to this Figure 3\" --> \"In addition to this, Figure 3\"\n\nLine 275: \"poems and Proverbs\" --> \"poems and proverbs\""}}]

Author rebuttal:

Meta-review: This paper introduces a novel information retrieval corpus as well as a list of stop words for Somali, thus addressing a gap in NLP research for a low-resource language. While the excitement and potential impact of this work deserves praise, some details are vague or missing, e.g. the total number of queries in the dataset, statistics regarding the length of the documents and queries, etc. These concerns were nevertheless addressed by the authors in their rebuttals, and reviewers acknowledged these responses as satisfactory. There is, however, substantiated concern over style and presentation, suggesting the paper would benefit from the input of a native speaker or at minimum, an automatic grammar checker like Grammarly.

In summary, the revisions required to address paper content concerns can be accomplished quite readily, as indicated by fruitful discussions during the rebuttal period. The bulk of revisions would likely be dedicated to ensuring the writing is clear and up to standard.
Decision: Accept-Main