Revisiting Anwesha:Enhancing Personalised and Natural Search in B angla (2022)

Abstract: Bangla is a low-resource, highly agglutinative language. Thus it is challenging to facilitate an effective search of Bangla documents. We have created a gold standard dataset containing query document relevance pairs for evaluation purposes. We utilise Named Entities to improve the retrieval effectiveness of traditional Bangla search algorithms. We suggest a reasonable starting model for leveraging implicit preference feedback based on the user search behaviour to enhance the results retrieved by the Explicit Semantic Analysis (ESA) approach. We use contextual sentence embeddings obtained via Language-agnostic BERT Sentence Embedding (LaBSE) to rerank the candidate documents retrieved by the traditional search algorithms (tf-idf) based on the top sentences that are most relevant to the query. This paper presents our empirical findings across these directions and critically analyses the results.

## introduction owing to india's multilingual diversity, it is important to ensure that a wide gamut of people from diverse backgrounds are able to access the web without any language barrier. bengali alternatively known as bangla, has 300 million speakers globally and has witnessed the fastest growth among the other indic languages in terms of internet usage @xcite . hence there is a pressing need to develop tools that facilitate semantic search over bangla text documents. previously, efforts have been put into creating bangla search engines. for example, anwesan 1 was built to search over rabindra rachanabali collection 2 @xcite 1 http://anwesan.iitkgp.ernet.in/ 2 https://rabindra-rachanabali.nltr.org/ 2012). sandhan foot_0 is a monolingual domainspecific search engine limited to tourism and health domains in nine indian languages: bangla, hindi, marathi, tamil, telugu, punjabi, odiya, gujarati and assamese. it is based on the bag of words model and focuses more on improving recall than precision @xcite . hence the top results are not always relevant for the query. pipilika foot_1 , launched on april 13, 2013, is designed for the residents of bangladesh. it crawls data from bangla news, bangla blogs and bangla wikipedia. however, anwesan and pipilika foot_2 are not presently accessible for exploration. this paper reports follow-up work based on @xcite recent work, which introduced an exploration toward building অে ষা (anwesha), a prototype for a search engine in bangla. anwesha demonstrated promise in addressing the existing search engines' shortcomings and advanced the research done in the information retrieval (ir) space for the bangla language. anwesha incorporated the use of diverse knowledge sources like indowordnet 6 @xcite , statistical co-occurrences (by way of latent semantic analysis (lsa) @xcite ) and external knowledge sources like wikipedia (by way of explicit semantic analysis (esa) @xcite ) for facilitating effective retrieval, opening gateways to further improvements in the search quality results. the authors have released a gold standard dataset 7 containing 94 query doc-ument relevance pairs over a test collection of 1182 documents. the collection contains 182 short stories, novels and essays written by rabindranath tagore foot_5 and 1000 newspaper articles published in 2013 crawled from the daily newspaper of bangladesh prothom alo foot_6 . every query was designed to belong to one of the four different complexity levels, as shown in in its present form, anwesha has been evaluated on a small set of queries. the lack of handling of multi-word expressions has adversely affected anwesha's performance in several cases, especially in complexity level 1 query. it does not use the implicit feedback the user provides via click preferences to improve the retrieval effectiveness. for best results, users are restricted by the choice of words that exactly match the contents of relevant documents. any change in word order in the query can potentially lead to contrasting results, which are not captured in anwesha. this paper presents four directions to address the current limitations of anwesha. first, we expand the gold standard dataset by creating an additional 100 query document relevance pairs over a new test collection foot_7 of 1000 documents for a more exhaustive evaluation and better analysis. second, we identify the technical terms and named entities in the documents and queries apart from considering only uni-gram word tokens as was done previously. third, we introduce a novel approach to improve the effectiveness of the ir system by incorporating implicit preference feedback via clickthrough data in an esa setting. lastly, we present the usage of contextual vector representations of documents and query using language-agnostic bert sentence embedding (labse) @xcite to rerank the documents based on the best sentences from the document that are relevant to the query. we believe that our approaches can be adapted for other low-resource, highly inflected and agglutinative languages similar to bangla, such as assamese, maithili, oriya and manipuri @xcite . the rest of this paper is organized as follows. in section 2, we position our approaches in the context of background work and relevant research. section 3 describes the implementation details of our approaches. section 4 presents a critical analysis of our empirical findings and observations. section 5 summarizes our key contributions and discusses potential extensions of the work. ## background and literature survey this section discusses the concepts that will be used in the rest of the paper. ## conclusion and future work we have enabled ne-based search to obtain single-unit ne tokens. to the best of our knowledge, ours is the first effort that delivers personalized results from diverse background knowledge sources via the clickthrough information of the user. we are also not aware of the past work that applies bert models and it's advancement on bangla ir. we have extended the previously existing gold standard dataset for diverse evaluation of search results and used this to systematically study the improvement of anwesha while addressing its current limitations. our technique can inspire research in ir for other low-resource, highly inflected languages. as part of future work, we plan to handle different forms of mwes like conjunct verbs (example "অনু ভব করা"/ anubhaba karā (en: to feel)), noun-verb collocations (example "েখেত যাওয়া"/khete yāōẏā (en: go eat)), reduplicated terms (example "েছাট েছাট"/choṭa choṭa (en: small small)), idiomatic compound nouns (example: "ভাই েবান"/bhāi bona (en: brother sister)) etc @xcite . in future, we can incorporate actions like bookmarking, saving a document, and time of viewing a document as other contributors to determining user preferences. esa is a recall-centric approach while labse, as a reranker on the tf-idf vector space approach, is precision oriented. studying the effect of the esa cosine similarity scores used as surrogates to weigh the labse sentence scores will be interesting. in some cases, where the queries were not precise, tf-idf could not include the relevant candidate documents in the initial pool of candidates. so labse could not perform well on them, suggesting that a recallcentric algorithm could plausibly be used to determine the initial candidate set. estimating the sentence-level relevance of a document to the query is a reasonable approach because labse is trained with a sentence-level objective, making it suitable to find the relationship between a query sentence and a sentence from the document. however, in this approach, we are losing context information. hence a more robust language model that could encode the "long" documents while effectively gauging their relevance to a "short" query would be a better alternative to our existing approach using birch. in future, we plan to conduct our experiments over a wider range of queries. the dataset and code of our work are present here: https://github.com/ arupdas15/revisiting_anwesha .