Term-Recency for TF - IDF , BM 25 and USE Term Weighting (2020)

Abstract: Effectiveness of a recommendation in an Information Retrieval (IR) system is determined by relevancy scores of retrieved results. Term weighting is responsible for computing the relevance scores and consequently differentiating between the terms in a document. However, the current term weighting formula (TF-IDF, for instance), weighs terms only based on term frequency and inverse document frequency irrespective of other important factors. This results in ambiguity in cases when both TF and IDF values the same for more than one document, hence resulting in same TF-IDF values. In this paper, we propose a modification of TF-IDF and other term-weighting schemes that weighs the terms based on the recency and the usage in the corpus. We have tested the performance of our algorithm with existing term weighting schemes; TF-IDF, BM25 and USE text embedding model. We have indexed three different datasets with different domains to validate the premises for our algorithm. On evaluating the algorithms using Precision, Recall, F1 score, and NDCG, we found that time normalized TF-IDF outperformed the classic TF-IDF with a significant difference in all the metrics and datasets. Time-based USE model performed better than the standard USE model in two out of three datasets. But the time-based BM25 model did not perform well in some of the input queries as compared to standard BM25 model.

## introduction term weighting is one of the most crucial tasks in information retrieval and recommender systems. it is method of quantifying terms in a document to determine the importance of the words in the document and the corpus @xcite . apart from recommendation engine and information retrieval, term weighting is effective in many scenarios such as text mining, text classification, duplicate image detection @xcite , document clustering, and even in medical science research. in text categorization and data mining, efficient. term weighting brings a considerable boost in effectiveness @xcite . several term weighting approaches are used in different applications basically derived from the frequency and distribution of words in documents @xcite . tf-idf is one of the classic term weighting approaches, that is most frequently used and was found to be used, for instance, by 83% of textbased research paper recommender systems @xcite . tf-idf as the name suggests, is made up of two parts, term frequency (tf) and inverse document frequency (idf). tf gives the number of times a term occurs in a document. the basis is that the more frequently a term occurs, the more it is important for the context of the document @xcite . idf is computed as the inverse frequency of documents containing the searched term. the idea behind this is that a rare term should be given higher importance as compared to frequently occurring terms such as articles, pronouns, etc. there have been numerous researches on tf-idf, and many extensions and alternatives are suggested. some other term weighting models used are bm25, lm dirichlet, divergence from independence, etc. text and sentence embedding models such as universal sentence encoder (use) @xcite ), google's bert, infersent, etc are also used in text classification tasks. these different approaches depend on the type, size of corpus, types of queries, and they use different term metrics to determine the effectiveness of term in a document and corpus. in case of information retrieval task, there are certain limitations in standard term weighting approaches. analyzing the simple approach of tf-idf, that weights term based on the frequency distribution in the corpus. the real issue in this method is the assumption that frequency distribution remains constant with time, without contemplating the diverse contexts for different terms. in short periods, this holds, however, over longer time this assumption fails. for example, consider two terms, "covid19" and "neural networks", that have different origin years. now, there are probably fewer documents containing the term "covid19" than documents containing the term "neural networks", simply because "covid19" is a relatively new term, while "neural networks" is a term being used since decades. however, they would be weighted similarly without considering the difference in the origins. the issue that terms have temporal distributions of frequency, not just space distribution is unaccounted when using the standard term weighting methodologies. considering this uncertainty in term weighting, we suggest a time-normalized term weighting approach, which reflects the age of a term. as the vocabulary changes over time, our intuition is to identify a term's age based on its first usage and current year and distinguish between the documents based on the age of the terms used. hence, we propose to weigh terms not only on their frequency distributions but also temporal distributions. furthermore, we demonstrate the significance of adding a time-based feature by comparing our method with state-of-the-art baseline models, that is, tf-idf, bm25 and use embedding. experimental results show substantial improvements over the baseline models for similar recommendations. in another paper, @xcite ) points out the problem of using idf in text classification. the basic idea behind idf is that a term occurring frequently has negligible distinguishing power, however, in the case of text classification, this might not be true, because, highly frequent terms in different documents of the same category can be helpful in text classification. hence, the authors suggested a supervised learning approach to calculate idf excluding the category under consideration. @xcite suggests a novel approach to term weighting based on the term positions along with the tf and idf terms. the authors studied the term patterns that occur in the documents using the wavelet transform method. the paper also suggests that the documents are ranked more relevant if the query terms are close to each other. utilizing temporal feature has also proved to be an efficient way for recommenders and time normalized recommendations are certainly receiving growing application in recent times @xcite ## time normalized term weighting in a classic term weighting approach, the terms are weighted irrespective of the different contexts or usage or recency. in this paper, we try to emphasize on the importance of term recency in relevant results retrieval. the premise for this algorithm is that, if a term is devised newly then there are probably a lesser number of documents containing the term compared to the term which is being used for a longer duration of time. in this algorithm, we introduce a time factor along with the regular tf-idf values. this time-based factor is formulated from the origin year of the word and the document frequency of the term giving the metric as documents per year. for a given term w in document d d, where d is the document corpus d with size n , term-age is calculated as: now, assume that a term is new and occurs in reasonable number of documents, then the value of t ( w, d) will be large and hence the term weight will be large. similarly, if the term is being used for many years and is occurring in many documents, it will relatively reduce the value of the time-factor, thus giving it low importance. a caution which needs to be taken while implementing this algorithm is to check for more commonly occurring non relevant terms which are normalized by using idf should not get boosted. this can be taken care of while calculating the value of y origin , and such terms can be ignored so they don't boost up the term weights based on non-relevant terms. this collection contains 608,180 news articles and blog posts, along with 50 queries from trec -2018 news background linking task @xcite , and expected set of results. for the purpose of testing our hypothesis, we use a sample of 20909 documents with approximately 2400 relevant documents. however, this has been done only for time-based index due to scalability and resource constraints. and the term age is still calculated considering the entire corpus and does not affect the algorithmic logic. the relevant fields in this dataset are id, url, title, author, and article text. ## conclusion in this paper, we proposed a novel algorithm for term weighting. the presented approach shows the significance of temporal distribution along with existing space distribution of terms. we suggest the scheming of a term recency parameter based on the origin of the word and the usage in the document corpus. this factor is used along with the standard weighting values (such as tf and idf) for relevance scoring in the information retrieval system. the algorithm is tested on a news dataset, with queries trying to find links with the documents, web answer retrieval dataset and research papers citations dataset. we have also extended the algorithm to other text embedding models that are bm25 and use. experiments conducted on the ir system show that term-recency based tf-idf and tuse model outperforms the classic tf-idf and classic use algorithms with a significant margin when measured in terms of average precision, recall, f1 and ndcg. it has set up a strong premise for our ongoing research on ways to improve recommendation effectiveness. future works for this can be to find ways to improve the time-based bm25 model and testing the algorithm's performance in other tasks such as text classification, and user modelling. furthermore, we also plan to test different normalization factors for calculating the term age and then using it in the scoring algorithm.