Query-Based Summarization and Sentiment Analysis for I ndian Financial Text by leveraging Dense Passage Retriever, R o BERT a, and F in BERT (2023)

Abstract: With the ever-expanding pool of information accessible on the Internet, it has become increasingly challenging for readers to sift through voluminous data and derive meaningful insights. This is particularly noteworthy and critical in the context of documents such as financial reports and large-scale media reports. In the realm of finance, documents are typically lengthy and comprise numerical values. This research delves into the extraction of insights through text summaries from financial data, based on the user’s interests, and the identification of clues from these insights. This research presents a straightforward, allencompassing framework for conducting querybased summarization of financial documents, as well as analyzing the sentiment of the summary. The system’s performance is evaluated using benchmarked metrics, and it is compared to State-of-The-Art (SoTA) algorithms. Extensive experimentation indicates that the proposed system surpasses existing pre-trained language models.

## introduction in recent years, the volume of data that we handle has increased exponentially, with vast quantities of textual data at our disposal. it can be a daunting task for a user to sift through such a significant amount of text in order to gain valuable insights. the process of reading through such extensive amounts of text places a considerable strain on short-term memory, ultimately leading to unfavorable outcomes. text summarization is an example of a natural language processing (nlp) task that entails producing a condensed version of a text while retaining the essential information. there are two primary methods for summarizing text: extractive summarization and abstractive summarization. extractive summarization entails selecting relevant sentences from the source text and combin-ing them to form a summary. conversely, abstractive summarization involves creating new sentences that capture the essential points of the original text. the financial services sector is expected to undergo a compound annual growth rate of 7.4% by 2026, which is twice the rate of growth that has been observed in the previous three years foot_0 . as a result, experts highly recommend utilizing artificial intelligence to improve various aspects of financial services. the finance industry generates and consumes vast amounts of information in the form of finance blogs/articles, reports, and other types of textual data. the major challenges in financial analytics include an extensive vocabulary, a large quantity of noisy unstructured data, and a robust model. @xcite in their research, showed that financial text analysis plays a critical role in making market decisions. @xcite revealed that automatic summarization systems considerably enhance employees' workflow in media monitoring and media response analysis through reduced processing time. this observation can be applied to the financial sector, where the proposed system caters to the needs of thousands of users dealing with a vast amount of textual data available on the internet. the system generates short summaries of text in 2-3 sentences, making it easier for users to read particular financial text documents. furthermore, with an available sentiment score, users can make more informed decisions quickly, improving efficiency and yielding better results. this sentiment analysis approach has been demonstrated to improve the performance of recommender systems in previous studies @xcite . in this research, our primary focus has been on the summarization and sentiment analysis of financial data. in order to achieve effective summarization of financial text documents, we have proposed a specialized system. our model relies on data collected from diverse sources, including news and official reports. given the voluminous and complex nature of financial documents, it is crucial to extract pertinent information by employing a query-based approach. this method allows for the retrieval of relevant documents based on user requirements and facilitates the generation of summaries. sentiment analysis refers to the process of extracting subjective information, such as opinions, emotions, and beliefs, from text. it is an incredibly powerful tool that can provide valuable insights into people's thoughts and feelings about a particular topic. in the financial @xcite and cryptocurrency @xcite trading sectors, sentiment analysis is used to track investor sentiment, identify emerging trends, and make informed investment decisions. many researchers have found that sentiment analysis is a key feature in stock price prediction, which has proven to be highly successful. according to the research conducted by @xcite , market sentiment plays a significant role in predicting stock prices, and this is why stock market prediction is so beneficial. it is widely acknowledged that carrying out effective and precise sentiment analysis for financial data can be an arduous undertaking. to address this challenge, we have put forward the implementation of a model that has undergone pre-training on vast quantities of financial data finbert @xcite . this has enabled us to generate precise sentiment scores for the summaries produced by our model. the primary contribution of our research can be outlined as follows 1. real-time financial data is acquired from the internet, and we construct an information retrieval system using dense passage retriever @xcite . to achieve efficient and prompt document retrieval, we have utilized the dense passage retriever (dpr) technique. dpr is capable of representing queries and documents through dense vectors, which have proven effective in capturing the semantic meaning of the text. moreover, dpr comes pre-trained on a vast corpus of textual data, simplifying the process of fine-tuning and enhancing the model's efficiency as opposed to creating a new one from scratch for every new task. query and document indexing are employed by dpr to store the dense representations of the documents, leading to swift retrieval of relevant documents. for indexing these representations, we have opted for the faiss (facebook ai similarity search) @xcite document store. for document summarization, we have utilized the roberta model, which is based on bert. bert based models are pre-trained on textual data, making them easy to fine-tune. additionally, roberta greatly enhances bert's performance by utilizing a larger corpus of pre-training data and implementing other model improvements. for sentiment analysis, we have employed finbert, which has already undergone extensive training on a vast amount of financial text, specifically fine-tuned for sentiment analysis on our dataset. the structure of the research paper is such that section 2 encompasses a discussion on related work pertaining to financial data summarization and sentiment analysis. moving on, section 3 delves into the proposed work, which entails a methodology for the collection of data, document indexing, data query processing, search, and sentiment analysis. the outcomes of the research are expounded upon in section 4, while section 5 provides a summary of the work, along with the conclusion and future prospects. ## literature review in this section, we have briefly outlined other approaches that have been used for extractive text summarization and sentiment analysis. one of the simplest techniques used for generating a summary for a given text is the term frequency-inverse document frequency (tf-idf) technique. @xcite in their research, showed that, tf-idf can be used to determine the relevance of words in documents. the words can then be ranked according to their tf-idf score and used to generate a summary for the given text. qaiser and ali (2018) have further explored the strengths and weaknesses of tf-idf. the work of @xcite was extended by @xcite through the addition of further features to their paper. the initial work involved the extraction of various features from a text document, combining them to generate a sentence score. the resulting sentences were then ranked according to their score, allowing for the creation of a summary specific to the given document. to create a multi-document summary, individual text summaries were merged based on cosine similarity. subsequently, the top n sentences are combined to form a summary. @xcite extracted word and sentence features in their system and employed the genetic algorithm to assign weights to each feature. the evaluation of each generation is carried out using rouge parameters, and fuzzy logic is then used to generate summaries. @xcite proposed a graph-clustering-based framework for generating financial news summaries. their model incorporates a graph structure that considers crosssentence relations and text embeddings to enhance representation and emphasize significant sentences. @xcite presented a system that comprises an encoder, an attention level decoder, and a copy mechanism, with the encoder employing a gru model. @xcite presented an extensive survey on sentiment analysis and opinion mining utilizing conventional machine learning techniques as well as cross-domain and crosslingual methods. they discovered that approaches like svm and naïve bayes exhibit the highest level of accuracy. additionally, they noted that combining ml models with the opinion lexicon method notably enhances the accuracy of sentiment classification. @xcite presented two approaches to summarize financial text: abstractive and extractive. the initial phase of the model identifies the commencement of the continuous narrative section in the document. subsequently, the model extracts the most significant sentences from the continuous narrative section. to identify the beginning of the continuous narrative section, the first stage of the model employs a similarity search. in contrast, the second stage of the model employs a range of features to extract the most important sentences. these features encompass the sentence's position in the document, length, topic, and sentiment. despite this, the model achieved a rouge score of below 0.5 on all rouge-n parameters. @xcite pre-sented two approaches for summarization, namely sentence-based and section-based summarization. the first approach involves the use of bert for embedding and classification, which resulted in superior performance compared to the baseline models. as for the second approach, bert was utilized for vector embeddings and weighted-section-clustering for classification, achieving a rouge-l f1 score metric of 0.31. kraaijeveld and de smedt (2020) in their research, explored the predictive power of public twitter sentiment in forecasting cryptocurrency prices. the authors utilize a dataset of more than 100 million tweets discussing bitcoin and ethereum, and they apply a range of sentiment analysis techniques to extract sentiment from the tweets. these techniques include lexicon-based sentiment analysis, machine learning-based sentiment analysis, and a hybrid approach that combines both methods. the authors' findings indicate that public twitter sentiment can serve as a valuable resource for investors seeking to make informed investment decisions. the study conducted by @xcite employs sentiment analysis in forecasting cryptocurrency prices via market tweets sentiment. a dataset of historical tweets was utilized to evaluate the proposed system, which was able to accurately predict the price of cryptocurrencies. the models examined were text blob and cnn-lstm, with the latter yielding a higher degree of accuracy. this literature survey highlights different techniques for extractive text summarization, including tf-idf, feature extraction with ranking, graphbased approaches, and bert-based models. it also emphasizes the value of sentiment analysis in forecasting cryptocurrency prices using twitter data. bert-based models have shown superior performance compared to other methods. however, feature extraction and sentence ranking continue to be widely employed for extractive summarization, with efforts to address ambiguity and redundancy in sentences to enhance performance. ## proposed work our model, as depicted in figure 1 , offers financial text summarization and sentiment analysis through a system of four distinct modules. 1. data collection module 2. document indexing module 3. data query processing and retrieval (search) module 4. sentiment analysis module ## conclusions in this research, we have presented a workflow for summarizing financial news and documents. we got the financial news and texts from the internet, specifically from sources/aggregators. we used dpr for document retrieval and roberta to create concise and accurate summaries based on user queries. furthermore, we also added sentiment analysis using finbert to provide sentiment scores, analyzing financial text sentiment allows for identification of overall market sentiment, positive/negative trends, and impact of specific events on sentiment, aiding investors and financial professionals in making informed decisions. we utilized various metrics, such as rouge and bleu scores, to gauge the caliber of the generated summaries in contrast to the ground truth summaries created by experts. the findings exhibited that our approach consistently surpassed the baseline methods, indicating its proficiency in capturing pertinent information and creating concise and precise summaries. to assess the sentiment analysis module, we juxtaposed the accuracy, precision, and f1 scores of finbert with those of other sota sentiment analysis models, thus illustrating that our model outperforms them. in conclusion, we have conducted research on query-based summarization for financial news and documents that demonstrates a comprehensive workflow utilizing cutting-edge techniques such as dpr, roberta, and finbert. through the integration of these approaches, we have created a robust system capable of extracting pertinent summaries that are personalized to user queries, while additionally conducting sentiment analysis to capture the overall market mood.