Backtesting Sentiment Signals for Trading: Evaluating the Viability of Alpha Generation from Sentiment Analysis (2025)

Abstract: Sentiment analysis, widely used in product reviews, also impacts financial markets by influencing asset prices through microblogs and news articles. Despite research in sentiment-driven finance, many studies focus on sentence-level classification, overlooking its practical application in trading. This study bridges that gap by evaluating sentiment-based trading strategies for generating positive alpha. We conduct a backtesting analysis using sentiment predictions from three models (two classification and one regression) applied to news articles on Dow Jones 30 stocks, comparing them to the benchmark Buy&Hold strategy. Results show all models produced positive returns, with the regression model achieving the highest return of 50.63% over 28 months, outperforming the benchmark Buy&Hold strategy. This highlights the potential of sentiment in enhancing investment strategies and financial decision-making.

## introduction sentiment analysis is a natural language processing task that has gained significant attention in recent years, with application domains such as product reviews, brand perception, and public sentiment towards political events or social issues. however, the influence of sentiments and opinions is not limited to consumer preferences and public discourse, reaching into the complex dynamics of financial markets. a growing body of research @xcite shows that sentiments and opinions play a pivotal role in shaping the behaviors and outcomes of market entities. sentiments and opinions expressed in financial microblogs and news articles can significantly impact market dynamics, often leading to abrupt fluctuations in asset prices. recent works address the challenge of sentiment analysis of financial microblogs and news to predict market reactions @xcite . therefore, integrating sentiment analysis into financial market analysis can potentially improve investment strategies and risk management. at the same time, it is essential to acknowledge that news outlets can also introduce distortions, resulting in producers' and consumers' biases, noisy information, and speculative sentiments @xcite . most of the literature in finance focuses on sentiment analysis at the level of sentences or news articles @xcite . sentiment analysis can be conceptually framed as either a classification or regression task, each offering varying degrees of granularity in sentiment interpretation. few works in the existing literature make a critical evaluation of the practical applicability of sentiment analysis in financial trading @xcite . an effective method to assess the impact of sentiment analysis on financial markets is to develop trading strategies exclusively based on sentiment data. these strategies operate according to specific rules to determine the optimal times to execute buy or sell orders for an asset. by analyzing the outcomes of these sentiment-driven trading strategies, researchers can evaluate whether the sentiment data provided by a model contributes to generating profits or incurs losses. such evaluations are instrumental in understanding the real-world practicality and effectiveness of sentiment analysis in financial trading. despite recent advances in sentiment model prediction, a persistent challenge remains in evaluating the practical efficacy of the predicted sentiment scores. although previous studies have focused mainly on assessing the accuracy of a model in predicting sentiment compared to human evaluations @xcite or combining sentiment/emotion with macroeconomic and technical data to forecast price returns @xcite , our study adopts a distinctive approach. in this work, our objective is to comprehensively evaluate the effectiveness of solo sentiment analysis strategies by examining their ability to produce actionable sentiment scores that inform trading strategies. this includes assessing their impact on trading performance and determining whether they can generate a positive alpha. financial backtesting is a critical tool that simulates how a particular strategy would have performed in historical market conditions. this approach offers valuable insights into risk management, profitability, and overall strategy viability. therefore, we propose a comparative backtesting trading strategy analysis that leverages sentiment predictions from three distinct sentiment recognition models. these models, which differ in their complexity, are used to forecast daily price return variations. we compared these methods with a benchmark strategy consisting of a buy&hold approach for the assets over the backtesting period. the backtesting involved determining daily sentiment trends from news articles published between january 1, 2020, and april 30, 2022, focusing on content relevant to the dow jones 30 stocks and generating daily signals to buy or sell an asset. in our analysis, all sentiment models generated positive cumulative returns at the end of the backtesting period. as expected, the regression model produced the highest cumulative return of 50.63%, which is 87.8% higher than the returns from the classification methods and the benchmark, thus generating a considerable alpha of 23.67%.this outcome highlights the potential of sentiment analysis-based trading systems as compelling investment strategies in financial markets. moreover, it highlights their versatility and potential for integration with additional information, such as macroeconomic indicators and technical data, to further enhance trading performance. the rest of this paper is structured as follows : section 2 reviews related work on the application of sentiment analysis in backtesting and its impact on financial markets. section 3 describes our backtesting procedure, including details on data collection, selected sentiment models, sentiment aggregation, and trading decision processes. section 4 presents the experiments conducted and discusses the results of our analysis. finally, section 5 concludes the paper with key insights and final remarks on our case study. ## related work sentiment analysis has been widely applied across diverse domains, including assessing product quality via user feedback or measuring public sentiment about a stock on social networks. however, relatively few studies have explored the application of sentiment analysis specifically in the context of trading evaluation. @xcite incorporated sentiment into a trading strategy using an svm-based sentiment classifier. their approach involved acquiring stocks when news articles were classified as having a positive sentiment and selling them when sentiment was negative. while their method demonstrated a strong correlation between sentiment signals and trading actions, it was sensitive to daily variations, mainly when multiple articles related to the same company were analyzed on the same day. lou (2023) extended this idea by examining the impact of human emotions on quantitative trading. by fine-tuning the bert model on user comments from east money and integrating it with the alpha191 model, lou achieved a 73.8% improvement in return rates compared to a baseline strategy. bird et al. (2023) further explored the integration of emotions into trading strategies, focusing on the timing of stock purchases, short sales, and reversals. their approach involved acquiring stocks shortly after exceptionally positive earnings news when positive emotions were subdued and reversing the position based on either an increase in positive emotions or the end of the holding period. for short positions, they employed a similar method following extremely negative earnings news. their findings underscored the significant influence of investor emotions on individual investment decisions, suggesting that price distortions are corrected only when emotions shift. @xcite analyzed the relationship between stock price movements and social media sentiment in china, leveraging data from platforms such as microblogs, chat rooms, and forums. their study revealed a strong correlation between chat room sentiment and stock price fluctuations. by developing a trading strategy based on chat room sentiment, they achieved a portfolio return of 19.54% over seven months, significantly outperforming a passive buy-and-hold approach. @xcite presented a trading strategy combining large language models (llms) with macroeconomic and technical indicators to enhance stock return predictions, focusing on smallcap stocks from the russell 2000 index. by incorporating sentiment analysis from financial news using gpt-4 and applying a decay function for news impact, their strategy achieved exceptional performance, with sharpe ratios of 3.64 and 5.10 in 2022 and 2023, respectively, even after accounting for transaction costs. focused on the bias in the backtesting, @xcite examined backtesting trading strategies based on sentiment derived from llms. they identified two key biases-look-ahead and distraction effects-and demonstrated that anonymizing financial news headlines by removing company identifiers improved trading performance. their findings suggest that the distraction effect has a more significant impact than look-ahead bias, particularly for larger companies, and their proposed anonymization technique offers a novel method for reducing bias in backtesting. our work focuses on developing a trading system based solely on sentiment analysis over an extended period of more than two years. while bird et al. (2023) focused on the emotions of earning news to generate their signals, we evaluate three distinct sentiment approaches to create trading systems that derive daily trading signals from major financial news providers (all types of news) on the dow jones 30 stocks. the following section outlines the backtesting methodology and protocol used in our study. ## conclusion in this paper, we proposed an evaluation framework based on financial backtesting to measure the effectiveness of incorporating sentiment scores predicted by machine learning approaches from news articles into a trading strategy. the backtesting from january 1, 2020, to april 30, 2022, demonstrates that utilizing news sentiment significantly contributes to creating a profitable trading strategy with relatively low risk. furthermore, we observed that, opposed to discrete classes, models predicting sentiment as floating-point values result in improved performance. the roberta+transfs approach achieved the highest cumulative return 50.63% which is 87.8% higher than all other methods analyzed in this study. the significant gap between finbert and roberta+transfs is primarily attributed to finbert predicting only three possible classes, whereas roberta+transfs can estimate various degrees of positivity or negativity in an article. moreover, the floating-based model generated a considerable alpha of 23,67% with a lower risk than the benchmark (buy&hold). future work will aim to conduct a more comprehensive comparison of trading strategies driven exclusively by news sentiment against hybrid approaches that integrate sentiment analysis with technical indicators and macroeconomic variables. the objective is to enhance the precision and effectiveness of daily order execution. this analysis will be strengthened by extending the backtesting period to cover at least 5 years, providing a more robust assessment of strategy performance across varying market cycles, including bull and bear phases. the analysis will also be expanded to include major indices such as the s&p 500 and nasdaq-100 to evaluate strategy robustness across different market environments.