Ask Asper at the Financial Misinformation Detection Challenge Task: Enhancing Financial Decision-Making: A Dual Approach Using Explainable LLM s for Misinformation Detection (2025)

Abstract: The integrity of the market and investor con- fidence are seriously threatened by the prolif- eration of financial misinformation via digital media. Existing approaches such as fact check, lineage detection and others have demonstrated significant progress in detecting financial mis- information. In this paper, we present a novel two-stage framework leveraging large language models (LLMs) to identify and explain finan- cial misinformation. The framework first em- ploys a GPT-4 model fine-tuned on financial datasets to classify claims as “True,” “False,” or “Not Enough Information” by analyzing rel- evant financial context. To enhance classifi- cation reliability, a second LLM serves as a verification layer, examining and refining the initial model’s predictions. This dual-model approach ensures greater accuracy in misinfor- mation detection through cross-validation. Beyond classification, our methodology empha- sizes generating clear, concise, and actionable explanations that enable users to understand the reasoning behind each determination. By com- bining robust misinformation detection with interpretability, our paradigm advances AI sys- tem transparency and accountability, providing valuable support to investors, regulators, and financial stakeholders in mitigating misinfor- mation risks.

## introduction the integrity of financial markets faces an unprecedented challenge from the proliferation of misinformation, which fundamentally undermines investor trust and threatens economic stability. financial misinformation, a particularly harmful subset of deceptive content, can significantly distort investor behavior, market perspectives, and lead to suboptimal financial decisions. this phenomenon manifests in various forms, from fraudulent financial statements to misleading investment advice, carrying severe implications for both individual and institutional stakeholders @xcite . the exponential growth of digital platforms facilitating real-time financial transactions has amplified the impact of such misinformation, necessitating robust detection and mitigation strategies @xcite . while existing frameworks primarily focus on identifying fraudulent claims, they often lack the transparency necessary to establish user trust. the emergence of advanced artificial intelligence (ai) models, particularly large language models (llms), presents promising avenues for detecting and understanding financial misinformation. however, the integration of these technologies with practical financial applications remains an underexplored area, especially concerning explainability and reliability. this research introduces a novel two-stage methodology that leverages llms, enhanced through financial dataset fine-tuning, to classify financial assertions into three categories ("true," "false," or "not enough information") while providing concise, comprehensible explanations for these classifications. our approach implements a refined gpt-4 model that evaluates the context of financial claims and predicts their veracity, followed by a secondary llm serving as a "judge" to review and refine initial classifications. this duallayer verification mechanism enhances reliability in the decision-making process through improved accuracy and comprehensibility @xcite . the next section focuses on the related prior work. in section 3, we will discuss the proposed architecture, its working, and its advantages. section 4 will provide an in-depth explanation of the experimental setup and evaluation methodology. following this, section 5 will present the results of our experiments, accompanied by a detailed analysis. finally, we conclude the paper in section 6, outlining the future work planned to extend this research. ## literature review deep learning and natural language processing (nlp) techniques have gained significant attention in detecting financial disinformation and fake news. numerous models have been proposed, each with unique strengths and limitations. fnfnet @xcite employs convolutional neural networks (cnns) for extracting information from news articles, achieving a remarkable accuracy of 98. @xcite fmdllama @xcite , built on llama3.1 and utilizing the financial misinformation detection instruction dataset (fmdid), excels in multitask learning for classification and explanation generation. despite its promise, its effectiveness is constrained by limited dataset diversity and the absence of real-world evaluation benchmarks. traditional machine learning methods have evolved into deep learning-based approaches like cnns and lstms, which improve classification precision through automated feature extraction @xcite . however, these methods often rely heavily on specific datasets, lack generalizability, and require multimodal data integration. finbert @xcite , a domainadaptive language model trained on financial texts, captures financial terminology and sentiment effectively. nonetheless, it struggles to keep up with changing market conditions and financial jargon, underscoring the need for continuous updates. dfdr @xcite takes a multimodal approach by integrating textual analysis with market data, including trading volumes and real-time signals. while this enhances detection capabilities, it encounters challenges like high computational costs and difficulties in maintaining real-time performance. the temporal-aware language model @xcite focuses on handling timesensitive financial data by incorporating temporal dependencies and market dynamics. despite its strength in timely detection, it faces resource constraints and struggles with long-term dependency modeling. crossfin @xcite unifies data from diverse sources, including social media, news platforms, and financial streams, enabling effective cross-platform detection. however, its performance consistency across platforms remains a challenge. finally, fingpt @xcite , an open-source financial language model with specialized pre-training, demonstrates a strong ability to understand complex financial narratives. its main drawbacks include slower inference speeds, optimization issues for model size, and challenges in adapting to rapidly changing market trends. while models like fnfnet, fmdllama, and finbert have advanced financial misinformation detection, significant gaps remain. these include the need for integrated multimodal approaches, better interpretability, robust benchmarks, and solutions for overfitting and dataset limitations. figure 1 demonstrates the logical architecture of our suggested two-step framework. ## proposed architecture in the first step, we categorize financial claims as true, false, or not enough information using a fine-tuned gpt-4 model specifically trained on financial data. this model leverages domain-specific financial traits and contextual knowledge, which are provided in the dataset, including claims, justifications, issues, evidence, image urls, and image content. by incorporating these elements, the model ensures that the classification aligns with accepted financial logic and principles. to further enhance the reliability and accuracy of the initial classification, we introduce a second layer of verification using the "llm as a judge" technique @xcite . in this stage, a second instance of gpt-4 serves as an impartial arbiter to assess the accuracy of the first model's predictions. this llm evaluates the classification's justification, compares it to pertinent financial information, and renders an assessment of the classification's accuracy. if any discrepancies are found, the judge updates the prognosis, providing a thorough justification for the change. this ensures that the final classification benefits from both increased accuracy and a clear, intelligible explanation. ## experimentation and evaluation the dataset was divided into training and validation sets using an 80-20 split, ensuring stratified sampling to preserve the class distribution across true, false, and nei labels. this resulted in 1500 samples for training and 443 for validation. to ensure robust model evaluation, we implemented 5-fold cross-validation, providing insights into performance across different data splits. seven experiments were conducted to evaluate strategies for fine-tuning and prompt engineering, adapting a large language model (llm) to the tasks of verifying financial claims and generating explanations (cf. experiment 1 used only prompt engineering without modifying the base model. while this approach achieved a high overall score (0.8348), task-specific metrics like f1 micro (0.2247) and rouge1 (0.2225) were low, indicating limitations in aligning the llm's reasoning with the problem domain. experiment 2 fine-tuned the base model using gpt-4o. this reduced the overall score to 0.5804 but significantly improved f1 micro to 0.8706, suggesting enhanced claim categorization. however, experiment 3 applied a single-layer prompting strategy with the fine-tuned model, yielding balanced improvements in rouge metrics (rouge1: 0.3267) and an overall score of 0.6033. experiment 4 introduced two-layer prompting, structuring intermediate reasoning steps to align better with task objectives. this approach improved rouge1 (0.4948) and rouge2 (0.4771), with an overall score of 0.6873. experiment 5 enhanced the fine-tuned model by incorporating synonym retrieval and lemmatization. this further improved rouge1 (0.5059) and stabilized the overall score at 0.6929. experiment 6 achieved the best task-specific performance by systematically improving the prompt template. this experiment recorded the highest overall score (0.6974) and rouge1 (0.5149), demonstrating the importance of refined prompt engineering. experiment 7 utilized a two-step framework. in the first step, multimodal attributes were added by extracting image content and url summaries using ai tools, which were then used to retrain the model. in the second step, a different model reviewed and updated the explanations and labels from the first step. this approach achieved a high f1 score ( 0.90), highlighting the effectiveness of integrating multimodal data into the evaluation process. these findings underscore the importance of harmonizing task-specific fine-tuning with iterative prompt design to achieve robust performance in both claim classification and explanation generation. ## conclusion this paper introduces a novel two-step framework for detecting financial misinformation, effectively combining the strengths of fine-tuned large language models (llms) with explainable ai principles. our approach achieves state-of-the-art per-formance with an f1 score of 0.90, while ensuring transparency through detailed explanations of its decision-making process. the dual-layer verification system, which includes an llm judge, significantly enhances the reliability of classifications and provides clear, actionable insights for financial stakeholders. our findings demonstrate that combining sophisticated prompt engineering with targeted finetuning yields superior performance compared to using either approach alone. additionally, integrating multimodal attributes in the final experiment further improved the model's ability to accurately contextualize and verify financial claims.