Capybara at the Financial Misinformation Detection Challenge Task: Chain-of-Thought Enhanced Financial Misinformation Detection (2025)

Abstract: Financial misinformation poses a significant threat to investment decisions and market stability. Recently, the application of Large Language Models (LLMs) for detecting financial misinformation has gained considerable attention within the natural language processing (NLP) community. The Financial Misinformation Detection (FMD) challenge @ Coling 2025 serves as a valuable platform for collaboration and innovation. This paper presents our solution to FMD challenge. Our approach involves using search engines to retrieve the summarized high-quality information as supporting evidence and designing a financial domain-specific chain-of-thought to enhance the reasoning capabilities of LLMs. We evaluated our method on both commercial closed-source LLMs (GPT-family) and open-source models (Llama-3.1-8B and QWen). The experimental results domonstrate that the proposed method improves veracity prediction performance. However, the quality of the generated explanations remains relatively poor. In the paper, we present the experimental findings and provides an in depth analysis of these results.

## introduction the proliferation of misinformation in the financial sector significantly impacts investor decisionmaking and market stability @xcite . manually verifying such financial misinformation demands substantial time and effort. consequently, the development of automated tools for detecting financial misinformation has become a critical area of research in fintech. previously, most frameworks for financial misinformation detection (fmd) relied on conventional deep learning approaches. for instance, @xcite developed a framework using roberta combined with a multi-channel network (cnn, bigru, and an attention layer) specifically for fmd task, while @xcite utilized multiple lstms to identify dynamic and covert patterns aiding in the detection process. recently, with the advent of large language models (llms), in response to the complexity of the financial context and the professionalism of financial information, fin-fact @xcite proposed a multimodal financial misinformation detection and interpretation generation dataset, and evaluated the capabilities of multiple popular llms on this dataset. furthermore, fmdllama @xcite has pioneered the use of open-source llms for identifying fraudulent financial information, setting a new benchmark in the field. despite these developments, the effectiveness of llms in fmd task warrants further exploration. the financial misinformation detection challenge @ coling 2025, as introduced by fmdllama @xcite , aims to explore the capabilities of llms in enhancing the accuracy of financial misinformation detection. this paper describes our technical solution for fmd challenge. the core idea of our solution is to involves enhancing the "justification" component of the dataset by retrieving summarized high-quality information from online as the extra evidence using search engines and designing a financial domainspecific chain-of-thought prompt to guide llm reasoning and explanation generation. we conducted experiments on both commercial closedsource models and open-source models. extensive evaluations on the fmd tasks yielded significant findings: (1) the proposed financial chain-of-thought prompt method effectively improves the pipeline's prediction results; and (2) despite this, the overall performance remains average. furthermore, the quality of the generated explanations is significantly inferior to that of the baseline method, which has undergone fine-tuning. this underscores the necessity of fine-tuning the model using highquality data. a more detailed analysis of the results is provided in section 4. ## methodology in this section, we outlines the proposed pipeline for financial misinformation detection. we integrate the retrieved summarized high-quality information with the original justification as whole support information and utilize a chain-of-thought prompt to enhance the prediction process (see in figure 1 ). ## conclusions and future work the paper presents a technical solution to the financial misinformation detection challenge, combining retrieved high-quality evidence with a financial chain-of-thought (cot) prompt to enhance prediction accuracy. however, the proposed pipeline demonstrates limitations in explanation quality compared to fine-tuned baselines. this emphasizes the necessity of incorporating fine-tuning steps to improve performance in future work.