Team FMD LLM at the Financial Misinformation Detection Challenge Task: Exploring Task Structuring and Metadata Impact on Performance (2025)

Abstract: The detection of financial misinformation (FMD) is a growing challenge. In this paper, we investigate how task structuring and metadata integration impact the performance of large language models (LLMs) on FMD tasks. We compare two approaches: predicting the label before generating an explanation, and generating the explanation first. Our results reveal that prediction-first models achieve higher F1 scores. We also assess the effect of auxiliary metadata, which surprisingly degraded performance despite its correlation with the labels. Our findings highlight the importance of task order and the need to carefully consider whether to use metadata in limited data settings.

## introduction recently, large language models (llms) @xcite @xcite @xcite has been transforming finance sectors with their adaptation @xcite @xcite . at the same time, there is a growing need to automate the detection of misinformation in finance, where misinformation can lead to market manipulation and instability @xcite @xcite . in this paper, we present our approach to the financial misinformation detection (fmd) shared task at coling 2025, where we developed models capable of both classifying financial claims and generating explanations for the predictions. our experiments revealed two key insights: (1) classifying claim labels prior to generating explanations significantly improved classification performance in f1 score, challenging the common practice of generating reasoning as a precursor to prediction, such as in chain of thought prompting; and (2) incorporating auxiliary metadata, such as summary fields, unexpectedly degraded model performance, despite the strong correlation of this metadata with the labels. this finding challenges conventional assumptions about feature engineering, in tasks requiring nuanced reasoning with limited data. ## related studies the growing interest in fact-checking spans various domains, from addressing misinformation related to covid-19 @xcite , to verifying health-related claims @xcite , to checking scientific assertions @xcite , and even to creating large-scale, multi-domain datasets such as fever @xcite . in the financial domain, the detection of misinformation has emerged as an important focus. for example, @xcite introduced the fin-fact dataset, specifically designed to address the gap in domain-specific fact-checking resources for financial misinformation. earlier research in financial misinformation detection primarily utilized traditional nlp techniques, including roberta @xcite , lstm-based models, and custom neural architectures @xcite @xcite . with increasing evaluations of llms in fields like the legal domain @xcite , there is a growing need for similar assessments in financial misinformation detection. recent advancements, particularly the work by @xcite leveraged domain-specific fine-tuning for llms. their fine-tuned version of llama3.1-8b foot_0 outperformed leading zero-shot models, such as mistral-7b-instruct (jiang et al., 2023) and gemma-instruct-7b (mesnard et al., 2024), highlighting the benefits of fine-tuning llms over general-purpose models in financial misinformation detection. ## approach our approach optimized financial misinformation detection by developing prompts tailored to two key factors: (1) subtask order, comparing whether classifying a financial claim (true/false/nei) before generating an explanation yields better performance than the reverse, and (2) the potential benefits of leveraging auxiliary metadata, particularly the availability of sci_digest field, which showed strong label correlations. ## results the impact of task order (classification prediction before explanation vs. explanation before classification prediction) and the inclusion of metadata on model performance. our findings indicate that models predicting the label before generating an explanation achieve higher f1 scores. prediction first without metadata (micro-f1: 0.7357) performed better than explanation first without metadata (micro-f1: 0.6063) by 0.1294. additionally, prediction first with metadata (micro-f1: 0.6969) performed better than explanation first with metadata (micro-f1: 0.4972) by 0.1997. this supports the hypothesis that beginning with the more constrained task of classification leads to better overall performance in financial misinformation detection. including whether sci_digest is empty (metadata) consistently lowered f1 scores, suggesting that while metadata correlates with labels, it may hinder model performance. specifically, the inclusion of metadata reduced the f1 score by 0.0388 in the prediction first approach and by 0.1091 in the explanation first approach. this implies that metadata may need to offer more than surface-level correlations to be effective in enhancing the model's reasoning process ## conclusion our results demonstrate that predicting the label before generating an explanation improves classification performance in financial misinformation detection, as evidenced by f1 score. this contrasts with conventional approaches that prioritize reasoning-first strategies. additionally, the inclusion of auxiliary metadata, such as the sci_digest field, despite its high correlation with the labels, hindered model performance. this finding challenges conventional assumptions regarding the benefits of metadata for prediction tasks, especially in cases where the metadata lacks semantic richness.