Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance? (2025)

Abstract: Fact-checking pipelines increasingly adopt the Decompose-Then-Verify paradigm, where texts are broken down into smaller claims for individual verification and subsequently combined for a veracity decision. While decomposition is widely-adopted in such pipelines, its effects on final fact-checking performance remain underexplored. Some studies have reported improvements from decompostition, while others have observed performance declines, indicating its inconsistent impact. To date, no comprehensive analysis has been conducted to understand this variability. To address this gap, we present an in-depth analysis that explicitly examines the impact of decomposition on downstream verification performance. Through error case inspection and experiments, we introduce a categorization of decomposition errors and reveal a trade-off between accuracy gains and the noise introduced through decomposition. Our analysis provides new insights into understanding current system’s instability and offers guidance for future studies toward improving claim decomposition in fact-checking pipelines.

## introduction fact-checking is a critical task that typically involves evaluating the veracity of claims or reports. with the rise of large language models (llms), the scope of fact-checking task has expanded to include verifying content generated by llms @xcite . while progress has been made in reducing llm hallucinations, the problem has not yet been resolved, presenting ongoing challenges in ensuring the factuality of llm outputs @xcite . at the same time, many studies focus on llm-driven automated fact-checking pipelines @xcite @xcite , aiming to improve the efficiency of the fact-checking processes. a common design pattern in these llm-driven pipelines is the decompose-then-verify paradigm, as adopted in frameworks like factscore @xcite , factool @xcite and veriscore @xcite . it involves decomposing input text into sub-claims (decompose), retrieving supporting information (e.g., wikipedia, google search) for each sub-claim, and using a verifier model to assess the veracity of each sub-claim (verify). the results are then aggregated to produce a final verification. by breaking down complex inputs (decomposition), these pipelines are effective in pinpointing misinformation, enabling a nuanced determination of whether a text is supported, unsupported @xcite @xcite , or assigned a quantified score @xcite . however, most existing studies primarily focus on the design of fact-checking pipeline architectures. while decomposition is commonly employed in these fact-checking frameworks, the reliability of the decomposition process itself remains insufficiently investigated. some studies show that the final predictions of these pipelines are sensitive to the decomposition outcomes @xcite . additionally, it has been observed that a given decomposer does not consistently lead to performance improvements when paired with different verifiers. for example, @xcite report improvements using a pre-trained natural language inference (nli) model as the verifier. however, an ablation study by minicheck @xcite , which employs the same dataset and decomposition method but different nli models, does not observe consistent benefits from decomposition. similar inconsistencies are observed in felm @xcite , which assesses the effect of decomposing responses into segments and further into claims. the study reveals distinct differences between chatgpt (ope-nai, 2022) and @xcite as verifiers. figure 1 : an overview of the decompose-then-verify pipeline employed in this study, which comprises four key stages: decomposition, retrieval, verification, and aggregation of sub-claim results. this figure illustrates how different decomposition methods, such as factscore @xcite and veriscore @xcite , can lead to divergent decomposing outcomes. in this example, factscore generates ambiguous sub-claims, while veriscore omits key information (e.g., "ultimately, the success of the su-57...") from the input. specifically, chatgpt's accuracy improves with claim-level decomposition, while gpt-4's performance declines. despite observed inconsistencies, the underlying reasons remain unclear, and there is limited in-depth analysis of how decomposition affects downstream performance. in this study, we investigate the underlying causes of performance variability by addressing three key questions. first, what determines decomposition's effect on factchecking performance? through experiments, we present performance variability across factors such as input complexity, decomposition method design, and verifier strength. second, what errors may decomposition introduce? we categorize decomposition error types, analyze their distribution, and validate their usefulness through error reflection. third, what explains the variability in fact-checking performance? our analysis reveals a trade-off between the accuracy gains by decomposing inputs into manageable sub-claims and the noise introduced by retrieval and decomposition as the number of sub-claims increases. this trade-off offers an explanation for the observed variability and provides insights for guiding future fact-checking pipeline design. here are some key takeaways: ## what determines decomposition's effect on fact-checking performance? based on the inconsistent impact of decomposition observed in the decompose-then-verify pipeline, we identify three primary factors contributing to the inconsistencies: decomposition method, verifier strength, and input granularity. our analysis demonstrates that each factor exerts a distinct influence, with no single method uniformly enhancing downstream performance across all granularities and verifiers. notably, verifier strength proves to be a key determinant of performance. ## what errors may decomposition introduce? while previous studies @xcite @xcite have identified 'atomicity', 'coverage', and 'verifiability' as key factors in decomposition, less focus has been placed on errors that may harm downstream factchecking performance. to explore this, we manually inspect cases where the baseline was correct but decomposition led to incorrect outcomes. ## what explains the variability in fact-checking performance? while factors contributing to variability and decomposition errors have been identified, the reasons for decomposition-induced performance variability remain unclear. we propose that this variability results from a decomposition trade-off and conduct experiments to empirically investigate it. ## conclusion we present a comprehensive analysis of decomposition's impact on downstream performance in the decompose-then-verify pipeline. we explicitly show the fact-checking performance variations across decomposition methods, input granularities, and verifiers. through detailed inspection, we categorize decomposition errors and reveal their distribution across methods and granularities. furthermore, we reveal a trade-off between the accuracy gains from decomposition and the noise introduced by decomposition, offering an explanation for the inconsistencies reported in prior studies.