DOI: 10.5281/zenodo.15267291
title: AFP-Sum: A Generative-AI-Driven Claim Retrieval System Capable of Detecting and Retrieving Claims from Social Media Platforms in Multiple Languages

description: A Generative-AI-Driven Claim Retrieval System Capable of Detecting and Retrieving Claims from Social Media Platforms in Multiple Languages

Abstract: Online disinformation poses a global challenge, placing significant demands on fact-checkers who must verify claims efficiently to prevent the spread of false information. A major issue in this process is the redundant verification of already fact-checked claims, which increases workload and delays responses to newly emerging claims. This research introduces an approach that retrieves previously fact-checked claims, evaluates their relevance to a given input, and provides supplementary information to support fact-checkers. Our method employs large language models (LLMs) to filter irrelevant fact-checks and generate concise summaries and explanations, enabling fact-checkers to faster assess whether a claim has been verified before. In addition, we evaluate our approach through both automatic and human assessments, where humans interact with the developed tool to review its effectiveness. Our results demonstrate that LLMs are able to filter out many irrelevant fact-checks and, therefore, reduce effort and streamline the fact-checking process.

Paper: https://arxiv.org/abs/2504.20668

GitHub Repository: https://github.com/kinit-sk/claim-retrieval

The data are available upon request for research purposes only.

References

If you use this dataset in any publication, project, tool or in any other form, please cite the following paper:



@misc{vykopal2025generativeaidrivenclaimretrievalcapable,      title={A Generative-AI-Driven Claim Retrieval System Capable of Detecting and Retrieving Claims from Social Media Platforms in Multiple Languages},       author={Ivan Vykopal and Martin Hyben and Robert Moro and Michal Gregor and Jakub Simko},      year={2025},      eprint={2504.20668},      archivePrefix={arXiv},      primaryClass={cs.CL},      url={https://arxiv.org/abs/2504.20668}, }


Content





afp-sum.csv - AFP-Sum dataset consisting of around 19K fact-checks across 23 languages






id - Article ID




url - A URL of a fact-checking article




text - A text extracted from the fact-checking article




summary - A summary extracted from the fact-checking article




processed_text - Text of the fact-checking article without the summary




language - Language of the fact-checking article





sample2.csv - Sample of 2 fact-checking articles per language from the AFP-Sum dataset






id - Article ID




url - A URL of a fact-checking article




text - A text extracted from the fact-checking article




summary - A summary extracted from the fact-checking article




processed_text - Text of the fact-checking article without the summary




language - Language of the fact-checking article





sample100.csv - Sample of 100 fact-checking articles per language from the AFP-Sum dataset






id - Article ID




url - A URL of a fact-checking article




text - A text extracted from the fact-checking article




summary - A summary extracted from the fact-checking article




processed_text - Text of the fact-checking article without the summary




language - Language of the fact-checking article





fact_checks_metadata.csv - Metadata for the MultiClaim dataset and especially for the fact-checking articles






fact_check_id - Id of the fact-checks from the original MultiClaim dataset




url - A URL of the fact-checking article




rating_category - Rating extracted from the fact-checks metadata




language - Language of the fact-checking article




published_at - Publication date of the fact-checking article




Acknowledgments

This project is funded by the European Media and Information Fund (grant number 291191). The sole responsibility for any content supported by the European Media and Information Fund lies with the author(s) and it may not necessarily reflect the positions of the EMIF and the Fund Partners, the Calouste Gulbenkian Foundation and the European University Institute.