Comparing LLM s and BERT -based Classifiers for Resource-Sensitive Claim Verification in Social Media (2025)

Abstract: The overwhelming volume of content being published at any given moment poses a significant challenge for the design of automated fact-checking (AFC) systems on social media, requiring an emphasized consideration of efficiency aspects.As in other fields, systems built upon LLMs have achieved good results on different AFC benchmarks. However, the application of LLMs is accompanied by high resource requirements. The energy consumption of LLMs poses a significant challenge from an ecological perspective, while remaining a bottleneck in latency-sensitive scenarios like AFC within social media. Therefore, we propose a system built upon fine-tuned smaller BERT-based models. When evaluated on the ClimateCheck dataset against decoder-only LLMs, our best fine-tuned model outperforms Phi 4 and approaches Qwen3 14B in reasoning mode — while significantly reducing runtime per claim. Our findings demonstrate that small encoder-only models fine-tuned for specific tasks can still provide a substantive alternative to large decoder-only LLMs, especially in efficiency-concerned settings.

## introduction while social media can be a space for public discourse, it can also be a place where misinformation and disinformation claims become dominant. in real-life claim verification, fast response times could be decisive in regard to the impact of harmful claims, such as providing verdicts before the claims start to spread. in the context of climate-related topics, where claims can be verified by a large amount of research, an opportunity is provided to combat misinformation by retrieving relevant research to verify said claims. like many other tasks in the natural language processing (nlp) domain, automated fact-checking systems are gaining significant performance boosts with the rise of large language models (llms). in the context of social media, however, the application of llms for tasks such as claim verification is greatly hindered by their high computational costs and latency. which, on a large scale, is problematic from an ecological point of view @xcite , as well as when considered from a latency-sensitive system design perspective @xcite . moreover, recent research indicates that bertbased models fine-tuned for specific tasks can still be competitive with zero-shot llms in text classification @xcite , or even outperform llms as shown in bucher and martini (2024) while also outperforming other classifiers in related challenging tasks like propaganda detection @xcite . as discussed in related studies such as @xcite , many encoderonly bert-based models like deberta-v3 @xcite are accompanied by significantly lower computational costs and therefore have a lower ecological impact due to a smaller number of parameters than many of their recent decoder-only counterparts like qwen3 @xcite or phi 4 @xcite . thus, we want to explore how both model classes perform on the climatecheck dataset @xcite ) -which was released in the context of the climatecheck@sdp 2025 shared task @xcite with respect to veracity prediction. in both cases, the input for prediction is acquired by an abstract retrieval pipeline, which we propose in this paper, and which also does not rely on llms. the main contributions of this paper can be summarized as follows: we have released our code foot_0 and the models foot_1 , foot_2 we fine-tuned in the context of this paper. ## related work in recent studies on the verification of climate claims @xcite , agent-based llm systems have been shown to achieve promising results when verifying claims based on retrieved knowledge from a corpus such as provided by the intergovernmental panel on climate change (ipcc). however, in a dynamic situation with a large unfiltered corpus of scientific papers and the frequency of social media claims, the cost and latency may limit the applicability of such a pipeline alone. at the same time, several datasets were published for the verification of claims outside of the climate domain. for example, pubhealth @xcite focuses on public health-related claims, which are accompanied by claims labeled with "true", "false", "mixture" and "unproven". the fever (fact extraction and verification) dataset @xcite aims at the development of systems for the verification of claims on different topics against textual sources, using the labels "supported", "refuted" or "notenoughinfo" -a label scheme similar to the labels in climate-check. averitec (automated verification of textual claims) @xcite focuses on retrieved evidence from the open web to verify claims, also providing samples with the additional label "conflicting evidence/cherrypicking". in @xcite , the averitec task is understood as related to natural language inference (nli) tasks, which focus on logical inference based on free-text data. in this paper, the authors proposed a label mapping scheme for pubhealth and averitec and fine-tuned a t5-3b model @xcite , whose initial training included data from nli datasets. this strategy inspired us to explore models beyond decoder-only architectures that were fine-tuned on nli datasets and to finetune them further in the context of climatecheck. ## methodology subtask i: abstract retrieval the first subtask focuses on the retrieval of relevant abstracts from a corpus of around 400k abstracts of publications from the climate science domains. we propose the following pipeline for this subtask, also illustrated in figure 1: 1. sparse retrieval: get the top 1500 most relevant abstracts from the corpus using each claim as the query via bm25 2. dense retrieval: get the most relevant top 150 results from ( 1) 3. rerank the results from ( 2 ) with a reranking model and return the final top 10 results the inclusion of step (1) was the result of preliminary experiments, where we first explored the strategy of running dense retrieval on the full set of the embeddings of all 400k abstracts. since this strategy yielded subpar results, we opted for a hybrid search approach by including sparse retrieval, which is a frequent approach in retrieval tasks to improve retrieval scores (as shown in sawarkar et al. ## discussion our results indicate that while recent decoder-only zero-shot llms such as qwen3 are able to receive impressive results on datasets like climatecheck just by prompting them without applying any finetuning strategies, fine-tuned encoder-only bertbased models can achieve comparable results at a fraction of the runtime. in conclusion, the smaller model class can still be a valid choice, particularly in scenarios where low latency is a critical factor.