Dunamu-ml’s Submissions on AVERITEC Shared Task (2024)

Abstract: This paper presents the Dunamu-ml’s submission to the AVERITEC shared task of the 7th the Fact Extraction and VERification (FEVER) workshop. The task focused on discriminating whether each claim is a fact or not. Our method is powered by the combination of an LLM and a non-parametric lexicon-based method (i.e. BM25). Essentially, we augmented the list of evidences containing the query and the corresponding answers using an powerful LLM, then, retrieved the relative documents using the generated evidences. As such, our method made a great improvement over the baseline results, achieving 0.33 performance gain over the baseline in AveriTec score.

## introduction the rise in misinformation has led to a greater need for fact-checking, which involves determining the accuracy of a claim through evidence. consequently, research on methods that automatically detect whether specific claims are true or false is being conducted actively. @xcite as part of this effort, the shared task fact extraction and verification (fever) 1 is held regularly @xcite @xcite . fact-checking requires large-scale retrieval. large-scale retrieval involves retrieving the most relevant documents from a vast collection containing millions to billions of entries in response to a text query. over the past ten years, deep representation learning techniques have become essential for large-scale retrieval, transitioning from traditional bag-of-words (bow) @xcite methods to pre-trained language models (plms) @xcite . the latest advancements in llms offer a quicker path to achieve zero-shot retrieval by enhancing a query with potential answers obtained from the llms @xcite . in this paper, we introduce our approach to the fever 2024 share task named averitec shared tasks @xcite . we aim to build our model powered by the generation and retrieval ability of recent llms @xcite . our method is inspired by @xcite which utilize a non-parametric lexicon-based method (such as bm25 @xcite ) as the retrieval component to directly measure the similarity between the query and document and boost the query using powerful llm. first, we generated initial question and answer pairs without any documents retrieved. then, we retrieved relevant documents and fix the initial answers using it. finally, we infer the final answer using the given evidences. our approach significantly enhanced the baseline outcomes, securing a 0.33 increase in performance compared to the baseline according to the averitec score. for evaluation, we used the given system foot_0 . ## task description the averitec challenge @xcite aims to evaluate the ability of systems to verify real-world claims with evidence from the web. the output format of each claim should be: the claim text itself. ## system overview in this section, we firstly provide a brief description of how we pre-processed the given knowledge store and present our approach to the task. ## conclusion in this work, we described the dunamu-ml's submission to the averitec shared tasks of the fever 2024. by integrating a language model (llm) with a non-parametric lexicon-based method (bm25), our approach bolstered the evidence list by integrating the query and associated answers using a robust llm. this strategy allowed us to pinpoint pertinent documents based on the