Automated Justification Production for Claim Veracity in Fact Checking: A Survey on Architectures and Approaches (2024)

Abstract: Automated Fact-Checking (AFC) is the automated verification of claim accuracy. AFC is crucial in discerning truth from misinformation, especially given the huge amounts of content are generated online daily. Current research focuses on predicting claim veracity through metadata analysis and language scrutiny, with an emphasis on justifying verdicts. This paper surveys recent methodologies, proposing a comprehensive taxonomy and presenting the evolution of research in that landscape. A comparative analysis of methodologies and future directions for improving fact-checking explainability are also discussed.

text: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6679–6692 August 11-16, 2024 ©2024 Association for Computational Linguistics
Automated Fact-Checking (AFC) is the automated verification of claim accuracy. AFC is crucial in discerning truth from misinformation, especially given the huge amounts of content are generated online daily. Current research focuses on predicting claim veracity through metadata analysis and language scrutiny, with an emphasis on justifying verdicts. This paper surveys recent methodologies, proposing a comprehensive taxonomy and presenting the evolution of research in that landscape. A comparative analysis of methodologies and future directions for improving fact-checking explainability are also discussed.

heading: 1 Introduction
text: The huge increase in both user-generated and automated content has led to a significant amount of misinformation. This poses risks to uninformed readers, highlighting the need for scalable, automated methods for verification and fact-checking (Nakov et al., 2021a). While predicting the veracity of claims is essential, relying solely on predictions without providing explanations can be counterproductive, potentially reinforcing belief in false claims and perpetuating misinformation (Lewandowsky et al., 2012).
Most fact-checking models use neural architectures, but interpreting these models is challenging. There is a need for fact-checking frameworks providing justifications to enhance effectiveness and trustworthiness. This survey presents recent efforts addressing automatic justification production for claim verification, emphasizing the move towards “Explainable” Automated Fact-Checking (AFC). Some work refers to the justification production process as the explanation generation process (Kotonya and Toni, 2020a). In this survey, the term “justification production” is used following the work of Guo et al. (2022).
This survey’s main contribution is as follows: Firstly, it introduces a multidimensional taxonomy for categorizing works based on various criteria. Secondly, it provides how research is progressing towards standard justifications. Thirdly, it conducts a comparative analysis of justification production approaches, pipeline architectures, input and output types. Lastly, it identifies challenges while proposing future directions in justification production. Appendix A outlines the methodology utilized for literature compilation, detailing the search strategy and selection criteria employed for the papers that form the cornerstone of this survey.

heading: 2 Related Surveys
text: Thorne and Vlachos (2018) provided a comprehensive review of early developments in fact-checking, but they don’t focus on verdicts with justifications. Other surveys such as Nakov et al. (2021b,a); Guo et al. (2022); Vladika and Matthes (2023) offer broad overviews of the entire fact-checking process and its various components. In contrast, our work specifically concentrates on the aspect of justification production. Moreover, recent multi-modal fact-checking surveys (Alam et al., 2022; Vladika and Matthes, 2023) mention that natural language justification production remains unexplored in the multi-modal AFC domain. In this survey, we highlight some of the emergent works in multi-modal justification production.
The survey by Kotonya and Toni (2020a), focusing on justification production, is closely related to ours. However, since then, there has been a significant progress driven by the rapid development of transformer-based architectures and Large Language Models (LLMs). Vallayil et al. (2023) only augmented the latter work’s taxonomy with counterfactual justifications. While partially covering some recent work, they do not provide a comprehensive, new, detailed multi-dimensional taxonomy as proposed in this survey.
6679

heading: 3 Justification Production within AFC
text: AFC consists of multiple stages forming a pipeline, as shown in Figure 1. One of these stages is justification production. In the upcoming subsections, a brief overview of the general stages in the AFC pipeline is provided, with a specific focus on the justification production stage.

heading: 3.1 Check-worthy Claims Detection Stage
text: This initial stage classifies the claims as checkworthy or not. If they are check-worthy, then they are selected from the corpus containing them. Deeming if a claim is check-worthy or not is based on the importance of the topic of the claim, if it is verifiable, and if the claim poses potential harm in case it is misleading (Guo et al., 2022).

heading: 3.2 Retrieval and Selection of Most Relevant Evidence
text: This stage retrieves data related to the claim from trustworthy sources and selects the most relevant information to make a decision, which is termed the ‘evidence.’ In the subsequent stage, this evidence is used to predict the veracity of the claim. The determination of veracity depends on the degree of alignment between the claim and the evidence. For example, the veracity of the claim ‘The director of the film ‘Legend’ is English’ could depend on the following evidence snippets gathered from multiple trustworthy websites: ‘Brian Helgeland is the director of the film ‘Legend”, and ‘Brian Helgeland only holds a U.S. citizenship’

heading: 3.3 Veracity Prediction of the Claim
text: This stage classifies claims according to a binary scheme, true or false, or through fine-grained multiclass classification including also other verdicts such as “partially correct”, or “correct but misleading without extra context”. Following the example in the previous section, the claim ‘The director of the film ‘Legend’ is English’ should be determined as ‘False’ as it is not aligned with the evidence.

heading: 3.4 Justification Production
text: This stage produces justifications to explain the verdict of an AFC model regarding a claim’s veracity. The process is known as justification production (Guo et al., 2022).
In the context of the previously discussed claim, ‘The director of the film ‘Legend’ is English,’ an example of a justification for the ‘False’ verdict,
grounded in the evidence, could be ‘Brian Helgeland, the director of the film ‘Legend,’ is American and not English.’ Hence, the inputs for a justification production component are the claim and the selected evidence. The veracity verdict may also be an input, depending on the pipeline architectures of the AFC systems that are explained in Section 6.3 and are shown in our proposed classification of pipelines (see Figure 2).
We propose categorizing the work in justification production not only based on these pipeline architectures but also on additional dimensions (see Figure 3). A key dimension is the explainability of the justification production process. The steps of the process leading to the prediction of the claim’s veracity and its justification can be self-explainable or not.
In addition, the input type is an important dimension. It can be either multi-modal or textonly. Another dimension is the nature of the justification output. It may be natural language text, or just highlighted parts of the input, like bold/highlighted words in the claim and evidence, or specific factual triples in the form Subject, Predicate, Object (SPO) (see Figure 4 for illustrative examples).
We can also differentiate studies based on the type of main approaches utilized, which include: attention based where specific segments of the input having the highest attention scores are highlighted based on the relationship between the evidence and the claim; knowledge graph based where a graph is used to represent the evidence. The relevant evidence rationals are selected nodes in the graph, and the edges represent the relations between these selected nodes. Symbolic logic is used to determine if the evidence is aligned with the claim; summarization based where the relevant evidence rationals are summarized as natural language text with a focus on whether the claim is aligned with the evidence or not; multi-hop based where the claim is decomposed into smaller parts related to each other and these parts are sequentially checked if they are aligned with the evidence or not.; and LLMs Retrieval Augmented Generation (RAG) or Fine-tuning based approaches where LLMs are used via prompting to verify the alignment between the claim and the evidence rationals producing the veracity verdict and the justification for the verdict.
Section 6.5 describes the approaches mentioned
above in more detail. The quality of the justifications produced by these approaches is evaluated based on the presence of certain desired properties, which will be discussed in Section 4.

heading: 4 Progression towards Justifications Standardization
text: The aim of justification production is to create a justification that aligns with specific, agreed-upon criteria (Sokol and Flach, 2019), which we refer to as a standard justification. Achieving high-quality justifications involves considering certain desired properties known as ‘desiderata’ (Kulesza et al., 2015). Researchers have collectively agreed upon these desired properties (Sokol and Flach, 2019). Producing justifications aligned with these desired properties is crucial for standardizing justifications in explainable AFC.
Graves (2018) identifies key desiderata for justifications, completeness, where the justification must be valid in full contextuality; coherence, ensuring the faithfulness/consistency between the veracity prediction and justification; interactivity, which is putting into consideration the users’ feedback; actionability, providing the user with the needed suggestions for modifying the claim to change it from non-factual to factual; chronology, giving preference to the timing of the claim; novelty, ensuring the justification offers new information; complexity, adjusting the justification’s language based on the user’s knowledge; parsimony, favouring more short and concise justifications; causality, where a comprehensive causal model is used for deducing causal connections between inputs and the predictions produced. These properties were defined with further details by Kotonya and Toni (2020a), who also added the desideratum of unbiased or impartial justifications. In the context of fact-checking, bias usually manifests as opinions masquerading as evidence.
Kotonya and Toni (2020b) started the first attempt to provide a standard justification evaluation process by measuring two different types of coherence in the produced justifications: the global coherence which assesses the relevance of a justification in relation to both the claim and its label; and the local coherence which evaluates the cohesion of sentences within a justification. To maintain local coherence, there should be no contradiction between any two sentences in the justification. Atanasova et al. (2022) started the first attempt to generate standard justifications by adding some desired properties (i.e. faithfulness/coherence, and data consistency) as additional learning signals in the loss function of a transformer-based model (Vaswani et al., 2017). The data consistency evaluates the similarity of justifications for similar input instances.

heading: 5 Datasets in AFC
text: It’s worth noting that this survey focuses on providing a new taxonomy, a comparative analysis of justification production approaches, investigating pipeline architectures, addressing challenges encountered, and proposing future directions in AFC justification production. Comprehensive examinations of datasets were covered thoroughly in previous surveys (mentioned in Section 2). However, some information about datasets in AFC is also provided in this section.
The dataset might contain the needed content for all the stages of the fact-checking pipeline: claim detection, evidence retrieval and selection, veracity verdict production, and justification production. The following paragraphs will discuss the type of content representing each stage with example datasets provided in Table 1.
Textual claims are the most common input for fact-checking because they are often produced after the claim detection stage. These claims are usually
sentence-level statements. Many researchers have created datasets by collecting real-world claims from specialized websites like Politifact because they are easily accessible. Some researchers concentrate on obtaining claims from specialized areas like climate change, science, and public health. Other sentence-level inputs, such as answers to questions in forums, have also been investigated. Some datasets are English, and some are multilingual. Metadata, including information such as publication date, sources, and user profiles, is a common type of evidence considered. Although metadata can provide complementary insights when textual sources or structural knowledge are lacking, it does not directly substantiate the claim.
Text-based sources, such as news articles, academic publications, and Wikipedia entries, are frequently employed as evidence for fact-checking. Multi-modal claims and evidence have recently been researched, and images and videos are considered more credible than text by most audiences.
Not all the datasets have a binary veracity verdict scheme. Fact-checkers often utilize multi-class labels to classify different levels of truthfulness, including categories such as ‘true,’ ‘mostly true,’ and ‘mixed.’
From the perspective of justification production, AFC datasets can be classified into two categories:
those without justifications and those with justifications. Complementing datasets lacking justifications is important. This was done by Zhu et al. (2023) on the HOVER dataset. Table 1 includes datasets from both categories, with some being synthetically generated and others extracted from external sources like Wikipedia.

heading: 6 Justification Production Taxonomy
text: Multiple dimensions or criteria for categorizing Explainable Automated Fact-Checking systems are outlined in this section. We propose five dimensions (illustrated by the first five levels/columns of the Taxonomy tree in Figure 3). The Justification Process Explainability category (Section 6.1) indicates whether the process leading to the justification production is explainable or not. Then the Type of Justifications criterion (Section 6.2) indicates whether these are a set of SOP triples, highlighted parts of selected rationals from the evidence input, or natural language textual justifications. Other discriminatory dimensions are the Pipeline Architecture of the AFC components (Section 6.3), the Input Type (Section 6.4), whether it is text or multi-modal, and the Main Approach (Section 6.5), which is the categorization of the predominant methods used for justification production. In the following sections, every dimension in the taxonomy is discussed in more details.

heading: 6.1 Explainability of Justification Process
text: The degree of clarity of the process through which the claim is processed and aligned with evidence to produce the justification makes the process selfexplanatory. For instance, Multi-hop approaches using QA pairs exemplify this clarity, decomposing claims into parts and then checking their alignment with each evidence snippet. Summarization approaches lack such clarity. For example, consider the claim: “The director of Interstellar was born in 1960.” and the corresponding evidence snippets: “Christopher Nolan was born on 30 July 1970”, “The name of the director of the film Interstellar is Christopher Nolan.” and “Interstellar is a 2014 epic science fiction film.” The justification of the multihop approach (self-explainable) is: "Interstellar is a 2014 science fiction film that was directed by Christopher Nolan. Christopher Nolan was born in 1970, not in 1960, so the claim is false." The justification of the summarization approach (nonself-explainable) is: "Christopher Nolan - born in
1970 - is the director of Interstellar." In the multi-hop approach, the process of justification production consists of decomposing the claim into three parts: ’Interstellar’, ’the director of Interstellar’, and ’born in 1960’ checking them against relevant evidence snippets. This approach offers clarity by revealing how the input claim is processed and aligned with evidence. In the nonself-explainable justifications, this process is not revealed. This is the case with surveyed summarization approaches. Additionally, the multi-hop justification example provided a sequential explanation. It starts with evidence (Interstellar is a 2014 science fiction film) to address the first part of the claim, i.e., ’Interstellar.’ It then proceeds to address the next part of the claim, ’the director of Interstellar’, by stating "that was directed by Christopher Nolan,". After that, it addresses the last part of the claim, ’born in 1960’, stating that "Christopher Nolan was born in 1970, not in 1960." It outlines a sequence of logical steps grounded in the alignment of claim-evidence to support the conclusion that "the claim is false."

heading: 6.2 Type of Justification
text: The type of justification varies depending on the approach used in the justification production process. Figure 4 shows examples of different types of output justifications. From Figure 3, we can observe that Natural Language justifications dominate in recent research as they are the most comprehensible for the readers compared to SOP triples or highlighted words in the evidence.

heading: 6.3 Justification Production Pipelines Architectures
text: We propose to differentiate various pipelines for Explainable AFC based on the relationship between the justification production stage (Section 3.4) and the veracity prediction stage (Section 3.3). These pipelines can be classified into four types, depicted in Figure 2.
In the ‘Separated-Veracity-Justification’ pipeline (Figure 2.a), the veracity prediction and justification production are independent processes. This architecture was investigated and used by Atanasova et al. (2020) and Kotonya and Toni (2020c). It is the earliest pipeline offering simpler error tracing capabilities but faces challenges with contradictions between justification and veracity predictions. Research interest in this pipeline is diminishing with the emergence of more robust al-
ternative pipelines like Justification-Then-Veracity and Joint-Veracity-Justification, as discussed later.

heading: In the ‘Veracity-Then-Justification’ pipeline
text: (Figure 2.d.), the veracity verdict is produced and then inputted into the justification production module to ensure consistency between the output justification and the verdict in contrast with SeparatedVeracity-Justification. Moreover, this pipeline allows the usage of different models separately. Each model can handle a different modality; for instance, Yao et al. (2023) trained a sentence-BERT model (Reimers and Gurevych, 2019) on the textual input while using CLIP (Radford et al., 2021) on the visual input ‘images.’ This pipeline is flexible, allowing a modular design while maintaining consistency between justifications and claim veracity predictions. It should be noted that this pipeline not only processes multi-modal input but it can also be employed for textual input.

heading: In the ‘Joint-Veracity-Justification’ pipeline
text: (Figure 2.c), veracity prediction and justification production are combined tasks carried out by the same model. According to Atanasova et al. (2020), this pipeline under-performed in summarization compared to the ‘Separated-Veracity-Justification’ pipeline. Yet, it excelled in completeness, incorporating essential details vital for the fact-checking process. Moreover, it demonstrated superiority in the overall quality of the justifications produced. This pipeline is also used in multi-modal explainable AFC through generating justifications by highlighting the most salient parts of the input having the highest attention scores (Kipf and Welling, 2016; Kou et al., 2020; Wu et al., 2019; Bonettini et al., 2021; Purwanto et al., 2021) as shown in Figure 3.

heading: In the ‘Justification-Then-Veracity’ pipeline
text: (Figure 2.b), a ‘reasoner’ breaks down the claims into smaller segments. It then evaluates each segment of the claim, using available evidence to verify their alignment. Essentially, it employs a logical ‘AND’ operator to determine if all segments of the claim are factual, leading to the final verdict. The verdict is reached after the justification is produced. This pipeline aligns with the most recent research (Figure 3). Techniques used in this pipeline include LLMs Chain-of-Thought (CoT) (Pan et al., 2023b), and Multi-hop approaches (Wang and Shu, 2023). Chakraborty et al. (2023) uses this pipeline in multi-modal explainable AFC. These approaches are further detailed in Section 6.5.

heading: 1. Images

heading: 6.4 Input Type
text: The input type can be text or multi-modal. The text input is predominant in Explainable AFC. Textonly datasets are more frequent than their multimodal counterpart (Figure 3). Multi-modal explainable AFC falls under three categories based on the main approaches dimension: attention based, multi-hop based and summarized natural language text (see Figure 3). The attention based approaches like (Zhang et al., 2023a) have ‘Joint-VeracityJustification’ pipeline architecture, where the input data, like the author of the claim and its timing, are inputted in a fine-tuned transformer based model. Using the attention mechanism, tokens of high attention scores in the evidence and the claim are presented as justification for the veracity verdict. In the new emerging approaches like (Yao et al., 2023), a sentence-BERT is used to process text corpus and a CLIP encoder model is used to present visual features in an image. All these features are then combined and given to a classifier for verdict prediction and also to BART model (Lewis et al., 2020) for justification production. Chakraborty et al. (2023) has used the ‘Justification-then-Veracity’ pipeline along with SOTA T5 (Raffel et al., 2020) for QA pairs gen-
eration during claim decomposition and a CNN to analyze visual claims and evidence. Figure 3 outlines the works that employ multi-modal input in Explainable AFC, according to the approach involved.

heading: 6.5 Main Approaches in Explainable AFC
text: The following sections detail the main approaches dimension in the taxonomy shown in Figure 3. Examples of justifications produced with these approaches are presented in Figure 4.

heading: 6.5.1 Attention Based Approaches
text: These approaches mostly use transformer based architectures with attention mechanisms, where justifications are the input segments with the highest attention scores, i.e. justifications are the highest attention score words from the claim and the evidence highlighted in a bold format. As shown in Figure 3, they are used with multi-modal input as well as textual input. The advantage of these approaches is the simplicity of the AFC pipeline compared to the others, as it doesn’t have a generator to produce natural language justifications. Thorne and Vlachos (2021)’s work takes a further step by employing a Masked Language Model (MLM) for correcting false claims by replacing
their salient false parts with appropriate generated text. However, there are several limitations. Guo et al. (2022) confirm that removing tokens with high attention scores doesn’t consistently impact model predictions, questioning the attention mechanisms reliability. Conversely, lower-scoring tokens have been found crucial for accurate predictions. The attention based approaches (Figure 3) are categorized into three distinct groups: (1) Yang et al. (2019) employ the ‘Separated-VeracityJustification’ architecture; (2) Popat et al. (2017, 2018); Shu et al. (2019); Lu and Li (2020); Wu et al. (2020) adopt the ‘Joint-Veracity-Justification’ architecture. Both groups utilize textual input. (3) Lourenço and Paes (2022); Bonettini et al. (2021); Shang et al. (2022); Purwanto et al. (2021); Kou et al. (2020); Wu et al. (2019); Zhang et al. (2023a) address multi-modal input by employing the ‘JointVeracity-Justification’ pipeline.

heading: 6.5.2 Knowledge Graph Based Approaches
text: In this approach, justifications are generated based on a graph with all the needed knowledge regarding nodes and relations between these nodes. The computational complexity of a knowledge graph creation from text can limit its scalability. While it provides a structured framework, knowledge graphs may not capture all nuances of natural language. Moreover, they may rely on predefined rules that
might not cover all possible scenarios. Additionally, the readability of SOP justifications typically produced with this approach might be difficult for non-expert users to comprehend. Logic rules are needed to search for relevant information in such graphs. For instance, Gad-Elrab et al. (2019) uses horn rules, which are an implication from an antecedent to a consequent. Ahmadi et al. (2019) extended the work by adding probabilistic answer set programming to the horn rules, while Dziri et al. (2021) used fine-tuned LLMs to traverse the knowledge graphs nodes.

heading: 6.5.3 Summarization Based Approaches
text: This approach can be extractive, providing short and concise information with less redundancy like in (Yao et al., 2023; Atanasova et al., 2020; Shen et al., 2023; Jolly et al., 2022) or extractiveabstractive where the extractive summary produced undergoes another process of abstraction. For instance, in the summarization of medical reports, a lot of medical terminology can confuse nontechnical audiences, so having a holistic, simpler summary with less technical terminology, such as an ‘abstractive summary,’ is important. This method is implemented in (Kotonya and Toni, 2020c; Russo et al., 2023). Most of the work on summarization is done using pre-trained models like when Augenstein et al. (2019) used distilled BERT (Sanh et al., 2019). Russo et al. (2023) gave an exhaustive study on enhancing extractiveabstractive summarization, and Jolly et al. (2022) improved extractive summarization with unsupervised post-editing. The extractive approach lacks the existence of desiderata, while the extractiveabstractive summarization has a higher probability of producing hallucinations than extractive summarization.

heading: 6.5.4 LLMs Reasoning via Prompting and RAG or Fine-tuning
text: LLM prompting, RAG and finetuning are being extensively used as approaches in the domain of explainable AFC. Stammbach and Ash (2020) was among the first researchers to use LLM prompting in explainable AFC. Using LLMs generally makes reasoning easier to implement. However, the computational costs are high, and sometimes, LLMs produce hallucinations. There are many types of hallucinations in LLMs and, in this survey, we focus on fact-conflicting hallucinations. As per Zhang et al. (2023b), fact-conflicting hallucinations
are produced when LLMs generate information or text that contradicts established world knowledge.
Note that as per Huang et al. (2023), LLMs can not correct themselves when hallucinating, therefore Gou et al. (2023) used Chain-of-Thought (CoT) as a possible solution. CoT -introduced by (Wei et al., 2022)- along with in-context learning and external tools -like search engines-, can greatly reduce hallucinations through reasoning.

heading: 6.5.5 Multi-hop Approaches
text: Multi-hop approaches are always associated with other methods like graph based methods (Xu et al., 2023), natural logic theorem (Krishna et al., 2022), and QA pair generation along with CoT in LLMs. Multi-hop approaches are being more frequently used in research works like (Wang and Shu, 2023; Peng et al., 2023; Pan et al., 2023b; Dhuliawala et al., 2023; Wang et al., 2023; Pan et al., 2023a). However, multi-hop fact checking is a complex reasoning task. Designing an effective method to generate justifications in the multi-hop setting requires consideration of the logical relationships between the claim and between multiple pieces of evidence. However, the prompts given to LLMs can be enhanced, e.g., using CoT, to exploit more potential of LLMs. Generally, Multi-hop LLMs reasoning methods are computationally and financially costly.

heading: 7 Challenges and Future Directions
text: This section outlines the challenges of producing justifications and highlights promising research efforts to address them, suggesting future directions.
Evaluating and Generating Justifications According to Desiderata One of the main goals of producing justifications in Explainable AFC is to align them with specific desiderata (Section 4). Developing quantitative frameworks, or mathematical formulations, is crucial for measuring desiderata in a structured manner. This allows for a systematic comparison of explainable AFC systems based on their incorporation of these desiderata. Furthermore, integrating these measurements into the model training process can significantly enhance justification quality. To date, only Kotonya and Toni (2020b) has explored modeling and integrating one particular desired property, coherence/faithfulness, as a learning signal in model training. This area has seen limited further exploration.
Another promising avenue for achieving several desiderata, given the recent proliferation of rea-
soners with LLMs (Section 6.5.4) and Multihop (Section 6.5.5), could be the production of counterfactual justifications, as suggested by Dai et al. (2022) and illustrated in Figure 4. Counterfactual justifications involve imagining scenarios or outcomes that did not actually occur and exploring their consequences. They involve alternate scenarios – for example, ‘if the Earth were flat, we would be able to see boats even when they are very far away using telescopes’ (as shown in Figure 4). When incorporated into the justifications, counterfactual reasoning can reinforce some desired properties such as completeness and coherence. Moreover, some desiderata not fully achieved by current work (Kotonya and Toni, 2020b), like actionability, could also be realized. For instance, counterfactual justifications can identify specific elements in a claim that, if altered, could render it factual, thus guiding users toward more accurate statements. By offering alternative perspectives on a claim, counterfactual justifications can provide novelty via new information that might not be apparent through traditional justification methods. Counterfactual justifications inherently involve understanding causal relationships, another highly desired property. The ultimate objective remains to incorporate most or all of the desiderata presented in Section 4.
Natural Language Justifications in Multimodal AFC The majority of works processing multi-modal input rely on attention-based approaches (Section 3). Commonly, these works use highlighted input segments as justifications. However, such justifications are less effective in meeting the desired properties compared to valid natural language. Only a few recent studies have incorporated multi-modality in the input while producing natural language justifications: Yao et al. (2023) and Chakraborty et al. (2023). Yao et al. (2023) use a ‘Veracity-Then-Justification’ process. However, this method is less intuitive compared to the much more popular ‘Justification-Then-Veracity’ pipeline. A key feature of this latter architecture is the inclusion of a reasoning component, which attempts to deduce veracity based on justifications grounded in the evidence. It is, however, predominantly used with text-only inputs. Chakraborty et al. (2023) are the only researchers so far to use this architecture with multi-modal input (Section 6.4). There is potential for further research in this area, especially with the advancements in LLMs
that can process multi-modal inputs and produce coherent, natural language justifications, similar to the approach used by Lin et al. (2024) in a different context of generating explanations for identifying harmful content in memes.
Non-factual Hallucinations in LLMs in AFC Nowadays, LLMs are used more frequently in AFC. The challenge is that they themselves can produce hallucinations. There are many types of hallucinations; however, the most related type to the domain of AFC is non-factual hallucinations. Aiming to address hallucinations, Du et al. (2023) introduced the Society of Minds (SOM) to improve the factuality and accuracy of the LLMs output. SOM is a method where multiple instances of the same language model produce results for the same query, and then they debate to unify and improve their answers, correcting hallucinations in multiple rounds. CoT is also used during these rounds. This method is based on the hypothesis that hallucinations are not produced consistently by LLMs. The debate rounds can also happen between different models like chatGPT versus BARD (Ahmed et al., 2023).
Complexity of Justification Production Generally, justification production via Multi-hop or LLMs reasoning methods are computationally and financially costly. For instance, employing FOLK (Pan et al., 2023b) led to an expense of 20 USD for every 100 examples when using the OpenAI API, or required 7.5 hours of processing time on locally deployed llama-30B models with an 8x A5000 cluster. Addressing the computational cost associated with reasoning methods justification production is essential, warranting exploration of techniques such as knowledge distillation (Hinton et al., 2015) and quantization (Choukroun et al., 2019).

heading: 8 Conclusion
text: In summary, this survey contributes a novel multidimensional taxonomy, comprehensively presents the architectures employed in justification production, explores emergent methodologies, conducts a comparative analysis of these methodologies, and proposes prospective avenues for further research.
Limitations
The limitations in this survey can be summarized in the following points:
1. We have not included work on AFC that focuses solely on claim verification based on the language and lexicons used in the claims.
2. The few related papers that were published before 2015 were not included in the taxonomy.
3. This survey focused only on the work on English justification production. Multi-lingual justification production should also be explored.

heading: A Methodology for Literature Compilation
text: This appendix outlines the methodology employed to compile the content of this survey. It details the search strategy and selection criteria used to curate the foundational content for this survey paper.
1. Search Strategy. Initially, we conducted a comprehensive search in the ACL Anthology, Google Scholar, and Google Search for related surveys. Within the ACL Anthology, we focused on venues such as EMNLP, ACL, and NAACL. The search involved using keywords like fact-checking, fact-checking survey, misinformation detection, explainable facts, and automatic fact-checking.
Furthermore, we gathered surveys related to the production of justifications in AFC. Our goal was to identify the earliest and most frequently cited papers in these surveys, which we considered as foundational or "pioneer" papers. Afterward, we tracked all papers that referenced these pioneer works up until the date of submission.
2. Selection Criteria. We only selected papers that were directly relevant to the subject matter of justification production in AFC. The selection was based on a careful review of the abstract, introduction, conclusion, and limitations of each paper. Following the selection phase, 73 relevant papers were chosen to form the foundational content of this paper.

heading: Future Work
text: This section outlines the challenges of producing justifications and highlights promising research efforts to address them, suggesting future directions. Evaluating and Generating Justifications According to Desiderata One of the main goals of producing justifications in Explainable AFC is to align them with specific desiderata (Section 4). Developing quantitative frameworks, or mathematical formulations, is crucial for measuring desiderata in a structured manner. This allows for a systematic comparison of explainable AFC systems based on their incorporation of these desiderata. Furthermore, integrating these measurements into the model training process can significantly enhance justification quality. To date, only Kotonya and Toni (2020b) has explored modeling and integrating one particular desired property, coherence/faithfulness, as a learning signal in model training. This area has seen limited further exploration. Another promising avenue for achieving several desiderata, given the recent proliferation of rea-
soners with LLMs (Section 6. 5. 4) and Multihop (Section 6. 5. 5), could be the production of counterfactual justifications, as suggested by Dai et al. (2022) and illustrated in Figure 4. Counterfactual justifications involve imagining scenarios or outcomes that did not actually occur and exploring their consequences. They involve alternate scenarios – for example, ‘if the Earth were flat, we would be able to see boats even when they are very far away using telescopes’ (as shown in Figure 4). When incorporated into the justifications, counterfactual reasoning can reinforce some desired properties such as completeness and coherence. Moreover, some desiderata not fully achieved by current work (Kotonya and Toni, 2020b), like actionability, could also be realized. For instance, counterfactual justifications can identify specific elements in a claim that, if altered, could render it factual, thus guiding users toward more accurate statements. By offering alternative perspectives on a claim, counterfactual justifications can provide novelty via new information that might not be apparent through traditional justification methods. Counterfactual justifications inherently involve understanding causal relationships, another highly desired property. The ultimate objective remains to incorporate most or all of the desiderata presented in Section 4. Natural Language Justifications in Multimodal AFC The majority of works processing multi-modal input rely on attention-based approaches (Section 3). Commonly, these works use highlighted input segments as justifications. However, such justifications are less effective in meeting the desired properties compared to valid natural language. Only a few recent studies have incorporated multi-modality in the input while producing natural language justifications: Yao et al. (2023) and Chakraborty et al. (2023). Yao et al. (2023) use a ‘Veracity-Then-Justification’ process. However, this method is less intuitive compared to the much more popular ‘Justification-Then-Veracity’ pipeline. A key feature of this latter architecture is the inclusion of a reasoning component, which attempts to deduce veracity based on justifications grounded in the evidence. It is, however, predominantly used with text-only inputs. Chakraborty et al. (2023) are the only researchers so far to use this architecture with multi-modal input (Section 6. 4). There is potential for further research in this area, especially with the advancements in LLMs
that can process multi-modal inputs and produce coherent, natural language justifications, similar to the approach used by Lin et al. (2024) in a different context of generating explanations for identifying harmful content in memes. Non-factual Hallucinations in LLMs in AFC Nowadays, LLMs are used more frequently in AFC. The challenge is that they themselves can produce hallucinations. There are many types of hallucinations; however, the most related type to the domain of AFC is non-factual hallucinations. Aiming to address hallucinations, Du et al. (2023) introduced the Society of Minds (SOM) to improve the factuality and accuracy of the LLMs output. SOM is a method where multiple instances of the same language model produce results for the same query, and then they debate to unify and improve their answers, correcting hallucinations in multiple rounds. CoT is also used during these rounds. This method is based on the hypothesis that hallucinations are not produced consistently by LLMs. The debate rounds can also happen between different models like chatGPT versus BARD (Ahmed et al. , 2023). Complexity of Justification Production Generally, justification production via Multi-hop or LLMs reasoning methods are computationally and financially costly. For instance, employing FOLK (Pan et al. , 2023b) led to an expense of 20 USD for every 100 examples when using the OpenAI API, or required 7. 5 hours of processing time on locally deployed llama-30B models with an 8x A5000 cluster. Addressing the computational cost associated with reasoning methods justification production is essential, warranting exploration of techniques such as knowledge distillation (Hinton et al. , 2015) and quantization (Choukroun et al. , 2019)