TELLER: A Trustworthy Framework For Explainable, Generalizable and Controllable Fake News Detection (2024)

Abstract: The proliferation of fake news has emerged as a severe societal problem, raising significant interest from industry and academia. While existing deep-learning based methods have made progress in detecting fake news accurately, their reliability may be compromised caused by the non-transparent reasoning processes, poor generalization abilities and inherent risks of integration with large language models (LLMs). To address this challenge, we propose TELLER, a novel framework for trustworthy fake news detection that prioritizes explainability, generalizability and controllability of models. This is achieved via a dual-system framework that integrates cognition and decision systems, adhering to the principles above. The cognition system harnesses human expertise to generate logical predicates, which guide LLMs in generating human-readable logic atoms. Meanwhile, the decision system deduces generalizable logic rules to aggregate these atoms, enabling the identification of the truthfulness of the input news across diverse domains and enhancing transparency in the decision-making process. Finally, we present comprehensive evaluation results on four datasets, demonstrating the feasibility and trustworthiness of our proposed framework. Our implementation is available at this link1.

heading: 1 Introduction
text: Fake news has emerged as a prominent social problem due to the rampant dissemination facilitated by social media platforms (Zhou and Zafarani, 2021). Additionally, the swift progress of generative artificial intelligence has further amplified this issue (Cardenuto et al., 2023). While human factchecking experts can accurately verify the authenticity of news, their efforts cannot scale with the overwhelming volume of online information. Con-
1https://github.com/less-and-less-bugs/Trust_ TELLER
sequently, researchers have turned to automatic fake news detection techniques.
Despite the improved predictive accuracy achieved by current deep learning-based detection approaches (Ma et al., 2023; Qi et al., 2021; Mehta et al., 2022), these methods suffer from the lack of transparency because of the black-box nature of neural networks (Cui et al., 2019) and a limited ability to generalize to unseen data of which the distribution is different from training data, given the inherent diversity of online information (e.g., topics, styles and media platforms) (Liu et al., 2024). Moreover, the increasing integration with LLMs is prone to uncontrollable risks due to hallucinations and societal applications. Thus, a growing awareness emphasizes trustworthiness2 of these systems (Liu et al., 2023; Sheng et al., 2022).
Unfortunately, the characteristics of a trustworthy fake news detector remain an open question. Hence, based on recent surveys of Trustworthy AI (Li et al., 2023; Jobin et al., 2019) and fake news detection (Shu, 2023), we identify three crucial aspects that go beyond accuracy for fake news detection technologies: explainability, generalizability, and controllability. These aspects work collectively
2In AI, trustworthiness refers to the extent to which an AI system can be trusted to operate ethically, responsibly, and reliably (Jobin et al., 2019).
15556
to enhance system security and trustworthiness. Firstly, explainability refers to understanding how an AI model performs decision (Miller, 2019). This mechanism serves as a fundamental requirement for establishing end-user trust in these tools, as it enables the disclosure of complex reasoning processes and the identification of potential flaws in neural networks. Secondly, generalizability represents the capability to acquire knowledge from limited training data to predict accurately in unseen situations (Wang et al., 2023a). Given the impracticality of exhaustively collecting and annotating vast amounts of data across various news domains, generalization ensures the affordable and sustainable deployment of data-driven fake news detection algorithms. Lastly, controllability encompasses the capacity for human guidance and intervention in the behavior of models (Ji et al., 2023a). This objective benefits models in understanding specific misinformation regulatory policies and rectifying deviations if necessary. While recent practices may satisfy the requirements of explainability (Xu et al., 2022; Liu et al., 2023) or generalization (Kochkina et al., 2018; Yue et al., 2023), they often fail to adhere to all three principles simultaneously.
To this end, we propose TELLER, a Trustworthy framework for Explainable, generaLizable and controLlabe dEtectoR, drawing inspiration from the dual-system theory3 (Daniel, 2017). This framework abstracts the existing pipeline of fake news detection into two components: the cognition and decision systems. As depicted in Fig. 1, the cognition system serves as the first step and is responsible for transforming meaningful human expertise from renowned journalism teams (Tsang, 2023; Sanders, 2023) into a set of Yes/No question templates that correspond to logic predicates. These decomposed questions are then answered using LLMs, which provide truth values for corresponding logic atoms.
On the other hand, the decision system, empowered by a differentiable neural-symbolic model (Cingillioglu and Russo, 2021), can integrate the output of the cognition system to deduce the final authenticity of input news by leveraging domain invariant logic rules learned from data automatically. This visible logic-based ensemble can mitigate the negative effects caused by inaccurate predictions of LLMs and allow for the correction of unreasonable
3System 1 provides tools for intuitive, imprecise, and unconscious decisions akin to deep learning, while system 2 handles complex situations requiring logical and rational thinking akin to symbolic learning (Booch et al., 2021).
rules through adjusting the weights in the model manually to align with human expertise.
Our framework ensures explainability by incorporating human-readable question templates (predicates) and a transparent decision-making process based on logic rules. This interpretability further enables the flexibility to adjust rules and enhances the model’s robustness against false LLM predictions, thereby guaranteeing controllability. Moreover, our model exhibits generalizability, attributed to the generalizable performance of LLMs combined with reliable human experience as guidance and the utilization of the neural-symbolic model, which can learn domain-generalizable rules.
To summarize, the contributions of this work include: 1) We introduce a systematic framework comprising cognition and decision modules, aiming to uphold three crucial principles for establishing a trustworthy fake news detection system: explainability, generalizability, and controllability. 2) We validate the effectiveness of our framework by conducting comprehensive experiments using various LLMs on four benchmarks. The results demonstrate the feasibility and trustworthiness of TELLER across different scenarios.

heading: 2 Related Work

heading: 2.1 Trustworthy AI
text: Establishing comprehensive trustworthiness in AI is non-trivial due to its multi-objective nature, including robustness, security, transparency, fairness, safety, and ethical standards (Jobin et al., 2019). Achieving such trustworthiness necessitates considering the entire lifecycle of an AI system, spanning from data preparation and algorithm design, development, and deployment to management and governance (Li et al., 2023; Eykholt et al., 2018). Recent researchers have explored diverse approaches to enhance AI trustworthiness across various goals and stages to address this challenge. For example, regarding algorithm design, several topics, such as transfer learning, federated learning, and interpretable AI, have been proposed to improve models’ robustness, security, and transparency. Moreover, the deployment of AI systems necessitates external government oversight, particularly for AGI (Bengio et al., 2023). Although our work focuses on enhancing the trustworthiness of detection systems from the algorithm design aspect, we acknowledge that there is still much room for improvement to achieve the ultimate goal.

heading: 2.2 Trustworthy Fake News Detection
text: Recent fake news detection research has witnessed a notable paradigm shift from prioritizing accuracy to considering trustworthiness. In line with our work, we primarily examine studies that aim to enhance algorithms’ explainability, generalizability, and controllability.
Regarding explainability, Cui et al. (2019); Xu et al. (2022); Liao et al. (2023) suggested obtaining key evidence for interpretation based on feature importance, while Liu et al. (2023) utilized logic clauses to illustrate the reasoning processing. However, these methods still need to be more transparent due to their probabilistic nature and complex architecture. Furthermore, another group of works (Huang and Sun, 2023; Hu et al., 2023; Yue et al., 2024; Qi et al., 2024), explored large generative language models (e.g., ChatGPT) and regarded the intermediate chain of thoughts as an explanation. Nevertheless, these explanations may not be reliable due to the hallucination phenomenon (Ji et al., 2023b) and the misalignment problem of AGI (Ji et al., 2023a). Moving on to generalizability, most methods, such as (Yue et al., 2023; Zhu et al., 2023; Qi et al., 2021), enhanced fake news detectors through transfer learning algorithms to learn domain-invariant features or domain-adaptive features. However, these methods inevitably introduce external costs of domain alignment, such as annotating domain labels. As for controllability, although some works (Silva et al., 2021; Mendes et al., 2023) incorporated the human-in-loop technique in data sampling and model evaluation, few works explore how to intervene and edit models to align with human expertise. More comparative discussion between TELLER and existing work can be found in Appendix E.

heading: 3 Methodology
text: Formally, given a piece of news T , the objective of the fake news detection task is to predict its label of truthfulness y ∈ Y where Y can fit in different levels of classification granularity. For example, in binary classification setting, Y = {true, false}, and T is identified as real (fake) when y is true (false).
As depicted in Fig. 2, TELLER involves two main components: cognition and decision systems. The cognition system decomposes human expertise into Yes/No question templates corresponding to logic predicates. When presented with a new input T , the templates and predicates can be instantiated
to form questions and logic atoms. By leveraging the parametric knowledge inside LLMs and gathering additional information from external tools (e.g., search engines), the cognition system can generate answers to these questions, represented as truth values of logic atoms. Then, the decision system takes these truth values as input and generates interpretable logic clauses to debunk misinformation by a neural-symbolic model, which can learn generic logic rules from data in an end-to-end manner.

heading: 3.1 Cognition System
text: To combat misleading information, existing deep learning-based algorithms fall short in gaining public trust, while fact-checking experts rigorously follow designated guidance and principles to facilitate transparent and fair evaluation. Our cognitive system aims to integrate the strengths of deep learningbased methods that can handle large-scale online information while maintaining the trustworthiness of manual checking.

heading: 3.1.1 Predicate Construction
text: To begin with, we describe the following symbol convention for clarity: calligraphic font Q and P for sets of question templates and predicates, capitalized letters Q, P, X for question templates, predicates, and variables, and corresponding lowercase letters q, p, x for instances of these entities (questions, logic atoms, values). The truth values of logic atoms are denoted by µ.
Inspired by the well-established fact-checking process in Table 5, we initially decompose it into a question template set, denoted as Q, containing eight questions as detailed in Appendix A.1. Each template Qi in Q consists of Ni variables and can be transformed into an Ni-ary logic predicate Pi(Xi,1, . . . ,Xi,Ni) in P . The logic semantics of Pi is interpreted as the affirmative answer to Qi and its truth value µi represents the probability that Pi holds. For instance, take Q1 (i.e., "Background Information: X1,1. Statement: X1,2. Is the statement true?") in Fig. 2 as an example. The corresponding predicate P1(X1,1,X1,2) can be explained as "Given the background information X1,1, the statement X1,2 is true".
For each predicate Pi(Xi,1, . . . ,Xi,Ni), we can instantiate the variables Xi,1, . . . ,Xi,Ni with the actual contents taken from any input news to obtain logic atoms. Since an input piece of news may contain multiple background information and statements (instantiations), we use k to denote the kth
instantiation where 1 ≤ k ≤ Ni∏ j=1
|Xi,j |. Here |Xi,j | indicates the total number of possible instantiations for variable Xi,j . Then we denote by pi,k the instantiated logic atom corresponding to the question qi,k. Next, we introduce how to acquire the truth value of each logic atom.

heading: 3.1.2 Logic evaluation with LLMs
text: While decomposed questions can provide a comprehensive explanation of how the decision is made (Chen et al., 2022; Fan et al., 2020), directly answering these questions poses a challenge due to the impracticality of annotating enormous data to train multiple models for different questions. To address this issue, we resort to the more general-purpose LLMs (e.g., FLAN-T5 (Chung et al., 2022), Llama2 (Touvron et al., 2023b), and GPT-3.5) as the foundation for effectively answering these questions. Existing LLMs can be categorized into two groups: LLMopen, such as FLAN-T5 and Llama2, where the logits of output vocabulary can be obtained, and LLMclose, such as GPT-3.5, where the logits are not accessible.
To ensure compatibility with both categories of LLMs, we propose two strategies to obtain the final truth values of logic atoms. Concretely, we first input the question qi,k with a suffix (i.e., "Yes or No? Response:") to LLMs in order to measure their preference for the affirmative answer "Yes" versus the negative one "No". This preference is subsequently used to compute the truth value of the corresponding logic atom pi,k.
For LLMopen, we follow (Gallego, 2023; Burns et al., 2023) to obtain pre-softmax logits of "Yes" and "No" tokens, denoted as vY es and vNo respec-
tively. Compared with post-softmax logits, presoftmax logits can mitigate the influence of other tokens in output vocabulary, particularly when LLMs tend to generate irrelevant tokens that may result in vY es or vNo becoming zero. Then the truth value µ for the logic atom p (here we omit the underscript i, k for ease of illustration) can be obtained as follows:
µ = 2 evY es
evNo + evY es − 1. (1)
For LLMclose, we sample m times during decoding and count the frequency of "Yes" and "No" responses as mY es and mNo. Then we compute
µ = 2 mY es
mNo +mY es − 1. (2)
In either case, µ is in the range of [−1, 1]. When µ ∈ [−1, 0), µ ∈ (0, 1], and µ = 0, the corresponding logic atom p is evaluated as false, true, and unknown, respectively. Once the truth values of all logic atoms for a single predicate Pi (corresponding to a single question template) are obtained, we concatenate them as one vector, denoted as µi. Then we concatenate the value vectors for all predicates as the input for the final decision system.
In conclusion, our cognition system can generate diversified questions and logic atoms based on the input news T . These human-readable entities enhance explainability by showcasing potential intermediate reasoning steps and ensure controllability by allowing adjustments to Q and P . Moreover, combining human expertise and LLMs provides the basis for the cognition system’s satisfactory generalization performance in unseen domains.

heading: 3.2 Decision System
text: After acquiring responses to all questions, it is imperative to develop a decision system to effectively aggregate them to predict the label of the input news T while preserving trustworthiness in the reasoning process. However, prevalent heuristic strategies (e.g., majority voting) lack the flexibility to handle complex relationships among different questions and cannot tolerate false predictions, and deep-learning-based models cannot be comprehended literally by humans (Wang et al., 2023b).
Hence, we utilize a neural-symbolic model, named Disjunctive Normal Form (DNF) Layer (Cingillioglu and Russo, 2021; Baugh et al., 2023), as our decision system. This model includes conjunctive layers (SL∧) and disjunctive layers (SL∨), which can progressively converge to symbolic semantics such as conjunction ∧ and disjunction ∨ respectively during model training. Consequently, this model can automatically learn logic rules from data in an end-to-end manner, capturing generalizable relationships between logic predicates and the target label. As illustrated in Fig. 2, we stack C conjunctive layers SL∧ beneath |Y| disjunctive layers SL∨ to construct the DNF Layer, where each SL∨ corresponds to a truthfulness label y ∈ Y .
However, the original DNF Layer proposed in (Cingillioglu and Russo, 2021) is not directly applicable to our work due to two issues. Firstly, the truth value of logic atoms µ ranges in [−1, 1], while the original model can only handle values of −1 and 1. Secondly, each logic atom in the original DNF Layer is treated differently which loses logic semantics where atoms for the same logic predicate should share similar functionality. To address the aforementioned challenges, we propose a modified DNF layer which takes continuous values µ ∈ [−1, 1] as input and assigns the same weight for those atoms instantiated from the same logic predicate. The detailed description of our modified DNF layer can be found in Appendix G.
More concretely, in our proposed DNF Layer, every SL∧ takes truth values µ of all logic atoms obtained in the cognition system as input, aiming to learn a conjunctive clause conj =
∧ pi,k∈A pi,k
where A ⊆ {p1,1, . . . , pN,MN }, referring to a subset of the complete logic atoms, and outputs the truth value of this conjunctive clause. Subsequently, each SL∨ receives the truth values of C conjunctive clauses to represent a disjunction of these conjunctions: ∨ c∈C conjc where C ⊆ {1, . . . , C}, referring
to a subset of all conjs. It then outputs the truth value of this disjunction formula, corresponding to the final probability that the input news T is identified as the label y. Hence, each label y will be associated with a DNF clause learned by the DNF layer. Intuitively, the conjunction simulates the idea that if the input news T gives affirmative answers to some questions simultaneously, it is highly probable that it should be assigned to label y. On the other hand, the disjunction provides more flexibility by considering different alternatives (the output is true if at least one of the conj is true) which makes the final decision less sensitive to incorrect atom values due to wrong predictions given by LLMs. For example, assume the learned rules are conj1 ∨ conj2 where conj1 = p1,1 ∧ p1,2 and conj2 = p2,1 ∧ p3,1. Suppose conj1 is true, then we can conclude that conj1 ∨ conj2 is true even if conj2 gives an incorrect value.
Last but not the least, we apply softmax function to the output of all disjunction layers SL∨ to obtain the probability z ∈ R|Y| for all possible labels. The entire decision system can be trained in an end-toend fashion by minimizing the cross-entropy loss function as below:
L = − |Y|∑
l=1
I(yl = yT ) log zl, (3)
where yT represents the ground truth label of T . During inference, we select the label corresponding to the highest value in z as the final result.
In summary, our decision system can extract interpretable symbolic rules from data that exhibit robustness across diverse domains and enable intervention by adjusting weights in the DNF Layer to align with prior knowledge (refer to Appendix C).

heading: 4 Experiments
text: In this section, we present the experiment setup and demonstrate the feasibility, explainability, generalizability and controllability of TELLER through extensive experiments.

heading: 4.1 Experimental Setting
text: Dataset. We conducted experiments using four challenging datasets, namely LIAR (Wang, 2017), Constraint (Patwa et al., 2021), PolitiFact, and GossipCop (Shu et al., 2020). LIAR comprises the binary classification and multi-classification setting with six fine-grained labels for truthfulness ratings. Moreover, Wang (2017); Alhindi et al.
(2018) curated relevant evidence (e.g., background information), serving as gold knowledge in an open setting. Constraint, PolitiFact and GossipCop are binary classification datasets related to COVID-19, politics, and entertainment domains, respectively. LLMs. We select the open-source FLAN-T5 and Llama2 series, which encompass various parameter sizes, as large language models for constructing the cognition system. We also conduct experiments using GPT-3.5-turbo on the LIAR dataset to examine the versatility of our framework. Baselines. We compare our model against Direct, Few-shot Direct, Zero-shot COT, Few-shot COT, Few-shot Logic. The baselines suffixed with Direct involve prompting large language models (LLMs) to predict the label of input news directly; those suffixed with COT utilize chain-of-thought techniques to enhance the performance of LLMs; those suffixed with Logic replace the thought process in COT with questions paired with their answers. We exclusively implement COT-related methods using GPT-3.5-turbo because they show no improvement over Direct on FLAN-T5 and Llama 2, as shown in Table 12. Additionally, we compare with small models, including BERT and RoBERTa, analyzed in Appendix E. Implementation Detail. We evaluate the performance of our framework using the accuracy and Macro-F1, which accommodates class imbalance. For each dataset, we train our decision system using the training split; select the optimal model based on its performance on the validation split; and report the results on the test split. To assess the generalizability of our model, we consider each dataset as a separate domain and train our models using the train split from source domains; choose the best model on the validation split of source ones; and report results on the test split from the target domain. Moreover, to highlight the robustness of our framework, we keep all hyperparameters fixed in each setting. Details of the experiment setting, data leakage analysis, baselines, and model training are elaborated in Appendix B.

heading: 4.2 Feasibility Study
text: To validate the feasibility of our framework, we compare it against multiple baselines across a wide range of LLMs and scenarios (e.g., different classification granularities) in Table 1 and Table 2. These results uncover two crucial findings listed below:
Firstly, our framework demonstrates satisfactory
performance in fake news detection tasks. Specifically, in the binary classification setting, TELLER achieves an accuracy of approximately 76% on the GossipCop dataset and over 80% on the other three datasets. Notably, when utilizing Llama 2 (13B) to drive the cognition system, TELLER outperforms all GPT-3.5-turbo based methods by a significant margin. These results highlight the effectiveness of TELLER in distinguishing between fake and genuine news. In the multi-classification setting on the LIAR dataset, our framework consistently outperforms Direct for FLAN-T5 and Llama2 series, even though these models may struggle to discriminate fine-grained labels. This observation underscores the capability of our decision system to mitigate the negative influences of noisy predictions in the cognition system, effectively unleashing the potential of LLMs through logic-based aggregation of answers to decomposed questions.
Secondly, our framework exhibits significant potential for the future. In the binary classification setting across four datasets, TELLER consistently outperforms Direct in terms of accuracy and macroF1 scores by an average of 7% and 6%, respectively. Considering the swift improvement of LLM intelligence, these results imply that the performance of our framework is likely to scale with the evolution of LLMs. Additionally, due to the notable performance difference between closed and open settings on the LIAR dataset, it is promising to integrate external tools to acquire extensive evidence from credible sources, such as official government websites, to enhance the performance of our systems.

heading: 4.3 Explainability Verification
text: Explainability is a fundamental factor for establishing trust in AI technology. We demonstrate that our framework satisfies this aspect through its inherent mechanism and the visualization of rules.
Unlike approaches that rely heavily on LLMs, our cognition system incorporates expert knowledge to construct a more well-grounded worldview by generating well-defined question templates and logic predicates. Moreover, our decision system can learn interpretable rules from data to deduce logic clauses to debunk fake news by converging implicit parameters to conjunctive and disjunctive semantics. These symbolic units (e.g., questions and logic atoms) and the interpretable DNF Layer contribute to our framework’s overall explainability and transparency.
highlighted in bold. The number indicates that the performance of w/ Intervention is worse than TELLER. The number with ↑ indicates the performance gain of TELLER over Direct.
tion. The best results for each setting are highlighted with bold numbers. The number and the number with ↑ have the same meaning as in Table. 1.
However, as the number of conjunctive and disjunctive layers grows, it is difficult for human beings to investigate logic rules derived from our decision system. To address this issue, we propose a strategy to prune unnecessary weights in the DNF Layer. For example, we present the rules extracted from the pruned model for GossipCop in Table 4, where each conjunctive clause identifies one candidate rule. The pruning algorithm and rules for other datasets are described in Appendix C.
Table 4 can be interpreted as learning DNF rules for both true and false labels of input news. Specifically, the true label is predicted if either ¬conj34 or ¬conj43 is true, i.e., either ¬P2 ∧P3 ∧P6 ∧P8 or P3 ∧ P6 ∧ P8 is false when removing the negation. Given the semantics of these logic predicates
shown in Table 6, we know that P2, P3 and P8 check the consistency between the background information and a given message, whereas P6 scrutinizes improper intention from the message alone. On the other hand, the news will be predicted as false if conj27 is true, i.e., P4 is false which means that the background information in the message is neither accurate or objective according to Table 6.

heading: 4.4 Generalizability Verification
text: Ensuring the generalization ability of fake news decision systems is vital for their sustainable and practical deployment. As observed in Table 3, TELLER consistently outperforms Direct across all domains and LLMs without the assistance of any generalization algorithm, while only exhibiting a negligible

heading: 4.5 Controllability Verification
text: Controllability ensures that fake news detection systems are subject to effective human oversight and intervention. We demonstrate TELLER satisfies this attribute from two aspects. Firstly, we verify the feasibility of manually rectifying rules learned by our decision system that may exhibit irrational behavior. For instance, we observe that P3 (i.e., "The message contains adequate background information") should have a positive logical relation with Ptrue instead of negation in Table 4. To correct this, we perform a manual adjustment by setting the corresponding weight to zero, effectively removing P3 from the logic rule. However, this modification only leads to a negligible improvement in the test split. Further investigation reveals that the truth value of logic atoms pertaining to P3 of most real samples is negative, possibly due to the prefer-
ence of LLMs. This suggests the superiority of our logic-based decision system in reducing the negative effect of incorrect predictions made by LLMs automatically. Secondly, we simulate human experts by intervening in the actions of our cognition system. We achieve this by guiding LLMs to expand the question template set Q using Algorithm 1, referred to as w/ intervention in Tables 1 and 2. The new question template set for intervention is shown in Table 7. The results consistently indicate that w/ intervention outperforms TELLER, highlighting the potential of LLMs as an agency for automatically regulating the behaviors of the cognition system. Thus, our framework ensures a comprehensive control mechanism by simultaneously facilitating human and AI agents’ oversight.
Furthermore, we conduct additional experiments to verify the effectiveness of the DNF Layer in logic formulation over other decision systems, namely decision trees, Naive Bayes classifiers and MLP. We replace the DNF Layers with these three algorithms to derive the final decisions. The results are shown in Tables 15 and 16 for in-domain and cross-domain settings, respectively in Appendix D.

heading: 5 Conclusion
text: In this work, we address the limitations of existing fake news detection methods, which struggle to establish reliability and end-user trust. To tackle this issue, we identify three crucial aspects for constructing trustworthy misinformation detection systems: explainability, generalizability, and controllability. By prioritizing these principles, we propose a dual-system framework TELLER that incorporates cognition and decision systems. To validate our framework’s feasibility, explainability, generalizability, and controllability, we conduct extensive experiments on diverse datasets and LLMs. These results affirm the effectiveness and trustworthiness of our approach and highlight its significant potential through evolving both subsystems in the future. While we achieve trustworthiness from an algorith-
mic perspective, we emphasize the importance of further research to improve the trustworthiness of the entire lifecycle of fake news detection systems.
Limitations
We identify three main limitations of our work. Firstly, although our framework focuses on enhancing the trustworthiness of fake news detection algorithms, trustworthiness is also influenced by other stages of the AI system lifecycle, such as data collection and deployment. Given the advancements in AI techniques and the importance of online information security, we encourage future research to address the challenges of building trustworthy AI systems comprehensively.
Secondly, as shown in Table 1, integrating external tools to acquire high-quality background knowledge significantly improves the performance of fake news detection systems. However, collecting information that can effectively support detection tasks using such tools is non-trivial due to the complexities of open-domain information retrieval and the diversity of news content. For instance, we search for background information by inputting check-worthy claims of P1 into a search engine and filter out as much useful information as possible using GPT-3.5-turbo. However, integrating this evidence led to a slight performance drop on Constraint, PolitiFact, and GossipCop datasets (Due to page limitations, we do not include this experiment in our paper). Therefore, we leave this for future research.
Thirdly, despite the excellent and robust performance of our decision system, especially in generalization ability, the expressiveness of the DNF Layer is still limited due to its simple architecture. For example, the DNF Layer learns rules from data without considering the semantics of logic predicates. It may be crucial to develop more powerful decision models to fully unleash the potential of large language models, such as incorporating the semantics of logic predicates. However, given the low-dimensional input and the need for trustworthiness, the DNF layer remains a prudent choice. Moreover, there also exists a trade-off between trustworthiness and the complexity of the decision system.
Ethics Statement
This paper adheres to the ACM Code of Ethics and Professional Conduct. Specifically, the datasets we
utilize do not include sensitive private information and do not pose any harm to society. Furthermore, we will release our codes following the licenses of any utilized artifacts.
Of paramount importance, our proposed dualsystem framework serves as an effective measure to combat fake news and safeguard individuals, particularly in the current era dominated by large generative models that facilitate the generation of deceptive content with increasing ease. Moreover, our approach fulfills explainability, generalizability, and controllability, thereby mitigating concerns regarding the security of AI products and enabling their deployment in real-world scenarios.

heading: Acknowledgment
text: This research is supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier 1 (023618-00001, RG99/23), Research Matching Grant Scheme, Hong Kong Government (9229106). We thank the SAC, PC and Editors of ACL 24/ARR February to read our appealing letter and coordinate the review.

heading: A Details of Cognition System
text: Deep learning has attracted increasing interest from academia and industry (Xiao et al., 2023, 2024). Unlike convolutional deep learning-based fake news detection frameworks that classify in a latent space, the cognition system of TELLER, aims to emulate human fact-checking experts by complying with specific policies to ensure transparency and controllability of the detection process. In this section, we describe the construction of the set of question templates Q and Q′ for TELLER and w/Intervention respectively in Appendix A.1. Furthermore, we introduce a trick for batch training by fixing the number of logic atoms for different inputs in Appendix A.2 and outline some potential techniques for further improvement of the cognition system in Appendix A.3.
A.1 Construction of Question Templates To provide an overview, we present the referenced human-checking process in Table 5. In this table, Steps I, VI and VII are excluded from detection algorithms, as they either fall into the preliminary procedures or the post-processing stages of the fake news detection pipeline. These steps may involve data crawling, human-computer interaction, machine translation, etc. As a result, we concentrate on the other steps.
Subsequently, we decompose the process into a Yes/No question template set Q, where each template Qi in Q corresponds to a predicate Pi in the predicate set P . All question templates and their corresponding predicates are listed in Table 6. Specifically, for Q1, our objective is to determine the trustworthiness of statements in the input news. Here, statements represent crucial information in news articles, playing a vital role in debunking misinformation. Additionally, extracting statements from news is a challenging task. While previous studies like Liao et al. (2023); Fung et al. (2021) used pre-trained language models to generate summaries as statements, we choose to utilize GPT3.5-turbo to generate statements for simplicity in implementation. The prompt used for this purpose is as follows:
To verify the MESSAGE, what are the critical claims related to this message we need to verify? Please use the following format to answer. If there are no important claims,
answer “not applicable”.
MESSAGE: CLAIM: CLAIM:
MESSAGE: $MESSAGE$.
Then, we replace the "$MESSAGE$" with input news and take the generated claims as statements for Q1 (P1).
Additionally, when verifying the controllability of our framework, we propose adjusting the question template set to deal with the diversity of fake news. While this adjustment should be done by fact-checking experts to ensure the reasonableness of new questions, our empirical findings demonstrate the feasibility of guiding large language models, such as GPT-3.5-turbo, to generate new question templates. These templates are then manually filtered by us to create the final question template set Q′, and the corresponding predicate set P ′ for intervention, as outlined in Algorithm 1. Such human verification is incorporated into our intervention method to ensure more controllability because the main point of controllability is to intervene via human knowledge instead of relying on models entirely. Moreover, such manual checking is not time-consuming, with only a few candidate questions being generated. Table 7 presents these newly added question templates and predicates. The prompt R used in this algorithm is as follows:
Write some questions that can be used to determine whether a news report is misinformation. The questions should be answerable by large language models in a close-book situation without requiring additional information. Please format each question using the 
 and 
 tags, such as 
A question
.
A.2 Trick for Batch Training To enable batch training, we fix the number of logic atoms, denoted as Mi for each predicate Pi. Specifically, If Mi < Ni∏ j=1 |Xi,j |, we randomly select Mi atoms. Conversely, if Mi > Ni∏ j=1
|Xi,j |, we pad the vector by 0 accordingly. In the end, µ can be repre-
sented as [µ1,1, . . . , µ1,M1 , . . . , µN,1, . . . , µN,MN ], where µ ∈ RM and M = N∑ i Mi.
A.3 The Potential of Cognition System It is noteworthy that specific techniques can be employed to improve the performance of our cognitive system. For instance, when obtaining the answers to questions as truth values for corresponding logic atoms in Sec. 3.1.2, we exclusively consider "Yes" and "No" tokens. However, considering the relationship between model outputs and final predictions, "Right" and "Wrong" tokens can also be suitable candidates. Therefore, drawing motivation from (Gao et al., 2021; Cui et al., 2022), existing manual or automatic verbalizer techniques that establish mappings between diverse model outputs and final labels can be leveraged to enhance performance. Additionally, the ensemble of prompts, similar to "Yes or No? The answer is: ", has proven effective for the "Yes" and "No" classification task in (Gallego, 2023). Consequently, our dual-system framework exhibits substantial potential for future improvements in the cognitive system.
Algorithm 1 Question Template Generation for Intervention Algorithm
Input: Prompt R, the original question template set Q, and a copy of Q denoted as Q̂ Output: The question template set Q′ for intervention
1: Set the number of iteration steps as T 2: for Iteration t = 1, . . . , T do 3: Use R to guide GPT-3.5-turbo in generating a set of new question templates Q′ 4: for each question template Q′i in Q′ do 5: Compute the average similarity score be-
tween Q′i and all templates in Q̂ using Sentence BERT.
6: end for 7: Add Q′i ∈ Q′ with the lowest similarity score to Q̂. 8: end for 9: Q′ = Q̂ \ Q 10: Manually refine Q′ by removing duplicate and impractical templates that are non-verifiable through LLMs, resulting in the final Q′.

heading: B Details of Experimental Setting
text: B.1 Datasets LIAR is a publicly available dataset for fake news detection, sourced from POLITIFACT.COM. This dataset comprises six fine-grained labels for truthfulness ratings: true, mostlytrue, halftrue, barelytrue, false, and pantsfire. To align with the binary classification problem, we merge true, mostlytrue into true and merge barelytrue, false, and pantsfire into false, following (Liao et al., 2023). Moreover, Wang (2017); Alhindi et al. (2018) curated relevant evidence from factchecking experts (e.g., publisher information, background information, etc.), which serve as gold knowledge in an open setting. Constraint is a manually annotated dataset of real and fake news related to COVID-19. We adopt the data pre-processing procedures described in (Patwa et al., 2021), which involve removing all links, non-alphanumeric characters, and English stop words. PolitiFact and GossipCop are two binary classification subsets extracted from FakeNewsNet (Shu et al., 2020). The PolitiFact subset comprises political news, while the GossipCop subset comprises entertainment stories. To optimize experimental costs and adhere to maximum context limitations, we exclude news samples longer than 3,000 words.
For dataset partitioning, we follow the default partition if specified; otherwise, we use a 7:1:2 ratio. Table 8 presents the statistics of each dataset.
B.2 Data Leakage Analysis In our work, we used four publicly available datasets to evaluate our proposed framework, TELLER. To begin with, following recent work (Oren et al., 2023), we refer to the problem of data leakage (data contamination) as the situation where the pretraining and finetuning dataset of LLMs contains the testing splits of datasets used in our work. To mitigate the risks associated with data leakage during our evaluation, we took three precautionary steps to ascertain that the probability of the
occurrence of data leakage is particularly low: Manual Check: For the open-public Flan-T5 and Llama 2 series, we double-checked the dataset cards of these two model families and did not find a data leaking problem. Concretely, we checked the finetuning data (i.e., Appendix F Finetuning Data Card of (Chung et al., 2022)) and pre-training data (i.e., C4 dataset in Sec. 3.4.1 of (Raffel et al., 2020)) for the family of Flan-T5 models and checked the pre-training data of Llama 1 (i.e., Sec. 2.1 of (Touvron et al., 2023a)) while the pretraining data of Llama 2 seems not publicly available yet. Assumption Experiment: If data leakage were present, we would expect the detection accuracy of LLMs to scale with model size, given that the memorization ability of LLMs is positively correlated to the size of models empirically (Kaplan et al., 2020). However, our results in Tables 1 and 2 do not support this hypothesis, suggesting a low likelihood of data leakage. Empirical Analysis: Some measurements for data leakage exist (Oren et al., 2023; Touvron et al., 2023b). We used the Sharded Rank Comparison Test, proposed by Oren et al. (2023) to analyze potential data leakage in our datasets on Llama2 (7B). We did not analyze the data leakage problem of the GPT series here due to the limited and expensive access, while Llama2 and FLAN-T5 are LLMs we mainly use. The results in Table 9 indicate no data leakage risk for Llama2 (i.e., when the p-value> 0.05 means there is no data leakage risk). However, these measurements of data leakage problems may compromise the accuracy of determining whether dataset contamination occurs and have contributed to evaluation performance sometimes because of many confounding factors (a detailed discussion in A.6 of (Touvron et al., 2023b)).
While TELLER has shown satisfactory accuracy on four open-public datasets, our main contribution is the systematic framework that adheres to explainability, generalizability, and controllability. As per our experimental results, TELLER’s detection performance can scale by integrating more powerful LLMs and external techniques, demonstrating the effectiveness of our approach as LLMs and related techniques continue to evolve. Consequently, even if the possible data leakage problem may have a deceptively good influence on the detection accuracy, we argue that it will not decrease our work’s contribution.
B.3 Illustration of Different Baselines We compare our model against Direct, Few-shot Direct, Zero-shot COT, Few-shot COT, Few-shot Logic. Direct utilizes LLMs to calculate the probability of each label using Eqs. 1-2 and then selects the label with the highest likelihood as the predicted label. Building upon Direct, Few-shot Direct incorporates demonstration samples with known labels as contextual information to enhance the model’s performance. Zero-shot COT and Fewshot COT employ the chain-of-thought (COT) technique (Wei et al., 2022), enabling LLMs to engage in step-by-step reasoning. While Zero-shot COT immediately adds the prompt "Let us think step by step!", Few-shot COT provides multiple COT exemplars. For Few-shot Logic, we replace the thought process in COT with instantiated questions accompanied by corresponding answers generated by our cognition system. We omit comparisons with Few-shot and COT-based prompt methods for Llama 2 and FLAN-T5 because COT prompts have been found to yield performance gains basically when used with models of approximately 100B parameters (Wei et al., 2022), and both Few-shot and COT-based methods show no additional improvement over Direct as revealed by Table 12, we exclusively implement COT-related methods using GPT-3.5-turbo.
Below we show the templates for these five baselines for the fake news detection task in the closed setting without access to any external knowledge source. Direct:
Message: $MESSAGE$. Is the message $Label$? Yes or No? Response:
Then, we replace the "$MESSAGE$" with input news, "$Label$" with candidate truthfulness labels. Few-shot Direct:
Following given examples to answer Yes/No questions.
Message: Says the Annies List political group supports third-trimester abortions on demand. Is the message true? Yes or No? Response: No
Message: Says the Annies List political group supports third-trimester abortions on demand. Is the message false? Yes or No? Response: Yes
(· · · more examples here · · ·)
Message: $MESSAGE$. Is the message $Label$? Yes or No? Response:
Then, we replace the "$MESSAGE$" with input news, "$Label$" with candidate truthfulness labels. Furthermore, during the testing phase, the examples are randomly selected from the training set. Zero-shot COT:
You will be provided with a statement, and your task is to classify its truthfulness into one of two categories: true and false. Message: $MESSAGE$. Let’s think step by step and give answer with the suffix “So the final answer is".
Then, we replace the "$MESSAGE$" with the input news. Few-shot COT:
You will be provided with a statement, and your task is to classify its truthfulness into one of two categories: true and false.
Example One Message: Says the Annies List political group supports third-trimester abortions on demand. Let’s think step by step and give answer with suffix “So the final answer is". Annie’s List was comfortable with candidates who oppose more limits on late-term abortions
while he also supported candidates who voted for more limits this year. Both dose not mention of third-trimester abortions. So the final answer is false.
(· · · more examples here · · ·)
Message: $MESSAGE$. Let’s think step by step and give answer with the suffix “So the final answer is".
Then, we replace the "$MESSAGE$" with the input news. Few-shot Logic:
You will be provided with a statement, and your task is to classify its truthfulness into one of two categories: true and false.
Example One Message: Says the Annies List political group supports third-trimester abortions on demand. Decomposed Questions: (1) Statement: The Annies List is a political group. Is the statement true? Yes (2) Statement: The Annies List supports third-trimester abortions. Is the statement true? No (3) Did the message contain adequate background information? False
(· · · more examples here · · ·)
Message: $MESSAGE$. Let’s think step by step and give answer with the suffix “So the final answer is".
Then, we replace the "$MESSAGE$" with the input news.
Additionally, we conducted supplementary experiments comparing our framework with other non-LLM-based misinformation detectors (referred to as small models following convention), including BERT4 and RoBERTa5, presented in Tables 13 and 14 for in-domain and cross-domain settings,
4https://huggingface.co/bert-base-uncased 5https://huggingface.co/FacebookAI/
roberta-base
respectively. These small models are finetuned on misinformation detection datasets. Especially for the cross-domain setting, we consider each dataset as a separate domain and fine-tune these models using the train split from source domains, choose the model on the validation split of source ones, and report results on the test split from the target domain. Moreover, we do not compare our framework here with existing transfer learning algorithms because we assume the domain label and target domain data are unavailable in our work.
B.4 Model Training for Decision System In the decision system of our framework, we employ the DNF Layer to learn human-readable rules from data differentially. To train this model, we utilize the Adam optimizer with a learning rate of 1e-3. Regarding the hyperparameters, we search the conjunction number C within the range [10, 20, 30, 40, 50], and the weight decay within the range [1e-3, 5e-4, 1e-4]. Furthermore, to showcase the superiority of our approach, we maintain consistent hyperparameters across different LLMs in each setting. For instance, all hyperparameters of TELLER in the closed setting for the binary classification task on the LIAR dataset remain unchanged. The batch size is set to 64, and the number of epochs is set to 30. Additionally, we progressively converge the model towards symbolic semantics by adjusting δ (refer to Appendix G for detail) to 1 or -1 before the first 15 epochs using exponential decay.

heading: C Details of Explainability Study
text: To enhance the accessibility of the rules generated by the DNF Layer, we propose a pruning algorithm that extracts more concise logic clauses by eliminating insignificant weights. The algorithm is described in Algorithm 2. Furthermore, to demonstrate the explainability of our framework, we visualize the extracted rules obtained from the pruned model for Constraint, PolitiFact, and GossipCop datasets in Tables 10, 11 and 4, respectively. In these tables, Ptrue and Pfalse represent the proposition that the input news is identified as true or false, respectively. In our visualization experiments, we employ Llama2 (13B) as the LLM in the cognition system. We set the number of conjunctive layers C as 50, the performance drop threshold ϵ as 0.005, and b as 0.0001 to reduce the number of conjunction clauses. More details regarding these parameters can be found in Appendix G.
Similar to Symbolic AI, such as expert systems, our learned rules can be intuitively translated into natural language. For instance, consider the rules provided in Table 4, conj27 = ¬P4, Pfalse = conj27 and the semantics of P4 in Table 6 is “The background information in the message is accurate and objective”, Pfalse can be translated as “The input message (news) is false when the background information in the message is not accurate and objective".
Algorithm 2 Pruning Algorithm for the DNF Layer Input: Trained DNF Layer Φ, performance drop threshold ϵ Output: Pruned DNF Layer Φ′ and extracted rule set R 1: Initialize R′ as an empty set 2: Initialize R by extracting rules from Φ 3: Initialize Φ′ using Φ 4: while |R′| ̸= |R| do 5: Initialize R by extracting rules from Φ′ 6: Prune disjunctions if the removal of a disjunction
results in a performance drop smaller than ϵ 7: Prune unused conjunctions that are not utilized by any disjunction 8: Prune conjunctions if the removal of a conjunction results in a performance drop smaller than ϵ 9: Prune disjunctions that use empty conjunctions 10: Prune disjunctions again if the removal of a disjunction results in a performance drop smaller than ϵ 11: Update the pruned model as Φ′ and extract rules from Φ′ to obtain R′; 12: end while

heading: D Comparison with Different Decision Models
text: In our work, we utilize the DNF Layer to construct our decision system, guaranteeing explainability and controllability. However, there are also other alternatives, such as existing neural symbolic architectures and interpretable machine learning algorithms. By comparing the DNF Layer with these candidates, we demonstrate that our dual-system framework can achieve better performance by inventing a more effective decision model to unleash the ability of LLMs.
While existing neural symbolic architectures can extract useful rules from data (Booch et al., 2021), they indeed have certain limitations. Firstly, these architectures often require complex mechanisms to implement logical operations, which makes them unsuitable for immediate application in fake news detection tasks. For example, Qu et al. (2021); Cheng et al. (2023) developed neural-symbolic models for knowledge graph completion, but their reliance on well-defined graph structures makes them infeasible for our task. Secondly, these architectures often suffer from efficiency issues. For instance, δLP proposed in (Evans and Grefenstette, 2018) had high computational complexity, and HRI (Glanois et al., 2022) was incompatible with batch training, which externally required users to predefine rule templates to constrain the search space. Furthermore, to the best of our knowledge, there may be no neural-symbolic framework available that can simultaneously handle the challenges of missing values and multi-grounding problems (i.e., one predicate can be instantiated as multiple logic atoms), which are common in our tasks. Therefore, we acknowledge the need for future research to develop a more suitable and powerful neuralsymbolic framework in the context of fake news detection.
Since each dimension in µ is precisely bonded to a question template (logic predicate), we can employ traditional machine learning classification algorithms, including decision tree6, naive Bayes Classifier7 and multi-layer perceptron (MLP), to replace the DNF Layer to drive our decision system, while maintaining partial aspects of trustworthy AI. Therefore, we compare the DNF Layer with these
6https://scikit-learn.org/stable/modules/tree. html
7https://scikit-learn.org/stable/modules/ naive_bayes.html
three methods in both in-domain and cross-domain settings on three datasets, shown in Tables 15 and 16, respectively.
According to the results, we conclude that the decision tree and MLP perform better when the training and testing data are from the same domain. Meanwhile, the naive Bayes Classifier demonstrates more satisfactory generalization performance in cross-domain experiments across various LLMs. This implies that our proposed dualsystem framework shows potential in developing a more powerful decision module, such as an ensemble of these algorithms. However, the DNF Layer still outperforms these three methods in most cases when using Llama2 (13B) as the driver of the cognition system, achieving a better trade-off between accuracy and generalization ability. Moreover, the DNF Layer also exhibits advantages over these methods in terms of its ability to handle missing values and multi-grounding problems, as well as its flexibility in efficiently searching logic rules in a large space, whereas the decision tree is constrained by depth and width.

heading: E Comparison with Existing Work on Three Principles
text: It is imperative to compare our LLM-based framework with prevailing misinformation detection methods across dimensions of explainability, generalizability, and controllability. We conduct additional experiments to compare with small models to demonstrate the strength of TELLER in generalizability. However, quantitatively measuring explainability and controllability in deep learning is presently challenging (Li et al., 2023), necessitating substantial research endeavors. Generalizability: We conduct additional experiments comparing with small models (BERT and RoBERTa) in Tables 13 and 14 for in-domain (with finetuning) and cross-domain settings, respectively. These results illustrate that small models only outperform TELLER in an in-domain setting, but TELLER excels in zero-shot generalization (around 30% improvement in terms of Accuracy and F1-Score) and can handle more complex misinformation detection tasks, exemplified by superior performance on the LIAR dataset. This advantage aligns with many real scenarios, characterized by the absence of training data and the presence of sophisticated misinformation (Pelrine et al., 2023). Consequently, TELLER proves significantly advantageous in such contexts.
Moreover, the feasibility and adaptability of TELLER are underscored by the resource-intensive nature of gathering adequate data for small models. Additionally, our framework, as a general and systematic framework, can achieve better in-domain accuracy by integrating small fine-tuned models into our cognition system, treating their binary classification outputs as truth values Explainability: Current interpretative methods using feature importance, attention visualization, and multiview learning (Cui et al., 2019; Xu et al., 2022; Liao et al., 2023; Ying et al., 2023) may be unreliable and possess limited explanatory power, as indicated by (Liu et al., 2022). Another approach (Liu et al., 2023), employing neural-symbolic learning for multimodal misinformation detection, falls short of clause length and readability caused by its unexplainable predicates. Unlike small-modelbased misinformation detectors, our cognition system incorporates expert knowledge to construct a more well-grounded worldview, which is unrealistic for small models to achieve. Furthermore, another group of work (Huang and Sun, 2023;
Hu et al., 2023; Yue et al., 2024; Qi et al., 2024) explored large generative language models (e.g., ChatGPT) and regarded the intermediate chain of thoughts as an explanation. Nevertheless, these explanations may not be reliable due to the hallucination phenomenon and the misalignment problem of AGI (Chen and Shu, 2023). Compared with them, our decision system can learn interpretable rules to explicitly aggregate generated logic atoms for further double-checking instead of relying on the implicit aggregation of LLMs. Controllability: As shown in Sec. 2.2, some studies integrated human-in-loop techniques (Wu et al., 2022) for data sampling and model evaluation, whereas our framework prioritizes algorithm design. Moreover, while recent RLHF techniques (Rafailov et al., 2023) can incorporate human guidance in model behaviors based on reinforcement learning, they indeed require external high-quality fine-tuning data and sophisticated finetuning. In contrast, our framework achieves controllability through natural manipulation of the question set and logic rules in our cognition and decision systems
In summary, TELLER effectively addresses challenges in explainability, generalizability, and controllability. We also emphasize TELLER is a general framework and does not sacrifice performance for explainability, generalizability, and controllability, considering its potential to integrate finetuned small models to improve the in–domain performance.

heading: F Cost Analysis
text: One crucial consideration of TELLER is the expense associated with the N queries to LLMs. The specific costs, including inference time and token cost, will be discussed below. Inference Time: Due to the limited access times of GPT-3.5-turbo in minutes, it is time-consuming to perform N queries for our framework. However, it is worthwhile that it may also require multiple queries for GPT-3.5-turbo to adopt self-consistency and least-to-most prompt techniques to achieve the comparable performance as our framework, given there is a performance gap between Direct and TELLER in Table 1.
Furthermore, our experiments indicate that utilizing smaller LLMs, like FLAN-T5 (XL and XXL) and Llama 2 (7B and 13B), suffices for effective misinformation detection. In this case, our framework stands out from COT-based methods (Pan et al., 2023; Pelrine et al., 2023; Wang and Shu, 2023) as it eliminates the necessity of generating numerous immediate reasoning steps sequentially. Specifically, our cognition system only requires decoding the first token (i.e., "yes"/"no") to compute truth values. Since the primary bottleneck in the inference time of LLMs arises from subsequential decoding, the cost of TELLER is lower than COT-based methods. For instance, consider a COTbased model that generates 100 tokens for input news. The theoretical inference time of our framework is thus 1100 of COT-based methods, assuming parallel decoding of the first token of N questions. Token Cost: Assuming our framework needs N queries and other LLM-based methods requires one with input query length L and output length M , cin is the price of input tokens, cout is the price of output tokens, and the token cost ratio between our framework and LLM-based methods is N×(L×cin+1×cout)
L×cin+M×cout . In general, cout is higher than cin. Then if M is significantly high when the output of other LLM-based methods contains lots of tokens such as COT, the total cost does not give much difference.
Additionally, we conduct experiments on various language models to verify the versatility of our framework, especially for FLAN-T5-large (780M) in Table 1. That is to say, our framework can build on smaller models (780M) while the size of bertlarge has been 334M. While there have been more and more distillation techniques for LLMs to obtain lightweight models, many engineering efforts can
be made to reduce the running cost, which is not the focus of our work. Consequently, we conclude that the cost of my framework is acceptable.

heading: G Formal Description of DNF Layer
text: In this section, we introduce modified Disjunctive Normal Form (DNF) Layer employed in our framework. The DNF Layer is built from semi-symbolic layers (SL), which can progressively converge to symbolic semantics such as conjunction ∧ and disjunction ∨.
Specifically, for the truth value vector µ ∈ RM mentioned in Sec. 3.1.2, SL can be formulated as follows:
µo = tanh
  M∑
j
wjµj + β
  , (4)
β = δ  b− ∑
j
|wjµj |   , (5)
where wj represents learnable parameters, b = max
j |wjµj | and δ ∈ [−1, 1] represents the semantic gate selector. µj is the truth value for the jth logic atom obtained from the cognitive system. The sign of the learned weight wj indicates whether µj (if wj is positive) or its negation (if wj is negative) contributes to µo. Thus, logical negation (e.g., ¬pj) can be computed as the multiplicative inverse of the input: −µj .
Eq. 4 resembles a standard feed-forward layer, aiming to compute a single truth value from a collection of values µj corresponding to different instantiations of a single predicate/question. β serves as the bias term. As shown by (Cingillioglu and Russo, 2021), by adjusting δ from 0 to 1 during training, SL tends to converge to conjunctive semantics as SL∧ (e.g., p1 ∧ p2, . . . ,∧pM ), indicating that if at least one wjµj is false, the output µo will be false; otherwise, µo will be true. Conversely, by gradually adjusting δ from 0 to −1, SL can attain disjunctive semantics as SL∨ (e.g., p1 ∨ p2, . . . ,∨pM ), where if at least one wjµj is true, µo will be true; otherwise, µo will be false. Additionally, b can guarantee µo being true (false) when all wjµj are true (false) for SL∧ (SL∨).
Since each dimension in µ corresponds to the same predicate for different inputs, SL effectively represents the relationship among different instantiations and the target output µo, enabling the learning of generic rules for various inputs. Moreover, by employing rule-based aggregation, our framework exhibits noise tolerance against incorrect predictions of LLMs in the cognition system, particularly owing to the SL∨.
Notably, one predicate can be instantiated by multiple assignments, i.e., Pi pertains to Mi logic atoms in Appendix A.2. Thus, the parameters bound to these Mi logic atoms should naturally share the logical semantics of Pi. Instead of gathering all possible combinations of Mi logic atoms for training ( Mi∏ j=1 j), we let these logic atoms share the same w. In this scenario, SL can be represented as follows:
µo = tanh(
N∑
i
Mi∑
j
wiµi,j + β), (6)
β = δ(b− N∑
i
Mi∑
j
|wiµi,j |), (7)
where N is the number of predicates.

heading: Future Work
text: Therefore, we acknowledge the need for future research to develop a more suitable and powerful neuralsymbolic framework in the context of fake news detection. Since each dimension in µ is precisely bonded to a question template (logic predicate), we can employ traditional machine learning classification algorithms, including decision tree6, naive Bayes Classifier7 and multi-layer perceptron (MLP), to replace the DNF Layer to drive our decision system, while maintaining partial aspects of trustworthy AI. Therefore, we compare the DNF Layer with these
6https://scikit-learn. org/stable/modules/tree. html
7https://scikit-learn. org/stable/modules/ naive_bayes. html
three methods in both in-domain and cross-domain settings on three datasets, shown in Tables 15 and 16, respectively. According to the results, we conclude that the decision tree and MLP perform better when the training and testing data are from the same domain. Meanwhile, the naive Bayes Classifier demonstrates more satisfactory generalization performance in cross-domain experiments across various LLMs. This implies that our proposed dualsystem framework shows potential in developing a more powerful decision module, such as an ensemble of these algorithms. However, the DNF Layer still outperforms these three methods in most cases when using Llama2 (13B) as the driver of the cognition system, achieving a better trade-off between accuracy and generalization ability. Moreover, the DNF Layer also exhibits advantages over these methods in terms of its ability to handle missing values and multi-grounding problems, as well as its flexibility in efficiently searching logic rules in a large space, whereas the decision tree is constrained by depth and width