Venue: NAACL 2022

# Cross-document Misinformation Detection based on

Event Graph Reasoning

Xueqing Wu, Kung-Hsiang Huang, Yi R. Fung, Heng Ji

University of Illinois Urbana-Champaign

{xueqing8,khhuang3,yifung2,hengji}@illinois.edu

###### Abstract

For emerging events, human readers are often exposed to both real news and fake news. Multiple news articles may contain complementary or contradictory information that readers can leverage to help detect fake news. Inspired by this process, we propose a novel task of **cross-document misinformation detection**. Given a cluster of topically related news documents, we aim to detect misinformation at both _document level_ and a more fine-grained level, _event level_. Due to the lack of data, we generate fake news by manipulating real news, and construct \(3\) new datasets with \(422\), \(276\), and \(1,413\) clusters of topically related documents, respectively. We further propose a graph-based detector that constructs a cross-document knowledge graph using cross-document event coreference resolution and employs a heterogeneous graph neural network to conduct detection at two levels. We then feed the event-level detection results into the document-level detector. Experimental results show that our proposed method significantly outperforms existing methods by up to \(7\) F1 points on this new task.1

Footnote 1: Codes and data are at [https://github.com/shirley-wu/cross-doc-misinfo-detection](https://github.com/shirley-wu/cross-doc-misinfo-detection).

+
Footnote †: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 543 - 558

July 10-15, 2022 ©2022 Association for Computational Linguistics

## 1 Introduction

The dissemination of fake news has become an important social issue. For emergent complex events, human readers are usually exposed to multiple news documents, where some are real and others are fake. News documents from different sources naturally form a cluster of topically related documents. We notice that articles about the same topic may contain conflicting or complementary information, which can benefit the task of misinformation detection. An example is shown in Figure 1. As shown in the knowledge graph, the death of Rosanne Boyland in 2021 US Capitol attack is a shared event across all four documents. Each document is internally consistent, making it difficult to identify misinformation when judging each news separately. However, the three real news documents complement each other's statements regarding the death of Boyland, while the fake news document contradicts the other stories. Such cross-document connections can be leveraged to help detect misinformation.

Most of the existing work on fake news detection is limited to judging each document in isolation. In contrast, we propose a novel task of **cross-document misinformation detection** that aims to detect fake information from a cluster of topically related news documents. We perform the task at both the document level and event level. Each event describes a specific type of real-world event mentioned in the text (e.g., the death of Boyland in Figure 1), and usually involves certain participants to represent different aspects of the event (e.g., the cause of death and the victim of the death event). **Document-level detection** aims to detect fake news documents. **Event-level detection** is a more fine-grained task that aims to detect fake events, thereby pinpointing specific fake information in news documents.

Existing work on fine-grained misinformation detection detects triplets of false knowledge Fung et al. (2021). However, we focus on identifying false events instead of relations or entities, because events are more important for storytelling and easier to compare across multiple documents through cross-document coreference resolution.

To the best of our knowledge, there are no fake news detection datasets with clusters of topically related documents. Therefore, we construct 3 new benchmark datasets based on existing real news corpus with such clusters. Following Fung et al. (2021), we train a generator that generates a document from a knowledge graph (KG), and feed manipulated KGs into the generator to generate fake news documents. Tracking the manipulation operations, we also obtain supervision for event

level detection.

We further propose a detection approach as shown in Figure 2. Given a cluster of documents, we first use an information extraction (IE) system Lin et al. (2020) to construct a within-document KG for each document. Then, we connect the within-document KGs to form a cross-document KG using cross-document event coreference resolution Lai et al. (2021); Wen et al. (2021). Eventually, we use a heterogeneous graph neural network (GNN) to encode the cross-document KG and conduct detection at two levels.

Our contributions are summarized as follows:

1. We propose the novel task of cross-document misinformation detection, and conduct the task at two levels, document level and the more fine-grained event level.
2. We construct \(3\) new datasets for our proposed task based on existing document clusters categorized by topics.
3. We propose a detector that leverages cross-document information and improve document-level detection by utilizing features produced by the event-level detector. Experiments on three datasets demonstrate that our method significantly outperforms existing methods.

## 2 Related Work

**Fake News Detection:** Early work on fake news detection uses hand-crafted features to perform document classification Rubin et al. (2016); Wang (2017); Rashkin et al. (2017); Perez-Rosas et al. (2018); Sarkar et al. (2018); Atanasova et al. (2019). Recent work uses neural networks such as recurrent neural networks Karimi et al. (2018); Nasir et al. (2021) and Transformer Zellers et al. (2019) to encode the news document. To model the internal structure of a news document, Karimi and Tang (2019) model the inter-sentence dependency tree, Vaibhav et al. (2019); Hu et al. (2021) model the interactions between sentences; and Pan et al. (2018) and Fung et al. (2021) model the knowledge graph extracted by IE systems. Similar to our work, Hu et al. (2021) compare each news with external knowledge base (KB) to check for inconsistencies. However, the correlation between news and KB is not as close as the correlation between related news documents due to the incompleteness of these KBs. Other work utilizes additional information such as user engagements and behaviors on social media Shu et al. (2019); Nguyen et al. (2020) and multi-modal features Khattar et al. (2019); Tan et al. (2020); Fung et al. (2021). However, to the best of our knowledge, no published work has considered using cross-document inference for misinformation detection.

In addition to document-level detection, the task of fine-grained detection is also important but rarely explored. The most relevant work detects fake knowledge triplets extracted from each individual news article Fung et al. (2021).

Another related task is fact verification which aims to verify a statement based on retrieved evidence. Fact verification has been explored in multiple domains such as general domain Thorne et al. (2018), climate change Diggelmann et al. (2020)

Figure 1: An example of cross-document misinformation detection, including the texts and knowledge graphs for four news documents. The three real news documents complement each other, while the fake news contradicts the other news. News 1 falsely speculates that Boyland was crushed to death, but it admits that the cause of death was not yet verified. News 2 and 3 complete the story by reporting that Boyland died of drug overdose. The fake news claims that Boyland was killed by police, which contradicts the other news. Additionally, the fake news states that the police attacked Boyland, which is inconsistent with News 3’s claim that the police was trying to help her.

 

and COVID-19 (Wadden et al., 2020; Saakyan et al., 2021). However, fact verification focuses on short single-sentence statements and cannot model the complicated internal structure of a news document.

**Fake News Datasets:** The main difficulty in constructing a fake news dataset is to obtain annotations. Rashkin et al. (2017) and Rubin et al. (2016) obtain labels from the source information and consider news from reliable sources as real news, and unreliable sources as fake news. A potential issue is that the detector may only learn to distinguish the style of different news sources, rather than the authenticity of the content. Shu et al. (2020) collect annotations from fact-checking websites, and Perez-Rosas et al. (2018) collect annotations via crowd-sourcing. These approaches produce datasets of higher quality, but require extensive manual efforts. With the development of powerful generative models capable of mimicking human-written news (Zellers et al., 2019), recent work has constructed datasets by using generative models to generate fake news (Tan et al., 2020; Fung et al., 2021). Fung et al. (2021) further generate fake news from manipulated KG, which we follow to construct our dataset.

## 3 Task Formulation

Given a cluster of documents about the same story, the task of cross-document misinformation detection aims to detect the fake information included in the cluster.

Formally, let \({\bf S}=\{{\bf d}_{1},\cdots,{\bf d}_{N}\}\) be the document cluster, and \(N=|{\bf S}|\) be the size of the cluster. Some documents in \({\bf S}\) are real, while others are fake. From each document \({\bf d}\in{\bf S}\), we extract events \({\bf E}({\bf d})=\{e_{1},\cdots,e_{m}\}\), where \(m=|{\bf E}({\bf d})|\) is the number of events in document \({\bf d}\). In an extracted event set \({\bf E}({\bf d})\), some events are real and others are fake.

We conduct the task of misinformation detection at two levels, document level and event level. **Document-level detection** aims to predict whether each document \({\bf d}\in{\bf S}\) is real or fake. **Event-level detection** is a more fine-grained task that aims to predict whether each event \(e\in{\bf E}({\bf d}),{\bf d}\in{\bf S}\) is real or fake. In the example in Figure 1, the _die_ event in the fake news is fake, since it falsely describes Boyland being killed by the police, but she actually died of drug overdose.

## 4 Approach

An overview of our approach is shown in Figure 2. Given a cluster of documents, we first construct a within-document KG for each document using an IE system (Lin et al., 2020), and then connect the within-document KGs into a cross-document KG using cross-document event coreference resolution. Based on the cross-document KG, we use a hetereogeneous GNN (Schlichtkull et al., 2018;

Figure 2: An overview of our approach. We first construct a within-document KG for each document based on IE output, where \(\Circle\) represents an entity and \(\triangle\) represents an event. Then, we construct a cross-document KG by (1) adding a node for each cross-document event cluster and connecting it with events in the cluster, and (2) introducing a document node \(\square\) for each document and connecting it with all entities and events in the given document. Finally, we use GNN to encode the cross-document KG, and use the event and document features to conduct misinformation detection at two levels. The two detectors are trained and deployed in a pipeline fashion, where event-level detection results are leveraged to improve document-level detection.

 

Hu et al., 2019) to perform detection. We further incorporate the results of event-level detection to help the document-level detector.

### Knowledge Graph Construction

**Within-document KG:** We first construct a within-document IE-based knowledge graph for each document. We leverage OneIE Lin et al. (2020), a joint IE system, to extract the entities, relations, and events contained in a given document. Then, we conduct entity linking Pan et al. (2017) and entity coreference resolution Lee et al. (2017) to merge multiple mentions of the same entities together. Eventually, we obtain a within-document KG where entities and events are nodes, relations are edges between entities, and arguments are edges between events and entities.

Cross-document KG:We leverage cross-document event coreference resolution to connect the within-document KGs into a cross-document KG as illustrated in Figure 2. We employ a cross-document event coreference resolution system Lai et al. (2021); Wen et al. (2021) to identify clusters of events from multiple documents that refer to the same real-world events. The system utilizes both textual contexts of the event mentions and symbolic features such as the event type information. An example of the detected event cluster is shown in Table 1, where the four events of four documents all refer to the same explosion attack on Venezuela's President Nicolas Marduro. These four events contain complementary or contradictory details, which can be used for misinformation detection. For each event cluster, we add a node to represent the overall information of the real-world complex event corresponding to the cluster. Then, an edge is added between each event node and corresponding cluster node to allow reasoning among cross-document coreferential events.

To indicate which document each entity or event belongs to and capture the global information of each document, we further introduce a document node and connect it to the associated entity and event nodes for each document.

The resulting KG contains \(4\) types of nodes (i.e. entity nodes, event nodes, document nodes, and event cluster nodes) and \(5\) types of edges (i.e. relation edges, event argument edges, document-to-entity edges, document-to-event edges, and edges connecting event nodes to event cluster nodes). Since all edges are directional, we add an inverse edge for each edge to propagate features along both directions, and the final KG contains \(10\) edge types, accounting for the inverse of existing edge types.

KG representation:We use BERT Devlin et al. (2019) to initialize the node and edge embeddings in the KG. For a document node, we use BERT to encode the entire document and take the embeddings of [CLS] tokens. Similarly, for an entity node, we encode its canonical mention. For an event node, we encode the sentence where the event trigger occurs. For an event cluster node, we take the average of the embeddings of all events in the cluster. For a relation edge or an event argument role edge, we encode the linearized representation of the relation tuple. For example, the _Leadership_ relation between "Nicolas Maduro" and "Venezuelan" is represented by "_Nicolas Maduro, Leadership, Venezuelan_" , and "guns" as the _ExplosiveDevice_ argument of the _DetonateExplode_ event is represented by "_DetonateExplode, ExplosiveDevice, guns_".

### Knowledge Graph Encoder

**Heterogeneous GNN:** Given the heterogeneous nature of the cross-document KG, we adopt a het

\begin{table}
\begin{tabular}{l|p{368.3pt}} Real & ...Venezuela’ s president, Nicolas Maduro, has survived an apparent assassination attempt after what officials described as drones armed with explosives\({}_{\text{argl}}\) detonated\({}_{\text{orig}}\) overhead during a speech he was making at a military event. \\ \hline Real & ...The BBC quotes anonymous firefighters at the scene who say “the incident was actually a gas tank explosiveion\({}_{\text{riq}}\) inside an apartment\({}_{\text{ug}}\), but did not provide further details.” \\ \hline Fake & ...Maduro was not targeted by the drones, the prime minister said, but state security services reported that the drones were meant for him. ”The explosion\({}_{\text{riq}}\) was caused by two machine guns\({}_{\text{siggi}}\),” Maduro said, adding that there were no injuries. \\ \hline Fake & ...Two drones armed with explosives\({}_{\text{detonated}}\) near PuntoDeCorte\({}_{\text{urgl}}\), where the Venezuelan Foreign Minister, Jorge Rodríguez, was performing, and near the stage where he was giving a speech. ... \\ \end{tabular}
\end{table}
Table 1: An example of cross-document event cluster from IED dataset, where trig, arg1 and arg2 represent the trigger, _ExplosiveDevice_ argument and _Place_ argument respectively. The four events from four documents all refer to the explosion attack on Nicolas Marduro. The two real news articles complement each other by providing different aspects of the event (_ExpulsiveDevice_ argument in the first news and _Place_ argument in the second news), while the two fake news articles contradict the real news with different details (i.e., different _ExplosiveDevice_ and _Place_ arguments).

 

erogeneous GNN to encode the KG.

Formally, let \({\mathcal{G}}\) denote KG and \({\mathcal{V}}\) denote the nodes in \({\mathcal{G}}\). We use \({\mathcal{R}}\) to denote the \(10\) types of edges as discussed in the previous section, and for each edge type \(r\in{\mathcal{R}}\), we use \({\mathcal{G}_{r}}\) to denote the subgraph of \({\mathcal{G}}\) that only contains edges of type \(r\). At the \(l\)-th layer, the inputs are output features produced by the previous layer denoted as \({\mathbf{h}}^{(l-1)}_{i},i\in{\mathcal{V}}\). For each edge type \(r\in{\mathcal{R}}\), we apply a separate GNN to encode \({\mathcal{G}_{r}}\) and produce a set of features denoted as \({\mathbf{h}}^{(l)}_{i,r}\). Then, we aggregate the outputs for all edge types into the final output as follows:

\[{\mathbf{h}}^{(l)}_{i}=\sum_{r\in{\mathcal{R}}}{\mathbf{h}}^{(l)}_{i,r}/|{ \mathcal{R}}| \tag{1}\]

For document-to-entity edges, document-to-event edges, and edges connecting event nodes to event cluster nodes, we use standard graph attention network (GAT). For relation edges and event argument edges, we apply edge-aware GAT to leverage the edge features. Here, the edge features refer to the BERT embeddings of text descriptions such as "_Nicolas Maduro, Leadership, Venezuela_" or "_DetonateExplode, ExplosiveDevice, guns_" as described in Section 4.1. The remainder of Section 4.2 presents details of GAT and edge-aware GAT, i.e., how to produce \({\mathbf{h}}^{(l)}_{i,r}\) based on \({\mathbf{h}}^{(l-1)}_{i}\).

**Graph attention network:** For each given node, GAT aggregates the node features of its neighbors via attention mechanism Velickovic et al. (2018). For a given edge type \(r\in{\mathcal{R}}\), let \({\mathcal{N}}_{i,r}\) denote the neighbors of node \(i\) in \({\mathcal{G}_{r}}\). At the \(l\)-th layer, the attention weights \(\alpha_{ij}\) are calculated as follows:

\[e_{ij}=\text{LeakyReLU}\left({\mathbf{a}}^{\top}\left[{\mathbf{W}}{\mathbf{ h}}^{(l-1)}_{i}\|{\mathbf{W}}{\mathbf{h}}^{(l-1)}_{j}\right]\right) \tag{2}\]

\[\alpha_{ij}=\text{softmax}_{j}(e_{ij})=\frac{\exp(e_{ij})}{\sum_{k\in{ \mathcal{N}}_{i,r}}\exp(e_{ik})} \tag{3}\]

where \({\mathbf{a}}\) and \({\mathbf{W}}\) are trainable parameters, and \(\|\) denotes the feature concatenation. The output features \({\mathbf{h}}^{(l)}_{i,r}\) for node \(i\) in \({\mathcal{G}_{r}}\) are calculated as follows:

\[{\mathbf{h}}^{(l)}_{i,r}=\sum_{j\in{\mathcal{N}}_{i,r}}\alpha_{ij}{\mathbf{ W}}{\mathbf{h}}^{(l-1)}_{j} \tag{4}\]

**Edge-aware graph attention network:** Edge-aware GAT is an extension of GAT that considers edge features in addition to node features Huang et al. (2020); Yasunaga et al. (2021). Let \({\bf r}_{ij}\) denote the features of the edge between nodes \(i\) and \(j\). For a given edge type \(r\in{\mathcal{R}}\), at the \(l\)-th layer, the attention weights \(\alpha_{ij}\) are computed as follows:

\[{\bf r}^{\prime}_{ij} ={\mathbf{W}}^{r}\left[{\mathbf{h}}^{(l-1)}_{i}\|{\mathbf{h}}^{( l-1)}_{j}\|{\bf r}_{ij}\right] \tag{5}\] \[\alpha_{ij} =\text{softmax}_{j}\left(({\mathbf{W}}^{Q}{\mathbf{h}}^{(l-1)}_{ i})({\mathbf{W}}^{K}{\bf r}^{\prime}_{ij})^{\top}\right) \tag{6}\]

where \({\mathbf{W}}^{r}\), \({\mathbf{W}}^{Q}\) and \({\mathbf{W}}^{K}\) are trainable parameters. The output features \({\mathbf{h}}^{(l)}_{i,r}\) for node \(i\) in \({\mathcal{G}_{r}}\) are computed as follows:

\[{\mathbf{h}}^{(l)}_{i,r}=\sum_{j\in{\mathcal{N}}_{i,r}}\alpha_{ ij}{\mathbf{W}}^{U}{\bf r}^{\prime}_{ij} \tag{7}\]

where \({\mathbf{W}}^{V}\) is a learnable matrix.

### Misinformation Detector

Using the previously described graph encoder, we are able to obtain representations of the document and event nodes. We conduct document-level detection using the document node representations, and event-level detection using the event node representations. We separately train two detectors for these two levels of tasks.

However, these two tasks are not mutually independent. Intuitively, document-level detection can benefit from the results of event-level detection, because the presence of a large number of false events indicates that the document is more likely to be fake. Therefore, we feed the results produced by a well-trained event-level detector into each layer of the document-level detector. Let \({\bf e}_{i}\) denote the representations of node \(i\) produced by the event-level detector. At the \(l\)-th layer of the document-level detector, instead of using the output features of the previous layer \({\mathbf{h}}^{(l-1)}_{i}\) as input features, we use a linear projection of the concatenation of \({\bf e}_{i}\) and \({\mathbf{h}}^{(l-1)}_{i}\) calculated as follows:

\[{\mathbf{W}}^{(l)}_{\text{proj}}\left[{\bf e}_{i}\|{\mathbf{h}}^{(l-1)}_{i}\right] \tag{8}\]

where \({\mathbf{W}}^{(l)}_{\text{proj}}\) is a learnable matrix.

## 5 Dataset Construction

Currently, there are no existing resources for cross-document misinformation detection. We propose to construct datasets based on real news datasets with clustering information. For each cluster, we randomly sample 50% real news and replace them with manipulated fake news. Figure 3 shows an 

overview of the fake news generation process, and more examples are presented in Appendix D.

Following Fung et al. (2021), we train a KG-to-text generator from the real news in our datasets, and generate fake news from manipulated KGs. The main differences between Fung et al. (2021)'s method and ours in terms of manipulating KG are: (1) we only conduct entity swapping, and do not adopt other types of manipulation including adding relations or events and subgraph replacement; (2) since we focus on events, we select entities to be replaced that are arguments of high-frequency events, instead of based on entity node degree; (3) we select entities from other documents in the same document cluster to replace the original entities, so that the entities before and after replacement are more similar.

We record the manipulation operations, and use a heuristic rule to obtain supervision for event-level detection as explained below. In a fake document, if an event involves manipulated entities as arguments, we consider this event as fake.

## 6 Experiments

### Data

We constructed three new benchmark datasets based on three datasets that naturally have clusters of topically related documents. **IED** is a complex event corpus, where each complex event refers to a real-world story (e.g., Boston bombing) and is described by multiple documents Li et al. (2021). Therefore, a complex event can be considered as a document cluster. **TL17** and **Crisis** are two timeline summarization datasets containing multiple news timelines. Each timeline contains multiple documents describing an evolving long-term event such as Influenza H1N1 and Egypt Revolution Tran et al. (2013, 2015), and thus can be regarded as a document cluster. The detailed statistics of the original datasets are shown in Appendix A.

However, documents within the same cluster may not be closely related as the story described by a cluster can span up to three years. To obtain smaller and more closely related clusters, we split each timeline into smaller clusters of approximately size \(10\) based on publication dates2. Then, we employ the methods described in Section 5 to generate fake documents. The statistics of the constructed datasets are in Table 2.

Footnote 2: For IED, we randomly split the clusters due to the lack of publication dates.

### Experimental Settings

For our proposed method, we use a \(4\)-layered heterogeneous GAT and use bert-base-uncased to initialize the node and edge embeddings. For comparison, on the document-level detection task, we compare our method against two baselines: **HDSF** that models the inter-sentence dependency tree Karimi and Tang (2019), and **GROVER**Zellers et al. (2019), a Transformer-based detector. On the event-level detection task, we compare our method against **random guessing**, **logistic regression** and **BERT**. For logistic regression, we use hand-crafted features to represent the event including the event type, the number of arguments, and the size of the event cluster. The detailed settings are presented in Appendix C.

For evaluation, we use F1 to evaluate document-level detection. Considering the label imbalance of event-level detection, we use F1 and the area under the ROC curve (AUC) to evaluate event-level

Figure 3: An overview of the fake news generation process. Based on the real news and its IE output, we select a high-frequency “_DetonateExplode_” event and replace its argument entity “device” with “machine guns”. We then generate the fake news from the manipulated KG. In the KG of generated fake news, the manipulated entity “guns” is an argument of the “_DetonateExplode_” event, so we consider the event as fake.

 

detection. For the F1 metric, we select the optimal threshold on the validation set.

### Document-level Detection Results

Table 3 shows the results of document-level detection. Our method yields consistent improvements on all three datasets and significantly outperforms the baselines that judge the authenticity for each document in isolation. To understand the effectiveness of each component, we conducted an ablation study and show the results in Table 4. We have the following findings:

(1) We remove the edges between event nodes and event center nodes to analyze the impact of cross-document event coreference resolution, and find that such information significantly improves the performance on IED and TL17. We also train our detector with smaller clusters on TL17 and get worse performance (\(84.53\%\) and \(87.37\%\) on clusters with size 1 and 2 respectively), which verifies that our model benefits from more cross-document information. The benefit of cross-document event coreference resolution is less significant on the large-scale Crisis dataset containing 1.7\(k\) documents. This may imply that cross-document misinformation detection is more useful for emerging new events where large-scale training data is not available.

(2) Using the event-level detection results consistently improves the performance by \(1\text{--}3\) points on all datasets. Since the projection modules introduce additional parameters, we further train a detector utilizing random features and find that using random features reduces the performance. This verifies that the improvement is brought by utilizing the knowledge learnt by the event-level detector rather than additional parameters.

### Event-level Detection Results

We track the manipulation operations during the dataset construction process, which allows us to obtain supervision for event-level detection. The results are shown in Table 5. We compare our method with random guessing, logistic regression with hand-crafted event features, and BERT. We find that random guessing performs the worst, logistic regression and BERT achieve satisfactory performance, and our method significantly outperforms all baselines by a large margin. As in document-level detection, we conduct an ablation study on the use of cross-document event coreference resolution by removing edges between event nodes and event cluster nodes, and find that such information brings slight improvements in the AUC metric.

### Analysis and Discussion

To demonstrate the benefits of using cross-document event coreference resolution, we show an example in Figure 4, with 4 documents from the same cluster. As shown in Figure 4, by performing cross-document reasoning on events in the same event cluster, our model achieves better performance compared to Ours\({}^{\rm(ABLATION)}\), i.e., our model without edges between event nodes and event cluster nodes.

We further analyze the remaining errors in our model. Figure 5 shows two representative cases

\begin{table}
\begin{tabular}{c c|c c c} \hline Cross-document & Event-level & **IED** & **TL17** & **Crisis** \\ event coreference & detection results & & & \\ \hline ✗ & ✗ & 80.59 & 86.55 & 93.64 \\ ✓ & ✗ & 84.57 & 88.99 & 93.67 \\ ✓ & Random & 83.63 & 84.86 & 92.18 \\ ✓ & ✓ & **86.76** & **90.21** & **93.89** \\ \hline \end{tabular}
\end{table}
Table 4: F1 results (in %) of ablation study over document-level detection. We analyze the use of cross-document event coreference resolution and event-level detection results. We further experiment with random features for event-level detection results. Results of our full method are presented in the last row.

\begin{table}
\begin{tabular}{l|c c c c} \hline  & **IED** & **TL17** & **Crisis** \\ \hline HDSF & 78.42 & 80.62 & 82.14 \\ GROVER-medium & 79.06 & 79.40 & 86.84 \\ GROVER-mega & 82.90 & 90.00 & 87.13 \\ \hline Ours & **86.76** & **90.21** & **93.89** \\ \hline \end{tabular}
\end{table}
Table 3: F1 results (in %) of document-level detection. We report the F1 scores of HDSF [1], GROVER of two settings [15], and our proposed method.

\begin{table}
\begin{tabular}{l|c c c c c} \hline  & \multicolumn{2}{c}{**IED**} & \multicolumn{2}{c}{**TL17**} & \multicolumn{2}{c}{**Crisis**} \\ \cline{2-6}  & F1 & AUC & F1 & AUC & F1 & AUC \\ \hline Random & 16.31 & 50.44 & 19.44 & 49.65 & 21.70 & 50.41 \\ LR & 31.26 & 77.87 & 29.14 & 68.19 & 31.67 & 68.17 \\ BERT & 26.43 & 71.12 & 31.95 & 71.42 & 33.89 & 71.86 \\ Ours & 44.86 & **88.46** & **41.56** & **82.59** & **48.48** & **85.60** \\ Ours\({}^{\rm(ABLATION)}\) & **45.00** & **88.54** & **41.66** & 82.28 & 47.78 & 85.17 \\ \hline \end{tabular}
\end{table}
Table 5: Results (in %) of event-level detection. We report the F1 and AUC scores of random guessing (Random), logistic regression (LR), BERT, and our method. We further conduct an ablation study and report the results of our method without cross-document event coreference information, denoted as Ours\({}^{\rm(ABLATION)}\).

 

where both document-level and event-level detectors fail to detect misinformation. In the first example, the manipulated entity is not captured by the IE system, and the error of IE system is propagated into the detector. A potential solution is to use an OpenIE system Stanovsky et al. (2018) that is able to cover more event and entity types. The second example is a more challenging case where the event containing fake information is not mentioned in any other document. This makes it difficult to either verify or disprove via cross-document reasoning, and may require the detector to actively search for external information related to the event.

There are some remaining challenges and limitations in our proposed methodology. First, some cross-document contradictions are difficult to capture by coreference resolution only. In the example in Figure 1, knowing that the police are unlikely to help and attack Boyland at the same time requires commonsense reasoning, which we leave as our future work. Second, an underlying assumption of our framework is that real news articles are consistent and complementary with each other, while fake news often contradicts each other. This assumption is true for our constructed datasets because we manipulate the KGs via random entity swapping. However, certain types of human-written fake news documents, such as conspiracy theories, tend to be closely related to each other and convey highly similar information because they share the same biases or aim to manipulate readers in the same way. This may limit the performance of our proposed system in real-world scenarios.

### Human Evaluation on Fake News Generation

To evaluate the quality of the generated fake news, we conducted a Turing Test by 13 human readers as in Fung et al. (2021). We randomly select 100 documents from the IED dataset, half real and half fake, and ask the human readers to assess the authenticity for each document. The overall accuracy achieved by human readers is 66.88%, with 77.44% accuracy on real documents but only 56.32% accuracy on fake documents. This shows that it is difficult for human readers to detect the generated fake news.

## 7 Conclusions and Future Work

We are the first to study the new task of cross-document misinformation detection. We conduct the task at two levels, document level and the more fine-grained event level, and construct three new datasets to handle the lack of training data. We further propose a graph-based cross-document detector that conducts reasoning over a cross-document knowledge graph and feed the event-level detection results into document-level detector. The ex

Figure 4: An example of four documents from the same cluster in the IED dataset. Event triggers are bolded and marked in gold, and fake information is marked in red. The tables report the detection results of our model with and without cross-document event coreference resolution, denoted by “Ours” and “Ours\({}^{(\text{ABLATION})}\)” respectively, and better results are bolded. The use of cross-document event coreference resolution significantly enhances both levels of detection, especially for detecting fake news 1.

Figure 5: Two examples where our detector fails to detect the fake information. Event triggers are bolded and marked in gold, and fake information is marked in red. In the first example, the fake event argument Abqaiq City is not captured by the IE system and thus cannot be detected. In the second example, the visit of Vajpaye to Mumbai is fake information but not mentioned by any other documents, and no coreference is detected for the _Transportation_ event. Therefore, our detector does not have enough information to detect the fake information.

 

perimental results show that our proposed method significantly outperforms existing methods.

For future work, we intend to extend our method to conduct cross-document reasoning over more types of information (e.g., entities and relations) in addition to events. We also plan to extend our method to multi-media news including texts, images, audios and videos, which requires the construction of cross-document multi-modal knowledge graphs. Finally, a challenging but important task is to construct a large-scale fake news detection corpus with human-written fake news containing document clusters and study our method in this scenario.

## 8 Ethical Considerations

The goal of this work is to advance state-of-the-art research in the field of misinformation detection by analyzing multiple documents on the same topic. We build new benchmark datasets using a fake news generator, and propose a detector that achieves high performance in such scenarios. We have released the constructed datasets and detector codes in this submission as a useful reference for future research. We hope that our work will encourage more efforts in this direction and benefit the community.

However, as with any work that utilizes text generation, our work involves the risk of being applied to produce false information to mislead or manipulate readers. Therefore, we promise not to share codes or checkpoints of our generator to avoid potential negative consequences. To improve reproducibility, we describe the general idea and a few crucial details of the fake news generator.

## Acknowledgement

This research is based upon work supported by U.S. DARPA SemaFor Program No. HR001120C0123 and DARPA KAIROS Program No. FA8750-19-2-1004. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of DARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.

## References

* A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzman, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov (2020)Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 8440-8451. External Links: Link Cited by: SS6.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019)BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), Vol. , pp. 4171-4186. External Links: Document, ISSN 1536-1212 Cited by: SS1.
* T. Diggelmann, J. L. Boyd-Graber, J. Bulian, M. Ciaramita, and M. Leippold (2020)CLIMATE-FEVER: A dataset for verification of real-world climate claims. CoRRabs/2012.00614. External Links: Link, 2012.00614 Cited by: SS6.
* Y. Fung, C. Thomas, R. Gargi Reddy, S. Policetty, H. Ji, B. Chang, K. R. McKeown, M. Bansal, and A. Sil (2021)Infourgeon: cross-media fine-grained information consistency checking for fake news detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, Virtual Event, August 1-6, 2021, pp. 1683-1698. External Links: Link, 2012.00614 Cited by: SS6.
* L. Hu, T. Yang, C. Shi, H. Ji, and X. Li (2019)Heterogeneous graph attention networks for semi-supervised short text classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pp. 4820-4829. External Links: Link, 1905.00614 Cited by: SS6.
* L. Huang, T. Yang, L. Zhang, W. Zhong, D. Tang, C. Shi, N. Duan, and M. Zhou (2021)Compare to the knowledge: graph neural fake news detection with external knowledge. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics 

the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, Volume 1: Long Papers), Virtual Event, August 1-6, 2021_, pages 754-763. Association for Computational Linguistics.
* Huang et al. (2020) Kung-Hsiang Huang, Mu Yang, and Nanyun Peng. 2020. Biomedical event extraction on graph edge-conditioned attention networks with hierarchical knowledge graphs. In _Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020_, volume EMNLP 2020 of _Findings of ACL_, pages 1277-1285. Association for Computational Linguistics.
* Joshi et al. (2020) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. Spanbert: Improving pre-training by representing and predicting spans. _Trans. Assoc. Comput. Linguistics_, 8:64-77.
* Karimi et al. (2018) Hamid Karimi, Proteek Roy, Sari Saba-Sadiya, and Jiliang Tang. 2018. Multi-source multi-class fake news detection. In _Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018_, pages 1546-1557. Association for Computational Linguistics.
* Karimi and Tang (2019) Hamid Karimi and Jiliang Tang. 2019. Learning hierarchical discourse-level structure for fake news detection. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_, pages 3432-3442. Association for Computational Linguistics.
* Khattar et al. (2019) Dhruv Khattar, Jaipal Singh Goud, Manish Gupta, and Vasudeva Varma. 2019. MVAE: multimodal variational autoencoder for fake news detection. In _The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019_, pages 2915-2921. ACM.
* Lai et al. (2021) Tuan Manh Lai, Heng Ji, Trung Bui, Quan Hung Tran, Franck Dernoncourt, and Walter Chang. 2021. A context-dependent gated module for incorporating symbolic semantics into event conference resolution. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021_, pages 3491-3499. Association for Computational Linguistics.
* Lee et al. (2017) Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017_, pages 188-197. Association for Computational Linguistics.
* Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_, pages 7871-7880. Association for Computational Linguistics.
* Li et al. (2021) Manling Li, Sha Li, Zhenhailong Wang, Lifu Huang, Kyunghyun Cho, Heng Ji, Jiawei Han, and Clare R. Voss. 2021. Future is not one-dimensional: Graph modeling based complex event schema induction for event prediction. _CoRR_, abs/2104.06344.
* Lin et al. (2020) Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020. A joint neural model for information extraction with global features. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_, pages 7999-8009. Association for Computational Linguistics.
* Nasir et al. (2021) Jamal Abdul Nasir, Osama Subhani Khan, and Iraklis Varlamis. 2021. Fake news detection: A hybrid cnn-rnn based deep learning approach. _International Journal of Information Management Data Insights_, 1(1):100007.
* Nguyen et al. (2020) Van-Hoang Nguyen, Kazunari Sugiyama, Preslav Nakov, and Min-Yen Kan. 2020. FANG: leveraging social context for fake news detection using graph representation. In _CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020_, pages 1165-1174. ACM.
* Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In _Proceedings of NAACL-HLT 2019: Demonstrations_.
* ISWC 2018
- 17th International Semantic Web Conference, Monterey, CA, USA, October 8-12, 2018, Proceedings, Part I_, volume 11136 of _Lecture Notes in Computer Science_, pages 669-683. Springer.
* August 4, Volume 1: Long Papers_, pages 1946-1958. Association for Computational Linguistics.
* Perez-Rosas et al. (2018) Veronica Perez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. 2018. Automatic detection of fake news. In _Proceedings of the 27th 

International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018_, pages 3391-3401. Association for Computational Linguistics.
* Proceedings of the Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes, EMNLP-CoNLL 2012, July 13, 2012, Jeju Island, Korea_, pages 1-40. ACL.
* Rashkin et al. (2017) Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of varying shades: Analyzing language in fake news and political fact-checking. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017_, pages 2931-2937. Association for Computational Linguistics.
* Rubin et al. (2016) Victoria L Rubin, Niall Conroy, Yimin Chen, and Sarah Cornwell. 2016. Fake news or truth? using sattical cues to detect potentially misleading news. In _Proceedings of the second workshop on computational approaches to deception detection_, pages 7-17.
* Saakyan et al. (2021) Arkadiy Saakyan, Tuhin Chakrabarty, and Smaranda Muresan. 2021. Covid-fact: Fact extraction and verification of real-world claims on COVID-19 pandemic. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, Volume 1: Long Papers), Virtual Event, August 1-6, 2021_, pages 2116-2129. Association for Computational Linguistics.
* Sarkar et al. (2018) Sohan De Sarkar, Fan Yang, and Arjun Mukherjee. 2018. Attending sentences to detect satirical fake news. In _Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018_, pages 3371-3380. Association for Computational Linguistics.
* 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, Proceedings_, volume 10843 of _Lecture Notes in Computer Science_, pages 593-607. Springer.
* Shu et al. (2020) Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. 2020. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. _Big Data_, 8(3):171-188.
* Shu et al. (2019) Kai Shu, Suhang Wang, and Huan Liu. 2019. Beyond news contents: The role of social context for fake news detection. In _Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM 2019, Melbourne, VIC, Australia, February 11-15, 2019_, pages 312-320. ACM.
* Song et al. (2015) Zhiyi Song, Ann Bies, Stephanie M. Strassel, Tom Riese, Justin Mott, Joe Ellis, Jonathan Wright, Seth Kulick, Neville Ryant, and Xiaoyi Ma. 2015. From light to rich ERE: annotation of entities, relations, and events. In _Proceedings of the The 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation, EVENTS@HLP-NAACL 2015, Denver, Colorado, USA, June 4, 2015_, pages 89-98. Association for Computational Linguistics.
* Stanovsky et al. (2018) Gabriel Stanovsky, Julian Michael, Luke Zettlemoyer, and Ido Dagan. 2018. Supervised open information extraction. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers)_, pages 885-895. Association for Computational Linguistics.
* Tan et al. (2020) Reuben Tan, Bryan A. Plummer, and Kate Saenko. 2020. Detecting cross-modal inconsistency to defend against neural fake news. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020_, pages 2081-2106. Association for Computational Linguistics.
* Thorne et al. (2018) James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and verification. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers)_, pages 809-819. Association for Computational Linguistics.
* 37th European Conference on IR Research, ECIR 2015, Vienna, Austria, March 29
- April 2, 2015. Proceedings_, volume 9022 of _Lecture Notes in Computer Science_, pages 245-256.
* Tran et al. (2013) Giang Binh Tran, Mohammad Alfriai, and Dat Quoc Nguyen. 2013. Predicting relevant news events for timeline summaries. In _22nd International World Wide Web Conference, WWW '13, Rio de Janeiro, Brazil, May 13-17, 2013, Companion Volume_, pages 91-92. International World Wide Web Conferences Steering Committee / ACM.
* Vaibhav et al. (2019) Vaibhav Vaibhav, Raghuram Mandyam Annasamy, and Eduard H. Hovy. 2019. Do sentence interactions matter? leveraging sentence level representations for 

fake news classification. In Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs@EMNLP 2019, Hong Kong, November 4, 2019, pp. 134-139. External Links: Link Cited by: SS5.
* May 3, 2018, Conference Track Proceedings, HLT '18, New York, NY, USA, pp. 134-147. External Links: ISBN 978-1-4503-387X, Document, ISSN 1530-1493 Cited by: SS1.
* D. Wadden, S. Lin, K. Lo, L. Lu Wang, M. van Zuylen, A. Cohan, and H. Hajishirzi (2020)Fact or fiction: verifying scientific claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, Online, November 16-20, 2020, pp. 7534-7550. External Links: Link Cited by: SS1.
* August 4, Volume 2: Short Papers, pp. 422-426. External Links: Link Cited by: SS1.
* H. Wen, Y. Lin, T. Lai, X. Pan, S. Li, X. Lin, B. Zhou, M. Li, H. Wang, H. Zhang, X. Yu, A. Dong, Z. Wang, Y. Fung, P. Mishra, Q. Lyu, D. Suris, B. Chen, S. Windisch Brown, M. Palmer, C. Callison-Burch, C. Vondrick, J. Han, D. Roth, S. Chang, and H. Ji (2021)RESIN: a dockerized schema-guided cross-document cross-lingual cross-media information extraction and event tracking system. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations, NAACL-HLT 2021, Online, June 6-11, 2021, pp. 133-143. External Links: Link Cited by: SS1.
* M. Yasunaga, H. Ren, A. Bosselut, P. Liang, and J. Leskovec (2021)QAGNN: reasoning with language models and knowledge graphs for question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pp. 535-546. External Links: Link Cited by: SS1.
* R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, and Y. Choi (2019)Defending against neural fake news. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 9051-9062. External Links: Link Cited by: SS1.

## Appendix A Statistics of Original Datasets

Statistics of the original IED, TL17 and Crisis dataset are presented in Table 6.

## Appendix B Method Details

### Information Extraction

We use **OneIE**Lin et al. (2020), a BERT-based end-to-end IE system to extract entities, relations, and events. OneIE conducts IE in four steps: (1) encode a sentence with a pre-trained BERT encoder, (2) identify entity mentions and event triggers using a conditional random fields layer, (3) classify types of entity mentions, events, entity relations, and event arguments using feed-forward networks, and (4) search for a globally optimal IE graph via beam search. In this work, we use the model released by Wen et al. (2021). The model achieves 64.1, 49.7, and 49.5 F1 on trigger extraction, argument extraction and relation extraction respectively on ACE 2005 and ERE Song et al. (2015).

In addition, we use entity linking and entity coreference resolution to identify coreferential entity mentions. For **entity linking**, we use an LSTM-based entity linker to link Pan et al. (2017) to link entity mentions to WikiData entries. The entity linker achieves 91.8 F1 and 84.3 accuracy. For **entity coreference resolution**, we use an extension of the e2e-coref model Lee et al. (2017) based on XLM-RoBERTa Conneau et al. (2020). The model is released by Wen et al. (2021) and achieves a 92.4 CoNLL score on OntoNotes Pradhan et al. (2012). Eventually, entity mentions that are linked to the same WikiData entry or identified as coreferences will be considered as the same entity, and their entity nodes in the KG will be merged.

### Event Coreference Resolution

For event coreference resolution, we use Lai et al. (2021), a within-document coreference resolution model. We extend it to the cross-document scenario following Wen et al. (2021). Given a cluster containing \(N\) documents, we concatenate each

\begin{table}
\begin{tabular}{c|r r r} \hline  & \multicolumn{1}{c}{**\# Cluster**} & \multicolumn{1}{c}{**\# Doc**} \\  & \multicolumn{1}{c}{**\# Doc**} & \multicolumn{1}{c}{**\# Doc****cluster**} \\ \hline
**IED** & 433 & 7403 & 17 \\
**TL17** & 17 & 4650 & 273 \\
**Crisis** & 4 & 20463 & 5116 \\ \hline \end{tabular}
\end{table}
Table 6: Statistics of the original datasets.

 

pair of documents into a "mega-document". The model then conducts coreference resolution on each mega-document. More specifically, for each event mention, the model uses SpanBERT Joshi et al. (2020) to extract contextualized text embeddings and builds manually designed symbolic features such as event types, attributes, and arguments. Then, the two features are combined selectively using a gated mechanism. Eventually, for each pair of event mentions in a mega-document, the model predicts whether they are coreferential. In this work, we use the model released by Wen et al. (2021). The model achieves 84.8 CoNLL score on ACE 2005.

### KG-to-Text Generator

We train the KG-to-text generator by following Fung et al. (2021).

We first linearize the IE-based KG. For example, the _Leadership_ relation between "Nicolas Maduro" and "Venezuelan" is represented by "_Nicolas Maduro, Leadership, Venezuela_", and the _DetonateExplode_ event with "drone" as _ExplosiveDevice_ argument and "flat" as _Place_ argument is represented by "[_DetonateExplode_ | _ExplosiveDevice_ = _drone_, _Place_ = _flat_]". We represent the entire KG in graph by concatenating the text representations of all relations and events.

Since generating the entire document is very challenging, we fine-tune a sentence-level KG-to-text generator from BART Lewis et al. (2020). The generator takes the linearized KG and the previous sentence as input and generates the next sentence. Here, the KG only contains information presented in the sentence rather than in the entire document. During inference, the generator generates the entire document sentence-by-sentence in an autoregressive manner.

## Appendix C Experiment Details

**Detailed settings of our method:** For our proposed method, we use a \(4\)-layered heterogeneous GNN, where each GAT layer contains \(8\) heads. To initialize the node and edge embeddings, we use bert-base-uncased model with the feature dimension of \(768\). Our model contains \(233\)M parameters.

For hyperparameters, we use a batch size of \(16\), and search the learning rate from \(\{10^{-3},10^{-4},10^{-5}\}\) and the number of layers within \(\{2,4,8\}\). Our best-found hyperparameters are a learning rate of \(10^{-5}\) and a number of layers of \(4\). We train our model with Adam optimizer until convergence. To reduce computation cost, we freeze BERT's parameters. The training process takes approximately \(6\) hours on a Tesla P100 GPU.

**Detailed settings of KG-to-text generator:** We fine-tune the model from bart-large model containing \(24\) layers, \(1024\) hidden dimensions, \(16\) heads, and \(406\)M parameters. on the three datasets respectively. We train the model on a Tesla P100 GPU using the batch size of \(1024\) tokens, the gradient accumulation step of \(16\), the learning rate of \(3\times 10^{-5}\), the warmup steps of \(500\) steps, and the total training steps of \(12000\).

**Document-level baselines:** For document-level detection, we compare our method against two baselines: HDSF that models inter-sentence dependency tree Karimi and Tang (2019), and GROVER Zellers et al. (2019), a Transformer-based detector. For HDSF, we use the implementation at [https://github.com/hamidkarimi/HDSF/](https://github.com/hamidkarimi/HDSF/). We train the model on our datasets using their default hyper-parameters. For GROVER, we use the implementation at [https://github.com/rowanz/grover](https://github.com/rowanz/grover) and experiment with two settings, medium setting and mega setting. Since fine-tuning the GROVER model is computationally expensive, we use GROVER in the zero-shot setting.

**Event-level baselines:** For event-level detection, since there are no existing methods, we use three baselines, random guessing, logistic regression, and BERT. In random guessing, for each event, we randomly draw a value from a uniform distribution between \([0,1]\) as the probability that the event is false. In logistic regression, we use the following features: event type (represented by one-hot feature), number of arguments, and the size of the event cluster that the given event belongs to. The features are normalized on the training set. We use the implementation of logistic regression and default parameters provided by sklearn. In the BERT baseline, we use the same BERT-based event features as our method, and replace the \(4\)-layer GNN in our model with a feed-forward network. We use the same hyper-parameters to train the model 

Examples of Fake News Generation

We present two examples of generated fake news in Figure 6 and 7, including the original real news, manipulated KG, and generated fake news. The generated fake news conveys the manipulated misinformation and meanwhile is stylistically similar to real news.

## Appendix E Scientific Artifacts

In this work, we use three datasets including IED Li et al. (2021), TL17 Tran et al. (2013) and Crisis Tran et al. (2015). There are no licenses or terms of use associated with all three datasets.

We use five software. Among them, HDSF Karimi and Tang (2019), OneIE Lin et al. (2020) and RESIN Wen et al. (2021) have no license or terms of use. GROVER Zellers et al. (2019) and huggingface are licensed under the Apache License 2.0. Fairseq Ott et al. (2019) is Licenced under the MIT License.

We use two models, BERT Devlin et al. (2019) and BART Lewis et al. (2020), licenced under the Apache License 2.0 and the MIT License respectively.

In summary, all artifacts involved either have no associated licenses or terms of use, or are licensed under the Apache License 2.0 or the MIT License. Both the Apache License 2.0 or the MIT License permit commercial and private use. Therefore, our use is consistent with their intended use. We will release the datasets and software with licenses compatible with the original access conditions.

 



### Real News

Arnia bombing: Argentina and Iran agree truth commission Published duration 28 January 2013, image caption: The two foreign minutes signed the agreement in Addis Babba on Sunday. Argentina and Iran are to jointly set up a commission to investigate the 1994 bombing of the Israeli-Agnetting Mutual Association (Anria) Jewish community centre in Buenos Aires. The commission will be made up of five independent judges, none of whom will be from either Argentina or Iran. Argentine courts have blameed Iran for the attack, which killed SS people. Iran has always denied any involvement. Israel's foreign ministry said it was "surpasset" by news the commission. Spokesman Yigal Plamor told the AFP news agency that it was waiting to receive yet details from Argentina. Arnia and another key Argentina Jewish organisation were meanwhile reported to be welmeny opposed to the move. High-profile suspect. Iran agreed last July to co-operate with Argentina in the investigation, which it said "was going down the wrong way. Such negotiations have aimed inserts government and Argentina jasake Jewsith country, who faxer Argentire is weakening in its resolve to put suspects on ruiu. We warned the Agerpretns from the start that the trainals would try to set at trap Town and that they should beware." Mr Palmor was quoted by AFP as saying Monday. The news agency also quoted a joint statement by Anna and the Dugelton of transine Agrapette Association as playing that he now moved Austell? They a decline in our severity? Image caption: The seven-storey Amia building was destroyed in the attack on B.July 1994. To ignore everything but Argentine justice has done and to replace it with a commission trial. In the best of cases, wil tissue, without any defined deadline, a "recommodulator" to the parties constitute, without doubt, a reversal in the common objective of obtaining justice, the statement said. However, Argentine President Cristina Fernandez de Kirchner called the agreement "listonc". It guarantees the right to due process of law, a fundamental principle of international criminal law," Ms Femaarde salt. She said the Argonne Minister Program Minister Hector Timmerman and his Iranian counterpart had signed a memorandum of understanding on the sidines of an African Union summit in Addis Ababa, in Ethiopia. It still has to be refilled by the parameters of both countries. The commission would "anlipses all the documentation presented to date by the judicial authorities of Argentina and Iran", Ms Femaarde wrote on her Twitter account. According to Mr Timeman, the agreement will make it possible for Argentine legal officials to question Iranian suspects Ir Tehran. Among the suspects named by _Apgenteus_ prosecutors when they made their case in 2007 is farars current Defence Minister, Gen Ahmed Vahid. At the time of the attack, Gen Vahid was the commander of a special unit of Iran's Revolutionary Guards. Argentine prosecutors said Iran planned and financed the attack, and that the Lebanese militator group Hezbolan carried out.

###### Acknowledgements.

This work was supported by the National Science Foundation under Grant No. 65040

An RA member who was part of the group behind the Birmingham path bombings has apologued. Michael Christopher Hayes - a self-outless bother maker - said he was story for the killing of record noise in the 1974 plants. Besacing to the BBC, Mr Hayes sad he hoped his analogy on behalf of all active republications will help give pictures related to "focusure" to exist: "My objectives and my heartful sympatencies at all you, for the termie, magic loss that you've been put through. And all these years you have been trying to fool closure," I hope at least God well be mercid and they you close... I would imagine red? A good impulse error for myself. I adoptable for all active republications who had no intention of hurting antibody and symbolise with you." An eight-minute delay before police were warned of the boming location led to the dead of an jogic and the jury of 182 engines, where employed is a pair of old young polles. My Hayes said he found that not been intended to kill people. On the evening of the 21 November 1974, an emun with nish account called the Birmingham Post and Evening Mal newspapers to say two borths were granted in the beam centre. As finished the call by giving the official cook used by the Provisional IR to authenticize a vamage can add any civilans to be evacuated. Former IRA officials have since sadd there was an environment deligi) it assigns the warning to drives past the R&A hit in Birmingham had been shocked by the deon itself. That wasn't meant it wouldn hard here done if that was the case; he said, The archaege left as the Midmery Bush up in Birmingham after a bowright ended on 21 November 1974 (On the Birmingham pub bombings caused west single losses of life in the Troubles. Six men were worthy convicted for the bbist but non-one sees her been brought inside. Mx Hayes, go, now horses in south Dalitit, salt he personally devised a third brother on Birmingham's Hedge Road after he became aware of the death kit in the first two lights. He studied to lay wide planet the bornes in the Mubeny Bush and the Tavern in the Tawn but she said he wanted to speak out to give the "port of view" or a participant. But relatives who who move twice Allied the explosive way "guides" against meispien and" Arveya]otomy came as an impostist into the bombings we seemed by a corner stating "we wishd of evidence" hand not yet been heard. West Island Police sat their investigation remained open hot that no further comments on the sunocotyl identifies such provided until the proceedings have conducted. Vidras famlies have repeatedly asked five frames of the sunects be disclosed. Mr Hayes was arrested and questioned by West Midman Police about the bombing in 1974 but he was reinbened. It said se had been a member of the IRA for more than 30 years in both Ireland and England and find participated in the group's activities in Birmingham as "an arcuive volunteer". APF us GRPly (K News: Bentin Stipo (centre) and Harris Ferguson (left) react as students at Norwich School receive her X-Level results. PA UK news: A train destined near Stonehaven has left three people dead.

### Kg manipulation

A man with nish accent made a telephone calf to the Birmingham Post and Evering Mall, clamping to be from the town centre and claiming to have information about the wavebands of an Irish Republican Army (RR) which had been planted in the team. Later claimed that the bomb had been intended to cause an explosion switch watch have killed all its "cembers". Anne uses killed and then others never injured much when they except caught up in the bonts which explored the two poles. The Birmingham pub bombings were believed to have been carried out by a member of the RR’s Povissonal IRA GROIN image corruption therapy images caption Twenty-one people were killed. Only one men have been convicted of the bombing button, but ro - one has been brought to justice. In fact been suggested that this RR calls driving may keep involved in the planning of the stock), but this has been denied. It has been suggested that the RR ut responsible for the Birmingham slimps may have been improved in the planning and execution. The features of those allied have been campaignying for a full public inquiry into the skirzing for more than 2 yearns. Imago copyright Fraomaker might expect that each turn of the Return Job upon buttons are hugent in a single day the Toxkos. In a statement, he said: ”It does not know what to glanted the bonds in the Muthery Bank and the Tavern in the Town but it clear during placed the bombs at the Tawrem.” It is known that Mr Hayes, who was being in South Dultin at the time, defeated a horn in Birmingham's Hedge Road After Hey before he carried out out the Mubery Bush forming bit so that no stream was involved in the Tavern in the Tom explosion. “A number of divisions were evacuated from the scene. Nctimary families have expressed their shock and get large enough Golf/Image Images Caption The attack has been described as a "teniscus set of terrorism" Image recognition Guide) message tag correction Street in Birmingham was the score of one of the Birmingham bomb heads in 1974. In County Astrium, Berita Stipgo and Mini Ferguson, our new strategy at the time of the bombing, were due to people who died in the suit, as was not how editorsthe. The attack was condemered by the brain Republican Army (RJR) and Skin Fun. During its knee at Ehrption Artificial, near York, Yox, Ori, in the UR, Carl Thomas had been archived in the construction of a radar station, which was to be tuned by the Royal Uister Consultatory (RIC) and the British Army. The gait, why one word and raised in the UR star grew in the mountains of the north-west of Event, were home-schooled shutters who had studented in Belgium before going to the Department for Education in London, batr father training brief studies in the Albanian mordias, the pair returned to their home country to work as tasking candidates in the Department of Education in the Country, and to work with ten teachers. After the attack, keeps Bowre away created by West Madison Police but he was released after being questioned by the police and released on ball, and was due to appear before a judge on Monday. Police believe he was a member of the IRA in trained and in England, and that he was based in Birmingham, but that he had been active in the Rx both in England and in Ireland. \(2\) of the 3 mcm concluded of the bombings have been released on appeal They had their conscious question on aposed after serving 6 years in prison and their trial was heard by the Court of Appead. They were released on crossionate grounds after a judge ruled that they had suffered a miscarriage of Justice.

Figure 7: An example of generated fake news, including the original real news, manipulated KG, and generated fake news. Real and fake information are marked in blue and red respectively. To save space, we only show some parts of the KG that are manipulated.