SlowGuess/NAACL_2022_OCR huggingface dataset:
Xueqing Wu, Kung-Hsiang Huang, Yi R. Fung, Heng Ji
{xueqing8, khhuang3, yifung2, hengji}@illinois.edu
For emerging events, human readers are often exposed to both real news and fake news. Multiple news articles may contain complementary or contradictory information that readers can leverage to help detect fake news. Inspired by this process, we propose a novel task of cross-document misinformation detection. Given a cluster of topically related news documents, we aim to detect misinformation at both document level and a more fine-grained level, event level. Due to the lack of data, we generate fake news by manipulating real news, and construct 3 new datasets with 422, 276, and 1,413 clusters of topically related documents, respectively. We further propose a graph-based detector that constructs a cross-document knowledge graph using cross-document event coreference resolution and employs a heterogeneous graph neural network to conduct detection at two levels. We then feed the event-level detection results into the document-level detector. Experimental results show that our proposed method significantly outperforms existing methods by up to 7 F1 points on this new task.
The dissemination of fake news has become an important social issue. For emergent complex events, human readers are usually exposed to multiple news documents, where some are real and others are fake. News documents from different sources naturally form a cluster of topically related documents. We notice that articles about the same topic may contain conflicting or complementary information, which can benefit the task of misinformation detection. An example is shown in Figure 1. As shown in the knowledge graph, the death of Rosanne Boyland in 2021 US Capitol attack is a shared event across all four documents. Each document is internally consistent, making it difficult to
identify misinformation when judging each news separately. However, the three real news documents complement each other's statements regarding the death of Boyland, while the fake news document contradicts the other stories. Such cross-document connections can be leveraged to help detect misinformation.
Most of the existing work on fake news detection is limited to judging each document in isolation. In contrast, we propose a novel task of cross-document misinformation detection that aims to detect fake information from a cluster of topically related news documents. We perform the task at both the document level and event level. Each event describes a specific type of real-world event mentioned in the text (e.g., the death of Boyland in Figure 1), and usually involves certain participants to represent different aspects of the event (e.g., the cause of death and the victim of the death event). Document-level detection aims to detect fake news documents. Event-level detection is a more fine-grained task that aims to detect fake events, thereby pinpointing specific fake information in news documents.
Existing work on fine-grained misinformation detection detects triplets of false knowledge (Fung et al., 2021). However, we focus on identifying false events instead of relations or entities, because events are more important for storytelling and easier to compare across multiple documents through cross-document coreference resolution.
To the best of our knowledge, there are no fake news detection datasets with clusters of topically related documents. Therefore, we construct 3 new benchmark datasets based on existing real news corpus with such clusters. Following Fung et al. (2021), we train a generator that generates a document from a knowledge graph (KG), and feed manipulated KGs into the generator to generate fake news documents. Tracking the manipulation operations, we also obtain supervision for event
1Codes and data are at https://github.com/shirley-wu/cross-doc-misinfo-detection.
Human Language Technologies, pages 543 - 558
Real News 1
...Rosanne Boyland, a 34-year-old Trump supporter from Georgia who died during the attack on the Capitol on Jan. 6, appears to have been killed in a crush of fellow rioters... The chief medical examiner of Washington, D.C., said Boyland's cause and manner of death are "pending". ...
Real News 2
...The medical examiner ruled that Rosanne Boyland, 34, of Kennesaw, Ga., died of accidental acute amphetamine intoxication. ...The drug cited in Boyland's death is addictive and can be prescribed to treat attention-deficit disorder and narcolepsy. ...
Shared across all news
Real News 3
...Officer A.W. stepped out of a tunnel where police were congregated and into the crowd to help Rosanne Boyland, 34, a rioter and QAnon supporter who was trampled by the mob. She died of an accidental drug overdose, according to Washington, DC's chief medical examiner. ...
Fake News
Figure 1: An example of cross-document misinformation detection, including the texts and knowledge graphs for four news documents. The three real news documents complement each other, while the fake news contradicts the other news. News 1 falsely speculates that Boyland was crushed to death, but it admits that the cause of death was not yet verified. News 2 and 3 complete the story by reporting that Boyland died of drug overdose. The fake news claims that Boyland was killed by police, which contradicts the other news. Additionally, the fake news states that the police attacked Boyland, which is inconsistent with News 3's claim that the police was trying to help her.
Newly-released video from the Jan. 6 U.S. Capitol display appears to show U.S. Capitol police kicking 34-year-old Rosanne Boyland to death. ...This video seems to confirm previous reports from The Gateway Pundit suggesting that Boyland was killed by U.S. Capitol police. ...
level detection.
We further propose a detection approach as shown in Figure 2. Given a cluster of documents, we first use an information extraction (IE) system (Lin et al., 2020) to construct a within-document KG for each document. Then, we connect the within-document KGs to form a cross-document KG using cross-document event coreference resolution (Lai et al., 2021; Wen et al., 2021). Eventually, we use a heterogeneous graph neural network (GNN) to encode the cross-document KG and conduct detection at two levels.
1. We propose the novel task of cross-document misinformation detection, and conduct the task at two levels, document level and the more fine-grained event level.
2. We construct 3 new datasets for our proposed task based on existing document clusters categorized by topics.
3. We propose a detector that leverages cross-document information and improve document-level detection by utilizing features produced by the event-level detector. Experiments on three datasets demonstrate that our method significantly outperforms existing methods.
Fake News Detection: Early work on fake news detection uses hand-crafted features to perform document classification (Rubin et al., 2016; Wang, 2017; Rashkin et al., 2017; Pérez-Rosas et al., 2018; Sarkar et al., 2018; Atanasova et al., 2019). Recent
work uses neural networks such as recurrent neural networks (Karimi et al., 2018; Nasir et al., 2021) and Transformer (Zellers et al., 2019) to encode the news document. To model the internal structure of a news document, Karimi and Tang (2019) model the inter-sentence dependency tree, Vaibhav et al. (2019); Hu et al. (2021) model the interactions between sentences; and Pan et al. (2018) and Fung et al. (2021) model the knowledge graph extracted by IE systems. Similar to our work, Hu et al. (2021) compare each news with external knowledge base (KB) to check for inconsistencies. However, the correlation between news and KB is not as close as the correlation between related news documents due to the incompleteness of these KBs. Other work utilizes additional information such as user engagements and behaviors on social media (Shu et al., 2019; Nguyen et al., 2020) and multi-modal features (Khattar et al., 2019; Tan et al., 2020; Fung et al., 2021). However, to the best of our knowledge, no published work has considered using cross-document inference for misinformation detection.
In addition to document-level detection, the task of fine-grained detection is also important but rarely explored. The most relevant work detects fake knowledge triplets extracted from each individual news article (Fung et al., 2021).
Another related task is fact verification which aims to verify a statement based on retrieved evidence. Fact verification has been explored in multiple domains such as general domain (Thorne et al., 2018), climate change (Diggelmann et al., 2020)
Figure 2: An overview of our approach. We first construct a within-document KG for each document based on IE output, where  $\bigcirc$  represents an entity and  $\triangle$  represents an event. Then, we construct a cross-document KG by (1) adding a node for each cross-document event cluster and connecting it with events in the cluster, and (2) introducing a document node  $\square$  for each document and connecting it with all entities and events in the given document. Finally, we use GNN to encode the cross-document KG, and use the event and document features to conduct misinformation detection at two levels. The two detectors are trained and deployed in a pipeline fashion, where event-level detection results are leveraged to improve document-level detection.
and COVID-19 (Wadden et al., 2020; Saakyan et al., 2021). However, fact verification focuses on short single-sentence statements and cannot model the complicated internal structure of a news document.
Fake News Datasets: The main difficulty in constructing a fake news dataset is to obtain annotations. Rashkin et al. (2017) and Rubin et al. (2016) obtain labels from the source information and consider news from reliable sources as real news, and unreliable sources as fake news. A potential issue is that the detector may only learn to distinguish the style of different news sources, rather than the authenticity of the content. Shu et al. (2020) collect annotations from fact-checking websites, and Pérez-Rosas et al. (2018) collect annotations via crowd-sourcing. These approaches produce datasets of higher quality, but require extensive manual efforts. With the development of powerful generative models capable of mimicking human-written news (Zellers et al., 2019), recent work has constructed datasets by using generative models to generate fake news (Tan et al., 2020; Fung et al., 2021). Fung et al. (2021) further generate fake news from manipulated KG, which we follow to construct our dataset.
Given a cluster of documents about the same story, the task of cross-document misinformation detec
tion aims to detect the fake information included in the cluster.
Formally, let  $\mathbf{S} = \{\mathbf{d}_1,\dots ,\mathbf{d}_N\}$  be the document cluster, and  $N = |\mathbf{S}|$  be the size of the cluster. Some documents in  $\mathbf{S}$  are real, while others are fake. From each document  $\mathbf{d}\in \mathbf{S}$ , we extract events  $\mathbf{E}(\mathbf{d}) = \{e_1,\dots ,e_m\}$ , where  $m = |\mathbf{E}(\mathbf{d})|$  is the number of events in document  $\mathbf{d}$ . In an extracted event set  $\mathbf{E}(\mathbf{d})$ , some events are real and others are fake.
We conduct the task of misinformation detection at two levels, document level and event level. Document-level detection aims to predict whether each document  $\mathbf{d} \in \mathbf{S}$  is real or fake. Event-level detection is a more fine-grained task that aims to predict whether each event  $e \in \mathbf{E}(\mathbf{d})$ ,  $\mathbf{d} \in \mathbf{S}$  is real or fake. In the example in Figure 1, the die event in the fake news is fake, since it falsely describes Boyland being killed by the police, but she actually died of drug overdose.
An overview of our approach is shown in Figure 2. Given a cluster of documents, we first construct a within-document KG for each document using an IE system (Lin et al., 2020), and then connect the within-document KGs into a cross-document KG using cross-document event coreference resolution. Based on the cross-document KG, we use a heterogeneous GNN (Schlichtkrull et al., 2018;
Hu et al., 2019) to perform detection. We further incorporate the results of event-level detection to help the document-level detector.
4.1 Knowledge Graph Construction
Within-document KG: We first construct a within-document IE-based knowledge graph for each document. We leverage OneIE (Lin et al., 2020), a joint IE system, to extract the entities, relations, and events contained in a given document. Then, we conduct entity linking (Pan et al., 2017) and entity coreference resolution (Lee et al., 2017) to merge multiple mentions of the same entities together. Eventually, we obtain a within-document KG where entities and events are nodes, relations are edges between entities, and arguments are edges between events and entities.
Cross-document KG: We leverage cross-document event coreference resolution to connect the within-document KGs into a cross-document KG as illustrated in Figure 2. We employ a cross-document event coreference resolution system (Lai et al., 2021; Wen et al., 2021) to identify clusters of events from multiple documents that refer to the same real-world events. The system utilizes both textual contexts of the event mentions and symbolic features such as the event type information. An example of the detected event cluster is shown in Table 1, where the four events of four documents all refer to the same explosion attack on Venezuela's President Nicolas Marduro. These four events contain complementary or contradictory details, which can be used for misinformation detection. For each event cluster, we add a node to represent the overall information of the real-world complex event corresponding to the cluster. Then, an edge is added between each event node and corresponding cluster node to allow reasoning among cross-document coreferential events.
To indicate which document each entity or event belongs to and capture the global information of each document, we further introduce a document node and connect it to the associated entity and event nodes for each document.
The resulting KG contains 4 types of nodes (i.e. entity nodes, event nodes, document nodes, and event cluster nodes) and 5 types of edges (i.e. relation edges, event argument edges, document-to-entity edges, document-to-event edges, and edges connecting event nodes to event cluster nodes).
Real	··· Venezuela's president, Nicolás Maduro, has survived an apparent assassination attempt after what of officials described as drones armed with explosivesarg1 detonatedtrig overhead during a speech he was making at a military event.···
Real	··· The BBC quotes anonymous firefighters at the scene who say “the incident was actually a gas tank explosiontrig inside an apartmentarg2, but did not provide further details.”···
Fake	··· Maduro was not targeted by the drones, the prime minister said, but state security services reported that the drones were meant for him. “The explosiontrig was caused by two machine gunsarg1,” Maduro said, adding that there were no injuries.···
Fake	··· Two drones armed with explosives detonatedtrig near PuntoDeCortearg2, where the Venezuelan Foreign Minister, Jorge Rodríguez, was performing, and near the stage where he was giving a speech.···
Table 1: An example of cross-document event cluster from IED dataset, where trig, arg1 and arg2 represent the trigger, ExplosiveDevice argument and Place argument respectively. The four events from four documents all refer to the explosion attack on Nicolas Marduro. The two real news articles complement each other by providing different aspects of the event (ExplosiveDevice argument in the first news and Place argument in the second news), while the two fake news articles contradict the real news with different details (i.e., different ExplosiveDevice and Place arguments).
Since all edges are directional, we add an inverse edge for each edge to propagate features along both directions, and the final KG contains 10 edge types, accounting for the inverse of existing edge types.
KG representation: We use BERT (Devlin et al., 2019) to initialize the node and edge embeddings in the KG. For a document node, we use BERT to encode the entire document and take the embeddings of [CLS] tokens. Similarly, for an entity node, we encode its canonical mention. For an event node, we encode the sentence where the event trigger occurs. For an event cluster node, we take the average of the embeddings of all events in the cluster. For a relation edge or an event argument role edge, we encode the linearized representation of the relation tuple. For example, the Leadership relation between "Nicolas Maduro" and "Venezuelan" is represented by "Nicolas Maduro, Leadership, Venezuelan", and "guns" as the ExplosiveDevice argument of the DetonateExplode event is represented by "DetonateExplode, ExplosiveDevice, guns".
4.2 Knowledge Graph Encoder
Heterogeneous GNN: Given the heterogeneous nature of the cross-document KG, we adopt a het-
erogeneous GNN to encode the KG.
Formally, let  $\mathcal{G}$  denote KG and  $\nu$  denote the nodes in  $\mathcal{G}$ . We use  $\mathcal{R}$  to denote the 10 types of edges as discussed in the previous section, and for each edge type  $r\in \mathcal{R}$ , we use  $\mathcal{G}_r$  to denote the subgraph of  $\mathcal{G}$  that only contains edges of type  $r$ . At the  $l$ -th layer, the inputs are output features produced by the previous layer denoted as  $\mathbf{h}_i^{(l - 1)}, i\in \mathcal{V}$ . For each edge type  $r\in \mathcal{R}$ , we apply a separate GNN to encode  $\mathcal{G}_r$  and produce a set of features denoted as  $\mathbf{h}_{i,r}^{(l)}$ . Then, we aggregate the outputs for all edge types into the final output as follows:
$$
\mathbf {h} _ {i} ^ {(l)} = \sum_ {r \in \mathcal {R}} \mathbf {h} _ {i, r} ^ {(l)} / | \mathcal {R} | \tag {1}
$$
For document-to-entity edges, document-to-event edges, and edges connecting event nodes to event cluster nodes, we use standard graph attention network (GAT). For relation edges and event argument edges, we apply edge-aware GAT to leverage the edge features. Here, the edge features refer to the BERT embeddings of text descriptions such as "Nicolas Maduro, Leadership, Venezuelan" or "DetonateExplode, ExplosiveDevice, guns" as described in Section 4.1. The remainder of Section 4.2 presents details of GAT and edge-aware GAT, i.e., how to produce  $\mathbf{h}_{i,r}^{(l)}$  based on  $\mathbf{h}_i^{(l - 1)}$ .
Graph attention network: For each given node, GAT aggregates the node features of its neighbors via attention mechanism (Velickovic et al., 2018). For a given edge type  $r \in \mathcal{R}$ , let  $\mathcal{N}_{i,r}$  denote the neighbors of node  $i$  in  $\mathcal{G}_r$ . At the  $l$ -th layer, the attention weights  $\alpha_{ij}$  are calculated as follows:
$$
e _ {i j} = \text {L e a k y R e L U} \left(\mathbf {a} ^ {\top} \left[ \mathbf {W h} _ {i} ^ {(l - 1)} \| \mathbf {W h} _ {j} ^ {(l - 1)} \right]\right) \tag {2}
$$
$$
\alpha_ {i j} = \operatorname {s o f t m a x} _ {j} \left(e _ {i j}\right) = \frac {\exp \left(e _ {i j}\right)}{\sum_ {k \in \mathcal {N} _ {i , r}} \exp \left(e _ {i k}\right)} \tag {3}
$$
where  $\mathbf{a}$  and  $\mathbf{W}$  are trainable parameters, and  $\parallel$  denotes the feature concatenation. The output features  $\mathbf{h}_{i,r}^{(l)}$  for node  $i$  in  $\mathcal{G}_r$  are calculated as follows:
$$
\mathbf {h} _ {i, r} ^ {(l)} = \sum_ {j \in \mathcal {N} _ {i, r}} \alpha_ {i j} \mathbf {W h} _ {j} ^ {(l - 1)} \tag {4}
$$
Edge-aware graph attention network: Edge-aware GAT is an extension of GAT that considers edge features in addition to node features (Huang et al., 2020; Yasunaga et al., 2021). Let  $\mathbf{r}_{ij}$  denote
the features of the edge between nodes  $i$  and  $j$ . For a given edge type  $r \in \mathcal{R}$ , at the  $l$ -th layer, the attention weights  $\alpha_{ij}$  are computed as follows:
$$
\mathbf {r} _ {i j} ^ {\prime} = \mathbf {W} ^ {r} \left[ \mathbf {h} _ {i} ^ {(l - 1)} \| \mathbf {h} _ {j} ^ {(l - 1)} \| \mathbf {r} _ {i j} \right] \tag {5}
$$
$$
\alpha_ {i j} = \operatorname {s o f t m a x} _ {j} \left(\left(\mathbf {W} ^ {Q} \mathbf {h} _ {i} ^ {(l - 1)}\right) \left(\mathbf {W} ^ {K} \mathbf {r} _ {i j} ^ {\prime}\right) ^ {\top}\right) \tag {6}
$$
where  $\mathbf{W}^r$ ,  $\mathbf{W}^Q$  and  $\mathbf{W}^K$  are trainable parameters. The output features  $\mathbf{h}_{i,r}^{(l)}$  for node  $i$  in  $\mathcal{G}_r$  are computed as follows:
$$
\mathbf {h} _ {i, r} ^ {(l)} = \sum_ {j \in \mathcal {N} _ {i, r}} \alpha_ {i j} \mathbf {W} ^ {V} \mathbf {r} _ {i j} ^ {\prime} \tag {7}
$$
where  $\mathbf{W}^V$  is a learnable matrix.
4.3 Misinformation Detector
Using the previously described graph encoder, we are able to obtain representations of the document and event nodes. We conduct document-level detection using the document node representations, and event-level detection using the event node representations. We separately train two detectors for these two levels of tasks.
However, these two tasks are not mutually independent. Intuitively, document-level detection can benefit from the results of event-level detection, because the presence of a large number of false events indicates that the document is more likely to be fake. Therefore, we feed the results produced by a well-trained event-level detector into each layer of the document-level detector. Let  $\mathbf{e}_i$  denote the representations of node  $i$  produced by the event-level detector. At the  $l$ -th layer of the document-level detector, instead of using the output features of the previous layer  $\mathbf{h}_i^{(l-1)}$  as input features, we use a linear projection of the concatenation of  $\mathbf{e}_i$  and  $\mathbf{h}_i^{(l-1)}$  calculated as follows:
$$
\mathbf {W} _ {\text {p r o j}} ^ {(l)} \left[ \mathbf {e} _ {i} \| \mathbf {h} _ {i} ^ {(l - 1)} \right] \tag {8}
$$
where  $\mathbf{W}_{\mathrm{proj}}^{(l)}$  is a learnable matrix.
5 Dataset Construction
Currently, there are no existing resources for cross-document misinformation detection. We propose to construct datasets based on real news datasets with clustering information. For each cluster, we randomly sample  $50\%$  real news and replace them with manipulated fake news. Figure 3 shows an
Real news
... The device exploded over an area where uniformed guardsmen stood at attention in ranks, injuring seven. Advertisement ...
Manipulated KG
Generated fake news
... "The explosion was caused by two machine guns," Maduro said, adding that there were no injuries. ...
KG of fake news
Figure 3: An overview of the fake news generation process. Based on the real news and its IE output, we select a high-frequency "DetonateExplode" event and replace its argument entity "device" with "machine guns". We then generate the fake news from the manipulated KG. In the KG of generated fake news, the manipulated entity "guns" is an argument of the "DetonateExplode" event, so we consider the event as fake.
overview of the fake news generation process, and more examples are presented in Appendix D.
Following Fung et al. (2021), we train a KG-to-text generator from the real news in our datasets, and generate fake news from manipulated KGs. The main differences between Fung et al. (2021)'s method and ours in terms of manipulating KG are: (1) we only conduct entity swapping, and do not adopt other types of manipulation including adding relations or events and subgraph replacement; (2) since we focus on events, we select entities to be replaced that are arguments of high-frequency events, instead of based on entity node degree; (3) we select entities from other documents in the same document cluster to replace the original entities, so that the entities before and after replacement are more similar.
We record the manipulation operations, and use a heuristic rule to obtain supervision for event-level detection as explained below. In a fake document, if an event involves manipulated entities as arguments, we consider this event as fake.