SlowGuess/NAACL_2022_OCR huggingface dataset:
Hongzhan Lin $^{1,2}$ , Jing Ma $^{2,*}$ , Liangliang Chen $^{1}$ , Zhiwei Yang $^{3}$ , Mingfei Cheng $^{1}$ , Guang Chen $^{1,*}$
<sup>1</sup>Beijing University of Posts and Telecommunications, Beijing, China
$^{2}$ Hong Kong Baptist University, Hong Kong SAR, China
$^{3}$ Jilin University, Changchun, China
{linhongzhan,outside,mingfeicheng,chenguang}@bupt.edu.cn
majing@comp.hkbu.edu.hk, yangzw18@mails.jlu.edu.cn
Massive false rumors emerging along with breaking news or trending topics severely hinder the truth. Existing rumor detection approaches achieve promising performance on the yesterday's news, since there is enough corpus collected from the same domain for model training. However, they are poor at detecting rumors about unforeseen events especially those propagated in different languages due to the lack of training data and prior knowledge (i.e., low-resource regimes). In this paper, we propose an adversarial contrastive learning framework to detect rumors by adapting the features learned from well-resourced rumor data to that of the low-resourced. Our model explicitly overcomes the restriction of domain and/or language usage via language alignment and a novel supervised contrastive training paradigm. Moreover, we develop an adversarial augmentation mechanism to further enhance the robustness of low-resource rumor representation. Extensive experiments conducted on two low-resource datasets collected from real-world microblog platforms demonstrate that our framework achieves much better performance than state-of-the-art methods and exhibits a superior capacity for detecting rumors at early stages.
With the proliferation of social media such as Twitter and Weibo, the emergence of breaking events provides opportunities for the spread of rumors, which is difficult to be identified due to limited domain expertise and relevant data. For instance, along with the unprecedented COVID-19 pandemic, a false rumor claimed that "everyone who gets the vaccine will die or suffer from auto-immune diseases" was translated into many languages and spread at lightning speed on social
media, which seriously confuses the public and destroys the achievements of epidemic prevention in related countries or regions of the world. Although some recent works focus on collecting microblog posts corresponding to COVID-19 (Chen et al., 2020a; Zarei et al., 2020; Alqurashi et al., 2020), existing rumor detection methods perform poorly without a large-scale qualified training corpus, i.e., in a low-resource scenario (Hedderich et al., 2021). Thus there is an urgent need to develop automatic approaches to identify rumors in such low-resource domains especially amid breaking events.
Social psychology literature defines a rumor as a story or a statement whose truth value is unverified or deliberately false (Allport and Postman, 1947). Recently, techniques using deep neural networks (DNNs) (Ma et al., 2018; Khoo et al., 2020; Bian et al., 2020) have achieved promising results for detecting rumors on microblogging websites by learning rumor-indicative features from sizeable rumor corpus with veracity annotation. However, such DNN-based approaches are purely data-driven and have a major limitation on detecting emerging events concerning about low-resource domains, i.e., the distinctive topic coverage and word distribution (Silva et al., 2021) required for detecting low-resource rumors are often not covered by the public benchmarks (Zubiaga et al., 2016; Ma et al., 2016, 2017). On another hand, for rumors propagated in different languages, existing monolingual approaches are not applicable since there are even no sufficient open domain data for model training in the target language.
In this paper, we assume that the close correlations between the well-resourced rumor data and the low-resourced could break the barriers of domain and language, substantially boosting low-resource rumor detection within a more general framework. Taking the breaking event COVID-19 as an example, we collect corresponding humorous and non-rumorous claims with propaganda
1https://www.bbc.com/news/
uk-wales-58103604
Findings of the Association for Computational Linguistics: NAACL 2022, pages 2543 - 2556
(a) TWITTER (Rumor)
(b) Twitter-COVID19 (Rumor)
(c) Weibo-COVID19 (Rumor)
(d) TWITTER (Non-rumor)
(e) Twitter-COVID19 (Non-rumor)
(f) Weibo-COVID19 (Non-rumor)
Figure 1: Word clouds of rumor and non-rumor data generated from TWITTER, Twitter-COVID19, and Weibo-COVID19 datasets, where the size of terms corresponds to the word frequency. Both TWITTER and Twitter-COVID19 are presented in English while Weibo-COVID19 in Chinese.
tion threads from Twitter and Sina Weibo which are the most popular microblogging websites in English and Chinese, respectively. Figure 1 illustrates the word clouds of rumor and non-rumor data from an open domain benchmark (i.e., TWITTER (Ma et al., 2017)) and two COVID-19 datasets (i.e., Twitter-COVID19 and Weibo-COVID19). It can be seen that both TWITTER and Twitter-COVID19 contain denial opinions towards rumors, e.g., "fake", "joke", "stupid" in Figure 1(a) and "wrong symptom", "exactly sick", "health panic" in Figure 1(b). In contrast, supportive opinions towards non-rumors can be drawn from Figure 1(d)-1(e). Moreover, considering that COVID-19 is a global disease, massive misinformation could be widely propagated in different languages such as Arabic (Alam et al., 2020), Indic (Kar et al., 2020), English (Cui and Lee, 2020) and Chinese (Hu et al., 2020). Similar identical patterns can be observed in Chinese on Weibo from Figure 1(c) and Figure 1(f). Although the COVID-19 data tend to use expertise words or language-related slang, we argue that aligning the representation space of identical rumor-indicative patterns of different domains and/or languages could adapt the features captured from well-resourced data to that of the low-resourced.
To this end, inspired by contrastive learning (He et al., 2020; Chen et al., 2020b,c), we propose an Adversarial Contrastive Learning approach for low-resource rumor detection (ACLR), to encourage effective alignment of rumor-indicative features in the well-resourced and low-resource data. More specifically, we first transform each microblog post into a language-independent vector by semantically
aligning the source and target language in a shared vector space. As the diffusion of rumors generally follows a propagation tree that provides valuable clues on how a claim is transmitted (Ma et al., 2018), we thus resort to a structure-based neural network (Bian et al., 2020) to catch informative patterns. Then, we propose a novel supervised contrastive learning paradigm to minimize the intraclass variance of source and target instances with same veracity, and maximize inter-class variance of instances with different veracity. To further enhance the feature adaption of contrastive learning, we exploit adversarial attacks (Kurakin et al., 2016) toplenish noise to the original event-level representation by computing adversarial worst-case perturbations, forcing the model to learn non-trivial but effective features. Extensive experiments conducted on two real-world low-resource datasets confirm that (1) our model yields outstanding performances for detecting low-resource rumors over the state-of-the-art baselines with a large margin; and (2) our method performs particularly well on early rumor detection which is crucial for timely intervention and debunking especially for breaking events. The main contributions of this paper are of three-fold:
- To our best knowledge, we are the first to present a radically novel adversarial contrastive learning framework to study the low-resource rumor detection on social media $^2$ .
- We propose supervised contrastive learning
2Our resources will be available at https://github.com/DanielLin97/ACLR4RUMOR-NAACL2022.
for structural feature adaption between different domains and languages, with adversarial attacks employed to enhance the diversity of low-resource data for contrastive paradigm.
- We constructed two low-resource microblog datasets corresponding to COVID-19 with propagation tree structure, respectively gathered from English tweets and Chinese microblog posts. Experimental results show that our model achieves superior performance for both rumor classification and early detection tasks under low-resource settings.
Pioneer studies for automatic rumor detection focus on learning a supervised classifier utilizing features crafted from post contents, user profiles, and propagation patterns (Castillo et al., 2011; Yang et al., 2012; Liu et al., 2015). Subsequent studies then propose new features such as those representing rumor diffusion and cascades (Kwon et al., 2013; Friggeri et al., 2014; Hannak et al., 2014). Zhao et al. (2015) alleviate the engineering effort by using a set of regular expressions to find questing and denying tweets. DNN-based models such as recurrent neural networks (Ma et al., 2016), convolutional neural networks (Yu et al., 2017), and attention mechanism (Guo et al., 2018) are then employed to learn the features from the stream of social media posts. However, these approaches simply model the post structure as a sequence while ignoring the complex propagation structure.
To extract useful clues jointly from content semantics and propagation structures, some approaches propose kernel-learning models (Wu et al., 2015; Ma et al., 2017) to make a comparison between propagation trees. Tree-structured recursive neural networks (RvNN) (Ma et al., 2018) and transformer-based models (Khoo et al., 2020; Ma and Gao, 2020) are proposed to generate the representation of each post along a propagation tree guided by the tree structure. More recently, graph neural networks (Bian et al., 2020; Lin et al., 2021a) have been exploited to encode the conversation thread for higher-level representations. However, such data-driven approaches fail to detect rumors in low-resource regimes (Janicka et al., 2019) because they often require sizeable training data which is not available for low-resource domains and/or languages. In this paper, we propose a novel framework to adapt existing models with the effective
propagation structure for detecting rumors from different domains and/or languages.
To facilitate related fact-checking tasks in low-resource settings, domain adaption techniques are utilized to detect fake news (Wang et al., 2018; Yuan et al., 2021; Zhang et al., 2020; Silva et al., 2021) by learning features from multi-modal data such as texts and images. Lee et al. (2021) proposed a simple way of leveraging the perplexity score obtained from pre-trained language models (LMs) for the few-shot fact-checking task. Different from these works of adaption on multi-modal data and transfer learning of LMs, we focus on language and domain adaptation to detect rumors from low-resource microblog posts corresponding to breaking events.
Contrastive learning (CL) aims to enhance representation learning by maximizing the agreement among the same types of instances and distinguishing from the others with different types (Wang and Isola, 2020). In recent years, CL has achieved great success in unsupervised visual representation learning (Chen et al., 2020b; He et al., 2020; Chen et al., 2020c). Besides computer vision, recent studies suggest that CL is promising in the semantic textual similarity (Gao et al., 2021; Yan et al., 2021), stance detection (Mohtarami et al., 2019), short text clustering (Zhang et al., 2021), unknown intent detection (Lin et al., 2021b), and abstractive summarization (Liu and Liu, 2021), etc. However, the above CL frameworks are specifically proposed to augment unstructured textual data such as sentence and document, which are not suitable for the low-resource rumor detection task considering claims together with more complex propagation structures of community response.
In this work, we define the low-resource rumor detection task as: Given a well-resourced dataset as source, classify each event in the target lowresource dataset as a rumor or not, where the source and target data are from different domains and/or languages. Specifically, we define a well-resourced source dataset for training as a set of events  $\mathcal{D}_s = \{C_1^s,C_2^s,\dots ,C_M^s\}$ , where  $M$  is the number of source events. Each event  $C^s = (y,c,\mathcal{T}(c))$  is a tuple representing a given claim  $c$  which is associated with a veracity label  $y\in \{\text{rumor, non-rumor}\}$ , and ideally all its relevant responsive microblog post in chronolog-
Figure 2: The overall architecture of our proposed method. For source and small target training data, we first obtain post-level representations after cross-lingual sentence encoding, then train the structure-based network with the adversarial contrastive objective. For target test data, we extract the event-level representations to detect rumors.
ical order, i.e.,  $\mathcal{T}(c) = \{c,x_1^s,x_2^s,\dots ,x_{|C|}^s\} ^3$  where  $|C|$  is the number of responsive tweets in the conversation thread. For the target dataset with low-resource domains and/or languages, we consider a much smaller dataset for training  $\mathcal{D}_t = \{C_1^t,C_2^t,\dots ,C_N^t\}$ , where  $N(N\ll M)$  is the number of target events and each  $C^t = (y,c',\mathcal{T}(c'))$  has the similar composition structure of the source dataset.
We formulate the task of low-resource rumor detection as a supervised classification problem that trains a domain/language-agnostic classifier  $f(\cdot)$  adapting the features learned from source datasets to that of the target events, that is,  $f(C^{t}|\mathcal{D}_{s})\to y$ . Note that although the tweets are notated sequentially, there are connections among them based on their responsive relationships. So most previous works represent the conversation thread as a tree structure (Ma et al., 2018; Bian et al., 2020).
In this section, we introduce our adversarial contrastive learning framework to adapt the features captured from the well-resourced data to detect rumors from low-resource events, which considers cross-lingual and cross-domain alignment. Figure 2 illustrates an overview of our proposed model, which will be depicted in the following subsections.
4.1 Cross-lingual Sentence Encoder
Given a post in an event that could be either from source or target data, to map it into a shared semantic space where the source and target lan
guages are semantically aligned, we utilize XLM-RoBERTa (Conneau et al., 2019) (XLM-R) to model the context interactions among tokens in the sequence for the sentence-level representation:
$$
\bar {x} = X L M - R (\mathbf {x}) \tag {1}
$$
where  $\mathbf{x}$  is the original post, and we obtain the postlevel representation  $\bar{x}$  using the output state of the  $
$  token in XLM-R. We thus denote the representation of posts in the source event  $C^s$  and the target event  $C^t$  as a matrix  $X^s$  and  $X^t$  respectively:  $X^* = [\bar{x}_0^*,\bar{x}_1^*,\bar{x}_2^*,\dots,\bar{x}_{|X^*|-1}^*]^\top$ ;  $*\in \{s,t\}$ , where  $X^s \in \mathbb{R}^{m\times d}$  and  $X^t \in \mathbb{R}^{n\times d}$ ,  $d$  is the dimension of the output state of the sentence encoder.
4.2 Propagation Structure Representation
On top of the sentence encoder, we represent the propagation of each claim with the graph convolutional network (GCN) (Kipf and Welling, 2016), which achieves state-of-the-art performance on capturing both structural and semantic information for rumor classification (Bian et al., 2020). It is worth noting that the choice of propagation structure representation is orthogonal to our proposed framework that can be easily replaced with any existing structure-based models without any other change to our supervised contrastive learning architecture.
Given an event and its initialized embedding matrix  $C^*, X^*$ ;  $* \in \{s, t\}$ , We model the conversation thread of the event as a tree structure  $\mathcal{T} = \langle V, E \rangle$  where  $V$  consists of the event claim and all its relevant responsive posts as nodes and  $E$  refers to a set of directed edges corresponding to the response relation among the nodes in  $V$ . Inspired by Ma et al.
${}^{3}c$  is equivalent to  ${x}_{0}^{s}$  .
(2018), here we consider two different propagation trees with distinct edge directions: (1) Top-Down tree where the edge follows the direction of information diffusion. (2) Bottom-Up tree where the responsive nodes point to their responded nodes, similar to a citation network.
Top-Down GCN. We treat the Top-Down tree structure as a graph and transform the edge  $E$  into an adjacency matrix  $\mathbf{A} \in \{0,1\}^{|V|\times |V|}$ , where  $\mathbf{A}_{i,j} = 1$  if  $\mathbf{x}_i$  has a response to  $\mathbf{x}_j$  or  $i = j$ , else  $\mathbf{A}_{i,j} = 0$ . Then we utilize a layer-wise propagation rule to update the node vector at the  $l$ -th layer:
$$
H ^ {(l + 1)} = \operatorname {R e L U} \left(\hat {\mathbf {A}} \cdot H ^ {(l)} \cdot W ^ {(l)}\right) \tag {2}
$$
where  $\hat{\mathbf{A}} = \mathbf{D}^{-1 / 2}\mathbf{A}\mathbf{D}^{-1 / 2}$  is the symmetric normalized adjacency matrix,  $\mathbf{D}$  denotes the degree matrix of  $\mathbf{A}$ .  $W^{(l)}\in \mathbb{R}^{d^{(l)}\times d^{(l + 1)}}$  is a layer-specific trainable transformation matrix.  $H^{(0)}$  is initialized as  $X^{*}$ . For a GCN model with  $L$ -layers, we obtain the final node representation  $H_{TD}$  w.r.t  $H^{(L)}$ .
Bottom-Up GCN. We also leverage the structure of Bottom-Up tree to encode the informative posts. Similar to Top-Down GCN, we update the hidden representation of nodes in the same manner as Eq. 2 and finally get the output node states  $H_{BU}$  at the  $L$ -th graph convolutional layer.
The Overall Model. Finally, we concatenate  $H_{TD}$  and  $H_{BU}$  via mean-pooling to jointly capture the opinions expressed in both Top-Down and Bottom-Up trees:
$$
o = \text {m e a n - p o o l i n g} \left(\left[ H _ {T D}; H _ {B U} \right]\right) \tag {3}
$$
where  $o \in \mathbb{R}^{2d^{(L)}}$  is the event-level representation of the entire propagation thread,  $d^{(L)}$  is the output dimension of GCN and  $[\cdot; \cdot]$  means concatenation.
4.3 Contrastive Training
To align the representation space of rumor-indicative signals from different domains and languages, we present a novel training paradigm to exploit the labeled data including rich sourced data and small-scaled target data to adapt our model on target domains and languages. The core idea is to make the representations of source and target events from the same class closer while keeping representations from different classes far away.
Given an event  $C_i^s$  from the source data, we firstly obtain the language-agnostic encoding for all the involved posts (see Eq. 1) as well as the propagation structure representation  $o_i^s$  (see Eq. 3) which is then fed into a softmax function to make rumor predictions. Then, we learn to minimize the
cross-entropy loss between the prediction and the ground-truth label  $y_{i}^{s}$ :
$$
\mathcal {L} _ {C E} ^ {s} = - \frac {1}{N ^ {s}} \sum_ {i = 1} ^ {N ^ {s}} \log \left(p _ {i}\right) \tag {4}
$$
where  $N^s$  is the total number of source examples in the batch,  $p_i$  is the probability of correct prediction. To make rumor representation in the source events be more discriminative, we propose a supervised contrastive learning objective to cluster the same class and separate different classes of samples:
$$
\begin{array}{l} \mathcal {L} _ {S C L} ^ {s} = - \frac {1}{N ^ {s}} \sum_ {i = 1} ^ {N ^ {s}} \frac {1}{N _ {y _ {i} ^ {s}} - 1} \sum_ {j = 1} ^ {N ^ {s}} \mathbb {1} _ {[ i \neq j ]} \mathbb {1} _ {[ y _ {i} ^ {s} = y _ {j} ^ {s} ]} \\ \log \frac {\exp \left(\operatorname {s i m} \left(o _ {i} ^ {s} , o _ {j} ^ {s}\right) / \tau\right)}{\sum_ {k = 1} ^ {N ^ {s}} \mathbb {1} _ {[ i \neq k ]} \exp \left(\operatorname {s i m} \left(o _ {i} ^ {s}, o _ {k} ^ {s}\right) / \tau\right)} \tag {5} \\ \end{array}
$$
where  $N_{y_i^s}$  is the number of source examples with the same label  $y_i^s$  in the event  $C_i^s$ , and  $\mathbb{1}$  is the indicator.  $\mathrm{sim}(\cdot)$  denotes the cosine similarity function and  $\tau$  controls the temperature.
For an event  $C_i^t$  from the target data, we also compute the classification loss  $\mathcal{L}_{CE}^t$  in the same manner as Eq. 4. Although we projected the source and target languages into the same semantic space after sentence encoding, rumor detection not only relies on post-level features, but also on event-level contextual features. Without constraints, the structure-based network can only extract event-level features for all samples based on their final classification signals while these features may not be critical to the target domain and language. We make full use of the minor labels in the low-resource rumor data by parameterizing our model according to the contrastive objective between the source and target instances in the event-level representation space:
$$
\begin{array}{l} \mathcal {L} _ {S C L} ^ {t} = - \frac {1}{N ^ {t}} \sum_ {i = 1} ^ {N ^ {t}} \frac {1}{N _ {y _ {i} ^ {t}}} \sum_ {j = 1} ^ {N ^ {s}} \mathbb {1} _ {[ y _ {i} ^ {t} = y _ {j} ^ {s} ]} \\ \log \frac {\exp \left(\operatorname {s i m} \left(o _ {i} ^ {t} , o _ {j} ^ {s}\right) / \tau\right)}{\sum_ {k = 1} ^ {N ^ {s}} \exp \left(\operatorname {s i m} \left(o _ {i} ^ {t} , o _ {k} ^ {s}\right) / \tau\right)} \tag {6} \\ \end{array}
$$
where  $N^t$  is the total number of target examples in the batch and  $N_{y_i^t}$  is the number of source examples with the same label  $y_i^t$  in the event  $C_i^t$ . As a result, we project the source and target samples belonging to the same class closer than that of different categories, for feature alignment with minor
Algorithm 1 Adversarial Contrastive Learning
Input: A small set of events  $C_i^t$  in the target domain and language; A set of events  $C_i^s$  in the source domain and language.
Output: Assign rumor labels  $y$  to given unlabeled target data.
1: for each mini-batch  $N^t$  of the target events  $C_i^t$  do:
2: for each mini-batch  $N^s$  of the source events  $C_i^s$  do:
3: Pass  $C_i^*$  to the sentence encoder and then structure-based network to obtain its event-level feature  $o_i^*$ , where  $* \in \{s, t\}$ .
4: Compute the classification loss  $\mathcal{L}_{CE}^{*}$  for source and target data, respectively.
5: Adversarial augmentation for target data and update  $\mathcal{L}_{CE}^{t}$
6: Compute the supervised contrastive loss  $\mathcal{L}_{SCL}^*$ .
7: Compute the joint loss  $\mathcal{L}^*$  as Eq. 8.
8: Jointly optimize all parameters of the model using the average loss  $\mathcal{L} = \mathrm{mean}(\mathcal{L}^s +\mathcal{L}^t)$
annotation at the target domain and language.
4.4 Adversarial Data Augmentation
Data augmentation techniques were successfully utilized to enhance contrastive learning models (Chen et al., 2020b). Some simple augmentation strategies are designed based on handcrafted features or rules, but they are not efficient and suitable for the propagation tree structures in rumor detection task. In this section, we introduce adversarial attacks to generate pseudo target samples at the event-level latent space to increase the diversity of views for model robustness in the contrastive learning manner. Specifically, we apply Fast Gradient Value (Miyato et al., 2016; Vedula et al., 2020) to approximate a worst-case perturbation as a noise vector of the event-level representation:
$$
\tilde {\boldsymbol {o}} _ {\text {n o i s e}} ^ {t} = \epsilon \frac {g}{| | g | |}; \text {w h e r e} g = \nabla_ {o ^ {t}} \mathcal {L} _ {C E} ^ {t} \tag {7}
$$
where the gradient is the first-order differential of the classification loss  $\mathcal{L}_{CE}^{t}$  for a target sample, i.e., the direction that rapidly increases the classification loss. We perform normalization and use a small  $\epsilon$  to ensure the approximate is reasonable. Finally, we can obtain the pseudo augmented sample  $o_{adv}^{t} = o^{t} + \tilde{o}_{noise}^{t}$  in the latent space to enhance our model.
4.5 Model Training
We jointly train the model with the cross-entropy and supervised contrastive objectives: