## 5 The Proposed Approaches  
### Bert\({}_{base}\)-Cnn-BiGRU-Att  
In this subsection, we will introduce the first deep learning architecture proposed in this paper for fake news detection. Specifically, as illustrated in Figure 1, we first perform BERT\({}_{base}\) on input text--suppose \(x_{ij}\in\mathbb{R}\), represents the d-dimensional word embedding of the \(j\)-th word in the \(i\)-th sentence, to obtain deep contextualized representations since BERT\({}_{base}\) plays a bidirectional attention mechanism on input text, ensuring better coverage of global semantic understanding. Then, a CNN layer is applied to capture local features as follows. Given a sentence, assume \(X_{1:i:m}=x_{11}\oplus x_{12}\oplus\ldots x_{1m}\), where \(m\) denotes the length of a given sentence \(i\) and \(\oplus\) refers to a concatenation operation. A feature map \(F\) is generated by performing a convolutional operation with a filter \(T\in\mathbb{R}^{h\times d}\) on \(X_{ii+h-1}\) using the equation \(f_{i}=\sigma(T\cdot X_{zi+h-1}+b)\) where \(\sigma\) represents a non-linear activation function and \(b\) denotes a bias term. The final feature map \(F\) is obtained by applying the previous step to all sliding windows of \(h\) words in a sentence. This \(F\) is fed into a max-pooling layer and to a global average pooling layer for capturing the most relevant and salient information, generating \(F_{nt}=[F_{m1},F_{m2},\ldots,F_{mk}]\) and \(F_{g}=[F_{g1},F_{g2},\ldots,F_{gk}]\), respectively, with \(k\) represents the number of filters used. The resulting feature map is max-pooled and global averaged-pool. In addition, BiGRU is applied to the output of the CNN layer. The BiGRU layer consists of a forward \(\overrightarrow{GRU}\) and a backward \(\overleftarrow{GRU}\) to process the information from both left-to-right and right-to-left directions.  
\[\overrightarrow{h_{ij}}=\overrightarrow{GRU}(F,h_{i(j-1)}),j\in\{1,\ldots,m\} \tag{11}\]  
\[\overleftarrow{h_{ij}}=\overleftarrow{GRU}(F,h_{i(j-1)}),j\in\{m,\ldots,1\} \tag{12}\]  
We denote the combination of the hidden state obtained from both the forward and backward GRUs as \(h_{ij}=[\overrightarrow{h_{ij}}\oplus\overrightarrow{h_{ij}}]\), which, in turn, captures full salient information about a given input text. In order to better capture salient and relevant features from the final output of BiGRU networks, the attention mechanism (see Equations (13)-(15)) is applied to the resultant output of BiGRU and the models, to assign different weights to different parts of the given input.  
\[u_{ij}=tanh(W_{w}h_{ij}+b_{w}) \tag{13}\]  
\[\alpha_{ij}=\frac{\exp(u_{ij}u_{w})}{\sum_{j}\exp(u_{ij}u_{w})} \tag{14}\]  
where \(u_{ij}\) is a hidden representation of \(h_{ij}\), \(u_{w}\) is a trainable parameter that is randomly initialized during the course of training, and the result of the product \((u_{ij}u_{w})\) is used to calculate the importance score of word \(j\), which is then normalized using the softmax function, generating the importance weight \(\alpha_{ij}\) for each word.  
Finally, the weighted sum of the concatenated word representations with the hidden states \(h_{ij}\) form the news vector \(n_{i}\).  
\[n_{i}=\sum_{1}\alpha_{ij}h_{ij} \tag{15}\]Ultimately, we concatenate the final output as follows: \(C=F_{m}\oplus F_{g}\oplus n_{i}\), where \(\oplus\) denotes the concatenation operation. We encode user posting behavior features using BiGRU. Assume \(U\) is the final representation of users encoded features, a concatenation layer is used to extract the final informative features of the two components (news text and user posting behavior features) as follows: \(O=U\oplus C\), where \(O\) is the final output representation of the given news article with its associated user-posted behavioral cues. Finally, the output is passed to a classification layer with one unit, activated using the sigmoid function.