# An Interpretable Fake News Detection Method Based on Commonsense Knowledge Graph
## 3 Extraction of Commonsense Knowledge
### Bi-GRU Based Entity Extraction  
Entity extraction [26] is to automatically identify named entities from a commonsense text corpus. Since entities are the most basic elements in a knowledge graph, their extraction completeness, accuracy, recall rate, and so on will directly affect the quality of the knowledge base. Therefore, entity extraction is the most basic and key step in knowledge extraction. The entity extraction model used in this paper mainly adopts the Bi-GRU-CRF architecture [12]. The specific details are described next. The network structure of the LAC model is shown in Figure 1. Bidirectional GRU (Bi-GRU) is an extension of GRU, which is suitable for lexical analysis tasks with input as a whole sentence. Reverse GRU combines with forward GRU to form a Bi-GRU layer. These two GRUs accept the same input, but train in different directions, and connect their results as output. In order to more effectively represent some functions and models' variable-length dependency relationships, two Bi-GRUs are stacked together in the LAC model to form a deep network to improve representation ability. As shown in the model structure diagram, a Conditional Random Field (CRF) [27] layer is used at the top of the GRU structure to decode the final label sequence jointly. CRF is a discriminative probability model, which is often used for tagging or analyzing sequence data. Its input is provided by a full connection layer, which converts the output of the top Bi-GRU layer into an L-dimensional vector. L represents the number of all possible labels. At the same time, hard constraints are applied to the decoding process to emphasize the dependency relationship between the output labels. Any sequence that does not conform to IOB2 transmission rules will not be accepted. The LAC model takes the initial character sequence as the input of the model. Given a character sequence \(\{c_{1},c_{2}\dots,c_{T}\}\), each character, \(c_{i}\), in the vocabulary, V, is projected to a real-valued vector, \(\mathrm{e}(c_{i})\), by looking up a table. Then, with the embedding of characters [28] as input, a deep GRU neural network is built to learn the structural information of a given sentence. The relevant formulas for the Bi-GRU layer are as follows:  
\[u_{t}=\sigma_{g}(W_{tx}x_{t}+W_{uh}h_{t-1}+b_{u}) \tag{1}\]  
\[r_{t}=\sigma_{g}(W_{rx}x_{t}+W_{rh}h_{t-1}+b_{r}) \tag{2}\]  
\[\overset{\sim}{h_{t}}=\sigma_{c}(W_{cx}x_{t}+W_{ch}(r_{t}\odot h_{t-1})+b_{c}) \tag{3}\]  
\[h_{t}=(1-u_{t})\odot h_{t-1}+u_{t}\odot\overset{\sim}{h_{t}} \tag{4}\]  
\(\odot\) is the element-wise product of vectors, \(\sigma_{g}\) is the activation function, \(u_{t}\) is the update gate, \(r_{t}\) is the reset gate, and \(\sigma_{c}\) is the activation function of the hidden layer. Figure 1: Illustration of Bi-GRU-CRF network. The CRF layer learns the conditional probability, \(p(y|h)\), \(h=\{h_{1},h_{2}\ldots,h_{T}\}\) is the representation sequence generated by the topmost Bi-GRU layer, and \(\{y_{1},y_{2}\ldots,y_{T}\}\) is the label sequence. The probability model of linear-chain CRF defines a conditional probability family, \(p(y|h;t,s)\), over all possible label sequences, \(y\) given \(h\), which has the following form:  
\[p(y|h;t,s)=\frac{\prod_{i=1}^{T}\varphi_{i}(y_{i-1},y_{i},h)}{\sum_{y^{\prime} \in\gamma(h)}\prod_{i=1}^{T}\varphi_{i}(y^{\prime}_{i-1},y^{\prime}_{i^{\prime }},h)} \tag{5}\]  
and \(\gamma(h)\) represent all possible label sequences. \(\varphi_{i}(y_{i-1},y_{i},h)=\exp(\sum_{i=1}^{T}t(y_{i-1},y_{i},h)+s\)\((y_{i},h))\). \(t\) is the transition probability from \(y_{i}-1\) to \(y_{i}\) given the input sequence \(h\). \(s\) is the output of a linear function implicitly defined by a fully connected layer, which transforms the output of the topmost Bi-RNN at step \(i\) into a score for \(y_{i}\).