# Construction of an Industrial Knowledge Graph for Unstructured Chinese Text Learning
## 3 Methods
### Semantic Relation Extraction
#### 3.1.2 Relation Classification  
After the embedding layer, the original sentence becomes the corresponding sentence vector. As shown in Figure 7, we use a softmax classifier to predict relation \(y\) from sentence set \(S\), just as these following equations demonstrate:  
\[\begin{split} p(y|S)&=\text{softmax}(WS+b),\\ y&=\text{arg}\max p(y|S),\end{split} \tag{2}\]  
Figure 5: The architecture of the GRU(gate recurrent unit) block.  
Figure 6: The architecture of the bidirectional GRU.  
where \(W\) is a trained parameter vector and \(b\) is a bias, and \(n\) indicates the number of sentence sets. The loss function is defined as \(J\):  
\[J(\theta)=-\frac{1}{m}\sum_{i=1}^{m}r_{i}\log\left(y_{i},\theta\right), \tag{3}\]  
where \(r\) is the one-hot representation of the truth relation and \(\theta\) represents all parameters of the model.  
The attention model was originally applied to image recognition, mimicking the focus of the eye moving on different objects when the person viewed the image [42; 43; 44]. Similarly, when recognizing an image or a language, a neural network is focused on a part of the feature each time, and the recognition is more accurate. This motivates determining how to measure the importance of features. The most intuitive method is to use a weight. Therefore, the result of the attention model is to calculate the weight of each feature first and then apply the weight to features.  
**Word-level attention.** As shown in Figure 6, the output layer \(H\) can be represented as a matrix consisting of vectors \([h_{1},h_{2},\dots,h_{n}]\), where \(n\) is the sentence length. The representation \(S\) of the sentence is formed by a weighted sum of these output vectors \(h_{i}\):  
\[M =\tanh(H), \tag{4}\] \[\alpha =\text{softmax}\left(w^{T}M\right),\] \[S =H\alpha^{T},\]  
where \(H\in\mathbb{R}^{d^{m}\times n}A^{w}\) is the dimension of the word vector, and \(w\) is a trained parameter vector.  
**Sentence-level attention.** As shown in Figure 7, if we regard each sentence equally, the wrong labeling of sentences will introduce a massive amount of noise during training and testing. Therefore, sentence-level attention is important for relation extraction. The set vector \(X\) is computed as a weighted sum of these sentence vectors \(s_{i}\):  
\[e_{i} =x_{i}Ar, \tag{5}\] \[\alpha =\text{softmax}\left(e_{i}\right),\] \[X =\sum_{i}\alpha_{i}S_{i}.\]  
Figure 7: Relation classification.  
As shown in Figure 8, every line is a sentence (the annotations are in parentheses). Red denotes the sentence weight and blue denotes the word weight. We normalize the word weight by the sentence weight to make sure that only important words in important sentences are emphasized. Figure 8 shows that the model can select the words carrying strong sentiment like \"middle-size\", \"MPV\", \"same price\" and their corresponding sentences. Sentences containing many words like \"common\", \"sales\", \"from\" are disregarded. Note that the model can not only select words carrying strong sentiment; it can also deal with complex across-sentence context.