# Heterogeneous Graph Neural Network for Short Text Classification
## 3 Methodology  
### Construction of the Graph  
The method proposed in this paper constructs a heterogeneous graph around text, words, and POS tags to which words belong. The T-W graph between text and words is shown in Figure 1a, which includes text nodes and word nodes, and links between text nodes and word nodes. The edge is characterized by the term frequency-inverse relationship between words and text document frequency term frequency-inverse document frequency (TF-IDF) value, it is a statistical method to evaluate the importance of words to a text set or a piece of text in a corpus. The importance of a word increases proportionally to the number of times it occurs in the text, but at the same time decreases inversely to the frequency it occurs in the corpus. The TF-IDF value is calculated as follows:  
\[\text{TF}-\text{IDF}_{T-W}=\frac{n}{N}\times log\frac{D}{1+d} \tag{1}\]  
where \(n\) is the number of occurrences of the word in the text, \(N\) is the sum of the occurrences of all words in the text, \(D\) is the total number of texts in the corpus, and \(d\) is the number of documents that contain the word. The W-W graph between words and words is shown in Figure 1b, which contains text nodes and word nodes, and the text nodes are connected to the word nodes they contain. The feature of the edge is Pointwise Mutual Information (PMI) value. The PMI value can solve the misleading calculation of high-frequency words. If a word co-occurs with many words, its weight will be reduced. Conversely, if a word only co-occurs with individual words, its weight is increased. This will make the fixed collocations in the text more recognizable, and make common words less recognizable, reducing the impact on the text. The co-occurrence method of words is obtained by window sliding in the text, and the calculation method of PMI value is as follows:  
\[\text{PMI}(i,j) =log\frac{p(i,j)}{p(i)p(j)} \tag{2}\] \[\text{P}(\text{i,j}) =\frac{\#W(i,j)}{\#W}\] (3) \[\text{P}(\text{i}) =\frac{\#W(i)}{\#W} \tag{4}\]  
where #W(i,j) refers to the number of sliding windows that nodes \(i\) and \(j\) appear together, #W(i) refers to the number of sliding windows that node \(i\) appears, and #W is the total number of sliding windows. A positive PMI indicates that the words co-occur, that is, they will appear together; a negative PMI indicates that the words do not appear together. This paper only builds edges between word nodes with a positive PMI. The W-P diagram between words and POS tag is shown in Figure 2. Since a word plays different components in a sentence. For example, in the sentence \"can you open a can?\", the first can is a modal verb, and the second can are nouns, and they belong to different POS tags, so a word may have one or several POS tags. In our constructed W-P graph, there are text nodes, word nodes, and POS tag nodes. The text node is connected to the word node it contains, the feature of the edge is 1, the connection between the word node and the POS tag node is also formed, and the feature of the edge is also the IDF value between the word and the POS tag. The calculation method is as follows:  
\[\text{IDF}_{T-P}=log\frac{P}{1+p} \tag{5}\]  
where \(P\) is the total number of POS tags in the corpus and \(p\) is the number of POS tag for the word. The heterogeneous graph constructed according to words, texts and POS tags is completed. Different from the homogeneous graph, the heterogeneous graph contains more abundant information. Through the information dissemination of different types of edges, POS tag nodes can help the network filter more important word nodes, and finally summarize the information into text nodes. Enlarging the characteristics of the text itself, focusing on the text through TF-IDF, and reducing noise, such a composition can give full play to its advantages in the classification of pictures and short texts. Figure 1: Yellow nodes represent text nodes, white nodes represent word nodes, black lines represent edges between text and words, and green lines represent edges between words and words.