# Social Media Opinion Analysis Model Based on Fusion of Text and Structural Features
## 4 Experimental Section
### Comparison of Experimental Results  
In addition to the aforementioned text vectorization models, several comparative models were added for ablation experiments in this article: the HGAT + ones method represents the use of an identity matrix instead of node feature vectors as input to the HGAT model in this article. We use this method to test the effect of pure network methods on the true opinion classification after ablation of the feature vectors of the text vectors. The GCN + BERT method represents the use of the BERT model to extract comment node text vector representations and updating comment node feature vectors through the GCN model. We use this method to test the effectiveness of the heterogeneity graph embedding model in HGAT for feature extraction of network structure. In this paper, we build on the above models by comparing several text classification methods, network structure-based methods, and two methods that combine text features with structural features. The same set of training and test samples was used during implementation so that the experimental results from different algorithms could be compared. Likewise, the training and test sets used for training with the text methods and updating the node feature vectors using the GCN and HGAT models were the same set of nodes. To balance the training and test samples, this paper divided the three comment types equally, and the split results are presented in Table 2. In this paper, the ratio of training set to test set is 90/10, we also did the experiment under 80/20 ratio, which has little effect on the result, here we choose the result under 90/10 ratio. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Data Set**} & \multicolumn{3}{c}{**Toutiao\#1**} & \multicolumn{3}{c}{**Toutiao\#2**} \\ \cline{2-7}  & **Positive** & **Neutral** & **Negative** & **Positive** & **Neutral** & **Negative** \\ \hline Training set & 904 & 899 & 899 & 1689 & 1689 & 1692 \\ Test set & 97 & 102 & 102 & 188 & 191 & 185 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The division result of datasets. Table 3 shows the experimental results based on the training and test sets above, where Acc. is the precision, Pre. represents the macro-precision, Rec. represents the macro-recall, and F1 represents the macro-F1. The experimental results show that:  
1. On both the Toutiao#1 and Toutiao#2 datasets, the text and structural feature combination method proposed in this paper performs better than the original text vectorisation algorithm, which states that the algorithm proposed in this paper, which optimizes and updates the text vectors based on the structural characteristics of the network, has some effect. 2. The HGAT + BERT model proposed in this paper outperforms all text vectorization methods as well as all structure-based algorithms on both datasets. 3. On the Toutiao#1 dataset, the HGAT proposed in this paper has a slightly weaker effect than the GCN, which could be because there are more neutral comments in the structure of the network. This paper's model uses graph embeddings of heterogeneous networks to update node vectors, as shown in Figures 4 and 5, but responses to neutral feedback are often still neutral feedback, making prediction of neutral feedback vectors slightly worse for HGAT than for GCN. In order to test whether the proposed HGAT model is universally applicable to a variety of text vectorization algorithms, in this paper, five text vectorization algorithms have been combined and compared with two graph embedding algorithms, GCN and HGAT, and the experimental results can be seen in Table 4. As can be seen in Table 4, the algorithm that updates the node based on the word embedding vectors Ernie, CPT, and GPT2 performs better than GCN, although it is weaker than GCN in terms of updating RoBERTa word embedding vectors but still has better performance than the original RoBERTa algorithm.