## 4 Experiments
### Comparison of Different LLMs  
We validate the effectiveness of our models by conducting experiments on the SEM16 dataset using three LLMs: Qwen-2.5, Llama-3.1, and GPT-3.5. These models are selected because they are comparable in terms of their capabilities and performance, representing a diverse range of LLMs. Table 4 and Figure 5 present a comparative analysis of stance detection performance across different LLM-generated graphs. The results highlight the significant influence of LLM-generated graph structures on the performance of our MVSD models.  
Table 4 shows that MVSD(GCN) consistently achieves the best performance across all LLM-generated graphs. Graphs generated by Qwen-2.5 (81.3%) and Llama-3.1 (80.5%) yield  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Model** & **Qwen-2.5** & **Llama-3.1** & **GPT-3.5** \\ \hline LLM-Direct & 70.3 & 75.1 & 54.5 \\ MVSD(LLM) & 78.4 & 78.9 & 75.0 \\ MVSD(GCN) & 81.3 & 80.5 & 75.4 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance comparison of different models using graphs generated by different LLMs on SEM16. The results are reported as Macro F1 scores (%).  
stronger results than those from GPT-3.5 (75.4%). MVSD(LLM) achieves 78.4% with Qwen-2.5 and 78.9% with Llama-3.1, although it slightly lags behind MVSD(GCN). The LLM-Direct approach, which omits graph structures, performs the worst, especially with GPT-3.5 (54.5%), highlighting the crucial role of graph-based features in stance detection. In summary, the LLMs can be ranked as Qwen-2.5 \(\approx\) Llama-3.1 \(>\) GPT-3.5 in terms of performance.  
Figure 5 visualizes the differences in node generation across LLMs, categorizing nodes into users, hashtags, and entities. Users and hashtags are extracted using regular expressions, ensuring the number of each is consistent across models. Qwen-2.5 and Llama-3.1 generate graphs with approximately 18% more entity nodes than GPT-3.5, resulting in denser graphs. This structural difference suggests that richer, more comprehensive graph representations improve stance detection performance.  
These findings emphasize the importance of choosing LLMs that generate structurally rich and contextually diverse graphs. Models like Qwen-2.5 and Llama-3.1, which generate more informative heterogeneous networks, significantly improve stance classification performance.  
To further illustrate the differences in entity extraction among various LLMs, we visualize the extracted entities using word clouds, as shown in Figure 6. This visualization provides an intuitive comparison of the models' extraction behaviors, revealing both differences and commonalities. Notably, while each LLM exhibits unique extraction patterns, certain key entities appear consistently across all models. For instance, the entity 'God' is extracted frequently by all three models, highlighting its significance in stance detection, particularly concerning the _Atheism_ label. Expressions of support for 'God' implicitly indicate opposition to _Atheism_, reinforcing the entity's relevance in stance classification. Similarly, entities such as 'feminist' are predominantly associated with the _Feminism_ label, reflecting their ideological importance. This figure visually represents a crucial aspect of how our HSN is constructed and further explains why incorporating HSN enhances performance. By capturing and structuring these key entities, HSN effectively models the underlying relationships between different stance categories, thereby improving the overall stance detection process.