Rumor Detection on Social Media with Crowd Intelligence and ChatGPT-Assisted Networks
Authors: Chang Yang
Peng Zhang
Wenbo Qiao
Hui Gao
JiaMing Zhao
Venue: EMNLP 2023 Main (2023)
Keywords: Rumor Detection; Crowd Intelligence; Large Language Model; Heterogeneous Graph; Semantic Feature Learning

Abstract: In the era of widespread dissemination through social media, the task of rumor detection plays a pivotal role in establishing a trustworthy and reliable information environment. Nonetheless, existing research on rumor detection confronts several challenges: the limited expressive power of text encoding sequences, difficulties in domain knowledge coverage and effective information extraction with knowledge graph-based methods, and insufficient mining of semantic structural information. To address these issues, we propose a Crowd Intelligence and ChatGPT-Assisted Network(CICAN) for rumor classification. Specifically, we present a crowd intelligence-based semantic feature learning module to capture textual content's sequential and hierarchical features. Then, we design a knowledge-based semantic structural mining module that leverages ChatGPT for knowledge enhancement. Finally, we construct an entity-sentence heterogeneous graph and design Entity-Aware Heterogeneous Attention to effectively integrate diverse structural information meta-paths. Experimental results demonstrate that CICAN achieves performance improvement in rumor detection tasks, validating the effectiveness and rationality of using large language models as auxiliary tools.

Rumor Detection on Social Media with Crowd Intelligence and ChatGPT-Assisted Networks
Chang Yang', Peng Zhang?; Wenbo Qiao², Hui Gao?, Jiaming Zhao College of Information Science and Engineering, Shenyang University of Technology, China College of Intelligence and Computing, Tianjin University, Tiajjin, China School of New Media and Communication, Tianjin University, Tiajjin, China yangchang@smail. sut edu. cn
{pzhang, , qiaobb, hui _ga j.e.eeuuuuuuuunnn edu.cn zn_ zn_zjm@ com com
Abstract
et al., 2021; Vosoughi et al., 2018). In response to this problem, rumor detection methods based on crowd intelligence have attracted much atten- tion. Crowd intelligence means social media users spread and interpret information by sharing per- sonal opinions positions, questioning, and provid- ing evidence (Phan et al., 2023). We realize that rumors the result of the interaction between cial media and swarm intelligence, a phenomenon of information dissemination among groups. That is, the characteristies of swarm intelligence are the fundamental reason why rumor detection becomes more complicated. From the perspective of crowd intelligence, we redefine rumors based on existing viewpoints (Phan et al., 2023; Zubiaga et al., 2018; Doan et al., 2011): the Rumor is an influential but unverified statement widely disseminated on social networks and other channels through people's col- lective participation and information interaction. Social media users usually share their views on the veracity of rumors through collective, subjective understanding (Tolmic et al., 2018; Ma et al., 2018), 8), which may help reveal the truth behind the rumors. Therefore, the crowd's cognitive mechanism for ru- mors has important guiding significance for rumor detection, especially the detection method based crowd intelligence. However, there are still some problems in effectively utilizing crowd intelligence in rumor detection:
In the era of widespread dissemination through social media, the task of rumor detection plays a pivotal role in establishing trustworthy and reliable information environment. Nonethe- less, existing research on rumor detection con- fronts several challenges: the limited expres- sive power of text encoding sequences, diffi- culties in domain knowledge coverage and ef- fective information extraction with knowledge graph-basee methods, and insufficient mining of semantic structural information. To address these issues, we propose Crowd Intelligence and ChatGPT-Assisted NeTWO))((ICAAN) rumor classification. Specifically, present a crowd intelligence-hasel semantic feature learning module to capture textual content's sequential and hierarchical features. Then, we design a knowledge- based semantic structural mining module that leverages ChatGPT for knowledge enhancement. Finally, we construct an entity-sentence heterogeneous graph and de- sign Entity-Aware Heterogeneous Attention to integrate diverse structural information meta- paths effectively. Experimental results demon- strate that CICAN achieves performance im- provement in rumor detection tasks, validating the effectiveness and rationality of using large language models as auxiliary tools.
1 Introduction
In recent years, the number of rumors posted on the Internet has increased significantly, and the harm of rumors is that they may cause panic and unnec- essary actions, undermine social stability and trust, and negatively affect individuals, organizations, and society (Lin et al., 2021). Unlike traditional media channels, information on social networks has the characteristics of fast dissemination and wide coverage, leading to a large amount of infor- mation that cannot be verified in real-time (Khan
Due to the complexity and irregularity of information on social networks, current research methods have certain limitations in extracting deep semantic features. Convolutional Neural Networks (CNNs) are commonly used for this purpose. Still, their max-pooling operations can lead to loss of detail and positional information, limiting the ex- pressive power of text-encoded sequences. There- fore. improving the model's ability detect deep semantic features while preserving the relationship between low and high-level features is crucial for optimal performance.
Corresponding author
This work is done when Chang Yang visited Tianjin University for exchange.

---

2) The method (Wei al., 2019; Tchakounté al., 2020) based on Crowd Intelligence has been proven effective in identifying information on so- cial networks. Since information on social net- works is noisy, it crucial to introduce additional knowledge to supplement semantic information. However, current primary supplement semantic information methods, such as knowledge graphs, face challenges in extracting information effec- tively. Therefore, is essential introduce ex- ternal knowledge effectively.
telligence and adequate external knowledge. Furthermore, we redefine rumors based on crowd intelligence and suggest meaningful fu- ture research directions in the field rumor detection.
We apply large language models(LLM)) ru- mor detection for the first time, using Chat- GPT as an auxiliary means for text enhance- ment, entity linking, and text conceptualiza- tion, and demonstrate the effectiveness and accuracy of the approach.
3) Fully mining the semantic structure infor- mation in the text is crucial for rumor detection, and there are currently two main problems to be solved. First, the study (Zhu et al., 2022) found that in- sufficient learning of entity information will cause entity bias and affect the model's generalization to future data. Second, important information is usu- ally distributed in various text positions rather than concentrated in specific position, which increases the difficulty of extracting complex long-distance semantically dependent features. Efficiently con- structing heterogeneous graphs and selecting ap- propriate meta paths to integrate semantic structure information are the keys to addressing these issues.
Our experiments on two real-world datasets show that our method is highly effective and outperforms the baseline approach. Our unique design has substantially enhanced the lask of rumor detection.
Related Work
2.1 Semantic Content-based Method
At present, the entry point of rumor detection work mainly focuses on the content characteristics of cial posts (Phan et al., 2023) because social posts carry crowd intelligence, which can be gathered in their creation, interaction, and dissemination. Traditional methods are mainly based on feature engineering (Castillo et al., 2011; Lim et al., 2017; Rayana and Akoglu, 2015; Al-Ghadir et al., 2021), using user opinions social media platforms and feature information of social networks extract features and use classifiers to detect rumors manu- ally.
Based on the above challenges and short- comings, this paper proposes Crowd Intelligence and CharGPT-Assisted NCCCO))(CICAN) that uses crowd intelligence and introduces a large language model as an auxiliary tool, combining semantic content and structure information for rumor detec- tion. The model includes the Crowd Intelligence- based Semantic Feature Learning Module (CIS) and the Knowledge-
Semantic Structure Min- ing Module (KSS). First, we propose a context- aware semantic encoder and a multi-granularity feature extractor for extracting semantic content features, leveraging TweetBERT to enhance the model's ability to handle semantic and contextual relevance, and combining BiLSTM and capsule networks for sequential and hierarchical feature learning. Then, by leveraging ChatGPT extract refined and structured knowledge from the raw data, we construct the Eintty-Senttnnd heterogeneous graph containing two types of nodes and design a new hierarchical attention mechanism to enhance the model's ability to capture complex relation- ships. Our main contributions are as follows:
However, the complexity of social media data is mainly reflected in the diversity of content, top- ies, and styles. In contrast, deep learning-base based methods can automatically learn representations of features. Existing studies (Ma et al., 2016; As- ghar et al., 2021; Liu and Wu, 2018; Yu al., 2017) mostly use CNN extract key features from the text content of related posts. However, the max pooling operation CNN comes at the cost of losing the precise encoding of object locations (Mazzia et al., 2021). )). To address this issue, Sabour et al. (2017) introduced capsule networks, which use vector outputs and the "routing trouiing-y--ggree agreemeemm algorithm to retain position information and mine features from various perspectives. Capsule net- works have proven effective in text classification tasks (Zhao et al., 2018: Yang et al., 2019). By incorporating capsule networks into NLP, we can capture different types of semantic information,
Our proposed rumor detection model (CI- CAN) extracts semantic content and semantic structure features by introducing crowd in-

---

In this paper, we formulate the rumor detection problem as a classification task. aim to learn a function p(Y|V, 2; where represents the semantic content information, and Z represents the semantic structure information. The imput consists of text information containing source tweets and comments, as well the entity-sentence hetero- geneous graph structure information constructed from external knowledge. All the learnable param- eters of the model are denoted as B. The class label Y {T, F, N, U} represents the four categories: representing word relationships, syntactic struc- tures, and similar word characteristics.
effectively integrate semantic structure information
and improve the ability of rumor detection.
2.2 External Knowledge-based Method
Problem Statement
Text on social networking platforms is usually lim- ited by the number of characters, which prevents an accurate understanding of content information based on sufficient context and details. Existing re- search in rumor detection has primarily focused on incorporating external knowledge, such as knowl- edge graphs. CompareNet (Hu et al., 2021) cap- tures content-knowledge consistency by comparing contextual entity representations with knowledge- based ones. The multimodal knowledge-aware event memory network (MKEMN) (Zhang al., 2019) integrates knowledge graphs and memory networks, considering both text and multimedia data to detect rumors. DDGCN (Sun et al., 2022) dynamically models message spreading and back- ground knowledge from knowledge graphs in uni- fied framework. However, the complexity and in- formality of social network information pose chal- lenges for knowledge graph-based methods ex- tract knowledge information in specific research fields effectively.
T (true rumor), F (false rumor), N (non-rumor), and U (unverified rumor).
The semantic content information V com- prises collection of event sentences, including source tweets and comments. For each event X {x, C1, C2, ...: .p) where x represents the source tweet, Gj represents the comment corre- sponding to the source tweet, and is the max- imum number of comments. To capture the se- mantic structure information, we construct the entity-sentence heterogeneous graph denoted E) where V.S.E and E represent the nodes and edges in the graph, respec- tively. V composed of series of sentences from the event (ssouree tweet, comments) along with reinforced entity nodes EY The graph inte- grates the entity and the event sentences, enabling the modeling of relationships between entities and sentences.
Recently, large-scale language models, such ChatGPT (OpenAI, 2022), have shown remark- able performance in natural language processing tasks, capable of extracting accurate and rich fea- tures beyond traditional methods (Feng et al., 2023). Therefore, utilizing ChatGPT as an auxiliary tool to improve the model's performance has become a promising trend.
C.S
2.3 Semantic Structure-based Method
Methodology
Graph Neural Networks (GNNs) have gained pop- ularity for capturing contextual relations, syntactic structures, and semantic dependencies in text. Yao al. (2019) constructed single text graph based on word co-occurrences and document-word re- lationships. Liu et al. (2018) proposed Concept Interaction Graph for text matching, employing Siamese Encoded Graph Convolutional Network to match results. Huang et al. (2020) proposed a meta- path-based heterogeneous graph attention network to fuse information from source tweets in rumor de- tection. Yuan al. (2019) employed global-local attention network capture local semantic rela- tions and global structural information propagated by source tweets. Therefore, the attention mecha- nism designed based on heterogeneous graphs can
We detail the overall framework of the proposed Crowd Intelligence and CharGPT-Assisted Net- (ork(CICAN), shown in Figure 1. We elaborate on the main constituent modules of CICAN and its rumor detection process.
4.1 Crowd Intelligence- Semantic Feature Learning Module
The diverse nature and the challenge of discerning truth from falschood in the discourse social net- works make it difficult to extract complex semantic content features for rumor detection from the mas- sive amount of information. To address this issue, we design a contextual semantic encoder and multi- granularity feature extractor, Recent experiment

---

Crowd Intelligeace-boseed Semantic Feature Learning Module
Ereading
Adl
Adl
Fornel
ingut
1STM
LSYR
INPISTR
Knowledge based Semantic Structure Mining Module
Early Avare Alteniie
Eallyy Scutenec Aeterogeneets sriph
Kohancedl Sentence
Socurn Tutterr
Edercod
Estty node
Sendence
Figure I: The architecture of Crowd Intelligence and ChatGPT-Assisted NeTWOr)(CICAA)
(Farinneya et al. 2021) shows that TweetBERT performs well in identifying rumors social me- dia. TweetBERT (Qudar and Mago, 2020) pre-trained language model explicitly tailored for tweets, which is trained on a large corpus of tweet data, enabling it to capture nuances and essential features in tweets effectively. Inspired by this, we employ TweetBERT as the contextual semantic en- coder to obtain the word embedding of each text for enhancing the model's ss ability handle semantic and contextual relevance.
"squashing function:
U
= Squash(us)
(2)
|u:l
Then, the prediction vector of the primary capsule ty is obtained by projection, dz W the i, E [e-xx(--- where W. is the transforma- tion matrix, the number of higher-level capsules is Cins and the dimension of the capsules Caim- Next, we use the dynamic routing mechanism generate prediction vectors for higher-level cap- sules based on the primary capsules. The higher- level capsule UR is the sum of prediction vector in primary capsule ūRحe and coupling coefficient Ckis.
sequence as HB [ .. ,H
HHTB)
In addition, we also combine BiLSTM and capsule network as the multi-granularity feature ex- tractor for text sequential and hierarchical feature learning. Specifically, BiLSTM is recurrent neu- ral network structure that can efficiently process sequential information. We obtain the hidden state representation H Bi..TM E m2*22h by concatenat- ing the output of BiLSTM,
Vk Σ Chie uNIs
(3)
Ckji sof coftmar(bHii)
(4)
where by initialized to and updates in each iteration bAis = =b2js + Uيي * 2%k Cjjz is determined by the iterative dynamic routing process. Finally, through rounds of dynamic routing iterations, we get the final output of the capsule network V [Ug vi, Up] Vk R X which represents the semantic content information of the event.
BILSTM (ASTM(H),,TTMEE
(1) where 2h is the hidden state dimension of BiLSTM, and is the sequence length. For hierarchical fea- ture learning, we use convolutional layer with convolution kernel size of k and the number of convolution kernels of k_num extract n-gram features different positions in the sentence, u; Conv( H. BiLSTM) E ( 1) which can be regarded as the primary capsules with a quantity (s k 1) and dimension of num, and use
4.2 Knowledge-basee Semantic Structure Mining Module
We design the Knowledge- Semantic Struc- ture Mining Module to capture mingddittnce struc- ture information. As shown in Figure 2.

---

CluAeit Stved Eapy Inhooeeed
better capture the long-distance semantic depen- dencies and complex structural information of sentences, we construct the entity-sentence het- erogeneous graph with enhanced sentence nodes = 16 C2 .2....,,4 and entity nodes E = E4...., m. The graph contains two meta- (ESE) considers the
paths: Serr--Seteeneeennn relationship between different entities in sentence, and Sentence- Sanriceee-nttt--SSSe SantitySSettence (SES) reflects the lationship between other sentences containing the same entity. Inspired by Heterogeneous Graph At- tention Network (Wang et al., 2019), the node-level attention mechanism is introduced to capture the information interaction between nodes and obtain richer feature representations.
Figure 2: Knowledge-based Semantic Structure Mining
Module
ChatGPT-based Knowledge Extraction: To deal with informal texts social networks, includ- ing abbreviations, misspellings, and spoken lan- guage, etc., we use ChatGPT to introduce external knowledge effectively.
To align the feature spaces of sentence nodes d/ and entity nodes e/, transformation matrices We and We are applied to project the features of these distinct node types into unified feature space. c'' = W. / oi' e,' = W. attention and entity attention are respectively calcu- lated for entity nodes with ESE meta-path structure and sentence nodes with SES meta-path structure, which can be expressed as:
First, we process the raw texts using ChatGPT to build a corrected set of texts = 41 22.-..,))) (p)- We design task-specific prompts that aim to guide the understanding pro- cess of ChatGPT better. As shown in Prompt I of Appendix C. Then, utilize ChatGPT to extract entities and concept information from the enhanced text via the "isA" relation, and the prompt used is shown in Prompt 2 of Appendix C. For a source tweet Co E Radd of length n, entities can be rep- resented as Eg E Rd concepts can be repre- sented as C, E Rmmkxxd m represents the maxi- mum number of entities, and k represents the max- imum number of concepts corresponding to each design the Context-based En-
/T ees'. Then, the sentence
expla(a% (e);)(]) B/B EKEN, expla(a" (e(;e
(6)
expla<a [c c))
d/ EREN. exp(a(a? (cj41)
(7)
entity. Finally, tity Enhancement method. Current research work (Zhang et al., 2019) mainly focuses on modeling the attention mechanism between concepts and enti- ties but does not fully consider the interrelationship between concepts and sentences. In fact, each con- cept can be regarded as semantic interpretation of the entity, which may have different importance in different contexts. Therefore, calculate the simi- larity between the concept C. corresponding the entity and the sentence representation co to deter- mine the importance of the concept for the context. By adding the calculation results to the original entity representation E,, we obtain the enhanced entity representation.
where o; and Cj represent node with SES struc- ture, e; and ej represents node with ESE struc- ture. ae, 0g are learned parameters, a is the non- linear activation function LeakyReLU (Xu et al., 2015). To capture multiple representations from different relations, a multi-head paradigm similar to heati-head attention is adopted, and the follow- ing output feature representation is obtained:
K 6 = k=
Σ iEN(c)
afieh
(8)
K - k=
E"
"
Σ (EN(c)
siel
(9)
E = E, + Σ softmax(co C.)- of (5)
where df represents the feature representation that df learned from the SES meta-path eii represents the feature representation that e' learned from the
Entity-Aware Hierarchical Attention: To

---

To maintain consistency, we set the batch size to 16 and the number of epochs to 30. The initial learning rate set to 0.001, gradually decreasing during training, and the weight decay coefficient is 0.0001. We employ L2 regularization with weight of 0.001 to control the model's complexity. Fur- (10) thermore, the dataset is divided into training set (70%), validation set (20%), and a test set (10%). To ensure the robustness of the results, we perform Table 1: Rumor detection results on two datascts.
Module
Tweet15
Tweet16
Method
Content
Struction
Knowledge
FT
F1
ACC
based
based
-based
F
N
F
T
U
PPC
0,668
0.579
0.604
0.591
0.610
0.626
0.630
0.617
0,636
0.628
dEPEND
0.731
0.631
0.646
0.617
0.668
0.721
0.649
0.603
0.611
0.637
TD-RvNN
0,713
0,702
0.691
0.746
0.654
0.737
0.662
0.743
0.801
0.768
BIGCN
5
0.798
0.716
0,758
0.843
0.876
0.803
0.792
0.788
0.796
0.814
ClaHi-GAT
0.823
0.817
0.802
0.831
0.813
0.802
0.817
0.833
0,808
0.824
AMGCN-NoVisnn
0.766
0.701
0.724
0.799
0.717
0.739
0.727
0.764
0,811
0.790
DDGCN
0,828
0.795
0.818
0.792
0.780
0.812
0.723
0.712
0.779
0.799
CICA-Net
0,855
0,816
0.811
0.870
0.901
0.840
0.838
0.826
0,802
0.862
ESE meta-path,,& k represents the number of atten- tion heads.
5
Experiments
5.1 Datasets and Experimental Settings
Different from the Heterogeneous Graph At-
We evaluate the proposed model using two publicly datasets recognized in the field of rumor detection, including Twitter15 (Ma et al., 2017) and Twitter1 6 (Ma al., 2017). Detailed dataset statistics are listed in Appendix A.
tention Network, We use co-attention (Lu et al.. 2016) to automatically learn the independence of meta-paths and the synergy between different meta- paths, enabling more accurate heterogeneous fu- sion. First calculate the affinity matrix of e'j and cy: Ce. tanh(e e Wese"). where Wes is a learnable parameter. Then, Ces is sent to the network as feature, and the attention maps of entity features and sentence features are learned as follows:
H. tanh( (Wee (
a, = soft ma H.)
(11) 5-fold validaaiio obtain reliable experi- mental outcomes. The detailed configuration of our model parameters is provided in Appendix B.
H. tanh WW. (W. (Wac))Cos)
(12)
5.2
Baselines
We have conducted a comparative analysis of state- of-the-art (SOTA) models in the field of rumor de- tection, including models based on semantic con- tent information, semantic structural information, and prior knowledge. Specifically, the following models were selected: 1) PPC (Liu and Wu, 2018): This approach combines Convolutional Neural Net- works and Recurrent Neural Networks to construct a time series classifier for rumor detection. 2) dE- FEND (Shu et al., 2019): This model explores the correlation between the source tweet and its corre- sponding comments using an attention mechanism to investigate semantic relationships. 3) RyNN (Ma et al., 2018): This model captures the traver- sal direction of the information propagation tree obtain clues about the spread and the semantic con- tent. 4) BiGCN (Bian et al., 2020): This model a rumor detection approach based on Graph Convo- lutional Networks to capture the structural features
ag soft mer(fax
(13)
where W. We, Ws WE represent the learnable parameters, ass and ae represent the attention weights of sentence features cj and entity features cy, as the importance of each meta-path. With the learned weights coefficients, we fuse meta-paths zj = azdy + aae%.. Finally, we use cascading to integrate semantic content features V = (vo, ......, {p} and semantic structure fea- tures = {20, 21, 21,..., 2p} into a unified represen- tation, and output accuracy prediction through a multilayer perceptron:
i softmax W [VV
(14)
where W is a Iearnalle parameter and represents a bias.

---

Table 2: Ablation experiment results our proposed CICAN model. (CIS represents Crowd Intelligence- Semantic
Feature Leariing Module, KSS represents Knowledge-base baade Semantic Structure Mining Module.)
CIS
KSS
Dataset
MODEL
TweetBert
Cap
BiLSTM
EE
ES graph
Twitter15 ACC
Twitter16 ACC
w/o
0.817
0.813
w/o CIS TweetBERT
0.828
0.821
w/o CIS-cup
/
0.84
0.829
w/o CIS-BiLSTM
0.843
0.832
w/o KSS
0.823
0.817
w/o KSS-ES gruph
0.836
0.829
w/o KSS-EE
0.827
0.82
CICAN
0.855
0.84
of rumor propagation. 5) ClaHi-GAT (Lin et al.. 2021): This model builds an undirected interaction graph and uses a hierarchical graph attention net- work to capture the semantic features of posts for (Wang al., 2020): KMGCN models semantic represen- tations by combining text, knowledge, and visual information. We use the KMGCN-No Visual model (which excludes visual information) for fair com- parison. 7) DDGCN (Sun et al., 2022): This model uses Graph Convolutional Networks to model two types of structural information, including the con- tent information in the propagation and the concep- tual information of the knowledge graph.
performs knowledge-based models, KMGCN and DDGCN, especially in dealing with complex se- mantic structures. The results confirm that CICAN can extract complex content features and capture semantic structure information, effectively enhanc- ing text understanding and key feature extraction capabilities.
rumor detection.
6) KMGCN-No
5.4 Ablation Study
We conduct series of ablation studies to evaluate the effectiveness of the individual modules of the CICAN model. The specific ablation experiments include the following aspects:
w/o CIS: Remove the crowd intelligence- based semantic feature learning module. w/o CIS -TweetBERT: Replace TweetBERT in CIS for BERT. w/o CIS-cap: Only remove the capsule net- work in CIS. w/o CIS- IIS-B:LTM: Only remove the BiLSTM in CIS. w/o KSS: Remove the knowledge- based semantic structure mining module. w/o KSS- ES graph: Remove the entity-sentence heteroge- neous graph KSS. w/o KSS-EE: Remove the Context-Bei Based Entity Enhancement in KSS.
5.3 Performance Comparison
As shown in Table 1, CICAN outperforms other methods with accuracy rates of 85.5% and 84.0% on Twitter15 and Twitter16 datasets, respectively. The semantic content-based models PPC and dE- FEND neglect deep semantic features and struc- tural information. At the same time, we propose a method combining BiLSTM and capsule network, which can extract higher-level semantic informa- tion and capture part-whole relationships. Regard- ing models utilizing semantic structure information, the tree structure model RvNN exhibits slightly inferior performance due to its limited ability to capture long- long-distance dependencies in sequences. Conversely, graph-based models demonstrate better capability in capturing semantic structure features. Distinguished from BiGCN, our method constructs entity sentene heterogeneous graphs, facilitating enhanced capture of structure features. Compared ClaHi-GAT, we introduce Entity-Aware Het- erogeneous Attention to discern the significance of different meta-paths automatically. Moreover, by leveraging chatGPT to extract knowledge and augment entities based on context, CICAN out-
The experimental results are summarized in Table 2. w/o CIS has significant decrease in the accuracy of the Twitter and Twitter16 16 datasets (decreased by 3.8% and 2.7%). which shows that this module plays vital role in enhancing seman- tic representation and mining semantic informa- tion. Similarly, w/o CIS-TweetBERT also signifi- cantly impacts model performance, indicating that TweetBERT has superior comtext-dependent pro- cessing capabilities when processing text on social networks compared to BERT. Compared with w/o BiLSTM, the accuracy rate of w/o cap is lower because the capsule network can aggregate inputs from multiple directions when processing shorter texts, which is more robust. This result further vali- dates the effectiveness the method in this paper.

---

w/o KSS leads 3.2% and 2.3% decrease in the accuracy of 1witter15 and Twitter16 datasets, re- spectively, emphasizing the significance of extract- ing structured knowledge and establishing global semantic connections to improve the model's un- derstanding and judgment of text. The performance drops significantly when directly replacing entity- sentence heterogencous graphs with directly con- catenated entity results (w/o KSS-ES graph) be- cause heterogeneous graphs can establish global semantic connections and contextual associations.
Twiiteer
DPC
0.3
CHID-GA GAT
BRGEN
062
counts
(a) Twitter15
Twimtrrl
The comparison between w/o CIS and w/o KSS shows that w/o CIS significantly impacts the model's performance, indicating the effectiveness of the pre-trained model based on social network and multi-granularity feature extraction in fully mining features. However, introducing external knowledge (mainly involving modeling entity rein- forcement and semantic structure) provides a com- plementary effect on the model's performance and enhances the model's generalization ability to un- seen data.
0.7
0.4
0.3
BAGEN
CIEAN
conminre courss
(b) Twitter16
Figure 3: Early Rumor Detection
5.5 Early Rumor Detection
One of the critical objectives in rumor detection is identify and intervene in rumors early as possible. In our experiments, we compare the per- formance different methods by incrementally increasing the test data. As shown in Figure 3, even in the initial stage with limited comment quanti- ties, our proposed CICAN model exhibited high accuracy. For instance, with only 30 comments, the CICAN model achieved accuracies of 74.3% and 71.8% 1witter15 and Twitter16, respectively, demonstrating its ability capture critical seman- tic features. It can be seen from the figure that the performance of all models fluctuates during train- ing due to the increase of semantic information and noise in social networks. However, the CICAN model comprehensively considers semantic con- tent and semantic structure information, so it has generalization and robustness and can consistently perform well. The CICAN model outperformed other models at each stage, confirming the effec- tiveness of early rumor detection and significantly improving detection performance.
mantic content and structure information for rumor detection. In the crowd intelligence- seman- tic feature learning module, we employ contextual semantic encoders and multi- multi-granularity feature extractors to capture the semantic content informa- tion. In the knowledge-based semantic structure mining module, to capture complex semantic struc- tural information, we extract refined knowledge utilizing ChatGPT to construct entity entity-sentence heterogeneous graph and design entity-aware het- erogeneous attention integrate representations from different meta-paths effectively. Experimen- tal results validate the effectiveness of the CICAN model in leveraging crowd intelligence and exter- nal knowledge, resulting in significant performance improvements in rumor detection tasks.
Future research directions can further expand the field of rumor detection in several ways. First, take inspiration from truth-fine truth-finding systems that leverage human intelligence to extract reliable in- formation from multiple sources. Second, to plore quantum cognition, using quantum theory to explain human behavior related to rumor detec- tion. Finally, the application of large models in rumor detection can be further mined to improve the performance and interpretability of the models.
6 Conclusion and Future Work
This study introduces a novel rumor detection model, the Crowd Intelligence and CharGPT- Assisted Network (CICAN), which combines se-

---

Limitations
Tian Bian, Xi Xiao, Tingyang Xu, Peilin Zhao. Wenbing Huang, Yu Rong, and Junzhou Huang. 2020. Ru- mor detection on social media with bi-directional graph convolutional networks. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 549-556.
In this section, we discuss the limitations of our work and propose corresponding solutions. First, we exploit the semantic information of crowd in- telligence for rumor detection. However, the user's profile, historical activity, or social network struc- ture should also be considered. Future research will focus on capturing user features related to rumor spreading to improve detection accuracy. Second, ChatGPT shows promising results in data augmen- tation and concept extraction in this study. How- ever, its effectiveness largely depends on the quality and design of the prompts used. Therefore, further research to explore more effective prompt engineer- ing techniques, especially for customizing rumor detection tasks, is expected to improve model per- formance.
Carlos Castillo, Marcelo Mendoza, and Barhara Poblete. 2011. Information credibility on twitter. In Pro- ceedings of the 20th international conference on World wide web, pages 675 684.
Anhai Doan, Raghu Ramakrishnan, and Alon Y Halevy. 2011. Crowdsourcing systems the world-wide web. Communications of the ACM, 54(4):86-9.
Parsa Farinneya, Mohammad Mahdi Abdollah Pour, Sardar Hamidian, and Mona Diab. 2021. Active learning for rumor identification on social media. Findings of the Association for Computational Linguistics: EMNLP 2021. pages 4556-4565.
Ethies Statement
Yutao Feng, Jipeng Qiang, Yun Li, Yunhao Yuan, and Yi Zhu. 2023. Sentence simplification via large language models. arXiv preprint arXiv:2302. 1
This article deals with social media data, specif- ically social media data on Twitter. The bench- mark datasets we experiment with for classification are publicly available in previous work, follow all Twitter regulations, and extract data through the official API. To mitigate the potential impact of us- ing such data, all tweets related to the dataset were anonymized, and URLs were removed to ensure individual users' privacy and data security. This re- search aims to analyze and explore the methods and techniques of rumor detection to improve the credi- bility and reliability of information on social media. We will report the experimental results objectively, transparently, and comprehensively following aca- demic norms and ethical standards.
Linmei Hu, Tianchi Yang, Luhao Zhang, Wanjun Zhong, Duyu Tang, Chuan Shi. Nan Duan, and Ming Zhou. 2021. Compare to the knowledge: Graph neural fake news detection with external knowledge. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 754-763.
Qi Huang, Junshuai Yu, Jía Wu, and Bin Wang. 2020. Heterogeneous graph attention networks for early detection of rumors on twitter. In 2020 Inter- national Joint Conference on Neural Networks (WCNN), pages 8. IEEE
Acknowledgements
Tanveer Khan, Antonis Michalas, and Adnan Akhun zada. 2021. Fake news outbreak 2021: Can we stop the viral spread? Journal of Network and Computer Applications, 190:103112.
This work is supported in part by the Natural Sci- ence Foundation of China (grant No. (2276188), Tianjin Research Innovation Project for Postgradu- ate Students (grant No. 2021YJSB167).
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980.
References
Wee Yong Lim, Mong Li Lee, and Wynne Hsu. 2017. ifact: an interactive framework to assess claims from tweets. In Proceedings of the 2017 ACM on Conference Information and Knowledge Man- agement, pages 787-796.
Abdulrahman A1-Ghadir, Aqil M Azmi, and Amir Hus- sain. 2021. A novel approach to stance detection in social media tweets by fusing ranked lists and sentiments. Information Fusion, 67:29-40.
Muhammad Zubair Asghar, Ammara Habib, Anam Habih, Adil Khan, Rehman Ali, and Asad Khattak. 2021. Exploring deep neural networks for rumor detection. Journal of Ambient Intelligence and Humanized Computing, 12:4315-4333.
Hongzhan Lin, Jing Ma, Mingfei Cheng, Zhiwei Yang. Liangliang Chen, and Guang Chen. 2021. Ru- mor detection twitter with claim-guided hier- archical graph attention networks. arXiv preprint arXiv:2110.04522

---

Bang Liu, Di Niu, Haojie Wei, Jinghong Lin, Yancheng He, Kunfeng Lai, and Yu Xu. 2018. Matching ar- ticle pairs with graphical decomposition and con- volutions, arXiv preprint arXiv: /rXivv/802.000 802.07459.
Kai Shu, Limeng Cui, Suhang Wang, Dongwon Lee, and Huan Liu. 2019. defend: Explainable fake news detection. In Proceedings of the 25th ACM SIGKDD international conference knowledge discovery & data mining, pages 395-405.
Yang Liu and Yi- Yi-Fang Wu. 2018. Early detection of fake
news on social media through propagation path classification with recurrent and convolutional net- works. In Proceedings of the AAAI conference on artificial intelligence, volume 32.
Mengzhu Sun, Xi Zhang, Jiaqi Zheng. and Guixiang Ma.
2022. Ddgen: Dual dynamic graph convolutional networks for rumor detection on social media. Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 461 1-469.
Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2016. Hierarchical question-image co-altention for visual question answering. Advances in neural information processing systems, 29.
Franklin Tchakounté, Ahmadou Faissal, Marcellin Atemkeng, and Achille Ntyam. 2020, reliable weighting scheme for the aggregation of crowd intelligence to detect fake news. Information. 11(6):319.
Jing Ma, Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard Jansen, Kam-Fai Wong, and Meeyoung Cha. 2016. Detecting rumors from microblogs with recurrent neural networks.
Peter Tolmie, Rob Procter, Mark Rouncefield, Maria Liakata, and Arkaitz Zubiaga. 2018. Microblog analysis pregram of work. ACM Transactions on Social Computing, 1(1):1-40.
Jing Ma, Wei Gao, and Kam-Fai Wong. 2017. Detect rumors in microblog posts using propagation struc- ture via kerne learning. Association for Computa- tional Linguistics.
Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. science, 359(6380):1 115-1
Jing Ma, Wei Gao, and Kam-Fai Wong. 2018. Rumor detection on twitter with tree-structured recursive neural networks. Association for Computational Linguistics.
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. 2019. Heteroge- neous graph attention network. In The world wide web conference, pages 2022-2032.
Vittorio Mazzia, Francesco Salvetti, and Marcello Chi- aberge. 2021. Efficient-capsnet: Capsule net-
Youze Wang, Shengsheng Qian. Jun Hu, Quan Fang. and Changsheng Xu. 2020. Fake news detection via knowledge-driven multimodal graph convo- lutional networks. Proceedings of the 2020 international conference on multimedia retrieval, pages 540-547.
work with self-attention routing. Scientific reports, 11(1):14634.
TB OpenAl. 2022. Chatgpt: Optimizing language mod- els for dialogue. OpenAl.
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word
Xuan Wei, Zhu Zhang, Mingyue Zhang, Weiyun Chen, and Daniel Dajun Zeng. 2019. Combining crowd and machine intelligence to detect false news on social media. Mis Quarterly.
representation. In Proceedings of the 2014 con- ference on empirical methody in natural language processing (EMNLP), pages 1532-1543.
Huyen Trang Phan, Ngoe Thanh Nguyen, and Dosam Hwang. 2023. Fake news detection: A survey of graph neural network methods. Applied Soft Computing, page 110235.
Bing Xu, Naiyan Wang, Tianqi Chen, and Mu Li. 2015. Empirical evaluation rectified activa- tions in convolutional network. arXiv preprint
Mohiuddin Md Abdul Qudar and Vijay Mago. 2020. Tweethert: a pretrained language representation model for twitter text analysis. arXiv preprint arXiv:2010.11091.
Min Yang, Wei Zhao, Lei Chen, Qiang Qu, Zhou Zhao, and Ying Shen. 2019. Investigating the transfer- ring capability of capsule networks for text classi- fication. Neural Networks, 118:247-261.
Shebuti Rayana and Leman Akoglu. 2015. Collective opinion spam detection: Bridging review networks and metadataa novel approach to stance detection in social media tweets by fusing ranked lists and sentiments. In Proceedings of the 2/th acm sigkdd international conference on knowledge discovery and data mining, pages 985-994.
Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. Graph convolutional networks for text classifica- tion. In Proceedings of the AAAI conference ar- tificial intelligence, volume 33, pages 7370-7377.
Yuan Yao, Lorenzo Rosasco, and Andrea Caponnetto. 2007. On early stopping in gradient descent learn- ing. Constructive Approximation, 26(2):289-315.
Sara Sabour, Nicholas Frosst, and Geoffrey E. Hinton. 2017. Dynamic routing between capsules. Ad- vances in nearal information processing systems, 30.
Feng Yu, Qiang Liu, Shu Wu, Liang Wang, Tieniu Tan, et al. 2017. convolutional approach for misinfor- mation identification. In IJCAI, pages -901-

---

Chunyuan Yuan, Qianwen Ma, Wei Zhou, Jizhong Han, and Songlin Hu. 2019. Jointly embedding the local and global relations of heterogeneous graph for rumor detection. In 2019 IEEE international conference on data mining (ICDM), pages 796- 805. IEEE.
B Implementation Details
We provide further details regarding the implemen- tation of the models. The models were imple- mented using the PyTorch framework and executed on an NVIDIA Tesla V100 GPU. Specific param- eter values were assigned to each module of the models:
Huaiwen Zhang, Quan Fang, Shengsheng Qian, and Changsheng Xu. 2019. Multi-modal knowledge- aware event memory network for social media ru- mor detection. In Proceedings of the 27th ACM in- lernational conference on mullimedia, pages 1942- 1951.
the crowd intelligence based semantic fea- ture learning module, the number of hidden units the TweetBert layer is 768. Pre-trained model pa- rameters are utilized, and their weights are frozen to prevent overfitting The BiLSTM layer consists of 256 hidden units. The size of the convolutional kernels (k) is fine-tuned among the options (2, 3, 4, 5, 6) and eventually set to {2, 3, 4). The num- ber of convolutional kernels (k_rum)) fixed 256. The capsule network layer contains a variable number capsules (Cin) selected from (2, 4, 6, 8, 16), with the final choice being 4. The output dimension of each capsule (Cdim) is set to 50. The number of iterations for capsule dynamic routing (r) is determined from (1, 3, 5, 7), with the chosen value being 3. In the knowledge-baid based semantic structure mining module,
Wei Zhao, Jianbo Ye. Min Yang. Zeyang Lei, Suofei Zhang, and Zhou Zhao. 2018. Investigating cap- sule networks with dynamic routing for text clas- sification. arXiv preprint arXiv: 1rXiv:1804 1804.00538.
Yongchun Zhu, Qiang Sheng, Juan Cao, Shuokai Li, Danding Wang, and Fuzhen Zhuang. 2022. Gen- eralizing to the future: Mitigating entity bias in fake news detection. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2120-20 2125.
Arkaitz Zubiaga, Ahmet Aker, Kalina Bontcheva, Maria Liakata, and Rob Procter. 2018. Detection and resolution of rumours in social media: A survey, ACM Computing Surveys (CSUR), 51(2):1-66.
utilized GloVe 300d (Pennington et al., 2014) word embedding to en- code individual entities and sentences. To enhance the effectiveness of Entity-Aware Hierarchical At- tention, we experimented with different numbers of attention heads (K) and evaluated their impact on performance. Specifically, we explored the options {1, 2, 4, 6, 8) for K and determined that setting K to 4 yielded optimal results.
A Dataset Details
To evaluate proposed approach, conducted experiments on two publicly available benchmark datasets, Twitter15 and Twitter16. The statistical details of these datasets are summarized Table 3,
Table 3: Statistics of the datasets
The cross-entropy loss function is employed during training assess the disparity between the predicted results and the true labels. Through back- propagation, parameter updates are carried out us- ing the Adam optimizer (Kingma and Ba, 2014). An early stopping (Yao et al., 2007) strategy is adopted to prevent overfitting. These parameter set- tings ensure reproducibility, fairness, and accurate experimental results.
Veracity
Twitter15
Twitter16
N
262
144
F
259
144
Training
T
262
142
144
Total
1043
573
N
75
41
F
Validing
T
U
Total
298
164
N
37
21
C Prompt Settings
F
Testing
T
20
Specifying clear instructions and tasks in the prompt is essential to enhance ChatGPT's perfor- mance in specific tasks. Therefore, adopt an it- erative experimental approach to optimize the final prompt, As shown in Figure 5. Prompt 1 used for data augmentation operations in ChatGPT-basee Knowledge Extraction, and Prompt is used for entity linking and conceptualization ChatGPT.
U
Total
149
82
N
374
205
F
205
T
203
Total
U
372
205
User
276663
173487
Com
331612
204820

---

Illustration of ChasGPT-baseed Knowledge Extraction Process
Type in: "possible" "robbery" took place and bam! like magic it's as if another unarmed
Prompt
Refined Content:" "There is suggestion a polential rubbery incident, and suddenly, it seems the memory unother unforturati incideni where un nuarmed black youth was killed has faded away, It's disteartening The incident associated with the hestiags Psmil and #ierguson."
black youth being murdered has been
Prompt2
forgotton #smb #ferguson'
Concept Set.
Entity Set:
C'sugeessoon ('dlvice/. 'proposall' 'reoouneedchloon ...)
I !ssggestion: "potcurial robbary incident, 'mounory" humortmnate incident' Ammrmedd black you!l.... ....
Cpottential pobbery incident' = l'robbery' 'crime', 'theff ...]
C Cmemoryy'= (remssmbrnece; 'recolliction", 'retention' ...
C'unfoctunate incideur- l'socident, 'mishapp' 'calamityy ...
C'unareed black you's- ("racial violence', social injustice'.....
Figure 4: Illustration of ChatGPT-based Knowledge Extraction Process
Promptt
You are processoe, your tass process the gven ramer detection text data, please
Prompt 3:
craame thrs procoeesl lests
1|
nusor detector, and your tadi judge whether evest sue, falle. ooo-tumour, or und-tmmioshle based on sonvce twcets rnd comment information gives the eveut. You will see the souuc mozets and celated commeatt of an evest. please judge step by stop according to the following requietmenss. Noriez
Th processed text should abstreet wathout changing an orginall meaning of the document.
2. The proceseed lndt shool: natuin all important olgacic cincagte and the wistionship sewwen cem.
3. The grocessed lnd doolld only centain in forrssiion from the ted)
4. processed should
underttndaatle wathout any abtrevations, specual marks
wher performing
detection, please consider the the
and relevant
bashaggs) and spilling mistakes
infocmation
judge authentiett possible rimee clocattetiours such is
You to generaz the following result:
source eweets commenss,
1ype in: the ned you need to peoors!
expeciinns uncertainty lack crodile sources confllcting intormation etc. Regreding the classitictioon caiecia, towe' menns that a true nuzor is sturtion that iniially lacks cridemee support but is confimeed to he inr afher fiat verification. which may be caused by Bettess soct incomplete intormation, rpi speead, or
Retines costent: the processed tost'
I
Moree is the ted you need to procoss:
Pronpe
insenmste proliminury imose gatims, 'oon -nmmat stands for non-nmerr The bipges whetaer there sufliceen rvidhmas cuppnt when fire sprod. 'iibel strads Fabus inferaation untertonaaly lahncured or speead miseedd anthenees spacifiz inteneds. When you mmmutter insuffiiii informiio or camee deserrine the sature of the camee, please select the tig 'uvveniee..
You ard hailding knowlagge graph. Your lask link and concaptuative untitics from the given || diffecence betwees
text. Please extaut it sepp by step.
fils,
()) Estract the untilies in the giver kest. Frritiie can he mouns or muan plaasee ihui refer to real-
presie
world objects abstract concepts. You can use named encry recognation (NER) tool or part
spech (POS) lagger ideelify iniiiies
Link the
entries the
companing and comppennding cnilty in berween entry and the knowledge hase esuhlisted.
knowdedge
Yiu. should only dineclly cupui the prodiced lihek for clasifictioo tasks. Your unower should be one of ['tuel, "lale, 'men-unoot "unveriied)) do net cuiput senieroe, other wonds
mitching entlt's contest, entily attributes,
associated
kanwladd ladeldee hase,
(3) orr each idcotified cntity, we oboxn concept information from the cassting koow gmph ttrough concepmallzationn acopting the "lak" celatioship to ebtais the concept. Concepuulizatann
quesitons, by rujjective assumptions.
When
resonrable
aroid
Map the extnected cniliss to related cnncppt or topics. this stop mesoiatic cutiiiee with sct of coucepts topies order better understiad the semantic meaning and domain of catties.
|| Esample:
the jodgment resul: you need to cotput: one of [tmee', Talse, 'zon-mmorr
Most impestinity if you find may kaowledgo output "Noae"
You need genemae the following sesult:
[
Here's the source nweet and comment info eatered:
Lecky Set Limitty Limity Linizy V..... Wutty X)
consogts eccrspending to ontity:
Chandry 1-- ('Cncepp L. Concept 2. Conrept F. Coucept N
C'Faaity ('Conege I,, Concept 2. Concept V. Concept N)
Duc space linitations, we give the prompt of d-shor here and the eusmpl of
CTFaiiyy N (Concept Nonnept 2, Konoop Y..., Vonnage NY
fem sbot is ozzimed
Here you process
Figure 5: Prompt instances used in the ChatGPT-based Knowledge Extraction part of the model
based Knowledge Extraction. The process of using ChatGPT for knowledge extraction is shown in Figure 4. Prompt 3 is designed to use ChatGPT alone for text classification tasks to enhance its understanding of text content and facilitate more accurate classification.
results of the experiments are shown in Figure 6.
It can be seen from the figure that the ChatGPT-Asssisted method can improve the accu- racy of text classification, but using ChatGPT alone for text classification may be limited because it is not designed explicitly for text classification tasks or pre-trained with data from social networks. On the contrary, using ChatGPT for knowledge extrac- tion can combine its powerful generative ability with the contextual information of the original text, which helps the model capture the latent features of the text, thereby improving the accuracy of text classification. At the same time, the training of ChatGPT is very dependent on the quality of the prompt. Few-shot results are better than zero-shot because when dealing with zero-shot classification, ChatGPT generates labels or data that do not corre- spond to the given label set, resulting in lower accu-
Quantitative Analysis of CICAN
This section provides a comprehensive analysis and effectiveness evaluation of the CICAN model.
ChatGPT vs ChatGPT-Assisted
We conducted experiments to compare the performance of using ChatGPT directly for down- stream classification tasks versus using it an auxiliary task. We developed prompts for ChatGPT (as shown in Prompt of Appendix C) and con- ducted experiments with 0-shot, 2-shot, and 5-shot samples for each class in the target domain. The

---

Table 4: The effectiveness analysis of the methods
Entity Enhancement
Hierarchical Attention
Dataset
CICAN
CICAN-Non
CICAN-ConcepIAut
CICAN-Coneat CICAN-SemanticAAtt
Tweet1
0.827
0.838
0.840
0.886
0.855
Tweet16
0.820
0.833
0.837
0.835
0.840
Effectiveness of Context-based Entity En- hancement Methods
O-sbot
2-ébot S-abor Chan@TTAAsstttd
Compared to CICAN-Non, CICAN achieved an accuracy improvement of 2.8% and 2% on Twit- ter15 and Twitter16, respectively. Furthermore. compared to CICAN-ConceptAAt CICAN demon- strated an accuracy improvement 1.7% and 0.7% on Twitter1s 15 and Twitter16, respectively. The context- entity augmentation method shows the best performance in the rumor detection task. The method combines attention computation to cap- ture the relationship between entities and the whole sentence and then applies attention weights to en- hance the context-based entity enhancement repre- sentation. This finding underscores the significance of context-aware entity enhancement the context of rumor detection tasks.
0.8
0.6
0.5
0.3
1witter1--Acc
Twitterl O-Acc
Figure Performance comparison of ChatGPT and ChatGPT-Assited
racy. To alleviate this problem, ChatGPT needs to be provided with examples that help it understand the content and facilitate accurate classification.
Effectiveness of Entity-Aware Hierarchical A Attention Method
Method Effectiveness Analysis
By comparing the experimental results, it can be observed that the performance of CICAN-concat and
We performed comparative analysis with commonly used methods to demonstrate the ef- ficacy of the optimization methods proposed in the model (Context-based Entity Enhancement and Entity-Aware Hierarchical Attention). Table presents the experimental results of CICAN and the following models based different modeling methods. According different modeling meth- ods, it mainly includes the following models.
are comparable, while Entity-Aware Hierarchical Attention demonstrates relatively superior performance. Our Entity-Aware Hierarchical Attention performs relatively superior, increasing accuracy by 1.5% and 0.3% on the two datasets, respectively. Duc to the complex and di- verse associations among different meta-paths in the heterogeneous graph, the co-attention mecha- nism employed in Entity-Aware Hierarchical At- tention more effectively capture meta-paths' independence and collaborative interactions.
1) CICAN-Non: No entity enhancement.
2) CICAN-ConcepIAtt: Use Concept Atten- tion to replace the Context-based Entity Enhance- ment method. Concept Attention is method first proposed in MKEMN (Zhang al., 2019) to mea- sure the semantic similarity between concepts and entity representations.
Overall, these results indicate the superior- ity of CICAN over its comparative models and highlight the importance of incorporating attention mechanisms and context-aware entity augmenta- tion during the modeling process in the context of rumor detection tasks to achieve more accurate and robust rumor detection.
3) CICAN-concald Concat the representation of the meta-path to replace the Entity-Aware Hier- archical Attention method.
Use Semantic At- tention to replace the Entity-Aware Hierarchical Attention method. Semantic Attention is a method proposed in HAN (Wang et al., 2019) 9) to calculate the importance different meta-paths through non- linear transformation.
4) CICAN-SemanticAl


Reviews:
[{"review_id": "T1HkhWUZ8e", "reviewer": "Reviewer_6ht6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewers for their attention to this aspect of our work. We did consider this matter in our original manuscript. As you've noted, the challenge of rumor detection using knowledge graphs is succinctly stated in the Introduction (Line 81). Moreover, *Section 2.2 External Knowledge-Based Approaches* comprehensively discusses knowledge graph-based methods and their inherent limitations. Additionally, we employed knowledge graph-based methods (elaborated in detail in *Section 5.2 Baselines*) and conducted sufficient comparative experiments to illustrate the distinctions and benefits of our model compared to the baseline methods in rumor detection (as shown in Table 1).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper well written and details are presented very well. This paper also have novelty.", "reasons_to_reject": "This gap in literature discusses in the introduction line number 81. There must be some more discussion regarding knowledge graphs and rumour detection. The author must discuss explicitly the methods using knowledge graphs and their limitations. This study must be compared with knowledge graph base methods in the literature for rumour detection to show the difference between knowledge graph chatGPT-based methods.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposed crowd-based intelligence methods which extract information from  to verify the authenticity of tweets. This paper relies on ChatGPT instead knowledge graph.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ByNKSjP9XJ", "reviewer": "Reviewer_oyJe", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1. The construction details of heterogeneous graphs are not provided.**\n\n\n\nA1: The construction of heterogeneous graphs is discussed in Section 4.2 of the manuscript under “Entity-Aware Hierarchical Attention”. We will elaborate further on this to show full build details in the final release.\n\n**Q2. The meaning of the horizontal and vertical coordinates in Figure 3 is unclear.**\n\n\n\nA2: We apologize for the unclear meaning of the coordinates in Figure 3. In the revised manuscript, we will add clear labels to the horizontal and vertical axes, clarifying that the horizontal axis represents the number of comments and the vertical axis represents the model accuracy.\n\n**Q3. The image quality of Figure 4 is not clear enough.**\n\n\n\nA3: Thank you for bringing up the issue regarding the image quality of Figure 4. In response, we will ensure that the image resolution is improved for Figure 4 in the final version of the paper, ensuring that all details are clearly visible.\n\n\n\n**Q4. The preprint paper should not be cited in the manuscript.**\n\n\n\nA4: We appreciate your feedback on the citation of the preprint paper. We will carefully review and revise our citations to ensure that only appropriate and peer-reviewed references are included in the final manuscript.\n\n**Q5. Typos Grammar Style And Presentation Improvements.**\n\n\n\nA5: Thank you for your thorough review of the manuscript. We admit there are some typos and grammatical issues, such as the example you provided. We will thoroughly check and fix syntax errors in the final commit.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.The method proposed in the manuscript is innovative. \n2.The author conducted extensive experiments on public datasets to verify the effectiveness of the proposed method.", "reasons_to_reject": "1.The construction details of heterogeneous graphs are not provided\n2. The meaning of the horizontal and vertical coordinates in Figure 3 is unclear.\n3. The image quality of Figure 4 is not clear enough\n4. The preprint paper should not be cited in the manuscript.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This manuscript proposes a swarm intelligence and chat GPT-assisted network for rumor classification. \nThe proposed model extracts semantic content and semantic structure features by introducing crowd intelligence and adequate external knowledge. \nThe manuscript applies large language models (LLM) to rumor detection for the first time, using Chat-GPT as an auxiliary means for text enhancement, entity linking, and text conceptualization, and demonstrates the effectiveness and accuracy of the approach.\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "There are some typos and grammar issues in the manuscript, e.g., \n“…Realated Work…”\n“…Where ci and cj represent a node with a SES structure, ei and ej represents a node with a ESE structure…”\n"}, "extra_text": {}}, {"review_id": "cBWDDGqeob", "reviewer": "Reviewer_2v6J", "summary": "", "questions": "None", "limitations": "", "rating": 4, "confidence": 4, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your positive feedback and agreement with our work. Thank you for your time and consideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper effectively identifies the problem of rumor detection and proposes a novel approach to tackle it. Through a well-designed methodology, the authors conduct experiments that provide substantial evidence supporting the effectiveness of their proposed solution.", "reasons_to_reject": "None", "questions_to_authors": "None", "paper_topic_and_main_contributions": "What is the paper about?  \n\nThe authors propose a new and innovative rumor detection model called Crowd Intelligence and ChatGPT Assisted Network (CICAN). The model aims to improve rumor detection by leveraging both semantic content and structure information. \n\nWhat is the problem?  \n\nFully mining the semantic structure information in the text is essential for accurate rumor detection, and the two main problems that need to be tackled are entity bias caused by insufficient learning of entity information and the difficulty in extracting complex long-distance semantically dependent features due to the dispersed distribution of important information in the text. Constructing heterogeneous graphs and selecting appropriate meta-paths can be key solutions to address these issues effectively. \n\nMain contributions of the paper: \n\n1. The proposed CI-CAN model extracts semantic content and semantic structure features, combining crowd intelligence and external knowledge, which provides a novel approach to rumor detection. The authors redefine rumors based on crowd intelligence and suggest future research directions in this field. \n\n2. This paper applies large language models (LLM) to rumor detection for the first time, using ChatGPT as an auxiliary tool for text enhancement, entity linking, and text conceptualization. The effectiveness and accuracy of this approach are demonstrated. \n\n3. Experimental results on two real-world datasets show that the proposed method outperforms baseline approaches and substantially enhances the task of rumor detection. \n\nIn summary, the paper introduces the CICAN model, which utilizes crowd intelligence and a large language model (ChatGPT) to improve rumor detection by extracting semantic content and semantic structure features. The experimental results indicate its improved results over baseline approaches, making it a promising and effective method for detecting rumors. ", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None", "Typos_Grammar_Style_and_Presentation_Improvements": "None"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper advances the state of the art for rumor detection using novel methodology that is backed with strong evidence. The reviewers appreciated the novelty of the work along with the experimentation. The authors are recommended to address the reviewers' concerns in the camera ready, if the paper is accepted.
Decision: Accept-Main