BIC: Twitter Bot Detection with Text-Graph Interaction and Semantic Consistency
Year: 2023

Abstract:
Twitter bots are automatic programs operated by malicious actors to manipulate public opinion and spread misinformation. Research efforts have been made to automatically identify bots based on texts and networks on social media. Existing methods only leverage texts or networks alone, and while few works explored the shallow combination of the two modalities, we hypothesize that the interaction and information exchange between texts and graphs could be crucial for holistically evaluating bot activities on social media. In addition, according to a recent survey (Cresci, 2020), Twitter bots are constantly evolving while advanced bots steal genuine users’ tweets and dilute their malicious content to evade detection. This results in greater inconsistency across the timeline of novel Twitter bots, which warrants more attention. In light of these challenges, we propose BIC, a Twitter Bot detection framework with text-graph Interaction and semantic Consistency. In particular, BIC utilizes a textgraph focused approach to facilitate the two communication styles of social media that may trade useful data throughout the training cycle. In addition, given the stealing behavior of novel Twitter bots, BIC proposes to model semantic consistency in tweets based on attention weights while using it to augment the decision process. Extensive experiments demonstrate that BIC consistently outperforms state-of-theart baselines on two widely adopted datasets. Further analyses reveal that text-graph interactions and modeling semantic consistency are essential improvements and help combat bot evolution.

heading: 1 Introduction
text: Twitter bots are controlled by automated programs and manipulated to pursue malicious goals
∗ These authors contributed equally to this work. † Corresponding author: Minnan Luo, School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China.
such as advocating for extremism and producing spam (Dickerson et al., 2014; Berger and Morgan, 2015). Bots are also involved in spreading misinformation during the pandemic (Shi et al., 2020). Since Twitter bots pose a threat to online society, many efforts have been devoted to detecting bots.
The majority of the existing approaches are textbased and graph-based. The text-based methods analyze the content to detect Twitter bots by natural language processing techniques. Kudugunta and Ferrara (2018) adopted recurrent neural networks to extract textual information. Guo et al. (2021) utilized the pre-trained language model BERT to help detect bots. The graph-based methods model the Twittersphere as graphs and adopt geometric neural networks or concepts of network dynamics to identify bots. Feng et al. (2022a) constructed a heterogeneous graph and leveraged different relational information. Magelinski et al. (2020a) exploited the ego-graph of Twitter users and proposed
10326
a histogram and customized backward operator. However, existing methods are faced with two challenges. On the one hand, these methods only adopt texts or graphs alone, and only a few works shallowly combine the two modalities as Figure 1(a) shows. The text-based model can not get the graph modality information while the graphbased model can not get the text modality information. We hypothesize that it is wise to interact and exchange information between texts and graphs to evaluate bot activities. On the other hand, Cresci (2020) pointed out that Twitter bots are constantly evolving. Advanced bots steal genuine users’ tweets and dilute their malicious content to evade detection, which results in greater inconsistency across the timeline of advanced bots as Figure 1(b) illustrates. Previous methods can not capture this characteristic. Namely, there is an urgent need for a method that can identify advanced bots.
In light of these challenges, we propose a framework BIC (Twitter Bot Detection with Text-Graph Interaction and Semantic Consistency). BIC separately models the two modalities, text and graph, in social media. A text module is adopted to encode the textual information and a graph module is used to encode graph information. BIC employs a text-graph interaction module to enable information exchange among different modalities in the learning process. To capture the inconsistency of advanced bots, BIC leverages a semantic consistency module, which employs the attention weights and a sample pooling function. Our main contributions are summarized as follows:
• We propose to interact and exchange information across text and graph modalities to help detect bots. We find that capturing novel bots’ inconsistency can increase detection performance.
• We propose a novel Twitter bot detection model, BIC. It is an end-to-end model and contains a textgraph interaction module to exchange modality information and a semantic consistency module to capture the inconsistency of advanced bots.
• We conduct extensive experiments to evaluate BIC and state-of-the-art models on two widely used datasets. Results illustrate that BIC outperforms all baseline methods. Further analyses reveal the effectiveness of the text-graph interaction module and semantic consistency module.

heading: 2 Problem Definition
text: We first define the task of Twitter bot detection with the text and graph modality. For a Twitter user ui ∈ U , the text modality contains the description Bi and the tweets Si = {Si,j}Tij=1, where Ti denotes the tweet count. The graph modality contains the attribution fi of ui and the heterogeneous graph G = G(U,E, φ,Re), where U denotes the user set, E denotes the edge set, φ : E −→ Re denotes the relation mapping function and Re is the relation type set. The neighbors of ui can be derived from G as Ni = {ni,j}Jij=1 where Ji is the neighbor count. The goal is to find a detection function f : f(ui) = ŷ ∈ {0, 1}, such that ŷ approximates ground truth y to maximize prediction accuracy.

heading: 3 Methodology
text: Figure 2 shows an overview of our proposed framework named BIC. Specifically, BIC first leverages a text module to encode textual information and a graph module to encode graph information. BIC then adopts a text-graph interaction module to interact and exchange modality information in the learning process. To further interact the two modalities, BIC repeats this process for M times. BIC extracts the semantic consistency from the attention weights derived from the text module with the help of the semantic consistency module. Finally, BIC leverages text modality, graph modality, and semantic consistency vectors to identify bots.

heading: 3.1 Modality Interaction
text: For simplicity, we omit the subscript of the user. BIC first encodes the text modality and graph modality information to obtain the initial representations. For text modality, BIC employs pre-trained RoBERTa (Liu et al., 2019) to encode description B and tweets {Si}Ti=1 into h (0) int and {h (0) i }Ti=1. BIC considers h(0)int as the text interaction modality because the description generally defines the user. For graph modality, BIC employs the same encoding methods as BotRGCN (Feng et al., 2021c) to get the user graph feature g(0)int as the graph interaction representation and representations of its neighbors {g(0)i }Ji=1.
After obtaining the initial representations, BIC employs M times modality interaction to ensure text and graph information interact completely. We describe the l-th interaction process as follows.
Text Module BIC puts text representations into a language model to extract textual information, i.e.,
{h̃(l)int, h (l) 1 , · · · , h (l) T } = LM({h (l−1) int , h (l−1) 1 , · · · , h (l−1) T }),
(1) where h̃(l)int denotes the interaction representation of text modality before interaction. BIC adopts transformer with multi-head attention (Vaswani et al., 2017) as the language model LM.
Graph Module BIC first feeds graph representations into a graph neural network to aggregate information between users and its neighbors, i.e.,
{ĝ(l)int, ĝ (l) 1 , · · · , ĝ (l) J } = GNN({g (l−1) int , g (l−1) 1 , · · · , g (l−1) J }).
BIC adopts relational graph convolutional networks (Schlichtkrull et al., 2018) due to its ability to extract heterogeneous information. To measure which neighbor is important for bot detection, BIC employs multi-head attention for the user, i.e.,
{g̃(l)int, g (l) 1 , · · · , g (l) J } = att({ĝ (l) int, ĝ (l) 1 , · · · , ĝ (l) J }),
where g̃(l)int denotes the interaction representation of graph modality before interaction and att denotes multi-head attention.

heading: 3.1.1 Text-Graph Interaction Module
text: BIC adopts a text-graph interaction module to interact and exchange information across text and graph modality in the learning process. Specifically, BIC employs an interaction function inter to interact
the text modality representation h̃(l)int and the graph modality representation g̃(l)int, i.e.,
(g (l) int, h (l) int) = inter(g̃ (l) int, h̃ (l) int).
For the details about inter function, BIC calculates the similarity coefficient between modality representations, i.e.,
whh = h̃ (l) int ⊗ (θ1 · h̃ (l) int), whg = h̃ (l) int ⊗ (θ2 · g̃ (l) int),
wgh = g̃ (l) int ⊗ (θ2 · g̃ (l) int), wgg = g̃ (l) int ⊗ (θ1 · h̃ (l) int),
(2)
where θ1 and θ2 are learnable parameters that transform the modality representations into the interaction-sensitive space, and ‘⊗’ denotes the dot product. BIC then applies a softmax function to derive final similarity weights, i.e.,
w̃hh, w̃hg = softmax(whh, whg),
w̃gg, w̃gh = softmax(wgg, wgh).
BIC finally makes the two representations interact through the derived similarity weights, i.e.,
h (l) int = w̃hhh̃ (l) int + w̃hg g̃ (l) int,
g (l) int = w̃gg g̃ (l) int + w̃ghh̃ (l) int.
So far, BIC could interact and exchange information between the two modalities.

heading: 3.2 Semantic Consistency Detection
text: Since attention weights from the transformer could indicate the correlations and consistency between tweets, BIC adopts the attention weights to extract the semantic consistency information. BIC can obtain the attention weight matrix Mi ∈ R(T+1)×(T+1) of text representation from equation (1) in i-th interaction process. BIC then employs a down-sample function to reduce the matrix size and obtain what matters in the matrix, i.e.,
M̃i = sample(Mi), M̃i ∈ RK×K , where K is a hyperparameter indicating the matrix size. BIC adopts a fixed size max-pooling as sample function in the experiments. BIC then flats the matrix and applies a linear transform to obtain the semantic consistency representation, i.e.,
di = θsc · Flatten(M̃i), where θsc is a shared learnable parameter of each interaction process. Finally, BIC applies an aggregating function to combine the representations of each interaction process, i.e.,
d = σ(WD · aggr({di}Mi=1) + bD), where WD and bD are learnable parameters, σ denotes the activation function, and aggr denotes the aggregating function (e.g., concatenate or mean).

heading: 3.3 Training and Inference
text: BIC concatenates text modality h(M)int , graph modality g(M)int , and semantic consistency d representation to obtain the representation of a user, i.e.,
z = WD · (h(M)int ∥g (M) int ∥d) + bD. (3)
BIC finally employs a softmax layer to get the predicted probability ŷ. We adopt cross-entropy loss to optimize BIC, i.e., l = − ∑
i∈U [yi log(ŷi) + (1− yi) log(1− ŷi)] + λ
∑ ω∈θ ω2,
where U denotes all users in the training set, θ denotes all training parameters, yi denotes the groundtruth label and λ is a regular coefficient.

heading: 4 Experiments

heading: 4.1 Experiment Settings
text: More detailed information about the experiment settings and the implementation details of BIC can be found in appendix A. We also included our code and the best model parameters as supplementary materials.
Datasets To evaluate BIC and baselines, we make use of two widely used datasets, Cresci15 (Cresci et al., 2015) and TwiBot-20 (Feng et al., 2021b). These two datasets provide user follow relationships to support graph-based models. TwiBot20 includes 229,580 Twitter users, 33,488,192 tweets, and 33,716,171 edges, while Cresci-15 includes 5,301 Twitter users, 2,827,757 tweets, and 14,220 edges.
Baselines We compare BIC with Botometer (Davis et al., 2016), Kudugunta et al. (Kudugunta and Ferrara, 2018), Wei et al. (Wei and Nguyen, 2019), Alhosseini et al. (Ali Alhosseini et al., 2019), BotRGCN (Feng et al., 2021c), Yang et al. (Yang et al., 2020), SATAR (Feng et al., 2021a), and RGT (Feng et al., 2022a).

heading: 4.2 Main Results
text: We first evaluate whether these methods leverage text modality, graph modality, and interact modalities. We then benchmark these baselines on Crescie-15 and TwiBot-20, and present results in Table 1. It is demonstrated that:
• BIC consistently outperforms all baselines including the state-of-art methods RGT (Feng et al., 2022a) with at least 1% improvement of performance on two datasets.
• The methods that leverage graph modality, such as RGT (Feng et al., 2022a), generally outperform other methods that only adopt text modality or other features. SATAR (Feng et al., 2021a) achieves competitive performance with the text modality and the graph modality. BIC further makes these two modalities interact to achieve the best performance.
• We conduct the significance test using the unpaired t-test. The improvement between BIC and the second-best baseline RGT is statistically significant with p-value < 0.005 on Creaci-15 and p-value < 0.0005 on TwiBot-20.
In the following, we first study the role of the two modalities and the interaction module in BIC. We then examine the effectiveness of the semantic consistency module in identifying advanced bots. Next, we evaluate the ability of BIC to detect advanced bots. We finally evaluate a specific bot in the datasets to explore how BIC makes the choice.
Table 1: Bot detection performance on Cresci-15 and TwiBot-20 benchmarks. For each baseline except for Botometer which has fixed results, we run 5 times on the same splits with different random seeds. Text, Graph, Modality-Int respectively denote whether baseline leverages text modality, graph modality and modality interaction. Bold and underline indicate the highest and second highest performance. ‘BIC w/o Graph’ and ‘BIC w/o Text’ indicate BIC without the Graph Module and without the Text Module. BIC achieves the best performance.
Method Modalities Cresci-15 TwiBot-20
Text Graph Modality-Int Accuracy F1-score Accuracy F1-score
Yang et al. 77.08 (±0.21) 77.91 (±0.11) 81.64 (±0.46) 84.89 (±0.42) Botometer 57.92 66.90 53.09 55.13 Kudugunta et al. ✓ 75.33 (±0.13) 75.74 (±0.16) 59.59 (±0.65) 47.26 (±1.35) Wei et al. ✓ 96.18 (±1.54) 82.65 (±2.47) 70.23 (±0.10) 53.61 (±0.10) BotRGCN ✓ 96.52 (±0.71) 97.30 (±0.53) 83.27 (±0.57) 85.26 (±0.38) Alhossini et al. ✓ 89.57 (±0.60) 92.17 (±0.36) 59.92 (±0.68) 72.09 (±0.54) RGT ✓ 97.15 (±0.32) 97.78 (±0.24) 86.57 (±0.41) 88.01 (±0.41) SATAR ✓ ✓ 93.42 (±0.48) 95.05 (±0.34) 84.02 (±0.85) 86.07 (±0.70) BIC w/o Graph ✓ 97.16 (±0.58) 97.80 (±0.46) 85.44 (±0.32) 86.97 (±0.41) BIC w/o Text ✓ 96.86 (±0.52) 97.57 (±0.39) 85.78 (±0.48) 87.25 (±0.57) BIC ✓ ✓ ✓ 98.35 (±0.24) 98.71 (±0.18) 87.61 (±0.21) 89.13 (±0.15)

heading: 4.3 Text-Graph Interaction Study
text: Modality Effectiveness Study We remove the text modality representation h(M)int and the graph modality representation g(M)int in equation (3), to evaluate the role of each modality. The results are illustrated in Table 1. We can conclude that: (i) Removing any modality will cause a drop in performance, which illustrates that leveraging and making the two modalities interact can help identify bots. (ii) BIC without graph modality can achieve the second-best performance on Cresci-15. Other ablation settings can achieve competitive performance. It is shown that BIC can derive useful information from either modality and the semantic consistency representation can help identify bots.
BIC adopts text and graph modalities and leverages the text-graph interaction module to facilitate information exchange across the two modalities. To further examine the ability of BIC to extract modality information, we gradually remove part of one modality information and conduct experiments. The results in Figure 3 demonstrate that: (i) Each data modality benefits the performance of bot detection. It suggests that bot detection relies on the text modality and the graph modality information. (ii) BIC could keep the performance with less information of one modality. It illustrates that the interaction module effectively exchanges information across the modalities.
Interaction Function Study BIC employs an interaction function, which transforms representa-
tions into an interaction-sensitive space and learns the similarity weights, to exchange the modality information. Apart from our proposed similaritybased interaction, there are several other interaction functions. We replace this function with other functions such as mean or MLP, to evaluate the effectiveness of our proposed interaction function. We apply the following different interaction functions:
• Hard function computes the average of two interaction representations to interact.
• Soft function utilizes two learnable parameters as weights for two interaction representations to generate new representations.
• MLP function concatenates two interaction representations and feeds the intermediate into an MLP layer to interact.
• Text function feeds the interaction representation from text modality into Linear layers.
• Graph function feeds the interaction representation from graph modality into Linear layers.
The results in Table 2 illustrate that:
• Almost all interaction strategies outperform methods with no interaction, which indicates the necessity of utilizing an interaction module to make two modalities interactive and exchange information.
• Our similarity-based modality interaction function outperforms others, which well confirms its efficacy, indicating that it can truly make two modalities inform each other and learn the relative importance of modalities.
Interaction Number Study To examine the role of the modality information interaction number M , we conduct experiments with different interaction numbers of layers and evaluate the model memory cost (Params). The results in Figure 4 demonstrate that BIC with 2 interactions performs the best over other settings. Besides, the two-layer interaction model has relatively less memory cost, which makes it the best selection. As the number of interaction number increases, the performance declines gradually, which may be caused by the higher complexity of increasing the training difficulty. Meanwhile, the one-layer interaction model may be deficient for learning rich information, thus leading to unappealing performance.

heading: 4.4 Semantic Consistency Study
text: Discrimination Case Study We check users’ tweets in the datasets to determine that humans and bots have different semantic consistency patterns and that advanced bots may steal genuine tweets.
We choose a genuine user, a traditional bot, and an advanced bot. Their representative tweets are displayed in Figure 5 and we can find that novel bots will have more tweet inconsistency than genuine users and traditional bots, which post similar spam tweets. Next, we check their semantic consistency matrices M̃i, and they are shown in Figure 6. We can find that the advanced bot has a relatively higher inconsistency in its matrices.
Discrimination Ability Study BIC adopts the attention weight from the text module to generate the semantic consistency representation d. We try to find out the extent to which our semantic consistency module can distinguish bots from genuine users. We derive consistency matrices M̃i and calculate the largest characteristic value. We draw box plots with these characteristic values to
find the differences between bots and humans excavated by the module. The results shown in Figure 7 demonstrate that the consistency matrices of bots and humans are quite different.
To evaluate that the semantic consistency representation d can distinguish bots and humans, we conduct the k-means algorithm to cluster the representations and calculate the V-measure, which is a harmonic mean of homogeneity and completeness. BIC achieves 0.4312 of v-measure on Cresci-15 and 0.3336 on TwiBot-20. More intuitively, we adopt t-sne to visualize the representation, and the results are shown in Figure 8, which shows clear clusters for bots and humans. It is proven that the semantic consistency representation can identify bots alone.

heading: 4.5 Advanced Bot Study
text: We claim that BIC could identify the advanced bots. To evaluate whether BIC can capture the advanced bots after 2020 (the TwiBot-20 published time), we
sample some users related to the pandemic from a new Twitter crawl (Feng et al., 2022b) to construct a new dataset. This dataset contains user-follow relationships including 5,000 humans and 5,000 bots. We compare BIC with RGT, the second-best baseline, and Botometer, the widely-used bot detection tool. We randomly split this dataset into the train set and the test set by 8:2 and train the methods. Table 3 illustrates the results. We can conclude that BIC achieves the best performance, which proves that BIC can capture advanced bots with the help of the text-graph interaction module and the semantic consistency module.

heading: 4.6 Case Study
text: We study a specific Twitter user to explain how BIC exchanges information across two modalities and learns the relative importance to identify bots. For this user, we study its tweets and neighbors with the top-3 highest attention weight. We then derive similarity weights in equation (2) to analyze it quantitatively. This user is visualized in Figure 9. We discovered that neighborhood information is more important in this cluster, due to more differences in attention weights of the selected bot’s bot neighbors and human neighbors than attention weights of tweets. The conclusion is also reflected
in similarity weights. The similarity weights of the original interaction representation from text modality are 0 and 0.051, while the similarity weights of the original interaction representation from graph modality are 1 and 0.949. The results further show the effectiveness of similarity-based interaction in that it indeed learns the emphasis on modalities.

heading: 5 Related Work

heading: 5.1 Twitter-bot Detection
text: Text-based Methods Text-based methods adopt techniques in natural language processing to identify bots. Wei and Nguyen (2019) adopted multiple layers of bidirectional LSTM to conduct bot detection. Stanton and Irissappane (2019) proposed to leverage generative adversarial networks to detect spam bots. Hayawi et al. (2022) adopted a variety of features and leveraged LSTM and dense layer to learn representations. Existing models can not capture the semantic consistency of users, which leads to failures to detect the advanced bots.
Graph-based Methods Social networks consist of rich information like social familiarity (Dey et al., 2017, 2018), attribution similarity (Peng et al., 2018), and user interaction (Viswanath et al., 2009). The graph constructs on Twittersphere help to detect bots. Feng et al. (2021a) leveraged user neighbor information combined with tweet and profile information. Graph neural networks are utilized to improve the Twitter bot detectors and can achieve great performance (Magelinski et al., 2020b; Dehghan et al., 2022; Yang et al., 2022). Ali Alhosseini et al. (2019) used graph convolutional networks to aggregate user informa-
tion. Feng et al. (2021c) constructed a heterogeneous graph and adopted relational graph convolutional networks to identify bots. Previous models leverage the text or graph modality alone without information interaction. We believe that exchanging modality information across two modalities can help improve performance.

heading: 5.2 Text-Graph Interaction
text: Text information is the basis of natural language processing, and pre-trained language models are the dominant framework in capturing text features (Devlin et al., 2019; Liu et al., 2019; Lewis et al., 2020). Meanwhile, graph neural networks are introduced to tackle NLP tasks, examples include fake news detection (Mehta et al., 2022), dialogue state tracking (Feng et al., 2022c), and machine translation (Xu et al., 2021). As both pretrained LMs and graph structure are proved to be effective, text-graph interaction was also widely used in the area of natural language processing. Some works interacted with two modalities hierarchically such as using encoded representations from knowledge graph to augment the textual representation (Mihaylov and Frank, 2018; Lin et al., 2019; Yang et al., 2019), or utilizing text representations to enhance the inferential capability of graph (Feng et al., 2020; Lv et al., 2020). More recently, GreaseLM (Zhang et al., 2022) proposed a model to allow two modalities to interact between layers by interaction nodes, in which a truly deep interaction was achieved.

heading: 6 Conclusion
text: Twitter bot detection is a challenging task with increasing importance. To conduct a more comprehensive bot detection, we proposed a bot-detection model named BIC. BIC interacts and exchanges information across text modality and graph modality by a text-graph interaction module. BIC contains a semantic consistency module that derives the inconsistency from tweets by the attention weight to identify advanced bots. We conducted extensive experiments on two widely used benchmarks to demonstrate the effectiveness of BIC in comparison to competitive baselines. Further experiments also bear out the effectiveness of modality interaction and semantic consistency detection. In the future, we plan to explore better interaction approaches.

heading: Acknowledgment
text: Qinghua Zheng and Minnan Luo are supported by the National Key Research and Development Program of China (No. 2022YFB3102600), National Nature Science Foundation of China (No. 62192781, No. 62272374, No. 62202367, No. 62250009, No. 62137002, No. 61937001), Innovative Research Group of the National Natural Science Foundation of China (61721002), Innovation Research Team of Ministry of Education (IRT_17R86), Project of China Knowledge Center for Engineering Science and Technology, and Project of Chinese academy of engineering “The Online and Offline Mixed Educational Service System for ‘The Belt and Road’ Training in MOOC China”. Minnan Luo also would like to express their gratitude for the support of K. C. Wong Education Foundation. All authors would like to thank the reviewers and chairs for their constructive feedback and suggestions.
Limitations
The BIC framework has two minor limitations:
• Our proposed BIC model utilizes representation from three different modalities, namely text, graph, and semantic consistency, and we introduce an interaction mechanism to allow information exchange between text and graph. However, whether interaction and information exchange are necessary among all three modalities is still an open question. We leave it to future work to study the necessity of introducing interaction modules.
• The new dataset we construct is limited to the topic of the pandemic, while other popular topics are not considered. However, Twitter bots are likely to behave differently with different topics. We leave it to future works to analyze how current approaches perform against bots with different topics.
Social Impact
Our proposed BIC is a Twitter bot detection model that leverages text-graph interaction and semantic consistency modules. However, there are potential biases or discrimination that exist among the text, graph, or semantic consistency-based representation. For instance, some users may be divided into the bot class for they may behave relevantly
"abnormal". In conclusion, we suggest that the application of the Twitter bot detection model should be guided by end users and experts.

heading: B Baseline Details
text: • SATAR (Feng et al., 2021a) leverages the tweet, profile, and neighbor information and employs a co-influence module to combine them. It pretrains the model with the follower count and finetunes it to detect bots.
• Botometer (Davis et al., 2016) is a publicly available service that leverages thousands of features to evaluate how likely a Twitter account exhibits similarity to the known characteristics of typical bots.
• Kudugunta et al. (Kudugunta and Ferrara, 2018) subdivide bot-detection task to account-level classification and tweet-level classification. In the
1https://github.com/Bjarten/early-stopping-pytorch 2https://github.com/Lyken17/pytorch-OpCounter
former, they combine synthetic minority oversampling (SMOTE) with undersampling techniques, and in the latter they propose an architecture that leverages a user’s tweets.
• Wei et al. (Wei and Nguyen, 2019) propose a bot detection model with a three-layer BiLSTM to encode tweets, before which pre-trained GloVe word vectors are used as word embeddings.
• Alhosseini et al. (Ali Alhosseini et al., 2019) utilize GCN to learn user representations from metadata such as user age, statuses_count, account length name, followers_count to classify bots.
• BotRGCN (Feng et al., 2021c) constructs a framework based on relational graph convolutional network (R-GCN) by leveraging representatives derived from the combination of user tweets, descriptions, numerical and categorical property information.
• Yang et al. (Yang et al., 2020) adopt random forest with account metadata for bot detection, which is proposed to address the scalability and generalization challenge in Twitter bot detection.
• RGT (Feng et al., 2022a) leverages relation and influence heterogeneous graph network to conduct bot detection. RGT first learns users’ representation under each relation with graph transformers and then integrates representations with the semantic attention network.

heading: C Evaluation Details
text: We elaborate on the evaluation of our baselines here. For methods without text and graph modalities, Lee et al. (2011) adopt a random forest classifier with Twitter bot features. Yang et al. (2020) adopt a random forest with minimal account metadata. Miller et al. (2014) extract 107 features from a user’s tweet and metadata. Cresci et al. (2016) encode the sequence of a user’s online activity with strings. Botometer (Davis et al., 2016) leverages more than one thousand features. All of them extract Twitter bot features, without dealing with these features in graph modality or text modality.
For methods with only text modality, SATAR (Feng et al., 2021a) leverages LSTM for its tweet-semantic sub-network. Kudugunta and Ferrara (2018) adopt deep neural networks for tackling user tweets. Wei and Nguyen (2019) propose
a model with a three-layer BiLSTM. All of them deal with user information in text modalities.
For methods with only graph modality, BotRGCN (Feng et al., 2021c) utilizes a relational graph convolutional network in its proposed framework. Ali Alhosseini et al. (2019) adopt graph convolution network to learn user representations and classify bots. RGT (Feng et al., 2022a) leverages heterogeneous graph networks to conduct bot detection. All of them deal with user information in graph modalities.

heading: D Modality interaction additional study
text: We conduct a qualitative experiment where we find that at least 54.8% of bots (51/93) that evaded the detection of only-text or only-graph models were captured by our proposed BIC, which also demonstrates BIC’s effectiveness.

heading: E Semantic consistency Study
text: Performance study To find how semantic consistency detection helps the overall BIC performance with different parameter settings, we experiment with different semantic consistency layers, consistency matrix pooling sizes, and consistency vector aggregation manners. The results shown in Figure 10 demonstrate that semantic consistency truly enhances the model performance. Although slight, differences are manifested in different parameter settings, which could be further studied.

heading: F Multi-task Learning Approach
text: Apart from using the interaction module to incorporate and exchange information between two modalities, we also consider a multi-task learning approach which might be more straightforward and intuitive, and have a similar effect. We conduct multi-task learning with both soft and hard parame-
ters. The results shown in Table 5 demonstrate that compared with multi-task learning, the interaction module could better exchange information between two modalities and lead to better performance.

heading: G Scientific Artifact
text: The BIC model is implemented with the help of many widely-adopted scientific artifacts, including PyTorch (Paszke et al., 2019), NumPy (Harris et al., 2020), transformers (Wolf et al., 2019), sklearn (Pedregosa et al., 2011), PyTorch Geometric (Fey and Lenssen, 2019). We commit to making our code and data publicly available to facilitate reproduction and further research.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
in the appendix
3 A2. Did you discuss any potential risks of your work? in the appendix
3 A3. Do the abstract and introduction summarize the paper’s main claims? introduction in Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? throughout the paper
3 B1. Did you cite the creators of artifacts you used? throughout the paper
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. in the appendix
C 3 Did you run computational experiments? in Section 4
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? figure 4 in Section 4 and appendix
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? table 4 in appendix
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? table 1 in Section 4
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? in appendix I
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: We leave it to future work to study the necessity of introducing interaction modules. • The new dataset we construct is limited to the topic of the pandemic, while other popular topics are not considered. However, Twitter bots are likely to behave differently with different topics. We leave it to future works to analyze how current approaches perform against bots with different topics. Social Impact
Our proposed BIC is a Twitter bot detection model that leverages text-graph interaction and semantic consistency modules. However, there are potential biases or discrimination that exist among the text, graph, or semantic consistency-based representation. For instance, some users may be divided into the bot class for they may behave relevantly
"abnormal"