Nigo SUMAILA†, Shoko WAKAMIYA†, and Eiji ARAMAKI† † Social Computing Lab., Nara Institute of Science and Technology (NAIST) 8916-5, Takayama-cho, Ikoma, Nara 630-0192, Japan E-mail: ↑{sumaila.nigo.sl8,wakamiya,aramaki}@is.naist.jp Several studies have shown that social bots might have impacts on various important areas of our society Abstract such as influencing the outcome of elections, the economy, or creating panic in time of crisis. Consequently, there is a growing need to develop and strengthen defense mechanism against these entities. So far, existing methods rely on users' global information, such as profile information, network-related, and from the text content only syntactic information have been used. In this paper, we propose a defense mechanism against social bots on Twitter, a neural network model that incorporates metadata features and semantic information, pairwise tweet similarities and ngram features from a user's content to differentiate genuine users from social bots. Our model outperforms baseline systems in three publicly available datasets. Twitter, content similarity, tweet semantic, ngram model, social bots detection, Convolutional Neural Key words Networks, social computing
Introduction
1
A recent study [3] suggests that social bots (social media accounts controlled algorithmically by software to generate content and interact with other users, automatically) may have played an important role in the 2016 U.S presidential election. An analysis of election-related tweets showed that bots produced near one-fifth of the entire conversation. Ear- lier, during the 2010 U.S midterm election, social bots were used to support some candidates and defame their opponents [30]. These kinds of incidents are not particular only to U.S. elections' arena, similar patterns have also been observed in other countries as well [13; 31; 19]. Outside of politics, there are also several reported impacts of social bots. For instance, bots have been associated with causing panic during a time of crisis by disseminating false and unverified information [35]; affecting the stock market by generating a vivid discussion about a company which created an apparent public interest [6]. As result of a widespread use of social media services, they have become a crucial data source in various academic re- search. Several studies concerned with correlating real-life events with observations made from social media data have been published. From detecting earthquakes [32; 12], elec¬ tion outcomes [25; 21] to disease outbreaks [34; 2; 8].  In addi- tion, [36] estimation places social bot population on Twitter between 9% to 15% of total active users. Therefore, con¬ tents generated by bots might undermine a useful predictions made from SNS data. Consequently, there is a growing need to develop and strengthen defenses against these entities. However, detecting bot is one of the most daunting tasks in social media analysis [13]. The challenge is understand- ing what modern bot can do. Bot have evolved from easily detectable ones which performs mainly one type of activity, such as posting or re-tweeting automatically, to increasingly sophisticated ones. Now the boundary between what consti- tutes human-like and bot-like behavior has become fuzzier Most of the previous work on bot detection fall in to two perspectives. First, network-based methods in which the de- cision is drawn from the users network analysis, with the assumption that social bots tend to be mostly connected to other bots [39; 6].  However, recent work show [29; 4] that ad- vanced social bots may successfully be connected to humans making them impossible to be detected solely on network- based assessments.  Second, machine-learning schemes are trained with language-based, and other features (extracted from users' tweets and metadata) to capture users' behav- ioral patterns in order to differentiate real users from so- cial bots. However, language-based features used in previous work were mostly syntactic inherent features, and there is still much to be explored in semantic features. In this paper, we propose a defense mechanism against so- cial bots on a social network service, Twitter, that leverage nuances in semantic features from a user's timeline tweets combined with tweet similarity, n-gram features and meta¬ data based features. Figure 1 illustrates the overview of the proposed method against previous methods based on many complex features. Some of the features require much time to obtain. Especially network and friend features are inconsis- tent. In contrast, the proposed method focused mainly on text-oriented features: semantic and content features, which are stable and easy to obtain, combined with users' metadata features. The contributions of the paper are summarized as follows: We propose a bot detection focused on semantic and . content features, which are simple statistics-based features to overhead computational complexity. We construct CNN (Convolutional Neural Network) models that incorporates semantic features together with tweet similarity, n-gram features and metadata based fea- tures. The experiments conducted with publicly available datasets show that the proposed model outperform baseline models. Our results show that we can improve performance from a base model that uses only semantic features and meta¬ data features by adding other features.  This might moti- vate other researchers to consider adding semantic features to their models.
Related Work
2
21   Bot Detection
Since the early days of social media, several research on automatic identification of non-genuine users on social me- dia network have been conducted [39; 11]. Most of the work done in this area fall into two categories: network-based and feature-based decision. On network-based bot detection, the decision is based on social graphs analysis.  For example, Sybil (# 1) Guard [39] assumes that non-Sybil region, region bounding human-to- human-established trust relation, is fast mixing and Sybil admission to the region is based on admission control proto- col. SybilRank [6] assumes that connection between users is based upon the trustworthiness among then, consequently, Sybil accounts show less connectivity to real users and more to other Sybils to appear trustworthy. However, sophisti¬ cated bots might gain trust of real users [29; 4], and to be able to penetrate their communities, making them impossible to be spotted solely by network-based assessments through trust assumption [14].  Therefore, other aspects should be considered. On feature-based detections, machine-learning schemes are trained to capture users' behavioral patterns and meta-data in order to differentiate real users from social bots. [37] and [1] focus on the detection of spam tweets, which optimizes the amount of data that needs to be gathered by relying only on tweet-inherent features.  [36] leverage more than one thousand features distributed in six categories (user- based, friend, network, temporal, content, and sentiment fea- tures) obtained from users' metadata, mentions and timeline tweets. [10] show that natural language text features are ef- fective at separating real users from social bots.  However, content and language features extracted from tweets text in the previous works [36; 10; 37] were mostly syntactic inherent features.
22   Chat Bot
In Natural Language Processing (NLP) context, the main goal is to realize natural conversation instead of the bot de- tection. Most current bot engines are designed to reply to user utterances based on existing utterance-response pairs [38]. In this framework, to capture the relevance between utterance and response is fundamental. Then, the relevance is calculated by the framework's ability to retrieve the most relevant pairs from the conversation database (retrieval ap- proach), or generate the response to the given utterance (gen- erative approach).  Although mostly the target relevance is limited to the short scope, usually, a single utterance- response pairs, our proposed method handle another wider relevance of the entire tweets (Section 3.2).
3 Model
We formulate the task of automatically identifying non- genuine users on social media as follows:  given a user u's posts (timeline tweets) X u = {x 1 ,x 2 , ...,x n} , our goal is to
find Y → {0,1} = { human , bot } for the user such that:
Y* = arg max P(Y|X u )
Representation of our model is illustrated in Fig. 2. This model is slight alteration of a shallow CNN for sentence clas- sification tasks [22; 40] to accommodate basic features, tweet similarity and n-gram lengths. Given a sequence of n tweets X u from a given user u , we apply it in three different parts. First, we compute word embedding from the concatenation of all tweets (Section 3.2); second, we compute basic features (Section 3.1), then pairwise tweet similarity from all tweets (Section 3.3), and lastly, we compute several statistics about lengths of different classes of n-gram (Section 3.4). The word embedding layer is followed by a convolution layer comprised of several filters of different sizes (3, 4 and 5), but with the same width as the dimension of the embedding vectors.  Every filter performs convolution over embedding matrix, producing feature map.  Next, we run the feature map through an element-wise non-linear transformation, us- ing Rectified Linear Unit (ReLU) [23].  Then, applying 1-max pooling [5], then we extract the maximum value from each feature map. We concatenate all extracted values from fea- ture map together with features from basic, pairwise tweet similarity and n-grams lengths. We then feed all features to a full connected SoftMax layer for final output. Dropout [17] is used as a mean of regularization. The aim of this study is to show the contribution of se¬ mantic features in spotting bots when combined with basic features, tweet similarity and n-gram features obtained from users' tweets.
31   Basic Features
Basic features mostly comprise those that are extracted from users' metadata and some from tweets.  For a given user u , we extract 9 basic features as follow: number of fol- lowers (number of users that follow u); number of following (number of users that u follows); ratio between the num- ber of followers and the sum of the number of followers and following; number of tweets authored by u; age of u's ac- count in years; length of u's profile description; length of u's screen name; average number of URL and average number of hashtags (both relatively to the number of analyzed tweets). Fig. 4 shows the distribution of the number of followings relative to URL average (a), and relative to the length of profile description (b) from honeypot dataset (see Section 4.1). We can observe that most genuine accounts, those be¬ longing to real people, followed less accounts and have URL average below 2, in contrast, bots have higher number of fol- lowings. Features extracted from users' meta-data have been successfully applied to classify users on Twitter [9; 36; 37]. However, advanced bots can camouflage human-like meta- data information which decrease the effectiveness of these features. Thus, we combine basic features with much more sophisticated features to counter bots' evasive tactics.
32   Word Embedding
Word embedding models enable us to represent a text in a dense vector of real numbers, one per word in a vocab- ulary.  On the contrary to vector space models [33], word embedding are built on the notion of distributional hypoth- esis [16], which assumes that words that are used and occur in the same context are semantically related to each other. Thus, word embedding entails efficiently encoding seman- tic information of a text or document. Therefore, we take X ′ u = x 1 + x 2 + ...x n as a concatenation of n tweets of a given user u and, we produce word embedding of 300 dimension that capture semantic features from that a user's content.
33   Pairwise Tweet  Similarity
A recent study [10] has shown that content from pure gen- uine users tend to be very dissimilar in average compared to content produced by automatons. Since we assume that bots generate tweets with similar structure and minor modi- fications from one tweet to another, we design the feature to capture the tweet similarity based on that assumption. We introduce three features that enable us to quantify the degree of similarity of a certain set of pairs of tweets from a given user u : mean of pairwise tweet similarity ( μ u ), stan- dard deviation of pairwise similarity ( δ u ) and coefficient of variation (cvu) of pairwise tweet similarity.  Being [x] the number of character in a tweet x, m the number of matching characters between two tweets ( x 1 , x 2 ) and t r the number of transpositions needed to change one tweet to another. And based on Jaro Similarity [20], the similarity between a pair of tweets, Sim(x1,x2), is given by: ifm=0
otherwise The mean μ u of all pairwise tweet similarities on a sample of n tweets of a given user u is calculated by:
Being δ ω the standard deviation of pairwise tweet similarity on a sample of n tweets of a given user u , where:
Then, the coefficient of variation of pairwise tweet similarity cv u of a given user u is calculated as cv u = δ u/μ u. Fig. 4(c) and (d) illustrate the distribution of the three similarity features. In general bot accounts have very simi- lar content compared to human accounts, but with also high standard deviation. However, similar to Fig. 4(a) and (b), Fig. 4(c) and (d) show that some bots are clustered together or close to genuine accounts. Therefore, similarity or basic features alone are not enough to separate bot accounts from genuine accounts in some instances.
3 4   n-gram Lengths
n-gram ( G n ) is a contiguous sequence of n items from a given sample of text. n-gram models are used over a broad range of tasks in Natural Language Processing such as text categorization [7], machine translation [26; 15], and speech recognition [18]. In social bot detection task, [37] used a combination of tf (term frequency) and tf-idf (term frequency times inverse document frequency) of unigram (1-gram), bi- gram (2-gram), and tri-gram (3-gram). They achieved their best results by combining user features (e.g, length of profile name, length of profile description, etc) with n-gram features which showed to be very effective. However, computing tf and tf-idf can be computationally expensive on large dataset. Therefore, we take a different approach by computing several statistics from the number of characters in n-grams (length of n-gram). We use word-based n-gram, e.g., for the given sentence "<user> it should be a good time !", we compute the fol- lowing: 1-gram ( G 1 ) ={ "<user>", "it", "should", "be", “a”, “good”, “time”, “!”}; 2-gram ( G 2 ) = { “<user> it”, “should be”, “be a”, “a good”, “good time”, "it should", Being |G n | = {len(g1),len(g2),...,len(gk) } a "time !"}. sequence of lengths of n-gram of class n (we compute n- gram of n = 1 up to n = 5) from a given user's tweets. As for 1-gram ( G 1 ) of the above example, it would be | G 1 | = From |G"| the following five statis- {6, 2, 6, 2, 1, 4, 4, 1}. tics are determined: median(|G n |), mean(|G n |), std(|G n |), kurtosis(|G n |) and skewness(|G n |), totaling 25 features (five statistics for |G 1 | to |G 5 |).  Fig. 4(e) and (f) shows the distribution of some ngram features.  Human accounts are clustered in the middle while bot accounts are spread.
prec and rec indicate precision and recall, respectively.
Experiments
4
We empirically investigated the proposed model by using two standard datasets, which were often utilized in previous works, and we also tested with a mixed dataset. This section initially describes the datasets used to train and evaluate our models. We then give implementation details of our models and the baseline models that we compared with. At last, we show the results followed by features importance and error analysis.
41 Dataset
We evaluate our models with two publicly available datasets ( DATAHP and DATAV AROL ) and the combina- tion of them DATAmix , respectively. DATAHP: this dataset is from the social honeypot . experiment [24]. It consists of 22K content polluters (bots) with over 2 million tweets and 19K legitimate users with over 3 million tweets. For our study, we randomly selected 7K bots and 3K genuine users, and 200 tweets for each user. . DATAVAROL: this is a manually labeled dataset as in [36]. It consists of about 2500 binary labeled twitter ac¬ counts (0 for genuine or human-controlled accounts and 1 for bot-controlled accounts). We ran our crawler in January 2018 and collected 200 tweets from each user's timeline to reduce the crawling time. Note that 200 tweets is the maxi- mum number of tweets per request under the standard Twit- ter API ( #2 ) as of January 2018. Since some accounts were deleted or had changed their privacy settings from public to private as of time of crawling, the number of successfully crawled accounts reduced to about 2000 (about 600 bots and 1400 humans). DATA MIX : we combined together the two datasets . ( DATAHP and DATAv AROL ), resulting in the dataset with about 12K users. We ran a series of tweet anonymization process where we substituted all mentions and links with tokens juserj and jlink ¿ from all tweets, respectively.  Using available li- brary (#3) , we removed all non-English tweets, and only ac- counts that remained with more than 20 messages were con- sidered for further analysis. For additional data sanitization, the cleaning method from [22] was applied to the datasets. 42 Baseline To evaluate the proposed models, we compared our results with three baseline systems. BL V AROL (Varol et al., 2017) [36]: a content-based . framework that leverages more than a thousand features dis- tributed in 6 categories: I. user-based features - extracted from user metadata (e.g, screen name length, number of dig- its in screen name, accounts age, etc); II. friend-based fea- tures - extracted from language use, local time, popularity, etc; III. network-based features - extracted from user's net- work structure; IV. temporal features - extracted from user activity, such as average rates of tweet production over var- ious time periods and distribution of time intervals between tweets; V. Content and language features - extracted from tweet text, mostly syntactic features such as POS tagging usage, statistics about number of words in a tweet, etc. VI. sentiment features - obtained from measurement of mood and emotions conveyed in the tweets. They trained a set of machine learning schemes with subset of 100 features and reported that random forest yielded the best result. . BL CLARK (Clark et al., 2016) [10]:  a machine- learning approach based on natural language text features to provide the base for identifying non-genuine accounts. They used three content-based features: I. average pairwise tweet dissimilarity; II. word introduction rate decay parameter and average number of URLs per tweets; . BLW ANG (Wang et al., 2015) [37]: a machine learn- ing approach that focuses on optimizing the amount of data that needs to be gathered by relying only on tweet-inherent features. They applied three feature categories: I. user fea- tures - similar of those used in BLv arol ; II. content fea- tures including n-gram based features such as tf (term fre- quency) and tf-idf (term frequency times inverse document frequency) of unigram (1-gram), bi-gram (2-gram), and tri- gram (3-gram).  III. sentiment features such as automatically and manually created sentiment lexicons from the tweet text. 43  Implementation Setup Our model is implemented in tensorflow on top of publicly available(#4) implementation of CNN for sentence classifica- tion as presented by [22]. 4 3.1  Models' settings To initialize our word embedding of dimension 300, word2vec pre-trained embeddings [28] were used. For convo- lutions, we set the number of filters to 128 for each filter-size of 3, 4, and 5. We applied 'Dropout' to the input to the penultimate layer with probability of 0.5.  Optimization is performed using stochastic gradient (SGD) with initial learn- ing rate 0.005 and 0.0001 for early stopping. 4 3. 2  Models' variants Four different variations of the proposed model, start- ing from one layer CNN with word embeddings [22] com- bined with basic features (BF + WE model) were imple- mented.  To the BF + WE model either/both pairwise tweet similarity (SIM) (explained in Section 3.3) or/and n- gram lengths (NG) (described in Section 3.4), creating BF + WE + SIM model, BF + WE + NG model, and BF + WE + SIM + NG model , respectively. It took up to 3 days to finish a 5-fold cross validation. Table 2: McNemar chi-squared test with Yates correction of 1.0 of the BF + WE model against: BF + WE + SIM model (a); BF + WE + NG model (b); BA + WE + SIM + NG model (c). On about 3.5K users assembled from original honeypot dataset. Both models were trained with DATAHP-

44 Results
Table 1 illustrates the results of our models' assessments through 5-fold cross validation. DATAHP and DATAMIX : The BF + WE model, which is just one layer CNN with pre-trained word embeddings from users' timeline tweets plus basic features, performs well on DATAHP and DATAMIX compared to the baseline sys- As we expected, adding more features (SIM, NG, tems. and SIM + NG) to the BF + WE model improves signif- icantly performance over the three datasets. This suggests that these features have comparatively low correlation among them.  We achieved our best results on all datasets when com- bining all features, which produces BF + WE + SIM + NG model (see fig. 2). DATAV arol : Despite not achieving almost-ceiling re- sult like on DATAHP , our models performed fairly well on DAT Avarol too, yielding a state of the art result as well. It is important to state that this dataset is more recent com¬ pared to DATAHP and possibly it contains more advanced bots. In addition, it has only 2K users and of those 600 are bots which might not have been enough data to train our models to understand underlying differences between bots and human users.  In summary, all our models performed well in all datasets, and the model that combines all features outperformed the three baselines.
45   Feature Contribution
In order to understand better the improvement of perfor- mance over the BF + WE model when adding more features (see Section 4.4), we employed McNemar test for paired nom- inal data [27]. This test is appropriate for binary classifica- tion tasks. Since we compare the results of the algorithms when applied on the same datasets. We assembled new data of about 3.5K users from original Honeypot dataset (see Section 4.1) completely exclusive with DATAHP. We next tested all models with this dataset and compared the models' outcomes to those of the BF + WE model. Using McNemar chi-squared test with one degree of freedom under the null hypothesis (p-value = 0.05) that the models have a negligible decrease of error rate, i.e., it would be determined that there is no significant performance im- provement from the BF + WE model if the p-value from the test is equal or greater than that of the null hypothesis. Table 2(a) shows the McNemar chi-squared test with Yates correction of 1.0 of the BF + WE model against the BF + WE + SIM model. The p-value of the test is equal to 0.009 (less than the p-value associated with the null hypothesis) which proves that adding pairwise tweet similarity features, indeed, we gain an evident performance improvement. Table 2(b) presents the p-value of the test against the BF + WE + NG model is equal to 0.0124, suggesting the significance to consider pairwise n-grams lengths features with the BF + WE model. Similarly, we can interpret the result of the BF + WE + SIM + NG model as shown in Table 2 (c). Analogous to what we have observed in Section 4.4, Table 2(c) shows that the greatest performance improvement from the BF + WE model is gained when applying the BF + WE + SIM + NG model which combined all the features, and produced the lowest p-value among all the three McNemar tests.
46   Error  Analysis
we conducted some empirical analysis in order to gain more insights on our model outputs.  As stated earlier, in recent years social bots have become sophisticated enough to gener- ate human-like content [14]. Thus, discriminating bots from human users is not a straightforward task. However, from the analysis of our models' inputs-outputs we observed that in general our model performed well even in a presence of non-obvious bots.  Table  3 shows a sample of our model output on two datasets, DATAvarol and DATAHP . 4 6. 1   False Positive Our models failed to correctly classify users labeled as hu- man but exhibited automated behaviour, bot-like behaviour. Some of these accounts belonged to human users but most of their content were generated by connected applications such as Spotify or Youtube. We also observed cases of miss label-
ing on the dataset, e.g., user u4 on table  3.  Accounts labeled as human/bot, although double check of their content and profile revealed that they are more likely to belong to the opposite label. 4 6. 2   False Negative Similar to false positive, our models also triggered false negative for users labeled as bot and yet, a checking on their tweets, meta-data and overall activity showed that these ac¬ counts might be human accounts(e.g. user u 1 on table  3).
Conclusions
5
This paper proposed an approach to classify Twitter users into social bots and human users with a CNN model con- sidering features obtained from their texts and metadata. Given a twitter user's tweets the model captures the seman- tic features through a pre-trained word embedding, find the content similarity by pairwise tweets comparison, statistics about lengths of various class of n-gram and extract features from the user's metadata. Our results shows that pairwise tweet similarity and n-gram features when combined with se- mantic features improve performance over the basic + word embedding model.  Our model outperformed the baseline sys- tem, yielding state-of-the-art results three datasets. In future work, we plan to consider more users information such as network, the temporal pattern of content generation, emotions or mood conveyed by the user's content. We also plan to create a large new dataset containing more recent bots.
Acknowledgement
This work was partly supported by JSPS KAKENHI Grant Number JP16K16057.