DBSJ Journal
Vol. 18, Article No. 1, March 2020

Regular Paper

Making Twitter Safer: UncoveringSocial-Bot on Twitter through User’s
Content Approach
Nigo Sumaila1

Shoko Wakamiya2

Eiji Aramaki3
Several studies have shown that social bots might have impacts
on various important areas of our society such as influencing the outcome of elections, the economy, or creating panic in time of crisis.
Consequently, there is a growing need to develop and strengthen
defense mechanism against these entities. So far, existing methods rely on users’ global information, such as profile information,
network-related, and from the text content only syntactic information
have been used. In this paper, we propose a defense mechanism
against social bots on Twitter, a neural network model that incorporates metadata features and semantic information, pairwise tweet
similarities and n-gram features from a user’s content to differentiate genuine users from social bots. Our model outperforms baseline
systems in three publicly available datasets.

1

Fig. 1: Overview of the proposed method against previous methods basedon many complex features. Some of the features require much time toobtain. Especially network and friend features are inconsistent. In contrast,
the proposed method focused mainly on text-oriented features: semantic
and content features, which are stable and easy to obtain.

Introduction

A recent study [3] suggests that social bots (social media accounts
controlled algorithmically by software to generate content and interact with other users, automatically) may have played an essential
role in the 2016 U.S presidential election. An analysis of electionrelated tweets showed that bots produced near one-fifth of the entire
conversation. Earlier, during the 2010 U.S midterm election, socialbots were used to support some candidates and defame their opponents [30]. These kinds of incidents are not particular only to U.Selections’ arena; similar patterns have also been observed in othercountries as well [13, 18, 31]. Outside of politics, there are also
several reported impacts of social bots. For instance, bots have been
associated with causing panic during a time of crisis by disseminating false and unverified information [35]; affecting the stock market
by generating a vivid discussion about a company which created anapparent public interest [14].
As a result of the widespread use of social media services, several
studies concerned with correlating real-life events with observations
made from social media data have been published. From detecting
earthquakes [12,32], election outcomes [20,25] to disease outbreaks[2, 8, 34]. Additionally, [36] estimation places social bot population
on Twitter between 9% to 15% of the total active users. Therefore,
contents generated by bots might undermine useful predictions madefrom SNS data. Consequently, there is a growing need to develop
and strengthen defences against these entities. However, detecting
bot is one of the most daunting tasks in social media analysis [13].
The challenge is understanding what advanced bot can do. Bot
have evolved from easily detectable ones which perform mainly one
type of activity, such as posting or re-tweeting automatically, toincreasingly sophisticated ones. Now the boundary between what
constitutes human-like and bot-like behaviour has become fuzzier[14].
Most of the previous work on bot detection fall into two perspectives. First, network-based methods in which the decision is drawn
from the users’ network analysis, with the assumption that social bots
tend to be mostly connected to other bots [6, 39]. However, recent
works [4, 29] show that advanced social bots may successfully beconnected to humans, making them impossible to be detected solely
on network-based assessments. Second, machine learning schemes
are trained with language-based, and other features (extracted fromusers’ tweets and metadata) to capture users’ behavioural patterns
to differentiate real users from social bots. However, languagebased features used in previous work were mostly syntactic inherent
features, and there is still much to be explored in semantic features.
This paper proposes a defence mechanism against social bots onsocial network service, Twitter, that leverages nuances in semantic
features from a user’s timeline tweets combined with tweet similarity, n-gram features and metadata-based features. Fig. 1 illustrates an
overview of the proposed method against previous methods based on
many sophisticated features. Some of the features require much time
to obtain. Especially network and friend features are inconsistent.
In contrast, the proposed method focused mainly on text-oriented
features: semantic and content features, which are stable and easy toobtain, combined with users’ metadata features. The contributionsof the paper are summarized as follows:• We propose a bot detection focused on semantic and content
features, which are simple statistics-based features to overhead
computational complexity.
• We construct CNN (Convolutional Neural Network) models
that incorporate semantic features together with tweet similarity, n-gram features and metadata-based features.
• The experiments conducted with publicly available datasetsshow that the proposed model outperforms baseline models.

1 Non member

Nara Institute of Science and Technology
nsumaila(at)gmail.com
2 Regular member Nara Institute of Science and Technology
wakamiya(at)is.naist.jp
3 Regular member Nara Institute of Science and Technology
aramaki(at)is.naist.jp

1

DBSJ Journal
Vol. 18, Article No. 1, March 2020

Regular Paper

2

Related Work

2.1

Bot Detection

Since the early days of social media, several studies on automatic
identification of non-genuine users on social media network havebeen conducted [11,39]. Most of the work done in this area fall into
two categories: network-based and feature-based decision.
On network-based bot detection, the decision is based on social
graphs analysis. For example, Sybil *1 Guard [39] assumes that nonSybil region, region bounding human-to-human-established trust
relation, is fast mixing and Sybil admission to the region is basedon admission control protocol. SybilRank [6] assumes that theconnection between users is based upon the trustworthiness among
them. Consequently, Sybil accounts show less connectivity to realusers and more to other Sybils to appear trustworthy. However,sophisticated bots might gain the trust of real users [4, 29], and to
be able to penetrate their communities, making them impossible
to be spotted solely by network-based assessments through trust
assumption [14]. Therefore, other aspects should be considered.
On feature-based detections, machine learning schemes aretrained to capture users’ behavioral patterns and metadata in order to differentiate real users from social bots. Wang et al. [37] andAlarifi et al. [1] focus on the detection of spam tweets, which optimizes the amount of data that needs to be gathered by relying only
on tweet-inherent features. Varol et al. [36] leverage more than one
thousand features distributed in six categories (user-based, friend,
network, temporal, content, and sentiment features) obtained fromusers’ metadata, mentions and timeline tweets. Clark et al. [10]shows that natural language text features are effective at separating
real users from social bots. However, content and language features
extracted from tweets text in the previous works [10, 36, 37] were
mostly syntactic inherent features.
2.2

Fig. 2: Visual illustration of the proposed modela sequence of 𝑛 tweets 𝑋𝑢 from a given user 𝑢, we apply it in four
different parts. First, basic features are computed (Section 3.1);
second, we compute word embedding from the concatenation of all
tweets (Section 3.2), then pairwise tweet similarity from all tweets(Section 3.3), and lastly, we compute several statistics about lengthsof different classes of n-gram (Section 3.4).
The word embedding layer is followed by a convolution layer
comprised of several filters of different sizes (3, 4 and 5), but withthe same width as the dimension of the embedding vectors. Everyfilter performs convolution over the embedding matrix, producing afeature map. Next, we run the feature map through an element-wise
non-linear transformation, using Rectified Linear Unit (ReLU) [23].
Then, applying 1-max pooling [5], the maximum value is extracted
from each feature map. All extracted values from feature map
are concatenated together with features from basic, pairwise tweet
similarity, and n-grams lengths. We then feed all features to a fullconnected SoftMax layer for final output. Dropout [22] is used as amean of regularization.
This study aims to show the contribution of semantic features inspotting bots when combined with basic features, tweet similarity,and n-gram features obtained from users’ tweets.

Chat BotIn Natural Language Processing (NLP) context, the main goal is torealize natural conversation instead of bot detection. Most current
bot engines are designed to reply to user utterances based on existingutterance-response pairs [38]. In this framework, to capture therelevance between utterance and response is fundamental. Then,the relevance is calculated by the framework’s ability to retrieve
the most relevant pairs from the conversation database (retrieval
approach) or generate the response to the given utterance (generative
approach). Although mostly the target relevance is limited to theshort scope, usually, a single utterance-response pair, the proposed
method handles another broader significance of the entire tweets(Section 3.2).

3

3.1

Basic Features

Basic features mostly comprise those that are extracted from users’
metadata and some from tweets. For a given user 𝑢, we extract 9
basic features as follow: number of followers (users that follow 𝑢);number of following (users that 𝑢 follows); ratio between the numberof followers and the sum of the number of followers and following;number of tweets authored by 𝑢; age of 𝑢’s account in years; length of𝑢’s profile description; length of 𝑢’s screen name; average number of
URL and average number of hashtags (both relatively to the numberof analyzed tweets). Fig. 3 shows the distribution of the numberof followings relative to URL average (a) and relative to the lengthof profile description (b) from honeypot dataset (see Section 4.1).
We can observe that most genuine accounts, those belonging toreal people, followed fewer accounts and have URL average below
2, in contrast, bots have a higher number of followings. Features
extracted from users’ metadata have been successfully applied toclassify users on Twitter [9, 36, 37]. However, advanced bots cancamouflage human-like metadata information which decreases theModel

We formulate the task of automatically identifying non-genuineusers on social media as follows: given a user 𝑢’s posts (timeline tweets) 𝑋𝑢 = {𝑥1 , 𝑥2 , ..., 𝑥 𝑛 }, our goal is to find 𝑌 → {0, 1} ={ℎ𝑢𝑚𝑎𝑛, 𝑏𝑜𝑡} for the user such that:𝑌 ∗ = arg max 𝑃(𝑌 |𝑋𝑢 )𝑌

where 𝑃 is the conditional probability of 𝑌 given 𝑋𝑢 .
Representation of our model is illustrated in Fig. 2. This model
is a slight alteration of a shallow CNN for sentence classification
tasks [21, 40] to accommodate our features; basic feature, wordembedding, pairwise tweet similarity, and n-gram lengths. Given*1 Social bots are also known as Sybils*1 We interchangeably use human and genuine to refer real users as opposed tosocial bots.

2

DBSJ Journal
Vol. 18, Article No. 1, March 2020

Regular Paper

effectiveness of these features. Thus, we combine basic featureswith much more sophisticated features to counter bots’ deceptive
tactics.
3.2

Word Embedding

Word embedding models enable us to represent a text in a dense
vector of real numbers, one per word in the vocabulary. On thecontrary to vector space models [33], word embedding is built on
the notion of distributional hypothesis [16], which assumes thatwords that are used and occur in the same context are semanticallyrelated to each other. Thus, word embedding entails efficiently
encoding semantic information of a text or document. Therefore,
we take 𝑋𝑢0 = 𝑥 1 + 𝑥2 + ...𝑥 𝑛 as a concatenation of n tweets of a given
user 𝑢 and produce word embedding of 300 dimension that capture
semantic features from the user’s content.
3.3(a) URL average vs following

Pairwise Tweet Similarity

A recent study [10] has shown that content from pure genuine users
tends to be very dissimilar on average compared to content producedby automatons. Since we assume that bots generate tweets withsimilar structure and minor modifications from one tweet to another,
we design the feature to capture the tweet similarity based on thatassumption.
We introduce three features that enable us to quantify the degreeof similarity of a particular set of pairs of tweets from a given user
𝑢: mean of pairwise tweet similarity (𝜇𝑢 ), standard deviation of
pairwise tweet similarity (𝛿𝑢 ), and coefficient of variation (𝑐𝑣 𝑢 ) of
pairwise tweet similarity. Furthermore, the Jaro Similarity metric [19] computes the edit distance to quantify how two sentences
are dissimilar by counting the minimum number of transposition
needed to transform one sentence into the other. Therefore, we apply the metric to compute the similarity between a pair of tweets,
𝑆𝑖𝑚(𝑥1 , 𝑥2 ), as follow:
0

𝑆𝑖𝑚(𝑥1 , 𝑥2 ) = { 1

𝑚−𝑡𝑟
𝑚
𝑚
3 ( |𝑥1 | + |𝑥2 | + 𝑚 )(c) Std vs coefficient of variation of (d) Std vs mean of tweet similarity
tweet similarity(e) Std of 1-gram length vs median of 3-gram length𝑖 𝑓 𝑚=0𝑜𝑡 ℎ𝑒𝑟 𝑤𝑖𝑠𝑒features from a sample of 𝐷 𝐴𝑇 𝐴 𝐻 𝑃

and speech recognition [17]. In social bot detection task, Wanget al. [37] used a combination of term frequency (tf) and term
frequency times inverse document frequency (tf-idf) of unigram (1gram), bi-gram (2-gram), and tri-gram (3-gram). They achieved
their best results by combining user features (e.g, length of profile
name, length of profile description, etc) with n-gram features whichshowed to be very effective. However, computing tf and tf-idf can becomputationally expensive on a large dataset. Therefore, we take adifferent approach by computing several statistics from the numberof characters in n-grams (length of n-gram).
We use word-based n-gram, e.g., for the given sentence “<user>
it should be a good time !”, we compute the following: 1-gram(𝐺 1 ) ={ “<user>”, “it”, “should”, “be”, “a”, “good”, “time”,“!”}; 2-gram (𝐺 2 ) = {“<user> it”, “it should”, “should be”,
“be a”, “a good”, “good time”, “time !”}. Being |𝐺 𝑛 | ={𝑙𝑒𝑛(𝑔1 ), 𝑙𝑒𝑛(𝑔2 ), ..., 𝑙𝑒𝑛(𝑔 𝑘 )} a sequence of lengths of n-gramof class 𝑛 (we compute 𝑛-gram of 𝑛 = 1 up to 𝑛 = 5) from agiven user’s tweets. As for 1-gram (𝐺 1 ) of the above example, itwould be |𝐺 1 | = {6, 2, 6, 2, 1, 4, 4, 1}. From |𝐺 𝑛 | the following fivestatistics are determined: 𝑚𝑒𝑑𝑖𝑎𝑛(|𝐺 𝑛 |), 𝑚𝑒𝑎𝑛(|𝐺 𝑛 |), 𝑠𝑡𝑑 (|𝐺 𝑛 |),𝑘𝑢𝑟𝑡𝑜𝑠𝑖𝑠(|𝐺 𝑛 |), and 𝑠𝑘𝑒𝑤𝑛𝑒𝑠𝑠(|𝐺 𝑛 |), totaling 25 features (fivestatistics for |𝐺 1 | to |𝐺 5 |). Fig. 3(e) and (f) show the distribution of some n-gram features. Human accounts are clustered in the
middle while bot accounts are spread.

Õ
2
𝜇𝑢 =
𝑆𝑖𝑚(𝑥𝑖 , 𝑥 𝑗 )𝑛(𝑛 − 1) 𝑥 ,𝑥 ∈𝑋𝑖𝑗𝑢

Being 𝛿𝑢 the standard deviation of pairwise tweet similarity on asample of 𝑛 tweets of a given user 𝑢, where:𝛿𝑢 =(f) Std of 1-gram length vs median of
2-gram length

Fig. 3: Distribution of some basic, pairwise tweet similarity, and n-gramBeing |𝑥| the number of characters in a tweet 𝑥, m the number ofmatching characters between two tweets (𝑥 1 , 𝑥2 ) and 𝑡𝑟 the numberof transpositions needed to change one tweet to another.
The mean 𝜇𝑢 of all pairwise tweet similarities on a sample of n
tweets of a given user 𝑢 is calculated by:s Í
2 𝑥𝑖 ,𝑥 𝑗 ∈𝑋𝑢 (𝑆𝑖𝑚(𝑥 𝑖 , 𝑥 𝑗 ) − 𝜇) 2𝑛(𝑛 − 1)

Then, the coefficient of variation of pairwise tweet similarity 𝑐𝑣 𝑢 ofa given user 𝑢 is calculated as 𝑐𝑣 𝑢 = 𝛿𝑢 /𝜇𝑢 .
Fig. 3(c) and (d) illustrate the distribution of the three similarity
features. In general bot accounts have very similar content comparedto human accounts, but with also high standard deviation. However,similar to Fig. 3(a) and (b), Fig. 3(c) and (d) show that some bots are
clustered together or close to genuine accounts. Therefore, pairwise
tweet similarity or basic features alone are not enough to separate
bot accounts from genuine accounts in some instances.
3.4(b) Profile length vs following

n-gram Lengths

n-gram (𝐺 𝑛 ) is a contiguous sequence of n items from a given
sample of text. n-gram models are used over a broad range of tasks
in NLP such as text categorization [7], machine translation [15, 26],
3

DBSJ Journal
Vol. 18, Article No. 1, March 2020

Regular Paper

4

Table 1: Performance of our models on different datasets

Experiments

Model
𝐵𝐿 𝑉 𝐴𝑅𝑂𝐿 [36]𝐵𝐿 𝐶 𝐿 𝐴𝑅𝐾 [10]𝐵𝐿 𝑊 𝐴𝑁 𝐺 [37]We empirically investigated the proposed models by using two standard datasets from previous works and also tested with a mixed
dataset. This chapter initially describes the datasets used to train
and evaluate our models. We then give implementation details of
our models and baseline models that we compared against our models. At last, we show the results followed by feature importance anderror analysis.
4.1

𝐷 𝐴𝑇 𝐴 𝐻 𝑃
0.960
0.960
0.940 prec
0.940 rec𝐷 𝐴𝑇 𝐴 𝑀 𝐼 𝑋
0.940
-

0.873
0.889
0.884
0.914

0.984
0.986
0.985
0.988

0.954
0.956
0.954
0.960

BF+WE
BF+WE+SIM
BF+WE+NG
BF+WE+SIM+NG

DatasetWe evaluated our models with two publicly available datasets(𝐷 𝐴𝑇 𝐴 𝐻 𝑃 and 𝐷 𝐴𝑇 𝐴𝑉 𝐴𝑅𝑂𝐿 ) and the combination of them
𝐷 𝐴𝑇 𝐴 𝑀 𝐼 𝑋 , respectively.

- values are not available.
Values in AUC except when prec and rec used.
prec and rec are precision and recall, respectively.

• 𝐷 𝐴𝑇 𝐴 𝐻 𝑃 : This dataset is from the social honeypot experiment[24]. It consists of 22000 content polluters (bots) with over 2
million tweets and 19K legitimate users with over 3 million
tweets. For our study, we randomly selected 7000 bots and3000 genuine users, and 200 tweets for each user.
• 𝐷 𝐴𝑇 𝐴𝑉 𝐴𝑅𝑂𝐿 : This is a manually labeled dataset as in [36].
It consists of about 2500 binary labeled twitter accounts (0 for
genuine or human-controlled accounts and 1 for bot-controlled
accounts). We ran our crawler in January 2018 and collected
200 tweets from each user’s timeline to reduce the crawling
time. Note that 200 tweets is the maximum number of tweets
per request under the standard Twitter API *2 as of January
2018. Since some accounts were deleted or had changed theirprivacy settings from public to private as of time of crawling,
the number of successfully crawled accounts reduced to about
2000 (about 600 bots and 1400 humans).
• 𝐷 𝐴𝑇 𝐴 𝑀 𝐼 𝑋 :We combined together the two datasets(𝐷 𝐴𝑇 𝐴 𝐻 𝑃 and 𝐷 𝐴𝑇 𝐴𝑉 𝐴𝑅𝑂𝐿 ), resulting in the dataset with
about 12000 users.

Table 2: McNemar chi-squared test with Yates correction of 1.0 of
the BF + WE model against: BF + WE + SIM model (a); BF + WE
+ NG model (b); BA + WE + SIM + NG model (c). On about 3.5K
users assembled from the original honeypot dataset. Both models
were trained with 𝐷 𝐴𝑇 𝐴 𝐻 𝑃 .
(a) BF + WE + SIM model

BF + WE + SIM

Correct
Incorrect

BF + WE
Correct Incorrect
3125
68
40
194
p = 0.0090

Correct
Incorrect

BF + WE
Correct Incorrect
3124
68
41
194
p = 0.0124(b) BF + WE + NG model

BF + WE + NG

We ran a series of tweet anonymization process where we substituted all mentions and links with tokens <user> and <link> from all
tweets, respectively. Using an available library *3, all non-English
tweets were removed and only accounts that remained with morethan 20 messages were considered for further analysis. For additional data sanitization, the cleaning method from [21] was applied
to the datasets.
4.2

𝐷 𝐴𝑇 𝐴𝑉 𝐴𝑅𝑂𝐿
0.890-(c) BF + WE + SIM + NG model

BF + WE + SIM + NG

Baseline

To evaluate the proposed models, we compared our results withthree baseline systems as follows.
• 𝐵𝐿 𝑉 𝐴𝑅𝑂𝐿 (Varol et al., 2017) [36]: A content-based framework that leverages more than a thousand features distributed in
6 categories: I. user-based features - extracted from user metadata (e.g, screen name length, number of digits in screen name,accounts age, etc); II. friend-based features - obtained fromlanguage use, local time, popularity, etc; III. network-based
features - extracted from user’s network structure; IV. temporalfeatures - derived from user activity, such as average rates oftweet production over various periods and distribution of timeintervals between tweets; V. content and language features - extracted from tweet text, mostly syntactic features such as POS
tagging usage, statistics about the number of words in a tweet,
etc. and VI. sentiment features - obtained from measurementof mood and emotions conveyed in the tweets. They trained aset of machine learning schemes with a subset of 100 featuresand reported that random forest yielded the best result.

Correct
Incorrect

BF + WE
Correct Incorrect
3126
68
39
194
p = 0.0065• 𝐵𝐿 𝐶 𝐿 𝐴𝑅𝐾 (Clark et al., 2016) [10]: A machine learning approach based on natural language text features to provide thebase for identifying non-genuine accounts. They used threecontent-based features: I. average pairwise tweet dissimilarity; II. word introduction rate decay parameter and III. average
number of URLs per tweets;• 𝐵𝐿 𝑊 𝐴𝑁 𝐺 (Wang et al., 2015) [37]: A machine learning approach that focuses on optimizing the amount of data that needs
to be gathered by relying only on tweet-inherent features. They
applied three feature categories: I. user features - similar of
those used in 𝐵𝐿 𝑉 𝐴𝑅𝑂𝐿 ; II. content features including n-grambased features such as term frequency (tf) and term frequency
times inverse document frequency (tf-idf) of unigram (1-gram),bi-gram (2-gram), and tri-gram (3-gram), and III. sentimentfeatures such as automatically and manually created sentimentlexicons from the tweet text.

*2 https://developer.twitter.com/en/docs.html*3 https://github.com/saffsd/langid.py

4

DBSJ Journal
Vol. 18, Article No. 1, March 2020

Regular Paper

Table 3: Excerpt of users’ tweets with their respective predicted and gold standard (denoted Gold) labels. The first three users are from𝐷 𝐴𝑇 𝐴 𝐻 𝑃 and the latter three users are from 𝐷 𝐴𝑇 𝐴𝑉 𝐴𝑅𝑂𝐿 .
User ID
𝑢1
𝑢2
𝑢3

𝑢4
𝑢5
𝑢6

4.3

Tweet ID
𝑡 11
𝑡 12
𝑡 21
𝑡 22
𝑡 31
𝑡 32
𝑡 41
𝑡 42
𝑡 51
𝑡 52
𝑡 61
𝑡 62

Tweetswe do not plant knowledge when young, it will give us no shade
he was posting him up user user the cavs didn’t have anybody that could help lebron
why can’t you just be straightforward ? <USER> find joy in the ordinary
i have so many thoughts <URL> firstly, <USER> is great wish he could do all ofhello <USER> we have arrived ! ! ! ! <USER> friday fun fact of the day a ford truck
is sold every 41 seconds ford150 <USER> the new chrysler
right amp center last spring concert so bittersweet qingatw if you can do high school
RT <USER>: #NowPlaying on #Spotify <USER> “One Night" <URL>...
RT <USER>: #NowPlaying on #Spotify Ayron Michael “One Night" <URL> <URL>...
<USER> why is dengue spray used in presence of students in schoolafer attock students at jehlum fainted due to effect of den <USER>
i finally get a saturday morning to sleep in and i’m awake at 8 am
<USER> mike i can’t believe it sully oh mike mike i’m on a t shirt

4.5

Models’ Settings

Models’ Variants

Four different variations of the proposed models, starting from one
layer CNN with word embeddings [21] combined with basic features(BF + WE model) were implemented. To the BF + WE model
either/both pairwise tweet similarity (SIM) (explained in Section3.3) or/and n-gram lengths (NG) (described in Section 3.4), creating
BF + WE + SIM model, BF + WE + NG model, and BF + WE
+ SIM + NG model, respectively. It took up to 3 days to finish a
5-fold cross validation.
4.4

bot

bot

human

human

bot

human

bot

bot

human

human

Feature Contribution

To understand better the improvement of performance over the BF+ WE model when adding more features (see Section 4.4), we employed McNemar test for paired nominal data [27]. This test isappropriate for binary classification tasks. Since we compare theresults of the algorithms when applied to the same datasets.
We assembled new data of about 3500 users from the original
honeypot dataset completely exclusive with 𝐷 𝐴𝑇 𝐴 𝐻 𝑃 . We next
tested all models with this dataset and compared the models’ outcomes to those of the BF + WE model. Using McNemar chi-square
test with one degree of freedom under the null hypothesis (p = 0.05)
that the models have a negligible decrease of error rate, i.e., it would
be determined that there is no significant performance improvementfrom the BF + WE model if the p value from the test is equal or
greater than that of the null hypothesis.
Table 2(a) shows the McNemar chi-square test with Yates correction of 1.0 of the BF + WE model against the BF + WE + SIM
model. The p value of the test is equal to 0.009 (less than the pvalue associated with the null hypothesis), which proves that adding
pairwise tweet similarity features; indeed, we gain evident performance improvement. Table 2(b) presents the p value of the test
against the BF + WE + NG model is equal to 0.0124, suggesting thesignificance to consider pairwise n-grams lengths features with the
BF + WE model. Similarly, we can interpret the result of the BF +WE + SIM + NG model, as shown in Table 2(c).
Analogous to what we have observed in Section 4.4, Table 2(c)shows that the most significant performance improvement from the
BF + WE model is gained when applying the BF + WE + SIM + NG
model which combined all the features, and produced the lowest pvalue among all the three McNemar tests.

To initialize our word embedding of dimension 300, word2vec pretrained embeddings [28] were used. For convolutions, we set the
number of filters to 128 for each filter-size of 3, 4, and 5. We applied‘Dropout’ to the input to the penultimate layer with a probability of
0.5. Optimization is performed using a stochastic gradient (SGD)with an initial learning rate of 0.005 and 0.0001 for early stopping.
4.3.2

Gold
bot

all our models performed well in all datasets, and the model thatcombines all features outperformed the three baselines.

Implementation Setup

Our model is implemented in tensorflow on top of publicly available *4 implementation of CNN for sentence classification as presented by [21].
4.3.1

Predicted
human

Results

Table 1 illustrates the results of our models’ assessments through a
5-fold cross validation.
𝐷 𝐴𝑇 𝐴 𝐻 𝑃 and 𝐷 𝐴𝑇 𝐴 𝑀 𝐼 𝑋 : The BF + WE model, which is justone layer CNN with pre-trained word embeddings from users’ timeline tweets plus basic features, performs well on 𝐷 𝐴𝑇 𝐴 𝐻 𝑃 and
𝐷 𝐴𝑇 𝐴 𝑀 𝐼 𝑋 compared to the baseline systems. As we expected,
adding more features (SIM, NG, and SIM + NG) to the BF + WE
model improves performance significantly over the three datasets.
This suggests that these features have comparatively low correlation
among them. We achieved our best results on all datasets whencombining all features, which produces BF + WE + SIM + NG
model (see Fig. 2).
𝐷 𝐴𝑇 𝐴𝑉 𝐴𝑅𝑂𝐿 : Despite not achieving almost-ceiling result
like on 𝐷 𝐴𝑇 𝐴 𝐻 𝑃 , our models performed reasonably well on
𝐷 𝐴𝑇 𝐴𝑉 𝐴𝑅𝑂𝐿 too, yielding the state-of-the-art result as well. It
is important to state that this dataset is more recent compared to
𝐷 𝐴𝑇 𝐴 𝐻 𝑃 , and possibly it contains more advanced bots. Furthermore, it has only 2000 users and of those 600 are bots which might
not have been enough data to train our models to understand underlying differences between bots and human users. In summary,

4.6

Error Analysis

We conducted some empirical analyses to gain more insights into
our model outputs. As stated earlier, in recent years, social bots havebecome sophisticated enough to generate human-like content [14].
Thus, discriminating bots from human users is not a straightforwardtask. However, from the analysis of our models’ inputs-outputs, weobserved that in general, our models performed well even in thepresence of non-obvious bots. Table 3 shows a sample of our model
output on the two datasets, 𝐷 𝐴𝑇 𝐴𝑉 𝐴𝑅𝑂𝐿 and 𝐷 𝐴𝑇 𝐴 𝐻 𝑃 .

*4 https://github.com/dennybritz/cnn-text-classification-tf

5

DBSJ Journal
Vol. 18, Article No. 1, March 2020

Regular Paper

4.6.1

ley. Using Social Media for Actionable Disease Surveillance andOutbreak Management: A Systematic Literature Review. PLOS ONE,10(10):e0139701, 2015.
[9] Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Emiliano
De Cristofaro, Gianluca Stringhini, and Athena Vakali. Mean Birds:
Detecting Aggression and Bullying on Twitter. In Proceedings of the2017 ACM on Web Science Conference, WebSci ’17, pages 13–22,
2017.
[10] Eric M. Clark, Jake Ryland Williams, Chris A. Jones, Richard A. Galbraith, Christopher M. Danforth, and Peter Sheridan Dodds. Sifting
robotic from organic text: A natural language approach for detecting
automation on Twitter. Journal of Computational Science, 16(Supplement C):1–7, 2016.
[11] George Danezis and Prateek Mittal. SybilInfer: Detecting Sybil Nodes
using Social Networks. In The Network and Distributed System Security Symposium, pages 1–15, 2009.
[12] Paul S. Earle, Daniel C. Bowden, and Michelle Guy. Twitter earthquake detection: Earthquake monitoring in a social world. Annals ofGeophysics, 54(6), 2012.
[13] Emilio Ferrara. Disinformation and Social Bot Operations in the Run
Up to the 2017 French Presidential Election. SSRN Scholarly Paper
ID 2995809, Social Science Research Network, Rochester, NY, 2017.
[14] Emilio Ferrara, Onur Varol, Clayton A. Davis, Filippo Menczer, andAlessandro Flammini. The rise of social bots. Commun. ACM, 59(7),2016.
[15] Nizar Habash. The use of a atructural n-gram language model ingeneration-heavy hybrid machine translation. In Natural Language
Generation, pages 61–69. Springer, 2004.
[16] Zellig S. Harris. Distributional Structure. WORD, 10(2-3):146–162,
1954.
[17] Teemu Hirsimäki, Janne Pylkkönen, and Mikko Kurimo. Importanceof High-Order N-Gram Models in Morph-Based Speech Recognition.
IEEE Trans. Audio, Speech & Language Processing, 17(4):724–732,
2009.
[18] Philip N. Howard and Bence Kollanyi. Bots, #strongerin, and #brexit:
Computational propaganda during the UK-EU referendum. SSRN
Scholarly Paper ID 2798311, Social Science Research Network, 2016.
[19] Matthew A Jaro. Advances in record-linkage methodology as applied
to matching the 1985 census of tampa, florida. Journal of the American
Statistical Association, 84(406):414–420, 1989.
[20] V. Kagan, A. Stevens, and V. Subrahmanian. Using Twitter Sentiment
to Forecast the 2013 Pakistani Election and the 2014 Indian Election.
IEEE Intelligent Systems, 30(1):2–5, 2015.
[21] Yoon Kim. Convolutional Neural Networks for Sentence Classification. arXiv:1408.5882 [cs], August 2014. arXiv: 1408.5882.
[22] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet
classification with deep convolutional neural networks. In Proceedingsof the 25th Advances in Neural Information Processing Systems, pages
1097–1105, 2012.
[23] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet
classification with deep convolutional neural networks. Commun.
ACM, 60(6):84–90, May 2017.
[24] Kyumin Lee, Brian David Eoff, and James Caverlee. Seven months
with the devils: A long-term study of content polluters on twitter. In
Proceedings of the Fifth International AAAI Conference on Weblogs
and Social Media, pages 185–192, 2011.
[25] T. Mahmood, T. Iqbal, F. Amin, W. Lohanna, and A. Mustafa. Mining
Twitter big data to predict 2013 Pakistan election winner. In IEEE
International Multi Topic Conference, pages 49–54, 2013.
[26] José B. Marino, Rafael E. Banchs, Josep M. Crego, Adrià de Gispert, Patrik Lambert, José AR Fonollosa, and Marta R. CostaJussà. N-Gram-Based Machine Translation. Computational linguistics, 32(4):527–549, 2006.
[27] Quinn McNemar. Note on the sampling error of the difference between
correlated proportions or percentages. Psychometrika, 12(2):153–157,
1947.
[28] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff
Dean. Distributed representations of words and phrases and aheir
compositionality. In Proceedings of the 26th International Conference
on Neural Information Processing Systems - Volume 2, pages 3111–
3119, 2013.
[29] A. Paradise, R. Puzis, and A. Shabtai. Anti-Reconnaissance Tools:
Detecting Targeted Socialbots. IEEE Internet Computing, 18(5):11–
19, 2014.
[30] Jacob Ratkiewicz, Michael Conover, Mark R. Meiss, Bruno

False Positive

Our models failed to correctly classify users labelled as human but
exhibited automated behaviour, bot-like behaviour. Some of theseaccounts belonged to human users but most of their content was
generated by connected applications such as Spotify or Youtube.
We also observed cases of miss labelling on the dataset, e.g., user𝑢 4 on Table 3. Accounts labelled as human/bot, although a double
check of their content and profile revealed that they are more likely
to belong to the different label.
4.6.2

False Negative

Similar to false positive, our models also triggered false negative
for users labelled as a bot. Yet, checking on their tweets, metadataand overall activities showed that these accounts might be humanaccounts (e.g. user 𝑢 1 on Table 3).

5 Conclusion
This paper proposed an approach to classify Twitter users into socialbots and human users with a CNN model considering features obtained from their texts and metadata. Given a Twitter user’s tweets,the model captures the semantic features through a pre-trained wordembedding, finds the content similarity by pairwise tweets comparison and statistics about lengths of various class of n-gram, andextracted features from the user’s metadata. Our results showedthat pairwise tweet similarity and n-gram features, when combinedwith semantic features, improve performance over the basic + wordembedding model (BF + WE model). Our model outperformed
the baseline systems, yielding state-of-the-art results on the threedatasets.
In future work, we plan to consider more users information suchas network, the temporal patterns of content generation, emotions
or mood conveyed by the user’s content. We also plan to create avast new dataset containing more new bots.

Acknowledgments
This study was supported in part by JSPS KAKENHI Grant Numbers JP19K20279 and JP19H04221 and Health and Labor Sciences
Research Grant Number H30-shinkougyousei-shitei-004.

References[1] Abdulrahman Alarifi, Mansour Alsaleh, and AbdulMalik Al-Salman.
Twitter turing test: Identifying social machines. Information Sciences,
372(Supplement C):332–346, 2016.
[2] Eiji Aramaki, Sachiko Maskawa, and Mizuki Morita. Twitter Catches
the Flu: Detecting Influenza Epidemics Using Twitter. In Proceedings of the Conference on Empirical Methods in Natural Language
Processing, EMNLP ’11, pages 1568–1576, 2011.
[3] Alessandro Bessi and Emilio Ferrara. Social Bots Distort the 2016
US Presidential Election Online Discussion. SSRN Scholarly Paper
ID 2982233, Social Science Research Network, 2016.
[4] Yazan Boshmaf, Ildar Muslukhov, Konstantin Beznosov, and Matei
Ripeanu. Design and analysis of a social botnet. Computer Networks,
57(2):556–578, 2013.
[5] Y-Lan Boureau, Jean Ponce, and Yann LeCun. A theoretical analysisof feature pooling in visual recognition. In Proceedings of the 27th
International Conference on Machine Learning (ICML-10), pages
111–118, 2010.
[6] Qiang Cao, Michael Sirivianos, Xiaowei Yang, and Tiago Pregueiro.
Aiding the detection of fake accounts in large scale social online
services. In Proceedings of the 9th USENIX conference on Networked
Systems Design and Implementation, page 15. USENIX Association,
2012.
[7] William B. Cavnar and John M. Trenkle. N-Gram-Based Text Categorization. Ann arbor mi, 48113(2):161–175, 1994.
[8] Lauren E. Charles-Smith, Tera L. Reynolds, Mark A. Cameron, Mike
Conway, Eric H. Y. Lau, Jennifer M. Olsen, Julie A. Pavlin, Mika
Shigematsu, Laura C. Streichert, Katie J. Suda, and Courtney D. Cor-

6

DBSJ Journal
Vol. 18, Article No. 1, March 2020

Regular Paper

spective on Preparing for High-Volume Trauma in an Urban Academic
Center. Journal of Orthopaedic Trauma, 29:S7, 2015.
[36] Onur Varol, Emilio Ferrara, Clayton A. Davis, Filippo Menczer,
and Alessandro Flammini. Online Human-Bot Interactions: Detection, Estimation, and Characterization. arXiv:1703.03107 [cs], 2017.
arXiv: 1703.03107.
[37] Bo Wang, Arkaitz Zubiaga, Maria Liakata, and Rob Procter. Making
the Most of Tweet-Inherent Features for Social Spam Detection on
Twitter. arXiv:1503.07405 [cs], 2015. arXiv: 1503.07405.
[38] Zhao Yan, Nan Duan, Junwei Bao, Peng Chen, Ming Zhou, Zhoujun
Li, and Jianshe Zhou. Docchat: An information retrieval approach for
chatbot engines using unstructured documents. In Proceedings of the54th Annual Meeting of the Association For Computational Linguistics(Volume 1: Long Papers), volume 1, pages 516–525, 2016.
[39] Haifeng Yu, Michael Kaminsky, Phillip B. Gibbons, and Abraham
Flaxman. SybilGuard: Defending Against Sybil Attacks via Social
Networks. In Proceedings of the 2006 Conference on Applications,Technologies, Architectures, and Protocols for Computer Communications, SIGCOMM ’06, pages 267–278, 2006.
[40] Ye Zhang and Byron Wallace. A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification. arXiv:1510.03820 [cs], October 2015. arXiv: 1510.03820.

Gonçalves, Alessandro Flammini, and Filippo Menczer. Detecting
and tracking political abuse in social media. In Proceedings of theFifth International AAAI Conference on Weblogs and Social Media,
pages 297–304, 2011.
[31] Marco Aurelio Ruediger, Amaro Grassi, Ana Freitas, Andressa Contarato, Carolina Taboada, Danilo Carvalho, Humberto Ferreira, Lucas
Roberto da Silva, Pedro Lenhard, Rachel Bastos, and Thomas Traumann. Robôs, redes sociais e política no Brasil: Estudo sobre interferências ilegítimas no debate público na web, riscos à democracia eprocesso eleitoral de 2018. Report, FGV DAPP, 2017.
[32] Takeshi Sakaki, Makoto Okazaki, and Yutaka Matsuo. Earthquake
Shakes Twitter Users: Real-time Event Detection by Social Sensors.
In Proceedings of the 19th International Conference on World Wide
Web, WWW ’10, pages 851–860, 2010.
[33] Gerard Salton, Anita Wong, and Chung-Shu Yang. A vector
space model for automatic indexing. Communications of the ACM,
18(11):613–620, 1975.
[34] Ramanathan Sugumaran and Jonathan Voss. Real-time Spatiotemporal Analysis of West Nile Virus Using Twitter Data. In Proceedings of the 3rd International Conference on Computing for Geospatial
Research and Applications, COM.Geo ’12, pages 39:1–39:2, 2012.
[35] Daniel Tobert, Arvind von Keudell, and Edward K. Rodriguez.
Lessons From the Boston Marathon Bombing: An Orthopaedic Per-

7