# An Evolutionary Computation Approach for Twitter Bot Detection
## 3 Data Preparation
### Feature Extraction  
As we mentioned before, the _TwiBot-20_ dataset comes with both user metadata information and tweets content information. For this reason, we extract a set of meaningful features from the available data to build a dataset where each user profile can be represented as a fixed-sized vector of real-number values. This step consists of a data preparation process in which feature engineering and feature selection are performed to map each user profile contained in the original dataset into a space contained in \(\mathbb{R}^{d}\) where \(d\in\mathbb{N}\). Hence, this preliminary step is executed for each user account independently. The most meaningful metadata properties associated with user accounts can be retrieved by Twitter API calls (Twitter API docs for user profile. [[https://developer.twitter.com/en/docs/twitter-api/v1/data-dictionary/object-model/user](https://developer.twitter.com/en/docs/twitter-api/v1/data-dictionary/object-model/user)]([https://developer.twitter.com/en/docs/twitter-api/v1/data-dictionary/object-model/user](https://developer.twitter.com/en/docs/twitter-api/v1/data-dictionary/object-model/user)), accessed on 16 December 2021). We describe these properties as follows. * _id_: unique numerical identifier of the user account;
* _screen_name_: unique string composed of letters, digits, and underscores, which identifies the username of the Twitter account;
* _name_: name visualized for the user account;
* _description_: optional field indicating the description visualized for the user account;
* _URL_: optional field indicating an URL for the user account;
* _location_: optional attribute indicating a location for the user account;
* _created_at_: timestamp indicating the user account creation time;
* _statuses_count_: total number of tweets (including retweets) published by the user during the entire account lifetime;
* _followers_count_: total number of followers of the user account;
* _friends_count_: total number of followings of the user account;
* _favourites_count_: total number of posts that the user has marked as favorites;
* _listed_count_: total number of public lists that the user is a member of. Additionally, _TwiBot-20_ provides us with a sample of approximately 200 tweets for each user account (each tweet is a text string), where these tweets correspond to the latest published tweets for a given user account, and thus they are the most representative of the behavior of that user on Twitter during the period in which the data has been collected. For this reason, we believe that deriving features from tweets that can be joined with user metadata information would lead to a significant improvement in the predictive power of an ML model for bot detection. As a first step, we employ natural language processing techniques to properly preprocess users' tweets while converting them from strings to lists of tokens. In particular, we perform the following operations for each tweet of a given user account using the NLTK (NLTK Library. [[https://www.nltk.org/](https://www.nltk.org/)]([https://www.nltk.org/](https://www.nltk.org/)), accessed on 4 January 2022) library in Python 3.8:  
Figure 1: Count plot of the number of instances of the partitions of the _TwiBot-20_ dataset for each class. * we first perform lowering of text and trimming. Then, we extract the hashtags and the mentioned users that are eventually contained in the current tweet. We also count the number of URLs and we check whether the tweet is an authentic published tweet or a retweet;
* we perform tokenization of the text using a Tweet Tokenizer, and part-of-speech tagging after proper punctuation, URLs, and digits removal;
* we use a Word Net Lemmatizer to lemmatize the tagged tokens of the previous step in this pipeline;
* we filter lemmatized tokens by excluding stop words and single-character words;
* we return the list of tokenized tweets obtained by following this pipeline.