Detecting State-backed Twitter Trolls With Transformers
By: Chua Chin Hon
Tags: Deep Learning, NLP, US Elections, Data Science, Machine Learning

1. GROUND TRUTH, DATA SOURCE, MODEL AND REPO

First things first: How did I ascertain whose tweets are deemed the work of state influence campaigns? In this case, the ground truth is established by Twitter’s election integrity team.

The state troll tweets used in this project are those which have been identified by Twitter and progressively released to the public since 2018. I chose six sets of state troll tweets from Twitter — three each from China and Russia — which I cleaned up, combined and downsampled to 50,000 rows.

I created an equivalent set of 50,000 rows of real tweets by using Tweepy to scrape 175 accounts, which comprise a mixture of verified users and those which I’ve personally checked for authenticity. The resulting combined dataset of 100,000 rows of state troll-plus-real tweets was further split into the usual train-test-validation sets in the standard proportion of 70:20:10 respectively.

Full details are in my notebooks, and I won’t repeat them here for brevity’s sake. The full repo is available here. Fine tuning was done on a Colab Pro account, and took about five-and-a-half hours.

The fine tuned Distilbert model is too big for Github, but I’ve uploaded a copy to Dropbox for those who just want to experiment with the model. The six raw CSV files containing state troll tweets have to be downloaded from Twitter if you wish to create a larger training set.

2. DATA PREPARATION

It ought to be pointed out that key assumptions in the data cleaning and preparation process will affect the outcomes. The assumptions are necessary to keep the scope of this project practical, but if you disagree with them, feel agree to slice a different version of the data based on your own preferred cleaning rules.

My main data cleaning rules for this project:

Exclude non-English tweets, since the working assumption is that the target audience is English-speaking. I also wanted to prevent the model from making predictions based on language.

Exclude retweets.

Exclude tweets which have fewer than three words after text cleaning.

I also kept the eventual combined dataset to 100,000 rows for practical reasons. My initial attempts to fine tune Distilbert with a 600,000 rows dataset resulted in repeated crashes of the Colab notebook and/or showed extremely long and impractical run times. A more ambitious version of this project just won’t be practical without access to more compute/hardware.

3. FINE TUNE THE DISTILBERT MODEL WITH CUSTOM DATASET

For this project, I picked the distilbert-base-uncased model (smaller and more manageable) and used Hugging Face’s trainer for the task. I abandoned attempts to do a hyperparameters search after several trials which ran on for too long, and hope to return to this topic at a future date (check out the discussion here).

The steps involved are fairly straightforward, as outlined in notebook2.0 of my repo. The code is mostly based on the excellent examples and documentation on Hugging Face here and here.

The immediate results of the fine tuning certainly look impressive:

'eval_accuracy': 0.9158421345191773,

'eval_f1': 0.9163813100629625,

'eval_precision': 0.9098486510199605,

'eval_recall': 0.9230084557187361

I ran a quick test against the validation set — 10,000 rows kept aside and which the model has not seen at all — and the excellent classification metrics barely budged:

'eval_accuracy': 0.9179,

'eval_f1': 0.9189935865811544,

'eval_precision': 0.9178163184864012,

'eval_recall': 0.9201738786801027

For a more thorough comparison and analysis, I trained two separate Logistic Regression and XGB classifier models using the same dataset used to fine tune the Distilbert model. Let’s see how they do in the same tests.