Sarcasm Classification (Using FastText) | by Sambit Mahapatra | Towards Data Science

FastText is a library created by the Facebook Research Team for efficient learning of word representations and sentence classification. It has gained a lot of attraction in the NLP community especially as a strong baseline for word representation replacing word2vec as it takes the char n-grams into account while getting the word vectors. Here we will be using FastText for text classification.
The data is collected from https://www.kaggle.com/rmisra/news-headlines-dataset-for-sarcasm-detection. Here the sarcastic ones are from TheOnion and nos-sarcastic ones are from HuffPost. Let’s now jump to coding.
First, let’s inspect the data to decide the approach. You can download the data from https://www.kaggle.com/rmisra/news-headlines-dataset-for-sarcasm-detection/data?select=Sarcasm_Headlines_Dataset_v2.json.
#load dataimport pandas as pddf = pd.read_json("Sarcasm_Headlines_Dataset_v2.json", lines=True)#shuffle the data inplacedf = df.sample(frac=1).reset_index(drop=True)# show first few rowsdf.head()
Basically, on reading the json in tabular form with pandas, the dataset contains 3 columns where ‘headline’ contains the headline texts of the news and ‘is_sarcastic’ contains 1 and 0 signifying sarcastic and non-sarcastic respectively. If we see the representation of sarcastic and non-sarcastic examples are
0 149851 13634Name: is_sarcastic, dtype: int64
Now to just look at how the texts are looking for sarcastic and non-sarcastic examples:
#word cloud on sarcastic headlinessarcastic = ‘ ‘.join(df[df[‘is_sarcastic’]==1][‘headline’].to_list())plot_wordcloud(sarcastic, ‘Reds’)
#word cloud on sarcastic headlines sarcastic = ' '.join(df[df['is_sarcastic']==0]['headline'].to_list()) plot_wordcloud(sarcastic, 'Reds')
Now before going to building the classifier model, we need to do some cleaning of the text to remove noise. Since these are the news headlines, they don’t contain much crap. The cleanings what I thought of having are getting all string lowercase, getting rid of anything which is not alphanumeric and replacing the numeric with a specific label.
df['headline'] = df['headline'].str.lower()df['headline'] = df['headline'].apply(alpha_num)df['headline'] = df['headline'].apply(replace_num)
Now, we are ready with the cleaned text and their corresponding labels to build a binary sarcasm classifier. As discussed, we will build the model using FastText python module.
First, need to install the FastText python module in the environment.
#Building fasttext for python\ !git clone https://github.com/facebookresearch/fastText.git !cd fastText!pip3 install .
We need to make the training and testing file ready in the format which FastText api can understand. The default format of text file on which we want to train our model should be __ label __ <Label> <Text>. We can use other prefix instead of __label__ by changing the parameter accordingly in the training time, which we will see ahead.
#data preparation for fasttextwith open('fasttext_input_sarcastic_comments.txt', 'w') as f:    for each_text, each_label in zip(df['headline'], df['is_sarcastic']):        f.writelines(f'__label__{each_label} {each_text}\n')
The data in the file looks like this:
!head -n 10 fasttext_input_sarcastic_comments.txt__label__1 school friends dont find camp songs funny__label__0 what cutting americorps would mean for public lands__label__0 when our tears become medicine__label__1 craig kilborn weds self in private ceremony__label__1 white couple admires fall colors__label__1 mom much more insistent about getting grandkids from one child than other__label__0 diary of a queer kids mom__label__1 sephora makeup artist helping woman create the perfect pink eye__label__1 kelloggs pulls controversial chocobastard from store shelves__label__0 winston churchills grandson introduces a new nickname for donald trump
Here __label__0 implies non-sarcastic and __label__1 implies sarcastic. Now we are in good shape to start training the classifier model. For that will divide the dataset into training(90%) and testing(10%) dataset. The FastText function to be used for this supervised binary classification is train_supervised.
''For classification train_supervised call will be used:The default parameters to it:    input             # training file path (required)    lr                # learning rate [0.1]    dim               # size of word vectors [100]    ws                # size of the context window [5]    epoch             # number of epochs [5]    minCount          # minimal number of word occurences [1]    minCountLabel     # minimal number of label occurences [1]    minn              # min length of char ngram [0]    maxn              # max length of char ngram [0]    neg               # number of negatives sampled [5]    wordNgrams        # max length of word ngram [1]    loss              # loss function {ns, hs, softmax, ova} [softmax]    bucket            # number of buckets [2000000]    thread            # number of threads [number of cpus]    lrUpdateRate      # change the rate of updates for the learning rate [100]    t                 # sampling threshold [0.0001]    label             # label prefix ['__label__']    verbose           # verbose [2]    pretrainedVectors # pretrained word vectors (.vec file) for supervised learning []'''model = fasttext.train_supervised('sarcasm_train.bin', wordNgrams=2)
To measure the performance in the testing dataset:
#measuring performance on test datadef print_results(sample_size, precision, recall):    precision   = round(precision, 2)    recall      = round(recall, 2)    print(f'{sample_size=}')    print(f'{precision=}')    print(f'{recall=}')print_results(*model.test('sarcasm_test.bin'))sample_size=2862 precision=0.87 recall=0.87
The results, though not perfect, look promising. We save the model object now for future inferences.
#save the modelmodel.save_model('fasttext_sarcasm.model')
FastText is also capable of compressing the model in order to have a much smaller size model file by sacrificing only a little bit of performance through quantisation.
# with the previously trained `model` object, callmodel.quantize(input='sarcasm_train.bin', retrain=True)\# results on test setprint_results(*model.test('sarcasm_test.bin'))sample_size=2862 precision=0.86 recall=0.86
If you see the precision and recall seem to suffer .01 score but, looking at the model file size:
!du -kh ./fasttext_sarcasm*98M ./fasttext_sarcasm.ftz774M ./fasttext_sarcasm.model
The compressed model is only 1/12th of the base model. So this is a trade-off between model size and performance which users have to decide depending on the use case. Since the classifier model is trained and ready, now is the time to have the inference script ready so you can plan the deployment.
def predict_is_sarcastic(text):    return SarcasmService.get_model().predict(text, k=2)if __name__ == '__main__':    ip = 'Make Indian manufacturing competitive to curb Chinese imports: RC Bhargava'    print(f'Result : {predict_is_sarcastic(ip)}')Result : (('__label__0', '__label__1'), array([0.5498156 , 0.45020437]))
As can be seen above the result label with maximum probability is __label__0 which means the used headline is non-sarcastic as per the trained model. In model.predict() call the value of k signifies the number of classes you want in output along with their respective probability score. Since we have used softmax activation (default one in FastText), the sum of the probability of the two labels is 1.
To conclude, FastText can be a strong baseline while doing any NLP classification and its implementation is very easy. All the source code of this article can be found at my git repo. The FastText python module is not officially supported but that shouldn’t be an issue for tech people to experiment :). In future post will try to discuss how can the trained model be moved to production.