fastText, and how to use it for text analysis?
By: Vaibhav Shukla
Tags: Machine Learning, NLP, Sentiment Analysis, Fasttext, Vaibhav Shukla

We all use facebook and you must have all experienced at some time that you have made some post and facebook starts showing you ads exactly related to that thing.

For example if you make a facebook post that you are going to quit your job to start some new venture of yours and suddenly facebook starts showing you ads like this-

So how does facebook exactly know what to show?

Well it’s the magic of its NLP library- fastText.

FastText is an open-source, free, lightweight library recently open sourced by Facebook.FastText is a library created by the Facebook Research Team for efficient learning of word representations and sentence classification.

This library has gained a lot of attention in the NLP community as it has shown great state of the art results in various NLP domains.

How to use it?

In this article I am going to tell you that how to use it for text classification.

Step 1: Putting your data in correct format

It is very important for fastText to have data in a prescribed correct format.

The format:

_label_0 your_text

Here ‘_label_’ has to be used as it is.

‘0’ tells your label.(the value in the label column for that particular row)

‘your_text’ is the text on which you want to train your classifier.

In the above given code I have shown the way to how to open a new file ‘train.txt’ for writing and write your dataframe into it.

Step 2: Cloning the repo

Next we need to clone the fastText repo into our notebook to use its functions.

Repo id: https://github.com/facebookresearch/fastText.git

After cloning it follow these steps:

Make sure that your ‘train.txt’ file is inside the fastText folder created by cloning the repo.

Step 3: Playing around with the commands

Now your model is ready to train, give the following command to train your classifier-

./fasttext supervised -input train.txt -output <path to save model>

-label __label__ -lr 0.1 -epoch 40

The following arguments are mandatory:

-input training file path

-output output file path

You could play around with these parameters to fine tune your model(be sure to use a dash(-) before using any argument)

Provided argument without a dash!

Usage:

The following arguments are optional:

-verbose verbosity level [2]

The following arguments for the dictionary are optional:

-minCount minimal number of word occurences [1]

-minCountLabel minimal number of label occurences [0]

-wordNgrams max length of word ngram [2]

-bucket number of buckets [2000000]

-minn min length of char ngram [0]

-maxn max length of char ngram [0]

-t sampling threshold [0.0001]

-label labels prefix [__label__]

The following arguments for training are optional:

-lr learning rate [0.1]

-lrUpdateRate change the rate of updates for the learning rate [100]

-dim size of word vectors [100]

-ws size of the context window [5]

-epoch number of epochs [40]

-neg number of negatives sampled [5]

-loss loss function {ns, hs, softmax, one-vs-all} [softmax]

-thread number of threads [12]

-pretrainedVectors pretrained word vectors for supervised learning []

-saveOutput whether output params should be saved [false]

The following arguments for quantization are optional:

cutoff number of words and ngrams to retain [0]

-retrain whether embeddings are finetuned if a cutoff is applied [true]

-qnorm whether the norm is quantized separately [false]

-qout whether the classifier is quantized [false]

-dsub size of each sub-vector [2]

Step 4: Predicting using saved model

The following command is to test the model on a pre-annotated test dataset and compare the original labels with the predicted labels and generate evaluation scores in the form of precision and recall values.

The precision is the number of correct labels among the labels predicted by fastText. The recall is the number of labels that successfully were predicted.

./fasttext test <path to model> <path to test file> k

Make sure that the test file is also in the same format as train file.

To predict new labels:

./fasttext predict <path to model> <path to test file> k > <path to prediction file>

where k signifies that the model will predict the top k labels for each review.

The labels predicted for the above reviews are as follows :

__label__2

__label__1

__label__2

__label__2

__label__2

__label__2

__label__2

__label__2

__label__1

__label__2

__label__2

The prediction file can then be used for further detailed analysis and visualization purposes.

fastText has shown great results in other fields also like word representations etc. which shall be covered in upcoming articles.

Thank you for reading this article. 😄