Package: fasttext-win
fasttext is a Python interface forFacebook
fastText.Requirementsfasttext support Python 2.6 or newer. It requiresCythonin order to build the
C++ extension.InstallationpipinstallfasttextExample usageThis package has two main use cases: word representation learning and
text classification.These were described in the two papers1and2.Word representation learningIn order to learn word vectors, as described in1, we can usefasttext.skipgramandfasttext.cbowfunction like the following:importfasttext# Skipgram modelmodel=fasttext.skipgram('data.txt','model')printmodel.words# list of words in dictionary# CBOW modelmodel=fasttext.cbow('data.txt','model')printmodel.words# list of words in dictionarywheredata.txtis a training file containingutf-8encoded text.
By default the word vectors will take into account character n-grams
from 3 to 6 characters.At the end of optimization the program will save two files:model.binandmodel.vec.model.vecis a text file containing the word vectors, one per line.model.binis a binary file containing the parameters of the model
along with the dictionary and all hyper parameters.The binary file can be used later to compute word vectors or to restart
the optimization.The followingfasttext(1)command is equivalent# Skipgram model./fasttextskipgram-inputdata.txt-outputmodel# CBOW model./fasttextcbow-inputdata.txt-outputmodelObtaining word vectors for out-of-vocabulary wordsThe previously trained model can be used to compute word vectors for
out-of-vocabulary words.printmodel['king']# get the vector of the word 'king'the followingfasttext(1)command is equivalent:echo"king"|./fasttextprint-vectorsmodel.binThis will output the vector of wordkingto the standard output.Load pre-trained modelWe can usefasttext.load_modelto load pre-trained model:model=fasttext.load_model('model.bin')printmodel.words# list of words in dictionaryprintmodel['king']# get the vector of the word 'king'Text classificationThis package can also be used to train supervised text classifiers and
load pre-trained classifier from fastText.In order to train a text classifier using the method described in2, we can use the
following function:classifier=fasttext.supervised('data.train.txt','model')equivalent asfasttext(1)command:./fasttextsupervised-inputdata.train.txt-outputmodelwheredata.train.txtis a text file containing a training sentence
per line along with the labels. By default, we assume that labels are
words that are prefixed by the string__label__.We can specify the label prefix with thelabel_prefixparam:classifier=fasttext.supervised('data.train.txt','model',label_prefix='__label__')equivalent asfasttext(1)command:./fasttextsupervised-inputdata.train.txt-outputmodel-label'__label__'This will output two files:model.binandmodel.vec.Once the model was trained, we can evaluate it by computing the
precision at 1 (P@1) and the recall on a test set usingclassifier.testfunction:result=classifier.test('test.txt')print'P@1:',result.precisionprint'R@1:',result.recallprint'Number of examples:',result.nexamplesThis will print the same output to stdout as:./fasttexttestmodel.bintest.txtIn order to obtain the most likely label for a list of text, we can useclassifer.predictmethod:texts=['example very long text 1','example very longtext 2']labels=classifier.predict(texts)printlabels# Or with the probabilitylabels=classifier.predict_proba(texts)printlabelsWe can specifykvalue to get the k-best labels from classifier:labels=classifier.predict(texts,k=3)printlabels# Or with the probabilitylabels=classifier.predict_proba(texts,k=3)printlabelsThis interface is equivalent asfasttext(1)predict command. The
same model with the same input set will have the same prediction.API documentationSkipgram modelTrain & load skipgram modelmodel=fasttext.skipgram(params)List of availableparamsand their default value:input_file     training file path (required)
output         output file path (required)
lr             learning rate [0.05]
lr_update_rate change the rate of updates for the learning rate [100]
dim            size of word vectors [100]
ws             size of the context window [5]
epoch          number of epochs [5]
min_count      minimal number of word occurences [5]
neg            number of negatives sampled [5]
word_ngrams    max length of word ngram [1]
loss           loss function {ns, hs, softmax} [ns]
bucket         number of buckets [2000000]
minn           min length of char ngram [3]
maxn           max length of char ngram [6]
thread         number of threads [12]
t              sampling threshold [0.0001]
silent         disable the log output from the C++ extension [1]
encoding       specify input_file encoding [utf-8]Example usage:model=fasttext.skipgram('train.txt','model',lr=0.1,dim=300)CBOW modelTrain & load CBOW modelmodel=fasttext.cbow(params)List of availableparamsand their default value:input_file     training file path (required)
output         output file path (required)
lr             learning rate [0.05]
lr_update_rate change the rate of updates for the learning rate [100]
dim            size of word vectors [100]
ws             size of the context window [5]
epoch          number of epochs [5]
min_count      minimal number of word occurences [5]
neg            number of negatives sampled [5]
word_ngrams    max length of word ngram [1]
loss           loss function {ns, hs, softmax} [ns]
bucket         number of buckets [2000000]
minn           min length of char ngram [3]
maxn           max length of char ngram [6]
thread         number of threads [12]
t              sampling threshold [0.0001]
silent         disable the log output from the C++ extension [1]
encoding       specify input_file encoding [utf-8]Example usage:model=fasttext.cbow('train.txt','model',lr=0.1,dim=300)Load pre-trained modelFile.binthat previously trained or generated by fastText can be
loaded using this functionmodel=fasttext.load_model('model.bin',encoding='utf-8')Attributes and methods for the modelSkipgram and CBOW model have the following atributes & methodsmodel.model_name# Model namemodel.words# List of words in the dictionarymodel.dim# Size of word vectormodel.ws# Size of context windowmodel.epoch# Number of epochsmodel.min_count# Minimal number of word occurencesmodel.neg# Number of negative sampledmodel.word_ngrams# Max length of word ngrammodel.loss_name# Loss function namemodel.bucket# Number of bucketsmodel.minn# Min length of char ngrammodel.maxn# Max length of char ngrammodel.lr_update_rate# Rate of updates for the learning ratemodel.t# Value of sampling thresholdmodel.encoding# Encoding of the modelmodel[word]# Get the vector of specified wordSupervised modelTrain & load the classifierclassifier=fasttext.supervised(params)List of availableparamsand their default value:input_file             training file path (required)
output                 output file path (required)
label_prefix           label prefix ['__label__']
lr                     learning rate [0.1]
lr_update_rate         change the rate of updates for the learning rate [100]
dim                    size of word vectors [100]
ws                     size of the context window [5]
epoch                  number of epochs [5]
min_count              minimal number of word occurences [1]
neg                    number of negatives sampled [5]
word_ngrams            max length of word ngram [1]
loss                   loss function {ns, hs, softmax} [softmax]
bucket                 number of buckets [0]
minn                   min length of char ngram [0]
maxn                   max length of char ngram [0]
thread                 number of threads [12]
t                      sampling threshold [0.0001]
silent                 disable the log output from the C++ extension [1]
encoding               specify input_file encoding [utf-8]
pretrained_vectors     pretrained word vectors (.vec file) for supervised learning []Example usage:classifier=fasttext.supervised('train.txt','model',label_prefix='__myprefix__',thread=4)Load pre-trained classifierFile.binthat previously trained or generated by fastText can be
loaded using this function../fasttextsupervised-inputtrain.txt-outputclassifier-label'some_prefix'classifier=fasttext.load_model('classifier.bin',label_prefix='some_prefix')Test classifierThis is equivalent asfasttext(1)test command. The test using the
same model and test set will produce the same value for the precision at
one and the number of examples.result=classifier.test(params)# Propertiesresult.precision# Precision at oneresult.recall# Recall at oneresult.nexamples# Number of test examplesThe paramkis optional, and equal to1by default.Predict the most-likely label of textsThis interface is equivalent asfasttext(1)predict command.textsis an array of stringlabels=classifier.predict(texts,k)# Or with probabilitylabels=classifier.predict_proba(texts,k)The paramkis optional, and equal to1by default.Attributes and methods for the classifierClassifier have the following atributes & methodsclassifier.labels# List of labelsclassifier.label_prefix# Prefix of the labelclassifier.dim# Size of word vectorclassifier.ws# Size of context windowclassifier.epoch# Number of epochsclassifier.min_count# Minimal number of word occurencesclassifier.neg# Number of negative sampledclassifier.word_ngrams# Max length of word ngramclassifier.loss_name# Loss function nameclassifier.bucket# Number of bucketsclassifier.minn# Min length of char ngramclassifier.maxn# Max length of char ngramclassifier.lr_update_rate# Rate of updates for the learning rateclassifier.t# Value of sampling thresholdclassifier.encoding# Encoding that used by classifierclassifier.test(filename,k)# Test the classifierclassifier.predict(texts,k)# Predict the most likely labelclassifier.predict_proba(texts,k)# Predict the most likely label include their probabilityThe paramkforclassifier.test,classifier.predictandclassifier.predict_probais optional, and equal to1by default.ReferencesEnriching Word Vectors with Subword Information[1] P. Bojanowski*, E. Grave*, A. Joulin, T. Mikolov,*Enriching Word
Vectors with Subword
Information*@article{bojanowski2016enriching,
  title={Enriching Word Vectors with Subword Information},
  author={Bojanowski, Piotr and Grave, Edouard and Joulin, Armand and Mikolov, Tomas},
  journal={arXiv preprint arXiv:1607.04606},
  year={2016}
}Bag of Tricks for Efficient Text Classification[2] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov,*Bag of Tricks for
Efficient Text
Classification*@article{joulin2016bag,
  title={Bag of Tricks for Efficient Text Classification},
  author={Joulin, Armand and Grave, Edouard and Bojanowski, Piotr and Mikolov, Tomas},
  journal={arXiv preprint arXiv:1607.01759},
  year={2016}
}(* These authors contributed equally.)Join the fastText communityFacebook page:https://www.facebook.com/groups/1174547215919768Google group:https://groups.google.com/forum/#!forum/fasttext-library