Package: pyfasttext
pyfasttextWarning!pyfasttextis no longer maintained: use the official Python binding from the fastText repository:https://github.com/facebookresearch/fastText/tree/master/pythonYet another Python binding forfastText.The binding supports Python 2.6, 2.7 and Python 3. It requiresCython.Numpyandcysignalsare also
dependencies, but are optional.pyfasttexthas been tested successfully on Linux and Mac OS X.Warning: if you want to compilepyfasttexton Windows, do not
compile with thecysignalsmodule because it does not support this
platform.Table of ContentspyfasttextTable of ContentsInstallationSimplest way to install pyfasttext: use
pipPossible compilation errorCloningRequirements for Python 2.7Building and installing
manuallyBuilding and installing without optional
dependenciesUsageHow to load the library?How to load an existing
model?Word representation
learningTraining using SkipgramTraining using CBoWWord vectorsWord vectors accessVector for a given wordNumpy ndarrayWords for a given vectorGet the number of words in the
modelGet all the word vectors in a
modelNumpy ndarrayMisc operations with word
vectorsWord similarityMost similar wordsAnalogiesText classificationSupervised learningGet all the labelsGet the number of labelsPredictionLabels and probabilitiesNormalized probabilitiesLabels onlyQuantizationIs a model quantized?SubwordsGet the subwordsGet the subword vectorsSentence and text vectorsUnsupervised modelsSupervised modelsMisc utilitiesShow the module versionShow fastText versionShow the model
(hyper)parametersShow the model version
numberExtract labels or classes from a
datasetExtract labelsExtract classesExceptionsInterruptible operationsInstallationTo compilepyfasttext, make sure you have the following compiler: *
GCC (g++) with C++11 support. * LLVM (clang++) with (at least)
partial C++17 support.Simplest way to install pyfasttext: use pipJust type these lines:pipinstallcythonpipinstallpyfasttextPossible compilation errorIf you have a compilation error, you can try to installcysignalsmanually:pipinstallcysignalsThen, retry to installpyfasttextwith the already mentionedpipcommand.Cloningpyfasttextuses gitsubmodules.So, you need to add the--recursiveoption when you clone the
repository.gitclone--recursivehttps://github.com/vrasneur/pyfasttext.gitcdpyfasttextRequirements for Python 2.7Python 2.7 support relies on thefuturemodule:pyfasttextneedsbytesobjects, which are not
available natively in Python2.You can install thefuturemodule withpip.pipinstallfutureBuilding and installing manuallyFirst, install all the requirements:pipinstall-rrequirements.txtThen, build and install withsetup.py:pythonsetup.pyinstallBuilding and installing without optional dependenciespyfasttextcan export word vectors asnumpyndarrays,
however this feature can be disabled at compile time.To compile withoutnumpy, pyfasttext has aUSE_NUMPYenvironment
variable. Set this variable to 0 (or empty), like this:USE_NUMPY=0pythonsetup.pyinstallIf you want to compile withoutcysignals, likewise, you can set theUSE_CYSIGNALSenvironment variable to 0 (or empty).UsageHow to load the library?>>>frompyfasttextimportFastTextHow to load an existing model?>>>model=FastText('/path/to/model.bin')or>>>model=FastText()>>>model.load_model('/path/to/model.bin')Word representation learningYou can use all the options provided by thefastTextbinary
(input,output,epoch,lr, …).Just use keyword arguments in the training methods of theFastTextobject.Training using Skipgram>>>model=FastText()>>>model.skipgram(input='data.txt',output='model',epoch=100,lr=0.7)Training using CBoW>>>model=FastText()>>>model.cbow(input='data.txt',output='model',epoch=100,lr=0.7)Word vectorsWord vectors accessVector for a given wordBy default, a single word vector is returned as a regular Python array
of floats.>>>model['dog']array('f',[-1.308749794960022,-1.8326224088668823,...])Numpy ndarrayThemodel.get_numpy_vector(word)method returns the word vector as anumpyndarray.>>>model.get_numpy_vector('dog')array([-1.30874979,-1.83262241,...],dtype=float32)If you want a normalized vector (i.e.the vector divided by its norm),
there is an optional boolean parameter namednormalized.>>>model.get_numpy_vector('dog',normalized=True)array([-0.07084749,-0.09920666,...],dtype=float32)Words for a given vectorThe inverse operation ofmodel[word]ormodel.get_numpy_vector(word)ismodel.words_for_vector(vector, k).It returns a list of thekwords closest to the provided vector.
The default value forkis 1.>>>king=model.get_numpy_vector('king')>>>man=model.get_numpy_vector('man')>>>woman=model.get_numpy_vector('woman')>>>model.words_for_vector(king+woman-man,k=1)[('queen',0.77121970653533936)]Get the number of words in the model>>>model.nwords500000Get all the word vectors in a model>>>forwordinmodel.words:...print(word,model[word])Numpy ndarrayIf you want all the word vectors as a bignumpyndarray, you can
use thenumpy_normalized_vectorsmember. Note that all these vectors
arenormalized.>>>model.nwords500000>>>model.numpy_normalized_vectorsarray([[-0.07549749,-0.09407753,...],[0.00635979,-0.17272158,...],...,[-0.01009259,0.14604086,...],[0.12467574,-0.0609326,...]],dtype=float32)>>>model.numpy_normalized_vectors.shape(500000,100)# (number of words, dimension)Misc operations with word vectorsWord similarity>>>model.similarity('dog','cat')0.75596606254577637Most similar words>>>model.nearest_neighbors('dog',k=2)[('dogs',0.7843924736976624),('cat',75596606254577637)]AnalogiesThemodel.most_similar()method works similarly as the one ingensim.>>>model.most_similar(positive=['woman','king'],negative=['man'],k=1)[('queen',0.77121970653533936)]Text classificationSupervised learning>>>model=FastText()>>>model.supervised(input='/path/to/input.txt',output='/path/to/model',epoch=100,lr=0.7)Get all the labels>>>model.labels['LABEL1','LABEL2',...]Get the number of labels>>>model.nlabels100PredictionTo obtain thekmost likely labels from test sentences, there are
multiplemodel.predict_*()methods.The default value forkis 1. If you want to obtain all the
possible labels, useNonefork.Labels and probabilitiesIf you have a list of strings (or an iterable object), use this:>>>model.predict_proba(['first sentence\n','second sentence\n'],k=2)[[('LABEL1',0.99609375),('LABEL3',1.953126549381068e-08)],[('LABEL2',1.0),('LABEL3',1.953126549381068e-08)]]If you want to test a single string, use this:>>>model.predict_proba_single('first sentence\n',k=2)[('LABEL1',0.99609375),('LABEL3',1.953126549381068e-08)]WARNING: In order to get the same probabilities as thefastTextbinary, you have to add a newline (\n) at the end of each string.If your test data is stored inside a file, use this:>>>model.predict_proba_file('/path/to/test.txt',k=2)[[('LABEL1',0.99609375),('LABEL3',1.953126549381068e-08)],[('LABEL2',1.0),('LABEL3',1.953126549381068e-08)]]Normalized probabilitiesFor performance reasons, fastText probabilities often do not sum up to
1.0.If you want normalized probabilities (where the sum is closer to 1.0
than the original probabilities), you can use thenormalized=Trueparameter in all the methods that output probabilities
(model.predict_proba(),model.predict_proba_file()andmodel.predict_proba_single()).>>>sum(probaforlabel,probainmodel.predict_proba_single('this is a sentence that needs to be classified\n',k=None))0.9785203068801335>>>sum(probaforlabel,probainmodel.predict_proba_single('this is a sentence that needs to be classified\n',k=None,normalized=True))0.9999999999999898Labels onlyIf you have a list of strings (or an iterable object), use this:>>>model.predict(['first sentence\n','second sentence\n'],k=2)[['LABEL1','LABEL3'],['LABEL2','LABEL3']]If you want to test a single string, use this:>>>model.predict_single('first sentence\n',k=2)['LABEL1','LABEL3']WARNING: In order to get the same probabilities as thefastTextbinary, you have to add a newline (\n) at the end of each string.If your test data is stored inside a file, use this:>>>model.predict_file('/path/to/test.txt',k=2)[['LABEL1','LABEL3'],['LABEL2','LABEL3']]QuantizationUse keyword arguments in themodel.quantize()method.>>>model.quantize(input='/path/to/input.txt',output='/path/to/model')You can load quantized models using theFastTextconstructor or themodel.load_model()method.Is a model quantized?If you want to know if a model has been quantized before, use themodel.quantizedattribute.>>>model=FastText('/path/to/model.bin')>>>model.quantizedFalse>>>model=FastText('/path/to/model.ftz')>>>model.quantizedTrueSubwordsfastText can use subwords (i.e.character ngrams) when doing
unsupervised or supervised learning.You can access the subwords, and their associated vectors, usingpyfasttext.Get the subwordsfastText’s word embeddings can be augmented with subword-level
information. It is possible to retrieve the subwords and their
associated vectors from a model usingpyfasttext.To retrieve all the subwords for a given word, use themodel.get_all_subwords(word)method.>>>model.args.get('minn'),model.args.get('maxn')(2,4)>>>model.get_all_subwords('hello')# word + subwords from 2 to 4 characters['hello','<h','<he','<hel','he','hel','hell','el','ell','ello','ll','llo','llo>','lo','lo>','o>']For fastText,<means “beginning of a word” and>means “end of
a word”.As you can see, fastText includes the full word. You can omit it using
theomit_word=Truekeyword argument.>>>model.get_all_subwords('hello',omit_word=True)['<h','<he','<hel','he','hel','hell','el','ell','ello','ll','llo','llo>','lo','lo>','o>']When a model is quantized, fastText mayprunesome subwords. If you
want to see only the subwords that are really used when computing a word
vector, you should use themodel.get_subwords(word)method.>>>model.quantizedTrue>>>model.get_subwords('beautiful')['eau','aut','ful','ul']>>>model.get_subwords('hello')['hello']# fastText will not use any subwords when computing the word vector, only the full wordGet the subword vectorsTo get the individual vectors given the subwords, use themodel.get_numpy_subword_vectors(word)method.>>>model.get_numpy_subword_vectors('beautiful')# 4 vectors, so 4 rowsarray([[0.49022141,0.13586822,...,-0.14065443,0.89617103],# subword "eau"[-0.42594951,0.06260503,...,-0.18182631,0.34219387],# subword "aut"[0.49958718,2.93831301,...,-1.97498322,-1.16815805],# subword "ful"[-0.4368791,-1.92924356,...,1.62921488,1.90240896]],dtype=float32)# subword "ul"In fastText, the final word vector is the average of these individual
vectors.>>>importnumpyasnp>>>vec1=model.get_numpy_vector('beautiful')>>>vecs2=model.get_numpy_subword_vectors('beautiful')>>>np.allclose(vec1,np.average(vecs2,axis=0))TrueSentence and text vectorsTo compute the vector of a sequence of words (i.e.a sentence),
fastText uses two different methods: * one for unsupervised models *
another one for supervised modelsWhen fastText computes a word vector, recall that it uses the average of
the following vectors: the word itself and its subwords.Unsupervised modelsFor unsupervised models, the representation of a sentence for fastText
is the average of the normalized word vectors.To get the resulting vector as a regular Python array, use themodel.get_sentence_vector(line)method.To get the resulting vector as anumpyndarray, use themodel.get_numpy_sentence_vector(line)method.>>>vec=model.get_numpy_sentence_vector('beautiful cats')>>>vec1=model.get_numpy_vector('beautiful',normalized=True)>>>vec2=model.get_numpy_vector('cats',normalized=True)>>>np.allclose(vec,np.average([vec1,vec2],axis=0)TrueSupervised modelsFor supervised models, fastText uses the regular word vectors, as well
as vectors computed using word ngrams (i.e.shorter sequences of words
from the sentence). When computing the average, these vectors are not
normalized.To get the resulting vector as a regular Python array, use themodel.get_text_vector(line)method.To get the resulting vector as anumpyndarray, use themodel.get_numpy_text_vector(line)method.>>>model.get_numpy_sentence_vector('beautiful cats')# for an unsupervised modelarray([-0.20266785,0.3407566,...,0.03044436,0.39055538],dtype=float32)>>>model.get_numpy_text_vector('beautiful cats')# for a supervised modelarray([-0.20840774,0.4289546,...,-0.00457615,0.52417743],dtype=float32)Misc utilitiesShow the module version>>>importpyfasttext>>>pyfasttext.__version__'0.4.3'Show fastText versionAs there is no version number in fastText, we use the latest fastText
commit hash (fromHEAD) as a substitute.>>>importpyfasttext>>>pyfasttext.__fasttext_version__'431c9e2a9b5149369cc60fb9f5beba58dcf8ca17'Show the model (hyper)parameters>>>model.args{'bucket':11000000,'cutoff':0,'dim':100,'dsub':2,'epoch':100,...}Show the model version numberfastText uses a versioning scheme for its generated models. You can
retrieve the model version number using themodel.versionattribute.version numberdescription-1for really old models
with no version number11first version number
added by fastText12for models generated
after fastText added
support for subwords
in supervised learning>>>model.version12Extract labels or classes from a datasetYou can use theFastTextobject to extract labels or classes from a
dataset. The label prefix (which is__label__by default) is set
using thelabelparameter in the constructor.If you load an existing model, the label prefix will be the one defined
in the model.>>>model=FastText(label='__my_prefix__')Extract labelsThere can be multiple labels per line.>>>model.extract_labels('/path/to/dataset1.txt')[['LABEL2','LABEL5'],['LABEL1'],...]Extract classesThere can be only one class per line.>>>model.extract_classes('/path/to/dataset2.txt')['LABEL3','LABEL1','LABEL2',...]ExceptionsThefastTextsource code directly calls exit() when something wrong
happens (e.g.a model file does not exist, …).Instead of exiting,pyfasttextraises a Python exception
(RuntimeError).>>>importpyfasttext>>>model=pyfasttext.FastText('/path/to/non-existing_model.bin')Modelfilecannotbeopenedforloading!Traceback(mostrecentcalllast):File"<stdin>",line1,in<module>File"src/pyfasttext.pyx",line124,inpyfasttext.FastText.__cinit__(src/pyfasttext.cpp:1800)File"src/pyfasttext.pyx",line348,inpyfasttext.FastText.load_model(src/pyfasttext.cpp:5947)RuntimeError:fastexttriedtoexit:1Interruptible operationspyfasttextusescysignalsto make all the computationally
intensive operations (e.g.training) interruptible.To easily interrupt such an operation, just typeCtrl-Cin your
Python shell.>>>model.skipgram(input='/path/to/input.txt',output='/path/to/mymodel')Read12MwordsNumberofwords:60237Numberoflabels:0...# type Ctrl-C during trainingTraceback(mostrecentcalllast):File"<stdin>",line1,in<module>File"src/pyfasttext.pyx",line680,inpyfasttext.FastText.skipgram(src/pyfasttext.cpp:11125)File"src/pyfasttext.pyx",line674,inpyfasttext.FastText.train(src/pyfasttext.cpp:11009)File"src/pyfasttext.pyx",line668,inpyfasttext.FastText.train(src/pyfasttext.cpp:10926)File"src/cysignals/signals.pyx",line94,incysignals.signals.sig_raise_exception(build/src/cysignals/signals.c:1328)KeyboardInterrupt>>># you can have your shell back!