Repository: piskvorky/gensim

Issue:
Loading the English wikipedia model hangs indefinitely when low on RAM
Initially reported [here](https://github.com/RaRe-Technologies/gensim/issues/2350#issuecomment-460293018) by @akutuzov.

<copypasta>

This is yet another regression after the [fastText code refactoring](https://github.com/RaRe-Technologies/gensim/pull/2313) in Gensim 3.7 (another one was fixed in #2341).
Indeed, Gensim 3.6 loads pre-trained fastText models without any trouble. Below are examples with the Wikipedia model from https://fasttext.cc/, but the same stuff happens with any models trained using native fastText.
```
import gensim
gensim.__version__
'3.6.0'
import logging
logging.basicConfig(format='%(asctime)s : %(levelname)s : (message)s', level=logging.INFO)
model = gensim.models.fasttext.FastText.load_fasttext_format('wiki_en')
2019-01-24 16:23:47,740 : INFO : loading 2519370 words for fastText model from wiki.en.bin
2019-01-24 16:29:54,820 : INFO : loading weights for 2519370 words for fastText model from wiki.en.bin
2019-01-24 16:37:43,068 : INFO : loaded (2519370, 300) weight matrix for fastText model from wiki.en.bin 
model
<gensim.models.fasttext.FastText at 0x7f8e98e2c320>
```
However, Gensim 3.7 is doing weird things here (retraining the model instead of loading it?):
```
import gensim
gensim.__version__
'3.7.0'
import logging
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)
model = gensim.models.fasttext.FastText.load_fasttext_format('wiki.en')
2019-01-24 16:25:50,816 : INFO : loading 2519370 words for fastTextmodel from wiki.en.bin
2019-01-24 16:30:14,701 : INFO : resetting layer weights
2019-01-24 16:30:14,702 : INFO : Total number of ngrams is 0
2019-01-24 16:30:14,702 : INFO : Updating model with new vocabulary
2019-01-24 16:30:40,839 : INFO : New added 2519370 unique words (50% of original 5038740) and increased the count of 2519370 pre-existing words (50% of original 5038740)
2019-01-24 16:31:02,325 : INFO : deleting the raw counts dictionary of 2519370 items
2019-01-24 16:31:02,325 : INFO : sample=0.0001 downsamples 650 most-common words
2019-01-24 16:31:02,326 : INFO : downsampling leaves estimated 4076481917 word corpus (103.2% of prior 3949186974)
```
After it went like this for an hour, I killed the process.

Gensim 3.7.1 does the same, nothing changed. I'm sorry, but it seems that fastText refactoring in 3.7 was extremely badly tested, with so many things broken :-(

</copypasta>


Discussion:
@akutuzov 

> Gensim 3.7.1 does the same, nothing changed.

I tried to reproduce with the current develop branch (commit hash 21376a48, FB I/O is identical to 3.7.1 release), and couldn't:

```
(devel.env) mpenkov@hetrad2:~$ ls -l wiki.en.bin
-rw-r--r-- 1 mpenkov mpenkov 8493673445 Oct 19  2017 wiki.en.bin
(devel.env) mpenkov@hetrad2:~$ md5sum wiki.en.bin
92341f7f94c1801db19bb6b114fddff9  wiki.en.bin
(devel.env) mpenkov@hetrad2:~$ cat 2372.py 
import gensim
print(gensim.__version__)
import logging
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)
model = gensim.models.fasttext.FastText.load_fasttext_format('wiki.en')
print(model)
(devel.env) mpenkov@hetrad2:~$ python --version
Python 3.5.2
(devel.env) mpenkov@hetrad2:~$ time python 2372.py                                                                                                                                                            3.7.1
2019-02-05 03:13:50,019 : INFO : loading 2519370 words for fastText model from wiki.en.bin
2019-02-05 03:14:18,547 : INFO : resetting layer weights
2019-02-05 03:14:18,547 : INFO : Total number of ngrams is 0
2019-02-05 03:14:18,548 : INFO : Updating model with new vocabulary
2019-02-05 03:14:28,589 : INFO : New added 2519370 unique words (50% of original 5038740) and increased the count of 2519370 pre-existing words (50% of original 5038740)
2019-02-05 03:14:38,003 : INFO : deleting the raw counts dictionary of 2519370 items
2019-02-05 03:14:38,003 : INFO : sample=0.0001 downsamples 650 most-common words
2019-02-05 03:14:38,003 : INFO : downsampling leaves estimated 4076481917 word corpus (103.2% of prior 3949186974)
2019-02-05 03:18:55,583 : INFO : loaded (4519370, 300) weight matrix for fastText model from wiki.en.bin
FastText(vocab=2519370, size=300, alpha=0.025)

real    5m13.520s
user    4m15.348s
sys     0m56.488s
(devel.env) mpenkov@hetrad2:~$
```

Perhaps the model you are using is different? Please check the bytesize and md5 hash above and let me know if your results are different.

Furthermore, how much memory do you have available when loading the model? What O/S and Python version?
Thanks for researching it.
However, the title of the issue is wrong, it doesn't have anything to do with Wikipedia models. It happens with any fastText model in *.bin format.
I checked now, and indeed a smaller model (with the vocabulary size about 200K words) does load correctly. However, with Gensim 3.6 it takes only 3 GBytes of RAM and 37 seconds to load this model, while with Gensim 3.7.1 it takes 6 (!) GBytes of RAM and 50 seconds. 
Thus, loading a fastText model now takes two times more memory and is almost twice longer. No wonder many users experience freezes and errors on models which previously worked without any problems. In my initial report, I used the Wikipedia model with very large vocabulary (2.5M words), so it was just taking forever to load it.  What is the reason for such drastic increase in resource consumption?

I have Gensim 3.6 installed on Python 3.5, and Gensim 3.7.1 installed on Python 3.6. Both on the same machine with Linux Mint and 8 GBytes of RAM.
Also, why log messages when loading a fastText model now look like the model is being **trained**, not **loaded** ('Updating model with new vocabulary', 'downsampling', etc)? 
```
Gensim: 3.7.1
2019-02-05 10:33:30,931 : INFO : loading 228671 words for fastText model from parameters.bin
2019-02-05 10:33:39,311 : INFO : resetting layer weights
2019-02-05 10:33:39,311 : INFO : Total number of ngrams is 0
2019-02-05 10:33:39,312 : INFO : Updating model with new vocabulary
2019-02-05 10:33:39,859 : INFO : New added 228671 unique words (50% of original 457342) and increased the count of 228671 pre-existing words (50% of original 457342)
2019-02-05 10:33:40,850 : INFO : deleting the raw counts dictionary of 228671 items
2019-02-05 10:33:40,850 : INFO : sample=0.0001 downsamples 1400 most-common words
2019-02-05 10:33:40,850 : INFO : downsampling leaves estimated 1521803161 word corpus (157.9% of prior 963672019)
2019-02-05 10:34:12,676 : INFO : loaded (2228671, 300) weight matrix for fastText model from parameters.bin
FastText(vocab=228671, size=300, alpha=0.025)
```
What does that `(50% of original 457342)` mean? There are 228671 words in the vocabulary of the model being loaded, what is the source of this number multiplied by 2?
This is very confusing, compare to Gensim 3.6, same model:
```
Gensim: 3.6.0
2019-02-05 10:32:09,784 : INFO : loading 228671 words for fastText model from parameters.bin
2019-02-05 10:32:23,655 : INFO : loading weights for 228671 words for fastText model from parameters.bin
2019-02-05 10:32:45,777 : INFO : loaded (228671, 300) weight matrix for fastText model from parameters.bin
FastText(vocab=228671, size=300, alpha=0.025)
```

> However, the title of the issue is wrong, it doesn't have anything to do with Wikipedia models. 

I don't think it's necessarily wrong, but now that you've provided more information, I can improve the title.

> It happens with any fastText model in *.bin format.
> I checked now, and indeed a smaller model (with the vocabulary size about 200K words) does load correctly. 

There's a contradiction in there somewhere ;)

> However, with Gensim 3.6 it takes only 3 GBytes of RAM and 37 seconds to load this model, while with Gensim 3.7.1 it takes 6 (!) GBytes of RAM and 50 seconds.
> Thus, loading a fastText model now takes two times more memory and is almost twice longer. No wonder many users experience freezes and errors on models which previously worked without any problems. In my initial report, I used the Wikipedia model with very large vocabulary (2.5M words), so it was just taking forever to load it. What is the reason for such drastic increase in resource consumption?

The previous implementation of the load_fasttext_format function was broken - it did not load the complete model. This prevented training continuation.

As part of the refactoring, we resolved the above problem by loading the full model, including the neural network that is necessary for training continuation. This change means the model now occupies more memory. If you want to use the previous, broken behavior, you can pass the full_model=False parameter to the load_fasttext_format function. For more info, please see the [documentation](https://radimrehurek.com/gensim/models/fasttext.html#gensim.models.fasttext.FastText.load_fasttext_format). 

In the future, we will make a better effort to clarify changes like the above in the change log.

> I have Gensim 3.6 installed on Python 3.5, and Gensim 3.7.1 installed on Python 3.6. Both on the same machine with Linux Mint and 8 GBytes of RAM.

I think you're running out of RAM. The load is taking a prohibitively long time cause the system is performing paging.
> As part of the refactoring, we resolved the above problem by loading the full model, including the neural network that is necessary for training continuation. This change means the model now occupies more memory

OK, this makes sense indeed in itself. However, see below

> If you want to use the previous, broken behavior, you can pass the full_model=False parameter to the load_fasttext_format function.

I would say that this `full_model=True` probably should not be the default behavior. First, it makes the existing code consume twice as much resources compared to the previous behavior. Second, arguably most users don't actually want to train loaded fastText models further. This is an important option to have, but it is required by only some (advanced) users. It hardly makes sense to waste memory of those who simply want to look up static vectors, not to continue training. I believe the best way to handle this is to load models in the old way by default, but show a warning message saying something like 'if you want to continue training, set this parameter to True'. Also should be mentioned in the [Gensim fastText tutorial notebook](https://github.com/RaRe-Technologies/gensim/blob/master/docs/notebooks/FastText_Tutorial.ipynb).

Anyway, I think if we introduce changes that double the consumption of resources, this should be stated in the changelog as clearly as possible.

And there is still the issue of these weird log messages,  showing inconsistent numbers. Once again, these are Gensim 3.7.1 logs when loading the previously mentioned large Wikipedia model (I now used a machine with larger RAM):
```
2019-02-05 15:45:27,129 : INFO : loading 2519370 words for fastText model from wiki.en.bin
2019-02-05 15:47:29,576 : INFO : resetting layer weights
2019-02-05 15:47:29,576 : INFO : Total number of ngrams is 0
2019-02-05 15:47:29,577 : INFO : Updating model with new vocabulary
2019-02-05 15:47:50,002 : INFO : New added 2519370 unique words (50% of original 5038740) and increased the count of 2519370 pre-existing words (50% of original 5038740)
2019-02-05 15:48:15,712 : INFO : deleting the raw counts dictionary of 2519370 items
2019-02-05 15:48:15,712 : INFO : sample=0.0001 downsamples 650 most-common words
2019-02-05 15:48:15,712 : INFO : downsampling leaves estimated 4076481917 word corpus (103.2% of prior 3949186974)
2019-02-05 15:58:29,511 : INFO : loaded (4519370, 300) weight matrix for fastText model from wiki.en.bin
FastText(vocab=2519370, size=300, alpha=0.025)
```
The vocabulary size of this model is 2519370; why the weight matrix is (4519370, 300)?  Are these 2 million 'extra' the vectors for ngrams? Then probably it should be reported as such.
And why we are 'downsampling most common words' and 'updating model with new vocabulary', when we were supposed to just load a pre-trained model?
> I would say that this `full_model=True` probably should not be the default behavior.

You make valid arguments. When we were discussing the best default, @menshikh-iv and I decided on the current behavior for two reasons:

1. It is correct
2. It is consistent with the rest of Gensim. If you can load a model, you can continue training it.

> Also should be mentioned in the [Gensim fastText tutorial notebook](https://github.com/RaRe-Technologies/gensim/blob/master/docs/notebooks/FastText_Tutorial.ipynb).

That tutorial doesn't mention anything about Facebook I/O and it never did. The Jupyter notebooks aren't a good place for this, because we don't have automatic checks for their correctness (unlike, for example, the docstrings).

The [docstring](https://github.com/RaRe-Technologies/gensim/blob/21376a48b6bd205527ace67cf4dcbbb81b694b7c/gensim/models/fasttext.py#L169-L178) for the fasttext module already contains detailed documentation about Facebook I/O functionality.

> Anyway, I think if we introduce changes that double the consumption of resources, this should be stated in the changelog as clearly as possible.

Yes, I agree, and regret that we were not able to do this for the 3.7.1 release.
 
> The vocabulary size of this model is 2519370; why the weight matrix is (4519370, 300)? Are these 2 million 'extra' the vectors for ngrams?

Yes.

> And why we are 'downsampling most common words' and 'updating model with new vocabulary', when we were supposed to just load a pre-trained model?

This is an implementation detail. From your perspective, we are "just loading a pre-trained model", but there is more than just I/O happening [here](https://github.com/RaRe-Technologies/gensim/blob/21376a48b6bd205527ace67cf4dcbbb81b694b7c/gensim/models/fasttext.py#L1223-L1278). As you can see, the code is converting the vocabulary from Facebook's plain dictionary to Gensim's OO design, among other things.

The log messages you are seeing are there because the refactoring re-used some of the training code when loading a native model. Previously, this was done by duplicated code, with different logging output (almost none).
I agree the current logging could be better. I find it confusing too, when I put myself in user shoes:

`INFO : Total number of ngrams is 0` What? I just asked you (Gensim) to load a FT model, why are there 0 ngrams?
`INFO : New added 2519370 unique words (50% of original 5038740) and increased the count of 2519370 pre-existing words (50% of original 5038740)` "New added X words" => "Added X new words"? Or was it a method called "New" that added them? Where do these "pre-existing words" come from, if we just loaded and updated a model from scratch, from 0?
`INFO : deleting the raw counts dictionary of 2519370 items` Why are you deleting my dictionary?
`INFO : sample=0.0001 downsamples 650 most-common words` I asked you (Gensim) to load a model, why are you downsampling anything?
`INFO : downsampling leaves estimated 4076481917 word corpus (103.2% of prior 3949186974)` Nooo, leave my model alone! Also, how can downsampling **increase** the size to 103%?!

`INFO` logs should tell a story to the user. Their purpose is to illuminate **what's going on**, show progress, not confuse the user with inactionable trivia.

`DEBUG` logs should also tell a story, but useful for debugging / developer, more internal details than workflow progress. It seems most of these logs should be either rephrased or DEBUG: not enough context to be useful to users without knowledge of internal implementation details. The gensim 3.6.0 log posted by @akutuzov looked significantly clearer to me, as a user.
> You make valid arguments. When we were discussing the best default, @menshikh-iv and I decided on the current behavior for two reasons:
>     1. It is correct

Well, I think both modes (with or without the matrices necessary to continue training) are correct, for different tasks.

>     2. It is consistent with the rest of Gensim. If you can load a model, you can continue training it.

This is not entirely correct. For example, we have a (widely used) `load_word2vec_format()` method. It returns an embedding model which can't be trained further, since only one vector array is present. But it is still immensely useful.

> The log messages you are seeing are there because the refactoring re-used some of the training code when loading a native model. Previously, this was done by duplicated code, with different logging output (almost none).

Yes, I remember seeing these (or very similar) log messages when I continue training word2vec models in the `vocabulary update` mode. And this makes them even more confusing when a user sees them in the context of loading a pre-trained model. I agree with Radim: it doesn't matter what code paths are running under the hood, the INFO log  messages should tell the user what happens conceptually.


So, in  the end this issue turned out to be about improving log messages and about whether loading 'full' fastText models should be the default behavior :-)
Does it make sense to start separate issues about this?
> For example, we have a (widely used) `load_word2vec_format()` method. It returns an embedding model which can't be trained further, since only one vector array is present. But it is still immensely useful.

From the POV of Gensim's FastText design, what you get from calling load_fasttext_format(..., full_model=False) isn't really a model. It's a dummy model wrapping a working KeyedVectors instance (for the difference between a FastText model and KeyedVectors, see [this part](https://radimrehurek.com/gensim/models/keyedvectors.html#why-use-keyedvectors-instead-of-a-full-model) of the docs).

It may be clearer to introduce a separate function that loads just the KeyedVectors from a FB binary. Users that don't want to continue training can use the new function.

@piskvorky WDYT about the default value for the full_model flag? I think we've addressed both sides of the argument above, so we should make a call: keep it as is (full_model=True) or roll back to the previous behavior as default (full_model=False)?

@akutuzov I think better logging should be a separate issue. It's relatively simple and someone who hasn't ever contributed to the repo can do it.
No matter what default we choose, there will be some people confused.

I like the idea of two separate functions (which may be thin wrappers over an internal function that accepts the parameter). And then have the docs promote whichever function is appropriate, to drive the message home, especially for the (numerous) copy-paste-happy crowd.

If that's not possible, `full_model=False` makes more sense to me. Both for backward compatibility and because it's the more common use-case (what users "typically" want).

Having good logging is deceptively simple. It's like good documentation: while anyone can do it, to do it well you need to understand the workflows and concepts, to know what's important and tell the story well. Not sure that's a 100% newcomer job.
Given that we're already thinking about simplifying the API, I think it's better to introduce a separate function as part of that effort.
I believe that then there should be a **separate new function to load full models**, not vice versa. 

Thus, the `load_fasttext_format()` function will continue to behave in the same way as it behaved before Gensim 3.7: loading only the parts of fastText models needed to lookup word vectors (including inference from char n-grams for OOV words). This will be good for backwards compatibility, as @piskvorky has said. This will also be consistent with the similarly named `load_word2vec_format()` function, where users also don't expect the loaded model to be ready for further training.

Since the fastText *.bin format (unlike word2vec binary format) does contain the information necessary to continue training, we can provide a new 'advanced' function: `load_fasttext_full()` or something like this.

I also agree with @piskvorky that a random new contributor will arguably not be able to come up with useful and meaningful log messages, since it requires deep understanding of both the fastText algorithm itself and its particular implementation in Gensim. As a quick fix, I would suggest moving all these confusing messages into the DEBUG category, and returning INFO messages back to the Gensim 3.6 state.
@akutuzov

> I believe that then there should be a **separate new function to load full models**, not vice versa.
> <snip>
> This will also be consistent with the similarly named `load_word2vec_format()` function, where users also don't expect the loaded model to be ready for further training.

Actually, if you have a look at [the load_word2vec_format function](https://radimrehurek.com/gensim/models/keyedvectors.html#gensim.models.keyedvectors.Word2VecKeyedVectors.load_word2vec_format), you'll see that it returns an _embedding_ (KeyedVectors subclass), not a model.

This means the behavior of the old `load_fasttext_format` function was already inconsistent with `load_word2vec_format`, because the former returns a model that wraps an embedding (where the model is essentially useless) whereas the latter returns an embedding.

If we _really_ wanted to make things consistent, then we'd make a new KeyedVectors.load_fasttext_format class method. That method would return FastTextKeyedVectors, which is a word embedding that you can use to calculate vectors. This would be the same embedding you get with full_model=False, minus the model wrapper (which is useless). In this scenario, the FastText.load_fasttext_format would continue to load full models.

So we have to make a tradeoff between backwards compatibility and correctness/consistency here:

1. Go back to the old behavior. It was broken and inconsistent, but people were used to it.
2. Refactor the API to make things consistent: loading models gives _working_ models, loading _embeddings_ gives working embeddings, and make it clear which one is which. We'd have an easier time documenting our API, too, since things would be less confusing.

Given that we're trying to improve Gensim's FastText implementation, my preference is to refactor, but I'll let @piskvorky make the call on how to proceed here.
> I also agree with @piskvorky that a random new contributor will arguably not be able to come up with useful and meaningful log messages, since it requires deep understanding of both the fastText algorithm itself and its particular implementation in Gensim.

Do I hear a volunteering call? :) Can you improve the INFO logging messages @akutuzov (not just DEBUG)?

@mpenkov I'm +1 for consistency. 3.7.0 was big and recent and we can still afford a bug-fix release, while the dust is settling (also need those NMF fixes from #2371).

Resolving patch:
diff --git a/gensim/models/fasttext.py b/gensim/models/fasttext.py
--- a/gensim/models/fasttext.py
+++ b/gensim/models/fasttext.py
@@ -171,27 +171,43 @@
 .. sourcecode:: pycon
 
     >>> cap_path = datapath("crime-and-punishment.bin")
-    >>> # Partial model: loads quickly, uses less RAM, but cannot continue training
-    >>> fb_partial = FastText.load_fasttext_format(cap_path, full_model=False)
-    >>> # Full model: loads slowly, consumes RAM, but can continue training (see below)
-    >>> fb_full = FastText.load_fasttext_format(cap_path, full_model=True)
+    >>> fb_model = load_facebook_model(cap_path)
 
 Once loaded, such models behave identically to those trained from scratch.
 You may continue training them on new data:
 
 .. sourcecode:: pycon
 
-    >>> 'computer' in fb_full.wv.vocab  # New word, currently out of vocab
+    >>> 'computer' in fb_model.wv.vocab  # New word, currently out of vocab
     False
-    >>> old_computer = np.copy(fb_full.wv['computer'])  # Calculate current vectors
-    >>> fb_full.build_vocab(new_sentences, update=True)
-    >>> fb_full.train(new_sentences, total_examples=len(new_sentences), epochs=model.epochs)
-    >>> new_computer = fb_full.wv['computer']
+    >>> old_computer = np.copy(fb_model.wv['computer'])  # Calculate current vectors
+    >>> fb_model.build_vocab(new_sentences, update=True)
+    >>> fb_model.train(new_sentences, total_examples=len(new_sentences), epochs=model.epochs)
+    >>> new_computer = fb_model.wv['computer']
     >>> np.allclose(old_computer, new_computer, atol=1e-4)  # Vector has changed, model has learnt something
     False
-    >>> 'computer' in fb_full.wv.vocab  # New word is now in the vocabulary
+    >>> 'computer' in fb_model.wv.vocab  # New word is now in the vocabulary
     True
 
+If you do not intend to continue training the model, consider using the
+:func:`gensim.models.FastText.load_facebook_vectors` function instead.
+That function only loads the word embeddings (keyed vectors), consuming much less CPU and RAM:
+
+.. sourcecode:: pycon
+
+    >>> from gensim.test.utils import datapath
+    >>>
+    >>> cap_path = datapath("crime-and-punishment.bin")
+    >>> wv = load_facebook_vectors(cap_path)
+    >>>
+    >>> 'landlord' in wv.vocab  # Word is out of vocabulary
+    False
+    >>> oov_vector = wv['landlord']
+    >>>
+    >>> 'landlady' in wv.vocab  # Word is in the vocabulary
+    True
+    >>> iv_vector = wv['landlady']
+
 Retrieve word-vector for vocab and out-of-vocab word:
 
 .. sourcecode:: pycon
@@ -417,7 +433,7 @@ class FastText(BaseWordEmbeddingsModel):
 
     The model can be stored/loaded via its :meth:`~gensim.models.fasttext.FastText.save` and
     :meth:`~gensim.models.fasttext.FastText.load` methods, or loaded from a format compatible with the original
-    Fasttext implementation via :meth:`~gensim.models.fasttext.FastText.load_fasttext_format`.
+    Fasttext implementation via :func:`~gensim.models.fasttext.load_facebook_model`.
 
     Attributes
     ----------
@@ -879,14 +895,6 @@ def train(self, sentences=None, corpus_file=None, total_examples=None, total_wor
             >>> model.train(sentences, total_examples=model.corpus_count, epochs=model.epochs)
 
         """
-        cant_train = hasattr(self.trainables, 'syn1neg') and self.trainables.syn1neg is None
-        if cant_train:
-            raise ValueError(
-                'this model cannot be trained any further, '
-                'if this is a native model, try loading it with '
-                'FastText.load_fasttext_format(path, full_model=True)'
-            )
-
         super(FastText, self).train(
             sentences=sentences, corpus_file=corpus_file, total_examples=total_examples, total_words=total_words,
             epochs=epochs, start_alpha=start_alpha, end_alpha=end_alpha, word_count=word_count,
@@ -937,82 +945,23 @@ def __contains__(self, word):
         return self.wv.__contains__(word)
 
     @classmethod
-    def load_fasttext_format(cls, model_file, encoding='utf8', full_model=True):
-        """Load the input-hidden weight matrix from Facebook's native fasttext `.bin` and `.vec` output files.
-
-        By default, this function loads the full model.  A full model allows
-        continuing training with more data, but also consumes more RAM and
-        takes longer to load.  If you do not need to continue training and only
-        wish the work with the already-trained embeddings, use `full_model=False`
-        for faster loading and to save RAM.
-
-        Notes
-        ------
-        Facebook provides both `.vec` and `.bin` files with their modules.
-        The former contains human-readable vectors.
-        The latter contains machine-readable vectors along with other model parameters.
-        This function effectively ignores `.vec` output file, since that file is redundant.
-        It only needs the `.bin` file.
-
-        Parameters
-        ----------
-        model_file : str
-            Path to the FastText output files.
-            FastText outputs two model files - `/path/to/model.vec` and `/path/to/model.bin`
-            Expected value for this example: `/path/to/model` or `/path/to/model.bin`,
-            as Gensim requires only `.bin` file to the load entire fastText model.
-        encoding : str, optional
-            Specifies the file encoding.
-        full_model : boolean, optional
-            If False, skips loading the hidden output matrix. This saves a fair bit
-            of CPU time and RAM, but **prevents training continuation**.
-
-        Examples
-        --------
-
-        Load, infer, continue training:
-
-        .. sourcecode:: pycon
-
-            >>> from gensim.test.utils import datapath
-            >>>
-            >>> cap_path = datapath("crime-and-punishment.bin")
-            >>> fb_full = FastText.load_fasttext_format(cap_path, full_model=True)
-            >>>
-            >>> 'landlord' in fb_full.wv.vocab  # Word is out of vocabulary
-            False
-            >>> oov_term = fb_full.wv['landlord']
-            >>>
-            >>> 'landlady' in fb_full.wv.vocab  # Word is in the vocabulary
-            True
-            >>> iv_term = fb_full.wv['landlady']
-            >>>
-            >>> new_sent = [['lord', 'of', 'the', 'rings'], ['lord', 'of', 'the', 'flies']]
-            >>> fb_full.build_vocab(new_sent, update=True)
-            >>> fb_full.train(sentences=new_sent, total_examples=len(new_sent), epochs=5)
-
-        Load quickly, infer (forego training continuation):
-
-        .. sourcecode:: pycon
-
-            >>> fb_partial = FastText.load_fasttext_format(cap_path, full_model=False)
-            >>>
-            >>> 'landlord' in fb_partial.wv.vocab  # Word is out of vocabulary
-            False
-            >>> oov_term = fb_partial.wv['landlord']
-            >>>
-            >>> 'landlady' in fb_partial.wv.vocab  # Word is in the vocabulary
-            True
-            >>> iv_term = fb_partial.wv['landlady']
+    @deprecated(
+        'use load_facebook_vectors (to use pretrained embeddings) or load_facebook_model '
+        '(to continue training with the loaded full model, more RAM) instead'
+    )
+    def load_fasttext_format(cls, model_file, encoding='utf8'):
+        """Deprecated.
 
-        Returns
-        -------
-        gensim.models.fasttext.FastText
-            The loaded model.
+        Use :func:`gensim.models.fasttext.load_facebook_model` or
+        :func:`gensim.models.fasttext.load_facebook_vectors` instead.
 
         """
-        return _load_fasttext_format(model_file, encoding=encoding, full_model=full_model)
+        return load_facebook_model(model_file, encoding=encoding)
 
+    @deprecated(
+        'use load_facebook_vectors (to use pretrained embeddings) or load_facebook_model '
+        '(to continue training with the loaded full model, more RAM) instead'
+    )
     def load_binary_data(self, encoding='utf8'):
         """Load data from a binary file created by Facebook's native FastText.
 
@@ -1220,6 +1169,105 @@ def _pad_ones(m, new_shape):
     return vstack([m, suffix])
 
 
+def load_facebook_model(path, encoding='utf-8'):
+    """Load the input-hidden weight matrix from Facebook's native fasttext `.bin` output file.
+
+    Notes
+    ------
+    Facebook provides both `.vec` and `.bin` files with their modules.
+    The former contains human-readable vectors.
+    The latter contains machine-readable vectors along with other model parameters.
+    This function effectively ignores `.vec` output file, since that file is redundant.
+    It only needs the `.bin` file.
+
+    Parameters
+    ----------
+    model_file : str
+        Path to the FastText output files.
+        FastText outputs two model files - `/path/to/model.vec` and `/path/to/model.bin`
+        Expected value for this example: `/path/to/model` or `/path/to/model.bin`,
+        as Gensim requires only `.bin` file to the load entire fastText model.
+    encoding : str, optional
+        Specifies the file encoding.
+
+    Examples
+    --------
+
+    Load, infer, continue training:
+
+    .. sourcecode:: pycon
+
+        >>> from gensim.test.utils import datapath
+        >>>
+        >>> cap_path = datapath("crime-and-punishment.bin")
+        >>> fb_model = load_facebook_model(cap_path)
+        >>>
+        >>> 'landlord' in fb_model.wv.vocab  # Word is out of vocabulary
+        False
+        >>> oov_term = fb_model.wv['landlord']
+        >>>
+        >>> 'landlady' in fb_model.wv.vocab  # Word is in the vocabulary
+        True
+        >>> iv_term = fb_model.wv['landlady']
+        >>>
+        >>> new_sent = [['lord', 'of', 'the', 'rings'], ['lord', 'of', 'the', 'flies']]
+        >>> fb_model.build_vocab(new_sent, update=True)
+        >>> fb_model.train(sentences=new_sent, total_examples=len(new_sent), epochs=5)
+
+    Returns
+    -------
+    gensim.models.fasttext.FastText
+        The loaded model.
+
+    """
+    return _load_fasttext_format(path, encoding=encoding, full_model=True)
+
+
+def load_facebook_vectors(path, encoding='utf-8'):
+    """Load word embeddings from a model saved in Facebook's native fasttext `.bin` format.
+
+    Parameters
+    ----------
+    path : str
+        The location of the model file.
+    encoding : str, optional
+        Specifies the file encoding.
+
+    Returns
+    -------
+    gensim.models.keyedvectors.FastTextKeyedVectors
+        The word embeddings.
+
+    Examples
+    --------
+
+    Load and infer:
+
+        >>> from gensim.test.utils import datapath
+        >>>
+        >>> cap_path = datapath("crime-and-punishment.bin")
+        >>> fbkv = load_facebook_vectors(cap_path)
+        >>>
+        >>> 'landlord' in fbkv.vocab  # Word is out of vocabulary
+        False
+        >>> oov_vector = fbkv['landlord']
+        >>>
+        >>> 'landlady' in fbkv.vocab  # Word is in the vocabulary
+        True
+        >>> iv_vector = fbkv['landlady']
+
+    See Also
+    --------
+
+    :meth:`gensim.models.fasttext.FastText.load_facebook_model` loads
+    the full model, not just word embeddings, and enables you to continue
+    model training.
+
+    """
+    model_wrapper = _load_fasttext_format(path, encoding=encoding, full_model=False)
+    return model_wrapper.wv
+
+
 def _load_fasttext_format(model_file, encoding='utf-8', full_model=True):
     """Load the input-hidden weight matrix from Facebook's native fasttext `.bin` and `.vec` output files.
 


Test patch:
diff --git a/gensim/test/test_fasttext.py b/gensim/test/test_fasttext.py
--- a/gensim/test/test_fasttext.py
+++ b/gensim/test/test_fasttext.py
@@ -20,6 +20,8 @@
 from gensim.models.keyedvectors import Word2VecKeyedVectors
 from gensim.test.utils import datapath, get_tmpfile, temporary_file, common_texts as sentences
 
+import gensim.models.fasttext
+
 try:
     from pyemd import emd  # noqa:F401
     PYEMD_EXT = True
@@ -56,24 +58,9 @@ def setUp(self):
         ft_home = os.environ.get('FT_HOME', None)
         self.ft_path = os.path.join(ft_home, 'fasttext') if ft_home else None
         self.test_model_file = datapath('lee_fasttext')
-        self.test_model = FT_gensim.load_fasttext_format(self.test_model_file)
+        self.test_model = gensim.models.fasttext.load_facebook_model(self.test_model_file)
         self.test_new_model_file = datapath('lee_fasttext_new')
 
-    def test_native_partial_model(self):
-        """Can we skip loading the NN and still get a working model?"""
-        model = FT_gensim.load_fasttext_format(self.test_model_file, full_model=False)
-
-        #
-        # Training continuation should be impossible
-        #
-        self.assertIsNone(model.trainables.syn1neg)
-        self.assertRaises(ValueError, model.train, sentences,
-                          total_examples=model.corpus_count, epochs=model.epochs)
-
-        model.wv['green']
-        model.wv['foobar']
-        model.wv['thisworddoesnotexist']
-
     def test_training(self):
         model = FT_gensim(size=10, min_count=1, hs=1, negative=0, seed=42, workers=1)
         model.build_vocab(sentences)
@@ -202,7 +189,7 @@ def model_sanity(self, model):
 
     def test_load_fasttext_format(self):
         try:
-            model = FT_gensim.load_fasttext_format(self.test_model_file)
+            model = gensim.models.fasttext.load_facebook_model(self.test_model_file)
         except Exception as exc:
             self.fail('Unable to load FastText model from file %s: %s' % (self.test_model_file, exc))
         vocab_size, model_size = 1762, 10
@@ -255,7 +242,7 @@ def test_load_fasttext_format(self):
 
     def test_load_fasttext_new_format(self):
         try:
-            new_model = FT_gensim.load_fasttext_format(self.test_new_model_file)
+            new_model = gensim.models.fasttext.load_facebook_model(self.test_new_model_file)
         except Exception as exc:
             self.fail('Unable to load FastText model from file %s: %s' % (self.test_new_model_file, exc))
         vocab_size, model_size = 1763, 10
@@ -308,10 +295,10 @@ def test_load_fasttext_new_format(self):
 
     def test_load_model_supervised(self):
         with self.assertRaises(NotImplementedError):
-            FT_gensim.load_fasttext_format(datapath('pang_lee_polarity_fasttext'))
+            gensim.models.fasttext.load_facebook_model(datapath('pang_lee_polarity_fasttext'))
 
     def test_load_model_with_non_ascii_vocab(self):
-        model = FT_gensim.load_fasttext_format(datapath('non_ascii_fasttext'))
+        model = gensim.models.fasttext.load_facebook_model(datapath('non_ascii_fasttext'))
         self.assertTrue(u'který' in model.wv)
         try:
             model.wv[u'který']
@@ -319,7 +306,7 @@ def test_load_model_with_non_ascii_vocab(self):
             self.fail('Unable to access vector for utf8 encoded non-ascii word')
 
     def test_load_model_non_utf8_encoding(self):
-        model = FT_gensim.load_fasttext_format(datapath('cp852_fasttext'), encoding='cp852')
+        model = gensim.models.fasttext.load_facebook_model(datapath('cp852_fasttext'), encoding='cp852')
         self.assertTrue(u'který' in model.wv)
         try:
             model.wv[u'který']
@@ -891,7 +878,7 @@ def load_native():
     # ./fasttext cbow -input toy-data.txt -output toy-model -bucket 100 -dim 5
     #
     path = datapath('toy-model.bin')
-    model = FT_gensim.load_fasttext_format(path)
+    model = gensim.models.fasttext.load_facebook_model(path)
     return model
 
 
@@ -1115,11 +1102,20 @@ def test_save_load_native(self):
             model.save(model_name)
 
     def test_load_native_pretrained(self):
-        model = FT_gensim.load_fasttext_format(datapath('toy-model-pretrained.bin'))
+        model = gensim.models.fasttext.load_facebook_model(datapath('toy-model-pretrained.bin'))
         actual = model['monarchist']
         expected = np.array([0.76222, 1.0669, 0.7055, -0.090969, -0.53508])
         self.assertTrue(np.allclose(expected, actual, atol=10e-4))
 
+    def test_load_native_vectors(self):
+        cap_path = datapath("crime-and-punishment.bin")
+        fbkv = gensim.models.fasttext.load_facebook_vectors(cap_path)
+        self.assertFalse('landlord' in fbkv.vocab)
+        self.assertTrue('landlady' in fbkv.vocab)
+        oov_vector = fbkv['landlord']
+        iv_vector = fbkv['landlady']
+        self.assertFalse(np.allclose(oov_vector, iv_vector))
+
 
 def _train_model_with_pretrained_vectors():
     """Generate toy-model-pretrained.bin for use in test_load_native_pretrained.
@@ -1170,7 +1166,7 @@ def setUp(self):
         #
         # ./fasttext skipgram -minCount 0 -bucket 100 -input crime-and-punishment.txt -output crime-and-punishment -dim 5  # noqa: E501
         #
-        self.model = FT_gensim.load_fasttext_format(datapath('crime-and-punishment.bin'))
+        self.model = gensim.models.fasttext.load_facebook_model(datapath('crime-and-punishment.bin'))
         with smart_open.smart_open(datapath('crime-and-punishment.vec'), 'r', encoding='utf-8') as fin:
             self.expected = dict(load_vec(fin))