Repository: explosion/spaCy

Issue:
pretrained_vectors meta written inconsistently by CLI
## How to reproduce the behaviour

I would like to build a new model more or less from scratch, using word vectors trained elsewhere:

```
spacy init-model nb path/to/model-with-vectors --vectors-loc input-vectors.txt  [...]
spacy train --vectors path/to/model-with-vectors [...]
```

This works well. But when I try to `spacy.load()` a trained model, it fails with:

`OSError: [E050] Can't find model 'nb_model.vectors'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory. `

If I manually remove `"pretrained_vectors": "nb_model.vectors"` from `{ner,tagger,parser}/cfg`, the model appears to work as expected.


<details>
  <summary>Full stack trace</summary>

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/__init__.py", line 21, in load
    return util.load_model(name, **overrides)
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/util.py", line 116, in load_model
    return load_model_from_path(Path(name), **overrides)
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/util.py", line 156, in load_model_from_path
    return nlp.from_disk(model_path)
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/language.py", line 647, in from_disk
    util.from_disk(path, deserializers, exclude)
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/util.py", line 511, in from_disk
    reader(path / key)
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/language.py", line 643, in <lambda>
    deserializers[name] = lambda p, proc=proc: proc.from_disk(p, vocab=False)
  File "pipeline.pyx", line 643, in spacy.pipeline.Tagger.from_disk
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/util.py", line 511, in from_disk
    reader(path / key)
  File "pipeline.pyx", line 625, in spacy.pipeline.Tagger.from_disk.load_model
  File "pipeline.pyx", line 535, in spacy.pipeline.Tagger.Model
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/_ml.py", line 447, in build_tagger_model
    pretrained_vectors=pretrained_vectors)
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/_ml.py", line 278, in Tok2Vec
    glove = StaticVectors(pretrained_vectors, width, column=cols.index(ID))
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/thinc/neural/_classes/static_vectors.py", line 41, in __init__
    vectors = self.get_vectors()
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/thinc/neural/_classes/static_vectors.py", line 52, in get_vectors
    return get_vectors(self.ops, self.lang)
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/thinc/extra/load_nlp.py", line 19, in get_vectors
    nlp = get_spacy(lang)
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/thinc/extra/load_nlp.py", line 11, in get_spacy
    SPACY_MODELS[lang] = spacy.load(lang, **kwargs)
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/__init__.py", line 21, in load
    return util.load_model(name, **overrides)
  File "/home/ubuntu/src/spacy-nb/.venv/lib/python3.6/site-packages/spacy/util.py", line 119, in load_model
    raise IOError(Errors.E050.format(name=name))
OSError: [E050] Can't find model 'nb_model.vectors'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.
```
</details>

## Your Environment

## Info about spaCy

* **spaCy version:** 2.1.0a4
* **Platform:** Darwin-18.2.0-x86_64-i386-64bit
* **Python version:** 3.6.1
* **Models:** de, fr


Discussion:
* It appears to only be a problem with non-final models (including `model-best`)
* It can also be fixed by setting vectors.name to "nb_model.vectors" in the top-level meta.json (which is why it works in `model-final`).
@jarib Thanks for the report. I think we should be able to patch this by adjusting the meta in `spacy train`. Would you be able to make a pull request?
Looks like the same problem occurs in `spacy package`, see #3067:

> First reported [on the Prodigy forum](https://support.prodi.gy/t/issue-with-exported-spacy-models/999/3):
>
>> You were right, that was caused by wrong meta.json, which was generated after `spacy package ... --create-meta`, without vector names: `"vectors":{ "width":XYZ, "vectors":XYZ, "keys":XYZ}` . I compared that with meta.json automatically generated from `prodigy textcat.batch-train ... en_core_web_md ` where vector info is stored in following format: `"vectors":{... , "name":"en_core_web_md.vectors"} `
>>
>> I just copied automatically generated meta.json, changed model name to my unique and was able to load my custom model with vectors inside prodigy commands :)
>
> This caused the following error:
>
>> ```
>> OSError: [E050] Can’t find model 'en_core_web_lg.vectors'. It doesn’t seem to be a shortcut link, a Python package or a valid path to a data directory.
>> ```

I'll merge those issues, since this should ideally be part of the same fix!

Resolving patch:
diff --git a/spacy/cli/package.py b/spacy/cli/package.py
--- a/spacy/cli/package.py
+++ b/spacy/cli/package.py
@@ -102,6 +102,7 @@ def generate_meta(model_path, existing_meta, msg):
         "width": nlp.vocab.vectors_length,
         "vectors": len(nlp.vocab.vectors),
         "keys": nlp.vocab.vectors.n_keys,
+        "name": nlp.vocab.vectors.name,
     }
     msg.divider("Generating meta.json")
     msg.text(
diff --git a/spacy/cli/train.py b/spacy/cli/train.py
--- a/spacy/cli/train.py
+++ b/spacy/cli/train.py
@@ -279,6 +279,7 @@ def train(
                     "width": nlp.vocab.vectors_length,
                     "vectors": len(nlp.vocab.vectors),
                     "keys": nlp.vocab.vectors.n_keys,
+                    "name": nlp.vocab.vectors.name
                 }
                 meta.setdefault("name", "model%d" % i)
                 meta.setdefault("version", version)


Test patch: