Download the model without executing a Python script
Author: lcswillems
Pull request: no

Hi,

Is there a command to download a model (e.g. BertForMaskedLM) without having to execute a Python script?

For example, in Spacy, we can do `python -m spacy download en`.

Comments:
Is this what you want?

```python
PRETRAINED_MODEL_ARCHIVE_MAP = {
    'bert-base-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz",
    'bert-large-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-uncased.tar.gz",
    'bert-base-cased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased.tar.gz",
    'bert-large-cased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-cased.tar.gz",
    'bert-base-multilingual-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-uncased.tar.gz",
    'bert-base-multilingual-cased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-cased.tar.gz",
    'bert-base-chinese': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-chinese.tar.gz",
}
```

from https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/pytorch_pretrained_bert/modeling.py

I feel like I'm not exactly understanding your question. They have hosted the pytorch dumps of the pretrained BERT models that were released by Google and hosted them on AWS. After downloading the model, what do you wish to do with it?

Thank you for the answer.

So with your code, I now have the url. But,  where should I put the files in my filesystem?

Spacy provide a command to download the weights, put them at the good location in the filesystem, and use them. Is there an equivalent in your repository?

For example, [DeepPavlov](https://github.com/deepmipt/DeepPavlov) provides this command:

```
python -m deeppavlov install -d squad_bert
```

to install and download a model.

Does my question make sense?

Not really. What is the reason you are trying to do that?
This library will automatically download pre-trained weights, you don't need to do that yourself (even though you can).

> Not really.

Do you understand what Spacy & Deeppavlov enable to do? If yes, I am asking if there is something similar here. But, if you don't understand, it surely means it is not possible.

> What is the reason you are trying to do that?

Because, when I put my code in production, I don't want to make the first query very long because it has to download the model.

I happen to know quite well SpaCy (if you look at the Huggingface github, you will see we have developed a coreference resolution extension for SpaCy, [NeuralCoref](https://github.com/huggingface/neuralcoref), which interfaces directly with the cython internals of SpaCy) so I know the download process they use which is there mainly because they need the models to install as python packages (which we don't need to do here).

You actually shouldn't have to do anything special to avoid a long first query (and we don't do anything special at HuggingFace with the model in production) for the following reason:
The model weights are downloaded and cached when you instantiated the model for the first time and this should be done before the first query is even received. If you create and load the model at each query, you will experience a very heavy overhead, you should avoid that.

If you want to download the weights yourself (you can also do that) you will need to download the weights, configuration and vocabulary files manually from the url that @chrisgzf has pointed and put these in a folder. You can then load the model and tokenizer from that folder as indicated in the readme.

Okay, thank you for your detailed and interesting answer. So there is not the feature I was asking for.

And I miswrote: I didn't want to say "I don't want to make the first query very long", but rather "I don't want to make the first server start very long" for some reasons.