unable to load the downloaded BERT model  offline in local  machine . could not  find config.json and  Error no file named ['pytorch_model.bin', 'tf_model.h5', 'model.ckpt.index'] |
Author: AjitAntony
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
I have downloaded the bert model [from the link in bert github page](https://storage.googleapis.com/bert_models/2018_10_18/cased_L-12_H-768_A-12.zip) offline but unable to load the model offline  .

from transformers import *
model = BertForMaskedLM.from_pretrained("/Users/Downloads/uncased_L-12_H-768_A-12/")



 Model name '/Users/Downloads/uncased_L-12_H-768_A-12/' was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased). We assumed '/Users/Downloads/uncased_L-12_H-768_A-12/' was a path or url to a configuration file named config.json or a directory containing such a file but couldn't find any such file at this path or url.

below are the files present in /Users/Downloads/uncased_L-12_H-768_A-12/ 
bert_config.json
bert_model.ckpt.data-00000-of-00001
bert_model.ckpt.index 
bert_model.ckpt.meta
vocab.txt
what should i do to load the downloaded model offline ?


since the error was saying config.json not found i changed the above 4 file names by removing the word bert from it.Below are the new file names 

config.json
model.ckpt.data-00000-of-00001
model.ckpt.index 
model.ckpt.meta
vocab.txt
now when i  load the downloaded model offline i get a different error 

from transformers import *
model = BertForMaskedLM.from_pretrained("/Users/Downloads/uncased_L-12_H-768_A-12/")

Error no file named ['pytorch_model.bin', 'tf_model.h5', 'model.ckpt.index'] found in directory /Users/Downloads/uncased_L-12_H-768_A-12/ or `from_tf` set to False


python version:3.7
tensorflow version:1.12



Comments:
Hi, you're downloading one of the original implementation BERT models, which is in TensorFlow and you are trying to load it into one of our Pytorch models. 

You can either download one of our checkpoints hosted on our S3 with:

```py
from transformers import BertForMaskedLM

model = BertForMaskedLM.from_pretrained("bert-base-cased")
```

This model will now be available offline as it will be saved in your pytorch cache.

Or you can convert the BERT model you downloaded to a checkpoint readable by our library by using the script [convert_bert_original_tf_checkpoint_to_pytorch.py](https://github.com/huggingface/transformers/blob/master/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py)

@LysandreJik thank you . 
**How to differentiate Bert tensor flow and pytorch models ?**

I followed the instruction and create a PyTorch model using this pyhton code ->convert_bert_original_tf_checkpoint_to_pytorch.py

INFO:transformers.modeling_bert:Initialize PyTorch weight ['cls', 'seq_relationship', 'output_weights']
INFO:transformers.modeling_bert:Skipping cls/seq_relationship/output_weights/adam_m
INFO:transformers.modeling_bert:Skipping cls/seq_relationship/output_weights/adam_v
INFO:transformers.modeling_bert:Skipping global_step
Save PyTorch model to /content/drive/My Drive/BMaskLang

the BMaskLang file was 402 MB size and it did not have any file extension ,now when i tired to load this pytorch model i get an error

_from transformers import BertForMaskedLM
model = BertForMaskedLM.from_pretrained("/content/drive/My Drive/BMaskLang")
Error:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte_


Basically what im trying to do is train a  BertForMaskedLM on a custom corpus .

what are the steps to train BertForMaskedLM model on custom corpus and 
After train model how to load it ?
After loading how to test it on a new sentence ?

For example if there was a sentence in sample_text.txt corpus like
"He went to space.He brought a moon"

if i want to test my pretrained BertForMaskedLM to check if it correctly predicts the masked word in sentences" He went to [Mask] .He brought a gallon [Mask]

so the model must predict the same words which was in sample_text.txt corpus "space","moon" rather than other words like "store","water" since it was trained on this sample_text.txt corpus .im expecting this behavior .Is this possible to pretrain and build language model using transformers  bert ?

 

> @LysandreJik thank you .
> **How to differentiate Bert tensor flow and pytorch models ?**
> 
> I followed the instruction and create a PyTorch model using this pyhton code ->convert_bert_original_tf_checkpoint_to_pytorch.py
> 
> INFO:transformers.modeling_bert:Initialize PyTorch weight ['cls', 'seq_relationship', 'output_weights']
> INFO:transformers.modeling_bert:Skipping cls/seq_relationship/output_weights/adam_m
> INFO:transformers.modeling_bert:Skipping cls/seq_relationship/output_weights/adam_v
> INFO:transformers.modeling_bert:Skipping global_step
> Save PyTorch model to /content/drive/My Drive/BMaskLang
> 
> the BMaskLang file was 402 MB size and it did not have any file extension ,now when i tired to load this pytorch model i get an error
> 
> _from transformers import BertForMaskedLM model = BertForMaskedLM.from_pretrained("/content/drive/My Drive/BMaskLang") Error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte_
> 
> Basically what im trying to do is train a BertForMaskedLM on a custom corpus .
> 
> what are the steps to train BertForMaskedLM model on custom corpus and
> After train model how to load it ?
> After loading how to test it on a new sentence ?
> 
> For example if there was a sentence in sample_text.txt corpus like
> "He went to space.He brought a moon"
> 
> if i want to test my pretrained BertForMaskedLM to check if it correctly predicts the masked word in sentences" He went to [Mask] .He brought a gallon [Mask]
> 
> so the model must predict the same words which was in sample_text.txt corpus "space","moon" rather than other words like "store","water" since it was trained on this sample_text.txt corpus .im expecting this behavior .Is this possible to pretrain and build language model using transformers bert ?

I haved the same problem that how to load bert model yesterday. And now I found the solution. 
1. run  convert_bert_original_tf_checkpoint_to_pytorch.py to create pytorch_model.bin
2. rename bert_config.json to config.json

after that, the dictionary must have

config.json (BertForMaskedLM.from_pretrained() need it)
pytorch_model.bin (BertForMaskedLM.from_pretrained() need it)
vocab.txt (BertTokenizer.from_pretrained() need it)

python version 3.7
pytorch version 1.3.1
tensorflow version 2.0.0

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


> Hi, you're downloading one of the original implementation BERT models, which is in TensorFlow and you are trying to load it into one of our Pytorch models.
> 
> You can either download one of our checkpoints hosted on our S3 with:
> 
> ```python
> from transformers import BertForMaskedLM
> 
> model = BertForMaskedLM.from_pretrained("bert-base-cased")
> ```
> 
> This model will now be available offline as it will be saved in your pytorch cache.
> 
> Or you can convert the BERT model you downloaded to a checkpoint readable by our library by using the script [convert_bert_original_tf_checkpoint_to_pytorch.py](https://github.com/huggingface/transformers/blob/master/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py)

hi, I can't open this link https://github.com/huggingface/transformers/blob/master/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py, where can I find this script? Thanks!

It's [here](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py)

Traceback (most recent call last):
File "/users/sroychou/BERT_text_summarisation/scripts/train_bert_summarizer.py", line 12, in
from metrics import optimizer, loss_function, label_smoothing, get_loss_and_accuracy, tf_write_summary, monitor_run
File "/users/sroychou/BERT_text_summarisation/scripts/metrics.py", line 16, in
_, _, _ = b_score(["I'm Batman"], ["I'm Spiderman"], lang='en', model_type='bert-base-uncased')
File "/users/sroychou/.local/lib/python3.7/site-packages/bert_score/score.py", line 105, in score
tokenizer = AutoTokenizer.from_pretrained(model_type)
File "/users/sroychou/.local/lib/python3.7/site-packages/transformers/tokenization_auto.py", line 298, in from_pretrained
config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
File "/users/sroychou/.local/lib/python3.7/site-packages/transformers/configuration_auto.py", line 330, in from_pretrained
config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/users/sroychou/.local/lib/python3.7/site-packages/transformers/configuration_utils.py", line 382, in get_config_dict
raise EnvironmentError(msg)
OSError: Can't load config for 'bert-base-uncased'. Make sure that:

'bert-base-uncased' is a correct model identifier listed on 'https://huggingface.co/models'

or 'bert-base-uncased' is the correct path to a directory containing a config.json file