MobileBert form google-research added
Author: mrm8488
Pull request: no
Labels: name: New model

# 🌟 New model addition

## Model description

[MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices](https://arxiv.org/abs/2004.02984) 
Repo [link](https://github.com/google-research/google-research/tree/master/mobilebert)

## Open source status

* [x] Uploaded to HF model hub 
[bert_config.json](https://s3.amazonaws.com/models.huggingface.co/bert/mrm8488/MobileBERT/bert_config.json)
[vocab.txt](https://s3.amazonaws.com/models.huggingface.co/bert/mrm8488/MobileBERT/vocab.txt)
[mobilebert_variables.ckpt.data-00000-of-00001](https://s3.amazonaws.com/models.huggingface.co/bert/mrm8488/MobileBERT/mobilebert_variables.ckpt.data-00000-of-00001)
[mobilebert_variables.ckpt.index](https://s3.amazonaws.com/models.huggingface.co/bert/mrm8488/MobileBERT/mobilebert_variables.ckpt.index)
[checkpoint](https://s3.amazonaws.com/models.huggingface.co/bert/mrm8488/MobileBERT/checkpoint)
* [ ] Convert model checkpoint to pytorch (or Keras h5, savedModel)

The [script](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py) to convert BERT models to Pytorch fails. Based on the error output the error comes from modeling [script](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py):
```
File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_bert.py", line 77, in load_tf_weights_in_bert
```
Whole error:
```
INFO:transformers.modeling_bert:Converting TensorFlow checkpoint from /content/mobilebert
Traceback (most recent call last):
  File "/content/convert_bert_original_tf_checkpoint_to_pytorch.py", line 61, in <module>
    convert_tf_checkpoint_to_pytorch(args.tf_checkpoint_path, args.bert_config_file, args.pytorch_dump_path)
  File "/content/convert_bert_original_tf_checkpoint_to_pytorch.py", line 36, in convert_tf_checkpoint_to_pytorch
    load_tf_weights_in_bert(model, config, tf_checkpoint_path)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_bert.py", line 77, in load_tf_weights_in_bert
    init_vars = tf.train.list_variables(tf_path)
  File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/checkpoint_utils.py", line 97, in list_variables
    reader = load_checkpoint(ckpt_dir_or_file)
  File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/checkpoint_utils.py", line 62, in load_checkpoint
    filename = _get_checkpoint_filename(ckpt_dir_or_file)
  File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/checkpoint_utils.py", line 384, in _get_checkpoint_filename
    return checkpoint_management.latest_checkpoint(ckpt_dir_or_file)
  File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/checkpoint_management.py", line 341, in latest_checkpoint
    if file_io.get_matching_files(v2_path) or file_io.get_matching_files(
  File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/lib/io/file_io.py", line 363, in get_matching_files
    return get_matching_files_v2(filename)
  File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/lib/io/file_io.py", line 384, in get_matching_files_v2
    compat.as_bytes(pattern))
tensorflow.python.framework.errors_impl.NotFoundError: /cns/tp-d/home/tensorflow-tpus/hongkuny/bert/pretrained_models/uncased_L-24_H-128_B-512_A-4_F-4_OPT/oss; No such file or directory
```
Based on last line of the error log:
```
tensorflow.python.framework.errors_impl.NotFoundError: /cns/tp-d/home/tensorflow-tpus/hongkuny/bert/pretrained_models/uncased_L-24_H-128_B-512_A-4_F-4_OPT/oss; No such file or directory
```
It seems TF keep a kind for directory with the paths of their *availiable* models and this one is not there yet.
But maybe we can fix it using the modeling script in the repo of MobileBert: https://github.com/google-research/google-research/blob/master/mobilebert/modeling.py

Thougths?


Comments:
Will close this to track this at more general #4185

Please the " checkpoint" file to reset the path to "mobilebert_variables.ckpt" 
This can solved my issue.