Imports for Roberta conversion appear to be outdated
Author: mortonjt
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

I'm trying to convert a custom Roberta model (from fairseq checkpoints) to a Tensorflow model.

The problem arise when using:
* [x] the official example scripts: (give details)
* [ ] my own modified scripts: (give details)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details)

## To Reproduce

Steps to reproduce the behavior:


It is possible to load checkpoints saved from Roberta directly?  From the documentation, it looks like it should be possible, but when I run the following code

```python
from transformers import TFRobertaModel
model = TFRobertaModel.from_pretrained('checkpoint_best.pt', from_pt=True)
```

I get
```
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-2-47fa7f7cf639> in <module>()
----> 1 model = TFRobertaModel.from_pretrained('checkpoint_best.pt', from_pt=True)

~/venvs/transformers-tf/lib/python3.6/site-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    208                 cache_dir=cache_dir, return_unused_kwargs=True,
    209                 force_download=force_download,
--> 210                 **kwargs
    211             )
    212         else:

~/venvs/transformers-tf/lib/python3.6/site-packages/transformers/configuration_utils.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    152
    153         # Load config
--> 154         config = cls.from_json_file(resolved_config_file)
    155
    156         if hasattr(config, 'pruned_heads'):

~/venvs/transformers-tf/lib/python3.6/site-packages/transformers/configuration_utils.py in from_json_file(cls, json_file)
    184         """Constructs a `BertConfig` from a json file of parameters."""
    185         with open(json_file, "r", encoding='utf-8') as reader:
--> 186             text = reader.read()
    187         return cls.from_dict(json.loads(text))
    188

/mnt/xfs1/sw/pkg/devel/python3/3.6.2/lib/python3.6/codecs.py in decode(self, input, final)
    319         # decode input (taking the buffer into account)
    320         data = self.buffer + input
--> 321         (result, consumed) = self._buffer_decode(data, self.errors, final)
    322         # keep undecoded input until the next call
    323         self.buffer = data[consumed:]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte
```
I'm guessing since the checkpoint is in a binary, I need to first convert this to a json format.
It looks like it should be done [here](https://github.com/huggingface/transformers/blob/master/transformers/convert_roberta_original_pytorch_checkpoint_to_pytorch.py) .  However, when I try to run that script, I get an error

```
(transformers-tf) [jmorton@pcn-7-01 checkpoints]$ python convert_roberta_original_pytorch_checkpoint_to_pytorch.py  --help
To use data.metrics please install scikit-learn. See https://scikit-learn.org/stable/index.html
Traceback (most recent call last):
  File "convert_roberta_original_pytorch_checkpoint_to_pytorch.py", line 26, in <module>
    from transformers import (BertConfig, BertEncoder,
ImportError: cannot import name 'BertEncoder'
```

From what I can tell, those imports are dated and will need to be fixed anyways.  


## Environment

* OS: Centos 7
* Python version: PY3.6
* PyTorch version: 1.2.0
* PyTorch Transformers version (or branch):2.0.0
* Using GPU ? Not yet
* Distributed of parallel setup ? Nope
* Any other relevant information:


Comments:
Getting the same error here

Will investigate, thanks for reporting. (And Hi, @louismartin :)

I had a similar issue trying to load BioBERT and I figured out what was going on in my case, sharing just in case that's what's going on in your case. In my case I converted TF BioBERT checkpoint to pytorch model. In my case the (first) problem was that I didn't provide a path to the config file. 

My local scripts are adapted from the python code that runs Glue. I have a `--config_name` parameter that specifies the json file from which the configuration is loaded. If you don't provide that one, it tries to infer it by using the model_name_or_path - and that's what caused my problem. 

Once I specified the config file, I had another problem that had to do with the following: `model_name_or_path` is supposed to be the path where you store the other info for the models and the model files are expected to follow a certain naming convention (e.g., in my case, it was looking for a `pytorch_model.bin` file; there is a similar file name for TF). 

Hope this helps!



This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This was superseded, and closed, by #1512