T5ForConditionalGeneration.from_pretrained load pytorch *.pt checkpoint fails
Author: Crista23
Pull request: no

I am using T5ForConditionalGeneration and want to load a custom *.pt checkpoint:

       model = T5ForConditionalGeneration.from_pretrained(my_custom_checkpoint) 
       model = model.to(device)

However, I am getting the error below:

        File "/home/usr/lib/python3.7/site-packages/transformers/modeling_utils.py", line 962, in from_pretrained
    **kwargs,
        File "/home/usr/lib/python3.7/site-packages/transformers/configuration_utils.py", line 372, in from_pretrained
        config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
        File "/home/usr/anaconda2/envs/maml_adaptation/lib/python3.7/site-packages/transformers/configuration_utils.py", line 423, in get_config_dict
        config_dict = cls._dict_from_json_file(resolved_config_file)
        File "/home/usr/anaconda2/envs/maml_adaptation/lib/python3.7/site-packages/transformers/configuration_utils.py", line 506, in _dict_from_json_file
        text = reader.read()
        File "/home/usr/anaconda2/envs/maml_adaptation/lib/python3.7/codecs.py", line 322, in decode
        (result, consumed) = self._buffer_decode(data, self.errors, final)
        UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

What is causing this error? I can load the *.pt checkpoint fine using torch.load, but how to do it using T5ForConditionalGeneration? Thank you!

Comments:
The `from_pretrained` method expects `pytorch_mode.bin` file in the checkpoint folder, if you have saved your checkpoint with a different name, then you could initialize the model with the config and load the weights from `state_dict`

```python
model = T5ForConditionalGeneration.from_config("....")
model.load_state_dict(torch_state_dict)

# save the model with save_pretrained, so next time it can be loaded using  from_pretrained
model.save_pretrained("....")
```

Also looking at the stack trace, there seems to be some issue with your config file, could you maybe check if it contains valid JSON ?


Thank you for your response. How do I convert from a *.pt checkpoint to a *.bin checkpoint in Pytorch? Does the transformers  library provide any such utility function? Thank you!

@Crista23, Simply rename your `.pt` checkpoint into `pytorch_model.bin` - it's a different name for the same file.

@LysandreJik Thank you for the answer! I have renamed the checkpoint I am trying to load from *.pt to *.bin, but I get the same UnicodeDecodeError.

`UnicodeDecodeError` error seems related to `config.json`, could you verify that it contains valid JSON, try to just load the json or maybe post the file, so we could take a look.

Hi @patil-suraj , thanks for your response. Where is this config.json supposed to be found? I keep on looking and cannot find it among my files.

It should be in the directory where you saved your checkpoint. the `save_pretrained` saves that file in the given directory.

@patil-suraj thank you for the clarification! Unfortunately there is no config.json file in the saved checkpoints directory. What I am doing is take t5-small, adapt it for my own purposes using the fairseq library, and then try to load the latest fairseq checkpoint into T5ForConditionalGeneration (which gives me the error) - the format in which fairseq saves checkpoints is a directory containing *.pt checkpoints only. 

I see in that case, see if you can save the model using `.save_pretrained` as well, which will let you use `from_pretrained`.

If it's a t5-small model then you could load the config and pass it to `from_pretrained`, for example

```python
config = T5Config.from_pretrained("t5-small")
model = T5ForConditionalGeneration.from_pretrained("path_to_model_dir", config=config)
```

this assumes you have at least `pytorch_model.bin` in the model dir. If not then rename the `.pt` file to `.bin` file as Lysandre suggested.

Thank you so much @patil-suraj, that fixed it! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.