UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte
Author: math-sasso
Pull request: no

## Description
I have fine tunned a transformers model ('bert-base') and saved the checkpoint. Now I want to use somewhere with the checkpoints I saved. But If I still use bert-base on my new env it will not retrieve my checkpoints, so I want to know the proper way to make my model (Pytorch Lightning model) understand that I want to retrieve my checkpoint


## Relavant info on my model class

```py
class NER_Model(pl.LightningModule):
    def __init__(self, hyperparams, model_parameters, dataset_infos, extra_infos):

        super(NER_Model, self).__init__()

        self.model_name = "/checkpoins_folder/epoch=2-step=167-v1.ckpt"
        self.model = AutoModelForTokenClassification.from_pretrained(
          self.model_name,
          num_labels=7,
          output_attentions = False,
          output_hidden_states = False
          )

    def predict(self, X: str):
        self.step = "Deployment"
        self.test_pred_tags = []

        batch = self.tokenizer.encode_plus(X, return_tensors="pt")
        batch["attention_masks"] = torch.ones_like(batch["input_ids"])
        batch = dict((key, input.to( self.device)) for key, input in batch.items())

        return self.test_step(batch, None)


## Call
model = NER_Model.load_from_checkpoint(
    checkpoint_path = "/checkpoins_folder/epoch=2-step=167-v1.ckpt",
    map_location={"cuda":"cpu"},
    hyperparams=hyperparams,
    model_parameters=model_parameters,
    dataset_infos=dataset_infos,
    extra_infos=extra_infos,  
)
```

##Error:

```out
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-26-c4ae8a2442c3> in <module>()
      6     model_parameters=model_parameters,
      7     dataset_infos=dataset_infos,
----> 8     extra_infos=extra_infos,
      9 )

7 frames
/usr/local/lib/python3.7/dist-packages/pytorch_lightning/core/saving.py in load_from_checkpoint(cls, checkpoint_path, map_location, hparams_file, strict, **kwargs)
    154         checkpoint[cls.CHECKPOINT_HYPER_PARAMS_KEY].update(kwargs)
    155 
--> 156         model = cls._load_model_state(checkpoint, strict=strict, **kwargs)
    157         return model
    158 

/usr/local/lib/python3.7/dist-packages/pytorch_lightning/core/saving.py in _load_model_state(cls, checkpoint, strict, **cls_kwargs_new)
    196             _cls_kwargs = {k: v for k, v in _cls_kwargs.items() if k in cls_init_args_name}
    197 
--> 198         model = cls(**_cls_kwargs)
    199 
    200         # give model a chance to load something

<ipython-input-10-ed914cb098e8> in __init__(self, hyperparams, model_parameters, dataset_infos, extra_infos)
     61           num_labels=len(self.tags_infos_dict["tag2idx"]),
     62           output_attentions = self.output_attentions,
---> 63           output_hidden_states = self.output_hidden_states
     64           )
     65 

/usr/local/lib/python3.7/dist-packages/transformers/models/auto/modeling_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1648         if not isinstance(config, PretrainedConfig):
   1649             config, kwargs = AutoConfig.from_pretrained(
-> 1650                 pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
   1651             )
   1652 

/usr/local/lib/python3.7/dist-packages/transformers/models/auto/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    366             {'foo': False}
    367         """
--> 368         config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
    369 
    370         if "model_type" in config_dict:

/usr/local/lib/python3.7/dist-packages/transformers/configuration_utils.py in get_config_dict(cls, pretrained_model_name_or_path, **kwargs)
    425             )
    426             # Load config dict
--> 427             config_dict = cls._dict_from_json_file(resolved_config_file)
    428 
    429         except EnvironmentError as err:

/usr/local/lib/python3.7/dist-packages/transformers/configuration_utils.py in _dict_from_json_file(cls, json_file)
    508     def _dict_from_json_file(cls, json_file: Union[str, os.PathLike]):
    509         with open(json_file, "r", encoding="utf-8") as reader:
--> 510             text = reader.read()
    511         return json.loads(text)
    512 

/usr/lib/python3.7/codecs.py in decode(self, input, final)
    320         # decode input (taking the buffer into account)
    321         data = self.buffer + input
--> 322         (result, consumed) = self._buffer_decode(data, self.errors, final)
    323         # keep undecoded input until the next call
    324         self.buffer = data[consumed:]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte
```


Comments:
You're using `AutoModelForTokenClassification` but you're specifying a `ckpt` file. It needs a folder with a `config.json`, and a state dict. Here it's expecting a PyTorch state dict named `pytorch_model.bin` since you're using the PyTorch version.

If you've retrieved your checkpoint from the original BERT repository, I recommend you take a look at the following documentation [Converting Tensorflow Checkpoints](https://huggingface.co/transformers/converting_tensorflow_models.html).

Also, you're using a PyTorch model's `from_pretrained` model, so I point you to the documentation of that method [here](https://huggingface.co/transformers/main_classes/model.html#transformers.PreTrainedModel.from_pretrained).

@LysandreJik first of all thanks for replying.

Is it the proper way to retrieve a model from a checkpoint generated by me? When I pass `self.model=bert-base` on my new env it doesnt retrieve my checkpoints, so when I call `model.precit('any text ')` it gives me wrong results [maybe because it gets original weights from bert-base instead of my finne tunned weights].

So following your suggestion, if changing `self.model` for my finne tunned model is the correct way, how do I generate `config.json` and  `pytorch_model.bin` from my **trained model**

Well you should use the `save_pretrained` method on your model. I haven't used PyTorch Lightning but if your module is named `model` and that the transformers is the `model` attribute of that module, it would be something like:

```py
model.model.save_pretrained("directory")
```

I did it and I worked. But it stills giving wrong results when I call the predict. On my model I have inserted the path to the created dir instead of bert-base.

 self.model = AutoModelForTokenClassification.from_pretrained(
          "/content/drive/MyDrive/CityZen/Explorations/BERT/weights/binaries",
          num_labels=len(self.tags_infos_dict["tag2idx"]),
          output_attentions = self.output_attentions,
          output_hidden_states = self.output_hidden_states
          )