Cammembert fine tuning from checkpoint
Author: UrszulaCzerwinska
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-28-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.3
- PyTorch version (GPU?): 1.3.1 (True)
- Tensorflow version (GPU?): 2.0.0 (False)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: no


### Who can help

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger 

## Information

Model I am using camembert:

The problem arises when using:
* [X] the official example scripts: 
run_language_modeling.py


The tasks I am working on is:
* [X ] an official GLUE/SQUaD task: LM fine tuning
* [X ] my own task or dataset: working with my own data corpus

## To reproduce

Steps to reproduce the behavior:

1. run the script run_language_modeling.py  was first run "from scratch" starting with cammembert model with my own data.
2. launching the same script but starting from checkpoint and the same data to continue training

```bash
python run_language_modeling.py     --output_dir=output2     --model_name_or_path="./LM/fine_tune_cammembert/output/checkpoint-27500"     --tokenizer_name="./sentencepiece.bpe.model"    --do_train     --train_data_file=corpus_camambert/train_1.txt     --do_eval     --eval_data_file=corpus_camambert/test_valid.txt     --mlm     --line_by_line     --evaluate_during_training     --overwrite_output_dir
```

## What happens

```bash
Traceback (most recent call last):
  File "run_language_modeling.py", line 281, in <module>
    main()
  File "run_language_modeling.py", line 186, in main
    tokenizer = AutoTokenizer.from_pretrained(model_args.tokenizer_name, cache_dir=model_args.cache_dir)
  File "/home/ccass/anaconda3/lib/python3.7/site-packages/transformers/tokenization_auto.py", line 209, in from_pretrained
    config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
  File "/home/ccass/anaconda3/lib/python3.7/site-packages/transformers/configuration_auto.py", line 272, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/home/ccass/anaconda3/lib/python3.7/site-packages/transformers/configuration_utils.py", line 355, in get_config_dict
    config_dict = cls._dict_from_json_file(resolved_config_file)
  File "/home/ccass/anaconda3/lib/python3.7/site-packages/transformers/configuration_utils.py", line 437, in _dict_from_json_file
    text = reader.read()
  File "/home/ccass/anaconda3/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 51: invalid start byte
```


## Expected behavior
Continue Taining


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Hello, I still didn't solve the problem... anyone?

It seems the tokenizer you've provided cannot be loaded. You've provided `--tokenizer_name="./sentencepiece.bpe.model"` which is a path to a file, and cannot work with an AutoTokenizer.

I recommend you put the tokenizer file in the same folder as your model, so that it can know the model type, and therefore the tokenizer type, from the configuration.

Also, there was an issue a few months back where the tokenizer wouldn't be saved by the script, and you would have to specify it like you just did. We patched this issue since, so I invite you to use the [`run_mlm.py` script](https://github.com/huggingface/transformers/tree/master/examples/language-modeling) instead, alongside upgrading your `transformers` version to the latest one. Thank you for your understanding.

Ok, thank you I will check it out