text: 
  
  TITLE
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 49: invalid start byte
  
  COMMENTS
  7
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ## Environment info
     
  - `transformers` version: 3.5.1
  - Platform: Linux-5.4.0-1029-gcp-x86_64-with-glibc2.10
  - Python version: 3.8.5
  - PyTorch version (GPU?): 1.7.0 (False)
  - Tensorflow version (GPU?): 2.3.1 (False)
  - Using GPU in script?: no
  - Using distributed or parallel set-up in script?: no
  
  ### Who can help
  I think:
   @patrickvonplaten @LysandreJik @VictorSanh
  
  Anyone is welcome!
  
  ## Information
  
  I am using `examples/language-modeling/run_mlm_wwm.py` to train my own Tiny BERT model.
  
  ## To reproduce
  Using Tiny BERT from Google [https://github.com/google-research/bert/blob/master/README.md](https://github.com/google-research/bert/blob/master/README.md)
  Using `examples/language-modeling/run_mlm_wwm.py` from HuggingFace to train a language model on raw text.
  
  
  
  files in my `google-bert-tiny` are `bert_config.json  bert_model.ckpt.data-00000-of-00001  bert_model.ckpt.index  vocab.txt`
  
  Steps to reproduce the behavior:
  
  1. install transformers torch and Tensorflow using pip
  2. Get `examples/language-modeling/run_mlm_wwm.py` from HuggingFace>Transformers [Link](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm_wwm.py)
  3. Running the following command:
  ```shell
  python run_mlm_wwm.py \
      --model_name_or_path google-bert-tiny/bert_model.ckpt.index \
      --config_name google-bert-tiny/bert_config.json \
      --train_file train.txt \
      --validation_file val.txt \
      --do_train \
      --do_eval \
      --output_dir test-mlm-wwm \
      --cache_dir cache
  ```
  
  Error:
  ```
  Traceback (most recent call last):
    File "run_mlm_wwm.py", line 340, in <module>
      main()
    File "run_mlm_wwm.py", line 236, in main
      tokenizer = AutoTokenizer.from_pretrained(
    File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/tokenization_auto.py", line 306, in from_pretrained
      config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
    File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/configuration_auto.py", line 333, in from_pretrained
      config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
    File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/configuration_utils.py", line 391, in get_config_dict
      config_dict = cls._dict_from_json_file(resolved_config_file)
    File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/site-packages/transformers/configuration_utils.py", line 474, in _dict_from_json_file
      text = reader.read()
    File "/home/3551351/.conda/envs/kuldeepVenv/lib/python3.8/codecs.py", line 322, in decode
      (result, consumed) = self._buffer_decode(data, self.errors, final)
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 49: invalid start byte
  ```
  
  <!-- If you have code snippets, error messages, stack traces please provide them here as well.
       Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
       Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->
  
  ## Expected behavior
  
  Want it to train
  
  <!-- A clear and concise description of what you would expect to happen. -->
  