AttributeError: 'BertTokenizerFast' object has no attribute 'max_len'
Author: zcain117
Pull request: no

## Environment info
    
- `transformers` version: 4.0.0-rc-1
- Platform: Linux-4.9.0-14-amd64-x86_64-with-debian-9.13
- Python version: 3.6.10
- PyTorch version (GPU?): 1.8.0a0+4ed7f36 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: 8-core TPU training
- **Using TPU**

### Who can help
 albert, bert, GPT2, XLM: @LysandreJik 

## Information

Model I am using (Bert, XLNet ...): bert and roberta

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: mlm
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

2 examples of failing commands:
```
E 2020-11-18T17:38:08.657584093Z python examples/xla_spawn.py \
E 2020-11-18T17:38:08.657588780Z   --num_cores 8 \
E 2020-11-18T17:38:08.657593609Z   examples/contrib/legacy/run_language_modeling.py \
E 2020-11-18T17:38:08.657598646Z   --logging_dir ./tensorboard-metrics \
E 2020-11-18T17:38:08.657604088Z   --cache_dir ./cache_dir \
E 2020-11-18T17:38:08.657609492Z   --train_data_file /datasets/wikitext-103-raw/wiki.train.raw \
E 2020-11-18T17:38:08.657614614Z   --do_train \
E 2020-11-18T17:38:08.657619772Z   --do_eval \
E 2020-11-18T17:38:08.657624531Z   --eval_data_file /datasets/wikitext-103-raw/wiki.valid.raw \
E 2020-11-18T17:38:08.657629731Z   --overwrite_output_dir \
E 2020-11-18T17:38:08.657641827Z   --output_dir language-modeling \
E 2020-11-18T17:38:08.657647203Z   --logging_steps 100 \
E 2020-11-18T17:38:08.657651823Z   --save_steps 3000 \
E 2020-11-18T17:38:08.657656739Z   --overwrite_cache \
E 2020-11-18T17:38:08.657661282Z   --tpu_metrics_debug \
E 2020-11-18T17:38:08.657667598Z  --mlm --model_type=bert \
E 2020-11-18T17:38:08.657672545Z --model_name_or_path bert-base-cased \
E 2020-11-18T17:38:08.657677441Z --num_train_epochs 3 \
E 2020-11-18T17:38:08.657682320Z --per_device_train_batch_size 16 \
E 2020-11-18T17:38:08.657687053Z --per_device_eval_batch_size 16
```
```
2020-11-18T17:51:49.357234955Z Traceback (most recent call last):
 E 
2020-11-18T17:51:49.357239554Z   File "/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py", line 329, in _mp_start_fn
 E 
2020-11-18T17:51:49.357245350Z     _start_fn(index, pf_cfg, fn, args)
 E 
2020-11-18T17:51:49.357249851Z   File "/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py", line 323, in _start_fn
 E 
2020-11-18T17:51:49.357254654Z     fn(gindex, *args)
 E 
2020-11-18T17:51:49.357272443Z   File "/transformers/examples/contrib/legacy/run_language_modeling.py", line 359, in _mp_fn
 E 
2020-11-18T17:51:49.357277658Z     main()
 E 
2020-11-18T17:51:49.357281928Z   File "/transformers/examples/contrib/legacy/run_language_modeling.py", line 279, in main
 E 
2020-11-18T17:51:49.357287863Z     data_args.block_size = tokenizer.max_len
 E 
2020-11-18T17:51:49.357292355Z AttributeError: 'BertTokenizerFast' object has no attribute 'max_len'
 E 
```

```
E 2020-11-18T06:47:53.910306819Z python examples/xla_spawn.py \
E 2020-11-18T06:47:53.910310176Z   --num_cores 8 \
E 2020-11-18T06:47:53.910314263Z   examples/contrib/legacy/run_language_modeling.py \
E 2020-11-18T06:47:53.910319173Z   --logging_dir ./tensorboard-metrics \
E 2020-11-18T06:47:53.910322683Z   --cache_dir ./cache_dir \
E 2020-11-18T06:47:53.910325895Z   --train_data_file /datasets/wikitext-103-raw/wiki.train.raw \
E 2020-11-18T06:47:53.910329170Z   --do_train \
E 2020-11-18T06:47:53.910332491Z   --do_eval \
E 2020-11-18T06:47:53.910335626Z   --eval_data_file /datasets/wikitext-103-raw/wiki.valid.raw \
E 2020-11-18T06:47:53.910340314Z   --overwrite_output_dir \
E 2020-11-18T06:47:53.910343710Z   --output_dir language-modeling \
E 2020-11-18T06:47:53.910347004Z   --logging_steps 100 \
E 2020-11-18T06:47:53.910350089Z   --save_steps 3000 \
E 2020-11-18T06:47:53.910353259Z   --overwrite_cache \
E 2020-11-18T06:47:53.910356297Z   --tpu_metrics_debug \
E 2020-11-18T06:47:53.910359351Z  --mlm --model_type=roberta \
E 2020-11-18T06:47:53.910362484Z --tokenizer=roberta-base \
E 2020-11-18T06:47:53.910365650Z --num_train_epochs 5 \
E 2020-11-18T06:47:53.910368797Z --per_device_train_batch_size 8 \
E 2020-11-18T06:47:53.910371843Z --per_device_eval_batch_size 8
```
```
2020-11-18T06:48:27.357394365Z Traceback (most recent call last):
 E 
2020-11-18T06:48:27.357399685Z   File "/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py", line 329, in _mp_start_fn
 E 
2020-11-18T06:48:27.357405353Z     _start_fn(index, pf_cfg, fn, args)
 E 
2020-11-18T06:48:27.357426600Z   File "/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py", line 323, in _start_fn
 E 
2020-11-18T06:48:27.357448514Z     fn(gindex, *args)
 E 
2020-11-18T06:48:27.357454250Z   File "/transformers/examples/contrib/legacy/run_language_modeling.py", line 359, in _mp_fn
 E 
2020-11-18T06:48:27.357460262Z     main()
 E 
2020-11-18T06:48:27.357465843Z   File "/transformers/examples/contrib/legacy/run_language_modeling.py", line 279, in main
 E 
2020-11-18T06:48:27.357471227Z     data_args.block_size = tokenizer.max_len
 E 
2020-11-18T06:48:27.357477576Z AttributeError: 'RobertaTokenizerFast' object has no attribute 'max_len'
 E 
```


The timing of this issue lines up with https://github.com/huggingface/transformers/pull/8586
Tests started failing on the evening of Nov 17, a few hours after that PR was submitted

Comments:
It is actually due to https://github.com/huggingface/transformers/pull/8604, where we removed several deprecated arguments. The `run_language_modeling.py` script is deprecated in favor of `language-modeling/run_{clm, plm, mlm}.py`.

Is it possible for you to switch to one of these newer scripts? If not, the fix is to change `max_len` to `model_max_length`. We welcome PRs to fix it, but we won't be maintaining that script ourselves as there exists better alternatives now (which run on TPU too :slightly_smiling_face:)

Thanks for taking a look!  I will try out the new script

The new runner is working for us on TPUs. Thanks again for the tip!

Hello, Everything was a few days. I am getting the same error  "   data_args.block_size = min(data_args.block_size, tokenizer.max_len)
**AttributeError: 'RobertaTokenizerFast' object has no attribute 'max_len"**. 

I can't switch to a new script as you mentioned. Kindly help me with this error. I do not know how to fix it. Here is my chunk of codes.
```

`!python "/content/transformers/examples/contrib/legacy/run_language_modeling.py" \
    --output_dir "/content/drive/MyDrive/Vancouver" \
    --model_name_or_path roberta-base \
    --do_train \
    --per_gpu_train_batch_size 8 \
    --seed 42 \
    --train_data_file "/content/input_textOC.txt" \
    --block_size 256 \
    --line_by_line \
    --learning_rate 6e-4 \
    --num_train_epochs 3 \
    --save_total_limit 2 \
    --save_steps 200 \
    --weight_decay 0.01 \
     --mlm`
```

> It is actually due to #8604, where we removed several deprecated arguments. The `run_language_modeling.py` script is deprecated in favor of `language-modeling/run_{clm, plm, mlm}.py`.
> 
> Is it possible for you to switch to one of these newer scripts? If not, the fix is to change `max_len` to `model_max_length`. We welcome PRs to fix it, but we won't be maintaining that script ourselves as there exists better alternatives now (which run on TPU too )

The fix is mentioned above:

>  fix is to change `max_len` to `model_max_length`

If you cannot switch scripts, I recommend pinning the library. You're having this error because you're using a legacy script with a `master` version that is not compatible.

You could pin it to v3.5.1.

Thanks, I appreciate your response. However, I am still a basic learner. Can you please explain it a bit? how to pin it to v3.5.1.. Is it mean to use the old version of huggingface.?

If you wish to stick to that deprecated example, yes! You can do so by checking out the tag v3.5.1:

```
git checkout v3.5.1
```

If you have installed transformers from pypi (and not from source), you should also update your transformers version:

```
pip install -U transformers==3.5.1
```

Please note that the script won't be in "/content/transformers/examples/contrib/legacy/run_language_modeling.py" anymore, but in "/content/transformers/examples/language-modeling/run_language_modeling.py"

> It is actually due to #8604, where we removed several deprecated arguments. The `run_language_modeling.py` script is deprecated in favor of `language-modeling/run_{clm, plm, mlm}.py`.

Hello, I am facing the same issue with `run_language_modeling.py` (and more). Where can I find this new file  `language-modeling/run_{clm, plm, mlm}.py`? Thanks!

You can find them here https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling

Thank you! 

> It is actually due to #8604, where we removed several deprecated arguments. The `run_language_modeling.py` script is deprecated in favor of `language-modeling/run_{clm, plm, mlm}.py`.
> 
> Is it possible for you to switch to one of these newer scripts? If not, the fix is to change `max_len` to `model_max_length`. We welcome PRs to fix it, but we won't be maintaining that script ourselves as there exists better alternatives now (which run on TPU too 🙂)

Change `max_len` to `model_max_length` where?

> fix is to change `max_len` to `model_max_length`

in which all model files should this be fixed ?