Italian roberta model takes 3 minutes to load
Author: AngledLuffa
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:  4.12.2
- Platform:   Centos 8
- Python version:  3.6.8
- PyTorch version (GPU?):   1.9.0 cu 111
- Using GPU in script?:   not when loading
- Using distributed or parallel set-up in script?:   no

### Who can help

Models:

@LysandreJik

The tokenizer for the Italian Roberta model (camembert model?) takes a long time to load.


## Information

Model I am using:

"idb-ita/gilberto-uncased-from-camembert"

This takes about 3 minutes to load.  Once loaded, it has the very suspicious `model_max_len=1000000000000000019884624838656`

## To reproduce

Steps to reproduce the behavior:

```
transformers.AutoTokenizer.from_pretrained("idb-ita/gilberto-uncased-from-camembert", do_lower_case=True)
```

This takes 3 minutes on my machine

## Expected behavior

Other tokenizers only take a few seconds to load.

Comments:
Also, while on the topic of the Italian model, is there any way to set it to automatically convert text so that capital letters don't become `<unk>`?  I would have expected the `do_lower_case` parameter to help with that, but both setting it to True and setting it to False causes capital letters to be `<unk>`.

Hey @AngledLuffa, thank you for opening an issue! There are a few issues here.

### The long loading time.

The `AutoTokenizer` API has two backends: the "slow", python-based tokenizers, and the "fast", rust-based tokenizers. There is a conversion path from slow tokenizers to fast tokenizers, and the `AutoTokenizer` API will try to use the fast every time it can. This means that it will perform the conversion on the fly when no fast tokenizer file is available, but a slow tokenizer file is available and the conversion script exists.

This is what's happening here: the API is converting the tokenizer to it's fast version, as that repository does not host a fast tokenizer file; yet a conversion script exists. The conversion is usually very fast, but for some specific tokenization types (like this one, SentencePiece), it may take longer.

In order to reduce the loading time, here's what you can do:
- Save the tokenizer once you have loaded it, and reload it from that path. It will save it as a fast tokenizer and will reload it as such:
```py
from transformers import AutoTokenizer

# Takes a while
tokenizer = AutoTokenizer.from_pretrained("idb-ita/gilberto-uncased-from-camembert")
tokenizer.save_pretrained("local-folder")

# Very fast
tokenizer = AutoTokenizer.from_pretrained("local-folder")
```
- Load it as a slow tokenizer:
```py
tokenizer = AutoTokenizer.from_pretrained("idb-ita/gilberto-uncased-from-camembert", use_fast=False)
```

Ideally, however, the owner of the repository would also upload a fast tokenizer file.

### The max length

This comes from a lack of a tokenizer configuration file on the repository. There should be one with the following contents:

```json
{"model_max_length": 512}
```

Here too, ideally, the owner of the repository would upload that tokenizer configuration file. However, you may override the value in your instantiation:

```py
tokenizer = AutoTokenizer.from_pretrained("local-folder", model_max_length=512)
```

### The `do_lower_case`

This is currently defined as a tokenizer-specific attribute, it's not available for all tokenizers (some tokenizers never had this capability, and it can be ambiguous for some tokenizers). Updating it right now would potentially result in a breaking change, so it's complex to do. Pinging @sgugger with whom we just discussed the issue.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.