DataCollatorForLanguageModeling is (unnecessary) slow
Author: robvanderg
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.43.1
- Platform: Linux-3.10.0-1062.18.1.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.11.3
- Huggingface_hub version: 0.23.4
- Safetensors version: 0.4.3
- Accelerate version: 0.32.1
- Accelerate config: 	not found
- PyTorch version (GPU?): 1.13.0+cu117 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No


### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoTokenizer
from transformers import DataCollatorForLanguageModeling

tokzr = AutoTokenizer.from_pretrained('bert-base-multilingual-cased', use_fast=False)

masker = DataCollatorForLanguageModeling(tokzr)
data = ['this is a very boring test sentence'] * 1000

for line in data:
    token_ids = tokzr.encode(line, return_tensors='pt')[0]
    #input_text, output_labels = masker.torch_mask_tokens(token_ids.view(1, -1))
```

### Expected behavior

I would expect this code to be not much slower when uncommenting the masking. However, uncommenting it makes the code 5 times slower. I have tried to profile where this happens, and it seems like it is mainly happening in finding the random words to replace with, as it gets the length of the vocabulary there which takes a long time (https://github.com/huggingface/transformers/blob/main/src/transformers/data/data_collator.py#L854). I am not sure how to make getting the length faster, but perhaps it can just be saved/cached?

Comments:
Was curious about this issue, so tried gathering some statistics on time spent in each line of [torch_mask_tokens](https://github.com/huggingface/transformers/blob/main/src/transformers/data/data_collator.py) (`transformers` commit ID `85a1269e19af022e04bc2aad82572cd5a9e8cdd9`) and this is what the statistics look like: 

```
Code block 'torch_mask_tokens': 18.94850 ms
Code block 'Line 833 to 842' took: 0.04782 ms
Code block 'Line 844 to 846' took: 0.05190 ms
Code block 'Line 849 to 850' took: 0.02048 ms
Code block 'Line 853 to 855' took: 0.01268 ms
Code block 'Line 854' took: 18.43938 ms
Code block 'Line 855' took: 0.06434 ms
```

Measured this using [linetimer](https://github.com/justasb/linetimer). Line 854 is: 
```
random_words = torch.randint(len(self.tokenizer), labels.shape, dtype=torch.long)
```

and is taking 97% of the execution time. And within Line 854, if we split it into:

```
                with CodeTimer("854: get tokenizer length"):
                    tok_len = len(self.tokenizer)
                with CodeTimer("854: generate randints"):
                    random_words = torch.randint(tok_len, labels.shape, dtype=torch.long)
```

we see: 

```
Code block '854: get tokenizer length' took: 18.28698 ms
Code block '854: generate randints' took: 0.03528 ms
```

You are right that this could be significantly sped up if this one thing is saved/cached. 

The length is being calculated in the [PreTrainedTokenizer's __len__ function](https://github.com/huggingface/transformers/blob/main/src/transformers/tokenization_utils.py#L495) function.  This function is doing `return len(set(self.get_vocab().keys()))` each time to calculate the length - which is why it is taking that long. 

It looks like this tokenizer class is setup such that new tokens can be added via the `_add_tokens` function. We could probably do something like:

- add a new attribute `self.vocab_size` to the `PreTreainedTokenizer` class when it is `init`ed.
- update `self.vocab_size` whenever `_add_tokens` is called. 
- modify the `len` function to just `return self.vocab_size`

I am actually curious whether the set has any effect at all. Shouldnt the keys already be unique?, shouldn't this call always have an equal result to `len(self.get_vocab)`, which is probably much faster?

@itazap if you have bandwidth do you want to take a look at what @rohitdwivedula offers?

cc @ArthurZucker 

I'll take a look into a `vocab_size` param @rohitdwivedula 😄 Will get back to you later today!

Looking into it, there are a few considerations: 

- `vocab_size` is used to calculate the vocab length used in training, and so does not include added tokens (`add_tokens`). 
-` len(self.tokenizer)` needs to consider both `vocab_size` and `added_tokens`

see #32240