AutoModelForMaskedLM produces NaN if no token is masked
Author: nreimers
Pull request: no

This is a more for discussion if a bugfix is wanted (and if yes, how it should look like). I already fixed it locally for my training script.


**Background**
I currently run MLM pre-training for large models where each batch can only consist of a single example. In some cases, the text can be rather short, for example, just a sentence.  Here it can happen that the `DataCollatorForLanguageModeling` does not mask any token and the computed loss is NaN, which produces problems down the multi-processing script as a NaN loss cannot be correctly back-propagated & shared across the workers.

Here a short simplified script that shows the problem:
```python
from transformers import DataCollatorForLanguageModeling, AutoTokenizer
from transformers import AutoModelForMaskedLM

model_name = "nreimers/BERT-Tiny_L-2_H-128_A-2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
coll = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm_probability=0.15)

print("MASK token:", tokenizer.convert_tokens_to_ids(tokenizer.mask_token))
text = "This is an example"

data = tokenizer(text, padding=True, max_length=512, return_special_tokens_mask=True)

model = AutoModelForMaskedLM.from_pretrained(model_name)

for _ in range(10):
    mini_batch = coll([data]) #Is equivalent to getting a mini batch from a DataSet / DataLoader
    print("Input", mini_batch['input_ids'])
    print("Labels", mini_batch['labels'])

    output = model(**mini_batch)
    print(output.loss)
    print("------")
```

Output:
```
// Here tokens 1-3 were masked
Input tensor([[ 101,  103,  103, 2019, 2742,  102]])
Labels tensor([[-100, 2023, 2003, 2019, -100, -100]])
Loss tensor(0.9181, grad_fn=<NllLossBackward0>)
------
// Here no tokens were masked
Input tensor([[ 101, 2023, 2003, 2019, 2742,  102]])
Labels tensor([[-100, -100, -100, -100, -100, -100]])
Loss tensor(nan, grad_fn=<NllLossBackward0>)
```

When there are masked tokens, then the loss is computed correctly. But as we mask just 15% of the tokens, it can happen for short sequences that no tokens are masked (i.e. labels are all -100), hence the loss is `nan`.

For long sequences and / or large batches the issues does not really happen, as the probability that no token is masked is rather low. But for short sequences with small batch sizes this happens fairly often. If you train large models, often you cannot increase you batch size and short text sequences in your dataset can kill your process.


**Discussion**

If we want to fix this, we could fix in two possible ways:
1) Update the `DataCollatorForLanguageModeling` to make sure that at least 1 token per text is masked
2) Update the loss in `AutoModelForMaskedLM` that the loss is 0 if no token is selected for masking.


**Work around**

My current solution (for Pytorch) looks like this: If there is no token selected for masking, I select the first token (the first token after the CLS token). Not a perfect solution, but it solves the issue for me.

```python
class MyDataCollatorForLanguageModeling(DataCollatorForLanguageModeling):
    def torch_mask_tokens(self, inputs, special_tokens_mask = None):
        """
        Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original.
        """
        import torch

        labels = inputs.clone()
        # We sample a few tokens in each sequence for MLM training (with probability `self.mlm_probability`)
        probability_matrix = torch.full(labels.shape, self.mlm_probability)
        if special_tokens_mask is None:
            special_tokens_mask = [
                self.tokenizer.get_special_tokens_mask(val, already_has_special_tokens=True) for val in labels.tolist()
            ]
            special_tokens_mask = torch.tensor(special_tokens_mask, dtype=torch.bool)
        else:
            special_tokens_mask = special_tokens_mask.bool()

        probability_matrix.masked_fill_(special_tokens_mask, value=0.0)
        masked_indices = torch.bernoulli(probability_matrix).bool()

        # Nils added code: Make sure at least 1 token is masked
        for idx in range(len(masked_indices)):
            if not torch.any(masked_indices[idx]):
                masked_indices[idx][1] = True
        # /Nils added code

        labels[~masked_indices] = -100  # We only compute loss on masked tokens


        # 80% of the time, we replace masked input tokens with tokenizer.mask_token ([MASK])
        indices_replaced = torch.bernoulli(torch.full(labels.shape, 0.8)).bool() & masked_indices
        inputs[indices_replaced] = self.tokenizer.convert_tokens_to_ids(self.tokenizer.mask_token)

        # 10% of the time, we replace masked input tokens with random word
        indices_random = torch.bernoulli(torch.full(labels.shape, 0.5)).bool() & masked_indices & ~indices_replaced
        random_words = torch.randint(len(self.tokenizer), labels.shape, dtype=torch.long)
        inputs[indices_random] = random_words[indices_random]

        # The rest of the time (10% of the time) we keep the masked input tokens unchanged
        return inputs, labels
```
 

Comments:
Hey @nreimers, thanks for the issue! I think the propositions are sensible, what do you think @sgugger ?

I think this should be solved in the `MaskedLM` directly to return a loss of 0.0 and no NaNs.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Any fix on this? I'm still facing this issue.

This still has not been fixed...