Improve DistilBert attn mask
Author: ydshieh
Pull request: yes

# What does this PR do?

Current (PT) `DistilBert ` use `-float("inf")` in self attention layer:
```
scores = scores.masked_fill(mask, -float("inf"))  # (bs, n_heads, q_length, k_length)`
```

When a sequence contains all 0s as attention mask, this will gives `-inf` and `nan` for `scores` and `weights` respectively (for that sequence) in the following block. This will cause loss to be `nan` if labels is passed.

_(It's very unlikely that a user will have examples with all 0s as attention mask in a sequence. But we have these cases generated in some tests.)_

https://github.com/huggingface/transformers/blob/637e81752aad01738d36ef816fadee21fa392317/src/transformers/models/distilbert/modeling_distilbert.py#L207-L209

`TFDistilBert` use a large but finite `-1e30`, which prevent `nan`. 

PyTorch `BERT` uses `-1e9` or `-1e4`:

https://github.com/huggingface/transformers/blob/637e81752aad01738d36ef816fadee21fa392317/src/transformers/modeling_utils.py#L239-L242

TF `BERT` uses `-1e4`:

https://github.com/huggingface/transformers/blob/637e81752aad01738d36ef816fadee21fa392317/src/transformers/models/bert/modeling_tf_bert.py#L854

This PR tries to use `BertModel`'s way of using attn mask to avoid inconsistency and failed test (introduced in #15256)

## Code snippet

```
import torch
from transformers import DistilBertModel
model = DistilBertModel.from_pretrained("distilbert-base-uncased")

device = "cpu"
input_ids = torch.tensor([[1, 1, 1]]).to(device)
# attention_mask = torch.tensor([[1, 1, 1]]).to(device)
# Use all `0` for `attention_mask`
attention_mask = torch.tensor([[0, 0, 0]]).to(device)
inputs = {"input_ids": input_ids, "attention_mask": attention_mask}
outputs = model(**inputs)
print(outputs)
```

### Output with master
```
BaseModelOutput(last_hidden_state=tensor([[[nan, nan, nan,  ..., nan, nan, nan],
         [nan, nan, nan,  ..., nan, nan, nan],
         [nan, nan, nan,  ..., nan, nan, nan]]],
       grad_fn=<NativeLayerNormBackward0>), 
```

### Output with this PR
```
BaseModelOutput(last_hidden_state=tensor([[[ 0.2162, -0.0174,  0.2730,  ...,  0.1114,  0.2780, -0.1252],
         [ 0.2946,  0.0555,  0.2918,  ...,  0.1332,  0.3355, -0.2091],
         [ 0.3039,  0.0415,  0.2762,  ...,  0.1108,  0.3250, -0.1843]]], ...)
```

gives all `nan` on current master.

## Future Improvement 

It might be a good idea to handle all these `-1e4`/`-1e9`/`-1e30`/`-inf` things in a unified way across the models & frameworks (in a future PR). See #14859


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15326). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

This could be treated in a future PR that will work with all models/frameworks.