Why `attention_mask` in Bert is 1D tensor and is doing 1D masking for keys, rather than 2D masking for both queries and keys
Author: KatarinaYuan
Pull request: no

### System Info

In Bert, as this tutorial(https://huggingface.co/docs/transformers/glossary) suggested, we should have an `attention_mask`, which masks out padding tokens. 

Intuitively, since the multiplication of queries and keys results in attention probability (`attention_probs`), we should mask out the padding tokens for both queries and keys. 

But as shown in the following link, `attention_mask` is constructed as a 1D tensor. https://github.com/huggingface/transformers/blob/450a181d8b963b4e896be4aac701815aa554a6bb/src/transformers/tokenization_utils_base.py#L3450

And as shown in this link, **1D tensor `attention_mask` is directly added upon 2D tensor `attention_probs`**. 
https://github.com/huggingface/transformers/blob/450a181d8b963b4e896be4aac701815aa554a6bb/src/transformers/models/bert/modeling_bert.py#L352

The addition operates without any running errors due to **broadcasting**. But what `attention_mask` does is **only masking out keys, not queries**.

I'm not sure whether my understanding is correct and feel quite confused. Thank you in advance for helping!

### Who can help?

@ArthurZucker and @younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

This is an intuitive question. There is no need of reproduction.

### Expected behavior

N/A

Comments:
Hey! You should check in the modelling code, the attention mask is processed as such:
```python 
        # We can provide a self-attention mask of dimensions [batch_size, from_seq_length, to_seq_length]
        # ourselves in which case we just need to make it broadcastable to all heads.
        extended_attention_mask: torch.Tensor = self.get_extended_attention_mask(attention_mask, input_shape)

        # If a 2D or 3D attention mask is provided for the cross-attention
        # we need to make broadcastable to [batch_size, num_heads, seq_length, seq_length]
        if self.config.is_decoder and encoder_hidden_states is not None:
            encoder_batch_size, encoder_sequence_length, _ = encoder_hidden_states.size()
            encoder_hidden_shape = (encoder_batch_size, encoder_sequence_length)
            if encoder_attention_mask is None:
                encoder_attention_mask = torch.ones(encoder_hidden_shape, device=device)
            encoder_extended_attention_mask = self.invert_attention_mask(encoder_attention_mask)
        else:
            encoder_extended_attention_mask = None
```
This kind of question should be asked on [the forum](https://discuss.huggingface.co/) 😉 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.