[BLOOM] Clean modeling code
Author: thomasw21
Pull request: yes

# What does this PR do?

**THERE'S A BREAKING CHANGE**: `past_key_values` changes in terms of format, is that acceptable @sgugger @patrickvonplaten ?

## Changes

Make a pass at cleaning up some code:
 - convert `causal_mask` to bool tensor and use it via mask.
 - simplify `causal_mask` creation function.
 - revert back to `baddbmm` instead of `bmm`, it was unclear why this was changed, and training codebase uses `baddbmm`.
 - switch back `attention_scores` upcasting to fp32 as it mimics the training procedure the closest.
 - remove `self.layer_number` normalization. It was introduced in Meg-DS to prevent overflow when computing attention scores in float16, but it doesn't seem to impact the model at all here.
 - remove a `reshape` in `.split_head()` which reduces memory footprint and increases throughput
 - remove multiple reshapes when computing `QKV` with past values. Though this introduces a **BREAKING CHANGE** where `past_key` instead of being stored in `[batch_size, num_heads, seq_length, head_dim]` it's stored in `[batch_size * num_heads, head_dim, seq_length]`.
 - explicit all dimensions when computing `.view`, `.reshape`
 - standardize namings for `batch_size`, `seq_length` etc ...
 - type hint improvements

## Speed-up in generation

Using the following script:
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from timeit import timeit

def main():
    model_name = "bigscience/bloom-350m"
    max_length = 50
    batch_size = 16
    model = AutoModelForCausalLM.from_pretrained(model_name).cuda()
    tokenizer = AutoTokenizer.from_pretrained(model_name)

    texts = ["Hello my name is"] * batch_size
    input_ids = tokenizer.batch_encode_plus(texts, return_tensors="pt").to("cuda")

    print(timeit(lambda: model.generate(**input_ids, max_length=max_length), number=100))


if __name__ == "__main__":
    main()
```
Results on A100:
```
This branch: 67.85658120410517
main: 78.72558180289343
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Ran quick test on the generation speed:
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from timeit import timeit

def main():
    model_name = "bigscience/bloom-350m"
    max_length = 50
    model = AutoModelForCausalLM.from_pretrained(model_name).cuda()
    tokenizer = AutoTokenizer.from_pretrained(model_name)

    text = "Hello my name is"
    input_ids = tokenizer.encode(text, return_tensors="pt").cuda()

    print(timeit(lambda: model.generate(input_ids, max_length=max_length), number=100))


if __name__ == "__main__":
    main()
```
The results are the following on A100:
```
This branch: 62.44300797022879
Main: 70.0128909018822
```

Now hitting: `62.44300797022879` on the test benchmark (Note that it's a small model, so it the improvement won't have the same percentage, but still I'd say that's quite nice).

Added `batch_size=16` and we see better performance improvement:
```
This branch: 67.85658120410517
main: 78.72558180289343
```

@thomasw21 

We could remove 

https://github.com/huggingface/transformers/blob/323c07316727939518d0966e027c98ecc86ca316/src/transformers/models/bloom/modeling_bloom.py#L100



Btw if that breaking change pattern is approved, I can probably implement that change on other models: `gpt2`, `opt` etc ... I think the smaller the model, the more influence the copies have on the throughput.

Agree with @sgugger regarding the shape of the past key values: fine for me for the shape to be changed. We did change the format for this output two years ago in https://github.com/huggingface/transformers/issues/9391 and https://github.com/huggingface/transformers/pull/9596, which could also have been considered breaking.

I'd like for this change to be contained to BLOOM (i.e., not ported to GPT-2) until @patrickvonplaten can comment as he was very involved in that reshape.

Thanks @LysandreJik ! Yeah this PR is only changing BLOOM modeling.

Okay tested 176b on lambada:
```
## This branch 
{
  "results": {
    "lambada": {
      "ppl": 3.9275610127374367,
      "ppl_stderr": 0.08456328894237608,
      "acc": 0.6716475839316903,
      "acc_stderr": 0.006542638265686496
    }
  },
  "versions": {
    "lambada": 0
  },
}

## 4.21.0
{
  "results": {
    "lambada": {
      "ppl": 3.933199250271968,
      "ppl_stderr": 0.08471163223917309,
      "acc": 0.6708713370851931,
      "acc_stderr": 0.006546580975553108
    }
  },
  "versions": {
    "lambada": 0
  },
}
```
I think this PR is ready to be checked.