Add `accelerate` support for `Whisper`
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR adds `accelerate` support for `Whisper`, this way `Whisper` can be loaded in 8-bit as follows:
```
from transformers import WhisperModel

model = WhisperModel.from_pretrained("openai/whisper-tiny", device_map="auto", load_in_8bit=True)
```

However some slow tests are not passing when running the whole `Whisper` testing suite but they pass if I run them independently one by one - I still need to understand why this is happening
```
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelTest::test_multi_gpu_data_parallel_forward
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_batched_generation
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_generation
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_generation_multilingual
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_logits_librispeech
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_small_en_logits_librispeech
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_en_batched_generation
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_en_generation
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_generation
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_logits_librispeech
```

cc @sgugger @ArthurZucker @ydshieh 

Thanks!

Comments:
For the failing tests, Wait a little bit a PR related to
The tests will be merged ☺️

Great! I'll try again then once the PR that you have mentioned will be merged 

_The documentation is not available anymore as the PR was closed or merged._

We now know the root cause of the initial issue explained above after an intense debugging session with @ArthurZucker !
the `test_disk_offload` test first overrides the `ACT2FN[config.activation_function]` variable that is initialized only once by the testing suite when importing the model. Therefore when running the slow tests that does not use `device_map=...` all submodules will be initialized without the `_hf_hook` variable except for the [`activation_fn`](https://github.com/huggingface/transformers/blob/072dfdaee4deee65328071e92ac48f471a78490c/src/transformers/models/whisper/modeling_whisper.py#L269) attribute since it is not re-initialized. A workaround could be to design a proper `tearDown` function for those tests, or to change the slow tests with `device_map=auto` ;) 
cc @sgugger 

The fix will be to change `ACT2FN`. It's not healthy that all models share the same instance of activation modules ;-) Will fix this morning.

I can confirm all tests are green after applying the fix proposed in #19728 ! Thanks a lot for adding the fix 💪 
```
======================= 76 passed, 3 skipped, 32 warnings in 146.28s (0:02:26) ========================
```

Since https://github.com/huggingface/transformers/pull/19728 has been merged, I will merge this PR once the tests below are green! 🟢