fix tensors on different devices in `WhisperGenerationMixin`
Author: faaany
Pull request: yes

## What does this PR do?
1. it fixes the bug in the following test.
2.  it makes the following test device-agnostic. 

```bash
pytest -rA tests/pipelines/test_pipelines_automatic_speech_recognition.py::AutomaticSpeechRecognitionPipelineTests::test_whisper_prompted
```
This above test fails because `prompt_ids` are on cpu, while `decoder_input_ids` is on cuda:0: 
```bash
        if any(do_condition_on_prev_tokens) and len(current_segments[0]) > 0:
            # according to https://github.com/openai/whisper/blob/e58f28804528831904c3b6f2c0e473f346223433/whisper/decoding.py#L609
            active_segments = [current_segments[i] if do_condition_on_prev_tokens[i] else None for i in batch_idx_map]
    
            if prompt_ids is not None and generation_config.prompt_condition_type == "all-segments":
                prev_ids = prompt_ids
            else:
                one_tensor = torch.ones((cur_bsz, 1), device=device, dtype=torch.long)
                prev_ids = prev_start_of_text * one_tensor[0] if prev_start_of_text is not None else None
    
            prev_tokens = _pad_to_max_length(
                active_segments,
                generation_config.pad_token_id,
                device=device,
                padding="left",
                bos_token_tensor=prev_ids,
                cut_off_length=cut_off_length,
            )
            decoder_input_ids = torch.cat([prev_tokens, decoder_input_ids], dim=-1)
    
            kwargs["decoder_attention_mask"] = decoder_input_ids != generation_config.pad_token_id
        elif prompt_ids is not None:
            prev_tokens = prompt_ids[None].repeat(decoder_input_ids.shape[0], 1)
>           decoder_input_ids = torch.cat([prev_tokens, decoder_input_ids], dim=-1)
E           RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument tensors in method wrapper_CUDA_cat)

src/transformers/models/whisper/generation_whisper.py:1629: RuntimeError
============================================= short test summary info ==============================================
FAILED tests/pipelines/test_pipelines_automatic_speech_recognition.py::AutomaticSpeechRecognitionPipelineTests::test_whisper_prompted - RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (w...
========================================== 1 failed, 3 warnings in 35.92s ==========================================
```
We need to move `prompt_ids` to the device of `decoder_input_ids`. 
After the fix, the test run through:
```bash
================================================ short test summary info ================================================
PASSED tests/pipelines/test_pipelines_automatic_speech_recognition.py::AutomaticSpeechRecognitionPipelineTests::test_whisper_prompted
============================================ 1 passed, 3 warnings in 42.88s =============================================
```

Comments:
good for me.

Hi @amyeroberts @ydshieh , could you take a look at this PR? Thx! 

Hi @sanchit-gandhi and @ylacombe , could you pls take a look at this PR? Many thx!

> The device fix for the test LGTM, thanks @faaany!
> 
> Regarding moving the `prompt_ids` to the correct device, this is actually due to a bit of a clash between the different design principles in the library:
> 
> 1. For the `model` + `processor` API, we expect users to move all input tensors to the correct device themselves, e.g.:
> 
> ```python
> prompt_ids = pipe.tokenizer.get_prompt_ids(whisper_prompt, return_tensors="pt")
> prompt_ids = prompt_ids.to(torch_device)
> ```
> 
> Consequently, we avoid moving tensors between the devices within the modelling (or generation) code.
> 
> 2. For the `pipeline` API, we handle all device transfer automatically within the pipeline abstraction. This is why you don't have to move the input audio onto the correct device; the pipeline pre-processes it and places it on the correct device for you.
> 
> The problem in this example is that we're mixing 1 and 2: we use the `processor` to pre-process the prompt ids, and then pass them to the `pipeline` for inference. Given we use the `processor` for the prompt ids, we need to ensure we manually move them to the correct device before inference, e.g.
> 
> ```python
> prompt_ids = pipe.tokenizer.get_prompt_ids(whisper_prompt, return_tensors="pt")
> prompt_ids = prompt_ids.to(torch_device)
> 
> prompted_result = pipe(sample, generate_kwargs={"prompt_ids": prompt_ids})["text"]
> ```
> 
> Given the above, I would be against moving the `prompt_ids` to the correct device within the generation code (since this violates design principle 1). Instead, I would be in favour of opening a new PR where we ensure any torch tensors in the [`_forward`](https://github.com/huggingface/transformers/blob/838d141fb47dd05c3ea9afeeaf7a263b16aba031/src/transformers/pipelines/automatic_speech_recognition.py#L467) method of the `AutomaticSpeechRecognitionPipeline` are on the correct device.
> 
> This can be handled quite easily with one line of code in this method:
> 
> ```
> generate_kwargs = self._ensure_tensor_on_device(generate_kwargs)
> ```
> 
> Let me know if that makes sense!

wow, thanks so much for the detailed explanation! yes, I totally see your point. Let me create a new PR for that! Thx! 

Hi @amyeroberts and @sanchit-gandhi , so actually there could be 2 solutions:
1. I can manually move the `prompt_ids` to the correct devcie in the `test_whispter_prompted` test
2. Or we can ensure the tensors in `generate_kwargs` to be on the same device. (I created this PR: https://github.com/huggingface/transformers/pull/32559 for this)

I tested both and both work on XPU. So which one would you guys prefer? For 1, I will need to push an additonal commit to move the prompt_id in the test to the correct device.  

Hi @amyeroberts and @sanchit-gandhi , could you take a look at my PRs? Thx a lot!

> * I can manually move the `prompt_ids` to the correct devcie in the `test_whispter_prompted` test

Thanks for iterating @faaany -device placement for the tests looks good! 

Regarding option 1 vs 2 - I think 1 is best and aligns with the design principles @sanchit-gandhi mentions above 

> > * I can manually move the `prompt_ids` to the correct devcie in the `test_whispter_prompted` test
> 
> Thanks for iterating @faaany -device placement for the tests looks good!
> 
> Regarding option 1 vs 2 - I think 1 is best and aligns with the design principles @sanchit-gandhi mentions above

Cool! New commit pushed.