batched generate using forced_decoder_ids
Author: ghadiaravi13
Pull request: no

### Feature request

Currently, forced_decoder_ids only accepts a single list of [index,token_id] to force the decoder output given an input. However, it does not support batched output forcing, where input itself is a batch. Can we have this support to have forced_decoder_ids = List(List(List([int,int]))) where 0th dimension corresponds to batch dimension?

### Motivation

This will help forcing the output for different inputs simultaneously.

### Your contribution

I could help submit a PR, wanted to first understand the feasibility of feature request and/or other implications.

Comments:
cc @gante 

Hey @ghadiaravi13 👋 

I see no major technical hurdles to implementing the feature. However, before we go that route, what is the use case you expect from that feature? (As any increase in complexity, we should make sure it is done for a good reason :) )

Hi @gante!

The expected usage is when we use generate function over a batch of inputs, and want to force the decoder output for each input in the batch. Having batch support rather than iterating manually will have computational benefits, I suppose. You may correct me though.

@ghadiaravi13 `forced_decoder_ids` should already work at batch level, assuming you want the same forced tokens for all members of the batch. Doesn't this solve your use case?

The alternative, where each member of the batch has its own `forced_decoder_ids`, requires significantly increasing the complexity of the code. As such, to include it in `transformers`, we need some demonstration that it is a valued feature :)

Yes I was referring to the latter case actually. Sure I could imagine the increase in code complexity, just wanted to check. I'll stick with manually iterating for now. Thanks for responding!

> @ghadiaravi13 `forced_decoder_ids` should already work at batch level, assuming you want the same forced tokens for all members of the batch. Doesn't this solve your use case?
> 
> The alternative, where each member of the batch has its own `forced_decoder_ids`, requires significantly increasing the complexity of the code. As such, to include it in `transformers`, we need some demonstration that it is a valued feature :)

I think it is crucial for cases when you want to force different prompt for member in the batch, e.g.: Training Whisper on transcribe and translate tasks  in the same dataset. In this case, some members in the batch needs to use the transcribe token forced and some the translate token.

How is it possible to solve it otherwise? 

`.generate()` is not used at training time, so that question doesn't apply. See our [blog post on fine-tuning Whisper](https://huggingface.co/blog/fine-tune-whisper#prepare-feature-extractor-tokenizer-and-data) for further reference.

At inference time, it is possible to build a solution to handle both tasks at once. However, the benefits are small (vs separating different tasks in different data batches) and we'd have the burden of long-term maintenance of the code. I'd still encourage you to build your own custom `LogitsProcessor` to solve the problem if it is relevant to your use case -- we've built a modular codebase precisely so anyone can easily build their custom solutions without depending on us 🤗 

Finally, I'd like to mention that [Whisper has its own `.generate()` function](https://github.com/huggingface/transformers/blob/d3046dad809b7b10019b142ae20b49fb58d21c28/src/transformers/models/whisper/modeling_whisper.py#L1232) that easily abstracts the parameterization for each task.