Enable the use of batch api using multiple speakers with Bark
Author: MidAtBest
Pull request: yes
Labels: name: Audio

# What does this PR do?

This PR aims to enable the use of the batch API when using multiple speaker prompts for Bark.

Currently, the batch API is available when using multiple text inputs but is limited to a single speaker. The objective of this PR is to enable this when multiple speaker prompts are being used. This is done keeping in mind that we want to minimise the amount of loops when processing the data while keeping the same audio quality.

cc @ylacombe  and @Selectorrr , could you take a look ? Thanks!

Fixes #26921

PS: Currently I have doubts on how to solve the issue in the `BarkCoarseModel` especially these 2 lines:

```
input_coarse = semantic_output[:, np.max([0, semantic_idx - max_semantic_history]) :]
input_coarse = input_coarse[:, :max_coarse_input_length]
```

My concern is that with the approach implemented in this PR, if we need to pad after the `preprocess_histories` method because the samples in the batch for `x_coarse` are not the same length then we might run into a scenario where some padding tokens would be used in `input_coarse` instead of "regular tokens" if the sample was processed outside of a batch.

Comments:
> Hi @MidAtBest, thanks for taking care of this, this is a great PR! The logic is quite difficult so congrats on handling it so nicely !
> 
> Two general comments:
> 
> 1. you should add test for bark modeling code as well !
> 2. it's best to stay with `voice_preset` instead of `voice_presets`.
> 
> I'm responding to your question below, I hope that it helps!
> 
> Thanks again!
> 
> In terms of your question, I believe that we should refactor the logic of the coarse generation quite a bit to make it work.
> 
> The key to that is IMO in the following lines:
> 
> These ones creates `semantic_output`:
> 
> https://github.com/huggingface/transformers/blob/87ae2a4632de9c3090272d5cd37db86c0e03a0a9/src/transformers/models/bark/modeling_bark.py#L1175-L1185
> 
> Those 3 lines are located in the generation loop and creates `input_coarse`:
> 
> https://github.com/huggingface/transformers/blob/87ae2a4632de9c3090272d5cd37db86c0e03a0a9/src/transformers/models/bark/modeling_bark.py#L1196-L1204
> 
> As you can see, the only thing that changes here is `semantic_idx`, the rest is quite static.
> 
> So in my opinion, you keep track of the length of each `semantic_history` and merge each `semantic_history` with the corresponding `semantic_output`.
> 
> Once arriving in the generation loop, you can do the 2 first lines for each sample (taking into account than each sample has its own `semantic_idx`) then use `torch.rnn.pad_sequence` to pad it to the right length.

Thanks a lot for the reply, yes it took a bit of time for me to understand both the model and the shape manipulations going on. 
Regarding the tests and other changes I had omitted them as I was a bit puzzled regarding the question I mentioned in the body of the PR. 
Setting the PR as WIP and will try to have something by Wednesday evening.


cc @ylacombe would be nice if you can review here!


Gentle ping @ylacombe 

Another ping @ylacombe 

Another ping @ylacombe