BarkModel: support multiple history prompt for batch api
Author: Selectorrr
Pull request: no
Labels: name: Good Second Issue

name: Feature request

### Feature request

Add multiple history prompt support for batch api

### Motivation

Batch API speeds up the audio generation process a lot. Unfortunately we can't use it to generate long sequences.

### Your contribution

I may be able to look at it soon

Comments:
cc @ylacombe 

Hi @Selectorrr, thanks for opening this issue!

it would actually be a great feature, in line with recent #26673 and #26672. 
I believe that it would need two main changes:
1. Allowing BarkProcessor to get a [list of history prompts](https://github.com/huggingface/transformers/blob/ad08137e473e00702fc3088a119da7026e1cb025/src/transformers/models/bark/processing_bark.py#L218-L228) and adapt the output in consequence
2. Adapt Bark modeling code -> making sure that every usage of history prompt is adapted and of the right shape. It might be more difficult

Basically, we would need to make sure that the number of history prompts is either 1 or the same as the batch size and throw errors otherwise. In the modeling code, we probably would need to adapt lines like this one to not repeat_interleave if the history prompt is of the right shape: https://github.com/huggingface/transformers/blob/ad08137e473e00702fc3088a119da7026e1cb025/src/transformers/models/bark/modeling_bark.py#L782 

Would you like to start working on this ?

@ylacombe Unfortunately my python development experience is not enough to say with certainty that I will do this. If I can do it, I will write it, but please consider it a free task.

@ylacombe and @Selectorrr Can I work on this? I understand history_prompt here represents speaker voices and what we want is more than one speaker for batch. Let me know if I need to know more.

@ylacombe and @Selectorrr Can I work on this? 

> @ylacombe and @Selectorrr Can I work on this? 

I don't do that, I'd be glad if you did.

Hey @shankarsharma8089 and @hackpk , feel free to work on this! Let me know if I can help you

@ylacombe I took an approach to loop the generation of semantic, coarse, and fine for every speaker it worked but it's not an optimized solution as we can have as many speakers as we want. At this moment I'm stuck to get an optimized solution. I was thinking maybe converting the list of speakers semantic, coarse, and fine to a list. I'm new to ML, Transformer, and Tensors which I'm learning on the go. Please guide me in the right direction. Thank you

Hey @hackpk, I believe that [my previous comment](https://github.com/huggingface/transformers/issues/26921#issuecomment-1797875170) should have everything you need to address this issue. It's okay to loop to load voice presets, but I don't think you should do that to generate. 
This PR is ideal for people having a little experience in the field. If this is out of reach for you, don't hesitate to let @shankarsharma8089 handle this! There are a bunch of issues that might be more suitable for you under the [Good First Issue](https://github.com/huggingface/transformers/issues?q=is%3Aissue+is%3Aopen+label%3A%22Good+First+Issue%22) tag!


Thanks @ylacombe I will work on some other suitable issue. @shankarsharma8089 you can take this

Hi @ylacombe @Selectorrr if it s ok with you, I'll have a stab at it. Not necessarily my area of expertise but happy to learn and contribute.