Improve bark batch generation
Author: ylacombe
Pull request: no
Labels: name: Good Second Issue

name: Feature request

### Feature request

Currently, when several samples are transmitted to Bark at the same time, i.e. during batch generation, the generated audios all have the same duration, which extends the length of the shortest audios to the longest audio in the batch.

This problem can be solved by keeping track of sample lengths as `Bark.generate` is called up, and then outputting the audio lengths with the generated audios at the end of the call.

### Motivation

This is something that would greatly improve Bark batch generation and that has been requested a few times. 

### Your contribution

A few pointers:
- This starts in [`BarkFineModel.generate`](https://github.com/huggingface/transformers/blob/b71f20a7c9f3716d30f6738501559acf863e2c5c/src/transformers/models/bark/modeling_bark.py#L1350-L1354), when computing `n_remove_from_end`, which is the number of tokens to remove, i.e a proxy of the length of the longest of the batch. You can add here a tracker of the length of each sample of the batch.
- You can then pass it to the output of `BarkFineModel.generate`, and finally to the output of `BarkModel.generate`

To keep backward compatibility, this feature should be enabled with an additional input boolean. Finally, we should ensure result consistency as compared to samples being generated one by one in the test suite [here](https://github.com/huggingface/transformers/blob/897a826d830e8b1e03eb482b165b5d88a7a08d5f/tests/models/bark/test_modeling_bark.py#L871).

Comments:
Hello, I'd like to work on this if no one is working on it.


Hello Yoach,
Since issue [#26672](https://github.com/huggingface/transformers/issues/26672) seems to already be taken on by a few people, I'd love to try and work on this one !
Thanks, Julien 

Hey, @ylacombe. If I understand correctly the input to fine model is output from coarse model. The dimensions are all the same for each of the sample in the batch. How can I specifically get lengths of each sample from here?

Hi @divinit7 and @JulienAjdenbaum, thanks for your motivation here!
Would one of you would like to start working on this?

BTW, there is this new issue #26921, which is quite interesting to work on as well.

> If I understand correctly the input to fine model is output from coarse model. The dimensions are all the same for each of the sample in the batch. How can I specifically get lengths of each sample from here?

Nice question, basically the length of each samples can be derived from here:
https://github.com/huggingface/transformers/blob/ad08137e473e00702fc3088a119da7026e1cb025/src/transformers/models/bark/modeling_bark.py#L958-L964

You can get each sample length by looking at the number of `coarse_generation_config.coarse_semantic_pad_token` ([which replace the eos_token_ids of the semantic model](https://github.com/huggingface/transformers/blob/ad08137e473e00702fc3088a119da7026e1cb025/src/transformers/models/bark/modeling_bark.py#L944-L949)) and substract it from `semantic_output.shape[1]`


Hello, I was a bit under the water in the past few days/weeks but I should have time to advance on this issue this week. I'll let you know when I have some code to show/questions !

Hello @JulienAjdenbaum, no worries::! let me know if I can help you

Hi @JulienAjdenbaum, any update on this ?