Add Vocos model
Author: Manalelaidouni
Pull request: yes
Labels: name: New model

name: Audio

# What does this PR do?

This PR aims at integrating  `Vocos` model to `transformers`.

Vocos is a neural vocoder designed for high quality audio synthesis in TTS pipelines and related tasks, outpeforms `HifiGan` and it is significantly faster. It has 2 main variants : 

- `VocosModel` can be used as a standalone vocoder in audio generation pipeline, the goal is to use it as a drop in vocoder in YuE model. It can also be used together with `VocosFeatureExtractor` to synthesis audio from mel-spectrogram features.
- `VocosWithEncodecModel` :  integrates the EnCodec neural audio codec model into Vocos for end-to-end audio compression and reconstruction.

This is a continuation of integrating model components for the new YuE model (mention in #36784).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed.
@ArthurZucker @eustlb @ylacombe 

Comments:
Thanks for reviewing! the failing tests seem unrelated to my changes, but I realized the latest datasets 4.0.0 loads different audio samples than earlier versions which was causing integration tests to fail in CI.

If you can merge main adress the small comment and we can merge!

Thanks for the review  @ArthurZucker, all comments addressed!

Hey @Arthur! just checking in on this PR, I think we’re all set as the CI failures appear unrelated, is there anything else needed from my side? Thanks again!



Hey! let's just fix the conflicts and the check repo consitency and we can merge!

Hey @ArthurZucker! CI is all green now should be good to go 👍

@ebezzam thanks for the review, I really appreciate the detailed feedback ☺️ 

I'm in the final stretch of a research lab application that’s due this friday, so I’ll handle the updates this weekend and push changes by sunday evening, I’ll also open a draft PR for YuE (finished some parts) to get early design feedback,  

Thanks again!

@Manalelaidouni you're welcome! It's much appreciated all the models you're adding and I'm happy to help 😀

And like YuE, other models use Vocos (xcodec2, Audio Flamingo 3) so this will be a great addition!

And no rush if you want to take a break after your application! All the best for that 💪 

Hey @ebezzam sorry for the delay, took longer than expected but I've pushed the full Vocos processor implementation you outlined and all tests pass, just a heads up there's a tiny 1e-5 difference in the melspecrogram variant since the original implementation uses the torchaudio backend while we use numpy backedn in the feature extractor, and I've left a comment about the Bark integration, again much appreciated!

@Manalelaidouni thank you 🙏 I'll take a look and may push some small changes if that's fine?

Absolutely! what small tweaks would you suggest?

I only made changes to the docs, and tried the examples myself which work great 👍 (I just had to fix the EnCodec example where you passed audio codes to the model (old way) instead of the processor.) 

The modeling code is much cleaner now with codec handling done in the processor, thanks for that!!

I'll try to give some comments today or early next week.

Hey @ebezzam! thanks for the input I've addressed all the requested changes, mainly I've added the torch backend alongside the numpy one, added tests for both backends and implemented batch integration tests for both the mel and encodec variants.


hey @ebezzam I've switched the feature extractor to use torchaudio Melspectrogram instead of torch.stft for the mel feature computation, and updated the batch tests for both the mel and encodec variants as you requested, everything is now passing.

@Manalelaidouni I pushed some small fixes for the integration tests. One of them is still failing (`test_inference_batch_mel_vocos`), can you try the following command?
```bash
RUN_SLOW=1 pytest tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest
```

I'm getting the following (truncated) output:
```
tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_batch_encodec_vocos PASSED                                                                                                          [ 25%]
tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_inference_batch_mel_vocos FAILED                                                                                                    [ 50%]
tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_inference_encodec_vocos PASSED                                                                                                      [ 75%]
tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_inference_mel_vocos PASSED                                                                                                          [100%]

====================================================================================================== FAILURES ======================================================================================================
______________________________________________________________________________ VocosModelIntegrationTest.test_inference_batch_mel_vocos ______________________________________________________________________________

self = <tests.models.vocos.test_modeling_vocos.VocosModelIntegrationTest testMethod=test_inference_batch_mel_vocos>

    def test_inference_batch_mel_vocos(self):
        repo_id = self.mel_batch_expected["hf_repo_id"]
        processor = VocosProcessor.from_pretrained(repo_id)
        model = VocosModel.from_pretrained(repo_id).to(torch_device).eval()
    
        audios = self._load_datasamples(3)
    
        inputs = processor(audio=audios, return_tensors="pt").to(torch_device)
        hf_batch_output = model(**inputs).audio
    
        for i, saved in enumerate(self.mel_batch_expected["reconstructed_audio"]):
            expected = torch.tensor(saved, dtype=torch.float32, device=torch_device)
>           torch.testing.assert_close(
                hf_batch_output[i, : expected.shape[0]],
                expected,
                rtol=1e-4,
                atol=1e-4,
            )
E           AssertionError: Tensor-likes are not close!
E           
E           Mismatched elements: 20 / 20 (100.0%)
E           Greatest absolute difference: 0.0004398916498757899 at index (0,) (up to 0.0001 allowed)
E           Greatest relative difference: 1.1968653202056885 at index (0,) (up to 0.0001 allowed)

tests/models/vocos/test_modeling_vocos.py:333: AssertionError
```

And also made some changes more amenable to Transformers, with naming and going back to original Vocos structure (modules for ISTFT and the ISFTT head) which will work well with Modular for Xcodec2 that I'm working on #37868

FYI: I updated the conversion scripts accordingly, pushed checkpoints to my HF account, and updated the repo IDs in the modeling tests. You can also run the conversions scripts yourself and go back to checkpoints on your account if you need to debug the integration test that's failing!

Hey @ebezzam good luck with the xcodec 2 model! all tests pass for me, including the test_inference_batch_mel_vocos
test 
```
tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_batch_encodec_vocos PASSED                                     [ 25%]
tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_inference_batch_mel_vocos PASSED                               [ 50%]
tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_inference_encodec_vocos PASSED                                 [ 75%]
tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_inference_mel_vocos PASSED                                     [100%]


============================================================== 4 passed in 51.04s ============================================================== 

```


run-slow: vocos

This comment contains run-slow, running the specified jobs: 

models: ['models/vocos']
quantizations: [] ...

@Manalelaidouni Thanks for checking the tests. `test_inference_batch_mel_vocos` is still failing for me locally, and also on the GitHub Actions (see [here](https://github.com/huggingface/transformers/actions/runs/18277033466/job/52031520604)).

I'm guessing you obtained the expected outputs and ran the tests on CPU? When I run the integration tests on CPU (like below), they pass.
```
CUDA_VISIBLE_DEVICES="" RUN_SLOW=1 pytest tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest
```
Could you re-compute the expected outputs (at least for `test_inference_batch_mel_vocos`) on GPU?

@Manalelaidouni no need anymore, I was able to re-create thanks to your reproducer scripts 🙂 https://gist.github.com/Manalelaidouni/853f4c902ab0ce0a512e5217d87d564c

could you update the scripts on Gist? some imports from Vocos and transformers were missing

run-slow: vocos

This comment contains run-slow, running the specified jobs: 

models: ['models/vocos']
quantizations: [] ...

hey @ebezzam, thanks, I already reporduced the fixtures on gpu but that's not the issue here;  `torch.testing.assert_close(orig_mels, hf_mels, rtol=1e-3, atol=1e-3)` still fails, I'm working on pin pointning the exact issue why the feature extractor between original Vocos and hf counterpart are identical on cpu but drift on cuda, I I don't want to change the tolerance level either, so i'm inspecting the numerical mismatch issue on gpu rn, will update soon 

@Manalelaidouni Thanks keep me posted! What is interested is that the slow test [passes](https://github.com/huggingface/transformers/actions/runs/18284649874/job/52056561043) (namely `test_inference_batch_mel_vocos` computed on GPU) with the new expected outputs, while `test_inference_batch_mel_vocos` now fails in [this test](https://app.circleci.com/pipelines/github/huggingface/transformers/148902/workflows/a64173c0-e784-4a6d-9803-fbeaa13b3991/jobs/1963519?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-checks-link&utm_content=summary).

From what I've seen `(Original-CPU, Transformers-CPU)` and `(Original-GPU, Transformers-GPU)` are close. What was being tested before, namely (Original-CPU, Transformers-GPU), was failing before. ~~Maybe this new failing test is computed on CPU (I'll ask), and we may have to store expected outputs for both CPU and GPU.~~ Yes those are computed on CPU!

EDIT: just noticed, the `@slow` decorator was missing around the integration tests. I have added it.



> @Manalelaidouni Thanks keep me posted! What is interested is that the slow test [passes](https://github.com/huggingface/transformers/actions/runs/18284649874/job/52056561043) (namely `test_inference_batch_mel_vocos` computed on GPU) with the new expected outputs, while `test_inference_batch_mel_vocos` now fails in [this test](https://app.circleci.com/pipelines/github/huggingface/transformers/148902/workflows/a64173c0-e784-4a6d-9803-fbeaa13b3991/jobs/1963519?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-checks-link&utm_content=summary).
> 
> From what I've seen `(Original-CPU, Transformers-CPU)` and `(Original-GPU, Transformers-GPU)` are close. What was being tested before, namely (Original-CPU, Transformers-GPU), was failing before. ~Maybe this new failing test is computed on CPU (I'll ask), and we may have to store expected outputs for both CPU and GPU.~ Yes those are computed on CPU!
> 
> EDIT: just noticed, the `@slow` decorator was missing around the integration tests. I have added it.

<br>

@ebezzam I think I found why the same test passes in the one CI job  [here](https://github.com/huggingface/transformers/actions/runs/18284649874/job/52056561043)  and not [here](https://app.circleci.com/pipelines/github/huggingface/transformers/148902/workflows/a64173c0-e784-4a6d-9803-fbeaa13b3991/jobs/1963519?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-checks-link&utm_content=summary) , if you check the torch and torchaudio wheels installed in each env (in *Show installed libraries and their versions*), the first installs 2.8.0+cu126 and the second 2.8.0+cpu, the wheel mismatch makes torchaudio.transforms.MelSpectrogram produce slightly different numbers, i reproduced this in this gist https://gist.github.com/Manalelaidouni/d015f5b33f69fc1db5a1b1343e268fdb

Hey @ebezzam, I just pushed an update the HF and original feature extractors now match on gpu and `torch.testing.assert_close(orig_mels, hf_mels, rtol=1e-3, atol=1e-3)` in the gist passes for both single and batched inputs, can you run the gpu tests in github actions when you get a chance? 

run-slow: vocos

This comment contains run-slow, running the specified jobs: 

models: ['models/vocos']
quantizations: [] ...

@Manalelaidouni the GitHub actions fail now [link](https://github.com/huggingface/transformers/actions/runs/18305498884) due to device mappings. But no worries! I can handle it, and I'll do some other tweaks for some standardization we are moving towards! 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

run-slow: vocos

This comment contains run-slow, running the specified jobs: 

models: ['models/vocos']
quantizations: [] ...

@ebezzam so you’ve removed the numpy support entirely and moved the encodec padding into the feature extractor, since we pivoted a few times, is this the final arrangement you’d like to keep?

@Manalelaidouni yes I wanted to update you on points that may help you for the YuE model addition.

And sorry for feedback that may seem "unnecessary" in hindsight. We're in the middle of a big standardization to make audio model interfaces/additions more intuitive. So rest assured, our iterations did help us better understand certain pain points and improvements. Thank you for your patience 🙏 

Some things you can keep in mind for your YuE model addition:
- Feature extractor should return a `padding_mask` so that we can get equivalent outputs between single and batch outputs (by using `padding_mask` to ignore/truncate corresponding samples). This was also why I moved the padding for encodec to the feature extractor (to have `padding_mask` for its audio inputs).
- Torch+batch processing should be default behavior for the feature extractor (for efficiency). Moreover, numpy is not necessary, which should hopefully simplify model addition and maintenance.*
- Using more intuitive variable names for the model inputs, e.g. `audio` and `audio_spectrogram` rather than `features`, `input_features`, etc... **

Hope that helps! And thanks again for your contributions 🙂 

_* For example, with Vocos we saw differences between torchaudio on CPU and GPU, and these would likely be present for numpy vs torch implementations. Also, more changes would have been necessary for the latest numpy branch to make sure it was equivalent to the torch processing: move padding to before computing the spectrogram (also so `padding_mask` is equivalent), adding tests because none of them use an environment that doesn't have torch to make sure feature extraction is similar, etc_
_** I still kept `input_features` for this model because embeddings from the audio_codes could be passed an input, and we haven't thought of a standardized name for such inputs (also not very common)_

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

hey @ebezzam, the feature extractor fails to properly handle gpu computation, the assertion between batched outputs for mel variants on GPU was failling  `torch.testing.assert_close(original_batch_output, hf_batch_output, rtol=1e-1, atol=1e-1)` (tests in gists), this meant even on the same device and same operation, the original and hf feature extractor were not close. It turns out the device inside feature extractor is still on cpu and due to the device mismatch in torchaudio it produces different values. 

This happens because when we pass a single or a list of numpy arrays in gpu (which we also do in modeling integration test `test in test_inference_batch_mel_vocos ` and expect users to pass) the device default to cpu.

Same issue occurs if you run the usage examples in docs with device being gpu, the computation in feature extractor would not be in gpu even if we explicitly move processor to cuda via  `processor(audio).to(device)` 

I added a warning to require users to pass device, but wanted to ask since there's no reliable way to auto detect the correct device should we just use  torch.cuda.is_available (haven't seen it used in core modeling code in Transformers). This would let `model.to(device)` handle everything, also is more intuitive to users to do than `processor(audio, device=device)` , thanks let me know what you think before I adjust modeling tests and docs to pass device to the processor as well.




**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

@Manalelaidouni thanks for cleaning up the doctrings and the feature extraction test! I would lean towards not having the warning in the feature extractor (to limit outputs to the user).

For `device` I took inspiration from [GraniteSpeech](https://github.com/huggingface/transformers/blob/be3fa93b298eec4182a0bb6c5e64270a5f1e8020/src/transformers/models/granite_speech/feature_extraction_granite_speech.py#L69) but a bit "smarter", where I default to CPU if the user provides a Numpy array, but if the user provides a Torch tensor it defaults to the device of the user-provided audio. That's what the snippet below is doing:
```python
        if device is None:
            if isinstance(audio[0], torch.Tensor):
                device = audio[0].device
            else:
                device = "cpu"
```

So the only reason why a user would need to do `processor(audio, device=device)` is if they want to process on a device different than that of `audio`. Moreover, if they don't have a GPU on their machine, the warning wouldn't help them (unless we only warn when cuda available is True).

I've asked @eustlb his feedback on the `device` input [here](https://github.com/huggingface/transformers/pull/39403#discussion_r2410844005), and we can wait to see what he thinks before changing the modeling tests and docs. I did however add a decorator to the test to only run them on GPU. In any case, that's also the case with `@slow` but it is more explicit that way.

EDIT: I've added an [example](https://huggingface.co/hf-audio/vocos-mel-24khz#feature-extraction-on-gpu) on the Hub for users to compute the spectrogram on GPU.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

@Manalelaidouni FYI I pushed a change for proper batch processing, namely `pad_to_multiple_of` has to be set corresponding to the hop length of the spectrogram or the codec.

I added a batch processing example to the hub as well ([mel](https://huggingface.co/hf-audio/vocos-mel-24khz#batch-processing) and [encodec](https://huggingface.co/hf-audio/vocos-encodec-24khz#batch-processing)).

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

Thanks @ebezzam, but with the padding change the output now no longer matches the original vocos output, for instance if we pass a single audio input to the mel path, the feature extractor now is padding the audio, e.g from 140520 to 140544, which makes the generated spectrogram come out different with a different shape [1, 100, 549] than the original [1, 100, 550]; for the encodec variant the output is also different than the original vocos output, I'll fix it and push a change soon.  

@Manalelaidouni thanks for checking. Yes I updated the feature extractor test for the larger spectrogram size. 

However, the integration tests still passed on my machine. Were you running another test where they failed? (Maybe from your gist, but what if you use the padding mask to trim?)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

hey @ebezzam so `pad_to_multiple_of` change was causing unintended padding even for single inputs for both mel and encodec (single audio [140520, 1] padded into [1, 140544] for mel and into [1, 140800] for encodec) causing diff output shapes , the batching between original and hf was also different because they're using different padding, and that was being used to generate values in the fixtures; now I updated the code and I also updated my batching gist so that the padding used is identical to pad to multiple of hop_length and not max length like we used to do; also the script in gist allow to inspect if each component of the model is accurate, comparing to the original, this way we catch issues,  btw I also removed the @slow decorators so the tests show on CI job, i'll put them back again :) 

**Update** : now everything passes, test assertions in gist (all match the original) and integration tests on both CPU and GPU. 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

run-slow: vocos

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

This comment contains run-slow, running the specified jobs: 

models: ['models/vocos']
quantizations: [] ...

@Manalelaidouni can you check again for `test_inference_batch_mel_vocos`? On my machine I'm getting errors. Maybe they were still computed on CPU?
```
# RUN_SLOW=1 pytest tests/models/vocos
    @slow
    @require_torch_gpu
    def test_inference_batch_mel_vocos(self):
        repo_id = "hf-audio/vocos-mel-24khz"
        processor = VocosProcessor.from_pretrained(repo_id)
        model = VocosModel.from_pretrained(repo_id).to(torch_device).eval()
    
        audios = self._load_datasamples(3)
    
        inputs = processor(audio=audios, return_tensors="pt").to(torch_device)
        hf_batch_output = model(**inputs).audio
    
        for i, saved in enumerate(self.mel_batch_expected["reconstructed_audio"]):
            expected = torch.tensor(saved, dtype=torch.float32, device=torch_device)
>           torch.testing.assert_close(
                hf_batch_output[i, : expected.shape[0]],
                expected,
                rtol=1e-4,
                atol=1e-4,
            )
E           AssertionError: Tensor-likes are not close!
E           
E           Mismatched elements: 20 / 20 (100.0%)
E           Greatest absolute difference: 0.0004398916498757899 at index (0,) (up to 0.0001 allowed)
E           Greatest relative difference: 1.1968653202056885 at index (0,) (up to 0.0001 allowed)

tests/models/vocos/test_modeling_vocos.py:333: AssertionError

FAILED tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_inference_batch_mel_vocos - AssertionError: Tensor-likes are not close!
```


I also launched the slow tests to see with the GitHub actions (no need to comment/remove the `slow` decorators in the future! I can run `run-slow: vocos` just to be sure we don't want to forget them before merging 🙈 )

EDIT: Slow tests finished, and it is the [same error](https://github.com/huggingface/transformers/actions/runs/18406689075/job/52448305744) I was seeing on my machine.

@ebezzam 

this is because the computation in the feature extractor does not map to device properly, and with the torchaudio mismatch, the batch test on gpu blows up with the fixtures generated on cpu, I think the solution here is to either to increase the tolerance or update processor passing `processor(audio, device=device)` in the tests and docs because this `processor(audio).to(device)` doesn't work or auto detect for device inside feature extractor by checking for cuda (I asked a while back which option you prefer but hadn’t heard back,  let me know which way you’d like to go and I’ll update).

btw an odd thing I was runnig on T4 gpu the test passed; 
```
tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_inference_batch_mel_vocos PASSED [100%]

============================== 1 passed in 6.21s ===============================

```
but then when I used A100 it failed
```
FAILED tests/models/vocos/test_modeling_vocos.py::VocosModelIntegrationTest::test_inference_batch_mel_vocos - AssertionError: Tensor-likes are not close!
========================= 1 failed, 3 passed in 26.47s =========================

```
so torchaudio not only has device mismatch on cpu vs gpu, even with different gpu cards 


@Manalelaidouni thanks for the info! Actually it is expected for different GPU devices/cards to produce different outputs, and we don't require having integration tests that pass for CPU + multiple GPU types (otherwise we would have to store multiple expected output types for each device).

I'm running on A100, and typically if it passes on that, it is fine for our GitHub Actions (which use A10G from what I see in the [GitHub action](https://github.com/huggingface/transformers/actions/runs/18406689075/job/52448305744)). In some rare cases where it passes on A100 but not on the GitHub Actions, we SSH to the GitHub action machines and compute the expected outputs on them.

But in this case, since they were passing, we can use the expected outputs that were passing before. I think before you did [this change](https://github.com/huggingface/transformers/pull/39403/commits/0a03739478444fa911f8ef8e2b0cdf6ced428830)?

Hope that's clear! (Your rigor is awesome btw but we typically don't go that far with our tests 😅 )



**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

@Manalelaidouni Thanks! Tests are again passing on my machine and will try GitHub actions soon.

Also I flipped back again to the functional ISTFT as you [used to have](https://github.com/huggingface/transformers/pull/39403#discussion_r2293771897) 🙈  because @eustlb and I had the [same idea](https://github.com/huggingface/transformers/pull/39403#discussion_r2419114404) of putting it in audio utils. 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

run-slow: vocos

This comment contains run-slow, running the specified jobs: 

models: ['models/vocos']
quantizations: [] ...

> @Manalelaidouni Thanks! Tests are again passing on my machine and will try GitHub actions soon.
> 
> Also I flipped back again to the functional ISTFT as you [used to have](https://github.com/huggingface/transformers/pull/39403#discussion_r2293771897) 🙈 because @eustlb and I had the [same idea](https://github.com/huggingface/transformers/pull/39403#discussion_r2419114404) of putting it in audio utils.

Nice, great idea actually, now it looks like the model is good to go right?

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos

@eustlb @ebezzam I’ll work on splitting this back into two models `Vocos`  and `VocosEncodec` 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

@Manalelaidouni thanks! I've started the splitting. Before you start/continue, let me confirm something with @eustlb first about the Encodec variant.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

hey @ebezzam are you done with the changes so I can handle the rest?

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=39403&sha=c5012c

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=39403&sha=5a8643

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=39403&sha=bd4727

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=39403&sha=b7bac4

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=39403&sha=d55b0f

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=39403&sha=c504d3

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=39403&sha=4cc616

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=39403&sha=39af5a

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=39403&sha=3e8df7

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, vocos, vocos_encodec

Hey @eustlb  @ebezzam I pushed few changes and the PR is in a mergeable shape again, would appreciate your review when you have a moment. The CI failures look unrelated except for the date updating in docs,

- Removed `return_audio_only` from feature extractor and simplified the processor flow.
- feature extractor and processor return a, `attention_mask`   so batch outputs can be trimmed consistently, models now accept and passes it through both `VocosModel` and `VocosEncodecModel` so that there is output trimming for batched audio uses the mask (similar to Parakeet style handling)
- Updated tests accordingly, fixtures, gist and the model doc cards.