Repository: huggingface/transformers
Pull request: Add codebook_dim attribute to DacVectorQuantize for DacResidualVectorQuantize.from_latents()

# What does this PR do?
This PR fixes an issue where DacResidualVectorQuantize.from_latents required an additional attribute from the DacVectorQuantize class.
- Added the missing instance attribute in the class initialization.
- Prevents runtime errors when calling from_latents.

## Before submitting
- [x] Bug-fix (non-breaking change)
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Your PR will be replied to more quickly if you can figure out the right person to tag with @
@eustlb


Changed files:
filename: src/transformers/models/dac/modeling_dac.py
patch: @@ -115,6 +115,7 @@ class DacVectorQuantize(nn.Module):
     def __init__(self, config: DacConfig):
         super().__init__()
 
+        self.codebook_dim = config.codebook_dim
         self.in_proj = nn.Conv1d(config.hidden_size, config.codebook_dim, kernel_size=1)
         self.out_proj = nn.Conv1d(config.codebook_dim, config.hidden_size, kernel_size=1)
         self.codebook = nn.Embedding(config.codebook_size, config.codebook_dim)

filename: tests/models/dac/test_modeling_dac.py
patch: @@ -375,6 +375,23 @@ def test_identity_shortcut(self):
         config.use_conv_shortcut = False
         self.model_tester.create_and_check_model_forward(config, inputs_dict)
 
+    def test_quantizer_from_latents(self):
+        config, inputs_dict = self.model_tester.prepare_config_and_inputs()
+        model = DacModel(config=config).to(torch_device).eval()
+        self.assertTrue(
+            all(hasattr(quantizer, "codebook_dim") for quantizer in model.quantizer.quantizers),
+            msg="All quantizers should have the attribute codebook_dim",
+        )
+        with torch.no_grad():
+            encoder_outputs = model.encode(inputs_dict["input_values"])
+            latents = encoder_outputs.projected_latents
+            quantizer_representation, quantized_latents = model.quantizer.from_latents(latents=latents)
+
+        self.assertIsInstance(quantizer_representation, torch.Tensor)
+        self.assertIsInstance(quantized_latents, torch.Tensor)
+        self.assertEqual(quantized_latents.shape[0], latents.shape[0])
+        self.assertEqual(quantized_latents.shape[1], latents.shape[1])
+
 
 # Copied from transformers.tests.encodec.test_modeling_encodec.normalize
 def normalize(arr):
@@ -872,3 +889,39 @@ def test_integration_batch(self, model_name):
             # make sure forward and decode gives same result
             enc_dec = model(inputs["input_values"])[1]
             torch.testing.assert_close(decoded_outputs["audio_values"], enc_dec, rtol=1e-6, atol=1e-6)
+
+    @parameterized.expand([(model_name,) for model_name in EXPECTED_PREPROC_SHAPE_BATCH.keys()])
+    def test_quantizer_from_latents_integration(self, model_name):
+        model_id = f"descript/{model_name}"
+        model = DacModel.from_pretrained(model_id).to(torch_device)
+        processor = AutoProcessor.from_pretrained(model_id)
+
+        # load audio sample
+        librispeech_dummy = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
+        librispeech_dummy = librispeech_dummy.cast_column("audio", Audio(sampling_rate=processor.sampling_rate))
+        audio_sample = librispeech_dummy[0]["audio"]["array"]
+
+        # check on processor audio shape
+        inputs = processor(
+            raw_audio=audio_sample,
+            sampling_rate=processor.sampling_rate,
+            return_tensors="pt",
+        ).to(torch_device)
+
+        input_values = inputs["input_values"]
+        with torch.no_grad():
+            encoder_outputs = model.encode(input_values)
+            latents = encoder_outputs.projected_latents
+
+            # reconstruction using from_latents
+            quantizer_representation, quantized_latents = model.quantizer.from_latents(latents=latents)
+            reconstructed = model.decode(quantized_representation=quantizer_representation).audio_values
+
+            # forward pass
+            original_reconstructed = model(input_values).audio_values
+
+        # ensure forward and decode are the same
+        self.assertTrue(
+            torch.allclose(reconstructed, original_reconstructed, atol=1e-6),
+            msg="Reconstructed codes from latents should match original quantized codes",
+        )