noamaanMulla-03/transformers-issues huggingface dataset:
Use deep copies instead of shallow copies for bbox_embed in GroundingDINO decoder (#37333).
Author: rabibastinj
Pull request: yes

Fixes [#37333](https://github.com/huggingface/transformers/issues/37333)

The decoder was using shallow copies of `GroundingDinoMLPPredictionHead`, causing all `bbox_embed` layers to share the same instance, which led to unintended parameter sharing. This PR updates it to use `copy.deepcopy()`.

### What I Changed

- Updated the model implementation to use `copy.deepcopy()` when `decoder_bbox_embed_share=True`.
- Added two unit tests under `tests/models/Rabi_Testcases/`:
  - ✅ `GD_Unique_Instances_Testcase.py` – verifies each decoder layer gets a unique instance
  - ✅ `GD_Layer_Independency_Testcase.py` – ensures that modifying one layer does not affect another

### Testing

- Ran: `pytest tests/models/Rabi_Testcases/`
- ✅ All tests passed successfully


Comments:
cc @qubvel, but @rabibastinj please delete the tests that copilot added - we don't want to add whole new test files just for this!
Add submodels support check function
Author: ivarflakstad
Pull request: yes


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39009). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

once ready (i.e. the new function is used to some test files), ping me :-)
Fix: Ensure wandb logs config in offline mode
Author: DavidS2106
Pull request: yes

- Fixes issue with wandb not logging config in offline mode
- Added config update after wandb.init()
- Verified with run_glue.py in WANDB_MODE=offline

Comments:
cc @sunmarc @mekkcyber

@bot /style

Style fixes have been applied. [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/15875478738).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38992). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[WIP] Fix DeepseekV3ModelTest::test_torch_compile_for_training
Author: ivarflakstad
Pull request: yes

`DeepseekV3ModelTest::test_torch_compile_for_training` fails with `torch._dynamo.exc.Unsupported: Dynamic shape operator`. Attempting to remedy by  `torch._dynamo.config` flags.

Pytorch was struggling to capture the graph when we were using `if token_indices.numel() > 0:`, so I switched to only loop over the experts that are matching from the expert mask. Should be an improvement regardless of capturing.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39012). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: deepseek_v3

This comment contains run-slow, running the specified jobs: 

models: ['models/deepseek_v3']
quantizations: [] ...

run-slow: deepseek_v3

This comment contains run-slow, running the specified jobs: 

models: ['models/deepseek_v3']
quantizations: [] ...

run-slow: deepseek_v3

This comment contains run-slow, running the specified jobs: 

models: ['models/deepseek_v3']
quantizations: [] ...

run-slow: deepseek_v3

This comment contains run-slow, running the specified jobs: 

models: ['models/deepseek_v3']
quantizations: [] ...
fix: remove undefined variable
Author: ybkurt
Pull request: yes

`MusicgenSinusoidalPositionalEmbedding` seems like a modified copy of `Speech2TextSinusoidalPositionalEmbedding`, however, in `__init__` method, `self.offset` is not defined for Musicgen. This removes the undefined `self.offset` variable from `MusicgenSinusoidalPositionalEmbedding` implementation.
 
@eustlb


Comments:
Update expected values (after switching to A10)
Author: ydshieh
Pull request: yes

# What does this PR do?

would merge as discussed offline

Comments:
run-slow: conditional_detr, convnext, cvt, d_fine, dab_detr, deformable_detr, detr

This comment contains run-slow, running the specified jobs: 

models: ['models/conditional_detr', 'models/convnext', 'models/cvt', 'models/d_fine', 'models/dab_detr', 'models/deformable_detr', 'models/detr']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39157). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: cvt

This comment contains run-slow, running the specified jobs: 

models: ['models/cvt']
quantizations: [] ...

run-slow: cvt

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: conditional_detr, convnext, cvt, d_fine, dab_detr, deformable_detr, detr

This comment contains run-slow, running the specified jobs: 

models: ['models/cvt']
quantizations: [] ...
fix `llama` tests
Author: ydshieh
Pull request: yes

# What does this PR do?

Update expected values of `test_llama_3_1_hard` on `A10`.

Also need

```python
    # TODO: check why we have the following strange situation.
    # without running in subprocess, this test causes subsequent tests failing with `RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!`
    @run_test_using_subprocess
    @slow
    def test_model_7b_dola_generation(self):
```

which is never run before 2025/06/24

Comments:
run-slow: llama

This comment contains run-slow, running the specified jobs: 

models: ['models/llama']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39161). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: llama
remove _run_third_party_device_tests
Author: jiqing-feng
Pull request: yes

Hi @SunMarc . I ran the transformers tests for XPU and found that the `_run_third_party_device_tests` blocks me from setting XPU as the device unless I set the parameter. I cannot see any reason to keep this parameter. There is no limitation for CUDA, so I suppose we should not have limitations for other devices like XPU.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@SunMarc, do we need more people to discuss on this from Hugging Face side. From our side, we think it's great that we remove `_run_third_party_device_tests` to let the UTs auto detect the accelerator in the platforms and run test.  

yeah should be safe to merge, looks like @ji-huazhong is fine with the change
fix mpt test of different outputs from cuda
Author: jiqing-feng
Pull request: yes

The tests `RUN_SLOW=1 pytest -rA tests/models/mpt/test_modeling_mpt.py::MptIntegrationTests::test_generation_batched` got result error on XPU.

The outputs of the MPT model are slightly different from CUDA, we need another result check for XPU. Besides, we need to use require_deterministic_for_xpu so xpu can get stable outputs.

Hi @SunMarc , please review this PR. Thanks

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37691). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
update bnb tests
Author: jiqing-feng
Pull request: yes

Update bnb tests results as bnb has new kernels, the output will be slightly different, but still reasonable.

@SunMarc @ydshieh  Please review it

cc @Titus-von-Koeller @matthewdouglas 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38011). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
assign the correct torchao data layout for xpu
Author: jiqing-feng
Pull request: yes

Hi @Rocketknight1 @ArthurZucker . Same as CPU, the XPU requires a specific data layout to make torchao work. Please review this PR. Thanks! 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @IlyasMoutawwakil 

gentle ping @IlyasMoutawwakil 

This is probably  better for @MekkCyber or @SunMarc 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37781). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Very nice 🔥 ! thanks for adding this 🤗 ! Can you add a test for xpu please ?

Yes, I will add tests after torch2.8 is released. Currently, we only support the pre-release torch version for advanced users.
update emu3 test
Author: jiqing-feng
Pull request: yes

Hi @zucchini-nlp. This PR fixed:
1. The correct usage of [decode_image_tokens](https://github.com/huggingface/transformers/blob/main/src/transformers/models/emu3/modeling_emu3.py#L1619) because the input only supports `**kwagrs`
2. Use atol=1 to check the result to avoid numerical precision error across different devices. The pixel values are uint8, ranging from 0 to 255. atol=1 will only handle very small precision errors, which is in our expectation.

Comments:
Rely on this [PR ](https://huggingface.co/datasets/raushan-testing-hf/images_test/discussions/2 )merge

run-slow: emu3

Hi @zucchini-nlp . You need to merge [this PR](https://huggingface.co/datasets/raushan-testing-hf/images_test/discussions/2) first so the test could pass

This comment contains run-slow, running the specified jobs: 

models: ['models/emu3']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38543). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Test is on large accelerator so not run. Verified locally, so merging
fix torch_dtype on awq
Author: jiqing-feng
Pull request: yes

Hi @SunMarc . The autoawq support CPU and XPU now, we should update the awq torch_dtype because CPU/XPU support both bf16/fp16. For CPU, the performance is better on bf16 in most cases.

Comments:
Hi @SunMarc . It seems that auto-merge didn't work. Could you merge this PR manually? Thanks!

Hi @SunMarc . The failed test is not related to my changes.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38463). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@bot /style

Hi @SunMarc . The `build / build_pr_documentation` is always pending, do you know why?

> Hi @SunMarc . The build / build_pr_documentation is always pending, do you know why?

I need to trigger its run.
Optimize 1st token for beam_search
Author: jiqing-feng
Pull request: yes
Labels: name: Generation

We can repeat the outputs after the first token instead of repeating the first token inputs. The outputs are identical because the 1st token inputs are the same in each beam.

It could avoid extra computation.

I will update the rest of the generation methods once we agree on this change.


Comments:
cc @gante @zucchini-nlp 

Hi @gante @zucchini-nlp . Could you please take a review? Thx!

Hi @jiqing-feng 👋 

The PR moves in the right direction, but we will want a different interface for it -- see "prefill" in [this tracker](https://github.com/huggingface/transformers/issues/30810). The prefill change is quite sensible with respect to interface, so we'll want to work on it ourselves. However, we'll be open to contributions after we add the prefill stage 🤗 

> Hi @jiqing-feng 👋
> 
> The PR moves in the right direction, but we will want a different interface for it -- see "prefill" in [this tracker](https://github.com/huggingface/transformers/issues/30810). The prefill change is quite sensible with respect to interface, so we'll want to work on it ourselves. However, we'll be open to contributions after we add the prefill stage 🤗

Thx! Please let me know when the PR is ready :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
fix low-precision audio classification pipeline
Author: jiqing-feng
Pull request: yes

Fix low-precision usage for audio-classification pipeline.

Require test

Comments:
cc @eustlb

Hi @LysandreJik @eustlb . The test is added, please run
`RUN_SLOW=1 pytest ./tests/pipelines/test_pipelines_audio_classification.py::AudioClassificationPipelineTests::test_small_model_pt_fp16 -rsx -v`
to check it. Thanks!

Waiting for review ....

Hi @SunMarc , please review this PR, thanks!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35435). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi @ArthurZucker @Rocketknight1 . Would you please review this PR? Thanks!
Fix vits low-precision dtype
Author: jiqing-feng
Pull request: yes

This PR fixed vits model dtype when torch_dtype=torch.float16.

To reproduce the error:
```python
import torch
from transformers import pipeline

pipe = pipeline("text-to-speech", model="facebook/mms-tts-eng", torch_dtype=torch.float16)
output = pipe("Hello, my dog is cooler than you!")
print(output)
```

Traceback:
```
......
  File "/home/jiqing/transformers/src/transformers/models/vits/modeling_vits.py", line 916, in forward
    query_states = self.q_proj(hidden_states) * self.scaling
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py", line 125, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 must have the same dtype, but got Float and Half
```

Required test?

Comments:
cc @eustlb

Hi @LysandreJik @eustlb . The test is added, please run
`RUN_SLOW=1 pytest ./tests/models/vits/test_modeling_vits.py::VitsModelIntegrationTests::test_forward_fp16`
to check the PR. Thanks

Waiting for review ...

Hi @SunMarc , do you mind reviewing this PR? The error is easy to reproduce. Thanks

Hi @ArthurZucker @Rocketknight1 . Would you please review this PR? Thanks!
fix document qa bf16 pipeline
Author: jiqing-feng
Pull request: yes

This PR enables low-precision document-question-answering pipeline, the error can be reproduced by
```python
from transformers import pipeline
from PIL import Image
import torch

pipe = pipeline("document-question-answering", model="impira/layoutlm-document-qa", torch_dtype=torch.bfloat16)

question = "What is the purchase amount?"
image = Image.open("document.png")

ans = pipe(image=image, question=question)
print(ans)
```

```
Traceback (most recent call last):
  File "/home/jiqingfe/jiqing_hf/HuggingFace/tests/workloads/test_qa.py", line 10, in <module>
    ans = pipe(image=image, question=question)
  File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/document_question_answering.py", line 278, in __call__
    return super().__call__(inputs, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/base.py", line 1354, in __call__
    return next(
  File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/pt_utils.py", line 125, in __next__
    processed = self.infer(item, **self.params)
  File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/document_question_answering.py", line 459, in postprocess
    answers = self.postprocess_extractive_qa(model_outputs, top_k=top_k, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/document_question_answering.py", line 488, in postprocess_extractive_qa
    starts, ends, scores, min_null_score = select_starts_ends(
  File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/question_answering.py", line 130, in select_starts_ends
    start = np.where(undesired_tokens_mask, -10000.0, start)
  File "/usr/local/lib/python3.10/dist-packages/torch/_tensor.py", line 1216, in __array__
    return self.numpy()
TypeError: Got unsupported ScalarType BFloat16
```

Comments:
cc @Rocketknight1 

The test has been added, please run 
`RUN_SLOW=1 pytest ./transformers/tests/pipelines/test_pipelines_document_question_answering.py::DocumentQuestionAnsweringPipelineTests::test_small_model_pt_bf16 -rsx -v` to check it. Thanks

Hi @SunMarc , please review this PR, thanks!

> SGTM ! Please update the description of the PR to add more detail or link the relevant PR

Hi @SunMarc . I have updated the description and showed how to reproduce the error, please check it. Thanks!

Hi @ArthurZucker @Rocketknight1 . Would you please review this PR? Thanks!
fix audio classification pipeline fp16 test on xpu
Author: jiqing-feng
Pull request: yes

Fix #36340 

Comments:
Thank you @jiqing-feng for this. I will wait for a response from @dvrogozh 🙏  

Hi @dvrogozh . I have fixed codes by your comments, please review the new changes. Thanks!

> Thank you @jiqing-feng for this. I will wait for a response from @dvrogozh 🙏

Hi @ydshieh . I suppose this PR is ready to be merged. Thanks!
Fix llava xpu tests.
Author: jiqing-feng
Pull request: yes

To reproduce:
`TRANSFORMERS_TEST_DEVICE=xpu ZE_AFFINITY_MASK=0 RUN_SLOW=1 pytest -rA tests/models/llava/test_modeling_llava.py::LlavaForConditionalGenerationIntegrationTest::test_pixtral_4bit`

Hi @SunMarc . This PR fixed llava tests on XPU because the output slightly differs from cuda. Please review it. Thanks!

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @ydshieh . I have fixed the format, please review it. Thanks!
feat: support FSDP config through a dataclass and allow for state_dict_type
Author: kmehant
Pull request: yes

# What does this PR do?

The PR introduces two changes broadly

1. Introduces a new data class to parse FSDP config for transformer + torchrun users
2. allows for `state_dict_type` FSDP option by passing it to FSDP accelerator plugin through env variable

For (1) changes in the trainer code are kept to 0 and impact is mostly concentrated on the trainer args by equipping the dataclass object with `get` and `subscriptable` dict APIs.


Fixes #29476

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. #29476
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@muellerzr and @SunMarc



Comments:
Closing this since, we have moved to using accelerate config for passing FSDP configs isntead of trainer args.
Why is return_assistant_tokens_mask and continue_final_message incompatible?
Author: nyxkrage
Pull request: no

I'm currently authoring a new chat template, and while debugging encountered the check for this, however when uncommenting the check, the resulting mask and template both seem to still be correct. So I'm curious as to why or whether this check is needed at all?

I can see it was introduced in [the original PR](https://github.com/huggingface/transformers/pull/33198), however there doesn't seem to be any justification/explanation for this assertion.

Comments:
Hi @nyxkrage,  there are two main reasons. The first is that they're usually used in different contexts. `continue_final_message` leaves the final message open for generation, but `return_assistant_tokens_mask` is generally used for masking during training.

The second reason is that `continue_final_message` does some hacky string post-processing on the template so that it works with multiple template formats, and we weren't sure if it would work well with the generation tags used for `return_assistant_tokens_mask`. If you make a PR to enable those options together and add a test confirming that it works, then we'd be happy to review it!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
fix(modeling_utils): Correctly call _init_weights in smart_apply
Author: Flink-ddd
Pull request: yes

## Description

This PR fixes issue #39124.

The `smart_apply` helper function in `modeling_utils.py` is used to recursively initialize weights in a `PreTrainedModel`. The current implementation checks if a submodule has a custom `_init_weights` method. However, if it finds one, it incorrectly attempts to pass `module._initialize_weights` in the subsequent recursive call.

This leads to an `AttributeError` for any module that defines a custom `_init_weights` method but not an `_initialize_weights` method (e.g., the `Resampler` module used in certain large models, or other custom architectures).

The error reported is: `AttributeError: 'Resampler' object has no attribute '_initialize_weights'`

## Solution

The fix is to align the function being passed in the recursive call with the condition being checked. I've changed the recursive call to use `module._init_weights`, ensuring that the intended custom initialization function is correctly dispatched.

related test screenshot:

<img width="791" alt="Screenshot 2025-07-01 at 23 44 58" src="https://github.com/user-attachments/assets/a7df27ee-520d-490b-9d9d-acfc07e45d52" />


Comments:
This was closed so I suspect you noticed it, but indeed this is not a bug and is perfectly intended. There are no issues with the current implementation 🤗

Hi @Cyrilvallez , I noticed it, I already tell this situation to questioner.