noamaanMulla-03/transformers-issues huggingface dataset:
Introduce GradientCheckpointingLayer
Author: qubvel
Pull request: yes

# What does this PR do?

A super minimal abstraction for a layer with gradient checkpointing that keeps the logic for enabling and disabling gradient checkpointing within PreTrainedModel for backward compatibility. It allows for a gradual rollout of the feature by supporting both checkpointing mechanisms: with a the current wrap of `_gradient_checkpointing_func` and using inheritance from `GradientCheckpointingLayer`.

I've applied this to Llama, but it's just a PoC for the discussion. Perhaps it's better to start with another less popular model that has fewer dependent models to see how it goes and check if it can be breaking for the hub custom code

## Who can review?


Comments:
run-slow: llama

This comment contains run-slow, running the specified jobs: This comment contains run-slow, running the specified jobs:

models: ['models/llama']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37223). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Ended up applying to all llama-based models and SigLIP/2 for the first iteration, all relevant tests pass

```
RUN_SLOW=1 pytest -k "gradient" tests/models/
```

CI error is unrelated

cc @ArthurZucker to merge if OK for you

This is very neat, Pavel!

Might be useful for us users to add to the doc:

1. that `"use_reentrant": True` is by default and perhaps pointing to where it gets set if not overridden by the user and 
2. point the reader to `gradient_checkpointing_enable` which defines the doc on how to override `use_reentrant` value.

Perhaps even add redundancy/shortcut by adding to the doc:
```
            gradient_checkpointing_kwargs = {"use_reentrant": True}
            model.gradient_checkpointing_enable(gradient_checkpointing_kwargs=gradient_checkpointing_kwargs)
```
it's not very intuitive, I first tried to call:
```
model.gradient_checkpointing_enable(use_reentrant=True)
```
and it failed `TypeError: PreTrainedModel.gradient_checkpointing_enable() got an unexpected keyword argument 'use_reentrant'`
fix `DbrxModelTest::test_offloaded_cache_implementation_0_offloaded` 
Author: faaany
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
cc @gante @IlyasMoutawwakil - @faaany can you explain the issue that requires this?

Hey 👋 

This test was deleted in #37724, so there is no need to fix it 🤗 Recent PR that might be relevant to you @faaany : #37873 

(I also noticed that overwrites of the deleted test were not deleted -- #37896 deletes them @Rocketknight1 )

great, thanks!

close as the test was deleted.
Fix incorrect type annotation in get_auxiliary_logits
Author: tanuj-rai
Pull request: yes

# What does this PR do?
Fixes #37909,
This PR fix invalid type hint syntax in get_auxiliary_logits (Dict[str, Tensor])

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@amyeroberts @qubvel 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37955). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thank you @qubvel
Add GraniteMoeHybrid support for 4.0
Author: Ssukriti
Pull request: yes

# What does this PR do?

The PR adds support for upcoming Granite4.0 models. It terms of model architecture, it is a hybrid class with shared MLP layer and Bamba layers.


Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:




Comments:
cc @arthurzucker for text models!

ccing @molbap for mamba2/bamba (feels like I'm pinging you constantly 😆)

Feel free to ping me again!

Thanks @ArthurZucker! It's ready for another look when you get the chance!

Thanks @berserkr! There were two because of `modular` expanding the superclass implementation that also set it. Updated to just pass the config value directly so it's less weird looking 🙂 

Thank you very much for the fast review @ArthurZucker! I've made all the changes 🙂 

Thanks @ArthurZucker! Added the missing TOC entry and removed the currently unused shared condition for the MLP, should pass now! 🤞
add xpu memory check 
Author: faaany
Pull request: yes

## What does this PR do?
The issue addressed in this PR https://github.com/huggingface/transformers/pull/37920 also applies to other GPUs. Otherwise, the device index in `device_map` on Intel GPU and NV GPU doesn't align with each other. On CUDA, this is {"": 0}, but on XPU this is {"": 1}, which leads to following test fails:

```bash
tests/peft_integration/test_peft_integration.py::PeftIntegrationTester::test_peft_from_pretrained_kwargs
```

cc @gante and @Rocketknight1


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37969). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[tests] Smaller model in slow cache tests
Author: gante
Pull request: yes

# What does this PR do?

Our CI is failing due to OOM in some slow tests (see `CacheHardIntegrationTest` failures [here](https://github.com/huggingface/transformers/actions/runs/14768710814/job/41465309462)).

This PR replaces the 7B model (requires ~15GB VRAM) with a 4B model (requires ~9GB VRAM) in tests that were using a 7B model. It also makes a few more minor modifications to ensure a green CI (commented in the diff)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37922). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

it's nice to try to use `run-slow: utils` maybe?

run-slow: utils

This comment contains run-slow, running the specified jobs: This comment contains run-slow, running the specified jobs:

models: ['utils']
quantizations: [] ...

(there are some multi-gpu issues, fixing them)

@ydshieh the multi-gpu issues require extensive changes on the offloaded caches, I'm merging this as-is and I'll open a new PR to make multi-gpu work!
ValueError: Tokenizer class LlamaTokenizerFast does not exist or is not currently imported.
Author: hkqiu
Pull request: no
Labels: name: bug

### System Info

transformers              4.26.1                   pypi_0    pypi



### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

tokenizer = AutoTokenizer.from_pretrained(model_path)
ValueError: Tokenizer class LlamaTokenizerFast does not exist or is not currently imported.

### Expected behavior

I use the base model and its config from https://huggingface.co/deepseek-ai/deepseek-llm-7b-chat/blob/main/tokenizer_config.json.
This bug is suddenly out today, while I have used this version for model traning and deploying.
I tried to update to 4.27.0 but still got this bug. Latest version like 4.51.X is not compatible with others needed requirement so maybe I need use 4.2X.X.
Hope someone can help! Thanks!!!

Comments:
add job links to new model failure report
Author: ydshieh
Pull request: yes

# What does this PR do?

Make it is easier to go to the job run page when checking new model failures.

Previous, it looks like

```json
{
    "ydshieh": {
        "vit": {
            "single-gpu": [
                {
                    "test": "tests/models/vit/test_modeling_vit.py::ViTModelTest::test_config",
                    "commit": "e93eb60c536481c646c34d8991fb4fdd78f44d5d",
                    "pr_number": null,
                    "author": "ydshieh",
                    "merged_by": null,
                }
            ]
        }
    }
}
```
Now it has `"job_link": ...` (see the last attribute) like below

```json
{
    "ydshieh": {
        "vit": {
            "single-gpu": [
                {
                    "test": "tests/models/vit/test_modeling_vit.py::ViTModelTest::test_config",
                    "commit": "e93eb60c536481c646c34d8991fb4fdd78f44d5d",
                    "pr_number": null,
                    "author": "ydshieh",
                    "merged_by": null,
                    "job_link": "https://github.com/huggingface/transformers/actions/runs/14856110212/job/41710279591"
                }
            ]
        }
    }
}
```

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37973). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Small typo lines 47 and 199 perf_infer_gpu_one.md
Author: nlhmnlhmnlhm
Pull request: yes

# What does this PR do?

Fix a small typo.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37938). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix typos
Author: omahs
Pull request: yes

Fix typos

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37978). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[speech2text] fix init of sinusoidal embeddings 
Author: gante
Pull request: yes

# What does this PR do?

Fixes #37874

Problem:
`nn.Parameter()` defined in `__init__` -> we initialize in the `meta` device -> whatever is stored in this parameter at init time gets shredded 

Solution:
Don't init the sinusoidal embeddings in `nn.Parameter`, e.g. use a non-persistent buffer. The buffer solution is used in other models with sinusoidal embeddings, e.g. [here](https://github.com/huggingface/transformers/blob/fa3c3f9cab1c45b449bd57e238c511c79637e314/src/transformers/models/kosmos2/modeling_kosmos2.py#L810)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37931). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix typo
Author: lkm2835
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes minor typo


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37964). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add param_to_hook_all_reduce parameter in HF Trainer
Author: awsankur
Pull request: no
Labels: name: Feature request

### Feature request

PyTorch DistributedDataParallel has `param_to_hook_all_reduce` option which is missing in trainer.py. This is needed to overlap gradient synchronization with backward pass

https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html

### Motivation

This will make Multi node DDP training more efficient

### Your contribution

Maybe

Comments:
"Maybe" 😅

It was funny but had to create this in a hurry.

What do you think of the `param_to_hook_all_reduce` parameter? I see all other parameters PyTorch DDP except this one.

never used it, in fact didn't know about it till now

cc @muellerzr @sunmarc

#37933 please check the implementation here

Happy to add this arg in HF trainer. This requires first a change in accelerate.
Enable granite speech 3.3 tests
Author: alex-jw-brooks
Pull request: yes

Turns on the tests for granite speech now that the 3.3 model is up! Integration tests guard with peft, since they will fail if the lora isn't applied.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@eustlb

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

run-slow: granite_speech

This comment contains run-slow, running the specified jobs: This comment contains run-slow, running the specified jobs:

models: ['models/granite_speech']
quantizations: [] ...

There are 3 tests failed if you could take a look first 

https://github.com/huggingface/transformers/actions/runs/14511526296/job/40711224726

Thanks @ydshieh - hopefully should be fixed now 🤞 


Hi @ydshieh, can you please take another look at this PR when you have a moment? Our team just released a 2b variant of the speech model, so I've updated the tests to use it

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37560). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix donut backtracking
Author: Rocketknight1
Pull request: yes

There's an exploitable regex in the donut processor, this replaces it with less blowup-prone code!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37788). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @Michellehbn! I think this is simple enough and we should be able to merge it without too much trouble