noamaanMulla-03/transformers-issues huggingface dataset:
Handle torch ver in flexattn
Author: Kh4L
Pull request: yes

Follow up #37399

@ArthurZucker 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Explain tp_plan for FBgemm
Author: MekkCyber
Pull request: yes

# What does this PR do?
Explains the tp_plan for fbgemm implementation


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37404). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix Llama4 offset
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

See title. It's the same as before + 1. Indeed, we need to add the +1 to match correctly the positions, it's also what is done in gemma2 here https://github.com/huggingface/transformers/blob/main/src/transformers/models/gemma2/modeling_gemma2.py#L912-L913 as it uses the length (which is equal to cache_position[-1] + 1).
I used `first_cache_position - attention_chunk_size + 1` here as I feel it simpler to understand when looking at the cache code (see the 1-offset here https://github.com/huggingface/transformers/blob/main/src/transformers/cache_utils.py#L1921-L1922), but it is striclty equal. I.e. we always have `max(first_cache_position - attention_chunk_size + 1, 0) == max(last_cache_position + 1 - key_length, 0)`

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37414). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Offloaded hybrid cache for Llama4
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37401). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
update `kernels` to 0.4.3
Author: ArthurZucker
Pull request: yes

# What does this PR do?
This makes sure we can disable the kernels if needed!

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37419). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Send trainer/fsdp/deepspeed CI job reports to a single channel
Author: ydshieh
Pull request: yes

# What does this PR do?

As discussed offline with @SunMarc 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37411). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Allow rocm systems to run these tests
Author: ivarflakstad
Pull request: yes

# What does this PR do?

Some tests were skipped on AMD gpus because they checked `torch.version.cuda` only.
Now we use `IS_CUDA_SYSTEM or IS_ROCM_SYSTEM`

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37278). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
use `rms_norm_eps` for the L2Norm for Llama4
Author: ArthurZucker
Pull request: yes
Labels: name: for patch

# What does this PR do?

It seems like this changes throughout the port, updating and will test!

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37418). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[bug] fsdp training save checkpoint twice
Author: cyr0930
Pull request: yes

# What does this PR do?
FSDP training save checkpoint twice as pytorch_model_fsdp.bin and model.safetensors.
And saving model during calling _save_optimizer_and_scheduler is not intuitive.

(Maybe I have to rename final checkpoint or save it as safetensors format.)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@zach-huggingface @SunMarc

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
[chat-template] Unify tests and clean up 🧼 
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, we added the video loading logic in rush for SmolVLM. But the API is a bit over-complicated, so the decision was to clean up later. As the first step of clean up, we will

- Remove `sampling_fn` from public kwargs, I don't think anyone uses it so we will not deprecate
- Squash typed dict manipulations into a smaller helper
- Put video loading for each model in model's processor. Until the PR for adding separate API for video processors is merged, we'll keep it private under processor. In the future, each video processor can have its own logic to load videos, following model-specific treatment
- Clean up tests to load only slow image/video processors. As we are moving to fast-only image processors, we need a way to run tests without relying only on torch. For torch inputs we already have `test_chat_template_return_dict_torch`
- Unify similar test under one and parameterize




Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37275). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

ready for review :)
Fix new failure reports not including anything other than `tests/models/`
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix new failure reports not including anything other than `tests/models/`

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@Copilot Shut up please?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37415). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Quark Quantization gated repo
Author: MekkCyber
Pull request: yes

# What does this PR do?
Fix the use of gated repo in quark quantization


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37412). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

ah, it's meta-llama/Llama-3.1-8B-Instruct ... I thought it was a new model ... we already have the access. 

any idea what's causing this error @ydshieh : 
`ERROR tests/quantization/quark_integration/test_quark.py - TypeError: function() argument 'code' must be code, not str`

`@require_read_token` is not the same as other `@require_xxx`. It is applied to function/method, while @require_xxx return test cases when it is applied to test class / test method.

I don't know why the error occurs when `require_read_token` is applied to both `QuarkTestDeviceMap and QuarkTest`, but currently this is not the correct usage of `require_read_token` for class. But you can add `require_read_token` to `setUpClass` maybe.

Considering these are slow tests, better to manually verify the changes work before merge.
(If the tests can run but just fails for some other reasons, that could be fixed in separate PRs) 

It's working right now after applying`require_read_token` to the `setUpClass` thanks a lot 🤗
Add image classifier donut & update loss calculation for all swins 
Author: eljandoubi
Pull request: yes

# What does this PR do?

- Add classifier head to Donut  
- Add image classifier loss to `LOSS_MAPPING` 
- Update classifier loss for all swin models

Models:

- vision models: @amyeroberts, @qubvel


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@zucchini-nlp @ArthurZucker @ydshieh  any feedback? 

sorry, let's merge it. Doesn't touch much core code, should be good to go
Missing SpeechT5ForVoiceConversion class - ImportError after installation
Author: ChanduBora
Pull request: no
Labels: name: Feature request

### Feature request

from transformers import SpeechT5Processor, SpeechT5ForVoiceConversion

ImportError: cannot import name 'SpeechT5ForVoiceConversion' from 'transformers'

### Motivation

pip show transformers
Name: transformers
Version: 4.51.1
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /home/cbora/Gemini_Field_310_env/lib/python3.10/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by: 

pip install --upgrade transformers (and the resulting version, e.g., 4.51.1)
pip install --force-reinstall transformers==4.35.0
pip install --force-reinstall --pre transformers (and the resulting pre-release version, if you noted it)
Installation from source (pip install . and the resulting dev version, e.g., 4.52.0.dev0)

### Your contribution

pip install --upgrade transformers (and the resulting version, e.g., 4.51.1)
pip install --force-reinstall transformers==4.35.0
pip install --force-reinstall --pre transformers (and the resulting pre-release version, if you noted it)
Installation from source (pip install . and the resulting dev version, e.g., 4.52.0.dev0)

Upgrading/reinstalling transformers (various versions).
Installing from source.
Inspecting __init__.py files (and the fact that grep found no definition).
Checking sys.path and PYTHONPATH.

Comments:
Transformers has never had a class called `SpeechT5ForVoiceConversion`. Possibly this comes from another library or a custom repo?
Correctly drop tokens in SwitchTransformer
Author: mario-aws
Pull request: yes

# What does this PR do?

Previously, the identity function was used for dropped tokens with a weight from the expert that was not applied to the hidden states. This was misleading, because dropping means, the expert weight is zero. Instead of trying to fix the weight, we take an easier approach by initializing with zeros.

Fixes #37017 

## Related work

https://github.com/tensorflow/mesh/blob/e6798a2610a2c2f4c4cd236d8214422cb1ecc00a/mesh_tensorflow/transformer/moe.py#L1144 mentions that it needs to be zeroed out.

https://github.com/tensorflow/mesh/blob/master/mesh_tensorflow/transformer/moe.py#L507C18-L507C31 combines the results without any clone initialization beforehand.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

text models: @ArthurZucker
last major changing person: @zucchini-nlp
person who requested PR: @Rocketknight1 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37123). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Would love to be part of this project one day.
I'm not a coder, but I do have a vision... :)

@Rocketknight1 

LGTM but I'm not an expert on MoE routing! @zucchini-nlp @arthurzucker if you're happy with it feel free to merge

run-slow: switch_transformers

This comment contains run-slow, running the specified jobs: This comment contains run-slow, running the specified jobs:

models: ['models/switch_transformers']
quantizations: [] ...

run-slow: switch_transformers

This comment contains run-slow, running the specified jobs: This comment contains run-slow, running the specified jobs:

models: ['models/switch_transformers']
quantizations: [] ...

[pytest/custom-tests](https://github.com/huggingface/transformers/actions/runs/14382730688) — Slow CI job

It's green, so might be ready to go, @zucchini-nlp ?

Cool thanks! Merging
Fix require_read_token
Author: MekkCyber
Pull request: yes

# What does this PR do?

Add require_read_token to functions instead of classes

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37422). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Adding to self_comment_ci.yml
Author: MekkCyber
Pull request: yes

# What does this PR do?
As stated above


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37426). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Apply torchfix to replace deprecated functions: `_pytree._register_pytree_node` and `torch.cpu.amp.autocast`
Author: bzhong-solink
Pull request: yes

- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@Rocketknight1 I tried before, but torch.amp is a new feature that is unavailable on PT 2.1

@cyyever Ack, you're right, and that potentially applies to the other fix as well. Can we keep the `pytree` change or will that break torch 2.1 as well?
allow accelerate prepare for torch compiled models
Author: winglian
Pull request: yes

# What does this PR do?

FSDP2 does't work with torch compiled models since it has an optimized wrap around it. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@S1ro1 
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.