noamaanMulla-03/transformers-issues huggingface dataset:
Skip all `AriaForConditionalGenerationIntegrationTest` on `T4`
Author: ydshieh
Pull request: yes

# What does this PR do?

Follow up of [this commment](https://github.com/huggingface/transformers/pull/37444#issuecomment-2801489013)

Skip all `AriaForConditionalGenerationIntegrationTest` except `test_tokenizer_integration` on `T4` runner.

These tests uses [this repository](https://huggingface.co/rhymes-ai/Aria/tree/main) that doesn't fit into T4 or even A10, even using  `load_in_4bit=True`.

(The `require_torch_large_accelerator` or `require_torch_large_gpu` is not well-written to specify the min. memory requirement despite it has an argument `memory`.)
 

 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37746). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

if fails on A10, I tried.

> create a smaller version as in MixtralTest

maybe, but the issue is that the new repository (used for tests) may not sync with the original repository's changes if any happens there.

>  cpu offloading 

Maybe yes


I ping @aymeric-roucher on Slack

Unfortunately, on T4, with

```
+        model = AriaForConditionalGeneration.from_pretrained("rhymes-ai/Aria")
+        import accelerate
+        accelerate.cpu_offload(model, execution_device=torch_device) 

```
the process gets killed

> tests/models/aria/test_modeling_aria.py::AriaForConditionalGenerationIntegrationTest::test_small_model_integration_test Killed
Fix qwen2-vl-docs.
Author: zhanluxianshen
Pull request: yes


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37879). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks review.
update Clean_up_tokenization_spaces typos.
Author: zhanluxianshen
Pull request: yes


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37865). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks.
Fix Qwen3 tp plan with FP8
Author: MekkCyber
Pull request: yes

# What does this PR do?
This PR : 
- Enables the device context for the FP8 activation kernel (if disabled, it causes a triton memory error)
- Updates the tp plan for the qwen3 fp8 models (since they use finegrained fp8, the scales will follow the sharding strategy of the weights)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37871). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Support `AOPerModuleConfig` and `include_embedding`
Author: jerryzh168
Pull request: yes

Summary:
This PR adds support per module configuration for torchao Also added per module quantization examples:

1. Quantizing different layers with different quantization configs
2. Skip quantization for certain layers

Test Plan:
python tests/quantization/torchao_integration/test_torchao.py -k test_include_embedding python tests/quantization/torchao_integration/test_torchao.py -k test_per_module_config_skip

Reviewers:

Subscribers:

Tasks:

Tags:



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @MekkCyber @SunMarc 

for formatting, do you want to setup pre-commit like torchao did? https://github.com/pytorch/ao/blob/main/.pre-commit-config.yaml

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37802). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
🚨 rm already deprecated padding max length
Author: itazap
Pull request: yes

this seems already deprecated since 2022 and no instances of its use throughout! everything was converted from using ` pad_to_max_length=True` --> `padding=True, max_length=n` 

there are plenty of other tests testing the `padding` parameters including `padding=True, max_length=n`, so safe to remove `def test_padding_to_max_length`.

need to remove all the overridden tests too, but want to confirm in review first

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37617). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
How to backpropagate the gradients of the embeddings output by the image processor to the input image tensor?
Author: weiminbai
Pull request: no
Labels: name: Feature request

### Feature request

I'm using the processor of Qwen2.5-VL, and the image processor within it should be Qwen2ImageProcessor. The input image I provide is a PyTorch tensor with gradients, and the processor outputs the feature embeddings of the image. How can I ensure that the gradient flow is not interrupted during this process?

### Motivation

I want to backpropagate the gradients of the embeddings output by the Qwen2 image processor to the input image tensor

### Your contribution

I can coporate to fix this issue

Comments:
Hi @weiminbai, ImageProcessor classes generally do not propagate gradients! They often use operations that are not differentiable, and inputs are usually `uint8` images. Integer dtypes cannot hold gradient because they are discrete rather than continuous values.
Fix: Optimize safetensors load by moving dtype check for meta device
Author: demoncoder-crypto
Pull request: yes

Addresses #37887, This is first implementation if the said path is correct I will iteratively fix the changes requested

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Closing as a duplicate of #37902!
[tests] fix `test_cache_copy`
Author: gante
Pull request: yes

# What does this PR do?

[Failure reported recently](https://huggingface.co/datasets/hf-internal-testing/transformers_daily_ci/raw/dbea6f08c0ef4104bcdb1e405085f40354a006c1/2025-05-01/ci_results_run_models_gpu/new_model_failures_with_bad_commit_grouped_by_authors.json).

👀 `test_cache_copy` has a different generation output depending on whether we run it with `RUN_SLOW=1 py.test tests/utils/test_cache_utils.py` (run all cache tests) or `RUN_SLOW=1 py.test tests/utils/test_cache_utils.py -k test_cache_copy` (run the test in isolation).

This means there is some stateful effect changing the generation output. I'll keep in mind for future occurrences but, since the purpose of this test is not the generation output (but rather the correctness of `copy.deepcopy(prompt_cache)`), the test is modified to output fewer tokens, which no longer triggers the issue.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37913). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Found the source of randomness, opening another PR 👼
[chat] clean code and add base help
Author: gante
Pull request: yes

# What does this PR do?

Cleans `chat.py` before start adding features. There are no functional changes, except for this short help message that is now printed at the start of the chat session.

![Screenshot 2025-04-30 at 14 13 03](https://github.com/user-attachments/assets/e8b210b1-24a9-47d0-92a8-12e45f5720d9)

Changes:
1. Adds a basic help message at the start of the chat session (previously, there was no information on e.g. how to exit the chat session)
2. Adds type hints and docstrings
3. Moves functions into class methods (these are chat-specific functions)
4. 120 char/line limit
5. Other related minor changes

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37892). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
🌐 [i18n-KO] Translated `gpu_selection.md` to Korean
Author: iambogeumkim
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `gpu_selection.md` file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 본인의 조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요!-->
May you please review this PR?
<!-- @cjfghk5697, @yijun-lee, @rlaalsrl0922 , @MinJu-Ha -->
@cjfghk5697, @yijun-lee, @devxaitist
<!-- @cjfghk5697, @yijun-lee, @Kim-Ju-won @junhkang @olccihyeon -->
<!-- @cjfghk5697, @yijun-lee, @garongkim @maximizemaxwell -->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)
<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
<!-- @stevhliu May you please review this PR? -->

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

@olccihyeon

Thank you for your review, @shethaadit ! I changed some files based on your comment.

Thanks for changing, the doc-build is failing because this [comment](https://github.com/huggingface/transformers/pull/36757#discussion_r1999264634) hasn't been addressed yet :)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36757). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update fastspeech2 model card
Author: ricalanis
Pull request: yes

#36979 

* Updated the FastSpeech2Conformer model card
* Could not replicate the pipeline example from original code, as a bug related to FastSpeech2ConformerConfig appears (AttributeError: 'FastSpeech2ConformerConfig' object has no attribute 'model_config') still left it as is.
* Did not add terminal call out as I could not find a direct way to do that.
* Added a small section to use the combined model with the vocoder.
* Did not add quantization as the model architecture is not linear layer heavy (limited impact of quantization, AFAIK)
* This was a little bit more challenging, still. really thank you for your patience!

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [] Did you write any new necessary tests? NA


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @stevhliu
Malformed config when saving & loading locally custom models
Author: Alicimo
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.47.1
- Platform: Linux-6.8.0-51-generic-x86_64-with-glibc2.39
- Python version: 3.11.10
- Huggingface_hub version: 0.27.0
- Safetensors version: 0.4.5
- Accelerate version: 1.2.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?:  NA
- Using GPU in script?: Yes, but also NA
- GPU type: NVIDIA GeForce RTX 4090

### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoModel, AutoTokenizer, AutoConfig

# Load a custom model from the Hub
model = AutoModel.from_pretrained("jinaai/jina-embeddings-v2-base-de", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("jinaai/jina-embeddings-v2-base-de")

# Save them locally
model.save_pretrained("tmp")
tokenizer.save_pretrained("tmp")

# Now load that model again, via a Config
config = AutoConfig.from_pretrained("tmp", trust_remote_code=True)
print(config.__class__)
# <class 'transformers_modules.jinaai.jina-bert-implementation.f3ec4cf7de7e561007f27c9efc7148b0bd713f81.configuration_bert.JinaBertConfig'>
local_model = AutoModel.from_pretrained("tmp", config=config, trust_remote_code=True)

# And save it again
local_model.save_pretrained("tmp_2")
tokenizer.save_pretrained("tmp_2")

# Now load that model again, via a Config
config_2 = AutoConfig.from_pretrained("tmp_2", trust_remote_code=True)
print(config_2.__class__)
# <class 'transformers_modules.tmp_2.configuration_bert.JinaBertConfig'>
# The second time around, the config is not the same as the first time
local_model_2 = AutoModel.from_pretrained("tmp_2", config=config_2, trust_remote_code=True)
# ValueError: The model class you are passing has a `config_class` attribute that is not consistent
# with the config class you passed (model has
# <class 'transformers_modules.jinaai.jina-bert-implementation.f3ec4cf7de7e561007f27c9efc7148b0bd713f81.configuration_bert.JinaBertConfig'>
# and you passed <class 'transformers_modules.tmp_2.configuration_bert.JinaBertConfig'>. Fix one of those so they match!
```

### Expected behavior

Model should load without raising a `ValueError`

Comments:
This is related to my PR #29854, and it's definitely a bug, yes. Let me see if I can loosen up that test a little.

@Alicimo this should now be fixed, please let me know if you're still getting issues after installing the latest version from `main`!
Aligning modling code for GPT2 to work with vLLM (fallback)
Author: ariG23498
Pull request: yes

This PR changes the modeling code for GPT2 to support working on vLLM using the `transformers fallback backend`.

The changes are as follows:
- Introduction of `kwargs` which is used to propagate information about `attention_indices` in vLLM
- Adding a `base_model_tp_plan`, which is currently empty. This is a required attribute for vLLM
- Changing the reshape structure for the `attn_outputs` (took help from the llama code)

This PR is dependent on https://github.com/vllm-project/vllm/pull/15290 on the vLLM side to work.

One can use the following snippet to check the implementation:

```py
from vllm import LLM, SamplingParams
llm = LLM("openai-community/gpt2", model_impl="transformers", tensor_parallel_size=1)
prompts = [
    "Hello, my name is",
    "The president of the United States is",
    "The capital of France is",
    "The future of AI is",
]
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
outputs = llm.generate(prompts, sampling_params)
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
```

```py
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")

inputs = tok("Hello there ", return_tensors="pt")
outputs = model.generate(**inputs)
print(tok.batch_decode(outputs))
```

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36934). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Just make sure CI is green!

@ArthurZucker I don't think the CI errors stem from the modeling changes. Do you want me to investigate further?

> Adding a base_model_tp_plan, which is currently empty. This is a required attribute for vLLM

I could make this optional so that a model which does not have it simply doesn't support TP, rather than not working at all?

@ArthurZucker would it be okay to merge? The CI is broken for issues not related to the PR it seems 😅 

Its missing some documentation maybe to linking to this exact PR ! 🤗 

@ArthurZucker I made a very small comment in the docs, was this something similar to what you had in mind? I can make it verbose if you would like.

Thanks 🤗
Gemma3 not supported in main branch
Author: xihuai18
Pull request: no
Labels: name: bug

### System Info

latest main branch of 0325

### Who can help?

@amyeroberts, @qubvel

Gemma3Config is not supported in the lastest main branch.

ValueError: Unrecognized configuration class <class 'transformers.models.gemma3.configuration_gemma3.Gemma3Config'> for this kind of AutoModel: AutoModel.

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

run gemma3 using `AutoModel.from_config`

### Expected behavior

No.

Comments:
@xihuai18 hey! We don't have a base `Gemma3Model` class yet, I am adding base classes for all vision LLMs currently in https://github.com/huggingface/transformers/pull/36367. If you want to get last hidden state from the model, you can do as follows:

```python
model = Gemma3ForConditionalGeneration.from_pretrained(model_id)
last_hidden_states = model(input_ids, pixel_values=pixel_values).hidden_states[-1]
```


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.