noamaanMulla-03/transformers-issues huggingface dataset:
[tests] fix `test_nemotron_8b_generation_sdpa`
Author: faaany
Pull request: yes

## What does this PR do?
Not sure whether the model got updated, but `test_nemotron_8b_generation_sdpa` fails on CUDA. This PR fixes this by adding `max_new_tokens` to make it pass. Otherwise, you will get the following error:

```bash
    def test_nemotron_8b_generation_sdpa(self):
        text = ["What is the largest planet in solar system?"]
        EXPECTED_TEXT = [
            "What is the largest planet in solar system?\nAnswer: Jupiter\n\nWhat is the answer",
        ]
        model_id = "thhaus/nemotron3-8b"
        model = NemotronForCausalLM.from_pretrained(
            model_id, torch_dtype=torch.float16, device_map="auto", attn_implementation="sdpa"
        )
        tokenizer = AutoTokenizer.from_pretrained(model_id)
        inputs = tokenizer(text, return_tensors="pt").to(torch_device)
    
        output = model.generate(**inputs, do_sample=False)
        output_text = tokenizer.batch_decode(output, skip_special_tokens=True)
>       self.assertEqual(EXPECTED_TEXT, output_text)
E       AssertionError: Lists differ: ['Wha[46 chars]er: Jupiter\n\nWhat is the answer'] != ['Wha[46 chars]er: Jupiter\n\nWhat is the answer: What is the name of the 19']
E       
E       First differing element 0:
E       'What[44 chars]wer: Jupiter\n\nWhat is the answer'
E       'What[44 chars]wer: Jupiter\n\nWhat is the answer: What is the name of the 19'
E       
E         ['What is the largest planet in solar system?\n'
E          'Answer: Jupiter\n'
E          '\n'
E       -  'What is the answer']
E       +  'What is the answer: What is the name of the 19']

tests/models/nemotron/test_modeling_nemotron.py:200: AssertionError
```


Comments:
cc @yao-matrix 

cc @ydshieh
Transformer pipelines erroneously invokes torch
Author: rishic3
Pull request: no
Labels: name: bug

### System Info

As originally pointed out by @radoslav-dimitrov-indeavr (see [this comment](https://github.com/huggingface/transformers/pull/37307#issuecomment-2786494331)), PR #37307 adds a torch invocation to pipelines, which will cause pure tensorflow applications to break. 

transformers/src/transformers/pipelines/base.py:

```
        if torch.distributed.is_initialized():
```

[Source](https://github.com/huggingface/transformers/blame/530322ccb6d109ba1383e852b17729a2185d816b/src/transformers/pipelines/base.py#L984)

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Repro: create an environment with transformers and tensorflow but without pytorch. 
```python
classifier = pipeline("sentiment-analysis"
```
Output:
```text
NameError: name 'torch' is not defined
```

### Expected behavior

Given that pipelines are supported for pure Tensorflow applications, said applications should not require a pytorch installation. 

Comments:
Indeed @rishic3 ! Thanks for catching this ! I opened a quick pr to fix it [here](https://github.com/huggingface/transformers/pull/37693)
Make sure torch_is_available before using torch.distributed
Author: MekkCyber
Pull request: yes

# What does this PR do?
Fixes https://github.com/huggingface/transformers/issues/37680


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37693). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Fix] InternVL3 automodel & InternVLProcessor import
Author: quoccuongLE
Pull request: yes

# What does this PR do?

This PR is to fix a model import problem mentioned in #37595 relating InternVL3 models

Fixes # (issue)

* Fixed loading InternVL model via AutoModel by adding ("internvl", "InternVLForConditionalGeneration") in MODEL_MAPPING_NAMES
* Fixed loading InternVLProcessor error while attributes "start_image_token", "end_image_token" and "context_image_token" missing from tokenizer.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Especically, I encourage anyone in HuggingFace team, InternVL team to review my PR.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @yonigozlan for review

@zucchini-nlp Thanks for your comment. I just saw your PR (https://github.com/huggingface/transformers/pull/37656) to update InternVLProcessor and since after removing  ("internvl", "InternVLForConditionalGeneration") from MODEL_MAPPING_NAMES. Then this PR becomes mainly to fix InternVLProcessor. It would be obsolete, in my opinion. If you think that so, feel free to close this PR :)

This PR is obsolete. So I close it !
[qwen-omni] fix training
Author: zucchini-nlp
Pull request: yes
Labels: name: for patch

# What does this PR do?

Fixes #37513 and fixes #37515. Ready for review!

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37517). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Should be ready for review now, doesn't seem we have any other issues reported with qwen training
Correctly raise errors when downloading tokenizer files
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. If an error arises in the middle of downloading tokenizer files, it was not properly raised and would propagate `None` values instead (because the check was performed on all files, i.e. aither they all pass or they all miss, but we may fail in the middle due to e.g. timeout)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37740). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update `MllamaForConditionalGenerationIntegrationTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

Followup of #37644

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37750). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[cache] fix `HybridCache` init when `device` is passed
Author: gante
Pull request: yes

# What does this PR do?

The following example, where we initialize the model with the `device` argument, results in cuda graph skips. It can be traced to a stray change in https://github.com/huggingface/transformers/pull/37007, where the device initialization was incorrectly changed. This PR reverts it.

```py
from transformers import pipeline
import torch

pipe = pipeline(
    "image-text-to-text",
    model="google/gemma-3-4b-it",
    device="cuda",
    torch_dtype=torch.bfloat16
)

output = pipe(text="Write a poem on Hugging Face, the company", max_new_tokens=10)
print(output)
```

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37718). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Bye bye env vars, keep everything as configs
Author: zach-huggingface
Pull request: yes

# What does this PR do?

As @BenjaminBossan found, (https://github.com/huggingface/accelerate/pull/3252), the `TrainingArguments` will set environmental variables automatically when using Accelerate because before it wouldn't work otherwise. Nowadays the only env variable required for things to run smoothly is the ones for model init (fsdp cpu eff ram). 

This PR does a few things:

1. We completely remove the need for environmental variables, creating the proper configs (dynamo relies on https://github.com/huggingface/accelerate/pull/3251)
2. I've refactored how `mixed_precision` gets set, to simplify the training arguments and combine 7 args into 2. 
3. Removes old references/updates the logic in `Trainer` to reflect the choices

Full version of https://github.com/huggingface/transformers/pull/34886

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@SunMarc @Rocketknight1 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37259). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Quantized int8 model evaluation using TP - only Tensors of floating point dtype can require gradients
Author: amd-xiaoyu12
Pull request: no
Labels: name: bug

### System Info

When I test quantized int8 model with TP,  the following error occurred: only Tensors of floating point dtype can require gradients

Detail:
[rank0]: File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/tensor_parallel.py", line 658, in shard_and_distribute_module
[rank0]: param = tp_layer.partition_tensor(
[rank0]: File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/tensor_parallel.py", line 385, in partition_tensor
[rank0]: return nn.Parameter(parameter) #, requires_grad=False)
[rank0]: File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/parameter.py", line 49, in new
[rank0]: t = data.detach().requires_grad_(requires_grad)
[rank0]: RuntimeError: only Tensors of floating point dtype can require gradient

Solution can be found at draft PR #37719 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Load quantized int8 model "amd/Llama-3.1-8B-Instruct-w-int8-a-int8-sym-test" and evaluate the model using tensor parallelism.

### Expected behavior

The model should run successfully using tensor parallelism. 

Comments:
Closing because #37719 has been merged!
[generate] skip compilation on cpu offload
Author: gante
Pull request: yes

# What does this PR do?

`torch.compile` + model CPU offload is resulting in crashes. It should work in theory, but it's not working atm.

```py
from transformers import AutoModelForCausalLM, AutoTokenizer

device_map = {"model.embed_tokens": 0, "model.layers.0": 0, "model.layers.1": "cpu", "model.norm": "cpu", "lm_head": 0}
model = AutoModelForCausalLM.from_pretrained(
    "hf-internal-testing/tiny-random-MistralForCausalLM", device_map=device_map
)
tokenizer = AutoTokenizer.from_pretrained("hf-internal-testing/tiny-random-MistralForCausalLM")
tokenized_inputs = tokenizer(["Hello world"], return_tensors="pt")
input_ids = tokenized_inputs.input_ids.to(0)

# Uses a compilable cache -> compilation happens under the hood
output = model.generate(input_ids, max_new_tokens=20, cache_implementation="static")
```

This PR:
1. Moves the logic to trigger "auto compile" into its own function
2. Disables "auto compile" when there is CPU offload (and disk offload too, which is not expected to support torch.compile)
3. Adds a test to prevent regressions

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37709). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> One question, do we expose "auto-compile" in generate to users through config or is this still under the hood?

There is `generation_config.compile_config`. Yes, agreed, we should throw a warning if it is set but we don't meet the conditions for compilation to happen -- adding a commit with it 👍
[Bug]: LLAMA_ATTENTION_CLASSES no longer found in newer versions, error when deploying models (Aria)
Author: musoles
Pull request: no
Labels: name: bug

### System Info

Environment install:

pip install vllm==0.8.4

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

This is related to a [vLLM issue I raised](https://github.com/vllm-project/vllm/issues/17077) when trying to deploy [Aria model](https://huggingface.co/rhymes-ai/Aria). 

The error I'm getting is related to a missing variable:

`LLAMA_ATTENTION_CLASSES' from 'transformers.models.llama.modeling_llama`

Upon initial investigation I noticed `LLAMA_ATTENTION_CLASSES` was present in `transformers<=4.47.1` but disappeared after. Since vLLM requires transformers >=4.51, this causes an issue.

I'm uncertain if this is a vLLM or a transformers issue, hence raising it in both.

The command to reproduce:
```

python -m vllm.entrypoints.openai.api_server \
  --model rhymes-ai/Aria-Chat \
  --host 0.0.0.0 --port 8080 \
  --tensor-parallel-size 1 \
  --pipeline-parallel-size 1
```

### Expected behavior

Successfully deployed Aria model.

Comments:
Closing as resolved, issue in remote code. For a solution, you can use Aria from core library, as commented under vllm issue  :)
updated hidden_features for FlaxDinov2SwiGLUFFN in Dinov2 
Author: premmurugan229
Pull request: yes

# What does this PR do?

Fixes #37745 

## Changes made

[FlaxDinov2SwiGLUFFN](https://github.com/huggingface/transformers/blob/b6d65e40b256d98d9621707762b94bc8ad83b7a7/src/transformers/models/dinov2/modeling_flax_dinov2.py#L400) has only 2 attributes (config,dtype).  It doesn't have hidden_features attribute. So, the previously calculated hidden_feature value is used. This change removes the error faced in the above linked issue

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@gante  @Rocketknight1 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37747). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
depreciating all occurances of clean_up_tokenization_spaces
Author: itazap
Pull request: yes

fixes #31187 
depreciating all occurances of clean_up_tokenization_spaces


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31232). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@itazap Any update on this PR?

@seastar105 yes, we began the deprecation with setting `clean_up_tokenization_space=False` by default (https://github.com/huggingface/transformers/pull/31938). For now, there is still a significant amount of older models that depend on this `clean_up_tokenization_space=True`, but by default, your model won't be using it anymore 😊
fix for loading gguf quantized model
Author: itazap
Pull request: yes

- "resolved_archive_file" --> "shard_file
- load_state_dict should not be called with gguf file
- gguf should use param = empty_param

snippet to reproduce error:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoModel

model_name = "Qwen/Qwen2.5-0.5B-Instruct-GGUF"
q4_file = "qwen2.5-0.5b-instruct-q4_k_m.gguf"

tokenizer = AutoTokenizer.from_pretrained(model_name, gguf_file=q4_file)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    gguf_file=q4_file,
    torch_dtype="auto",
    device_map="auto"
)
```

TODO: Is it alright to add a gguf model on the internal testing hub for testing?

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36563). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[generate] fix default autocompile case on gpu
Author: gante
Pull request: yes

# What does this PR do?

(I didn't run slow tests before merging #37709 , the CPU tests of the push ci didn't caught this issue)

reproducer, requires GPU
```py
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct", device_map="auto")
inputs = tokenizer(["The quick brown"], return_tensors="pt").to(model.device)

# compilable cache -> compilation will happen
gen_out = model.generate(**inputs, do_sample=False, cache_implementation="static")
print(tokenizer.batch_decode(gen_out, skip_special_tokens=True))
```


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37756). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.