noamaanMulla-03/transformers-issues huggingface dataset:
fix cached file error when repo type is dataset
Author: hiyouga
Pull request: yes

# What does this PR do?

Pass the repo type correctly to `try_to_load_from_cache` in `cached_files`

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@SunMarc 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

Seems okay to me, but in general this code will only be downloading from `model`  type repos anyway!
Fix quantized model initialization for int8 dtypes
Author: Krish0909
Pull request: yes

# Fix quantized model initialization for int8 dtypes

This PR resolves a critical issue where loading quantized models (particularly llmcompressor W8A8 models) fails with:
RuntimeError: expected a floating-point or complex dtype, but got dtype=torch.int8

**Root Cause**: The model initialization code calls `normal_()` on int8 tensors during weight initialization, but PyTorch only supports this operation on floating-point tensors.

**Solution**: Skip weight initialization for quantized models since their weights are already loaded from checkpoints. Changed the conditional in `_load_pretrained_model` from `else:` to `elif not is_quantized:` to prevent calling `initialize_weights()` on quantized models.

**Impact**: 
- Enables loading of llmcompressor quantized models without crashes
- No impact on non-quantized models
- Minimal, targeted fix with backward compatibility

Fixes the issue reported in the original GitHub discussion about RedHatAI/Qwen2.5-VL-7B-Instruct-quantized.w8a8 model loading.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? (No doc changes needed for this internal fix)
- [ ] Did you write any new necessary tests?

## Who can review?

@SunMarc @MekkCyber (quantization experts)

Comments:
@SunMarc Saw this as a good first issue, fixed it. 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: idefics3, smolvlm
Improve grammar and clarity in perf_hardware.md
Author: ridima11
Pull request: yes

This PR improves the readability and corrects grammatical issues in the perf_hardware.md documentation file:

- Fixed subject-verb agreement and pluralization issues.
- Rephrased awkward sentences for better clarity.
- Improved formatting of temperature ranges and technical terminology.
- Simplified the explanation about pigtail cables.

These changes enhance the documentation’s professional tone and make it easier for readers to follow.


Comments:
cc @stevhliu

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39428). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
create ijepa modelcard (ref : PR  #36979 ).
Author: dhruvmalik007
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

#36979 : adds the descriptive model card based on the latest standard.


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu


 

Comments:
> Two more things:
> 
> * add the API references back since they shouldn't be removed (`[[autodoc]]`)
> * remember to add the image back

thanks for reminding the  image.  this was indeed skipped by me . 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39354). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix do_convert_rgb default value
Author: MohitIntel
Pull request: yes

# What does this PR do?

Sets the default value for optional parameter `do_convert_rgb` to True as described in the class definition in Line 81. 
Otherwise, lm-eval on this model results in "ValueError: Unable to infer channel dimension format."

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma3

I submitted new PR https://github.com/huggingface/transformers/pull/39438 for making the suggested change in image_processing_gemma3_fast.py. However, this PR and the new PR alone do not work. In conjunction with these two, https://github.com/huggingface/transformers/pull/39376 is also needed to make lm-eval work on Gemma3.

@MohitIntel can you collect all updates in one PR? Separately it doesn't make sense and it will fail

@zucchini-nlp , closing this PR as all related changes are now in newer PR ready for review : https://github.com/huggingface/transformers/pull/39438
When will transformers 4.51.4 be released?
Author: FightingZhen
Pull request: no

As title, when will transformers 4.51.4 be released? Some new merged features are required for a specific release version :)

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix quantized model dispatch with device_map='auto'
Author: Krish0909
Pull request: yes

# What does this PR do?

Fixes issue #39461 where quantized models fail to load with `device_map="auto"` on Linux but work on Windows.

**Root cause:** `dispatch_model` calls `model.to(device)` on bitsandbytes quantized models, which isn't supported. Platform differences in device detection/handling cause this code path to be hit on Linux but not Windows.

**Fix:** Skip `dispatch_model` for bitsandbytes quantized models since they're already correctly placed during quantization.

Fixes #39461


please verify quantization experts (bitsandbytes, autogpt): @SunMarc @MekkCyber 

Comments:
Add `pruna` integration for loading model through `transformers.from_pretrained` / `pipeline`.
Author: davidberenstein1957
Pull request: no
Labels: name: Feature request

### Feature request

[Pruna](https://github.com/PrunaAI/pruna) is an open source AI model Optimization framework. 
As discussed with @SunMarc, it would be nice to load Pruna optimized models through the `transformers.from_pretrained` interface as an alternative to the [PrunaModel](https://docs.pruna.ai/en/stable/docs_pruna/user_manual/save_load.html#saving-and-loading-models) interface.

Currently, the code looks as follows.
```python
from pruna import PrunaModel

loaded_model = PrunaModel.from_hub("PrunaAI/opt-125m-smashed")
```

We could go for something like. 

```python
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PrunaAI/opt-125m-smashed")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("PrunaAI/opt-125m-smashed")
model = AutoModelForCausalLM.from_pretrained("PrunaAI/opt-125m-smashed")
```

### Motivation

It would be a nice integration.

### Your contribution

We can submit a PR from our side. 


Comments:
Hey, @davidberenstein888, can I work on this issue? 

I think it will be best that the PR comes from their team @Tanuj-rai !

@SunMarc, could you give us some pointers on how to best tackle this? Are there other PRs you know of that we can use as guideline. Do you feel it makes sense to directly add Pruna to the `from_pretrained` flow, or should it be more like an integration where we could start with something like the "Use this model" option https://huggingface.co/Qwen/Qwen3-Embedding-0.6B-GGUF?library=llama-cpp-python?

@SunMarc, reminder

@SunMarc, reminder :) 

Let's maybe focus first on diffusers as i think that it can have the most impact
Better pipeline type hints ✨
Author: qubvel
Pull request: yes
Labels: name: typing

# What does this PR do?

With the power of `@overload` this PR adds several features to make our pipelines more IDE-friendly (and as a bonus more Curosor, Copilot and autocompletion friendly).

### 1. Correctly resolve pipeline `type` in case `task` is specified for pipeline function

![pipe_type](https://github.com/user-attachments/assets/41402181-ae96-4b24-81fc-12abf8b746c7)

As a result, we have **pipeline-specific** docs and method signatures fetched by IDE

<img width="600" alt="Screenshot 2025-05-09 at 17 09 12" src="https://github.com/user-attachments/assets/0e774621-7505-4a52-829f-30fffaac346c" />

### 2. Resolve ambiguities regarding the input and output types for pipelines

For example, 
 - if we provide one image, we will get a dict output 
 - if we provide multiple images we will get list of dict outputs

![output_type](https://github.com/user-attachments/assets/495ebcf4-074e-4d9f-b3d4-b601decb188d)

The part of the code is autogenerated and added to the `pipelines/__init__.py` + added CI check / fix

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38049). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

The code is a bit bloated, but since `pipeline` acts more like an API (and is already somewhat bloated 😅), I think it's worth optimizing the user experience.
@Rocketknight1 @Cyrilvallez curious to hear your thoughts!
Update the auto batch size we expect to see in transformers tests
Author: Rocketknight1
Pull request: yes

https://github.com/huggingface/accelerate/pull/3684 changed accelerate's behaviour to backoff to 90% batch size instead of 50% when a batch is too large for memory. We need to update our tests so they know about that!

Comments:
Closed in favour of #39451

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39475). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix tests due to breaking change in accelerate
Author: SunMarc
Pull request: yes

# What does this PR do?
This PR updates trainer tests due to a breaking change in https://github.com/huggingface/accelerate/pull/3684.

To be merged when accelerate will be released 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39451). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update `GemmaIntegrationTest::test_model_2b_bf16_dola`
Author: ydshieh
Pull request: yes

# What does this PR do?

The value is changed after #39120 but remain reasonable. Not sure the root reason why this might be changed though

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39362). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma
[torch.export] Unhandled FakeTensor Device Propagation for two different devices
Author: Stonepia
Pull request: no
Labels: name: bug

### System Info

transformer version: latest main 3d2c6b90914823c78809874f56e3dfecb1b7105c (06/23)



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Current `torch.export` feature only tests on CPU. When changed the following tests' device from `cpu` to `torch_device`. The error will happen.

https://github.com/huggingface/transformers/blob/2166b6b4ff09f6dd3867ab982f262f66482aa968/tests/utils/test_cache_utils.py#L637

```
Unhandled FakeTensor Device Propagation for aten.embedding.default, found two different devices cuda:0, cpu')
```
### Expected behavior

The error happens similar like below. THe root cause is that there are some hard-coded tensor that is on CPU.

https://github.com/huggingface/transformers/blob/2166b6b4ff09f6dd3867ab982f262f66482aa968/src/transformers/integrations/executorch.py#L356


```
  File "/root/miniforge3/envs/torch_env/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 924, in _find_common_device
    merge_devices(arg)
  File "/root/miniforge3/envs/torch_env/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 919, in merge_devices
    raise RuntimeError(
torch._dynamo.exc.TorchRuntimeError: Dynamo failed to run FX node with fake tensors: call_module L__self___model_model_embed_tokens(*(FakeTensor(..., size=(1, 1), dtype=torch.int64),), **{}): got RuntimeError('Unhandled FakeTensor Device Propagation for aten.embedding.default, found two different devices cuda:0, cpu')


from user code:
   File "/root/miniforge3/envs/torch_env/lib/python3.10/site-packages/transformers-4.48.3-py3.10.egg/transformers/integrations/executorch.py", line 111, in forward
    outs = self.model(
  File "/root/miniforge3/envs/torch_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1766, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniforge3/envs/torch_env/lib/python3.10/site-packages/transformers-4.48.3-py3.10.egg/transformers/models/llama/modeling_llama.py", line 834, in forward
    outputs = self.model(
  File "/root/miniforge3/envs/torch_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1766, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniforge3/envs/torch_env/lib/python3.10/site-packages/transformers-4.48.3-py3.10.egg/transformers/models/llama/modeling_llama.py", line 548, in forward
    inputs_embeds = self.embed_tokens(input_ids)
```

Comments:
cc @gante I think since it's a generation function, but feel free to forward it to someone else! See also the PR at #38976
fix: ImageTextToTextPipeline handles user-defined generation_config
Author: peteryschneider
Pull request: yes

# What does this PR do?

This allows an ImageTextToTextPipeline object to handle a user-defined generation_config passed to the pipeline, the same way a TextGenerationPipeline object does. Currently, it ignores these generation_config parameters set on pipeline object initialization, causing unexpected text generation behavior.

@Rocketknight1

Comments:
Makes sense - cc @zucchini-nlp, I assume there are no issues with passing `generation_config` to VLMs the same way we pass it to text models?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39374). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Enable some ruff checks for performance and readability
Author: cyyever
Pull request: yes

# What does this PR do?
Enables ruff checks and fixes some code. These checks are:
```
PERF102: Checks for uses of dict.items() that discard either the key or the value when iterating over the dictionary.
PLC1802: Checks for len calls on sequences in a boolean test context.
PLC0208: Checks for iteration over a set literal where each element in the set is itself a literal value.
```

 

Comments:
All of the changes look good, but cc core maintainers @arthurzucker @Cyrilvallez since we're adding global ruff rules!

Merging!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39383). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, bridgetower, donut, grounding_dino, llama4, luke, mluke, omdet_turbo, xmod, zoedepth
Use newer typing notation
Author: cyyever
Pull request: yes

# What does this PR do?

Use newer typing for source files with `from __future__ import annotations`.

Comments:
Seems like a cool idea but cc @stevhliu will these types still be rendered correctly for docstrings / docs?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38934). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Checked the rendered docs/docstrings and they look correct! 👍 

@Rocketknight1 this work is also worth paying attention.

@Rocketknight1 Thank you! Also note that the test failures are unrelated.

Yes, they should be fixed by a rebase, sorry about that!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: albert, bart, bert, blenderbot, blenderbot_small, blip, camembert, clip, convbert, convnext, convnextv2, ctrl, cvt, data2vec, deberta, deberta_v2
Use OSError
Author: cyyever
Pull request: yes

# What does this PR do?
Gradually apply Python 3.9 syntax to existing files. This PR changes OSError aliases to OSError.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38712). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
More typing in src/transformers/training_args.py
Author: cyyever
Pull request: yes

# What does this PR do?
 
More typing in `TrainingArguments`

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@cyyever I think some of these errors are actually real! The issue is likely the line `self.distributed_state: Optional["PartialState"] = None`. This creates an attribute that previously did not exist, and this is likely causing other checks to fail.

@Rocketknight1 There are indeed issues in tests that have not been triggered before..

@cyyever I dug into this. The error is real and is caused by the type hint on the `framework` attribute. Giving this a `str` type causes the `AttributeError: 'RegressionTrainingArguments' object has no attribute 'distributed_state'` error, but I have no idea why. I'm guessing something parses `TrainingArguments` and treats typed attributes differently.

cc @sunmarc, do you have any idea why that's happening?

@Rocketknight1 Thank you, let me check `framework` related code..

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38106). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
HfArgumentParser cannot parse `str` for local path
Author: Tavish9
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.52.4
- `transformers` version: 4.53.x
- `transformers` version: 4.54.0.dev0

---
- Platform: Linux-5.10.134-008.16.kangaroo.al8.x86_64-x86_64-with-glibc2.35
- Python version: 3.10.17

### Who can help?

@SunMarc 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

- case1:
  ```python
  from transformers import (
      HfArgumentParser,
      Trainer,
      TrainingArguments,
  )
  
  
  def main():
      parser = HfArgumentParser(TrainingArguments)
      parser.parse_args_into_dataclasses()[0]
      print("parse success")
  
  
  if __name__ == "__main__":
      main()
  ```

  ```bash
  python test.py --deepspeed ./zero1.json
  test.py: error: argument --deepspeed: invalid dict value: './zero1.json'
  ```

- case2:
  ```python
  from transformers import (
      HfArgumentParser,
      TrainingArguments,
  )
  
  
  def main():
      parser = HfArgumentParser(TrainingArguments)
      parser.parse_args_into_dataclasses()[0]
      print("parse success")
  
  
  if __name__ == "__main__":
      main()
  ```

  ```bash
  python test.py --deepspeed ./zero1.json
  parse success
  ```

### Expected behavior

- case1 and case2 should parse local path successfully.

- `transformers<=4.52.4` works well
- `transformers>=4.53.0` fails when importing `Trainer`

Comments:
It seems to be working in `transformers 4.54.0.dev0`

> It seems to be working in `transformers 4.54.0.dev0`

Double check: not working on `transformers>=4.53.0`, containing `transformers 4.54.0.dev0`