noamaanMulla-03/transformers-issues huggingface dataset:
Fix `init empty weights` without accelerate
Author: Cyrilvallez
Pull request: yes
Labels: name: for patch

# What does this PR do?

As per the title. People started reporting the issue as they sometimes uses Transformers without Accelerate

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37337). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Use Python 3.9 syntax in examples
Author: cyyever
Pull request: yes

# What does this PR do?
 
A follow-up work to fix examples.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
ImportError: cannot import name 'HfEngine' from 'transformers.agents'
Author: HariNuve
Pull request: no

Tried out the follwing :

```
from transformers.agents import HfEngine, ReactJsonAgent

llm_engine = HfEngine("meta-llama/Meta-Llama-3-8B-Instruct")

agent = ReactJsonAgent(
    tools=[RetrieverTool(vectordb, all_sources)],
    llm_engine=llm_engine
)

agent_output = agent.run("Please show me a LORA finetuning script")

print("Final output:")
print(agent_output)

```

 But resulted  with following error : 
```
ImportError                               Traceback (most recent call last)
Cell In[11], line 1
----> 1 from transformers.agents import HfEngine, ReactJsonAgent
      3 llm_engine = HfEngine("meta-llama/Meta-Llama-3-8B-Instruct")
      5 agent = ReactJsonAgent(
      6     tools=[RetrieverTool(vectordb, all_sources)],
      7     llm_engine=llm_engine
      8 )

ImportError: cannot import name 'HfEngine' from 'transformers.agents' (/home/paperspace/anaconda3/envs/doc_env/lib/python3.10/site-packages/transformers/agents/__init__.py)
```


I have followed :[Transformers Agents 2.0]( https://huggingface.co/blog/agents#what-is-an-agent)

Comments:
We deprecated agents in tranformers! You should use `small-agents` library! https://github.com/huggingface/smolagents 

@ArthurZucker Thank You
Latest TorchAO config breaks serialization
Author: airMeng
Pull request: no

PR #36526 breaks https://github.com/huggingface/transformers/blob/49b5ab6a27511de5168c72e83318164f1b4adc43/tests/quantization/torchao_integration/test_torchao.py#L345


_Originally posted by @airMeng in https://github.com/huggingface/transformers/pull/36526#discussion_r2015946292_


Transformers version 49b5ab6a27511de5168c72e83318164f1b4adc43
TorchAO version ab3792e3d91e04f85992a659c1664a6a1a6d733c


Reproduce scripts:
```python
import torch
from transformers import TorchAoConfig, AutoModelForCausalLM
from torchao.dtypes import Int4CPULayout

model_name = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"
quantization_config = TorchAoConfig("int4_weight_only", group_size=32, layout=Int4CPULayout())
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="cpu", quantization_config=quantization_config)
model.save_pretrained("./qwen-ao-int4", safe_serialization=False)
q_model = AutoModelForCausalLM.from_pretrained("./qwen-ao-int4", torch_dtype=torch.bfloat16, device_map="cpu")

```

@SunMarc @drisspg 


```shell
Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.51.0.dev0
- Platform: Linux-5.15.0-73-generic-x86_64-with-glibc2.35
- Python version: 3.10.15
- Huggingface_hub version: 0.29.1
- Safetensors version: 0.5.2
- Accelerate version: 1.4.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.7.0a0+git924a247 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: NO
- Using XPU in script?: NO
- XPU type: Intel(R) Data Center GPU Max 1100
```

Comments:
Hi @SunMarc . I think we got the wrong test, which prevented us from detecting this bug. I have fixed the test in #37034 . Now we can see the save and from pre-trained bug from the new test. 

Can you have a look at this bug @jiqing-feng ? Otherwise, I will try to find some time later ! 

Yes, please assign to me to solve this.

Fixed in #37034
byebye torch 2.0
Author: ydshieh
Pull request: yes

# What does this PR do?

For #37137

Fixes #37238

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@SunMarc  @muellerzr  You may want to double check the changes in trainer / fsdp / training args / quantizer

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37277). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

not urgent, but @Rocketknight1 this PR is ready. The failing tests are irrelevant.

Going to merge, the failing tests

> tests/models/llama4/test_processor_llama4.py

are irrelevant to this PR and known already failing on main.
transformers 4.50 does not work with pytorch 2.0
Author: koen-dejonghe
Pull request: no
Labels: name: bug

### System Info

```
pip install torch==2.0.1  transformers==4.50.3
python <<-EOF
from transformers.integrations import CodeCarbonCallback
EOF
```

results in 

```
RuntimeError: Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
Failed to import transformers.modeling_utils because of the following error (look up to see its traceback):
module 'torch' has no attribute 'compiler'
```

I cannot run `transformers-cli env` for the same reason.

transformers 4.49 does not have this issue.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
pip install torch==2.0.1  transformers==4.50.3
python <<-EOF
from transformers.integrations import CodeCarbonCallback
EOF
```

### Expected behavior

it should not give an error.

transformers 4.49 does not have this issue.

Comments:
Is there a specific reason you are using `torch==2.0.1`?

no.
it's just an observation of a bug.
feel free to do with it as you please.

as a matter of fact, yes, there is a specific reason, but I do not wish to go into the details, as that is beside the point.

Your problem lies in `flex_attention.py` in the `tranformers` library in the following line:
```py
@torch.compiler.disable(recursive=False)
```
which was added in `transformers 4.50` for optimizing the performance.

Fix:

For your use-case, you can just comment out the decorator - Your code will then be compiled with TorchDynamo (Some performance degradation is expected).

**Note - This can lead to graph breaks so I would advise you to keep using 4.49 as long as you are using 2.0.1 version of PyTorch.**

duplicate of #37137 

Yes - the latest versions of Transformers do not support versions of Torch lower than 2.1. We didn't update our requirements correctly, but we're fixing that in #37137
Fix deepspeed with quantization
Author: Cyrilvallez
Pull request: yes
Labels: name: for patch

# What does this PR do?

Alright, this should fix all deepspeed issues (finally). See here https://github.com/huggingface/transformers/pull/36963/files#diff-6b72b98c4c2dcfc6cc606843917733f5d858374fbc22a735ff483bbc0c1e63eaL4248-L4252, `low_cpu_mem_usage` was forbidden with deepspeed, but then still overriden here in the quantized case https://github.com/huggingface/transformers/pull/36963/files#diff-6b72b98c4c2dcfc6cc606843917733f5d858374fbc22a735ff483bbc0c1e63eaL4358-L4361

cc @stas00, @winglian 

@stas00, next steps will be making sure tests are robust, and once this is done we'll try to see how we can truly harmonize the paths, so that everything is taking the same one! But this will be done after the tests, to avoid potential similar issues as what happened these past days 🙃

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37324). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Cyrilvallez, I validated that this PR still works for our use case.

Let's wait for @winglian to confirm that this PR unbreaks their use-case.

Also I'd insist on adding the missing test https://github.com/huggingface/transformers/pull/37315#issuecomment-2781196713 since clearly @winglian's use case isn't covered by deepspeed tests under `tests/deepspeed`



All good on my end ✅ , thanks! we'll put together a follow up PR to add a test for the other deepspeed test cases.

FYI: if necessary, feel free to use the comment

> run-slow: quantization

to trigger a CI running `quantization` test jobs.
Update translation template
Author: stevhliu
Pull request: yes

Updates translation template so we stop pinging Maria

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37294). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove HQQ from caching allocator warmup
Author: Cyrilvallez
Pull request: yes
Labels: name: for patch

# What does this PR do?

HQQ quantizer injects items in the state dict of the model which are neither parameters nor buffers, and do not seem to be moved to device with usual params (even though they are Tensor).
See https://github.com/huggingface/transformers/issues/37263 here for the bug that was introduced when allowing all quantizers to go through the caching allocator warmup.
So skipping this quantizer from allocation for now, but we truly need to standardize these quantizers.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37347). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
updated model card for Mistral
Author: NahieliV
Pull request: yes

This PR updates the model card for Mistral and follows the template outlined in the issue. Please let me know if any changes need to be made. 

Fixes https://github.com/huggingface/transformers/issues/36979

### Before submitting

- [x] This PR improves the docs.

### Who can review?
@stevhliu 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@stevhliu , I have incorporated your suggestions. Should be good now! 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37156). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@stevhliu , the modifications are done :)
Update falcon mamba card
Author: ricalanis
Pull request: yes

#36979 

* Updated the FalconMamba model card
* Second contrib, thank you again for your patience <3

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [] Did you write any new necessary tests? NA


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Fixed comments, ready for review.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37253). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Clean up the compressed-tensors integration
Author: MekkCyber
Pull request: yes

# What does this PR do?
Cleanup compressed-tensors integration as no longer used for Llama4

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37349). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update Model Card for Jamba
Author: ParagEkbote
Pull request: yes

# What does this PR do?

As described in the issue, this PR updates the model card for Jamba. I've chosen to also add the newer Jamba-Large-1.6 and Mini models in the model card as well and notes for loading the model using accelerate and BnB. Please let me know if any modifications are required and I will make the necessary changes.

#36979

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

@stevhliu

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Could you please review the changes?

cc: @stevhliu 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37152). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Idefics2 Fast Image processor 
Author: sushmanthreddy
Pull request: yes

close #36978 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@yonigozlan


This is the Idefics 2 fast image processor ,got some issue closed that pr
fix llama4 training
Author: hiyouga
Pull request: yes
Labels: name: for patch

# What does this PR do?

Fix some issues during Llama 4 training using transformers.

![image](https://github.com/user-attachments/assets/0191e111-6ea5-4f72-a369-f72e58975f4f)

Reproduce: https://github.com/hiyouga/LLaMA-Factory/pull/7611

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
AutoModel.from_pretrained without accelerate raises a NameError because `init_empty_weights` is not available
Author: LoicGrobol
Pull request: no

[d1b92369ca193](https://github.com/huggingface/transformers/blame/d1b92369ca193da49f9f7ecd01b08ece45c2c9aa/src/transformers/modeling_utils.py#L3736) introduced a call to `init_empty_weights` that leads to an error if accelerate is not installed. This should be guarded by a condition that accelerate is available (one already guards the import, but not the actual call).

Comments:
same, this problem is introduced in 4.50.3->4.51.0

downgrade to transformers=4.49.0 solved my problem

@samuelan installing accelerate is a better solution imo but still, that's a bug.

Yep accelerate should be installed, we will fix that ASAP sorry all ! 

This hurts especially in inference-only cases. I'm using sentence-transformers (pinned version) to compute embeddings, and this morning my builds started failing for no good reason. We have no need to install accelerate, and I'm not referencing transformers directly.

But, simply instantiating a SentenceTransformer calls the magic line, and everything fails.

same here @vladiliescu

Closing as this was fixed on main #37337

Sorry everyone this was not planned
enable 2 llama UT cases on xpu
Author: yao-matrix
Pull request: yes

case 1: pytest -rA tests/models/llama/test_modeling_llama.py::LlamaIntegrationTest::test_model_7b_logits
case 2: pytest -rA tests/models/llama/test_modeling_llama.py::LlamaIntegrationTest::test_model_7b_logits_bf16 

both don't have XPU criteria, put them as key 0 and reuse A100/A100 ground truth. Both can pass in Ponte Vecchio XPU.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @yao-matrix Thank you for this PR. It's not good to use `0` as the key IMO.

In #36569, @ivarflakstad introduced a way to better deal with more general expected values. Could you try to use that new approach in this PR? Thank you!

> Hi @yao-matrix Thank you for this PR. It's not good to use `0` as the key IMO.
> 
> In #36569, @ivarflakstad introduced a way to better deal with more general expected values. Could you try to use that new approach in this PR? Thank you!

@ydshieh i switched to use Expectations, pls help review, thx.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37126). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Let me run on CI runner and merge if everything is good !
Thank you a lot for the iterations.
Improvements in Gemma2 model card
Author: devesh-2002
Pull request: yes

# What does this PR do?
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #36979, 
This PR aims to improve model card for Gemma2 based on the given format mentioned [here](https://github.com/huggingface/transformers/issues/36979).

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@stevhliu, Please let me know, if there are any changes needed here.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @stevhliu. I have tried to make changes as per the suggestions. Please let me know if there are any further adjustments needed.

 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37076). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@stevhliu, I have made the changes, as suggested. Please let me know, if there is still anything.

Thank you for all the help! @stevhliu