noamaanMulla-03/transformers-issues huggingface dataset:
NameError: "init_empty_weights" not defined - ESM family models
Author: Eloffredo
Pull request: no
Labels: name: bug

### System Info

### **Issue** 

Using the latest release ``v4.51.0`` I get a NameError when loading the ESM model with ``facebook/esm2_t33_650M_UR50D``. Using the previous ``v4.50.3`` everything is working fine.

**system info**
```
- `transformers` version: 4.51.0
- Platform: Linux-6.8.0-1021-aws-x86_64-with-glibc2.35
- Python version: 3.13.1
- Huggingface_hub version: 0.30.2
- Safetensors version: 0.5.3
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A10G
```

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

### **Reproducibility**
I run ``uv run python hello.py`` with these two files:
**hello.py**
```
import transformers
from transformers import EsmForMaskedLM

def main():
    print(transformers.__version__)
    model_name = "facebook/esm2_t33_650M_UR50D"
    model = EsmForMaskedLM.from_pretrained(model_name)
    print(model)

if __name__ == "__main__":
    main()
```
**pyproject.toml**
```
[project]
name = "temp"
version = "0.1.0"
description = "Add your description here"
requires-python = ">=3.13"
dependencies = [
    "torch",
    "transformers==4.51.0",
]
```
### **Full traceback**
```
  File "/home/ssm-user/transformer-project/./hello.py", line 11, in <module>
    main()
    ~~~~^^
  File "/home/ssm-user/transformer-project/./hello.py", line 7, in main
    model = EsmForMaskedLM.from_pretrained(model_name)
  File "/home/ssm-user/transformer-project/.venv/lib/python3.13/site-packages/transformers/modeling_utils.py", line 279, in _wrapper
    return func(*args, **kwargs)
  File "/home/ssm-user/transformer-project/.venv/lib/python3.13/site-packages/transformers/modeling_utils.py", line 4333, in from_pretrained
    model_init_context = cls.get_init_context(is_quantized, _is_ds_init_called)
  File "/home/ssm-user/transformer-project/.venv/lib/python3.13/site-packages/transformers/modeling_utils.py", line 3736, in get_init_context
    init_contexts = [no_init_weights(), init_empty_weights()]
                                        ^^^^^^^^^^^^^^^^^^
NameError: name 'init_empty_weights' is not defined
```

### Expected behavior

The expected behavior is to instantiate the pretrained model without any issue.

Comments:
@Eloffredo A patch was released for this bug in the `v4.51.1`. So installing latest version should resolve this issue


yes! closing!
Preserve requires_grad in pre quantized model
Author: jerryzh168
Pull request: yes

Summary:
discovered this when running lm-eval for some models, current code will set requires_grad to True always

Test Plan:
lm_eval --model hf --model_args pretrained=jerryzh168/phi4-torchao-gguf-q4_k --tasks hellaswag --device cuda:0 --batch_size 8

Reviewers:

Subscribers:

Tasks:

Tags:

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @SunMarc
Use Python 3.9 syntax in tests
Author: cyyever
Pull request: yes

# What does this PR do?
A follow-up work to fix tests.
The changes were made by executing 
```
 pyupgrade --py39-plus tests/**/*py
```

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @cyyever 

Could you share how to produce the changes in this PR. It would be nice if the PR description could contain more context and details 🙏 

@ydshieh Added 

@ydshieh Thank you for your review. Admittedly there are 600+ files to fix but most changes are quit simple. ``Tuple --> tuple and List --> list`` because py 3.9+ support using these types as annotations instead of the counterparts from typing. For oversimplify of the open parameters, unfortunately there is not an option to disable that change.. 

BTW, there is no need to update with `main` frequently. We can do it just before we want to merge :-) once I have a final look, hopefully tomorrow.

@ydshieh  EnvironmentError is an alias of OSError since Python 3.3:
```
Changed in version 3.3: 
[EnvironmentError](https://docs.python.org/3/libraryexceptions.html#EnvironmentError), 
[IOError](https://docs.python.org/3/library/exceptions.html#IOError), [WindowsError](https://docs.python.org/3/library/exceptions.html#WindowsError), [socket.error](https://docs.python.org/3/library/socket.html#socket.error), [select.error](https://docs.python.org/3/library/select.html#select.error) 
and mmap.error have been merged into [OSError](https://docs.python.org/3/library/exceptions.html#OSError), and the constructor may return a subclass.
```

Thanks a lot, You are a real expert!

> @ydshieh EnvironmentError is an alias of OSError since Python 3.3:
> 
> ```
> Changed in version 3.3: 
> [EnvironmentError](https://docs.python.org/3/libraryexceptions.html#EnvironmentError), 
> [IOError](https://docs.python.org/3/library/exceptions.html#IOError), [WindowsError](https://docs.python.org/3/library/exceptions.html#WindowsError), [socket.error](https://docs.python.org/3/library/socket.html#socket.error), [select.error](https://docs.python.org/3/library/select.html#select.error) 
> and mmap.error have been merged into [OSError](https://docs.python.org/3/library/exceptions.html#OSError), and the constructor may return a subclass.
> ```



(@ydshieh I'm not merging since you've self-requested a review, I'm assuming you still want to double-check a few things :) )

> (@ydshieh I'm not merging since you've self-requested a review, I'm assuming you still want to double-check a few things :) )

Yes, I had an question regarding the exception type, and @Rocketknight1 answered. I am merge now.
Update composition flag usage
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Actual fix for https://github.com/huggingface/transformers/pull/36230. The very first PR that added the flag wasn't what we want, since the root cause of Pixtra config reloading is not in nested configs. The cause is how Mistral config is written with a `None` head-dim which is later inferred from `hidden_dim`. Bug from https://github.com/huggingface/transformers/issues/36048 can be reproduced by reloading only the mistral config

TL;DR; We can't do that because the modeling code has no assumption about `hidden_size = head_dim * num_heads`. So the solution is to set the `head dim` only when config has the value, otherwise it will get inferred during modeling attention

Also, this PR renames the flag which is very misleading in its name. The flag was first added for models like RAG, when the config can't be init without args. The flag therefore is used only once, when trying to get default values of the config class (if any). Renaming shouldn't break BC I believe

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36263). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @ydshieh this should solve the quick fix we did a few day ago :)

> cc @ydshieh this should solve the quick fix we did a few day ago :)

(For the record, here means #36230)
[Cache] rename `max_batch_size` -> `batch_size` in compilable caches
Author: gante
Pull request: yes

# What does this PR do?

WIP, see [this comment](https://github.com/huggingface/transformers/pull/37389#issuecomment-2789228023)

___________________________________

Uses `deprecate_kwarg` to rename `max_batch_size` to `batch_size` in all compilable caches. `max_batch_size` is a bad arg name: it implies that batch sizes smaller than `max_batch_size` can use the cache too, which is not the case.

_________________________
Note that this deprecation was started before, but we messed it up along the way:
1. Deprecation process started: https://github.com/huggingface/transformers/pull/32657
2. Deprecation message got changed to the opposite of the goal: https://github.com/huggingface/transformers/pull/34921
3. User-contributed PR that respected the (modified) deprecation message: https://github.com/huggingface/transformers/pull/37007

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37389). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@zucchini-nlp I see, the export use cache with a batch size > input batch size makes sense (export once, reuse with any batch size). 

It's feasible, the questions are a) code complexity; b) throughput. I'm going to give it a go 🤞 

Yeah, I am also concerned if it will add too much complexity. Thanks, will be cool if it's doable with minimal maintenance cost :)

It's actually quite clean and doesn't seem to have throughput disadvantages 👀 closing this PR in favor of expanding capabilities
Add glm4
Author: ArthurZucker
Pull request: yes
Labels: name: New model

# What does this PR do?



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37388). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
update deepspeed docker
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR updates deepspeed docker so that we run our tests with pytorch 2.6

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37371). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for your help @ydshieh !
Add bnb to the list of supported quantization methods for LLama4
Author: MekkCyber
Pull request: yes

# What does this PR do?
This pr adds support for bitsandbytes quantization, and loading bnb quantized models in transformers.

Some quantized Llama4 models can be found here : 

- https://huggingface.co/bnb-community/Llama-4-Scout-17B-16E-Instruct-bnb-4bit
- https://huggingface.co/bnb-community/Llama-4-Scout-17B-16E-Instruct-bnb-8bit

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37348). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

let's do the patch attribute that i talked about before this change breaks other bnb models !
Add "selecting a quantization method" doc
Author: DerekLiu35
Pull request: yes

# What does this PR do?
Modify docs to help users determine which quantization methods to use

## Who can review?
@SunMarc @stevhliu

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37159). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sounds good, but not sure where's the best place to include estimate of the calibration time.
Fix some failing AWQ tests
Author: DerekLiu35
Pull request: yes

This PR fixes some failures observed in the AWQ quantization tests (`tests/quantization/autoawq/test_awq.py`) 

Also, some notes:

1.  `test_quantized_model_bf16`
    *   Error: `RuntimeError: expected scalar type Half but found BFloat16`
    *   Reason: `transformers>=4.50` preserves `bfloat16` dtype for scales when loading with `torch_dtype=torch.bfloat16`. However, AWQ implementation expects `float16` scales.
    *   Solution: Cast the scales to `torch.float16` if the model's `torch_dtype` is `bfloat16` in AWQ quantizer.
    *   Note: This resolves the `RuntimeError` but causes an `AssertionError` comparing output to a `bfloat16` baseline. This is expected, as the fix makes AWQ operate correctly in its supported `float16` mode, which differs from `bfloat16` output.

2.  `test_quantized_model_no_k_proj_quantized`
    *   Error: `RuntimeError: expected scalar type Int but found Half`
    *   Reason: The `lm_head` module was being incorrectly identified for quantization and converted to `WQLinear_GEMM`.
    *   Solution: Adjusted logic within the AWQ quantizer that determines the set of modules to be excluded from quantization.

3.  `test_generation_fused` & `test_generation_fused_batched`
    *   Error (CI - T4 GPU): `RuntimeError: FlashAttention only supports Ampere GPUs or newer.`
    *   Reason (CI): The CI environment uses T4 GPUs (Compute Capability 7.5), which lack hardware support for FlashAttention
    *   Error (Ampere+ GPU - A100): `ValueError: too many values to unpack (expected 2)` during the attention forward pass.
    *   Reason (Ampere+): The `QuantAttentionFused` module provided by `AutoAWQ` returns 3 values. (PR #35235) have modified the expected return signature to 2 values, removing `present_key_value`, also cache implementation seems to have been changed.
    *   Solution (Ampere+): Further investigation needed

4.  `test_generation_llava_fused`
    *   Error: `ValueError: Image features and image tokens do not match: tokens: 1, features 576`
    *   Reason: It also seems related to (PR #35235), which seems to be first failing commit.
    *   Solution:  Further investigation needed


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37383). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @ydshieh @mekkcyber

LMK if you want us to merge this or you want to take care of the other issues 

Yeah, I think we should merge this PR first. 
I think the remaining issues (Fused Attention on Ampere+ and Llava Fused) might require significant follow-up, potentially involving coordination with AutoAWQ.
Attention Quantization with FBGemm & TP
Author: MekkCyber
Pull request: yes
Labels: name: for patch

# What does this PR do?

Adds support for attention quantization with FBGemm and TP

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37384). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
the fix that did not get in
Author: ArthurZucker
Pull request: yes
Labels: name: for patch

# What does this PR do?
 One fix did not get in the patch, I hate the github merging interface

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37370). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix in HybridChunkedCache
Author: Kh4L
Pull request: yes

Handles seq length smaller that the cache size in HybridChunkedCache

Fixes #37380


@ArthurZucker for review



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

In my case, this change made it work properly 👍 

but for smaller sequence length, we can return this indeed if not compiling prefill, if we compile prefill this is a regression

> Thanks a lot for the PR! We are working on the patch in #37370 we'll make sur this is fixed

Sure, closing as https://github.com/huggingface/transformers/pull/37370 's better fix
handle torch version edge cases
Author: winglian
Pull request: yes

# What does this PR do?

If torch is installed as `torch==2.6.0+cu126`, then this check fails to detect it as 2.6.0

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker @SunMarc 
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@vasqu you're right, using that helper func in #37400
Fix image processor speedup fixed
Author: Aravind-11
Pull request: yes

# What does this PR do?

This PR fixes copy inconsistencies in the image processor files for Deformable DETR and Grounding DINO. It synchronizes the optimization pattern between these two image processors to resolve CI check failures in PR #34432.

## Before submitting
- [x] This PR fixes issues in the code (not just a typo or docs improvement).
- [x] I've read the contributor guidelines.
- [x] This change is related to PR #34432 by yaswanth19.
- [x] Documentation updates are not needed for this change.
- [x] No new tests are needed as this is fixing inconsistencies in existing code.

## Who can review?
@yaswanth19 (since this is related to their PR)
@amyeroberts or @qubvel (vision models maintainers)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

@Aravind-11 Refer this https://github.com/huggingface/transformers/pull/34432#issuecomment-2724804020

Hey @Aravind-11, isn't it a duplicate of 
 - https://github.com/huggingface/transformers/pull/34432
 
 cc @yonigozlan
Add warning when failed to acquire other user's lock at model download
Author: manueldeprada
Pull request: yes

In cache directories that are shared between multiple users, it is common that a user starts downloading a model but gets tired and kills the process. This leaves the folder structure and their own locks on the filesystem.

If another user comes later and tries to use the same model, the loading will fail with a cryptic message:
```bash
OSError: meta-llama/Llama-3.1-8B-Instruct does not appear to have a file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt or flax_model.msgpack
```
What is happening under the hood is that it failed to acquire the lock, catches the exception, leaves no trace, then tries to recover here:

https://github.com/huggingface/transformers/blob/f834ca2c19215f1e4fb0959cc3faafeaf56cd4f7/src/transformers/utils/hub.py#L453C4-L472C10

Since it fails to find the files, the user gets the confusing OSError.

This PR just adds proper error handling to make clear what is happening, so that they know they can manually delete the locks. I don't think we should catch the exception and delete the locks ourselves, since the other user might be actually downloading the file, and UNIX locks are not shareable.

## Who can review?

@gante 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37395). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.