noamaanMulla-03/transformers-issues huggingface dataset:
RuntimeError: Failed to import transformers.models.bert.modeling_bert
Author: JaehyunsLee
Pull request: no
Labels: name: bug

### System Info

sentence-transformers==3.4.1
spacy-transformers==1.2.4
transformers==4.51.2
numpy==2.0.2
pandas==2.2.3
polars==0.19.19

Python 3.9.14

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Type `python `on terminal
2. Type `from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig`

### Expected behavior

Produces the following error: 
```
RuntimeError: Failed to import transformers.models.auto.modeling_auto because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
```


Comments:
Hi @JaehyunsLee, this usually indicates some other module that depends on Numpy (maybe Torch or Scipy?) was built against Numpy 1.0, but you're now using Numpy 2.0, which is incompatible. The easiest solution in these cases is usually to just make a new environment and install up-to-date packages if you're using `conda` or some other environment manager like that.

Closing because I don't think this is a `transformers` bug we can fix!
Fixed broken links
Author: cypherpepe
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

HI! I fixed broken links to the `convert-hf-to-gguf.py` script in the Arabic and Korean documentation files. The script was renamed from `convert-hf-to-gguf.py` to `convert_hf_to_gguf.py` in the llama.cpp repository.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Documentation: @stevhliu


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Add Fast Image Processor for EfficientFormer
Author: grewalsk
Pull request: yes

Add Fast Image Processor for EfficientFormer (#36978)

Fixes # (issue)
Part of the ongoing effort to add fast image processors to all vision models (issue #36978)

Before submitting
- [x] This PR improves performance for EfficientFormer image processing
- [x] I read the contributor guideline, Pull Request section
- [x] This was discussed as part of issue #36978 for adding fast image processors to all models
- [x] I updated the documentation with proper docstrings following the documentation guidelines
- [x] I wrote new tests for the fast image processor implementation

Implements a GPU-accelerated image processor for EfficientFormer using PyTorch/torchvision operations instead of PIL/NumPy. This significantly improves performance, especially for batch processing.

Key changes:
- Added EfficientFormerImageProcessorFast class with GPU-accelerated methods
- Updated __init__.py to include the fast processor
- Created comprehensive test suite

Note: 11 tests related to AutoImageProcessor integration and serialization compatibility are skipped and will be addressed in a future PR.

Who can review?
@yonigozlan

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @grewalsk ! Sorry about that but I missed that this model was deprecated, so we shouldn't add a fast image processor for it... Feel free to choose another fast image processor from the community issue, I've removed the deprecated models from the list so this shouldn't happen again :)
Llama Kernel integration
Author: MekkCyber
Pull request: yes

# What does this PR do?
PoC of Integrating kernels into the llama model. The kernels used are for `attn`, `mlp`, and `rmsnorm`

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37092). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Flag to disable is coming1

I believe the implementation should be compatible with torch.compile.
For most simple modules, such as RMSNorm and operations like silu_and_mul in MLP, torch.compile already provides sufficient optimization. Therefore, for these cases, we should avoid overriding the forward method when torch.compile is enabled.

Suggestions:
In the kernels-community kernels, consider leveraging torch.compile for optimization, for example by using PyTorch ops or Triton implementations.
For custom kernels that are neither PyTorch nor Triton based, compatibility with torch.compile should also be taken into account. 
For details on supporting custom kernels, refer to the official documentation: https://pytorch.org/docs/stable/library.html



Yeah that's right, once we have a stable api, we can think about how to make it compatible with torch.compile, but for now using kernels can be deactivated, by uninstalling the kernels library or just setting the `DISABLE_KERNEL_MAPPING` to 1 for example.
I'm also doing some benchmarks to see how much we gain or lose from using kernels instead of torch.compile, that would lay a good basis to see what we can do to improve the performance
Fix mask handling for flex attention in llama/gemma2/mistral/qwen2
Author: flukeskywalker
Pull request: yes

# What does this PR do?

**Issue** 
In gemma2 and mistral modelling code, where the `_update_causal_mask()` methods were not handling the case of the `atn_implementation` being the recently added `"flex_attention"`.

**Main Fix**
For Mistral and Gemma2: Added logic for converting attention mask passed as as a tensor to a BlockMask required for flex attention, or skip any conversion if a BlockMask is given directly by the user. This logic is borrowed from `LlamaModel`.

This change also fixes the issue for Gemma3 and Qwen2 models due to inheritance.

**Additional Changes**
~~- For Llama, Mistral, Gemma2: Slightly modified the previous behavior when flex_attention is enabled. Previously in `LlamaModel`, the behavior was unclear/prone to failure if flex_attention was enabled but the attention mask was neither a Tensor nor a BlockMask. The updated code adds an assertion to explicitly fail in this situation.~~

- Updated type hints and docs to reflect that attention mask may also be a BlockMask.


Fixes #37299 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @flukeskywalker, don't forget to `pip install -e .[quality]` and `make fixup` to get the code quality checks to pass!

Ah it looks like the failing checks were repo consistency checks because I updated `LlamaModel.update_causal_mask()` as well as type hints in derived models such as `LlamaForSequenceClassification` etc.,  but this method and the type hints are meant to be copied over to lots of other models. 

I will try to fix by running `make fixup` and removing some type hints that I added that create conflicts downstream due to being copied from Llama models.

CI looks green now, cc @arthurzucker for flexattention review

Merging since the review comments have been addressed!
Add Fast LeViT Processor
Author: keetrap
Pull request: yes

Related #36978
cc @yonigozlan

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37154). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add Fast Mobilenet-V2 Processor
Author: keetrap
Pull request: yes

Related #36978
cc @yonigozlan

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37113). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix wrong argparse type in modular checker script
Author: seven-mile
Pull request: yes

# What does this PR do?

In the `argparse` library, the `type` parameter is applied to each argument individually. When using `type=list` in combination with `nargs="+"`, the command `python utils/check_modular_conversion.py --files abc def` is parsed as `[['a', 'b', 'c'], ['d', 'e', 'f']]`. This behavior occurs because the script typically relies on no arguments being passed, defaulting to `['all']`, which has allowed it to function correctly until now.

This PR corrects this behaviour.

CC: @ArthurZucker


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Fixing gated repo issues
Author: MekkCyber
Pull request: yes

# What does this PR do?

Using unsloth model alternative for gated repos in quark quantization, instead of `require_read_token`.

https://github.com/huggingface/transformers/actions/runs/14415368554/job/40431139879

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37463). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Segmentation Fault
Author: lordsoffallen
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.48.3
- Platform: Linux-5.15.0-76-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.30.2
- Safetensors version: 0.4.5
- Accelerate version: 1.2.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: yes
- GPU type: NVIDIA A100 80GB PCIe

### Who can help?

@Rocketknight1 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

depth = pipeline(
        task="depth-estimation",
        model="depth-anything/Depth-Anything-V2-Large-hf",
    )

Returns segmentation fault( core dumped) error. 

### Expected behavior

Expected to load the model

Comments:
Hi @lordsoffallen, I don't get a segmentation fault when I run that code, so it might be an environment issue. Can you try narrowing down the cause, for example by:

1) Updating to the latest version of `transformers`
2) Making a new environment e.g. with `conda` and installing fresh packages into it
3) Seeing if you still get the error on another system (you can get a free instance on Colab)

If there's a specific environment that triggers it, we can investigate further!

It's very likely environment related but upgrading or downgrading didn't help. I switched to a different docker image and it worked there so I'm not really sure what to debug honestly. I can update here if I encounter this error again but for now I'm good. 

No problem, it happens! I'll close for now, but if you can find a specific environment that reliably triggers the issue, feel free to reopen and let us know.
[qwen-omni] fix processor
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes #37491 and deflakes a test

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37493). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[BUG] Qwen2.5-Omni-7B processor numpy view error.
Author: kobenaxie
Pull request: no

https://github.com/huggingface/transformers/blob/4b8c6d4cf8c779bf0895deb980669f5b2cb5d182/src/transformers/models/qwen2_5_omni/processing_qwen2_5_omni.py#L247

This bug was introduced by #https://github.com/huggingface/transformers/pull/36752/commits/605da8121c4a708102ce04d7092eb61cf7229e8a


Comments:
oops, sorry, will fix now

> oops, sorry, will fix now

Another bug https://github.com/huggingface/transformers/blob/a5079a2c84e9ca5c88a1ca124823e4a89626bd7b/src/transformers/models/qwen2_5_omni/processing_qwen2_5_omni.py#L330-L331
The `content` of `system` is `str` not `List[Dict[str, str]]`, so can't use `conversation[0]["content"][0]["text"] ` to get the default system prompt.

It should be list of dicts, reference in the docs
https://huggingface.co/docs/transformers/main/en/model_doc/qwen2_5_omni#usage-example

> It should be list of dictx, reference in the docs https://huggingface.co/docs/transformers/main/en/model_doc/qwen2_5_omni#usage-example

Ok, get it, examples in [github](https://github.com/QwenLM/Qwen2.5-Omni/tree/main) and [huggingeface](https://huggingface.co/Qwen/Qwen2.5-Omni-7B) should be updated.
Add Fast owlvit Processor
Author: keetrap
Pull request: yes

related #36978 
cc @yonigozlan 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37164). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Vocab Size does not change when adding new tokens
Author: ncoop57
Pull request: no

Env:
- `transformers` version: 4.8.2
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.10
- PyTorch version (GPU?): 1.9.0+cu102

When adding new tokens to an existing tokenizer, the tokenizer's vocab size variable doesn't change. I believe it should be updated every time the tokens change.

Here is a google colab to reproduce: https://colab.research.google.com/drive/1mC_eSmHOgA_F5fPX7AsUt86jAbC7iSSw?usp=sharing

Specifics:
```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("gpt2")
current_size = tokenizer.vocab_size
tokenizer.add_tokens(["new_token"])
tokenizer.vocab_size, current_size, len(tokenizer.vocab)
```

Outputs: (50257, 50257, 50258)

The same happens when I do the following as well
`tokenizer = AutoTokenizer.from_pretrained("gpt2", additional_special_tokens=["new_token"])`

Comments:
I think you should use `print(len(tokenizer))` instead of `print(tokenizer.vocab_size)` (as the `vocab_size` is a fixed attribute, referring to the base vocabulary without any additional tokens). Refer to [this](https://github.com/huggingface/transformers/issues/1413#issuecomment-538083512) and [this](https://github.com/huggingface/transformers/blob/2dd9440d0835782e41ae415a68e71fd15051c428/src/transformers/tokenization_utils.py#L161).

ah okay, didn't realize this was expected behavior. Thanks!

So, shouldn't we change the vocab size, too, after adding the new tokens? But when I try to change the vocab size in the config, we can't load the pre-trained model as there is an error for size mismatch in the weights.
Remove deprecation warning for `num_logits_to_keep`
Author: Cyrilvallez
Pull request: yes

# What does this PR do?



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37149). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Don't auto-assign reviewers when the author is in HF
Author: Rocketknight1
Pull request: yes

As the title says, we don't want to auto-assign reviewers for staff authors because they can do it themselves (and probably more accurately)

Comments:
cc @ydshieh!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37500). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
make test_snowman_image_captioning pass on XPU, by sharing same atol w/ ROCM
Author: yao-matrix
Pull request: yes

`pytest -rA tests/models/kosmos2/test_modeling_kosmos2.py::Kosmos2ModelIntegrationTest::test_snowman_image_captioning` PASS.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Tensor parallel support for LLM training.
Author: czkkkkkk
Pull request: no
Labels: name: Feature request

### Feature request

Hi HF team,

I'm wondering about the current status of tensor parallelism (TP) support in Hugging Face. I've noticed that some standard models, such as llama4 and mixtral, include TP sharding plans, and `.from_pretrained` appears to support [loading models with a TP plan](https://github.com/huggingface/transformers/blob/4e63a1747ce6a4b5f75e8d2318857c2b76c3ba23/src/transformers/integrations/tensor_parallel.py#L620). So it seems that TP is supported for inference.

However, I'm curious about training support. Does the transformers library support TP combined with data parallelism (DP) during training? Also, it looks like `.save_pretrained` doesn't currently support saving TP-sharded models—can you confirm if that's the case, or if there's a workaround?

Thanks in advance!

### Motivation

To support large-scale LLM training with TP.

### Your contribution

Happy to contribute if there is a specific way to support TP+DP training.

Comments:
Hello, we currently work on supporting up-to 3D parallelism (TP+DP+FSDP) with Accelerate, you can follow the progress and discussion in [this](https://github.com/huggingface/accelerate/pull/3498) PR. Going to open matching PR to transformers soon.

As for TP-only, this should now work on main of Accelerate and Transformers for training AND inference with these 2 PRs (#37054) and ([#3457](https://github.com/huggingface/accelerate/pull/3457)) merged.

RE model saving: I'm not entirely sure on this, I think it's being worked on in #36434 .
Refixing require_read_token
Author: MekkCyber
Pull request: yes

# What does this PR do?



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37427). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.