noamaanMulla-03/transformers-issues huggingface dataset:
TypeError: Llama4TextModel._prepare_4d_causal_attention_mask_with_cache_position() missing 1 required positional argument: 'device'
Author: monk1337
Pull request: no
Labels: name: bug

### System Info

I am trying to evaluate the latest Llama-4 model but keep getting this error  

File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/accelerate/hooks.py", line 170, in new_forward
    output = module._old_forward(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/transformers/models/llama4/modeling_llama4.py", line 1018, in forward
    outputs = self.model(
              ^^^^^^^^^^^
  File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/transformers/models/llama4/modeling_llama4.py", line 657, in forward
    causal_mask, chunk_causal_mask = self._update_causal_mask(
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/transformers/models/llama4/modeling_llama4.py", line 783, in _update_causal_mask
    causal_mask = self._prepare_4d_causal_attention_mask_with_cache_position(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Llama4TextModel._prepare_4d_causal_attention_mask_with_cache_position() missing 1 required

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

using the lm eval repo 

### Expected behavior

Should be eval as other models.

Comments:
Skip is_flaky tests in the CI
Author: Rocketknight1
Pull request: yes


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37723). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Rocketknight1  don't hesitate to chat if you need some help on this PR regarding my above suggestion.

hi @ydshieh, are you sure we want to keep the old behaviour (retrying flaky tests) on the nightly runs, when it's not blocking PRs? Won't that just hide the flaky failures from us? 

Yes, at this moment, it's best to keep retry behavior. We don't run flaky tests on PRs, but we do want to run flaky tests on `merge to main` and `nightly CircleCI` and `nighly GitHub Actions CI`.

So run flaky tests without retry is going to make `merge to main` quite red 🔴 which is going to distract our attention from daily priorities. Also it would make the daily CI report quite difficult to follow (i.e. can't identify non-flaky tests' failure). Both will lead to the situation where team members having trouble and lack of motivation to fix them.

As you can see, even with retry, we already some flakiness. Let's focus on those and fix them if we see them failed on `merge to main`. For others, it depends on team members' engagement. We can try to remind from time to time (I know it's not perfect).

cc @ydshieh I reverted most of the changes, and added one extra `@is_flaky()` decorator (because that test was failing and blocking this PR!)

It works well. Let's merge tomorrow once team members are informed 🙏
`check_imports` unnecessarily verifies packages that may not be needed
Author: HichTala
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.51.3
- Platform: Linux-6.11.0-21-generic-x86_64-with-glibc2.39
- Python version: 3.12.4
- Huggingface_hub version: 0.30.2
- Safetensors version: 0.5.3
- Accelerate version: 1.6.0
- Accelerate config: 	- compute_environment: LOCAL_MACHINE
	- distributed_type: NO
	- mixed_precision: no
	- use_cpu: False
	- debug: False
	- num_processes: 1
	- machine_rank: 0
	- num_machines: 1
	- gpu_ids: 1
	- rdzv_backend: static
	- same_network: True
	- main_training_function: main
	- enable_cpu_affinity: False
	- downcast_bf16: no
	- tpu_use_cluster: False
	- tpu_use_sudo: False
	- tpu_env: []
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA RTX 3500 Ada Generation Laptop GPU

### Who can help?

@Rocketknight1

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In an environment without tensorflow run the following

```python
from transformers import AutoImageProcessor

image_processor = AutoImageProcessor.from_pretrained(
    "HichTala/reproduction_example",
    trust_remote_code=True
)
```

Here is the link to the [repo](https://huggingface.co/HichTala/reproduction_example).

### Expected behavior

Here the code crash with the error message

```
ImportError: This modeling file requires the following packages that were not found in your environment: tensorflow. Run `pip install tensorflow`
```

While in the code tensorflow is not necessary since there is a check function already:

```python
if is_tf_available():
    import tensorflow as tf
```

In the new way of parsing remote code imports using `AST`, it checks every imports except those from flash_attn.
May be we want to also ignore more library. I propose to ignore the library if the check function exists in `transformers.utils.import_utils`.

If you're ok with that. I'll be happy to submit a PR.

Comments:
@HichTala yes, I think that makes sense, particularly for `tensorflow`!

Hi, I'll submit a PR then !
[modular] Fix the prefix-based renaming if the old and new model share a common name suffix
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. See https://github.com/huggingface/transformers/pull/36895#issuecomment-2815598690 for details. It's mostly an issue for all the `detr` models, as they share the suffix in the base name.

cc @qubvel for viz!


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37829). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Transformers cli clean command
Author: LysandreJik
Pull request: yes

This updates the `transformers-cli` commands towards a cleaner `transformers` command.
It also updates the `chat` command to accept the model name or path as a positional argument as well as keyword argument.

This allows the following command:
```
transformers-cli chat --model_name_or_path Qwen/Qwen2.5-3B-Instruct
```
to become
```
transformers chat Qwen/Qwen2.5-3B-Instruct
```

A follow-up will be done for the `run` command (which is a different method than the `chat` command) to run efficiently. 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37657). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Option for save_pretrained() to Export Model Source Code Files
Author: MufanQiu
Pull request: no
Labels: name: Feature request

### Feature request

Add an optional flag (e.g., include_source_code=True) to PreTrainedModel.save_pretrained() / PreTrainedTokenizer.save_pretrained() that copies the currently-used Python implementation files (e.g., modeling_xxx.py, configuration_xxx.py, tokenization_xxx.py) into the target directory—even when those files already exist in the Transformers package.

### Motivation

- **Editable snapshots**
When iterating on research ideas, I often need to tweak the model’s forward pass or config class after fine-tuning. Editing code inside site-packages/transformers is brittle (breaks with upgrades, affects unrelated projects, requires virtual-env gymnastics).
- **Reproducibility & archiving**
Archiving both weights and exact code in a single artifact dramatically simplifies sharing and long-term reproducibility, especially when the upstream library may evolve.

### Your contribution

I’m not very familiar with the internal mechanism Transformers uses to store model files; if the feature is feasible, I’m willing to submit a PR myself later.

Comments:
Hi @WhenMelancholy!! The mechanism Transformers uses to store/lookup model files is one of two options, depending on whether it's a "custom code" model or not:

1) If it's not a custom code model, then its code is contained in the Transformers library. You can find it in [src/transformers/models/[model_name]](https://github.com/huggingface/transformers/tree/main/src/transformers/models). The name of the model directory should match the model type in the model's `config.json`.

2) If it is a custom code model, then the modeling code exists in a repo on the Hub instead. We're actually working on this right now, so that custom code models save their modeling code along with their weights in future. 

We probably won't add a mechanism to save library code with models - it would add complexity and the library code might go out of date, plus the relevant library code is very easy to find and download automatically if needed. However, we're working on something relevant for custom code models!
Fix duplicate init self attention in Qwen3 MoE
Author: yzlnew
Pull request: yes

# What does this PR do?

Remove the duplicated lines, maybe due to a manual error.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @yzlnew, your fix is correct but we have another PR open already! #37848 I'm going to merge that one instead just because it was opened earlier
uniformize kwargs for VisionTextDualEncoder
Author: tibor-reiss
Pull request: yes
Labels: name: Processing

Adds uniformized processors for VisionTextDualEncoder following https://github.com/huggingface/transformers/issues/31911.

@qubvel @molbap 



Comments:
@molbap I changed the order of `text` and `images` (I saw that in a similar PR as a comment). But this breaks backwards compatibility. I tried to do a `isinstance` check and flip after, but it errors because how these type aliases are defined. Any other idea how BC can be maintained - if it is a strict requirement?

Hi @tibor-reiss , you can use `_validate_images_text_input_order` from processing_utils!

> Hi @tibor-reiss , you can use `_validate_images_text_input_order` from processing_utils!

Done, could you please check again @molbap?

Friendly reminder @molbap 

Rebased
update comment in image_processing_base.py to reference image_process…
Author: arjunaskykok
Pull request: yes

…ing_utils_fast

Fixes #37815

## Who can review?

@Rocketknight1


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37864). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Misleading comment at image_processing_base.py
Author: arjunaskykok
Pull request: no

![Image](https://github.com/user-attachments/assets/2163745f-5803-457c-a7c3-ef922415bc15)

I think it should be "imported by both image_processing_utils and image_processing_utils_fast".

Comments:
I think you're right! You can check `git blame` to see who wrote it and ping them to check what it should be.

@amyeroberts Is the comment an oversight?

Amy's out at the moment, actually, so I'm happy to replace the comment with `imported by both image_processing_utils and image_processing_utils_fast`. Would you be willing to make a PR for that?

@Rocketknight1 Okay, I'll do it.
Fix device mismatch by moving num_items_in_batch to loss device in fixed_cross_entropy (#37886)
Author: iamsernine
Pull request: yes

Fixes: #37886 

This PR ensures that the num_items_in_batch tensor is moved to the same device as the loss tensor before performing division inside the fixed_cross_entropy function. This prevents runtime device mismatch errors when models are trained on non-default devices (e.g., CUDA).

🔧 Changes made:
Updated fixed_cross_entropy to move num_items_in_batch to loss.device before division when reduction is set to 'sum'.

This fix is particularly relevant for ForCausalLMLoss, where num_items_in_batch may be on a different device than logits or loss.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Remove redundancies for Qwen3MoeDecoderLayer 
Author: guoqingbao
Pull request: yes

# What does this PR do?
There are duplicate constructions for mlp and self attention layer in Qwen3MoeDecoderLayer. This PR remove those redundancies @ArthurZucker 

**From**
```python
class Qwen3MoeDecoderLayer(nn.Module):
    def __init__(self, config: Qwen3MoeConfig, layer_idx: int):
        super().__init__()
        self.hidden_size = config.hidden_size

        self.self_attn = Qwen3MoeAttention(config, layer_idx) #duplicated here
        self.mlp = Qwen3MoeMLP(config) #duplicated here

        self.self_attn = Qwen3MoeAttention(config, layer_idx)

        if (layer_idx not in config.mlp_only_layers) and (
            config.num_experts > 0 and (layer_idx + 1) % config.decoder_sparse_step == 0
        ):
            self.mlp = Qwen3MoeSparseMoeBlock(config)
        else:
            self.mlp = Qwen3MoeMLP(config, intermediate_size=config.intermediate_size)

        self.input_layernorm = Qwen3MoeRMSNorm(config.hidden_size, eps=config.rms_norm_eps)
        self.post_attention_layernorm = Qwen3MoeRMSNorm(config.hidden_size, eps=config.rms_norm_eps)

```

**To**
```python
class Qwen3MoeDecoderLayer(nn.Module):
    def __init__(self, config: Qwen3MoeConfig, layer_idx: int):
        super().__init__()
        self.hidden_size = config.hidden_size

        self.self_attn = Qwen3MoeAttention(config, layer_idx)

        if (layer_idx not in config.mlp_only_layers) and (
            config.num_experts > 0 and (layer_idx + 1) % config.decoder_sparse_step == 0
        ):
            self.mlp = Qwen3MoeSparseMoeBlock(config)
        else:
            self.mlp = Qwen3MoeMLP(config, intermediate_size=config.intermediate_size)

        self.input_layernorm = Qwen3MoeRMSNorm(config.hidden_size, eps=config.rms_norm_eps)
        self.post_attention_layernorm = Qwen3MoeRMSNorm(config.hidden_size, eps=config.rms_norm_eps)

```

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Update modular_qwen3_moe.py
Author: tanuj-rai
Pull request: yes

# What does this PR do?

Fixes #37813,
This PR removed redundant initializations of self.self_attn and self.mlp in Qwen3MoeDecoderLayer in modular_qwen3_moe.py.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@SunMarc 

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@Tanuj-rai this PR is correct :) You can:
1. close https://github.com/huggingface/transformers/pull/37826
2. run `make fix-copies` locally and push the changes

We got like 4 different PRs for this fix, so I have to pick one and I'm merging #37848! Thank you for the PR though - it was totally correct.

No problem at all! Thanks for reviewing and for the update.
[tests] Stricter generate + compilation test -- no recompilations allowed
Author: gante
Pull request: yes

# What does this PR do?

Follow-up to #37447 

This PR upgrades `test_generate_compile_model_forward` to catch recompilation issues. This is done by a) activating recompilation logs b) catching recompilation messages in the logs. The improved test would have failed with the changes that broke gemma 2/3 + compile 🤗 

In the process, a few extra things were standardized in the test, and a few skips were removed.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @manueldeprada (since you mentioned you are interested in torch.compile)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37629). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@zucchini-nlp 

> So this means all VLMs are actually compiling without any breaks, right. I am still wondering about the slow downs we experienced when compiling certain models

Yes, they are compilable without graph breaks. That by itself is useful, it means they can probably be exported with `torch.export` 💪 We'll have to profile to understand why we don't observe as big speedups as in LLMs 🔍  It may be due to the number of input tokens, it may be due to suboptimal image processing ops, ...

> One q about the test: it guards torch>=2.6, are the runner installing the latest version? Would be nice to get this test running on each PR to avoid accidentally breaking gemma again

Yes, they are running torch 2.6 (the runners' images install the latest version e.g. [here](https://github.com/huggingface/transformers/blob/1077603410cd73ba71d64a522033574d66d64b55/docker/torch-light.dockerfile#L8)).

On this PR's `ci/circleci: tests_torch`:
<img width="195" alt="Screenshot 2025-04-21 at 15 42 20" src="https://github.com/user-attachments/assets/0e4defb3-2170-4c8e-b8ce-94c5772df08e" />
[tests] reset logs in `torch.compile` test
Author: gante
Pull request: yes

# What does this PR do?

We set special log options in `test_generate_compile_model_forward`, but we don't reset them. This PR ensures we reset them.

(thank you @ydshieh for noticing it! https://github.com/huggingface/transformers/pull/37629#discussion_r2068663475)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37894). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.