noamaanMulla-03/transformers-issues huggingface dataset:
Audio encodings now match conv2d weight dtype in Gemma3nAudioSSCPConvBlock
Author: Malav-P
Pull request: yes

# What does this PR do?

The change ensures that the audio encodings are cast to the same type as `conv.weight` tensor. This is done by appending `to(self.conv.weight.dtype)` to the `audio_encodings_padded` rvalue.  

Fixes an issue where the conv2d forward pass throws an error. Code for reproducible error (run on Mac M1) :


```python
from transformers import pipeline
import torch

pipe = pipeline(
    "image-text-to-text",
    model="google/gemma-3n-e2b-it",
    device=0,
    torch_dtype=torch.bfloat16,
    cache_implementation="static"
)


messages = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": "5676.wav"},
            {"type": "text", "text": "Transcribe this audio file."}
        ]
    }
]

output = pipe(text=messages, max_new_tokens=200, torch_dtype=torch.bfloat16)
print(output[0]["generated_text"][-1]["content"])
```
Full stack trace is attached [error.txt](https://github.com/user-attachments/files/21478915/error.txt).

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Suggested Reviewers
@ArthurZucker 



Comments:
cc @eustlb 

> This makes sense, happy to merge!

Awesome, ty!


Hey @ArthurZucker thanks again for looking at the PR! It seems there's one more workflow (build pr docs) that needs to run before I can merge, but I don't have the rights to run it. Could I get an assist here? Once it's done I'll be able to merge! 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma3n

Oups sorry approved and enabling auto merge!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39743). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Oups sorry approved and enabling auto merge!

Awesome, thanks for the assist!
Enable SIM rules
Author: cyyever
Pull request: yes

# What does this PR do?
Enable SIM rules of ruff, except those for code styles.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: aya_vision, biogpt, clvp, cohere2_vision, deepseek_vl, deepseek_vl_hybrid, fsmt, janus, jetmoe, mm_grounding_dino, oneformer

cc @arthurzucker @cyrilvallez, more Ruff style rules

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39806). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Missing einops dependency causing ModuleNotFoundError
Author: iforgetmyname
Pull request: no
Labels: name: bug

### System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.
- `transformers` version: 4.53.2
- Platform: Linux-4.19.90-vhulk2211.3.0.h1543.eulerosv2r10.aarch64-aarch64-with-glibc2.35
- Python version: 3.11.13
- Huggingface_hub version: 0.34.3
- Safetensors version: 0.5.3
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.6.0+cpu (NPU)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: yes
- Using NPU in script?: yes
- NPU type: Ascend910B4
- CANN version: 8.2.RC1.alpha003

### Who can help?

@ivarflakstad 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Background: running SGLang on Ascend NPU

1. Run with ascend/cann:8.2.rc1.alpha003-910b-ubuntu22.04-py3.11 docker image
2. Install deps: `pip install torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cpu && pip install torch_npu==2.6.0`
3. Clone sglang and run `pip install -e "python[srt_npu]"`
4. Check `transformers env`

<img width="898" height="641" alt="Image" src="https://github.com/user-attachments/assets/4fee73a0-1dc7-48b5-a126-9b50e8ece8d0" />

### Expected behavior

import transformers successfully

Comments:
Answered on the PR this is NPU specific

#40002 will handle this but let me know if not, so closing this. Feel free to reopen if the issue persists.

> [#40002](https://github.com/huggingface/transformers/pull/40002) will handle this but let me know if not, so closing this. Feel free to reopen if the issue persists.

looks good, i will give it a try, thx!
🌐 [i18n-KO] Translated `tiny_agents.md` to Korean
Author: AhnJoonSung
Pull request: yes

# What does this PR do?

Translated the `tiny_agents.md` file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 KREW 팀원들에게 리뷰를 요청하는 아래 주석을 노출해주세요!-->
May you please review this PR?
<!-- @jungnerd, @luckyvickyricky, @chelsseeey, @skwh54, @amo33, @maximizemaxwell, @D15M4S -->
<!-- @harheem, @nsbg, @Youngdong2, @xhaktm00, @ssunbear, @ChoHyoungSeo, @judy-choi -->
<!-- @4N3MONE, @Kim-Ju-won, @ahnjj, @FacerAin, @ssum21, @TaskerJang, @HyunZ118 -->
@yijun-lee, @songi104, @chhaewxn, @jihyun-0611, @seopp, @pyapyapya

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review? (Final)
@stevhliu May you please review this PR?

Comments:
build-doc

[Building docs for all languages...](https://github.com/huggingface/transformers/actions/runs/16914014479)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39913). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

build-doc

[Building docs for all languages...](https://github.com/huggingface/transformers/actions/runs/16927175848)
save_pretrained not working correctly when using device_map="auto" for big models in from_pretrained 
Author: bZehner-git
Pull request: no

### System Info

- `transformers` version: 4.21.3
- Platform: Windows-10-10.0.19044-SP0
- Python version: 3.10.5
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.12.0 (True)
- Tensorflow version (GPU?): 2.9.1 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <True>
- Using distributed or parallel set-up in script?: <True>

### Who can help?

@patrickvonplaten

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
ML_MODEL = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xxl", device_map="auto")
ML_MODEL.save_pretrained("custom_path")
```

1) Load Flan-T5-xxl using from_pretrained with device_map="auto"
In my case the model is loaded to 24 GB GPU RAM (RTX 3090) and the rest of the model ist loaded to the CPU RAM

2) Save Model to custom path using save_pretrained
In my case save_pretrained seems only to save the model information from the GPU RAM. Checking the file sizes in the custom directory only chunk 1 and 2 of the model.bin files have the expected 10GB. Chunk 3 is only partly saved and chunk 4 and 5 have only a few kB. 

### Expected behavior

When using from_pretrained without device_map="auto" the model is completely loaded into CPU RAM and also completely saved using save_pretrained. Checking the file sizes in the custom path chunk 1 to 4 have the expected 10GB and chunk5 has the expected 6GB. Same file sizes as in the Transformer Cache Directory. This behaviour is expected when using from_pretrained with device_map="auto".

Comments:
Thanks for the report! Using `device_map="auto"` is only for inference, and it indeed does not work with `save_pretrained` yet, especially with offloaded weights.

We will look at adding support for this in the future!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add model card for MobileViT
Author: Shivamjan
Pull request: yes

# What does this PR do?

This PR adds a detailed and beginner-friendly model card for MobileViT to the Hugging Face Transformers documentation. The previous model card was minimal and lacked clear explanations about the model architecture. This model retains several elements from the earlier version, as they remain applicable and effective for users.

The new version includes:
- A clear explanation of the MobileViT architecture.
- Notes on preprocessing and image format.
- Clarifies how to use the model for classification and segmentation.
- Highlights TensorFlow Lite compatibility for mobile use.
- Primary references to the original paper and related resources.

Fixes # (issue)


## Before submitting
- [ x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.



Comments:
@stevhliu Please take a look at your convenience and do let me know if there is any further changes required.

@stevhliu I have made the suggested changes, do take a look!

Take a look, I hope it looks good now.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40033). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Kudos this is the 20 000th commit! 🤗
changed xLSTMRMSNorm to RMSNorm
Author: nikitazuevblago
Pull request: yes

xLSTMRMSNorm doesn't exist and hence gives an ImportError, without it users can't test NX-AI/xLSTM-7b.

cc @Cyrilvallez

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: xlstm
Question: Llama4 weight reshaping
Author: gskorokhod
Pull request: no

Hi all

I am trying to extract the original Llama4 MoE weights, specifically:

- `experts.w1` (aka `experts.moe_w_in_eD_F`)
- `experts.w3` (aka `experts.moe_w_swiglu_eD_F`)

I need both of these in the shape `[E, D, N]`, where:

- E is the number of experts (16 for Scout)
- D is the embedding dimension (5120)
- N is the intermediate dimension (8192)

I tried just splitting `experts.gate_up_proj` in half along the last dimension to get w1 and w3, but although the dimensions match, the model is outputting nonsense, so I assume the actual order of the weights is wrong.

Could someone help me make sense of this snippet (from `convert_llama4_weights_to_hf`)?
Why is this hard coded indexing / reshaping being done and do you have any suggestions for how to get the original weight back?

```python
elif re.search(r"(gate|up)_proj", new_key):
        path = new_key.split(".")
        gate_key = re.sub(r"(gate|up)_proj", lambda m: "gate_proj", new_key)
        up_key = re.sub(r"(gate|up)_proj", lambda m: "up_proj", new_key)
        if gate_key == new_key:
            state_dict[new_key] = torch.cat(current_parameter, dim=concat_dim)
        elif new_key == up_key:
            if "experts" not in new_key:
                state_dict[new_key] = torch.cat(current_parameter, dim=concat_dim)
            else:
                # gate_proj = moe_w_in_eD_F = w1
                gate_proj = state_dict.pop(gate_key)
                gate_proj = [
                    gate_proj.reshape(num_experts, -1, 8, 1024)[:, :, k, :].reshape(num_experts, -1, 1024)
                    for k in range(8)
                ]
                gate_proj = torch.cat(gate_proj, dim=-1)

                # up_proj = moe_w_swiglu_eD_F = w3
                up_proj = [
                    k.reshape(num_experts, -1, 8, 1024).reshape(num_experts, -1, 1024)
                    for k in current_parameter
                ]
                up_proj = torch.cat(up_proj, dim=-1)

                gate_up_proj = torch.cat((gate_proj, up_proj), dim=-1)
                new_key = new_key.replace("up_proj", "gate_up_proj")
                state_dict[new_key] = gate_up_proj.contiguous()

            tqdm.write(f"Processing: {key.ljust(50)}  ->\t {new_key}, {state_dict[new_key].shape}")
```

Thank you!

Comments:
Fix QuantoQuantizedCache import issues
Author: manueldeprada
Pull request: yes

Revert some changes from #39797 to fix QuantoQuantizedCache.

cc @Cyrilvallez 

Fixes #40099

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40109). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
QuantoQuantizedCache is not working
Author: maxjeblick
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.56.0.dev0
- Platform: Linux-6.14.0-27-generic-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 0.34.3
- Safetensors version: 0.6.1
- Accelerate version: 1.9.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.7.1+cu126 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: Yes
- GPU type: NVIDIA RTX A6000


### Who can help?

@manueldeprada  @SunMarc @MekkCyber

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I'm using the ame script as in the `QuantoQuantizedCache` docstring

```
from transformers import AutoTokenizer, AutoModelForCausalLM, QuantoQuantizedCache

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-0.5B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-0.5B-Instruct")

inputs = tokenizer(text="My name is Qwen2", return_tensors="pt")

past_key_values = QuantoQuantizedCache(config=model.config, nbits=4)
outputs = model(**inputs, past_key_values=past_key_values, use_cache=True)
outputs.past_key_values
```
which yields to
```
Traceback (most recent call last):
  File "/home/mjeblick/.config/JetBrains/PyCharm2025.2/scratches/scratch_41.py", line 8, in <module>
    past_key_values = QuantoQuantizedCache(config=model.config, nbits=4)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/2tb/PyCharmProjects/kvpress/.venv/lib/python3.12/site-packages/transformers/cache_utils.py", line 1305, in __init__
    super().__init__("quanto", config, nbits, axis_key, axis_value, q_group_size, residual_length)
  File "/mnt/2tb/PyCharmProjects/kvpress/.venv/lib/python3.12/site-packages/transformers/cache_utils.py", line 1257, in __init__
    layer_class(nbits, axis_key, axis_value, q_group_size, residual_length)
  File "/mnt/2tb/PyCharmProjects/kvpress/.venv/lib/python3.12/site-packages/transformers/cache_utils.py", line 587, in __init__
    super().__init__(
  File "/mnt/2tb/PyCharmProjects/kvpress/.venv/lib/python3.12/site-packages/transformers/cache_utils.py", line 515, in __init__
    super().__init__(self)
TypeError: CacheLayerMixin.__init__() takes 1 positional argument but 2 were given

```

(I'm using optimum_quanto-0.2.7, I don't think, however, the version matters here, as it seems to be an inheritance issue).


### Expected behavior

`QuantoQuantizedCache` works as expected.

Comments:
cc @manueldeprada if you have time :)

should be fixed in the PR! Thanks for reporting!
[serve] allow array `content` inputs for LLMs
Author: gante
Pull request: yes

# What does this PR do?

LLMs in `transformers serve` now accept `content` messages containing arrays, as [expected in the API](https://platform.openai.com/docs/api-reference/chat/create).

Adds tests to prevent regressions.

Example of a command that gets fixed in this PR:
```bash
curl -X POST http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{"messages": [{"role": "system", "content": [{"text": "Can you help me write tests?", "type": "text"}]}], "temperature": 0.9, "max_tokens": 1000, "stream": true, "model": "Qwen/Qwen2.5-0.5B-Instruct"}'
```

Fixes #39791

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39829). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker this one too 🤗
`decoding_method` argument in generate
Author: manueldeprada
Pull request: yes

`custom_generate` already allows a Hub/local repository to override `generate()` with a custom implementation.

This PR extends the feature to also accept **custom decoding functions** directly.
When a function is provided, GenerationMixin.generate still handles all input preparation and configuration, and the function only needs to implement the decoding loop.

This enables moving built-in decoding methods such as DoLa and contrastive decoding to the Hub, and power users to easily plug in their own custom loops.

[Docs preview of the new feature here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40085/en/generation_strategies#reusing-generate-s-preparation-steps-by-passing-a-callable)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40085). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Collated reports
Author: ivarflakstad
Pull request: yes

Generates a report of reports for easy access to all results of a CI run

```json
{
    "gpu_name": "MI300",
    "machine_type": "multi-gpu",
    "commit_hash": "1234567",
    "total_status_count": {
        "passed": 99999,
        "failed": 420,
        "skipped": 69,
        "error": 0,
        "null": 0
    },
    "results": [
        {
            "model": "aimv2",
            "results": [
                {
                    "status": "passed",
                    "line": "tests/models/aimv2/test_modeling_aimv2.py::Aimv2VisionModelTest::test_can_load_with_global_device_set",
                    "count": 1
                },
... snip ...
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40080). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

For gpu name, we might be able to check the workflow file name that trigger  the run and use it to assign the value.
`TypeError: 'builtins.safe_open' object is not iterable` in `load_pytorch_state_dict_in_tf2_model `
Author: harupy
Pull request: no

```
Traceback (most recent call last):
  File "/home/runner/work/dev/dev/tests/transformers/helper.py", line 321, in <module>
    prefetch_models()
  File "/home/runner/work/dev/dev/tests/transformers/helper.py", line 317, in prefetch_models
    func()
  File "/home/runner/work/dev/dev/tests/helper_functions.py", line 672, in decorated_func
    return test_func(*args, **kwargs)
  File "/home/runner/work/dev/dev/tests/transformers/helper.py", line 46, in load_small_qa_tf_pipeline
    model = transformers.TFAutoModelForQuestionAnswering.from_pretrained(architecture)
  File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 600, in from_pretrained
    return model_class.from_pretrained(
  File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/transformers/modeling_tf_utils.py", line 2964, in from_pretrained
    return load_pytorch_state_dict_in_tf2_model(
  File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/transformers/modeling_tf_pytorch_utils.py", line 333, in load_pytorch_state_dict_in_tf2_model
    for key in pt_state_dict:
TypeError: 'builtins.safe_open' object is not iterable
```

Code to reproduce:

```python
import transformers

transformers.TFAutoModelForQuestionAnswering.from_pretrained("csarron/mobilebert-uncased-squad-v2")
```

Package versions:

- `transformers`: 4.55.0
- `safetensors`: 0.6.1

Comments:
Our TF code is deprecated, unfortunately, and will probably be removed at some point! We're not actively maintaining it anymore.

Thanks for the reply! I'll migrate to torch.

@Rocketknight1 and whoever might face the same issue
this broke in `v4.55.0`, so you'd need to downgrade to `transformers<=4.54.1`