noamaanMulla-03/transformers-issues huggingface dataset:
Support of Qwen3 GGUF model
Author: Auth0rM0rgan
Pull request: no

Hi, I am getting the following error when I want to use the GGUF model with Qwen3
"ValueError: GGUF model with architecture qwen3 is not supported yet."

I have the latest transformers and gguf-0.17.0
```
self.tokenizer = AutoTokenizer.from_pretrained(model_name, gguf_file= "Qwen3-0.6B-Q2_K_L.gguf",use_fast=True)
        if self.tokenizer.pad_token is None:
            self.tokenizer.pad_token = "<pad>"
            self.tokenizer.add_special_tokens({"pad_token": "<pad>"})
        self.tokenizer.padding_side = "left"
        self.model = AutoModelForCausalLM.from_pretrained(
            model_name,
            gguf_file = "Qwen3-0.6B-Q2_K_L.gguf",
            pad_token_id=self.tokenizer.pad_token_id,
            trust_remote_code=True,
            torch_dtype=torch.bfloat16,
            device_map="auto",
        )
```
How can I use the gguf model of Qwen3 with transformers? Could you please add the support of it?

Thanks!

Comments:
There is an open PR for this #38645

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Force real tensors and clone state_dict in src/transformers/modeling_utils.py
Author: MutugiD
Pull request: yes
Labels: name: Flax

What does this PR do?

This PR changes the default behavior of Flax→PyTorch conversion to always allocate real CPU tensors (by setting ```low_cpu_mem_usage=False``` in ```from_pretrained)``` and replaces the old offload logic in ```save_pretrained``` with an up-front detach-and-clone of the entire state_dict. This guarantees no meta-tensor placeholders or shared-storage checks, so converted ViT models now save and reload without errors.

Fixes #37999
Who can review?

    Flax integration: @gante

    PyTorch save/load logic: @Rocketknight1

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Update phi4_multimodal.md
Author: tanuj-rai
Pull request: yes

# What does this PR do?

This PR updates the Phi-4 Multimodal card documentation as described in #36979, in an attempt to standardize all model-cards.

#36979

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38830). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Thanks!
> 
> We'll merge for now and fix the code examples later :)

Great @stevhliu!. Thank you so much for every change you have suggested.
Ignore extra position embeddings weights for ESM
Author: Rocketknight1
Pull request: yes

Before #38089, ESM created `self.position_embeddings` even when `config.position_embedding_type == "rotary"`. This weight was not used. The error was fixed in #38089, but most ESM checkpoints still have the unused `self.position_embeddings` weight. This PR adds that it the unexpected_ignore list, so users don't get scary warnings for a benign issue.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39063). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: esm
Fix invalid property
Author: cyyever
Pull request: yes

# What does this PR do?

ruff detects some invalid property definitions. This PR fixes them.


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gptq, higgs

Hi @cyyever, I'm not sure what makes those type hints invalid! I know that after `from __future__ import annotations` that we can just do `Optional[PreTrainedModel]`, but the existing hints are correct until then, right?

@Rocketknight1 Python properties should not accept other parameters than `self`, they are not used.

Oh, I see! I completely misread, I thought this was about the type hint. You're entirely correct!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39384). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix bugs from pipeline preprocessor overhaul
Author: Rocketknight1
Pull request: yes

I overhauled pipeline processor loading in #38947, which caused some test failures. They should mostly be fixed by the user PR at #39411, but the remaining issues are resolved here.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39425). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
FP8 training support for Model Parallel / Tensor Parallel (MP/TP)
Author: edgeinfinity1
Pull request: no
Labels: name: Feature request

### Feature request

I recieve message "ValueError: The model you are trying to fine-tune is quantized with QuantizationMethod.FP8 but that quantization method do not support training. Please open an issue on GitHub: https://github.com/huggingface/transformers to request the support for training support for QuantizationMethod.FP8" when trying to finetune a fp8 model.
I have learned from the documentations that fp8 models can be trained with ddp, zero or fsdp. Is there a way to do it with MP/TP for huge fp8 models?

### Motivation

Enable finetuning huge fp8 models, like Qwen/Qwen3-235B-A22B-FP8

### Your contribution

I'm afraid it's too tough for me, but I'll do whatever I can if you need.

Comments:
cc @sunmarc since that's a trainer error!

Hey @edgeinfinity1, unfortunately, we don't have yet recipes to do TP + FP8 training.
Pytorch language_modelling example run_clm fails when streaming is enabled
Author: HRezaei
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.53.0.dev0
- Platform: macOS-15.5-arm64-arm-64bit
- Python version: 3.10.16
- Huggingface_hub version: 0.30.0
- Safetensors version: 0.5.2
- Accelerate version: 1.8.1
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.5.1 (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No



### Who can help?

@ArthurZucker 
@sgugger 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the example file: `examples/pytorch/language-modeling/run_clm.py` with the argument `--streaming` enabled:

1. Clone the repository
2. Install transformers from source, `cd transformers && pip install .` 
3. For instance, run the command below: 
```shell
python examples/pytorch/language-modeling/run_clm.py --model_type gpt2 \ 
--tokenizer_name openai-community/gpt2 --dataset_name=HuggingFaceFW/fineweb \ 
--dataset_config_name=sample-350BT --streaming True --do_train --do_eval \ 
--overwrite_output_dir --max_steps=100 --max_train_samples=2000 --max_eval_samples=1000 \ 
--output_dir=/tmp/gpt2TestClm
```
3. You will see an error like `ValueError: Bad split: train[:5%]. Available splits: ['train']`

That's because in streaming mode, range splits are not available.

Also there are other bugs like running `len(train_dataset)` which raises an error if the dataset is in streaming mode.

[Here](https://colab.research.google.com/drive/10AhU2vT1QfTfHCRM2kdxEsu6ofpPICJq?usp=sharing) is a colab notebook to see the issue in action.

### Expected behavior

It is expected to pre-train a GPT2 model from scratch using the dataset in streaming mode


I'm working on a [PR](https://github.com/huggingface/transformers/pull/39286) that fixes this issue.


Comments:
Remove residual quantization attribute from dequantized models
Author: dwarez
Pull request: yes

# What does this PR do?
The `quantization_method` attribute was not being deleted during model dequantization, causing methods like `to()` to fail even when the operation should be valid for dequantized models.

Fixes # 39295


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
https://github.com/huggingface/transformers/issues/39295
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@SunMarc 


Comments:
Quick note: since I couldn't run the test with `model_name = "bigscience/bloom-1b7"` (which doesn't fit on my machine 😔) I ran the test with `model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"`and it passes. It shouldn't make any difference, I'm mentioning it just in case.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bnb

run-slow: bnb

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39373). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
`quantization_method` is not cleared after calling `.dequantize()`
Author: karol-szustakowski
Pull request: no
Labels: name: bug

### System Info

bitsandbytes==0.46.0
transfomers==4.51.3
Python 3.9.21

### Who can help?

@SunMarc @MekkCyber 


### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction


After calling `model.dequantize()` I would expect to be able to handle the model as if it were never quantized.
For example, I expected to be able to do this:
```py
model = model.dequantize()
model = model.to(dtype=torch.float16)
```
The above fails with the exception:
`
ValueError: You cannot cast a bitsandbytes model in a new `dtype`. Make sure to load the model using `from_pretrained` using the desired `dtype` by passing the correct `torch_dtype` argument.
`


### Expected behavior

The `quantization_method` should be cleared after the model is dequantized, this currently can be circumvented by:
```py
model = model.dequantize()
model.quantization_method = None
model = model.to(dtype=torch.float16)
```

Comments:
This happens because the `dequantize()` method is not removing the `quantization_method` attribute from the model, as [you can see here](https://github.com/huggingface/transformers/blob/fe5f3c85d292e34bed52e02a53edd5fa2acfc010/src/transformers/quantizers/base.py#L240). Therefore, in the `to` method of `PretrainedModel`, [this ValueError](https://github.com/huggingface/transformers/blob/fe5f3c85d292e34bed52e02a53edd5fa2acfc010/src/transformers/modeling_utils.py#L4134) gets raised.

I don't know if it's intended, but the `dequantize()` method seems quite straightforward in its semantic, it should remove all traces of quantization from the model's object. I did a fix locally and I managed to move the model on CPU after dequantizing it, so I assume there's no technical limitation that prevents to delete the `quantization_method` attribute in the `dequantization()` method.

What do you think? @ArthurZucker 

We should indeed remove all trace of quantization after dequantization ! Mind opening a PR to fix this @DWarez ? 

Hello @SunMarc. I've opened PR [#39373](https://github.com/huggingface/transformers/pull/39373) to fix this issue. The fix removes a residual quantization attribute during model dequantization, which should resolve the `to()` method failures.

~~I've also added a test case, but I'm unable to run it locally. It gets skipped with `SKIPPED (test is slow)`. Could you help me understand how to properly test this? Is there a specific flag or setup needed for quantization tests?~~

I found out how to not skip the test, however unfortunately the target model doesn't fit on my machine (I'm gpu poor 😔). I replaced it locally with `model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"` and the test passes.

Thanks :]
[vlm] fix loading of retrieval VLMs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, reported internally that slow tests are failing. We need to apply same changes as in VLMs to the models that use VLMs in their architecture




Comments:
run-slow: colpali, colqwen2

This comment contains run-slow, running the specified jobs: 

models: ['models/colpali', 'models/colqwen2']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39242). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I wanted to use `AutoModel` as we shouldn't be loading the lm-head for these model. But the Qwen2-based model was released after refactor, and can work without any `conversion_key_mapping` currently, and I don't want us to add another key mapping just to use `AutoModel` instead of `AutoModelForImageTextToText`

Hmm, ColQwen for me wasn't failing in a sense that the weights matched when laoding. But the tensors aren't close enough even after model was released. I can check out on runners and see what's the issue. 

ColQwen shouldn't have the same issue, it was released after the major refactor

Hi, sorry, I think my memory got messed

https://github.com/huggingface/transformers/pull/39043#issuecomment-3018650688

So that issue was already fixed, but my brain wasn't yet. 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: colpali, colqwen2
docs: update SuperGlue docs
Author: sbucaille
Pull request: yes

# What does this PR do?

Updates SuperGlue model card for #36979 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

@stevhliu


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39406). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
docs: update LightGlue docs
Author: sbucaille
Pull request: yes

# What does this PR do?

Updates LightGlue model card for #36979 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

@stevhliu


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39407). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix bugs in pytorch example run_clm when streaming is enabled
Author: HRezaei
Pull request: yes

# What does this PR do?

Fixes #39285


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ArthurZucker
@sgugger



Comments:
Thanks 🤗 

My pleasure @ArthurZucker! 🤗

I suggested to make this functionality built-in in the `datasets` [here](https://github.com/huggingface/datasets/issues/2962#issuecomment-3077006825). If it's accepted and done, then most of this PR can be reverted in future versions.
Responses API (to be merged into #39155)
Author: gante
Pull request: yes

# What does this PR do?

To be merged into #39155 

### Testing

Works with the [OpenAI responses starter app](https://github.com/openai/openai-responses-starter-app/):
1. launch server as `transformers serve --force-model HuggingFaceTB/SmolLM3-3B`
2. launch app as `OPENAI_API_KEY="<KEY>" OPENAI_BASE_URL=http://localhost:8000/v1 npm run dev`

Test script:
```py
# Launch `transformers serve` in another terminal first
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="<KEY>")

response = client.responses.create(
    model="Qwen/Qwen2.5-0.5B-Instruct",
    instructions="You are a helpful assistant.",
    input="Hello!",
    stream=True,
    metadata={"foo": "bar"},
)

for event in response:
    print(event)
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39338). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

☝️ rebased w/completion server tests. Now has OAI output types. All tests passing

☝️ test script seems operational against this commit, i.e. responses API starts to be operational here

☝️ now with metadata support and kv caches
[autodocstring] add video and audio inputs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, I think these are quite common to be in the auto docstring


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39420). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: clap, clvp, gemma3n, glm4v, granite_speech, llava_next_video, llava_onevision, musicgen_melody, perception_lm, pop2piano, qwen2_5_omni, qwen2_5_vl, qwen2_audio, qwen2_vl, speech_encoder_decoder, speech_to_text
model_type = self._reverse_config_mapping[key.__name__] KeyError: 'Qwen2RMConfig'
Author: Lelege0
Pull request: no
Labels: name: bug

### System Info

File "/anaconda/envs/openrlhf/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 767, in __getitem__
    model_type = self._reverse_config_mapping[key.__name__]
KeyError: 'Qwen2RMConfig'


transformers version is 4.51.3

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

we use the lastest version of openrlhf、vllm、transformers

### Expected behavior

Can you merge QwenRMconfig into the lastest version of transformers


Comments:
Hi @Lelege0, can you paste some simple code that we can run to see the cause of this issue?

same problem, I commented the trust_remote_code=True temporarily. I'm using transformers==4.52.4

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Whisper chunking algorithm increases WER
Author: asusdisciple
Pull request: no
Labels: name: bug

name: Whisper

### System Info

So I made a few experiments with whisper and seamless m4tv2 on the fleurs (concatenated files to 5min samples) dataset. I used the batching functionality by setting `chunk_length_s` to 30s and as is turns out the WER increases by **20%** over all languages compared to long form transcription (sequentially going through each file). Do you have the same behaviour? Is this a bug or expected to happen because of the chunking? 20% seems to be far too much from my point of view.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Just use the default pipeline implementation of whisper on files which are a few minutes long. Its far worse when chunking is enabled.

### Expected behavior

I would expect the same transcription quality or maybe a few % less but 20% is far from that.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
lazy cache init
Author: ArthurZucker
Pull request: yes

# What does this PR do?
CB needs lazy cache init as well.
Making sure all layers are local, but casting to DTensor appropriately sounds a lot easier. 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38495). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@bot /style

Style fixes have been applied. [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/15395747840).