noamaanMulla-03/transformers-issues huggingface dataset:
AutoModel cant load Qwen/Qwen2.5-0mni-7B
Author: liwenju0
Pull request: no
Labels: name: WIP

name: bug

### System Info

```
from transformers import AutoModel
model = AutoModel.from pretrained("Qwen/Qwen2.5-0mni-7B",torch dtype="auto"trust remote code=True)
```
will raise error：

![Image](https://github.com/user-attachments/assets/419c274f-bfc6-491a-8e77-55f3c8953889)

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoModel
model = AutoModel.from pretrained("Qwen/Qwen2.5-0mni-7B",torch dtype="auto"trust remote code=True)
```

### Expected behavior

load sucessfully

Comments:
@liwenju0 
My understanding is AutoModel is designed for simple single-modality models, while Qwen is multi-modal and could make it hard for AutoModal to figure out how to assemble encoders/decoders.

You can workaround using the customized APIs: https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/qwen2_5_omni.md



@liwenju0 for multimodal models we didn't have a base model in most cases because the models are usually a composition of LM and encoders. Thus the straightforward way was to compose a generative model only. I am currently adding a base model for multimodals in https://github.com/huggingface/transformers/pull/37033 but it might not be usable with official checkopints

If you wanted to obtain last hidden state from Qwen-Omni, you can still load the `ConditionalGeneration` and get model outputs with `out.hidden_states[-1]`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

not stale
[fp8] fix scales param name
Author: MekkCyber
Pull request: yes

# What does this PR do?



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42434). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I have verified that this resolves the FP8Linear issue in #42028.
Fix an edge case for `get_encoder()`
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, fixes https://github.com/huggingface/transformers/pull/42156#issuecomment-3553776255 and adjusts the test case slightly

I didn't not change the setter, we don't call `set_encoder` unless we're sure there is an encoder right?


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42295). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

CI is not feeling good today

My last try to pass the flaky test 🤞🏻
Fix DataParallel dtype access in smolvlm
Author: Abdennacer-Badaoui
Pull request: yes

Handle StopIteration when accessing self.dtype in DataParallel mode by
falling back to input tensor dtype or torch.float32.

Fixes `test_multi_gpu_data_parallel_forward`.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: smolvlm

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42446). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Cheap CLI
Author: LysandreJik
Pull request: yes

Better guard the CLI commands, so that `transformers add-new-model-like` works without the serving dependencies (bug) and so that `transformers chat` does not import `torch` unnecessarily (improvement)

The `transformers run` (previously `transformers-cli run`) is an artefact of the past, was not documented nor tested,
and isn't part of any public documentation. Removing it for now and writing about it in the migration guide.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42447). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Benchmark simplification
Author: remi-or
Pull request: yes
Labels: name: run-benchmark

This PR does two things:
- in the benchmark_v2 suite, merge the two `time_generate` function (with and without CB) into one
- rename the the attributes of `RequestState` so they make more sense

This will make the code more readable and concise. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42408). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Future annotations break FastAPI
Author: LysandreJik
Pull request: yes

Unfortunately, `from __future__ import annotations` breaks pydantic/FastAPI -> everything results in "422: unprocessable entity".

The following is therefore not possible: `"PreTrainedTokenizerFast" | None`, reverting to an `Optional`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42450). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
How to average sub-words embeddings to obtain word embeddings?
Author: speedcell4
Pull request: no
Labels: name: wontfix

Hi~

How to average sub-words embeddings to obtain word embeddings?
I only want word-level embedding instead of sub-word-level, how can I get them?

Is there any tokenizer that provides a method that can output the index/mask of sub-words or something?

Comments:
You may use the word as the input and make the sentence embedding as the word embedding.
for example, input is 
"puppeteer"
tokens as
'[CLS]', 'puppet', '##eer', '[SEP]'
and then get embedding of this tokens list output.

I have similar usage as well, I did a simple experiment, and observe that the subword embedding [subword1, subword2, subword3...] when input a whole sentence,
 the cosine similarity of [subword1,subword2],[subword1,subword3]... tends to above 90%.
 So that sum and average subwords' embedding doesn't change much.
Btw, I tested this with Roberta models, and I observe quite different result for Bert models.

Take a look at how bert-sense does it :)
https://github.com/uhh-lt/bert-sense/blob/bfecb3c0e677d36ccfab4e2131ef9183995efaef/BERT_Model.py#L342

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Excuse me did someone solve it ?

@mathshangw I turn to use the `TransformerWordEmbeddings` of `flair` library to handle this. Here is an [example](https://github.com/flairNLP/flair/blob/master/examples/ner/run_ner.py#L119).
examples(android): add FastImageProcessor to avoid ImageProcessor ove…
Author: CodersAcademy006
Pull request: yes

This PR fixes a slowdown we hit when using TensorFlow Lite Task’s `ImageProcessor` on full-HD FP32 images (for example, **1920×1080×3**). On large images the ImageProcessor pipeline allocates temporary objects, performs multiple buffer copies, and runs per-pixel work in Java. On our test device that added roughly **150–180 ms** of overhead per frame, pushing total frame time to about **200 ms**, even though the actual model/QNN inference was only **~90 ms**.

To address this I added a small, focused helper called **FastImageProcessor** that skips ImageProcessor for big FP32 frames and does lightweight, reusable preprocessing/postprocessing instead:

- Allocate a direct, native-order `ByteBuffer` / `FloatBuffer` once at construction and reuse it every frame.
- Read pixels using `bitmap.getPixels(IntArray)` once per frame, then convert the IntArray → normalized floats in a tight loop (RGB order, normalized to `[0,1]`).
- Postprocess model output by converting floats → an `IntArray` and calling `bitmap.setPixels(...)` once.
- Reuse buffers and a single output `Bitmap` to avoid per-frame allocations and GC pressure.

In practice this reduces preprocess + postprocess time substantially (we see it drop into the tens of ms on the tested hardware), so the pipeline becomes model-bound again instead of CPU-bound.

### Files added
- `FastImageProcessor.kt` — the small reusable helper class for FP32 pipelines  
- `README.md` — short usage notes and testing instructions

### How to test
Replace the current ImageProcessor usage with `FastImageProcessor` (create one instance, reuse per frame), run your FP32 model (e.g., a super-resolution test), and measure: preprocess → model → postprocess. You should see a large reduction in CPU time and allocations.

**Fixes:** #105217


Comments:
Got tensorflow in this, my bad.
untie?
Author: molbap
Pull request: yes

# What does this PR do?

Slow tests are failing for a couple VLMs since tied weights change, attempting to fix


Comments:
run-slow: internvl

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: internvl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/internvl"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42237). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19427972178)

✅ No failing test specific to this PR 🎉 !


For info, was to confirm current failures are linked to weight tying- we should just need to change the config key tie_word_embeddings

Superseded by #42385
[WIP] attempt to fix ooms in tests
Author: molbap
Pull request: yes

# What does this PR do?

 as per title, focused on qwen series

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3_omni_moe, qwen3_vl, qwen3_vl_moe

run-slow: qwen3_omni_moe, qwen3_vl, qwen3_vl_moe

This comment contains `run-slow`, running the specified jobs: 

models: ["models/qwen3_omni_moe", "models/qwen3_vl", "models/qwen3_vl_moe"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42424). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19707459337)

✅ No failing test specific to this PR 🎉 !
[cleanup] Remove deprecated load config from file
Author: Wauplin
Pull request: yes

This PR removes the possibility to load a Config using a file URL directly. This is a deprecated feature expected to be removed in transformers v5:

```py
    warnings.warn(
        f"Using `from_pretrained` with the url of a file (here {url}) is deprecated and won't be possible anymore in"
        " v5 of Transformers. You should host your file on the Hub (hf.co) instead and use the repository ID. Note"
        " that this is not compatible with the caching system (your file will be downloaded at each execution) or"
        " multiple processes (each process will download the file in a different temporary file).",
        FutureWarning,
    )
```

I've also removed the legacy test `test_legacy_load_from_one_file` testing this feature.

---

In addition, this allowed me to cleanup the `is_remote_url` and `download_url` helpers and to entirely avoid importing `http_get` from `huggingface_hub` (which is to be considered a "semi-private" helper. I have refactored a few tests using it to default to `hf_hub_download` instead.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: metaclip_2

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42383). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This PR is ready for review. CI is failing on 

```
FAILED tests/models/kosmos2/test_modeling_kosmos2.py::Kosmos2ModelTest::test_assisted_decoding_matches_greedy_search_1_same - AssertionError: False is not true
```

which looks unrelated to the PR. I already retriggered the CI once but it didn't help.
[`FA`] Cleanup loading logic
Author: vasqu
Pull request: yes

Multitude of miscellaneous fixes in regards to how we load flash attention
- Less important but it enables also flash attention via config setting, i.e. you can freely set `config._attn_implementation` and it will properly switch
- Fallback to kernels was broken and activated when other kernels were requested like the metals kernel
- Switch to new kernels naming for fa2 (even if it's redirected)
- CB specific fixes
  - FA3 is now included 
  - Centralized loading logic, should make it easier to directly include things for base fa and CB at the same time

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41427). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker updated the logic a bit to include CB now as well. The loaded attention is now properly dependent on the config like in the base fa version + we can now use fa3 as well for CB.

Seems like the `implementation` kwarg was only used in `flash_paged`, the other implementations don't have it:
- eager https://github.com/huggingface/transformers/blob/a99b1be3c73173509f71c261834d93d3d5e30ef1/src/transformers/integrations/eager_paged.py#L21-L29
- sdpa https://github.com/huggingface/transformers/blob/a99b1be3c73173509f71c261834d93d3d5e30ef1/src/transformers/integrations/sdpa_paged.py#L20-L29
- flash removed in this PR https://github.com/huggingface/transformers/blob/1ece1936f9d8293abfbbd6006d84980f344e504d/src/transformers/integrations/flash_paged.py#L9-L21
So we should be good @ArthurZucker?
tiny fix for deepseekocr support [vllm]
Author: molbap
Pull request: yes

# What does this PR do?

As per title.

Comments:
run-slow: deepseek_v2

This comment contains `run-slow`, running the specified jobs: 

models: ["models/deepseek_v2"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42423). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19706646198)

✅ No failing test specific to this PR 🎉 !


Removed modeling code, config is enough for vLLM side. Merging!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: deepseek_v2
Transformers serve -> list all generative models from the cache 
Author: LysandreJik
Pull request: yes

Instead of relying on an inadequate list, look for generative models within the cache. Generative models are identified by taking checking in their config whether the architecture defined is in the `MODEL_FOR_CAUSAL_LM_MAPPING_NAMES` mapping.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42146). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks Lucain!
🚨 [v5][PEFT] Bump min version requirement of PEFT to  0.18.0
Author: BenjaminBossan
Pull request: yes

# What does this PR do?

PEFT is an optional dependency of transformers with a min version of 0.5.0. However, starting with transformers v5, older PEFT versions will not work anymore (see #41406). The minimum PEFT version will be 0.18.0.

This PR updates the PEFT integration to require PEFT 0.18.0. This allows us to eliminate some obsolete checks and tests that were required for backwards compatibility.

Note: 

- PEFT 0.18.0 is not yet released, so **don't merge** this PR yet. However, we will release it once we have tested it with the transformers v5 release candidate, before the transformers v5 released.
- If this commit is not included in the transformers v5 release, users who want to use PEFT will get an `ImportError` without indication what's wrong. Therefore, I would highly recommend to merge this before the v5 release.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests? <= Tests updated

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41889). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This PR should be included in the v5 release. Not sure who can best review it, ping @ydshieh. The failing tests seem to be unrelated to this PR.

> Happy to merge, wondering why we don't update the `dependency_table` and `setup.py` ? 🤗

PEFT is an optional dependency, as such there is no entry there.

> Happy to merge, wondering why we don't update the `dependency_table` and `setup.py` ? 🤗

You can still put it in those files. Most of them are optional dependency, even `torch` .

The benefits is that an user could do something like `pip install transformers[dev]` or even `pip install transformers[peft]` (if you want such entries).

But it's just minor, up to you.

@ydshieh I didn't know that, I added PEFT to that list. The failing CI looks unrelated.

@ArthurZucker I updated the PR, could you please give a final review?

Merged the latest main, the failing test looks unrelated to this PR.
Correctly return finish reason length when finished
Author: LysandreJik
Pull request: yes

This PR updates `transfsormers serve` so that it accurately returns the stop reason -> instead of only returning "stop" (stop token), it now accurately returns "length" as well, therefore indicating when the max new token limit has been reached.

Transformers chat is updated to check on this reason, and to offer to continue:

```
Generation stopped after reaching the token limit.

Continue generating? (y/N):
```

This results in chats like the following:
<img width="1256" height="626" alt="image" src="https://github.com/user-attachments/assets/863a0828-6f2d-43b0-a6d6-24cf47d6d070" />

Implemented for both continuous batching and generate.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42157). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix the "test_offline" test
Author: LysandreJik
Pull request: yes

Unfortunately it turned out to be a bit harder to patch than initially thought.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42458). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
LlamaRMSNorm and equivalent module implementations using torch ops
Author: mstojkovicTT
Pull request: no
Labels: name: Feature request

### Feature request

I would like to change `LlamaRMSNorm` and other similar modules implementation to use `nn.function.rms_norm` instead of python math.

### Motivation

I am curious as to why it was decided to use python math for `RMSNorm` modules, instead of just using torch op for it (it has same interface and potential benefits for perf).

### Your contribution

I would file a PR, explained in feature request.

Comments:
I believe this was because we needed to maintain compatibility with older versions of Torch that didn't have the op! I think at this point we can probably start updating though, but it might introduce small numerical shifts. If you make a PR, can you check to see what the differences look like, especially in `bfloat16`?

@Rocketknight1 PR is up, I have also included the test for `bfloat16` scenario.
`transformers chat` launched without base_url has a direct tie to localhost:8000
Author: LysandreJik
Pull request: yes

To significantly simplify the `transformers chat` launch, specifying a base URL is no-longer necessary as it will default to `http://localhost:8000`.

It will perform a health-check to ensure the server is running, and error out otherwise.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42463). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
update with more recent tts models
Author: Deep-unlearning
Pull request: yes

# What does this PR do?

<!-- Remove if not applicable -->

Update TTS docs with more recents TTS models released on transformers

waiting for this [PR](https://github.com/huggingface/transformers/pull/42326) to be merged which fixes TTS pipeline


## Who can review?

@eustlb @ebezzam 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42328). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.