noamaanMulla-03/transformers-issues huggingface dataset:
Rework dependencies and extras + Remove outdated `templates` folder
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

The dependencies and extras have almost never been touched and are quite old.
We are suffering quite a bit from the following:
- duplicated dependencies (e.g. being still in extra etc even if they are main dependencies, such as tokenizers or tqdm, or an extra needing 2 extras that are redundant)
- dependencies upon non-used library (e.g. `ftfy`)
- dependencies that are already main dependencies of REQUIRED libs (i.e. `hf_xet` il only ever used as a dep of `huggingface_hub`, not directly in Transformers)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43536). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Can you run the docker file generation btw to make sure we test it


All right, dockers have been re-built and run on this PR, and all failed tests have been seen elsewhere and are due to the recent CI difficulties.
Will rebase to remove the change to dockerfile, then merge it
Change from v4 to v5c1: DeprecationWarning: builtin type SwigPyPacked/Object has no __module__ attribute
Author: jrp2014
Pull request: no

When I run a purest, the tests work as expected, after updating transformers to 5rc1 but I get a deprecation warning that is new.  This seems harmless, for now, but may mean something to someone!


```
<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
```

Comments:
This warning is actually being thrown when `sentencepiece` is imported! I don't think it's occurring anywhere else in the lib. It seems like a minor incompatibility between `sentencepiece` and recent Python versions. There's not a lot we can do about it!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
fix(converter): speed up `MistralConverter.extract_vocab_merges_from_model`
Author: tarekziade
Pull request: yes

# What does this PR do?

This test was taking 3 to 4mn on the CI  `test_voxtral_tokenizer_converts_from_tekken`

https://github.com/huggingface/transformers/blob/main/tests/models/auto/test_tokenization_auto.py#L213

Turns out we have a super slow function on large vocabularies (100k+ tokens)

This patch improves `MistralConverter.extract_vocab_merges_from_model`

Performance improvement:
- Before: O(n) linear search for each lookup, causing O(n²) behavior overall - 3/4mn
- After: O(1) dictionary lookup, reducing to O(n log n) for sorting - 1 second



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43557). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Benchmark] Disable GPU monitoring by default on AMD GPUs
Author: Abdennacer-Badaoui
Pull request: yes

GPU monitoring causes ~2-3x slowdown on AMD GPUs during benchmark runs. This PR automatically disables GPU monitoring when an AMD GPU is detected, with a warning message explaining the reason.
Users can still force-enable GPU monitoring with `--force-gpu-monitoring` if needed.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43534). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This will be closed if we merge this https://github.com/huggingface/transformers/pull/43552
[Model] Add PP-DocLayoutV3 Model Support
Author: zhang-prog
Pull request: yes
Labels: name: New model


Comments:
@ArthurZucker 
1. The model is not yet publicly released, but we aim to merge it into the transformers library as soon as possible.
2. I’ve adopted a modular approach and further optimized the code. 

PP-DocLayoutV3 model_doc and huggingface repo are comming soon. Could I please request a review?


@molbap  
Thank you for your effort! I’ve made changes based on your comments.
PTAL.

@molbap
Model overview and `PPDocLayoutV3ModelIntegrationTest` cannot be completed now, but they are coming soon.
PTAL.

@vasqu I have modified the code according to your suggestions. PTAL.

@molbap @vasqu I’ve updated the code based on your suggestions. Let’s take a look at the latest version!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43098). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: pp_doclayout_v3

This comment contains `run-slow`, running the specified jobs: 

models: ["models/pp_doclayout_v3"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21249367774)

✅ No failing test specific to this PR 🎉 !


@zhang-prog Are we waiting for the weights or will they be released after merge? Re slow integration tests

> @zhang-prog Are we waiting for the weights or will they be released after merge? Re slow integration tests

@vasqu We will get back to you after we have an internal discussion.

No worries, you can also write us on slack then :hugs: 

@molbap hi, Pablo, we have made three changes, and this should be the final update:

1. We overrode the `_preprocess` method, because we require `self.resize(..., antialias=False)` to approximate the behavior of `cv2.resize` in order to ensure model accuracy, but the `antialias` argument cannot be passed in the current implementation.
2. The `decoder_order_head` has been changed from a single `nn.Linear` layer to a stack of `nn.Linear` layers, with the depth defined by `config.decoder_layers`.
3. We added `polygon_points` to the post-processing stage, which are derived from the model’s output masks.

Ready for review, PTAL!

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43098&sha=cbbc37

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, pp_doclayout_v3

run-slow: pp_doclayout_v3

This comment contains `run-slow`, running the specified jobs: 

models: ["models/pp_doclayout_v3"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21473280178)

✅ No failing test specific to this PR 🎉 !


Merging, CI is on fire these days and it only touches the new model which passes all tests - thanks a lot @zhang-prog
[CI][AMD] Fix Pipeline CI 
Author: Abdennacer-Badaoui
Pull request: yes

This PR fixes a large portion of the failing pipeline tests on AMD. 



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43178). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: falcon_mamba, gemma3n, janus, lfm2, lfm2_moe, mamba, qwen2_5_omni

This comment contains `run-slow`, running the specified jobs: 

models: ["models/falcon_mamba", "models/gemma3n", "models/janus", "models/lfm2", "models/lfm2_moe", "models/mamba", "models/qwen2_5_omni"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20994413202)

✅ No failing test specific to this PR 🎉 !


Asked @Cyrilvallez to force merge, but it seems we are going to merge https://github.com/huggingface/transformers/pull/43256 before. You'll probably have to rebase afterwards.

I tried to simplify this PR by removing many unnecessary changes. For the safetensors check, using `None` works well, so I removed the try/except block. I also removed the Whisper-specific changes from the automatic-speech-recognition pipeline.

I updated the tapas tokenizer to always include` token_type_ids` in its default `model_input_names` so the table QA pipeline consistently emits the 7‑field tapas token type matrix that `TapasTokenizer.convert_logits_to_predictions` and the sequential inference path expect. 

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43178&sha=80778d

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma3n, qwen2_5_omni, tapas
model.dtype and model.qformer.dtype remain float32 when loading Blip2 model with dtype=torch.float16 or torch.bfloat16
Author: rebel-seinpark
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 5.0.0rc3
- Platform: Linux-6.8.0-59-generic-x86_64-with-glibc2.35
- Python version: 3.12.12
- Huggingface_hub version: 1.3.4
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.8.0+cpu (NA)
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@zucchini-nlp 

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import Blip2ForConditionalGeneration
import torch

dtype = torch.bfloat16  # or torch.float16

model = Blip2ForConditionalGeneration.from_pretrained(
    "Salesforce/blip2-opt-2.7b",
    dtype=dtype,
)

# These pass ✓
print("language_model.dtype:", model.language_model.dtype)  # bfloat16
print("vision_model.dtype:", model.vision_model.dtype)      # bfloat16

# These fail ✗
print("qformer.dtype:", model.qformer.dtype)  # float32 (expected: bfloat16)
print("model.dtype:", model.dtype)            # float32 (expected: bfloat16)
```

### Expected behavior

Hello,

I noticed an inconsistency when loading `Blip2ForConditionalGeneration` with a specific `dtype` argument.

- Problem
When loading the Blip2 model using `from_pretrained` with `dtype=torch.float16` or `dtype=torch.bfloat16`, the `language_model` and `vision_model` submodules are correctly cast to the target dtype. However, both `model.qformer.dtype` and `model.dtype` still return torch.float32.

- Expected Behavior
`model.qformer.dtype` should return the target dtype (bfloat16 or float16)
`model.dtype` should return the same dtype as its submodules when loaded with a specific dtype argument

Comments:
@rebel-seinpark hey, in blip models we need to keep Qformer in full precision to get a reasonable performance. Therefore `qformer` is added in models' `keep_in_fp32_modules` which I believe matches the model's behavior in "Salesforce" repo

Closing as not-a-bug!
Fix mistral checkpoint loading in `utils/fetch_hub_objects_for_ci.py`: avoid too many requests and/or timeout
Author: ydshieh
Pull request: yes

# What does this PR do?

Just reuse the local files as much as possible

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43584). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Increase default ReadTimeout in tests
Author: Wauplin
Pull request: yes

# What does this PR do?


This PR sets `HF_HUB_DOWNLOAD_TIMEOUT=60` as environment variable when running tests. I've also added [`pytest-env`](https://github.com/pytest-dev/pytest-env) as a dev dependency to do that.

I've also reverted https://github.com/huggingface/transformers/pull/43560 which is not needed anymore.


**Note:** there might still be some `ReadTimeout`s in CI when running scripts that are not run with `pytest`. I preferred not to touch that part as I don't have a broad understanding on what the CI does but any failing workflow (due to timeouts) should be fixable with a `HF_HUB_DOWNLOAD_TIMEOUT=60` env variable.

**Note 2:** this PR requires [huggingface_hub v1.3.5](https://pypi.org/project/huggingface-hub/1.3.5/) to work. I don't think we should pin this version as minimal dependency though, since it's just to fix some tests. In theory the CI will always pull the latest version so it should be fine.



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43586). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add XPU support to the tests for solar_open
Author: YangKai0616
Pull request: yes

# What does this PR do?

Hi @ydshieh , please help review this PR, thanks!

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: solar_open

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43579). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix failing tests due to no attribute `pad_token_id`
Author: Sai-Suraj-27
Pull request: yes
Labels: name: for patch

# What does this PR do?

Fixes these failing [PhiIntegrationTest](https://github.com/huggingface/transformers/actions/runs/21272911412/job/61226967021#step:14:949), [Glm46VIntegrationTest](https://github.com/huggingface/transformers/actions/runs/21199210283/job/60982515419#step:14:1128), [Qwen3VLMoeIntegrationTest](https://github.com/huggingface/transformers/actions/runs/21272911412/job/61226967735#step:14:950) & [Qwen2_5OmniModelIntegrationTest](https://github.com/huggingface/transformers/actions/runs/21199210283/job/60982513946#step:14:971) tests.

<img width="2233" height="410" alt="image" src="https://github.com/user-attachments/assets/67330ceb-c31c-4c62-9adc-50b06eda4b44" />





## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@zucchini-nlp (Continuation to https://github.com/huggingface/transformers/pull/41541+ https://github.com/huggingface/transformers/pull/43398) @Rocketknight1

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm4v, glm_image, glm_ocr, phi, qwen2_5_omni, qwen3_vl_moe

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43453). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Regex post processing in loading
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

The change introduced in https://github.com/huggingface/transformers/pull/43261 with `__setattr__` is quite dangerous, as `source_patterns` and `target_patterns` live together and cannot be unentangled from one another if there is a capturing group. 
Thus, changing one with `__setattr__` without changing the other, or simply changing the wrong one first can lead to very unexpected behavior and wrong pattern matching/replacing.
For this reason, it's better to reinstantiate an object directly in `peft` instead of switching the targets/sources.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43585). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Qwen3 omni - fix get video features
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/43540


Comments:
run-slow: qwen3_omni_moe

This comment contains `run-slow`, running the specified jobs: 

models: ["models/qwen3_omni_moe"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43588). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21476603995)

✅ No failing test specific to this PR 🎉 !


**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3_omni_moe
ValueError when processing video inputs in Qwen3OmniMoe
Author: BBiering
Pull request: no
Labels: name: bug

### System Info

## System Info

- `transformers` version: 5.0.1.dev0 (also affects v4.57.0 through v5.0.0)
- Platform: Linux-5.10.0-37-cloud-amd64-x86_64-with-glibc2.31
- Python version: 3.10.17
- PyTorch version: 2.10.0+cu128
- CUDA available: Yes
- CUDA version: 12.8
- GPU: NVIDIA A100-SXM4-80GB

### Who can help?

@zucchini-nlp, @BakerBunker 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

### Code to reproduce

```python
import torch
from transformers import Qwen3OmniMoeForConditionalGeneration, Qwen3OmniMoeProcessor
from qwen_omni_utils import process_mm_info

MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Thinking"

model = Qwen3OmniMoeForConditionalGeneration.from_pretrained(
    MODEL_PATH,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

processor = Qwen3OmniMoeProcessor.from_pretrained(MODEL_PATH)

conversation = [
    {
        "role": "user",
        "content": [
            {"type": "video", "video": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen2.5-Omni/draw.mp4"},
        ],
    },
]

text = processor.apply_chat_template(conversation, add_generation_prompt=True, tokenize=False)
audios, images, videos = process_mm_info(conversation, use_audio_in_video=True)

inputs = processor(
    text=text,
    videos=videos,
    return_tensors="pt",
    padding=True,
    use_audio_in_video=True
)

inputs = inputs.to(model.device).to(model.dtype)

# This will fail with ValueError
outputs = model.generate(**inputs, max_new_tokens=100)
```

### Error traceback

```
Traceback (most recent call last):
  File "/path/to/serve_qwen3.py", line 73, in <module>
    outputs = model.generate(**inputs, max_new_tokens=100)
  File "/path/to/transformers/models/qwen3_omni_moe/modeling_qwen3_omni_moe.py", line 4087, in generate
    thinker_result = self.thinker.generate(input_ids=input_ids, **thinker_kwargs)
  File "/path/to/transformers/generation/utils.py", line 2669, in generate
    result = decoding_method(...)
  File "/path/to/transformers/generation/utils.py", line 2864, in _sample
    outputs = self._prefill(input_ids, generation_config, model_kwargs)
  File "/path/to/transformers/generation/utils.py", line 3853, in _prefill
    return self(**model_inputs, return_dict=True)
  File "/path/to/transformers/models/qwen3_omni_moe/modeling_qwen3_omni_moe.py", line 2218, in forward
    video_embeds, video_embeds_multiscale = self.get_video_features(
ValueError: too many values to unpack (expected 2)
```

### Expected behavior

The model should process video inputs successfully without errors, similar to how it handles image inputs.

Comments:
Could have been affected by the recent refactor of `get_xxx_features` utilities. Taking a look 👁️
simplify extra tokens logic in base
Author: itazap
Pull request: yes

simplify extra tokens logic in base

some light refactoring, hopefully a bit more readable


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43230). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[run-sl](run-slow: florence2, nllb, tapas)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43230&sha=b0a156

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: florence2, nllb, qwen2_5_omni, qwen3_omni_moe, tapas

Force merging as the errors are overall the same as on main

Seeing some processor errors after this PR including `Qwen3VLProcessorTest::test_apply_chat_template_video_frame_sampling` and `Glm46VProcessorTest::test_apply_chat_template_video_frame_sampling`, plus possibly others. I think the CI would have caught them but it was hard to tell in the cloud of other errors!

Pushed fixes for Glm46 and Glm4 in #43537
Fix: Add missing pad_token_id to StableLmConfig
Author: tobyliu2004
Pull request: yes

## What does this PR do?

Fixes #43572

Adds the missing `pad_token_id` parameter to `StableLmConfig` to resolve the `AttributeError` when creating StableLM models from config.

## Changes made:
- Added `pad_token_id` parameter to `__init__` method (defaults to `None`)
- Added `self.pad_token_id = pad_token_id` assignment
- Updated docstring to document the new parameter

## Testing:
Verified that `AutoModelForCausalLM.from_config(StableLmConfig())` now works without errors.

## Before submitting
- [x] This PR fixes a bug
- [x] Code follows existing patterns (matches bos_token_id and eos_token_id)
- [x] Documentation updated in docstring

cc @ArthurZucker @Cyrilvallez @itazap

Comments:
- Do we also need a test to cover such usage example?
- Do we need to update existing configs like https://huggingface.co/stabilityai/stablelm-3b-4e1t/blob/main/config.json? likely not

Thanks for reviewing!
You're right, I think a test would be a good idea, I've just added an automated test in test_modeling_stablelm.py that verifies StableLmConfig has the pad_token_id attribute and that models can be instantiated from config without errors. As for the existing configs, I agree that since pad_token_id is defaulted to None that they should work without updates.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: stablelm

cc @zucchini-nlp

Thanks for the PR @tobyliu2004 ! I'd prefer to close it in favor of https://github.com/huggingface/transformers/pull/43592 which is more complete

Thanks for reviewing! Makes sense to fix all models at once rather than one by one. Appreciate the feedback!
Operation Green CI II
Author: Rocketknight1
Pull request: yes

We get flaky generation tests in the CI a lot. A very common cause is the `has_similar_generate_outputs` helper function - this function is called by many tests, but it has very narrow tolerances by default in the CI (`atol=1e-5, rtol=1e-5`). Example here: https://app.circleci.com/jobs/github/huggingface/transformers/2127052

Especially for `bfloat16` models, this seems very likely to cause unnecessary errors, so I expanded the tolerances to `1e-3`.

Example failing run [here](https://app.circleci.com/pipelines/github/huggingface/transformers/161587/workflows/f87424b2-d037-4309-a864-08a8d62ae201/jobs/2127052)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43537). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, cpmant, plbart, qwen3_omni_moe, seamless_m4t