noamaanMulla-03/transformers-issues huggingface dataset:
Add supported kwargs to fixed_cross_entropy
Author: Rocketknight1
Pull request: yes

This PR allows some specific kwargs to be passed through to `fixed_cross_entropy`. Fixes #43240

cc @arthurzucker because I think you wrote the original?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43254). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43254&sha=f05ae7

After this change I'd then drop `**kwargs` arg, since it silently ignores them. This is not good for the user - it either should do something with kwargs and assert if the use is unexpected or it shouldn't absorb args that shouldn't be there.

There is another version of this PR here https://github.com/huggingface/transformers/pull/43251

Hey @stas00, good to hear from you again! I suspect the `**kwargs` there is because the function is often called with `**kwargs` in other contexts, so doing it like that lets it catch any named kwargs that match while ignoring ones that don't. That makes me a little nervous about updating it, because I think those kwargs are flowing through from a lot of different possible sources.

I understand your reasoning, Matt

Perhaps with v5 transformers could fix the silent `kwargs` black holes in its code - `kwargs` is designed for passing forward arguments to the next function in the stack. In this situation the function isn't passing `kwargs` forward therefore it shouldn't have `kwargs` in its API.
[Qwen2VL] Fix missing min_pixels/max_pixels attributes in fast image processor
Author: tomaszcichy98
Pull request: yes

## What does this PR do?

Fixes a bug where `Qwen2VLImageProcessorFast` doesn't set `min_pixels` and `max_pixels` instance attributes, breaking compatibility with code that expects these attributes.

## The Problem

The slow processor (`Qwen2VLImageProcessor`) sets `self.min_pixels` and `self.max_pixels` as instance attributes. The fast processor converts these to `size["shortest_edge"]` and `size["longest_edge"]` but never sets the original attributes.

This causes vLLM (and potentially other libraries) to fail when loading Qwen2-VL/Qwen2.5-Omni models:

```
File "transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 92, in smart_resize
    if h_bar * w_bar > max_pixels:
TypeError: '>' not supported between instances of 'int' and 'NoneType'
```

### Why config values don't help

Even if `preprocessor_config.json` contains `min_pixels` and `max_pixels` values, this doesn't fix the issue because:

1. `use_fast: false` in the config is **ignored** - transformers 5.0 always auto-upgrades to the fast processor
2. The fast processor receives these values as kwargs but only stores them in `self.size` dict, not as instance attributes
3. Code accessing `processor.min_pixels` gets `None` instead of the config value

## The Fix

Set `min_pixels` and `max_pixels` attributes after `super().__init__()` to maintain backward compatibility:

```python
super().__init__(size=size, **kwargs)
self.min_pixels = self.size.get("shortest_edge")
self.max_pixels = self.size.get("longest_edge")
```

## Verification

```python
from transformers import AutoProcessor

# Before fix
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")
print(processor.image_processor.min_pixels)  # None
print(processor.image_processor.max_pixels)  # None

# After fix
print(processor.image_processor.min_pixels)  # 3136
print(processor.image_processor.max_pixels)  # 12845056
```

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm_image, qwen2_vl, video_llama_3

cc @zucchini-nlp maybe?

fyi @hmellor ,can be added in https://github.com/vllm-project/vllm/pull/30566 if it's indeed not working

> It was removed as part of v5 release in favor of `self.size`. I believe the error is raised from within vLLM code?
> 
> We can ask vLLM team to update the code on their side, not sure if they already bumped to v5

Yeah it is coming from vLLM @zucchini-nlp 

This is already fixed in vLLM https://github.com/vllm-project/vllm/pull/33208
Fix `make_batched_video` with 5D arrays
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, it was not working when input was a batched 5D array

Fixes https://github.com/huggingface/transformers/issues/43450

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43486). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Video processors return incorrect shape when input is batched
Author: shaform
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.57.6
- Platform: Linux-5.10.0-37-cloud-amd64-x86_64-with-glibc2.31
- Python version: 3.10.19
- Huggingface_hub version: 0.36.0
- Safetensors version: 0.7.0
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.10.0+cu128 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: No
- GPU type: Tesla T4

### Who can help?

@zucchini-nlp 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

### Install Packages
```bash
pip install pillow==12.1.0 torch==2.10.0 torchvision==0.25.0 transformers==4.57.6 torchcodec==0.10.0
```

### Run the following script and observe that the output shape is unexpected
```
import torch
from transformers import AutoVideoProcessor

processor = AutoVideoProcessor.from_pretrained("qubvel-hf/vjepa2-vitl-fpc16-256-ssv2")

# input: B x T x C x H x W
pixel_values_videos = processor(
    torch.zeros(4, 5, 3, 100, 100), return_tensors="pt", do_sample_frames=False
).pixel_values_videos

# output: 1 x 1 x B x T x C x H x W
assert pixel_values_videos.shape == (
    4,
    5,
    3,
    256,
    256,
), "Shape should be B x T x C x H x W, not {}".format(pixel_values_videos.shape)
```

### Expected behavior

If batched videos are given to the video processor, it should be able to detect that the input is already batched, and avoid adding some extra dimensions.

### Additional information
- #39600 

Comments:
Fixes configuration default values
Author: zucchini-nlp
Pull request: yes
Labels: name: for patch

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/43525
Fixes https://github.com/huggingface/transformers/issues/43572

Adds missing `pad_token_id` and `tie_word_embeddings` to config classes with their defaults

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43592). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Adding fix to `tie_word_embeddings`, don't merge!

run-slow: cohere2, deformable_detr, emu3, exaone4, falcon_mamba, fast_vlm, flava, florence2, glm46v, got_ocr2, gpt_bigcode, gpt_neox, gptj, internvl, jetmoe, mamba

This comment contains `run-slow`, running the specified jobs: 

models: ["models/cohere2", "models/deformable_detr", "models/emu3", "models/exaone4", "models/falcon_mamba", "models/fast_vlm", "models/flava", "models/florence2", "models/glm46v", "models/got_ocr2", "models/gpt_bigcode", "models/gpt_neox", "models/gptj", "models/internvl", "models/jetmoe", "models/mamba"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21482710104)

✅ No failing test specific to this PR 🎉 !


@bot /style

Deformable detr is flaky now, apparently related to the random order of tests 😢  Not reproducible locally if I run a single testcase

@bot /repo

Repo. Consistency bot fixed some files and pushed the changes.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: cohere2, cohere2_vision, deepseek_vl, deepseek_vl_hybrid, deformable_detr, emu3, exaone4, falcon_mamba, fast_vlm, flava, florence2, glm46v, got_ocr2, gpt_bigcode, gpt_neox, gptj

run-slow: llava_onevision, llava_next_video

This comment contains `run-slow`, running the specified jobs: 

models: ["models/llava_next_video", "models/llava_onevision"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21512999444)

✅ No failing test specific to this PR 🎉 !
missing pad_token_idx in StableLmConfig after 5.0 update
Author: mario-aws
Pull request: no
Labels: name: bug

### System Info

transformers 5.0.0
trainium 2
python 3.10
pytorch 2.9

### Who can help?

@ArthurZucker @Cyrilvallez @itazap 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

1. from transformers import StableLmConfig, AutoModelForCausalLM
2. AutoModelForCausalLM.from_config(StableLmConfig())

Error:

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/shared/mmkrell/vscode/moduscope_ws/TorchNeuronEager/.venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 236, in from_config
    return model_class._from_config(config, **kwargs)
  File "/shared/mmkrell/vscode/moduscope_ws/TorchNeuronEager/.venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1508, in _from_config
    model = cls(config, **kwargs)
  File "/shared/mmkrell/vscode/moduscope_ws/TorchNeuronEager/.venv/lib/python3.10/site-packages/transformers/models/stablelm/modeling_stablelm.py", line 702, in __init__
    self.model = StableLmModel(config)
  File "/shared/mmkrell/vscode/moduscope_ws/TorchNeuronEager/.venv/lib/python3.10/site-packages/transformers/models/stablelm/modeling_stablelm.py", line 464, in __init__
    self.padding_idx = config.pad_token_id
  File "/shared/mmkrell/vscode/moduscope_ws/TorchNeuronEager/.venv/lib/python3.10/site-packages/transformers/configuration_utils.py", line 164, in __getattribute__
    return super().__getattribute__(key)
AttributeError: 'StableLmConfig' object has no attribute 'pad_token_id'. Did you mean: 'bos_token_id'?
```

### Expected behavior

Normal model creation.

Comments:
I can reproduce this and have a fix ready. I've added the missing 'pad_token_id" parameter to 'StableLmConfig' following the same pattern as 'bos_token_id' and 'eos_token_id'. submitting a PR now.

PR: #43573

Do we know, why it was not an issue before 5.0?

After looking at the git history, line 464 self.padding_idx = config.pad_token_id has existed since StableLM was first added in Feb 2024, but pad_token_id was never defined in StableLmConfig. The bug went unnoticed because most users load pretrained models via .from_pretrained() which loads pad_token_id from the saved config.json file. The bug only surfaces when instantiating a fresh config directly with StableLmConfig(). The fix adds the missing attribute following the same pattern as bos_token_id and eos_token_id.

cc @Rocketknight1  as you were planning on a PR to cover all models (ref https://github.com/huggingface/transformers/pull/43539). I prefer to fix all at once than opening separate PR for each model
AttributeError: 'Llama4Config' object has no attribute 'pad_token_id'
Author: xin3he
Pull request: no
Labels: name: bug

### System Info

transformers == 5.0.0

### Who can help?

@zucchini-nlp 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

To reproduce:
```python
import transformers
model_name = "/dataset/Llama-4-Scout-17B-16E-Instruct"
config = transformers.Llama4Config.from_pretrained(model_name)
config.vision_config.num_hidden_layers = 1  # Reduce layers for testing
config.text_config.num_hidden_layers = 1
model = transformers.Llama4ForConditionalGeneration(config)
```

```log
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/xinhe/miniforge3/lib/python3.12/site-packages/transformers/models/llama4/modeling_llama4.py", line 1190, in __init__
    self.pad_token_id = self.config.pad_token_id if self.config.pad_token_id is not None else -1
                                                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xinhe/miniforge3/lib/python3.12/site-packages/transformers/configuration_utils.py", line 164, in __getattribute__
    return super().__getattribute__(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Llama4Config' object has no attribute 'pad_token_id'
```

### Expected behavior

The demo code should work well.

Comments:
@Rocketknight1  is on it, will be fixed today!

PR open at #43539
[MoE] Use int input for histc on CUDA to support deterministic algorithms
Author: YangKai0616
Pull request: yes

# What does this PR do?

`torch.histc` with deterministic algorithms enabled behaves differently across devices:

**CPU**: only supports float input
**CUDA**: only supports int input

This PR updates `grouped_mm_experts_forward` to use the appropriate dtype based on device type, enabling compatibility with `torch.use_deterministic_algorithms(True)` on CUDA.

Simple reproduction script:
```
import torch

def test_histc_deterministic(device: str):
    print(f"\n=== Testing on {device.upper()} ===")
    
    num_experts = 8
    expert_ids = torch.randint(0, num_experts, (100,), device=device)
    
    torch.use_deterministic_algorithms(True)
    
    # Test float input
    try:
        result = torch.histc(expert_ids.float(), bins=num_experts, min=0, max=num_experts - 1)
        print(f"✅ float input works with deterministic mode")
    except RuntimeError as e:
        print(f"❌ float input fails: {e}")
    
    # Test int input
    try:
        result = torch.histc(expert_ids, bins=num_experts, min=0, max=num_experts - 1)
        print(f"✅ int input works with deterministic mode")
    except (RuntimeError, NotImplementedError) as e:
        print(f"❌ int input fails: {e}")
    
    torch.use_deterministic_algorithms(False)

if __name__ == "__main__":
    print(f"PyTorch version: {torch.__version__}")
    
    test_histc_deterministic("cpu")
    
    if torch.cuda.is_available():
        test_histc_deterministic("cuda")
```

Hi @vasqu , please help review this PR, thanks!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43583). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Separate renaming from merging in Mixtral conversion mapping
Author: hmellor
Pull request: yes

This allows the Transformers modeling backend for vLLM to load Mixtral with Transformers v5 because it currently (once https://github.com/vllm-project/vllm/pull/31545 is merged) only supports `WeightRenaming` but not `WeightConverter`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43071). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43071&sha=8e700a

Humm, in theory should not be too much of a problem, but tests complain here (`test_reverse_loading_mapping` failing is I believe due to the test itself that checks source patterns, so expected to fail without combining both Renaming and Conversion after this change), but others seem to indicate that conversion may not happen as expected.
Do you want to dive in it @hmellor? Otherwise I can do it, let me know!

Yeah I noticed the test failed but didn't know enough about the new system to figure out why. 

I won't be able to look at it for a few weeks so if you can find time that'd be awesome

Superseded by https://github.com/huggingface/transformers/pull/43071
Add GIT support to visual-question-answering pipeline
Author: DowellHd
Pull request: yes

# What does this PR do?

This PR adds GIT to the auto-model mapping for the visual-question-answering pipeline and updates the corresponding GIT pipeline test mapping.

This ensures that GIT models (e.g. GitForCausalLM) are correctly recognized and usable with the visual-question-answering pipeline, consistent with existing generative model support.

## Issue

Partially addresses #21110

## Tests
- pytest tests/models/git/test_modeling_git.py -v -k "pipeline" -x
- pytest tests/pipelines/test_pipelines_visual_question_answering.py -v -x

---

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the contributor guideline, Pull Request section?
- [x] Was this discussed/approved via a Github issue or the forum?  
  Issue reference: #21110
- [ ] Did you make sure to update the documentation with your changes?  
  (Not required — registration/test-only change)
- [x] Did you write any new necessary tests?  
  (No new tests required; existing pipeline tests cover this change)

---

## Who can review?

@Rocketknight1

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, git

That entire pipeline was deleted in v5, I'm afraid! We usually just recommend `image-text-to-text` for VLMs.
[CB] [Serve] Fix broken serve tests
Author: remi-or
Pull request: yes

This PR fixes two issues related to CB and serve:
- when the `max_length` argument as set to `None` when creating a request, CB had an error because it always expected max_length to be an int. This is now an expected behavior, and we have a hidden variable to avoid checking if the variable is None
- the CB streamer did not yield the last generated token when the request finished. This was not an issue before the last PR, because the result of each request was yielded twice (with the same amount of tokens). Now that this is fixed, it revealed the bug related to the streamer, which is now fixed.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43594). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Guard torch.is_autocast_enabled(device_type) for torch<2.4
Author: oliverholworthy
Pull request: yes

This PR restores compatibility with PyTorch versions < 2.4 while preserving the current behavior on newer versions.

`torch.is_autocast_enabled(device_type)` was introduced in PyTorch 2.4, but Transformers currently supports torch>=2.2.  
On torch 2.2 and 2.3, calling this function with an argument raises a `TypeError`.

This change gates the call based on the installed torch version:

```python
if is_torch_greater_or_equal_than_2_4:
    is_autocast_active = torch.is_autocast_enabled(device_type)
else:
    is_autocast_active = torch.is_autocast_enabled()
````

This ensures:

* Newer PyTorch versions use the device-specific autocast API.
* Older supported PyTorch versions continue to work without runtime errors.

Motivation: avoid breaking users on torch 2.2/2.3 while keeping the existing minimum torch requirement unchanged.

Fixes #43508


## Who can review?

Tagging for awareness based on prior related changes and pipeline context:

@Rocketknight1


Comments:
I think we're just going to modify https://github.com/huggingface/transformers/pull/41307 to bump the minimum version to 2.4 instead! It's about time to do so anyway
Transformers 5.0.0 calls torch.is_autocast_enabled(device_type) breaking compatibility with torch<2.4
Author: oliverholworthy
Pull request: no
Labels: name: bug

In `transformers==5.0.0`, a call to `torch.is_autocast_enabled(device_type)` was introduced.  
That function signature (accepting a `device_type: str` argument) was only added in **PyTorch 2.4.0**.

However, the current minimum PyTorch version specified by Transformers is **torch>=2.2**.  
On torch 2.2.x and 2.3.x, `torch.is_autocast_enabled()` takes **no arguments**, and passing one raises a `TypeError`.

The call was introduced in this PR:
- https://github.com/huggingface/transformers/pull/42747

The PyTorch signature confirming when this was added:
- https://github.com/pytorch/pytorch/blob/v2.4.0/torch/_C/__init__.pyi.in#L1298

---

### Who can help?

@Rocketknight1 (tagging since you handled the last PyTorch minimum version bump in  
https://github.com/huggingface/transformers/pull/40317)

---

### Information

- [ ] The official example scripts  
- [X] My own minimal reproduction  

---

### Reproduction

#### 1) Environment setup

```bash
pip install "torch==2.2.0" "transformers==5.0.0"
````

#### 2) Minimal repro script

```python
from transformers.utils.generic import maybe_autocast
maybe_autocast("cuda")
# => TypeError: torch.is_autocast_enabled() takes no arguments (1 given)
```
---

### Expected behavior

Either:

1. Transformers should guard the call and fall back to the zero-argument version when running on older PyTorch, **or**
2. The minimum supported PyTorch version should be bumped to the first version that supports `torch.is_autocast_enabled(device_type)` (PyTorch 2.4.0).

---

### Actual behavior

With torch 2.2.x or 2.3.x:

```
TypeError: torch.is_autocast_enabled() takes no arguments (1 given)
```

and Transformers can crash at runtime if this codepath is hit.

---

### Additional context

From the discussion in PR #41307 (attempting to bump min torch to 2.3), it seems there is no policy like “support last N PyTorch versions”, and bumps are done for internal technical reasons rather than tracking PyTorch releases.

Given that, this appears to be an unguarded usage of a newer PyTorch API while still advertising compatibility with older ones.

---

### Possible fixes

**Option A (compatibility guard):**

#43509

```python
try:
    enabled = torch.is_autocast_enabled(device_type)
except TypeError:
    enabled = torch.is_autocast_enabled()
```

**Option B (dependency bump):**

* Raise minimum PyTorch version for Transformers 5.0.0 to >= 2.4.0.


Comments:
@ydshieh Please could you provide advice on which fix is preferred for this issue. 

I have an open PR for Option A as one route we could take:
https://github.com/huggingface/transformers/pull/43509

https://github.com/huggingface/transformers/pull/41307 is likely our solution here - we're just going to bump the minimum version.