noamaanMulla-03/transformers-issues huggingface dataset:
OLMo and OLMo 2 models do not match original models for low precisions
Author: 2015aroras
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.52.0.dev0
- Platform: macOS-14.5-arm64-arm-64bit
- Python version: 3.12.9
- Huggingface_hub version: 0.31.2
- Safetensors version: 0.5.2
- Accelerate version: 1.6.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Steps to reproduce behavior:

1. Obtain a copy of an OLMo or OLMo 2 model and its corresponding HF model.
2. Run the original and HF model in a lower precision like float16.

Giving code for a repro is particularly complicated due to running the models in the original codebase, but I intend to fix this bug myself so hopefully it is fine to omit this.

### Expected behavior

Expected behavior is that the output logits match or are close (say, all within 1e-4). In practice, this is not the case for bfloat16 and float16.

```
...
[2025-05-13 16:54:03] INFO     [__main__:246, rank=0] blocks.11.feed_forward_norm|output, model.layers.11.post_feedforward_layernorm|output element diff abs mean: 0.0077401818707585335
[2025-05-13 16:54:03] INFO     [__main__:246, rank=0] lm_head.norm|input, model.norm|input element diff abs mean: 0.015348772518336773
[2025-05-13 16:54:03] INFO     [__main__:246, rank=0] lm_head.norm|output, model.norm|output element diff abs mean: 0.01983717642724514
[2025-05-13 16:54:03] INFO     [__main__:246, rank=0] lm_head.w_out|input, lm_head|input element diff abs mean: 0.01983717642724514
[2025-05-13 16:54:03] INFO     [__main__:234, rank=0] lm_head.w_out|output, lm_head|output shape mismatch: torch.Size([1, 120, 100352]) torch.Size([1, 120, 100278])
[2025-05-13 16:54:03] INFO     [__main__:246, rank=0] lm_head.w_out|output, lm_head|output element diff abs mean: 0.03660527244210243
```

Comments:
Hi @2015aroras, in general you will not get outputs matching to within `1e-4` for float16 or bfloat16 models, because the reduced precision generally adds more numerical "noise". However, if you think you can improve the accuracy of the lower-precision models, we can definitely review that PR!
AutoConfig.from_pretrained on Llama4 models only returns the inner text_config
Author: cg123
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.51.3
- Platform: Linux-6.8.0-40-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.30.2
- Safetensors version: 0.5.3
- Accelerate version: 1.3.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: no
- GPU type: NVIDIA H100 PCIe

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```python
>>> import transformers
>>> cfg = transformers.AutoConfig.from_pretrained("meta-llama/Llama-4-Scout-17B-16E-Instruct")
>>> type(cfg)
<class 'transformers.models.llama.configuration_llama.LlamaConfig'>
```

### Expected behavior

I expect `AutoConfig.from_pretrained` to return the full `Llama4Config`, with both vision and text configs. As is it's currently quite difficult to work with Llama 4 models.

Comments:
Hi @cg123, when I run this code on `main` I get a `Llama4Config` object. Can you confirm that the issue is resolved when you install from `main` with `pip install git+https://github.com/huggingface/transformers.git`?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[SAM-HQ] Update names in the docs
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR updates the `repo_id` used in the docs of SAM-HQ (as it was updated to https://huggingface.co/syscv-community/sam-hq-vit-base)


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38058). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[image-text-to-text pipeline] Accept a chat as a positional arg
Author: gante
Pull request: yes

# What does this PR do?

See title. 

Calling `pipe(chat)` is not working, but `pipe(text=chat)` is. Intuitivelly, if we prepare a chat message containing the full data, text and images, it should work as a positional argument.

### Example

```py
import torch
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="google/gemma-3-4b-it", torch_dtype=torch.bfloat16)
messages = [
    {
        "role": "system",
        "content": [{"type": "text", "text": "You are a helpful assistant."}]
    },
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"},
            {"type": "text", "text": "Describe this image in detail."}
        ]
    }
]
print(pipe(messages, max_new_tokens=100))
```

before:
```
...
File "/home/joao/transformers/src/transformers/pipelines/image_text_to_text.py", line 351, in preprocess
    images = load_images(inputs["images"], timeout=timeout)
KeyError: 'images'
```

this PR:
```
[{'input_text': [{'role': 'system', 'content': [{'type': 'text', 'text': 'You are a helpful assistant.'}]}, {'role': 'user', 'content': [{'type': 'image', 'image': 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg'}, {'type': 'text', 'text': 'Describe this image in detail.'}]}], 'generated_text': [{'role': 'system', 'content': [{'type': 'text', 'text': 'You are a helpful assistant.'}]}, {'role': 'user', 'content': [{'type': 'image', 'image': 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg'}, {'type': 'text', 'text': 'Describe this image in detail.'}]}, {'role': 'assistant', 'content': "Here's a detailed description of the image:\n\n**Overall Impression:**\n\nThe image is a close-up, vibrant photograph of a garden scene featuring several pink cosmos flowers and a bumblebee. It has a slightly shallow depth of field, which brings the bee and the central cosmos flower into sharp focus, while the background is softer.\n\n**Foreground:**\n\n*   **Cosmos Flower:** The main focus is a large, bright pink cosmos flower. It has five delicate, slightly ruffled petals"}]}]
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38204). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
🚨🚨🚨  [pipelines] update defaults in pipelines that can `generate`
Author: gante
Pull request: yes

# What does this PR do?

TL;DR our defaults for text generation are very outdated. Most notably, the default maximum length.

This PR adds the tooling to easily add pipeline-specific `generate` defaults, and adds new defaults to all pipelines that call `generate`.

⚠️ ⚠️ ⚠️  

In all pipelines that call `generate`, the new `max_new_tokens` default is `256` 

⚠️ ⚠️ ⚠️ 

### Tests

This PR reviewed the tests of the pipelines that call `generate` such that their CI is now green. Overall changes:
- broken TF tests -> deleted
- tests that were already broken on `main` -> added a skip with a TODO
- slow tests that were not slow (< 3s) -> removed `@slow`
- tests were the actual output contents were not checked -> changed to a smaller model whenever possible, and reduced `max_new_tokens`

### Example

```py
from transformers import pipeline
generator = pipeline(model="Qwen/Qwen3-0.6B")
print(generator("Tell me a story about a cat:"))
```

On main it prints
```
[{'generated_text': 'Tell me a story about a cat: a cat named Bella, who is a member of a small town in the countryside. 
She has a'}]
```

With this PR it prints
```
[{'generated_text': "Tell me a story about a cat: a cat with the name of Cesar, who has a special ability to create things
 out of his imagination, and he loves playing with toys. He has a dog named Luna who loves playing with him. What are the 
possible themes and messages that can be conveyed through this story? Also, could you provide an example of a plot 
twist that could be added to the story? Additionally, could you write a short story of at least 1000 words, including the 
opening scene, the development of the story, and the closing scene?\nAnswer:\n\n### A Cat with a Special Ability\n\nIn a 
quiet village nestled between rolling hills and a distant forest, nestled in a cozy cottage, lived a cat named Cesar. He was 
small and playful, with a gentle gaze that made people smile. However, something about Cesar was truly unique. He could 
create things out of his imagination, a gift that made him both curious and endearing.\n\nCesar's ability to create things 
out of his imagination was unlike anything he had ever known. He could conjure up whimsical creatures, build elaborate 
structures, and even make things that seemed impossible at first. He loved to play with toys, and his favorite toy was a 
small wooden box that he always filled with colorful feathers and"}]
```

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38129). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Tensor parallel docs
Author: S1ro1
Pull request: yes

Improve the existing docs for tensor parallelism, with more details etc. Relies on #37877 (needs to be edit a tiny bit when final version of that PR lands). 


cc @ArthurZucker 



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38178). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[doc] fix bugs in `how_to_hack_models.md`
Author: faaany
Pull request: yes

## What does this PR do?
When following the tutorial in `how_to_hack_models.md`, I find that I got multiple errors:

1. task_type "mask-generation" is invalid:
```bash
Traceback (most recent call last):
  File "/home/sdp/fanli/transformers/../test_doc.py", line 72, in <module>
    config = LoraConfig(
  File "<string>", line 33, in __init__
  File "/home/sdp/miniforge3/envs/ipex-2.7/lib/python3.10/site-packages/peft/tuners/lora/config.py", line 539, in __post_init__
    super().__post_init__()
  File "/home/sdp/miniforge3/envs/ipex-2.7/lib/python3.10/site-packages/peft/config.py", line 67, in __post_init__
    raise ValueError(
ValueError: Invalid task type: 'mask-generation'. Must be one of the following task types: SEQ_CLS, SEQ_2_SEQ_LM, CAUSAL_LM, TOKEN_CLS, QUESTION_ANS, FEATURE_EXTRACTION.
```

2. `SamVisionAttentionSplit` is not replaced successfully with `modeling_sam.SamVisionAttention = SamVisionAttentionSplit`
```bash
#model = SamModel.from_pretrained("facebook/sam-vit-base")
#print(type(model.vision_encoder.layers[0].attn))
<class 'transformers.models.sam.modeling_sam.SamVisionAttention'>
```

3. `SamVisionAttentionSplit` object has no attribute `add_decomposed_rel_pos`


This PR fixed all these error to make the example code works. 

Comments:
cc @stevhliu

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38198). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix: make docs work better with doc builder
Author: S1ro1
Pull request: yes

This pr fixes docs to work with doc builder properly. #38178 broke the formatting somehow.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38213). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[fix] sliding window attention mask
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/38002. Sliding mask should not be applied when `config.use_sliding_window = False`

Added a small test as well


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38045). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[bug] `use_sliding_window` doesn't work as expected
Author: ZhiyuLi-Nvidia
Pull request: no
Labels: name: bug

### System Info

* transformer: main
* pytorch, cuda: anyversion

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoModelForCausalLM
import numpy as np
import torch


MODEL_NAME = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"
model = AutoModelForCausalLM.from_pretrained(MODEL_NAME)

input_ids = torch.randint(0, 100, (1, 8192))

with torch.no_grad():
    output_raw = model(input_ids)

# correct situaion: should be the same as the original model since use_sliding_window is False
model_no_sliding = AutoModelForCausalLM.from_pretrained(MODEL_NAME, sliding_window=None)

with torch.no_grad():
    output_non_sliding = model_no_sliding(input_ids)


np.testing.assert_allclose(output_raw.logits[:, :4096], output_non_sliding.logits[:, :4096])

# wrong: the logits are unexpectedly different with sliding_window=4096
np.testing.assert_allclose(output_raw.logits[:, 4096:], output_non_sliding.logits[:, 4096:])
```

### Expected behavior

# description
What is expected:   

```
  "sliding_window": 4096,
  "use_sliding_window": false,
```
`use_sliding_window` is set as false in `deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B` [here](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B/blob/main/config.json#L20). We do expect sliding window is disabled. In other words, we should expect the same results even with different `sliding_window`.

However, the results are different in the repro script.


# Root cause
Attention Mask is changed according to `sliding_window` without respect on `use_sliding_window`.

https://github.com/huggingface/transformers/blob/3c0796aaeac722152836ea063d542a4c628a75be/src/transformers/models/qwen2/modeling_qwen2.py#L708-L715

If we add some printing under this conditional block, we can clearly see attention mask is changed even with `use_sliding_window=false`

Comments:
Not sure who's the right code owner here - cc @gante @zucchini-nlp, but feel free to tag Arthur instead if you think he's more appropriate!

Nice catch! Indeed I think we should be checking both values, given that in attention layer we pass sliding window only when `use_sliding_window`. I can open a PR for that 🤗
Remove trust_remote_code=True tests from bnb quantization tests (MPT now integrated)
Author: Titus-von-Koeller
Pull request: yes

This PR removes all tests in the bitsandbytes (bnb) quantization test suite that rely on `trust_remote_code=True`, specifically the `test_get_keys_to_not_convert_trust_remote_code` test and its usage of the MosaicML MPT model.

**Context and reasoning:**
- The MPT model is now directly integrated into Transformers and no longer requires `trust_remote_code=True` or custom code from the Hub.
- The removed test was the only one using `trust_remote_code=True` and was causing CI failures due to missing dependencies (e.g., `triton_pre_mlir`).
- As discussed [here](https://huggingface.slack.com/archives/C06ALV91VML/p1747668043103049?thread_ts=1744637333.150599&cid=C06ALV91VML) and agreed by the team.

This should help stabilize the CI and avoid unnecessary dependency issues for the bnb quantization tests.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38206). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
New cache tests and modular Hybrid Cache
Author: manueldeprada
Pull request: yes

## What does this PR do?

1. Refactor out the cache update logic for static and sliding window attention mechanisms.
Extracts the core update logic into two new utility functions: `_static_cache_update_logic` and `_sliding_cache_update_logic`. This way, there is only one implementation for StaticCache, SlidingWindowCache, and HybridCache.
@ArthurZucker @gante this is a first step towards per-layer modular cache definitions.
2. Added new synthetic tests for caches. Fixes #37574 and should catch similar bugs.

This is preliminary work for #38077

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37972). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

All suggestions applied and all the tests moved into clear "hardcoded" ones!! thanks a lot for the feedback :)

Thanks @Cyrilvallez for quickly merging the fix into main(#38046)! In hindsight, I could’ve split the PR into the fix and the refactor + tests.

That said, I think this version is better long-term: having the sliding logic in one place with clear names and comments:
https://github.com/huggingface/transformers/blob/8548b8f3eb6c7344111c4d58dde0eccf3cd45b27/src/transformers/cache_utils.py#L97-L105
In Llama4 fix wrongly inverted causal attention mask when using SDPA implementation
Author: sogartar
Pull request: yes

When preparing the causal attention mask at this point the mask comes in as a float tensor with min value as a masked value.
It is not correct to convert it to bool and treat it as a bool mask as this inverts the mask. `torch.nn.functional.scaled_dot_product_attention` expects that a bool masked value is `False`.

I suspect that the `sdpa` implementation variant may not have been thoroughly tested and that is why this error was not caught earlier.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @arthurzucker!
Make `train_dataset` attribute in `_get_train_sampler` optional 
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR fixes a breaking change that was caused by https://github.com/huggingface/transformers/pull/38090. We removed in that PR `train_dataset` arg but TRL lib uses it. We make it optional instead now

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38226). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix dead flax links modeling_flax_pytorch_utils.py
Author: dizer-ti
Pull request: yes

Hey team—noticed a dead link, replaced it with a working URL

https://flax.readthedocs.io/en/latest/installation.html - old links
https://flax.readthedocs.io/en/latest/index.html#installation - new links

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38212). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Minor llama4 fixes
Author: molbap
Pull request: yes

# What does this PR do?

Fixes a couple nits/wrong defaults in Llama4 code for DynamicCache init - not sure we should even default to it, I think.
The scaling however has to be passed else `eager_attention_forward" will fail.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38123). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sounds good :D
Fix Llama4
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. It would currently fail with `RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.`, at least when using TP.

Unfortunately, flex block mask and generate got broken as well, but this will be solved by the mask refactor. Solving it here would involve a lot of duplicated efforts.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38222). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Llama Guard updates
Author: pcuenca
Pull request: yes

Reviewed by @molbap privately.

#37852 should be merged too.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37872). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This broke quite a few things, will have a look