noamaanMulla-03/transformers-issues huggingface dataset:
Add EdgeTam to Transformers 🤗
Author: yonigozlan
Pull request: no
Labels: name: New model

### Model description

Happy to take this one, as I'm working on Sam2 and the implementation looks very similar.
I'll have to dive deeper in the code to check if this even need to be a separate model.


### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

https://github.com/facebookresearch/EdgeTAM

Comments:
Hey @yonigozlan

Great to hear about this new initiative. Lmk, if can help you anyhow :-)

Fixed in https://github.com/huggingface/transformers/pull/39800

I think this should be closed? cc @zucchini-nlp
Fix some deprecated practices in torch 2.9
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. It otherwise triggers the following warning:

`UserWarning: Using a non-tuple sequence for multidimensional indexing is deprecated and will be changed in pytorch 2.9; use x[tuple(seq)] instead of x[seq]. In pytorch 2.9 this will be interpreted as tensor index, x[torch.tensor(seq)], which will result either in an error or a different result`

and will result in wrong outputs.

Also takes care of the following warning:

`UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument.`

by providing the default


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43167). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: dab_detr, efficientloftr, funnel, omdet_turbo, oneformer, superglue

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43167&sha=17e50b
Fix Fuyu processor width dimension bug in `_get_num_multimodal_tokens`
Author: Abhinavexists
Pull request: yes

# What does this PR do?

This PR fixes a critical bug in the Fuyu processor's `_get_num_multimodal_tokens` method where the image width calculation incorrectly uses the height dimension.

## The Bug

In `src/transformers/models/fuyu/processing_fuyu.py` at line 629, the code was:

```python
image_unpadded_h = min(int(image_size[0] * optimal_scale_factor), image_size[0])
image_unpadded_w = min(int(image_size[0] * optimal_scale_factor), image_size[0])  # BUG: uses [0] twice!

```

Both height and width were being calculated using `image_size[0]` (the height dimension). The width should use `image_size[1]`.

## The Fix

Changed line 629 to correctly use `image_size[1]` for the width calculation:

```python
image_unpadded_h = min(int(image_size[0] * optimal_scale_factor), image_size[0])
image_unpadded_w = min(int(image_size[1] * optimal_scale_factor), image_size[1])  # Fixed

```

## Impact

This bug caused incorrect image padding calculations for multimodal inputs in Fuyu, resulting in:

* **Wrong aspect ratio preservation** for images.
* **Incorrect token count computation** for variable-sized images.
* **Distorted image processing** when .

The fix ensures proper scaling of both dimensions independently, matching the intended behavior shown in the corresponding `image_processing_fuyu.py` file.

---

## Before submitting

* [x] This PR fixes a bug in model implementation.
* [x] Did you read the [[contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md)](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md)?
* [ ] Was this discussed/approved via a Github issue or the forum?
* [x] This is a minimal change that does not require documentation updates.
* [ ] Did you write any new necessary tests? (Fix is self-evident and covered by existing tests).

## Who can review?

@zucchini-nlp (multimodal models) @molbap (vision models)

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: fuyu

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43137). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix failing `BartModelIntegrationTest`
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?

Fixes this failing [BartModelIntegrationTest](https://github.com/huggingface/transformers/actions/runs/20706072009/job/59437072399#step:14:2431).

<img width="1329" height="139" alt="image" src="https://github.com/user-attachments/assets/68d49d77-aadc-4de6-8aa8-9b859b42a659" />



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@vasqu

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bart

run-slow: bart

This comment contains `run-slow`, running the specified jobs: 

models: ["models/bart"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20818898084)

✅ No failing test specific to this PR 🎉 !


The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43160). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Generation config boolean defaults
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Continues on https://github.com/huggingface/transformers/pull/42958 and makes sure the opposite case, when config has a non-default value saved but the user wants to set it back to a default boolean, is also covered


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43000). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: llama, gemma2, qwen2, bart, t5, whisper

This comment contains `run-slow`, running the specified jobs: 

models: ["models/bart", "models/gemma2", "models/llama", "models/qwen2", "models/t5", "models/whisper"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20432504592)

### Model CI Report

#### ❌ Failed tests

- [bart](https://github.com/huggingface/transformers/actions/runs/20432504592/job/58706641170):
    tests/models/bart/test_modeling_bart.py::BartModelIntegrationTests::test_xsum_config_generation_params



> LGTM, could you just double check if we still need

my bad, didn't see the comment on it. Not needed!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bart, clvp
[`Fp8`] Fix experts
Author: vasqu
Pull request: yes

As per title, the current fp8 experts implementation is wrong - likely related to #42456 (cc @3outeille)

Before:
- The number of experts was treated as the number of `top k` experts that are hit, i.e. `num_experts = top_k_weights.shape[1]`
  - This is simply wrong as now our indices which can cover all experts is limited to `top_k`
  - This can trigger device assert errors as one hot might not have the correct indices for the number of classes indicated by num_experts
- The hit expert is now used which is correct if we passed the selection before
  - Only due to the indices being correctly limited 

After:
- We now follow the current eager implementation, e.g. Mixtral: https://github.com/huggingface/transformers/blob/88a5623361e1b3d844daef3c6c95535d12e70056/src/transformers/models/mixtral/modeling_mixtral.py#L82-L86
- The portion where we iterate through experts should only have changed the linear calls of https://github.com/huggingface/transformers/blob/88a5623361e1b3d844daef3c6c95535d12e70056/src/transformers/models/mixtral/modeling_mixtral.py#L88-L98
  - The key is to redirect the correctly hit expert to the top k weight via https://github.com/huggingface/transformers/blob/88a5623361e1b3d844daef3c6c95535d12e70056/src/transformers/models/mixtral/modeling_mixtral.py#L92

I'm not sure how long it has been this way, but it definitely is not correct atm. I'd be curious if I'm overseeing something here but without this fix #42028 will not work for the original model - my suspicion is on mixtral having a bias towards low indices which might have saved this oversight.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43154). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@3outeille note that this breaks TP but same as it is broken for now as well

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: finegrained_fp8

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43154&sha=d0d860
qwen2 tok regression
Author: itazap
Pull request: yes

handle when merges are a list format - happens when loading from json

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen2

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42644). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Check kernel FA when creating mask
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

If we request kernels FA2 when the model is loaded, the attention mask is created as 4D which causes issues downstream (e.g in Siglip). We can either manually force `config.attn_implementation` to be only `flash_attention_2/3` by mapping it back from kernels, or we can check both until new masking API is used everywhere

IMO we need to keep `config.attn_implementation` pointing to correct impl, so I went to adjust the name in all other files

Btw, some models don't even use the general attention interface yet 🫠 I didn't check model-by-model and made changes in batch automatically. 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: altclip, bamba, bark, bloom, clipseg, codegen, deepseek_v2, deepseek_v3, falcon, falcon_h1, git, gpt_bigcode, gpt_neo, gpt_neox_japanese, gptj, idefics

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43172&sha=6c6dfd

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43172). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Does #43121 solve these issues, had the same thought as you and it annoyed me so I went into it :joy: 

oh cool, did not see that 😄
The _crop_past_key_values function should be a member function of Cache.
Author: jiqing-feng
Pull request: no
Labels: name: Feature request

name: Cache

### Feature request

The _crop_past_key_values function should be a member function of Cache.

### Motivation

I suppose all models will use Cache class instead of tuple to save past_key_values. It will make more sense to make [_crop_past_key_values](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/candidate_generator.py#L375) as a member function of Cache.

### Your contribution

cc @gante @echarlaix 

Comments:
Hi @jiqing-feng, I would like to work on this issue. I have an approach, Could you please tell me if it works and is as of expectation. 

I need to add a cache function in the Cache of cache_utils.py and I need to call this function in the generation code instead of manual computation.

```Python
class Cache:
    def _crop_past_key_values(self, new_length: int) -> None:
        for layer in self.layers:
            if hasattr(layer, "key"):
                layer.key = layer.key[..., :new_length, :]
            if hasattr(layer, "value"):
                layer.value = layer.value[..., :new_length, :]
```

Sure, it works for me.

This was already done in one of the old PRs, so ig it can be closed now. We have `cache.crop()` for some of the classes, if it supports assisted decoding
Processor's attribute is mis-detected for child class from #41633
Author: MilkClouds
Pull request: no
Labels: name: WIP

name: bug

### System Info

transformers: v5.0.0.rc1

### Who can help?

@yonigozlan @Cyrilvallez 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

From #41633, transformers removed manually defined `attributes` and replaced this into auto-detected from argument of `__init__` using `inspect`.

https://github.com/huggingface/transformers/blob/05c0e1d39082ee8b69064ed4ea9c239cd17405e9/src/transformers/processing_utils.py#L1405-L1409

But it does not deal with the case for inherited class, e.g: there's no `name` present on signature of class B's init.
```py
In [5]: import inspect

In [6]: class A:
   ...:     def __init__(self, *args, name: str, **kwargs):
   ...:         self.name = name
   ...:

In [7]: class B(A):
   ...:     def __init__(self, *args, name2: str, **kwargs):
   ...:         self.name2=name2
   ...:         super().__init__(*args, **kwargs)

In [12]: inspect.signature(B.__init__).parameters.keys()
Out[12]: odict_keys(['self', 'args', 'name2', 'kwargs'])
```

Although above is toy example, I'm using inherited class as Processor in my own research code. Following is tiny snippet from my own code. My own code broken and does not save attributes properly when `save_pretrained` is called after #41633.

```py
class ActionTokenizerMixin:
    def __init__(self, *args, action_tokenizer: Optional[BaseActionTokenizer] = None, **kwargs):
        super().__init__(*args, **kwargs)
        self._action_tokenizer = action_tokenizer
        self._is_action_tokenizer_prepare_called = False

class AutoProcessorWithActionMixin(ActionTokenizerMixin, AutoProcessor):
    pass

```

### Expected behavior

See above.

Comments:
And IMO, I believe goal of "auto-detecting everything" is neither realistic nor useful. Manual specification has it's own value, especially for `Processor` which is not strictly defined and has wide variety. Also detecting attributes from `__init__` signature seems not that reliable, possibly leading to many loopholes.

cc @yonigozlan , maybe we can allow to set explicit attributes as before and fallback to auto-detect if not set. It will give freedom to overwrite and define their own preprocessing classes for users

Thanks for raising the issues @MilkClouds . Agreed with @zucchini-nlp that we can first check if sub_processor_class attributes are manually defines before trying to auto-detect everything. I'll open a PR for that today.
Docs: improve wording for documentation build instructions
Author: Sailnagale
Pull request: yes

What does this PR do?
- Improves wording and clarity in the documentation build instructions.

Notes:
- Documentation-only change.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43007). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[makefile] Cleanup and improve the rules
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Along the years, the rules accumulated and are now weirdly named/hard to remember.
This PR cleans everything, and adds the following 3 important rules:

- `style` -> runs only linting/formatting automatic checks
- `check-repo` -> performs all repo health checks, without fixing even if a fix is available
- `fix-repo` -> performs all repo automatic fixes, e.g. modular conversions, Copied from, dummy etc

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43171). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43171&sha=46aef1
Some new models added stuff that was already removed
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. They don't exist

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: pe_audio, pe_audio_video, pe_video, switch_transformers, video_llama_3

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43179). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Error with text generation pipeline in v5
Author: davidmezzetti
Pull request: no
Labels: name: bug

### System Info

transformers 5.0.0rc0

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import pipeline

model = pipeline(task="text-generation", model="hf-internal-testing/tiny-random-LlamaForCausalLM")
model([{"role": "user", "content": "Test"}])
```

```
  File "transformers/pipelines/text_generation.py", line 293, in __call__
    return super().__call__(text_inputs, **kwargs)
  File "transformers/pipelines/base.py", line 1267, in __call__
    outputs = list(final_iterator)
  File "transformers/pipelines/pt_utils.py", line 127, in __next__
    processed = self.infer(item, **self.params)
  File "transformers/pipelines/text_generation.py", line 489, in postprocess
    if self.tokenizer.response_schema:
  File "transformers/tokenization_utils_base.py", line 1329, in __getattr__
    raise AttributeError(f"{self.__class__.__name__} has no attribute {key}")
AttributeError: LlamaTokenizer has no attribute response_schema
```

### Expected behavior

Generate text

Comments:
Yes, this is definitely a bug. On it!

Fix: #42300

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Need to understand logging_steps in Trainer.
Author: quic-meetkuma
Pull request: no

When using Trainer for training in DDP setting, does the logging_steps argument perform all-reduce for that step just to get the single loss value? 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Can anyone help me understand the internal workings?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fixes and compilation warning in torchao docs
Author: merveenoyan
Pull request: yes

cc @SunMarc as discussed and also @stevhliu 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42909). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
torch.compile of model.forward() or model.generate() not working anymore
Author: jeasinema
Pull request: no
Labels: name: Usage

name: bug

### System Info

- Ubuntu 24.04
- `torch==2.10.0a0+b4e4ee81d3.nv25.12`
- `transformers==4.57.3`

### Who can help?

@gante  @ArthurZucker 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Just run the [original sample code](https://github.com/huggingface/transformers/pull/30788#issue-2293357958).

### Expected behavior

Per https://github.com/huggingface/transformers/pull/30788, `torch.compile(model.forward)` or `torch.compile(model.generate)` should both work for `model.generate()` now. However, with `transformers==4.57.3`, I got the following error when running the [original sample code](https://github.com/huggingface/transformers/pull/30788#issue-2293357958) by @gante:

```
Traceback (most recent call last):
  File "/workspace/compile_vla/gemma_test.py", line 108, in <module>
    time_generate_call(model, "static + fwd compiled", ms_per_token, first_step, static_cache=True)
  File "/workspace/compile_vla/gemma_test.py", line 68, in time_generate_call
    torch.compiler.reset()
  File "/usr/local/lib/python3.12/dist-packages/torch/compiler/__init__.py", line 65, in reset
    torch._dynamo.reset()
  File "/usr/local/lib/python3.12/dist-packages/torch/_dynamo/__init__.py", line 149, in reset
    _reset_guarded_backend_cache()
  File "/usr/local/lib/python3.12/dist-packages/torch/_dynamo/eval_frame.py", line 334, in _reset_guarded_backend_cache
    for backend in cached_backends.values():
RuntimeError: dictionary changed size during iteration
```

It works when switching back to some older versions like `transformers==4.48.0` tho.

Any thoughts?

Comments:
Okay, so you should not compile yourself if you use generate, you should rely on this: https://github.com/huggingface/transformers/blob/57278c904c5158999d31a0db8bfcd63360c37b48/src/transformers/generation/utils.py#L2852-L2857. 

That is because there is `prefill` and `decode` that require different inputs. 

Thank you. I got it working by switching to `CompileConfig`. Here is an updated sample code:

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, StaticCache
from transformers.generation import CompileConfig

MODEL_ID = "google/gemma-2b"
ATTN_IMPL = "sdpa"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, padding_side="left")
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID, torch_dtype=torch.float16, attn_implementation=ATTN_IMPL
).to("cuda")

torch.compiler.reset()
model.generation_config.compile_config = CompileConfig(
    fullgraph=True,
    backend='inductor',
    dynamic=False,
    mode='max-autotune',
)
# !! you still need to manually specify static cache to make compilation work for generate() 
model.generation_config.cache_implementation = 'static'

model.generate()
```



thanks 🤗