noamaanMulla-03/transformers-issues huggingface dataset:
Some methods in TrainerControl seem not to be utilized.
Author: mst272
Pull request: no

Looking at the callback code has caused me a great deal of confusion. It seems that this function has never been used. I'm not sure if I've missed something. 

https://github.com/huggingface/transformers/blob/6966fa190172b48b2fb46fe4552a13b943e692cf/src/transformers/trainer_callback.py#L275

Comments:
cc @muellerzr @SunMarc 

These are indeed not used. Not sure the utility to be exact but probably here to let us know how the variable should be set to have specific behavior
[CI] Automatic rerun of certain test failures
Author: gante
Pull request: yes

# What does this PR do?

Adds `pytest-rerunfailures` to our push CI. This should help stabilize our CI and allow faster dev cycles 🤗 

In a nutshell, if a test failure has a substring in the newly added `FLAKY_TEST_FAILURE_PATTERNS` variable (`.circleci/create_circleci_config.py`), that test is repeated up to 5 times. `FLAKY_TEST_FAILURE_PATTERNS` in this PR includes the following categories of failures:
1. `'OSError'` / `'Timeout'`  -- generic transient machine/connection errors
2. `'HTTPError.*502'` / `'HTTPError.*504'` -- hub HTTP failures
3. `"AssertionError: Tensor-likes are not close!"`  -- unlucky `torch.testing.assert_close` cases
4. [several] -- exceptions thrown after a hub failure handling

Thank you for the suggestion @Wauplin 💛 
______________________________
Follow up: remove some uses of `@is_flaky()`, since they serve the same purpose:
1. Tests with old `@is_flaky()` additions that I couldn't reproduce with `--flake-finder --flake-runs 1000`
2. Tests with `@is_flaky()` that were tagged because one of the criteria in the list above

See diff in [this commit](https://github.com/huggingface/transformers/pull/36694/commits/65ccbeef450d4bbb0c76ac2eb485973022cec22c)

______________________________________________________
Corresponding PR for docker image update: https://github.com/huggingface/transformers/pull/36427

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36694). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> I think we need to add pytest-rerunfailures to `extras["testing"]`

@ydshieh good catch! I added it to the PR that builds the dev image, but forgot to add it here. Added ✅ 


image pushed in #36427 , merging
Final CI cleanup
Author: Rocketknight1
Pull request: yes

@gante's recent PR has mostly resolved the CI issues, but there are a couple more problems left to fix! One is a flaky issue with an unusual error message that isn't caught by the pytest retry pattern, and the other two are actual failures that are blocking the CI. I'm skipping them for now, but we'll need to deal with those.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36703). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add support for fast image processors in add-new-model-like CLI
Author: yonigozlan
Pull request: yes
Labels: name: Core: CLI

# What does this PR do?

Fix add-new-model-like CLI for image processor and fast image processors.
Also encourage contributors to add a fast image processor to new model, even if the old model on which the new is based on doesn't have one.
Fixes/improve add-fast-image-processor CLI.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36313). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Updated the docs! @stevhliu would you mind having a look when you have some bandwidth? Thanks!
`disable_compile` not honored as a kwarg in generate
Author: pcuenca
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.49.0
- Platform: Linux-6.8.0-49-generic-x86_64-with-glibc2.39
- Python version: 3.10.16
- Huggingface_hub version: 0.29.1
- Safetensors version: 0.5.3
- Accelerate version: 1.4.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 4090

### Who can help?

@gante, @SunMarc, @ArthurZucker 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The following snippet should disable `torch.compile`, note the use of `disable_compile` as a kwarg. From the documentation, it should replace the corresponding value in `generation_config`:

```python
import os
os.environ["TORCH_LOGS"]="+dynamo"

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "google/gemma-2-2b-it"
device = "cuda:0"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16).to(device)

prompt = "<start_of_turn>user\nWrite a poem about the Kraken.<end_of_turn>\n<start_of_turn>model\n"
inputs = tokenizer.encode(prompt, return_tensors="pt").to(device)

outputs = model.generate(inputs, max_length=50, disable_compile=True)
text = tokenizer.decode(outputs[0])
```

But we can still see dynamo tracing calls when we run it.

The reason appears to be [this line](https://github.com/huggingface/transformers/blob/66f29aaaf55c8fe0c3dbcd24beede2ca4effac56/src/transformers/generation/utils.py#L3219), which uses `self.generation_config` instead of `generation_config`.

Note that this behaviour will be fixed by #36519 when it's merged. Alternatively, we could fix this issue first if that PR takes long to be approved.

### Expected behavior

As discussed above.

Comments:
(should be fixed in #36519)

@gante Yes, noted at the end of my comment :) Keeping track in case that PR takes long, we are temporarily needing this for an upcoming integration.

Closing, this was in case we needed to add it for the Gemma 3 release, but @gante fixed it :)
Component loading incorrect dtype
Author: dsocek
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.50.0.dev0
- Platform: Linux-6.8.0-52-generic-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 0.29.3
- Safetensors version: 0.4.5
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0a0+df5bbc09d1.nv24.12 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: yes
- GPU type: NVIDIA A100-SXM4-80GB


### Who can help?

@zucchini-nlp @ArthurZucker

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Some components are loaded with incorrect `dtype`. @zucchini-nlp  `git bisect` analysis shows that this bug was introduced in https://github.com/huggingface/transformers/commit/84a6789145c3d728f2e405d31e9a35df5d74f05c

```
84a6789145c3d728f2e405d31e9a35df5d74f05c is the first bad commit
```

Please install diffusers for a simple reproducer:
```
pip install diffusers
```

Reproducer script (we instantiate SDXL pipeline with BF16 `dtype`, and expect components to be of this type):
```py
import torch
from diffusers import StableDiffusionXLPipeline
pipe = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.bfloat16,
)
print(pipe.text_encoder.dtype)
print(pipe.text_encoder_2.dtype")
```

Output (correct) before https://github.com/huggingface/transformers/commit/84a6789145c3d728f2e405d31e9a35df5d74f05c :
```
torch.bfloat16
torch.bfloat16
```

Output (incorrect) after https://github.com/huggingface/transformers/commit/84a6789145c3d728f2e405d31e9a35df5d74f05c was merged:
```
torch.bfloat16
torch.float16
```

Now 2nd text encoder is incorrectly loaded as `torch.float16` which can cause a series of issues (e.g. see https://github.com/huggingface/optimum-habana/pull/1815)

@zucchini-nlp  can you please help address this issue?

### Expected behavior

2nd text encoder should be loaded as BF16 `torch.bfloat16` and not as FP16 `torch.float16`.  This was correctly the case before the bug was introduced.

Comments:
Thanks for reporting. Might be fixed by https://github.com/huggingface/transformers/pull/36679. I can check out later tomorrow, if not fixed by that PR will dig into root causes

@zucchini-nlp FYI I tested #36679 PR and this resolves the issue on my side.  Also tested SD3 pipeline (which had 2 out of 3 text encoders loaded as FP16 instead of assigned BF16) and all 3 text encoders are loading OK. Looking forward to merging this fix :)

Great, the PR has been merged so I'll close this issue as resolved :)
ValueError: Unrecognized model. Should have a model_type key in its config.json
Author: LujainAlawwad
Pull request: no

I have trained a custom transformer model as a sequence-to-sequence model, did not finetune an LLM.

My config doesn't specify the model-type, it is as follows:
config = {
  "architecture": "Seq2SeqTransformer",
  "vocab_size": tokenizer.vocab_size,
  "d_model": d_model,
  "num_layers": num_layers,
  "num_heads": num_heads,
  "dim_feedforward": dim_feedforward,
  "save_steps": save_steps
}

But when I try to load it from huggingface, it gives me this error:
**Lu...... Should have a model_type key in its config.json, or contain one of the following strings in its name: albert, align, altclip......**

I do not know which type should be specified.

Please find the layout of my work in this notebook: https://colab.research.google.com/drive/1qwT4mqaTPoVW7HwlWNRKheR7FAzIdR-L#scrollTo=DtU3whBevE31

Comments:
Hey! Your notebook seems to have stopped at the:
```
MODEL_MAPPING.update({"seq2seq_transformer": CustomSeq2SeqModel})
```
and did not execute it as the above failed. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I had the same issue. Basically I was pushing my model `trainer.push_to_hub("model_name")` but I did not set the argument `push_to_hub=True,` in the `TrainingArguments` class.

Following the documentation, It should be like this:

`training_args = TrainingArguments(output_dir="my-awesome-model", push_to_hub=True)`

Reference: https://huggingface.co/docs/transformers/model_sharing
Some questions of `Gemma3` processor
Author: Kuangdd01
Pull request: no
Labels: name: VLM

Thanks for bringing us a nice implementation of the `Gemma3` model!  After reading the code, I have a question about `gemma3.processing.py`. 
This segment of code is as follows:  [code](https://github.com/huggingface/transformers/blob/d84569387fb1f88c86fb8d82a41f20c9e207d09e/src/transformers/models/gemma3/processing_gemma3.py#L126C16-L133C60)
```python 
 for batch_idx, (prompt, images, num_crops) in enumerate(zip(text, batched_images, batch_num_crops)):
    image_indexes = [m.start() for m in re.finditer(self.boi_token, prompt)]

    if len(images) != len(image_indexes):
        raise ValueError(
            f"Prompt contained {len(image_indexes)} image tokens but received {len(images)} images."
        )

    # Insert additional image tokens for Pan-and-Scan crops
    for num, idx in reversed(list(zip(num_crops, image_indexes))):
        if num:
            formatted_image_text = (
                f"Here is the original image {self.boi_token} and here are some crops to help you see better "
                + " ".join([self.boi_token] * num)
            )
            prompt = prompt[:idx] + formatted_image_text + prompt[idx + len(self.boi_token) :]
            text_with_crops[batch_idx] = prompt
```
I can see that this code is handling the placeholders for the image and when `Pan-and-Scan` is on, the crops of the image will also be added to the sentence before feeding into the tokenizer.  

But `text_with_crops` seems never to be used after that.   The sub-fig in a sentence is a nice feature for me!
Is there something I missed or is the code in `if num` incomplete? @RyanMullins  @zucchini-nlp 

Comments:
@Kuangdd01 thanks for reporting that! Indeed seems to be a typo. Would you like to submit a PR to fix that? We'll also need a small test where the `do_pan_and_crop=True` to make sure the crops are being added to the input prompt

> @Kuangdd01 thanks for reporting that! Indeed seems to be a typo. Would you like to submit a PR to fix that? We'll also need a small test where the `do_pan_and_crop=True` to make sure the crops are being added to the input prompt

I will try to make a PR and a small test.
Make the flaky list a little more general
Author: Rocketknight1
Pull request: yes

cc @gante, I also saw `HfHubHTTPError` in flaky tests, so I rewrote the list to include that too!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36704). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Cleanup the regex used for doc preprocessing
Author: Rocketknight1
Pull request: yes

We had an overly-complex regex designed to capture docstrings, and as a result it has potentially exponential runtime for malicious inputs. This PR makes the following simplifying changes:

- Explicit calls to `re.compile` without saving the compiled object are removed - these only hurt performance (see [note](https://docs.python.org/3/library/re.html#re.compile))
- `re.MULTILINE` was removed because this only [affects the behaviour](https://docs.python.org/3/library/re.html#re.MULTILINE) of `^` and `$` characters, which are not used in this regex
- Over-complex non-capturing group replaced with much simpler regex, because `re.DOTALL` allows `.*?` to go through newlines

cc @Michellehbn!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36648). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @ydshieh for review as the original author (I think!) of the function
[model loading] don't `gc.collect()` if only 1 shard is used
Author: gante
Pull request: yes

# What does this PR do?

Fixes a test-related regression introduced by #36033. Despite speeding up `from_pretrained` in general, most tests that relied on `from_pretrained` became much slower (see tests section below).

Measuring time, the entire slowdown can be traced to a [single line](https://github.com/huggingface/transformers/blob/8cb522b4190bd556ce51be04942720650b1a3e57/src/transformers/modeling_utils.py#L4900), a `gc.collect()`. This line was not added in #36033, it was moved. [Before #36033](https://github.com/huggingface/transformers/blame/7652804d237fb8768f0f0b8129a05e4f0576114b/src/transformers/modeling_utils.py#L5060), if the checkpoint was not sharded, a `state_dict` would have been passed to `_load_pretrained_model`, and the `gc.collect()` branch would not be reached.

This PR adds a tiny `if` to skip `gc.collect()` if the checkpoint is not sharded. Since many tests rely on unsharded `from_pretrained`, we can immediately observe faster tests.

### Tests 

`py.test tests/models/gpt2/test_modeling_gpt2.py` times, which includes a mix of tests with and without `from_pretrained`, on my machine:
- commit before #36033: 25.41s
- `main`: 61.14s
- this PR:  21.78s 💪 (this means #36033 resulted in 15%+ test speedup, after this fix)




Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36721). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[tests] Parameterized `test_eager_matches_sdpa_inference`
Author: gante
Pull request: yes

# What does this PR do?

## Problem

`test_eager_matches_sdpa_inference` on `main` is running many nested test cases (48!). This is not only a bad practice but also slows down our workflow: when it breaks, we need to parse the test outputs to see which configuration(s) broke. An example:
<img width="925" alt="Screenshot 2025-03-11 at 18 38 46" src="https://github.com/user-attachments/assets/7a126a31-52cd-4b7b-a15e-33e1786760ec" />

Which expands into
<img width="1513" alt="Screenshot 2025-03-11 at 18 39 37" src="https://github.com/user-attachments/assets/0e74c5b6-daca-4511-9651-048f9f0b86b2" />

Did it crash because of a specific parameterization? Or because we are running many subtests in a test? Answering those questions is not clear atm.

## Fix

This PR replaces the test's `for` loops by `@parameterized.expand`, making sure the test name immediately identifies the test case.

In the process, I've noticed many skips/overwrites are no longer needed. The test is still super ugly, and I've left a few TODO for whenever we decide to touch the test again.

An example -- what's being tested is now clear, minimal difference in run time ✨ 
<img width="1600" alt="Screenshot 2025-03-11 at 18 42 19" src="https://github.com/user-attachments/assets/3b164d0d-8f78-46cc-91fa-b99125e56e47" />


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36650). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

One more nit: for the name in the test, we can make even short by `attention --> attn`
Handling an exception related to HQQ quantization in modeling
Author: MekkCyber
Pull request: yes

# What does this PR do?

Adds exception handling to allow HQQ quantization to proceed when parameter names don't match between the model and state dict. It's due to the fact that in HQQ we only replace the linears on the fly by the hqqlinears, and this is because of some internal hqq considerations (to quantize each layer with a different precision...)

## Who can review ?
@SunMarc 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36702). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Not sure where else to mention this, but the new `_infer_parameter_dtype` function broke the `HIGGS` quantization method because it thinks that `weight` in the state dict should be directly cast to the`int16` dtype of the quantized `weight` buffer in `HiggsLinear`. This `_infer_parameter_dtype`shall be reconsidered.

cc @MekkCyber maybe we should think of a better to deal with that ? 

Can you share a quick reproducer @BlackSamorez ? 

@SunMarc 
```
from transformers import AutoModelForCausalLM, HiggsConfig

PATH = "unsloth/Llama-3.2-1B"

model = AutoModelForCausalLM.from_pretrained(
    PATH,
    quantization_config=HiggsConfig(
        modules_to_not_convert=["embed_tokens"],
    ),
    device_map="balanced",
)
```
raises
```
RuntimeError: Error(s) in loading state_dict for HiggsLinear:
	size mismatch for weight: copying a param with shape torch.Size([2048, 8192]) from checkpoint, the shape in current model is torch.Size([512, 8192]).
```
because the tensor being loaded is a full model weight cast to Int16 while the allocated buffer is 4x smaller for Int4 values densely packed into Int16.
Fix post_init() code duplication
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. The loop got duplicated in https://github.com/huggingface/transformers/pull/36677 but it should not be

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36727). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.