noamaanMulla-03/transformers-issues huggingface dataset:
Test: generate with `torch.compile(model.forward)` as a fast test
Author: gante
Pull request: yes

# What does this PR do?

Follow-up to #34464 

This PR:
1. Converts `test_generate_compile_model_forward` to a fast test. This means we will check generate with `torch.compile(model.forward)` at each commit on ALL models that support `StaticCache` 💛 
2. Fixes failing cases of `test_generate_compile_model_forward` whenever possible
3. Tags models with `_supports_static_cache = False  #Reason` when the model doesn't support `torch.compile(model.forward)`

__________________________

✅  `py.test tests/models/ -k test_generate_compile` is all green, takes ~2 mins to run on all models on my machine

Comments:
> Q: Is it really fast ...?

@ydshieh yes :D 
![Screenshot 2024-10-31 at 18 41 14](https://github.com/user-attachments/assets/69a0c89e-51fa-4d1c-a8f7-f3168473885b)


The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34544). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Could you just make sure it's run 

We need to remove `@require_torch_gpu` too for `def test_generate_compile`

Before merge, feel free to ping me for a check for (if there is any) flakyness :-) or anything you think I can double check again.

(sorry, the PR is not ready yet, a few cases are still failing 👀 I didn't mean to request a review)

Now it's working on all models, including encoder-decoder + cache 🤗 

It's not too heavy on our CI, it should add ~2 mins if all models are run. And it should prevent us from many headaches! As we can see in diff, we had compilation enabled for a bunch of models that don't support it.

![Screenshot 2025-01-28 at 12 48 05](https://github.com/user-attachments/assets/bfe180e8-d5ec-46a2-a537-c76ce61960d9)
Add rust to deepspeed image so it can build tokenizers
Author: ivarflakstad
Pull request: yes

# What does this PR do?
The deepspeed image build CI has been [failing](https://github.com/huggingface/transformers/actions/runs/13001128582/job/36259855640) because the base image is not able to build tokenizers.
This PR adds rust to the image so that it can get past this error.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35932). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks. I will take a look as I didn't see why other docker images are successfully built and CI are run with them without problem.

it turns out that deepseed docker file should use a base image with python >= 3.9 for which I forgot to update. I will update that and the changes in this PR shouldn't be necessary.

Thank you anyway.
use torch constraints to check if covariance is positive definite during mean resizing.
Author: abuelnasr0
Pull request: yes

# What does this PR do?

this PR implements this solution https://github.com/huggingface/transformers/issues/35075.
using torch constraints to check if the covariance is positive definite.


cc: @Rocketknight1


Comments:
There's a CI issue that I can't fix with a retrigger, though - can you push an empty commit, or make a change and then revert it, to trigger the CI again? After that the tests should be green and we can merge

Going to merge this as soon as I can get those flaky tests to pass - they're not your fault, it's just CI issues!

Sorry for the endless CI issues here - we'll keep trying and rebasing and merge as soon as it's green.

merged as the failing test is mentioned being flaky
Fix mask slicing for models with HybridCache
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Models with HybridCache need to correctly slice the key/value states when using FA2 as inputs needs to be unpadded on the right as well (and the mask has shape [bs, seq_len]). Moreover, mask slicing was wrong in all cases when the sequence length is larger than the sliding window. It is currently broken and leads to garbage generation when using padding. This fixes it.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35681). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @ArthurZucker slicing is now correct for all length (beyond sliding window) and attention functions. To not break dynamo tracing, the simplest IMO is to propagate the necesary `int` to slice (I initially wanted to only propagate though `kwargs`, but gradient checkpointing complained, so made a new explicit keyword argument). The other solution as discussed would be to pass a full other mask, but the logic and args to add would be similar (passing the new mask instead of just the int and slicing).
Let me know what you think of this solution. It's annoying to have to add a keyword arg just for that, but I reckon there are no other solutions that do no break tracing 🥲

PS: slow tests are similar as on main for both models (and the new tests showing full equivalence all pass)
lr_scheduler needs to be at the epoch level
Author: guidothekp
Pull request: no
Labels: name: bug

### System Info

In the page: https://huggingface.co/docs/transformers/training, 

note the following blurb.

Isn't lr_scheduler updated per epoch? The code below calls lr_scheduler.step() per batch in stead of per epoch. So the line needs to be moved out to the outer for loop.

```
from tqdm.auto import tqdm

progress_bar = tqdm(range(num_training_steps))

model.train()
for epoch in range(num_epochs):
    for batch in train_dataloader:
        batch = {k: v.to(device) for k, v in batch.items()}
        outputs = model(**batch)
        loss = outputs.loss
        loss.backward()

        optimizer.step()
        lr_scheduler.step()
        optimizer.zero_grad()
        progress_bar.update(1)
```

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Follow the steps in the page above.

### Expected behavior

```
from tqdm.auto import tqdm

progress_bar = tqdm(range(num_training_steps))

model.train()
for epoch in range(num_epochs):
    for batch in train_dataloader:
        batch = {k: v.to(device) for k, v in batch.items()}
        outputs = model(**batch)
        loss = outputs.loss
        loss.backward()

        optimizer.step()
        optimizer.zero_grad()
        progress_bar.update(1)
    lr_scheduler.step()
```

Comments:
Learning rate scheduler updates can be either per-epoch or per-batch depending on configuration. However, modern large models often train on massive datasets with very few epochs (typically 1-3 epochs). In such cases with limited epochs, epoch-based scheduling can be problematic as it provides very few adjustment points for the learning rate. To address this, many modern training approaches calculate the total number of training steps upfront and update the learning rate based on global steps instead of epochs. This is why you'll often see lr_scheduler.step() called within the batch loop rather than the epoch loop in modern training code.
Update `unwrap_and_save_reload_schedule` to use `weights_only=False`
Author: ydshieh
Pull request: yes

# What does this PR do?

Our CircleCI already has torch 2.6. With this torch version, We need to use `weights_only=False` explicitly if the checkpoint contains other stuffs than weights.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35952). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update `squad_convert_example_to_features` to work with numpy v2
Author: ydshieh
Pull request: yes

# What does this PR do?

We get `numpy==2.1.2` now instead of `numpy==1.26.3` on our CircleCI runner env. (likely due to the `torch==2.6` is out).
We need to use `np.atleast_1` now for a scalar, as

> span["input_ids"] == tokenizer.pad_token_id
is `False` in many case.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35955). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Cannot import name 'LRScheduler' from 'torch.optim.lr_scheduler'
Author: maverick-2030
Pull request: no

Run the following command and get an error

<img width="361" alt="Image" src="https://github.com/user-attachments/assets/9565b42c-6aba-486b-8713-f05766ca5acb" />

RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
cannot import name 'LRScheduler' from 'torch.optim.lr_scheduler' 

How can I fix this issue? Any suggestion welcome!

Attach pytorch and transformer version information

<img width="411" alt="Image" src="https://github.com/user-attachments/assets/94eabc8a-f650-41bb-bbf4-7e2f21560730" />

Comments:
Hi @maverick-2030, our support for Torch 1.x is very limited at this point. I recommend upgrading to 2.x and seeing if that resolves the issue!
Output dicts support in text generation pipeline
Author: jonasrohw
Pull request: yes

# What does this PR do?

It is a minor fix to the text generation pipeline. When calling with the generation argument `return_dict_in_generate=True,` the code breaks because it does not expect a dict from the `model.generate(...)` call. If you want to view logits, for example, a dict is required. This fix can handle `return_dict_in_generate=True` as a pipeline parameter.

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@Rocketknight1




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35092). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Rocketknight1 Hi. Thanks for looking at the PR and pointing that out. I agree with you; I got a bit ahead of myself. I've removed most of the changes and just made the changes to support dictionary output. I'd be happy if you could take a look. Thanks again!

@Rocketknight1 Any update on this? Thanks!

@gante Friendly reminder if you could take a look. Thank you very much!

@Rocketknight1 Can't seem to get a hold of @gante. Any idea how to proceed? 

Hang on, I'll get him!

@Rocketknight1 Any success in finding him?

@jonasrohw I exist, but I'm being pinged in many places :D 

(fixed conflicts, will merge as soon as CI gets green)

(failing tests are unrelated to this PR, and are being fixed in other PRs. Waiting for them to be merged first)
Trainer Refactor: Part 1
Author: muellerzr
Pull request: yes

# What does this PR do?

This is the first chunk in my attempt to clean up the trainer **without** breaking core capabilities (and integration capabilities/projects).

The aim is for *readability* rather than enhancement of features. I've started this process by:
* Refactoring how we get the "core" training values (max_steps, update_steps_per_epoch, num_train_tokens) in a more flow-based pattern than before (e.g. grouping num_train_tokens together)
* All `*_steps` based setups for the `Trainer.state` are now a func inside of `TrainerState` (a user should only need to know that's being done, realistically)
* Similarly there is a `init_training_references` which transparently hints that things are being stored in the `state`
* I've also refactored the RNG state setters to a helper func, to reduce repeated code that will repeat as more device support comes. 
* Lastly I've simplified a bit of the accelerate-based setup items. 


Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@SunMarc 
@tomaarsen 
@Rocketknight1 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35567). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Doesn't seem to break anything in TRL yet 👌

@qgallouedec @tomaarsen going to be running Trainer tests thoroughly now to make sure all is good (was still that way but will do a just-in-case). However this has gotten :heavy_check_mark: so consider this the "testable final version" for us to see if anything breaks across packages :crossed_fingers: 

@muellerzr Sentence Transformers tests are running right now👍

Sentence Transformers tests are passing!

All green on the transformers GPU and Multi-GPU tests as well :heavy_check_mark:
Fix flaky `test_assisted_decoding_matches_greedy_search`
Author: ydshieh
Pull request: yes

# What does this PR do?

I just used @gante method to fix this flakyness, you are the best!

(run 1000 times)
previously: 1% failures 
this PR:       0%

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35951). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> was there a reason for not adding this VLM processor

Question @gante , but I guess it's just we got focus on the failed tests and fixed them and didn't try to do it for all tests that were not failing at that time.

I guess it won't hurt to add it for all models, since it will anyway fail for some VLMs, existing or new. If Joao agrees ofc 

Let me merge this one first, and wait @gante 's input for further changes in a separate PR.

@zucchini-nlp @ydshieh Yup, we should make models have their own generation config with custom defaults. 

But this won't be an easy change, btw 😅 Possibly each model would need a function in its `PreTrainedModel` class

@gante I think @zucchini-nlp only means applies the same changes you and me made (like in this PR) to all generation tests, and not saying change anything in the non-testing code.

Ah, I see.

But we should fix the underlying issue in the models too :D 

> applies the same changes to all generation tests,


Yep, exactly what I meant, as a better workaround until the default generation config is added. It is quite annoying otherwise to see new flakiness when a new VLM is added
update docker file `transformers-pytorch-deepspeed-latest-gpu`
Author: ydshieh
Pull request: yes

# What does this PR do?

Previously one has python 3.8, but that is dropped recently and cause docker image failing to build.

Just an update with a newer base image with python 3.10

image is now built : https://github.com/huggingface/transformers/actions/runs/13015529608/job/36303666404



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35940). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ivarflakstad Thank you for the review ❤️ 

Since @muellerzr maintains `deepspeed CI` within `transformers`, I will wait a bit for they to also has a look before merging.
Byebye `test_batching_equivalence`'s flakiness
Author: ydshieh
Pull request: yes

# What does this PR do?

Au revoir flakyness 👋 

Some tests are decorated with `is_flaky` but with a clear reason. Some `is_flaky` are removed as they are no longer flaky.

**For reviewers**: The main changes are in `tests/test_modeling_common.py`.

## The major ways of the fix

- ensure the norm layers get patched by `set_xxx_less_flaky` methods
- don't use the cosine similarity in the equivalence check. It's bad as:
  - with the very small values (sometimes we get `1e-20` or `1e-39`) and the `eps=-38` used, the cosine similarity is not in the range of `[-1, 1]` and it's not stable.
  - the cosine similarity doesn't take magnitude into account

### Still some flakyness

- when a model using `max()` or `topk` methods to compute some indices and use them in later computation, this is not stable (small difference even in the order of `1e-9` could still get difference indices and cause later values to differ a bit larger)
- Timm's backbone has somehow different weight initialization and it may sometimes give larger intermediate values (so the difference would be larger too)
- `AutoformerAttention`: it uses `topk`, but also the computation of `tmp_delay` is wrong I believe (it won't keep the batch equivalence - I am sure about that, but not sure that is the original design)


Comments:
run-slow: vit

run-slow: vit

This comment contains run-slow, running the specified jobs: ['models/vit'] ...
[generate] move max time tests
Author: gante
Pull request: yes

# What does this PR do?

2-in-1 PR:
1. Moves the tests regarding `generate()`'s `max_time` flag from specific models to the generation integration tests. It is a `generate` feature, and not a model feature
2. Fixes the test -- e.g. the one on `gpt2` had an incorrect assertion

(found while working on #35802 )

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35962). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ydshieh it is not 🙈 moving to the right class

Thank you for noticing it!
`tokenizer` should be replaced to `processing_class` in `Seq2SeqTrainer`?
Author: zzaebok
Pull request: no
Labels: name: Core: Tokenization

name: trainer

name: bug

### System Info

- `transformers` version: 4.47.1
- Platform: Linux-5.4.0-200-generic-x86_64-with-glibc2.31
- Python version: 3.10.16
- Huggingface_hub version: 0.27.0
- Safetensors version: 0.4.5
- Accelerate version: 1.2.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 2070 SUPER

### Who can help?

@amyeroberts @ArthurZucker 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

In [`trainer_seq2seq.py`](https://github.com/huggingface/transformers/blob/5c75087aeee7081025370e10d1f571a11600f1ae/src/transformers/trainer_seq2seq.py#L372) file, there is still calling `self.tokenizer.` which produces deprecation warning "Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead."

```python
def _pad_tensors_to_max_len(self, tensor, max_length):
    if self.tokenizer is not None and hasattr(self.tokenizer, "pad_token_id"):
        # If PAD token is not defined at least EOS token has to be defined
        pad_token_id = (
            self.tokenizer.pad_token_id if self.tokenizer.pad_token_id is not None else self.tokenizer.eos_token_id
        )
```

### Expected behavior

I believe self.tokenizer should be replaced to self.processing_class

```python
def _pad_tensors_to_max_len(self, tensor, max_length):
    if self.processing_class is not None and hasattr(self.processing_class, "pad_token_id"):
        # If PAD token is not defined at least EOS token has to be defined
        pad_token_id = (
            self.processing_class.pad_token_id if self.processing_class.pad_token_id is not None else self.processing_class.eos_token_id
        )
```

Is it okay for me to make a PR for this issue? :smile:

Comments:
Thanks @zzaebok! Would you like to open a PR to fix this warning?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
🌐 [i18n-KO] Translated `altclip.md` to Korean
Author: cjfghk5697
Pull request: yes

# What does this PR do?

Translated the `altclip.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
@jungnerd, @cjfghk5697, @yijun-lee, @mreraser

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR?

Comments:
cc @stevhliu