noamaanMulla-03/transformers-issues huggingface dataset:
[performance_optim] define flash attention mask on NPU device directly
Author: FightingZhen
Pull request: yes

# What does this PR do?

When using Flash Attention2 on Ascend NPU, we have found that CPU memory keep increasing when calling func `npu_flash_attn_varlen_func` or `npu_flash_attn_func`. 

The root cause is that the attention mask generated by func `torch.ones()` is initially defined on the CPU side, occupying CPU memory before being transferred to the NPU device. As the func `npu_flash_attn_varlen_func` or `npu_flash_attn_func` is called repeatedly, the CPU memory consumption continues to accumulate, which is not optimal solution. Below is one example: https://github.com/huggingface/transformers/blob/12f65ee7520404b511c7fe716bc5d48d93d8297d/src/transformers/integrations/npu_flash_attention.py#L225

Therefore, this PR is committed for solving this problem by defining attention mask tensor with `torch.ones()` on NPU device directy.

Fixes # (issue)
Not releated.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@MekkCyber @SunMarc please help me review it, thanks : )

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37698). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@MekkCyber @SunMarc I have rebased the code to `main` branch and repushed it, I am not clear why `build_pr_documentation` CI pending, and the CI error seems not related with my PR, and  can you please help me merge this PR? thanks : )

@MekkCyber @SunMarc All CI passes and this PR seems to be ready for merge :)

Merged 🎊 ! Thanks a lot
[Bugfix] Fix flash-attention func param mismatch and softmax_scale default value mistake on Ascend NPU
Author: FightingZhen
Pull request: yes

# What does this PR do?
After we support using Flash Attention on Ascend NPU[(PR)](https://github.com/huggingface/transformers/pull/36696), we found 2 bugs in the implementation of Flash Attention on Ascend NPU. This PR is committed for solving both.

**Bugfix1:** 
We found that there exists a kind of situation in the code of `main` branch, where func `flash_attn_varlen_func` is used **without key-value style params passing**, like following code: https://github.com/huggingface/transformers/blob/3bc44eaaeee01b7f0d2d55c9991900b43cafe62d/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L203

In that case, params order between func `flash_attn_varlen_func` in package `flash-attn` and func `npu_flash_attn_varlen_func` in package `transformers` **is not aligned**, which may cause unexpected errors 😞 

Therefore, we solve this problem by aligning mismatch params `max_seqlen_q` and `max_seqlen_k`.

At the same time, we have also checked the func `npu_flash_attn_func`, and it does not have param order mismatch problem.

**Bugfix2:**
For func `npu_flash_attn_func` and `npu_flash_attn_varlen_func`, when param `softmax_scale` is set to `None`, it should be set to `1.0 / sqrt(q.shape(-1))` as default value. 

Fixes # (issue)
Not related.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@ArthurZucker please help me review and merge it, thanks~
eval_loss not found when training a peft model using trainer.py / losses not retrieved from base model where appropriate
Author: kreil
Pull request: no
Labels: name: bug

### System Info

pip list |grep transf; python --version; uname -a

transformers              4.51.3
Python 3.12.10
Linux gap 6.11.10+bpo-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.11.10-1~bpo12+1 (2024-12-19) x86_64 GNU/Linux


### Who can help?

@zach-huggingface @SunMarc

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Train any peft model. I am training a LoRA adapter using the AutoModelForMaskedLM from ModernBERT.

### Expected behavior

Trainer and its most regular features should handle looking up the losses in the base model, which at several places in the code they apparently don't. So instead of finding `eval_loss` they only see the new losses added by `peft` such as `eval_steps_per_second`.

One such issue is described, including a fix, in #33420. This fix should be merged with the regular sources.

The second such issue, triggered by setting `load_best_model_at_end` to True, is described in the comments to #33420 (no fix yet).


Comments:
cc @benjaminbossan and @SunMarc 

Hey @kreil, if you are share a reproducer, that would help us fix the bug. Happyy to review the PR if you have the solution also ! 

I also had this problem and solved it according to [#33420](https://github.com/huggingface/transformers/issues/38130) hopefully this pr will be merge

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

why this pr not merged? i meet this issue in transformers==4.52.4

"The second such issue, triggered by setting load_best_model_at_end to True, is described in the comments to https://github.com/huggingface/transformers/issues/33420 (no fix yet)."
yep! i think so ! when i use "load_best_model_at_end", i will have the same problem. Has anyone solved this problem?

 > The second such issue, triggered by setting `load_best_model_at_end` to True, is described in the comments to [#33420](https://github.com/huggingface/transformers/issues/33420) (no fix yet).

emmm, i saw the code, i found that we need to set `label_names` this args (by the way it is a List[str]) in `TrainingArguments`, and it will not emerge `"The metric_for_best_model training argument is set to 'eval_loss', which is not found in the evaluation metrics. The available evaluation metrics are: ['eval_runtime', 'eval_samples_per_second', 'eval_steps_per_second', 'epoch']. Consider changing the metric_for_best_model via the TrainingArguments."` again when we set `load_best_model_at_end` to True.
🌐 [i18n-KO] Translated `t5.md` to Korean
Author: taemincode
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `t5.md` file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 KREW 팀원들에게 리뷰를 요청하는 아래 주석을 노출해주세요!-->
May you please review this PR?
<!-- @jungnerd, @luckyvickyricky, @chelsseeey, @skwh54, @amo33, @maximizemaxwell, @D15M4S -->
<!-- @harheem, @nsbg, @Youngdong2, @xhaktm00, @ssunbear, @ChoHyoungSeo, @judy-choi -->
<!-- @4N3MONE, @Kim-Ju-won, @ahnjj, @FacerAin, @ssum21, @TaskerJang, @HyunZ118 -->
<!-- @yijun-lee, @songi104, @chhaewxn, @AhnJoonSung, @jihyun-0611, @seopp, @pyapyapya -->
@yijun-lee @harheem @4N3MONE @jungnerd

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. KREW 팀원들의 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
<!-- @stevhliu May you please review this PR? -->

Comments:
Fix CI: Use correct import for torchvision InterpolationMode
Author: manueldeprada
Pull request: yes

https://github.com/huggingface/transformers/pull/32317 renamed `from torchvision.transforms.v2 import functional as F_t` leaving `F` for `torch.functional`

but some usages of `F` remained unchanged, breaking CI ([example](https://app.circleci.com/pipelines/github/huggingface/transformers/142032/workflows/d0770631-92de-4f3c-96df-5832a7322bc2/jobs/1879488)):

```
FAILED tests/models/sam/test_image_processing_sam.py::SamImageProcessingTest::test_slow_fast_equivalence
...
E           AttributeError: module 'torch.nn.functional' has no attribute 'InterpolationMode'

/usr/local/lib/python3.9/site-packages/transformers/models/sam/image_processing_sam_fast.py:246: AttributeError
```

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: sam

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40160). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Continous Batching] set head_dim when config.head_dim is None
Author: kashif
Pull request: yes

# What does this PR do?

Some models have empty `config.head_dim` so we now check if the value is not None instead of setting it to `None`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40159). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[EfficientLoFTR] Fixed bugs and added original implementation bug
Author: sbucaille
Pull request: yes

# What does this PR do?

Fixes a couple of bugs introduced in the implementation of EfficientLoFTR.
Adds a cross attention bug introduced by the authors which makes the weights dependant on (see https://github.com/zju3dv/MatchAnything/issues/26)

## Who can review?

@qubvel 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40141). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I think it would be better to write the integration tests differently, because to my surprise, the tests passed, because we only look at the top k matches

run-slow: efficientloftr

This comment contains run-slow, running the specified jobs: 

models: ['models/efficientloftr']
quantizations: [] ...

> I think it would be better to write the integration tests differently, because to my surprise, the tests passed, because we only look at the top k matches

😄 hah!

Ok, I don't know what I did but indeed the tests don't pass, like on the custom tests, let me adjust them

run-slow: efficientloftr

This should be fixed now

run-slow: efficientloftr

This comment contains run-slow, running the specified jobs: 

models: ['models/efficientloftr']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: efficientloftr
Add model card for BERT Base Uncased
Author: Meghanap24
Pull request: yes

This PR adds a beginner-friendly model card for `bert-base-uncased` under `docs/source/model_cards/`.

Includes:
- Model description (plain language)
- Usage example
- Intended use & limitations
- Resources and references



Comments:
Hi @sbucaille, @qubvel. 
Please review my PR when you get a chance. Thank you!
Support "output_attentions" while using Flash Attention
Author: eslambakr
Pull request: no
Labels: name: Feature request

### Feature request

I am requesting to add support to return the "output_attentions" while using Flash Attention as it is currently not supported, where "output_attentions" is set to False (hard-coded) and ignores the configuration while using Flash Attention.

### Motivation

It is very useful for visualizing attention maps.
So, not supporting it during Flash attention limits a lot of applications where attention maps are needed.

### Your contribution

Sorry, I can not support an implementation for it as I don't understand why the attention maps in the Flash attention case are untrustworthy (#28903).
I mean, they are already there, but as they are not trusted, [they were set to "None"](https://github.com/huggingface/transformers/blob/1bd604d11c405dfb8b78bda4062d88fc75c17de0/src/transformers/models/llama/modeling_llama.py#L431).



Comments:
Hey! This is inherently not supported by flash attention! That is because to efficiently compute the final attention output, you don't materialize the attention scores! 

If you look at the calls to the `Flash Attention` kernels, they don't return the intermediate `attn_weights = softmax(Q @ K^T * softmax_scale)` but fused the entire `(softmax(Q @ K^T * softmax_scale) @ V)` 🤗 

Hi @ArthurZucker 

Thanks for your prompt response!
I got it. That's why it is okay to use it during training, but during testing, I can disable it to get the attention maps, and still, the attention maps should be accurate.

Thank you so much for the clarification!

Hi @eslambakr ! I am having the same trouble, and may i ask how did you eventually get the attention maps if flashattn2 is not allowed?

You can calculate attention map by hands if your L is not very big.
build: Add fast image processor tvp
Author: adutchengineer
Pull request: yes

Adds TVP from https://github.com/huggingface/transformers/issues/36978
cc: @yonigozlan


Comments:
Once this one is approved, I will also get started on Vivit :) 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, tvp

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39529). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add GptOssForSequenceClassification for GPT-OSS models
Author: zyfedward
Pull request: yes

# What does this PR do?

Add GptOssForSequenceClassification for GPT-OSS models following generic practice for #40050 

Testing:
- Tested with single-node (8 A100) and multi-node (16 A100) options, loss curve and AUC are all expected with no issue.
- All testes passed
- `make fixup` passed

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker @Rocketknight1



Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, gpt_oss

> Make sure tests are unrelated, otherwise LGTM

Confirmed unrelated, no error found in circleci but timeout: `Too long with no output (exceeded 10m0s): context deadline exceeded`.

I’m training with GptOssForSequenceClassification using the Hugging Face Trainer, and I’ve noticed that both the loss and grad norm are very large. I suspect the losses are not being averaged properly.
For example, when I train on 32 × H20 GPUs, the loss is about 4× larger than when I train on 8 × H20 GPUs.
Has anyone encountered a similar issue?
Environment:
transformers==4.56.2
accelerate==1.10.1
Standardize BARTpho model card: badges, new examples, fixed broken im…
Author: eshwanthkartitr
Pull request: yes

Updated bartpho.md

# What does this PR do?

Fixes # (issue)
Standardize BARTpho model card


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Please review this @stevhliu



Comments:
Hi @stevhliu,

Thank you for your dedication to maintaining high-quality documentation across the repository. I've standardized the BARTpho model card with several key improvements:

**Key Changes:**
- Standardized model card badges for consistency with other model documentation
- Added new practical examples to help users better understand implementation
- Fixed broken links and images that were impacting user experience

I'd appreciate your feedback on whether any additional elements would enhance the documentation further - specifically, would an architecture diagram improve clarity for users? I'm also open to any other suggestions you might have to make this model card even more valuable for the community.

Looking forward to your review and any insights you can share!

Best regards

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40051). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix misspelled issues 
Author: ganlerseian
Pull request: yes

# What does this PR do?

fixed spelling issues 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: conditional_detr, maskformer, perception_lm, speech_to_text

@ganlerseian can you run `make fix-copies` to fix the repo consistency checks? After that you can ping me and I'll merge it
Pin torch to 2.7.1 on CircleCI for now
Author: ydshieh
Pull request: yes

# What does this PR do?

It's possible torch 2.8 causing much more `Too long without output` on CircleCI jobs.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40174). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update dynamic attnt setter for multimodals
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Dynamic attn setting doesn't work for multimodal currently and also for any other code where the attention layer is imported, and not defined. This PR updates the heuristic to check if an attention layer exists in the model code as well.

Also updated the warning when requested attention wasn't found to `ValueError`. Most ppl don't read warnings and raising an explicit error is more useful.

The tests are now running on 36 multimodal model classes and are all green 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39908). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Alright, @zucchini-nlp I needed this so I took the liberty to push some changes to fix the failing test (sam2 was failing) and make it slightly more robust at the same time! Everything green now, so merging! Thanks again for the PR!! 🤗
Update utils.py: fix nan
Author: MengAiDev
Pull request: yes

# What does this PR do?
- Update utils.py: fix nan
- Suggested in https://github.com/huggingface/transformers/pull/40093/commits/7e230e5464705e6e235aec47d8a7312a3404ba15


 @ArthurZucker @zucchini-nlp 


Comments:
Hi @MengAiDev 👋 Thank you for opening this issue 🤗 

Agreed with @zucchini-nlp. Before considering code changes, we need to understand the problem -- even if we end up merging your suggestion as-is :)

Questions:
1. Can you share a short stand-alone reproducer for the issue? If not (e.g. private model), can you share which operation introduces the `nan`?
2. Have you tried running generate with `remove_invalid_values=True`? (this removes `nan` in the logits)

I see, but this is just I try to fix the circle ci's failed, and I actually don't have the way to propose it., Sorry.