noamaanMulla-03/transformers-issues huggingface dataset:
Add resume checkpoint support to ClearML callback
Author: aarbelle
Pull request: no
Labels: name: Feature request

### Feature request

Link ClearML task when resuming from checkpoint

### Motivation

Currently, when resuming from checkpoint Clearml overwrites the task and starts a new one. A lot of the data  and history is lost.

### Your contribution

I will try to see if I can fix it and submit a PR

Comments:
Hi! @aarbelle 
Are you still working on this feature request?
I've already written some code about this, and I can open up a pull request for you.
The parameter 'text' may be None as the comments says, there is a confuse.
Author: ralgond
Pull request: no

The comments of method "\_\_call\_\_" say:

        Main method to prepare for the model one or several sequences(s) and audio(s). This method forwards the `text`
        and `kwargs` arguments to Qwen2TokenizerFast's [`~Qwen2TokenizerFast.__call__`] if `text` is not `None` to encode
        the text. To prepare the audio(s), this method forwards the `audios` and `kwrags` arguments to
        WhisperFeatureExtractor's [`~WhisperFeatureExtractor.__call__`] if `audios` is not `None`. Please refer to the doctsring
        of the above two methods for more information."

but here "text" must be not None:

https://github.com/huggingface/transformers/blob/7652804d237fb8768f0f0b8129a05e4f0576114b/src/transformers/models/qwen2_audio/processing_qwen2_audio.py#L106

Comments:
I agree the text is a little unclear! Would you be willing to write a PR to update the docstring?

> I agree the text is a little unclear! Would you be willing to write a PR to update the docstring?

This is a hard work, I haven't figured out how the whole method works yet. Why don't we find the original author and ask him to revise it?

I can polish the description a little bit. @ralgond 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
🌐 [i18n-KO] Translated `prompting.md` to Korean
Author: chhaewxn
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `prompting.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
<!-- @jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave -->
<!-- @win2dvp21, @enchantee00, @fabxoe, @1kmmk1, @junejae -->
<!-- @SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd -->
<!-- @mjk0618, @cjfghk5697, @ahnjj, @JinukHong, @nuatmochoi -->
@010kim, @chhaewxn, @boyunJang, @jeongiin, @harheem 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32294). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Thanks for your contribution! Just need to resolve the conflict then we can merge 🤗

@stevhliu I have resolved the conflict. Thank you🤗
Facing RunTime Attribute error while running different Flax models for RoFormer
Author: ctr-pmuruganTT
Pull request: no
Labels: name: Flax

name: bug

when running FlaxRoFormerForMaskedLM model, I have encountered an issue as 

>  AttributeError: 'jaxlib.xla_extension.ArrayImpl' object has no attribute 'split'.

This error is reported in the file `transformers/models/roformer/modeling_flax_roformer.py:265`

The function responsible for this error in that file is as below

```
def apply_rotary_position_embeddings(sinusoidal_pos, query_layer, key_layer, value_layer=None):
      sin, cos = sinusoidal_pos.split(2, axis=-1)
```

While changing this particular line from `sinusoidal_pos.split(2, axis=-1)` to `sinusoidal_pos._split(2, axis=-1)` , I didn't get that error

My observation is when I replace `split()` with `_split()` , my issue is resolved

### System Info
My environment details are as below :

>  - `transformers` version: 4.49.0
> - Platform: Linux-5.4.0-208-generic-x86_64-with-glibc2.35
> - Python version: 3.10.12
> - Huggingface_hub version: 0.29.3
> - Safetensors version: 0.5.3
> - Accelerate version: not installed
> - Accelerate config: not found
> - DeepSpeed version: not installed
> - PyTorch version (GPU?): 2.6.0+cu124 (False)
> - Tensorflow version (GPU?): not installed (NA)
> - Flax version (CPU?/GPU?/TPU?): 0.10.2 (cpu)
> - Jax version: 0.4.36
> - JaxLib version: 0.4.36

I am attaching a screenshot for reference 

<img width="1642" alt="Image" src="https://github.com/user-attachments/assets/a488444c-6095-4fc5-a5a0-bc400409d8ba" />


### Who can help?

@gante @Rocketknight1 

I am facing this issue for Models like 

> FlaxRoFormerForMultipleChoice

> FlaxRoFormerForSequenceClassification

> FlaxRoFormerForTokenClassification

> FlaxRoFormerForQuestionAnswering

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Steps to recreate the error:

Run the below code in any python editor

```
from transformers import AutoTokenizer, FlaxRoFormerForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("junnyu/roformer_chinese_base")
model = FlaxRoFormerForMaskedLM.from_pretrained("junnyu/roformer_chinese_base")

inputs = tokenizer("The capital of France is [MASK].", return_tensors="jax")

outputs = model(**inputs)
logits = outputs.logits
```

### Expected behavior

The model should run and produce error free output

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Need Option to Disable Flash Attention in VideoLLaMA2.1-7B-AV (SiglipVisionModel)
Author: harshmoothat
Pull request: no

When using SiglipVisionModel inside VideoLLaMA2.1-7B-AV, I encounter the following error:

**ValueError: SiglipVisionModel does not support Flash Attention 2.0 yet.**

I do not need Flash Attention for my use case and would like to disable it.
Could you provide an official way to toggle it off?

![Image](https://github.com/user-attachments/assets/9bbf32b3-6cfa-4e20-bdff-e58812be52ee)

Comments:
Hmm Siglip does support FA2 as per this code block, Ca you check transformers version, and that FA2 was available for Siglip in that version? If not update to the latest

https://github.com/huggingface/transformers/blob/main/src/transformers/models/siglip/modeling_siglip.py#L439

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I'm encountering the same issue when using VideoLLaMA2.1-7B-AV

Video Llama is not maintained by transformers and is hosted with custom code. I see now that it has q requirement to use an older transformers version (siglip support FA2 was added only in transformers v4.43.0). If you cannot update the version, you can disable the attention implementation with `XXXModel.from_pretrained(attn_implementation="sdpa")`, or using video-llama repo it should be doable with `model_init(MODEL_PATH, attn_implementation="sdpa")`

I'll close this issue as resolved, since FA2 is supported in new versions and the problem stems from custom code
enable 6 gemma2 cases on XPU
Author: yao-matrix
Pull request: yes

**2 pass**
PASSED tests/models/gemma2/test_modeling_gemma2.py::Gemma2IntegrationTest::test_export_static_cache
PASSED tests/models/gemma2/test_modeling_gemma2.py::Gemma2IntegrationTest::test_generation_beyond_sliding_window_3_eager

**1 should skip: XPU doesn't support flash-attention-2 package as of now**
SKIPPPED tests/models/gemma2/test_modeling_gemma2.py::Gemma2IntegrationTest::test_generation_beyond_sliding_window_0_flash_attention_2   

**2 flex_attention failed, already in development plan**
FAILED tests/models/gemma2/test_modeling_gemma2.py::Gemma2IntegrationTest::test_generation_beyond_sliding_window_2_flex_attention
FAILED tests/models/gemma2/test_modeling_gemma2.py::Gemma2IntegrationTest::test_model_2b_pipeline_bf16_flex_attention

**1 repetition output when out seq is long, under debug, will submit separate PR if needed**
tests/models/gemma2/test_modeling_gemma2.py::Gemma2IntegrationTest::test_generation_beyond_sliding_window_1_sdpa


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@yao-matrix I have seen the `run-test` is not triggered. Could you click "Convert to draft" then "ready for review"? I am just trying to see what might cause this situation 🙏 

> @yao-matrix I have seen the `run-test` is not triggered. Could you click "Convert to draft" then "ready for review"? I am just trying to see what might cause this situation 🙏

done, thx
enable 6 modeling cases on XPU
Author: yao-matrix
Pull request: yes

**4 PASSED**
PASSED tests/models/mpt/test_modeling_mpt.py::MptIntegrationTests::test_model_logits
PASSED tests/models/nemotron/test_modeling_nemotron.py::NemotronIntegrationTest::test_nemotron_8b_generation_eager
PASSED tests/models/bamba/test_modeling_bamba.py::BambaModelIntegrationTest::test_simple_batched_generate_with_padding
PASSED tests/models/bamba/test_modeling_bamba.py::BambaModelIntegrationTest::test_simple_generate

**2 FAILED**
FAILED tests/models/gemma/test_modeling_gemma.py::GemmaIntegrationTest::test_compile_static_cache, will fix in separate PR
FAILED tests/models/nemotron/test_modeling_nemotron.py::NemotronIntegrationTest::test_nemotron_8b_generation_sdpa, will fix in separate PR

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @yao-matrix, we probably don't want to add tests that we know are failing! I think it works better if you only add/enable them in the PR where you fix them. cc @ydshieh

> Hi @yao-matrix, we probably don't want to add tests that we know are failing! I think it works better if you only add/enable them in the PR where you fix them. cc @ydshieh

@Rocketknight1 , my current practice is:
1. If the cases failed and the bug is in Hugging Face libraries or testing adaptation, I will enable them and fix them.
2. If the cases failed and we triaged the bug is in Hugging Face dependent libraries(for example: PyTorch XPU op issue), I will discuss with our PyTorch team and ask them to fix. In this case, I'll enable the case in Hugging Face and expose it to our PyTorch team or op team for reproduce. And after their fix, the failures will disappear by only upgrading PyTorch, so it's OK.

Let me know if this philosophy make sense to you, thx.

I think what @Rocketknight1 originally intend to mean is that we don't want to add tests that we know are failing on our CI. But what @yao-matrix does is to enable tests for XPU, and the mentioned `2 FAILED` are failure on XPU.

Their work won't introduce new failure on our CI runs as we only runs on GPU (Nvidia / AMD).
fix 2 encoder_decoder issues on XPU
Author: yao-matrix
Pull request: yes

PASSED pytest -rA tests/models/encoder_decoder/test_modeling_encoder_decoder.py -k test_roberta2roberta_summarization 
PASSED pytest -rA tests/models/encoder_decoder/test_modeling_encoder_decoder.py -k test_bert2gpt2_summarization

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @ydshieh

Thank you
Include "time" as option to save_strategy (and log and eval too!)
Author: davidhughhenrymack
Pull request: no
Labels: name: Feature request

### Feature request

When building a training config for trainer I'd love to be able to do something akin to:

``
training_args = SFTConfig(
        eval_strategy="time",
        eval_minutes=15,
        save_strategy="time",
        save_minutes=30
)
``

### Motivation

My motivation for this is that I'm always using steps to proxy time, but this is painful when I run on different GPUs or most commonly run the code locally (Macbook pro) to check it runs, then run it remotely on a beefy GPU.

For saving I want to "not lose significant amounts of work" and for me this is always time based (E.g. I don't want to wait for 6 hours but see the model crash and loose all progress at hour 5) since both my waiting and my paying for compute is time based. Similarly for eval, I don't want to eval every minute (expensive!) but do want to eval often enough to know the model is still progressing. Time based solves this.

### Your contribution

I'm down to write code for this! I don't know the deeper architectural considerations so greatly welcome the community's insights. Thanks :)

Comments:
cc @sunmarc @muellerzr 

That could be a nice feature if you feel like steps or epoch are sometimes a bit tricky to set correctly. Feel free to submit a PR that that. 

Hi @davidhughhenrymack !
Are you still working on this issue?
If not, can I claim this issue?

It’s all yours! 💕I got swallowed up into other reinforcement learning stuff
On Mon, Apr 7, 2025 at 11:52 AM Minjun Zhu ***@***.***> wrote:

> Hi @davidhughhenrymack <https://github.com/davidhughhenrymack> !
> Are you still working on this issue?
> If not, can I claim this issue?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/36310#issuecomment-2784263778>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADSQKIR4W5PVW4KGKWWFBET2YLCPPAVCNFSM6AAAAABXRQH25GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOBUGI3DGNZXHA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
> [image: ailunc]*ailunc* left a comment (huggingface/transformers#36310)
> <https://github.com/huggingface/transformers/issues/36310#issuecomment-2784263778>
>
> Hi @davidhughhenrymack <https://github.com/davidhughhenrymack> !
> Are you still working on this issue?
> If not, can I claim this issue?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/36310#issuecomment-2784263778>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADSQKIR4W5PVW4KGKWWFBET2YLCPPAVCNFSM6AAAAABXRQH25GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOBUGI3DGNZXHA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


Hi! @davidhughhenrymack @Rocketknight1 @SunMarc Allen and I just submitted a PR to implement this feature. Will be grateful if you could help us review it!
Adding Paged Attention to Qwen1.5-MoE-A2.7B-Chat models using PyTorch XLA and Pallas
Author: ranwangmath1988
Pull request: no
Labels: name: Feature request

### Feature request

[Qwen1.5-MoE-A2.7B-Chat](https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B-Chat)(https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B-Chat) is one of the models that is closest to DeepSeek with improvement. In the current state, FlashAttention is only implemented for GPUs. On the other hand, for TPU training, there has already been written [Paged Attention](https://pytorch.org/xla/master/features/pallas.html) for PyTorch. 

### Motivation

In general, TPU v5e and v6e have demonstrated significant cost efficiency for training large models. In addition, as language models become large, the [vLLM](https://github.com/vllm-project/vllm) library has been continuously developed to tackle low-level computation. Moreover, [Pallas](https://docs.jax.dev/en/latest/pallas/index.html) has been actively developed to allow better integration of Jax/Flax into the PyTorch-XLA ecosystem. Despite numerous efforts to perform inference, the possibility of training larger language models, combined with custom training methods such as LoRA, enables the use of TPU to train huge models. 

### Your contribution

I want to offer testing using Google Cloud TPU, including fine-tuning and other tasks. Unfortunately, currently, I do not know how to 

1. Use PagedAttention to replace regular multihead attention written in `nn.module`.
2. How to load the pre-trained weights into the Paged Attention module.

Comments:
[kernels] use original forward at compile time
Author: gante
Pull request: yes

# What does this PR do?

`kernels` and `torch.compile` are not yet compatible with each other. Although we can skip custom kernels when the package is not installed, adding an error message is also not feasible -- we can't throw exceptions at compile time.

This PR hijacks the `kernels` decorator to add a compile-friendly path: until `kernels` supports `torch.compile`, let's use the original `forward`. 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@qubvel I'm not sure how to implement the manual escape path (as discussed on slack and [here](https://github.com/huggingface/transformers/pull/36853#issuecomment-2743825833)) 🤔 

From the decorator's perspective, we only have access to `cls`. Some classes where the kernel is applied don't have any way we can set extra flags at model definition time ([e.g.](https://github.com/huggingface/transformers/blob/f9742143538cfeb65c3a25632b83276266819c41/src/transformers/models/phi3/modeling_phi3.py#L240)), so we can't do something like `model.disable_custom_kernels = False` with the decorator approach. Not unless we add a lot of extra code.

Using `DISABLE_KERNEL_MAPPING=1` should work, though.

Any suggestions or ideas?  

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37604). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@qubvel @Cyrilvallez let me know if you'd like further changes

(and sorry for being pushy -- CI is broken on several places until this is merged or we update a bunch of CI images)
Remove torchvision requirement from AutoImageProcessor
Author: LysandreJik
Pull request: yes


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37457). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Is it correct that the repetition penalty is applied to the input_ids encompassing all inputs and outputs, rather than solely on the generated tokens?
Author: Ostrichpie818
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.44.2
- Platform: Linux-3.10.0-1160.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.10.14
- Huggingface_hub version: 0.26.2
- Safetensors version: 0.4.5
- Accelerate version: 1.0.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.1+cu124 (True)

### Who can help?

@gante 

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

https://github.com/huggingface/transformers/blob/c4d4e8bdbd25d9463d41de6398940329c89b7fb6/src/transformers/generation_utils.py#L101    

    def enforce_repetition_penalty_(self, lprobs, batch_size, num_beams, prev_output_tokens, repetition_penalty):
        """repetition penalty (from CTRL paper https://arxiv.org/abs/1909.05858). """
        for i in range(batch_size * num_beams):
            for previous_token in set(prev_output_tokens[i].tolist()):
                # if score < 0 then repetition penalty has to multiplied to reduce the previous token probability
                if lprobs[i, previous_token] < 0:
                    lprobs[i, previous_token] *= repetition_penalty
                else:
                    lprobs[i, previous_token] /= repetition_penalty

    def postprocess_next_token_scores(
        self,
        scores,
        input_ids,
        no_repeat_ngram_size,
        bad_words_ids,
        cur_len,
        min_length,
        max_length,
        eos_token_id,
        repetition_penalty,
        batch_size,
        num_beams,
    ):
        # repetition penalty (from CTRL paper https://arxiv.org/abs/1909.05858)
        if repetition_penalty != 1.0:
            self.enforce_repetition_penalty_(
                scores, batch_size, num_beams, input_ids, repetition_penalty,
            )
    ...

### Expected behavior

Maybe it should just rely on generated tokens?

Comments:
I also found this to be problematic, especially when using generation function with cot prompts: repetition penalty will falsely penalize the phrase "let's think step by step" and even worse, causing weird strings to be  generated.

> I also found this to be problematic, especially when using generation function with cot prompts: repetition penalty will falsely penalize the phrase "let's think step by step" and even worse, causing weird strings to be generated.

I found the problem is gone when i use chat_template for the input.