noamaanMulla-03/transformers-issues huggingface dataset:
Add DBRX GGUF Support
Author: cjfghk5697
Pull request: yes

# What does this PR do?
Add DBRX GGUF loading support

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
to it if that's the case. Link: https://github.com/huggingface/transformers/issues/33260
- [x] Did you make sure to update the documentation with your changes? Here are the
[documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
[here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?
## Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Comments:
cc @SunMarc 

Do you want to finish this PR or I should assign it to someone else ? @cjfghk5697 

Hi @SunMarc,
I’m sorry, but I won’t be able to finish this work right now. My schedule has become too busy due to my business obligations. Thank you for your understanding.
🌐 [i18n-KO] Translated `blip-2.md` to Korean
Author: cjfghk5697
Pull request: yes

# What does this PR do?

Translated the `blip-2.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
@jungnerd, @cjfghk5697, @yijun-lee, @mreraser

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR? 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33516). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
🌐 [i18n-KO] Translated `blip.md` to Korean
Author: cjfghk5697
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `blip.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
@jungnerd, @cjfghk5697, @yijun-lee, @mreraser

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR? 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33515). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[bart] minor test fixes
Author: gante
Pull request: yes

# What does this PR do?

Makes `RUN_SLOW=1 py.test tests/models/bart/test_modeling_bart.py` green.

(Issues found while working on #35802)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35965). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Whisper: fix static cache CI
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Some tests started failing after https://github.com/huggingface/transformers/pull/35164. This PR fixes it by:
1. Removing manual forward compilation, as it is compiled again in the generation's `_sample`
2. Removes the overriden `prepare_inputs_for_generation` 

Ran tests:
- `pytest -k greedy tests/models` (moshi is failing even on `main`)
- `pytest  tests/models/whisper/test_modeling_whisper.py`


Comments:
Merging after tests are fixed on `main`

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35852). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Revert p_mask to a list in DQA pipeline
Author: Rocketknight1
Pull request: yes

We had two PRs to fix the same issue, #32076 and #32208. I don't think the conflict was noticed, and the fix in #32076 seems to cause issues! This PR tests reverting #32076 but keeping #32208 and seeing if that resolves everything.

Fixes #35948

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35964). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

The failing examples in both #35948 and #32208 work after this PR

Yep, this looks good - cc @Cyrilvallez for review!
adjust beam search early stopping to any criteria as opposed to all
Author: secrettoad
Pull request: yes

# This PR addresses https://github.com/huggingface/transformers/issues/34843, adjusting the requirement to early stop during generation via beam search to ANY of the supplied criteria as opposed to ALL

<!--

This is a proposed solution to the bug documented in https://github.com/huggingface/transformers/issues/34843. Any early stopping criteria being met should be enough to stop generation.

-->

<!-- Remove if not applicable -->

Fixes #34843 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?

@Rocketknight1 @gante 

Comments:
I'll take a look at the failing tests

PyTorch and Tensorflow equivalence test that failed on previous build seems to be non-deterministic, or at the least not reproducible or related to this change. Passing now

cc @gante @zucchini-nlp 

For beam search I'd recommend to wait for gante as he had some plans for general refactoring, he'll be back later next week.

@gante this seems like an easy fix to the bug mentioned. what do you think? 

This is actually not the fix -- we might hit the stopping criteria with one of the batch items, but the other batch items would not be finished. The change would also break the case when `early_stopping=False`

Have a look at #34843 for related discussion, and at #35802 for the proper fix :)
Less flaky for `TimmBackboneModelTest::test_batching_equivalence`
Author: ydshieh
Pull request: yes

# What does this PR do?

As in the (previous) comment, Timm model in `transformers` has no `_init_weights` method, and it relies the (some how different) timm's way of weight init.

This seems giving somehow larger values and differences. Decorated `is_flaky` and use `1e-5` still fails with 1% ratio.

Let's just use a `1e-4`


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35971). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

0% In the sense of running 2000 times.

(but when it comes to 1 millions times, I won't promise it's 0% 😆 )
Any plans for a Test-Time Compute Scaling Package or Module?
Author: August-murr
Pull request: no
Labels: name: Feature request

### Feature request

Last month, @lewtun , @edbeeching , and @srush posted this blog post about test-time compute scaling, which got me curious about whether there are any plans to create a Transformers module or a separate package for test-time compute scaling methods like self-consistency, best-of-n, beam search, etc.

### Motivation

Scaling compute at test time can be combined with Agentic workflows to improve productivity, as well as paired with reinforcement fine-tuning methods for o1 like performance.

### Your contribution

I am very interested in the development of these modules; however, I am skeptical about their place in the HF ecosystem, whether as a module or a package, and how they will interact with the HF ecosystem. Therefore, I would like to seek your opinions and feedback on this matter.

Comments:
I think for the time being this feature is best kept in an external repo. I think TRL would be a more suitable place, but the field is moving so fast that it would be hard to maintain and the TRL team is stretched as it is. We welcome contributions, of course.

@August-murr the transformers team will also be exploring this topic, but we haven't decided on the format -- notebooks with recipes? pipelines? something else?

One thing is for granted, a few parts of `transformers` will have to updated to accommodate these tools. E.g. we're vectorizing beam search [here](https://github.com/huggingface/transformers/pull/35802), which should help with beam-based techniques. 

> [@August-murr](https://github.com/August-murr) the transformers team will also be exploring this topic, but we haven't decided on the format -- notebooks with recipes? pipelines? something else?
> 
> One thing is for granted, a few parts of `transformers` will have to updated to accommodate these tools. E.g. we're vectorizing beam search [here](https://github.com/huggingface/transformers/pull/35802), which should help with beam-based techniques.

Yea, there were lots of ideas around but I guess R1 caught everybody off guard. although I honestly doubt **Reinforcement Fine Tuning**, which is what OpenAI is calling it, or just Reasoning with RL is gonna be the end of Test-time Compute Scaling. 

Heuristic algorithms that allow models to explore will be essential to AGI and It won't be limited to RL.
not to use A100 for `benchmark.yml`
Author: ydshieh
Pull request: yes

# What does this PR do?

From internally discussion

> yes technically we can use A100 on CI. I did the conf for @McPatate, but this WF only use 1 GPU, so i disabled this p4d instance for now because we wasted 7 GPU on it

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35974). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Handle empty change indices in SAM's mask to rle conversion
Author: MSt-10
Pull request: yes

# What does this PR do?
It handles empty change indices when the predicted mask is converted to rle. 
Previously this caused an exception.

Fixes #35664 

## Who can review?
@ArthurZucker 




Comments:
Hi @MSt-10! Thanks for the fix 🤗

Do you have a minimum reproducing example for the error? It would be also great to add a test

Not really, it is highly dependent on the used image and the state of the fine-tuned model. I haven't observed the bug with the pre-trained model only after fine-tuning it.
(I have a fine-tuned model and an image that produces the error, but I don't think this is a practical test...)

One could call the method directly with an empty mask. This should also produce the error I think.

> One could call the method directly with an empty mask. This should also produce the error I think.

Such an example would be fine!

Where should I add this test? In this [file](https://github.com/huggingface/transformers/blob/main/tests/models/sam/test_processor_sam.py)?

Yes!

Are my tests sufficient? @qubvel 

Hey @MSt-10, hope you are doing well! Let me know if any help is needed from our side to finish this PR 🤗 

@qubvel  Sorry for the delay but I don't use TF, hence I overlooked it. I hope it's correct now🤗
Restore is_torch_greater_or_equal_than for backward compatibility
Author: tlrmchlsmth
Pull request: yes

This PR restores booleans such as `is_torch_greater_or_equal_than_1_13` for backward compatibility reasons. These were removed in https://github.com/huggingface/transformers/pull/35358, and I've set them to True as Tranformers not requires PyTorch >= 2.0

For context, I am trying to upgrade vLLM to use transformers 4.48.0 in https://github.com/vllm-project/vllm/pull/12120. One of the issues I'm running into is that [MiniCPM3-4B
](https://huggingface.co/openbmb/MiniCPM3-4B)'s  checks `is_torch_greater_or_equal_than_1_13` here https://huggingface.co/openbmb/MiniCPM3-4B/blob/e5715484011d723a1892db91da8b59d979d14aee/modeling_minicpm.py#L63-L65.

I've put up a PR to remove that check in MiniCMP3-4B but it seems likely that there are other models that will hit the same issue, I think it would be good to restore these at least for the short term.

cc @ydshieh


Comments:
@ArthurZucker WDYT? I had to do [some emergency PRs to MiniMax](https://huggingface.co/MiniMaxAI/MiniMax-Text-01/discussions/5) because of this, so maybe keeping them as static `True` makes sense for backward compatibility

@Rocketknight1 FYI the link to my PR that does the same thing for MiniCPM3 is https://huggingface.co/openbmb/MiniCPM3-4B/discussions/39 

OK to restore for backward compatibility, but let's do it as before and not just set to `True` to avoid unpleasant surprise.
Thank you for bringing this issue and the PR


> OK to restore for backward compatibility, but let's do it as before and not just set to `True` to avoid unpleasant surprise. Thank you for bringing this issue and the PR

Sounds good -- updated.

@Rocketknight1  I just merged this PR

Yes, sounds good! Thanks @ydshieh!

I still happens in 
Torch version: 2.5.1+cu124
Transformers version: 4.48.1

Hmm, I don't think the fix is included in the patch.

Would it be good for you to using (for now) the dev version or 4.48 or an older version v4.47.1?

Sorry for the inconvenience.


For vLLM, we're on `4.45.2`, but would like to pick up 4.48 for `bamba` support

yep working on the patch! 

will probably ship now and have another one for the num items in batch issue

> For vLLM, we're on `4.45.2`, but would like to pick up 4.48 for `bamba` support

Same error with transformers `4.48.1` and torch `2.5.1` here.

> > For vLLM, we're on `4.45.2`, but would like to pick up 4.48 for `bamba` support
> 
> Same error with transformers `4.48.1` and torch `2.5.1` here.

the fix in this PR is not included in `4.48.1`. We will have a new patch.

im also hitting this on intel mac with torch-2.2.2, numpy1, transformers-4.48.1, etc

No luck! Still facing error: 
```ImportError: cannot import name 'is_torch_greater_or_equal_than_1_13' from 'transformers.pytorch_utils' (\site-packages\transformers\pytorch_utils.py). Did you mean: 'is_torch_greater_or_equal_than_2_1'?```

@MileanCo  @Vaishnav2804 Could you try v4.48.2 (patched released today)
RoPE: model-agnostic RoPE refactor
Author: gante
Pull request: yes

# What does this PR do?

This PR:
- Refators RoPE such that it is model-agnostic. 
  - RoPE models now only need one class
  - The class is parameterized by the model config.
  - Based on the model config, the appropriate type of rope will be loaded into the class
- Adds `longrope`, as part of the model-agnostic refactor on Phi3 (closes #31992); With `longrope`, phi3's checkpoints are now loadable.

👉 Built on top of the Yarn PR (#30910)
____________________________________________
### Review

Key files to check, IN THIS SPECIFIC ORDER:
```
src/transformers/models/llama/modeling_llama.py 
src/transformers/models/llama/configuration_llama.py
src/transformers/modeling_rope_utils.py
```

👉 Other relevant files include `phi3` (`longrope`) and `recurrentgemma` (a few custom changes)

__________________________________________

### Models that require future changes for standardization

⚠️ Some models don't support `cache_positions`, and therefore they are not changed as part of this PR (the new classes is built with the new pattern in mind). A future PR is needed on these models, where both `cache_positions` and this new model-agnostic RoPE is added.

Models that were NOT changed but have RoPE:
- ESM
- Falcon
- GPTNeoX
- GPTNeoXJapanese
- Idefics
- Mixtral
- Persimmon
- Phi
- Qwen2
- Qwen2MoE
- StableLM
- Starcoder2




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31999). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

(all RoPE models with `cache_positions` upgraded, now fixing CI)

I'm trying to train the `Phi-3-small-128k-instruct` model and the configuration loading leads to an error in the rope_validation function [here](https://github.com/huggingface/transformers/blob/main/src/transformers/models/phi3/configuration_phi3.py#L190-L194) because [the config](https://huggingface.co/microsoft/Phi-3-small-128k-instruct/blob/main/config.json) has more than 3 hyper-parameters which fails the check. 

Will the PR fix this issue? If yes, when can we expect this to merge in main? 

MMmm what's weird is that this model uses code on the hub. 
Anyways if we broke something we can do a patch but we need a proper reproducer!

> MMmm what's weird is that this model uses code on the hub. Anyways if we broke something we can do a patch but we need a proper reproducer!

Way to Reproduce:

```
from transformers import  Phi3ForCausalLM
Phi3ForCausalLM.from_pretrained(<path/to/Phi3_small_128k_instruct>")
```

That model is "code on the hub" so it's kind of expected


@gante hello can you help me to review this [PR](https://github.com/huggingface/transformers/pull/32424) for fixed ntk scaling?

Note: splitting this PR into multiple smaller ones, as the refactor needs extra attention in some models (e.g. `cohere`'s RoPE is not exactly the same as `llama`'s)

Keeping the PR open as a reference, until all models have the new RoPE structure

(we now have `modular`, no longer relevant)
[Moshi] disable automatic compilation
Author: gante
Pull request: yes

# What does this PR do?

Fixes moshi + generation, which is making our ci quite red.

#34544 expanded automatic compilation when any compileable cache is used

Moshi uses sliding window cache by default, which is a compileable cache. However, Moshi doesn't support compilation. This PR ensures all caches are tagged as non-compileable for Moshi at generation time, to prevent automatic compilation from happening.

Comments:
Sorry, I'm closing this -- I've found a much better solution 😅 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35986). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Moonshine] compute head_dim_padding at init
Author: eustlb
Pull request: yes

# What does this PR do?

Following @ArthurZucker's [comment](https://github.com/huggingface/transformers/pull/35922#discussion_r1936215568), this PR ensures `head_dim_padding` is computed offline at init. Moreover, there is no need to change the scaling as it is correctly computed on the unpadded head dimension.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35984). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Moshi] disable automatic compilation if the model can't compile
Author: gante
Pull request: yes

# What does this PR do?

Don't trigger automatic compilation if the model can't compile. https://github.com/huggingface/transformers/pull/34544 enabled automatic compilation with any compilation-ready cache. However, some models default to compilation-ready caches AND don't support compilation, e.g. Moshi

Supercedes #35986; Carved from #35989 to reduce the size of that PR.

Fixes Moshi generation tests, e.g. `py.test tests/models/moshi -k test_greedy_generate -vv`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35992). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ydshieh I agree the name is misleading, and has been repurposed for compilation things. Happy to rename it to `_supports_compilation`, WDYT?

I would love it renamed, but since it's an (despite being private) attribute of the important/base class `PreTrainedModel`, I feel more comfortable if Arthur is not against this change.

(and we can do it in a separate PR, so you can merge this one quickly if you wish)

yes, let's do it in a separate PR :)
use torch 2.6 for daily CI
Author: ydshieh
Pull request: yes

# What does this PR do?

It's time :-)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35985). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.