noamaanMulla-03/transformers-issues huggingface dataset:
[`docs`] Add link to ModernBERT Text Classification GLUE finetuning script
Author: tomaarsen
Pull request: yes

# What does this PR do?

Add a link to a Text Classification finetuning script that finetunes a ModernBERT model on GLUE.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Documentation: @stevhliu
cc @ohmeow @NohTow @bclavie @warner-benjamin 

- Tom Aarsen


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35347). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Mamba2 `torch_forward` reduction dimension possibly incorrect?
Author: HanGuo97
Pull request: no
Labels: name: bug

### System Info

NA

### Who can help?

@ArthurZucker

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

NA

### Expected behavior

In the `torch_forward` part of Mamba2, it seems like the reduction dimension should be `dim=3` instead of `dim=2`?

https://github.com/huggingface/transformers/blob/30335093276212ce74938bdfd85bfd5df31a668a/src/transformers/models/mamba2/modeling_mamba2.py#L560

with `dim=3`, the output seems to more or less match that of Mamba-2's [`ssd_minimal`](https://github.com/state-spaces/mamba/blob/main/mamba_ssm/modules/ssd_minimal.py) implementation, but not with `dim=2`

Comments:
Yes, that seems correct. Good spotting! I have a fairly extended ramble into this below (ignore if its too much :) cc @molbap 

We can also see it based on the einsum notation in the ssd minimal script:
`bhzc,bchpn->bzhpn`

In this case as we do not use the einsum notation I will notate the same dimension notations before the sum and broadcasted (via none) values as simple 1:
decay chunk: `bhzc11`
permuted states: `bh1cpn`
So based on the multiplication before the sum we get `bhzcpn` and since we wanted shape `bzhpn` we need to sum along `c` (on dim=3) and reshape afterwards.

Just a quick idea: I'm not sure if we even have to reshape twice instead of once by reshaping the decay chunks only (not checked): 
states: `bc1hpn`
permuted decay chunks: `bczh11`
Resulting in `bczhpn` and finally to `bzhpn` (after sum on dim=1) - hence we avoid the double permutation and just do it "once".

I'm a bit suprised that the following operations after that don't fail. Have you tested your fixed version on a forward? 

As far as I remember, the following operations won't fail because the reductions was on the number of (source) chunks even though it should be on the number of (target) chunk. During training, these two are of the same size.

It's been a while but yea that makes sense. Thx for clarifying! 

A tad late, but I've verified it myself now based on my [test](https://github.com/vasqu/mamba2-torch/blob/main/tests/TestSSDMinimal.py) and modifying the respective local ssd minimal:
- Either we need to sum on `dim=3`
- Or use transposed decays as I led to before (1 permutation less):
```python
decay_chunk = decay_chunk.transpose(1, 3)
new_states = (decay_chunk[..., None, None] * states[:, :, None, ...]).sum(dim=1)
```

Will have a look asap thanks @vasqu !

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Reduce CircleCI usage
Author: ydshieh
Pull request: yes

# What does this PR do?

We have seen the usage are almost doubled in the past 2 month. We are investigating why this happens, but we also try to reduce the usage bu downgrade the runners we need to use.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35355). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Implement AsyncTextIteratorStreamer for asynchronous streaming
Author: CISC
Pull request: yes

# Asynchronous text generation streaming

Added a new `AsyncTextIteratorStreamer` class that allows you to have fully asynchronous text generation streaming in your application. Works identically to `TextIteratorStreamer` except that you iterate the stream as follows:
```python
generated_text = ""
async for new_text in streamer:
    generated_text += new_text
```

~~Requires Python 3.11+ due to use of `asyncio.timeout` to handle timeout.~~ Falls back to `asyncio.wait_for` if `asyncio.timeout` is not available.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

~~Any hints on how best to implement an async test would be appreciated. :)~~ pytest-asyncio seems to be the way to go...

## Who can review?

@gante @ArthurZucker

Comments:
~~Hmmm, looks like I'm missing something important regarding documentation, not sure what though?~~
> Exception: The following objects are in the public init so should be documented:
> - AsyncTextIteratorStreamer

Was missing autodoc entry.

@gante @ArthurZucker Ready for review.

The `check_code_quality` workflow failure makes no sense to me, not sure how to fix?

@ArthurZucker @gante ping

@ArthurZucker ping with sugar on top?

Seems @gante is on holiday, anyone else who can review?

Gentle ping @gante, or maybe @zucchini-nlp if I haven't totally overloaded her with pings at this point!

@Rocketknight1 Thanks! BTW, know what's up with `check_code_quality` and/or how to fix it?

@CISC that's usually a code style thing. Try `pip install transformers[quality]` followed by `make fixup`

@Rocketknight1 It's not, it's complaining about the import order in `src/transformers/__init__.py`, which I did not change... :(

@CISC `make fixup` will also resolve issues like that! In this case, the `__init__.py` has actually been modified in this PR to allow the class to be imported (check `Files changed` - it's in there!). Our code styling tools just want to correctly sort/order inputs into the standard format.

@Rocketknight1 Sure, I changed the file, but I didn't change the sort order, it should already be sorted! :P

Sorry @CISC reviewing!

Do you want us to fix the CIs for you? I can push directlyu

> Do you want us to fix the CIs for you? I can push directlyu

Yes please, thank you. :)

thanks @CISC 🤗
Cleaner attention interfaces
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Improve the attention interfaces with comments/dtype fix


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35342). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

LGTM
Fix tokenizer model handling for LLaMA, converting Base64 to SentencePiece (#35037)
Author: aka133
Pull request: yes

# What does this PR do?

Fixes #35037


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ArthurZucker 
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->

Comments:
Did I do this right? My first time making a PR.
@ArthurZucker
bugfix: torch.export failure caused by `_make_causal_mask`
Author: jiwoong-choi
Pull request: yes
Labels: name: torch export

# What does this PR do?

Fix the `torch.export` failure caused by `AttentionMaskConverter._make_causal_mask`.

Recent changes in torch dynamo prevent mutations on tensors converted with aten::_to_copy. To address this, we can clone such tensor before performing in-place operation `masked_fill_` only when the code is being compiled by torch dynamo. (relevant issue on PyTorch: https://github.com/pytorch/pytorch/issues/127571)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

PyTorch: @gante @Rocketknight1 


Comments:
This seems legit to me, but cc @arthurzucker for core maintainer review
Update test fetcher when we want to test all
Author: ArthurZucker
Pull request: yes

# What does this PR do?
There is no need to check the diff when you want to test all. As computing the diff might sometimes take a looong time. 

Comments:
Nice.

I have

> glob.glob("tests/**/test_**.py", recursive=True)

a few lines below, but it's just a nit if you find helpful

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35364). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Use `weights_only=True` with `torch.load` for `transfo_xl`
Author: ydshieh
Pull request: yes

# What does this PR do?

Just to move forward with a [huntr report](https://huntr.com/bounties/adc0155f-4af7-455a-a7c1-33ad799c1d24). Use `weights_only=True` for `torch.load` in `src/transformers/models/deprecated/transfo_xl/tokenization_transfo_xl.py`.

### Remark

Although `TransfoXL` is deprecated, I am trying to check if this PR breaks something by running

>  tokenizer = TransfoXLTokenizer.from_pretrained("transfo-xl-wt103")

but that is failing even on `main`. I  don't do anything trying to fix it though.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35241). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Make `test_generate_with_static_cache` even less flaky
Author: ydshieh
Pull request: yes

# What does this PR do?

#34464 make some generate tests less flaky. However, with 

> tests/models/paligemma/test_modeling_paligemma.py::PaliGemmaForConditionalGenerationModelTest::test_generate_with_static_cache

it fails 0.6% of the time.

This PR takes the same idea in #34558 to make `test_generate_with_static_cache` even much less flaky. Running 2000 times and all pass.

It also make this process easier to applied to other tests in the future whenever necessary

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34995). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

also fixed

> tests/models/persimmon/test_modeling_persimmon.py::PersimmonModelTest::test_generate_with_static_cache

> tests/models/persimmon/test_modeling_persimmon.py::PersimmonModelTest::test_generate_with_static_cache

200 runs: 8 failed (before) v.s 0 failed (PR)
Fixed typo in audio_classification.md
Author: Uvi-12
Pull request: yes

This PR resolves the documentation issue in audio_classification.md where the term "finetune" was incorrectly used. The correct form, "fine-tune" (with a hyphen), has been applied throughout the document.

@stevhliu

Comments:
Linking the issue as well: https://github.com/huggingface/transformers/issues/35304
Fixed typos in Audio Classification Documentation
Author: Uvi-12
Pull request: yes

Fixes #35261 

- Fixed possessive form: "it's" corrected to "its" in reference to the MInDS-14 dataset.
- Corrected unit capitalization: "khz" updated to "kHz" for proper scientific notation.
- Added a comma for clarity: A missing comma was added to improve grammatical correctness in the sentence about the output_dir parameter.

Comments:
Hi @stevhliu, While reviewing the documentation, I noticed a few areas where the clarity and quality could be enhanced. These aren't necessarily typos or errors, but rather improvements that could make the content easier to understand. Would it be okay for me to proceed with making these improvements?

Hi, feel free to open an issue describing your proposed improvements

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35263). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Hi, feel free to open an issue describing your proposed improvements

Thank you, I will do as you said. For this PR I have made the changes as per your review. If everything looks good please proceed with merging the PR.
DOC: Enhance Documentation Of Audio Classification
Author: Uvi-12
Pull request: no

The documentation for the Audio Classification requires improvements in grammar and phrasing to enhance clarity and readability. Some Suggested updates include:
- "contain" to "contains"
- "called to load and resample the audio file" to "used to load and resample the audio file"
- "Take a look at an example now:" to "Here's an example:", and more.

Comments:
[docs] Follow up register_pipeline
Author: stevhliu
Pull request: yes

Forgot to add the contents of the json file to `register_pipeline`! 😅 cc @tcapelle

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35310). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Improve modular transformers documentation
Author: joelpaulkoch
Pull request: yes
Labels: name: Documentation

name: Modular

# What does this PR do?

Added some more details to the modular transformers documentation that could have helped me.

- Adds hints to general contribution guides 
- Lists which utils scripts are available to generate single-files from modular files and check their content

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu 

Comments:
Question on OWLv2 Model Input Size Flexibility in Hugging Face
Author: heyunwang1
Pull request: no
Labels: name: Vision

I noticed that in Google Research's OWLv2 implementation, the model can accept images of varying sizes, as it allows the input image size to be adjusted. Does Hugging Face’s version of OWLv2 support this same flexibility in input image sizes?

Comments:
Maybe cc @molbap @qubvel 

Hi @heyunwang1! Thanks for opening an issue! There is a PR for enabling variable input size for OWLv2 model
 - https://github.com/huggingface/transformers/pull/34764

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
owlvit/2 dynamic input resolution.
Author: bastrob
Pull request: yes
Labels: name: Vision

name: run-slow

# What does this PR do?

Towards https://github.com/huggingface/transformers/issues/30579
Hey, this is a draft, Im wondering how can we manage the variables impacted by the dynamic input change in the OwlViTForObjectDetection class (self.sqrt_num_patches, self.box_bias) ?
Is there a better way to handle this ?

The interpolate_pos_encoding allows new input size respecting height==width strictly  ? 
In that case I should ensure this.
If not, I think  sqrt_num_patches needs to be decomposed too (_h, _w), some examples where it might throws exc:
https://github.com/bastrob/transformers/blob/30f3c2d56729974ec0d1d9e2fc4fd633ab697eb2/src/transformers/models/owlvit/modeling_owlvit.py#L1355
https://github.com/bastrob/transformers/blob/30f3c2d56729974ec0d1d9e2fc4fd633ab697eb2/src/transformers/models/owlvit/modeling_owlvit.py#L1459
https://github.com/bastrob/transformers/blob/30f3c2d56729974ec0d1d9e2fc4fd633ab697eb2/src/transformers/models/owlvit/modeling_owlvit.py#L1719

Fixes: #34622

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@amyeroberts
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
Yes, Im agree it makes sense to enable any image resolution, i just wanted to clarify this point. I will work on it, thanks for your feedback !

Hi @qubvel, I pushed a version to handle image size at any resolution, what do you think about it ?

If it sounds good to you @qubvel, Im ready to push the last commit :)

Hello @qubvel, `box_predictor` now takes `interpolate_pos_encoding` as extra parameter to enable the dynamic computation of `box_bias` inside.
Plus, I've added `# copied from` statement.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34764). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[WIP] Add flex attention for Qwen2VL
Author: jla524
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Addresses #34809 (issue)

## Who can review?

@ArthurZucker

Comments: