noamaanMulla-03/transformers-issues huggingface dataset:
[docs] Fix quantization overview
Author: stevhliu
Pull request: yes

Fixes the links the quantization overview table

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36523). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Closing in favor of #36528
Fix links in quantization doc
Author: MekkCyber
Pull request: yes

# What does this PR do?

Fixing the links in the quantization doc by deleting the .md extension from the local paths.



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).
[save_pretrained ] Skip collecting duplicated weight
Author: wejoncy
Pull request: yes

# What does this PR do?

Many weight in sub moduoles are collected multiple times. It's useless and time-wasting. Sometimes it even induces some issue when CPU/disk offload is enabled.


<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?




@Rocketknight1 @ArthurZucker


Comments:
Hi @wejoncy 👋 

I'm not a reviewer for this PR, but I can foresee a request: please share with us a script with an example where your change results in an upgrade 🤗 

@SunMarc @gante  Thanks for your review.
I did a comparation of with/without this change in local. weight are not changed and time keeps same.  I think this might be most of the time are consumed on saving to disk. But this will resolve a issue when enabled CPU-offloading with VPTQ models. `Centroid.weight` would be overwrited as meta tensor.

This change fixed it. I can of-couse build a test with it.
Thanks again.
Add weight norm rename in _load_state_dict_into_model
Author: dannywhuang
Pull request: yes

# What does this PR do?

#33275 fixed old parametrization of weight_norm for _load_state_dict_into_meta_model, but the issue also holds for _load_state_dict_into_model when using an old version of torch, e.g., 1.13.

Namely, weights are not loaded correctly for wav2vec2.encoder.pos_conv_embed.conv.weight_g, wav2vec2.encoder.pos_conv_embed.conv.weight_v for a model trained on a PyTorch version where weight_g and weight_v were renamed to original0 and original1. 

What happens is that 
- cls._load_pretrained_model is called in from_pretrained
- _fix_key is called to ensure no more warnings for renamed keys as nicely fixed by #33275
- _load_state_dict_into_model is called, with the state_dict containing still unrenamed keys https://github.com/huggingface/transformers/blob/7f95372c6267d3163fd2aa74aeff9d84ddb6cc35/src/transformers/modeling_utils.py#L4712
- _load_state_dict_into_model adjusts beta and gamma names but not weight_g and weight_v: https://github.com/huggingface/transformers/blob/7f95372c6267d3163fd2aa74aeff9d84ddb6cc35/src/transformers/modeling_utils.py#L666
- This is however, done in _load_state_dict_into_meta_model creating an inconsistency between the two: https://github.com/huggingface/transformers/blob/7f95372c6267d3163fd2aa74aeff9d84ddb6cc35/src/transformers/modeling_utils.py#L832

Related to #31970.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

cc @LysandreJik and @ArthurZucker 


Comments:
Not sure exactly who to ping on this one, but since meta models are involved let's try @muellerzr and @SunMarc
guard torch version for uint16
Author: SunMarc
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/36472#issuecomment-2695005664 
torch.uint16, torch.uint32, torch.uint64 are available only for torch>=2.3. See https://github.com/pytorch/vision/issues/8359

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36520). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Request: Add Flash Attention 2.0 Support for ViTMAEForPreTraining
Author: noelEOS
Pull request: no
Labels: name: Good Second Issue

name: Feature request

name: Vision

name: Flash Attention

Hi Hugging Face team!

I am currently working on pre-training a Foundation Model using ViTMAEForPreTraining, and I was hoping to use Flash Attention 2.0 to speed up training and reduce memory usage. However, when I attempted to enable Flash Attention, I encountered the following error:

`ValueError: ViTMAEForPreTraining does not support Flash Attention 2.0 yet. 
Please request to add support where the model is hosted, on its model hub page: https://huggingface.co//discussions/new 
or in the Transformers GitHub repo: https://github.com/huggingface/transformers/issues/new`

Since MAE pre-training is heavily dependent on the attention mechanism, adding Flash Attention support would be a valuable enhancement—especially for larger ViT models and high-resolution datasets, like Landsat data we are working with.

**Feature Request**

- Please add support for Flash Attention 2.0 to ViTMAEForPreTraining.
- This would help make MAE pre-training more efficient in terms of speed and memory consumption.

**Why This Matters**

- Many users working with large imagery datasets (like remote sensing, medical imaging, etc.) would greatly benefit from this.
- Flash Attention has already proven useful in other ViT variants, so bringing this to MAE feels like a natural next step.

**Environment Details**

- Transformers version: v4.41.0.dev0
- PyTorch version: 2.5.1
- Running on multi-GPU with NCCL backend

Comments:
It might take a while before we get to this, but we'd welcome a PR if anyone in the community wants to add it! You can use attention code from other models that do support FlashAttention2 as a template

Hey @noelEOS, please check the PR linked, it should work now if you install from the branch

```
pip install -U git+https://github.com/qubvel/transformers@refactor-vit-attention
```

@qubvel, thanks so much for the quick response and for adding this feature! I really appreciate the fast turnaround. 🙌

I have not had the chance to test the branch yet, but I’ll try it out soon and report back if I encounter anything unexpected. Looking forward to seeing how it performs!!
[docs] minor typo in modular converter cli
Author: pcuenca
Pull request: yes

- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Documentation: @stevhliu

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

Thanks for raising a PR! There is already a duplicate PR [here](https://github.com/huggingface/transformers/pull/36558), so maybe we can close this one? :)

lol, sorry!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36568). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
What is the difference between the function of add_tokens() and add_special_tokens() in tokenizer
Author: kugwzk
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->
When I read the code of tokenizer, I have a problem if I want to use a pretrained model in NMT task, I need to add some tag tokens, such as '2English' or '2French'. I think these tokens are special tokens, so which function should I use: add_tokens() or add_special_tokens(). What is the difference between them?


Comments:
For some reasons those functions do not appear in the documentation, will have a look at why. The docstrings state of `add_special_tokens` states:

> Add a dictionary of special tokens (eos, pad, cls...) to the encoder and link them to class attributes. If special tokens are NOT in the vocabulary, they are added to it (indexed starting from the last index of the current vocabulary).
> Using `add_special_tokens` will ensure your special tokens can be used in several ways:
>  - special tokens are carefully handled by the tokenizer (they are never split)
>  - you can easily refer to special tokens using tokenizer class attributes like `tokenizer.cls_token`. This makes it easy to develop model-agnostic training and fine-tuning scripts.

Though the second point would not apply in your case. The docstring of `add_tokens` states:

> Add a list of new tokens to the tokenizer class. If the new tokens are not in the vocabulary, they are added to it with indices starting from length of the current vocabulary.
        When possible, special tokens are already registered for provided pretrained models (ex: BertTokenizer cls_token is already registered to be '[CLS]' and XLM's one is also registered to be '
')

Hope that helps!

Thank a lot for your help. Therefore, if I use the add_tokens(), the tokens may still split?

I would also appreciate some clarification on the difference between the function and when to use which one. 

> Thank a lot for your help. Therefore, if I use the add_tokens(), the tokens may still split?

In my practice yes, 


> I would also appreciate some clarification on the difference between the function and when to use which one.

For example if you want your `<SEG>` as whole, don't split to < ,  SEG, >  three token, you need add_special_tokens()
AttributeError: 'dict' object has no attribute '_attn_implementation_internal'
Author: lucasjinreal
Pull request: no
Labels: name: bug

### System Info

```
ForCausalLM.from_pretrained(model_path, attn_implementation='flash_attention_2')
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4084, in from_pretrained
    config = cls._autoset_attn_implementation(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/lib/python3.12/site-packages/transformers/modeling_utils.py", line 1516, in _autoset_attn_implementation
    sub_config._attn_implementation_internal = curr_attn_implementation
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'dict' object has no attribute '_attn_implementation_internal'
```

Loading from a saved_pretrained model got error.

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

ef

### Expected behavior

ef

Comments:
@lucasjinreal can you provide a small reproducer with the model id you are using?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
How to get the score of each token when using pipeline
Author: xin0623
Pull request: no
Labels: name: Usage

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.7,
    top_p=0.95,
    top_k=40,
    repetition_penalty=1.1,
    output_scores=True
)

The model I use is Qwen2-7B-Instruct. When I try to output the score of each token by modifying the parameters, it doesn't work.

Comments:
Hey! You might want `return_dict_in_generate` see https://huggingface.co/docs/transformers/en/internal/generation_utils#generate-outputs 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

HI, Have you found a way now? How to get the score of each token when using pipeline？ @xin0623
how to use transformers with musicgen with float16
Author: ghost
Pull request: no

```
import transformers, torch, builtins, numpy

processor = transformers.AutoProcessor.from_pretrained(' facebook/musicgen-stereo-melody-large', torch_dtype=torch.float16)
model = transformers.MusicgenMelodyForConditionalGeneration.from_pretrained('facebook/musicgen-stereo-melody-large ,torch_dtype=torch.float16).to('cuda')

result = []
for _ in builtins.range(2):
    inputs = processor(audio=result[-1] if result else None, sampling_rate=model.config.audio_encoder.sampling_rate, text='A grand and majestic symphony with soaring strings, powerful brass, and dynamic orchestration. Inspired by Beethoven and Tchaikovsky, featuring dramatic crescendos, delicate woodwind passages, and a triumphant finale. The mood is epic, emotional, and timeless', padding=True, return_tensors='pt').to('cuda')
    audio_values = model.generate(**inputs, max_new_tokens=1000)
    result += audio_values[0, 0].cpu().numpy(),

from IPython.display import Audio
Audio(numpy.concatenate(result), rate=model.config.audio_encoder.sampling_rate)
```
i alwayse get
```
<ipython-input-12-348220656bb8> in <cell line: 0>()
      7 for _ in builtins.range(2):
      8     inputs = processor(audio=torch.from_numpy(result[-1]).to(dtype=torch.float32) if result else None, sampling_rate=model.config.audio_encoder.sampling_rate, text='A grand and majestic symphony with soaring strings, powerful brass, and dynamic orchestration. Inspired by Beethoven and Tchaikovsky, featuring dramatic crescendos, delicate woodwind passages, and a triumphant finale. The mood is epic, emotional, and timeless', padding=True, return_tensors='pt').to('cuda')
----> 9     audio_values = model.generate(**inputs, max_new_tokens=1000)
     10     result += audio_values[0, 0].cpu().numpy(),
     11 

5 frames
/usr/local/lib/python3.11/dist-packages/torch/nn/modules/linear.py in forward(self, input)
    123 
    124     def forward(self, input: Tensor) -> Tensor:
--> 125         return F.linear(input, self.weight, self.bias)
    126 
    127     def extra_repr(self) -> str:

RuntimeError: mat1 and mat2 must have the same dtype, but got Float and Half
```


Comments:
Hi @fancydaddy, these questions are usually more suited to the Discord or forums! Older models may not always natively support float16 or bfloat16 inputs, so you should check that the processor is actually returning `float16` objects. If passing these through the network still results in errors, then it's likely that the model's code includes a cast to `float32` somewhere, probably for numerical stability in the attention layers. There's not much you can do in that case except to rewrite the model code to cast dtypes correctly!
Do trailing padding tokens get a forward pass?
Author: rvashurin
Pull request: no

Imagine I'm performing an **inference** using causal LLM with `batch_size=2`, `do_sample=False`, `num_beams=1` (i.e. greedy decoding). 

Suppose one of the inputs in the batch results in a shorter generated sequence than the other. After this shorter sequence get an EOS token, all subsequent token ids for this sequence will be set to whatever `pad_token_id` is set to be, while other sequence keeps getting meaningful tokens added to it.

<img width="430" alt="Image" src="https://github.com/user-attachments/assets/e6103cf3-12ae-49fc-955a-d0b44519dc73" />

If I look at the values of `outputs.hidden_states` from the last layer for these trailing pad tokens, I see that they are present **and** vary between subsequent decoding steps. 

<img width="719" alt="Image" src="https://github.com/user-attachments/assets/67e3207b-723b-40aa-be7d-3bcb19e97ff9" />

Do these pad tokens actually produce these embeddings through a full scale forward pass, even though the sequence is already finished, and no new tokens will be added to them? If, say, I want to train some classifier on latent representations of generated serquences from LLMs, I still should probably mask these embeddings out, before doing some pooling over them, right?

Comments:
Hi @rvashurin, this is a question that's more suited to the forums or the Discord, since it's not a bug in Transformers! Batched generation always continues 'finished' sequences after the EOS token, until all sequences have finished. If you want to train a classifier, it's up to you to find the end of the "real" generated tokens in each sequence.
Add support for StableAdamW optimizer in Trainer
Author: capemox
Pull request: no
Labels: name: Feature request

### Feature request

StableAdamW is an optimizer first introduced in [Stable and low-precision training for large-scale vision-language models](https://arxiv.org/pdf/2304.13013), an AdamW and AdaFactor hybrid optimizer, leading to more stable training. Most notably, however, it has been used in the [modernBERT paper](https://arxiv.org/pdf/2412.13663):
> StableAdamW’s learning rate clipping outperformed standard gradient clipping on downstream tasks and led to more stable training

It would be great is this is available as an optimizer in `Trainer`!

### Motivation

More models in the future may use StableAdamW because of its success in training modernBERT, and having it as an option in `Trainer` (as `optim` in `TrainingArguments`) would be convenient.

### Your contribution

I'm interested to contribute! The modernBERT paper uses the implementation from [optimi](https://github.com/warner-benjamin/optimi), which can be added as an import. I'd love to submit a PR.

Comments:
cc @muellerzr @sunmarc

Hi @capemox, feel free to submit a PR for that !
Modular Conversion --fix_and_overwrite on Windows
Author: hlky
Pull request: yes

# What does this PR do?

Uses `encoding="utf-8"` and `newline="\n"`

FYI `huggingface_hub` introduced a style bot GitHub action https://github.com/huggingface/huggingface_hub/pull/2898 we're using that in Diffusers https://github.com/huggingface/diffusers/blob/main/.github/workflows/pr_style_bot.yml to reduce the burden on external contributors.

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36583). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi @hlky, this looks good! I think the `newline` kwarg is only needed on the **writer** file handle `(mode="w")`, though. On the reader file handles, universal newlines are already used, so setting `newline="\n"` will have no effect, and might even reduce compatibility (it will break if the input file has "\r\n" newlines): https://docs.python.org/3/library/functions.html#open

Thanks @Rocketknight1. `.gitattributes` will handle CRLF->LF if `newline="\n"` wasn't set on `write` but avoids the warning.

> warning: in the working copy of 'src/transformers/models/siglip2/modeling_siglip2.py', CRLF will be replaced by LF the next time Git touches it