noamaanMulla-03/transformers-issues huggingface dataset:
Fix num_heads in _upad_input
Author: fs4r
Pull request: yes

# What does this PR do?

The variable num_key_value_heads in the FlashAttention Module has falsely been named num_heads, which led to reshaping the query_layer using the wrong attention head count. (It would have been enough to use the correct variable self.num_heads instead of num_heads, but I renamed num_heads to num_key_value_heads for clarity)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker and @younesbelkada


Comments:
@younesbelkada Just did 👍 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26490). All of your documentation changes will be reflected on that endpoint.

I do not have time at the moment. Leave it as a TODO and I might pick up the task in the coming weeks.
Fix broken link to video classification task
Author: HelgeS
Pull request: yes

# What does this PR do?

It fixes a broken link in the VideoMAE documentation.

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu and @MKhalusova


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26487). All of your documentation changes will be reflected on that endpoint.
[`PEFT`] Pass token when calling `find_adapter_config`
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR fixes an issue that was reported on Spaces. I was not able to reproduce the issue locally though. 
When loading a model with `token=True` (i.e. on a gated or private repository), `find_adapter_config` will try to look for an adapter file inside a private repository without the token, leading to an authentication error. 

The fix is to pass the token to `adapter_kwargs` and remove the duplication here: https://github.com/huggingface/transformers/blob/main/src/transformers/modeling_utils.py#L2529 

cc @LysandreJik 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This seems to fix the issue for me. Shall we merge it? Or figure out the actual cause?

@younesbelkada `adapter_kwargs` can be `None`, in which case the code breaks. [Here](https://github.com/huggingface/transformers/blob/main/src/transformers/models/auto/auto_factory.py#L467) it can be set to `None`, then it passed via [this line](https://github.com/huggingface/transformers/blob/main/src/transformers/models/auto/auto_factory.py#L548). 

For reference, I was running
```
python main.py \
  --model path_to_mistral_7B \
  --tasks mbpp \
  --temperature 0.1 \
  --n_samples 15 \
  --batch_size 4 \
  --precision fp16 \
  --allow_code_execution \
  --save_generations \
  --max_length_generation 512 \
  --save_generations_path predictions.json \
  --metric_output_path metrics.json
```

Nice catch!  Will open a PR soon to fix that
[`PEFT`] Protect `adapter_kwargs` check
Author: younesbelkada
Pull request: yes

# What does this PR do?

Protect `adapter_kwargs` check in case it is excplicitly None

Addresses: https://github.com/huggingface/transformers/pull/26488#issuecomment-1742821190

cc @LysandreJik 

The fix could be also to pop with an empty dict here: https://github.com/huggingface/transformers/blob/main/src/transformers/models/auto/auto_factory.py#L467

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Doctest] Add configuration_roformer.py 
Author: hegdeadithyak
Pull request: yes

Adds configuration_roformer.py to utils/documentation_tests.txt

Based on https://github.com/huggingface/transformers/issues/19487

@ydshieh Please have a look at this 🌞

Comments:
@ydshieh Please have a look at this 🌞

@ydshieh Please Check this!!

Sorry, I forgot to check, but we should remove the entry `src/transformers/models/roformer/configuration_roformer.py` from the file `utils/not_doctested.txt`. Otherwise the doctest won't be triggered.

@ydshieh Sir, Please Check this 😁

OK, you corrected it - I was somehow confused by the previous commit! Thanks

Thankyou

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26530). All of your documentation changes will be reflected on that endpoint.
Fix padding for IDEFICS 
Author: shauray8
Pull request: yes

# What does this PR do?

Fixes padding issues - 
```python
from transformers import AutoProcessor

processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics-9b", padding_side="right")

sents = [["hello world"], [" this is a longer sentence to testing padding"]]

# This is the correct behaviour:
a = processor(sents, padding="max_length", truncation=True, max_length=20)
print(processor.tokenizer.decode(a['input_ids'][0]))
# => 
 hello world<unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk>

# This is the incorrect behaviour:
b = processor(sents, padding="longest", truncation=True, max_length=30)

print(processor.tokenizer.decode(b['input_ids'][0]))
# => <unk><unk><unk><unk><unk><unk>
 hello world
```
With this fix the results would be 
```
# for max_length

 hello world<unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk>
# for longest

 hello world<unk><unk><unk><unk><unk><unk>
```

Fixes #26354


## Before submitting
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.

## Who can review?
@VictorSanh @ArthurZucker @LysandreJik 

*let me know if there's anything that needs follow-ups*

Comments:
sure!

@ArthurZucker CI looks green, PR's ready for another review.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26396). All of your documentation changes will be reflected on that endpoint.

thank you @shauray8 !
Code-llama-nit
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes #26156, the `fast` CodeLLama tokenizer excepts the fill token to be != None. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I'll add a few more update it seems that even when performing infilling the prefix tokens is removed, which should not be done (for both fast and slow): TODO for patch if needed
LLamaV2 tokenizer prepare_for_model produces different results for fast and slow tokenizers
Author: zerogerc
Pull request: no

### System Info

transformers==4.33.2

Hi, I noticed that LlamaV2 tokenizer produces different results depending on whether is_fast is enabled or not:



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Here is a code snippet:

```

    tokenizer_fast = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf", use_fast=True)
    tokenizer_slow = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf", use_fast=False)

    print(tokenizer_fast.prepare_for_model([100, 200, 300])["input_ids"])
    >>> [100, 200, 300]
    print(tokenizer_slow.prepare_for_model([100, 200, 300])["input_ids"])
    >>> [1, 100, 200, 300]
```

### Expected behavior

The results should be the same

Comments:
Yep, this is because `prepare_input_for_model` uses the `build_inputs_with_special_tokens`, which was never added for LlamaFast as using the template processor is the favoured way to go. I'll add it back for now, good catch!
add build_inputs_with_special_tokens to LlamaFast
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes #26287 where the ouptuts of `prepare_for_model` are different. There is a test for  `prepare_for_model` but we don't really make sure that the outputs match. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

On our test update roadmap cc @ydshieh
[i18n-DE] contribute chapter
Author: flozi00
Pull request: yes

# What does this PR do?
Another continue of https://github.com/huggingface/transformers/issues/18564
This PR adds PR and testing translation for the contribute chapter in docs

I will do another review on live docs, when its ready

pinging for docs
@stevhliu and @MKhalusova


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26481). All of your documentation changes will be reflected on that endpoint.
🌐 [i18n-KO] Translated `tokenizer_summary.md` to Korean
Author: wonhyeongseo
Pull request: yes

# What does this PR do?

@HanNayeoniee translated the `tokenizer_summary.md` file of the documentation to Korean. I made sure hanging suggestions were resolved.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
Moved from https://github.com/huggingface/transformers/pull/25023

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
<!-- Team OSSCA, may you please review this PR? @bolizabeth, @nuatmochoi, @heuristicwave, @mjk0618, @keonju2, @harheem, @HongB1, @junejae, @54data, @Sunmin0520, @seank021, @augustinLib, @sronger, @TaeYupNoh, @kj021, @eenzeenee -->
Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- May you please review this PR? @sgugger, @ArthurZucker, @stevhliu -->

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26243). All of your documentation changes will be reflected on that endpoint.
[RFC, Logging] Change warning to info
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

Change warning to info when adding a new text embedding. The reason is that this warning is trigger every time someone runs:

which confuses users: https://github.com/huggingface/diffusers/issues/5212 .

I don't think we should use `pad_to_multiple` here as it would mean that we add 8 new tokens every time we call `load_textual_inversion`


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Wav2Vec2 and Co] Update init tests for PT 2.1
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Wrapping modules in `nn.utils.parameterization.weight_norm` changes the weight signature vs vanilla `nn.utils.weight_norm`: https://github.com/huggingface/transformers/blob/1b8decb04c246ec8e1c4ba7f2749043d0876d24e/src/transformers/models/wav2vec2/modeling_wav2vec2.py#L379-L381

I've updated the tests to catch these new weight signatures, keeping the old ones in there for backwards comp


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fixed inconsistency in BertTokenizerFast
Author: Towdo
Pull request: yes

`if token_ids_1:` will fail if `token_ids_1` is a empty list.
In `BertTokenizerFast.create_token_type_ids_from_sequences` and `BertTokenizer` we check `if token_ids_1 is None:`.
A empty list is not considered None, but it will return False if converted to bool.
Which makes this an inconsistency.

Fixes #26123


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

( I don't know what to tick here. It's just a small fix :L)


@ArthurZucker


Comments:
[`Nougat`] from transformers import *
Author: ArthurZucker
Pull request: yes

# What does this PR do?

Doing `python -c "from transformers import *"` on a fresh env fails

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26562). All of your documentation changes will be reflected on that endpoint.
Nit-added-tokens
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes #26500, fixes #26536

Comments:
_The documentation is not available anymore as the PR was closed or merged._

A small benchmark on the `get_added_vocab()`:
```python 
from transformers import AutoTokenizer
import time 
tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-moe-54b")
start = time.time();tokenizer.get_added_vocab();print(time.time()-start)
>>> 0.17021536827087402

start = time.time();{k.content: v for v, k in sorted(tokenizer.added_tokens_decoder.items(), key=lambda item: item[0])};print(time.time()-start)
>>> 0.0054759979248046875

start = time.time();tokenizer.added_tokens_decoder;print(time.time()-start)
0.0007669925689697266
```
will update rust to make `tokenizer.added_tokens_encoder` available. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26538). All of your documentation changes will be reflected on that endpoint.
[Whisper] Allow basic text normalization
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Supersedes #20707. This PR adds the option of performing **basic** text normalisation, which is used for multilingual decoding. Rather than replacing English text normalisation altogether, we add it as an alternative option. The rationale for this is:
1. Users running a multilingual checkpoint (e.g. large-v2) might still want to apply the English normaliser if they are using it for English ASR
2. Prevent a breaking change in the API, where passing `normalize=True` switches from using the English normaliser to the basic normalizer silently

=> instead, we add a new argument `basic_normalizer`, which can be toggled to enable/disable basic text normalisation

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Gently pinging @ArthurZucker since this one might have slipped through the net!

I don't agree with using one argument! See the original comment for the rationales for using two arguments: https://github.com/huggingface/transformers/pull/26149#issue-1894810529
docs: feat: add clip notebook resources from OSSCA community
Author: junejae
Pull request: yes

# What does this PR do?
Adds a notebook resource of CLIP on: 
- How to fine-tune CLIP with Korean multimodal dataset
- How to use CLIP for image-text similarity.

The notebook is created by our OSSCA community.
Part of https://github.com/huggingface/transformers/issues/20055

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@bolizabeth, @nuatmochoi, @heuristicwave, @mjk0618, @keonju2, @harheem, @HongB1, @junejae, @54data, @seank021, @augustinLib, @sronger, @TaeYupNoh, @kj021, @eenzeenee, @stevhliu
<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
That's cool! Feel free to ping us when you'd like that to be merged :)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26505). All of your documentation changes will be reflected on that endpoint.
[`PEFT`] Final fixes
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR fixes multiple bugs with PEFT and some corner cases such as:

- loading an adapter model with `token` argument that currently fails
- logger.warning that errors out since it does not seem to accept the argument `FutureWarning`
- Saving that fails with 4-bit quantized models

Added some nice tests

cc @LysandreJik 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@younesbelkada thanks for this, does this allow pushing 4-bit nf4 models to hub? thanks

Hi @RonanKMcGovern 
For pushing 4-bit weights on the Hub, please refer to this PR: https://github.com/huggingface/transformers/pull/26037 
The current PR enables pushing adapter weights with 4bit base models
[`core`] Fix keep in fp32 silent bug
Author: younesbelkada
Pull request: yes

# What does this PR do?

Before this PR we were performing a simple check `if module_name in key` but that lead to some modules silently converted in fp32. 

For example instructblip models got their `word_embedding` layers converted in fp32 because `_keep_in_fp32_modules` includes `"wo"` which is contained in the string `word_embedding`. The fix is to check if `module_name in key.split(".")`

cc @ydshieh 

Related bnb and T5 test all pass

Need to investigate if instructblip tests pass

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add add_generation_prompt argument to apply_chat_template
Author: Rocketknight1
Pull request: yes

This PR adds a new `add_generation_prompt` argument to `apply_chat_template`. We need this when we want to chat with a chat model - if the model has special tokens that indicate the start of a bot message, then we need to append these to the end of a generation prompt to indicate to the model that it should write a bot reply, and not continue the user message or something like that.

Note that many prompts (e.g. LLaMA) don't include special tokens at the start of bot messages - this makes them very easy to generate for. This argument would have no effect for them, so I don't need to update their chat templates to support it.

Fixes #26539

Comments:
_The documentation is not available anymore as the PR was closed or merged._