noamaanMulla-03/transformers-issues huggingface dataset:
[`MllamaProcessor`] Update errors and API with multiple image
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes #33706 to support batches:
- if one image or more per batch, padd and don't error our
- if less than one image in one of the prompts, error out
- if all text, don't error out 
- if images are not passed, but token is found, error out

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33715). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix paligemma `eager` vs `sdpa` + image transforms test fixup
Author: molbap
Pull request: yes
Labels: name: run-slow

# What does this PR do?

Should fix a couple bugs identified in testing by @ydshieh, related to `eager` mode not being passed correctly + a torch dynamo breaking test due to `cache_position[0] == 0`. 

I also found out we rescale with -1 images to check the image processor in common tests, however we create an uint8 image first - so I changed up the `rescale` to detect this edge case. It should not be a common codepath as we don't rescale by negatives that often. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33699). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

hmm, very strange it didn't find any model to run slow ci. I am checking

oh no. The last commit is `mark test as very slightly flaky` so it's normal
Fix position embeddings singular/plural
Author: molbap
Pull request: yes
Labels: name: run-slow

# What does this PR do?

It seems a few Blip-derived models have both `self.position_embedding` and `self.position_embeddings`, plural, defined in the same module and causing some errors. This aims at fixing those. Noticed it in #31368 .

Causes many things to break because self.position_embedding isn't defined since it's self.position_embeddings (with an s) instead. 

Comments:
cc @amyeroberts for review!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33678). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@amyeroberts sorry for the ping, just keeping this one higher in the notifications - hub tests were just flaky :) 

Found out about an issue in `test_expansion_in_processing` for `instructblip` @zucchini-nlp and fixed it here, not a big deal I think but FYI (list + tensor concat messing expansion!)
clean_up_tokenization_spaces=False if unset
Author: itazap
Pull request: yes

FUTURE DEPRECATION

fixes #31884 

start of deprecating `clean_up_tokenization_spaces`. Right now it defaults to True, update to default to False.

- BERT based models need `clean_up_tokenization_spaces=True`, so set this in class __init__
- some models like wav2vec needed test updated since they don't really expect `clean_up_tokenization_spaces=True`. 

@ArthurZucker 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31938). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

i believe the issue is 
```python
if "clean_up_tokenization_spaces" not in kwargs:
            warnings.warn(
                "`clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This "
                "behavior will be depracted in transformers v4.45, and will be then set to `False` by default. "
                "For more details check this issue: https://github.com/huggingface/transformers/issues/31884",
                FutureWarning,
            )

# By default, cleaning tokenization spaces for both fast and slow tokenizers
self.clean_up_tokenization_spaces = kwargs.pop("clean_up_tokenization_spaces", True)
```

by default it setting but i don't understand where to update it to remove the warning completely 


@rishi23root I'll update the message!
Generate: `can_generate()` recursive check
Author: gante
Pull request: yes

# What does this PR do?

The deprecation warning added in #33203 was incomplete: classes inheriting from classes that have received the update were throwing the warning. This PR adds a recursive check to `can_generate()` and tests the warnings thrown by the function.

Example of script that is throwing a warning, but shouldn't (and is fixed in this PR):
```py
from transformers.models.llama import LlamaForCausalLM

class NewLlamaForCausalLM(LlamaForCausalLM):
    pass

model = NewLlamaForCausalLM.from_pretrained("hf-internal-testing/tiny-random-LlamaForCausalLM")
model.generate()
```

Thank you @regisss for warning me about this one and providing a reproducer 🙏 



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33718). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Ok! @regisss can you comment whether this fixes the issue you encountered?

Yep it's all good on my side, the warning doesn't appear anymore with this fix. Thanks!
🌐 [i18n-KO] Translated `llm_optims.md` to Korean
Author: yijun-lee
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `llm_optims.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
@jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave
<!-- @win2dvp21, @enchantee00, @fabxoe, @1kmmk1, @junejae -->
<!-- @SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd -->
<!-- @mjk0618, @cjfghk5697, @ahnjj, @JinukHong, @nuatmochoi -->
<!-- @010kim, @chhaewxn, @boyunJang, @jeongiin, @harheem -->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR?

Comments:
원문의 내용이 새로운 commit으로 변경되었습니다.

긴 문서인데, 번역하느라 고생 많으셨습니다! 😄 @yijun-lee 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32325). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Fantastic, LGTM! Thanks for adding those additional sections!
> 
> @mreraser @shinhyunji36 would you mind reviewing these new sections?

Hello @stevhliu! Yes, I will review the newly added sections. Thank you 😄
Add a language model probability pipeline
Author: ianporada
Pull request: no
Labels: name: Core: Pipeline

name: Feature request

### Feature request

A common use case of language models is to estimate the probability (or log-probability, or perplexity) of a sequence of tokens. It could be convenient to add a pipeline that returns the log-probability / score of a sequence.

### Motivation

It's possible to calculate model predictions for the probability of a sequence, e.g.: https://huggingface.co/docs/transformers/en/perplexity

But pipelines automatically handle tokenization, batching, etc. so it would be more convenient to have a pipeline.

Alternatively, it might be possible to achieve this behavior by creating a "text-generation" pipeline and calling the pipeline with  `max_new_tokens=0` as well as passing certain `generate_kwargs` to get the sequence LM scores, but how to do this is not immediately obvious.

### Your contribution

This should be quite easy to achieve by creating a new pipeline with `_forward` essentially containing the logic for calculating sequence probability at https://huggingface.co/docs/transformers/en/perplexity

Comments:
cc @Rocketknight1 

Hmmn, it's an interesting idea but I'm not sure it justifies a pipeline. The main reason is that calculating sequence perplexity is something that more advanced users do, but the implementation is quite simple using lower-level tools than pipelines. Therefore, I suspect the users who would want this mostly don't need the pipeline!


Personally I am a big fan of the ease of use of pipelines (because they automatically handle a lot of things) and would use such a pipeline, and I believe many may be interesting in getting a "score" for a sequence probability even if not an advanced user looking to calculate perplexity (in the same way that many non-experts are interested in prob. scores for token classification or QA pipelines).

@Rocketknight1 I do agree with the sentiment that this use case might not justify an entirely new pipeline. Do you think it would make sense to add something like a "score" option to text-generation?

Hey @ianporada, we also have custom pipelines that you can push to the Hub for others to use: https://huggingface.co/docs/transformers/add_new_pipeline#share-your-pipeline-on-the-hub

In case you're interested in doing that to make your pipeline available to others :slightly_smiling_face: 

Thanks! I didn't know about that. I'll close this request and look to create a custom pipeline if I get the chance.

Awesome! Don't hesitate to share it back here so that we can check it out!
transformers - installation
Author: jzaba123
Pull request: no


Hello,

Going via the install and I see some issues 

https://huggingface.co/docs/transformers/installation

A)
PLATFORM WIN

pip install 'transformers[torch]'

ERROR: Invalid requirement: "'transformers[torch]'": Expected package name at the start of dependency specifier
    'transformers[torch]'

pip install 'transformers[tf-cpu]'

ERROR: Invalid requirement: "'transformers[tf-cpu]'": Expected package name at the start of dependency specifier
    'transformers[tf-cpu]'

pip install 'transformers[flax]'

ERROR: Invalid requirement: "'transformers[flax]'": Expected package name at the start of dependency specifier
    'transformers[flax]'

CORRECT SHOULD BE
==================

pip install "transformers[torch]"


pip install "transformers[tf-cpu]"


pip install "transformers[flax]"

B)
pip install "transformers[tf-cpu]" DID NOT WORK

Failed to build tokenizers
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (tokenizers)

https://huggingface.co/learn/nlp-course/en/chapter8/5
transformers-cli env

- `transformers` version: 4.44.0
- Platform: Windows-10-10.0.22631-SP0
- Python version: 3.11.9
- Huggingface_hub version: 0.24.5
- Safetensors version: 0.4.4
- Accelerate version: 0.33.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.4.0+cu118 (True)
- Tensorflow version (GPU?): 2.17.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.7.0 (cpu)
- Jax version: 0.4.13
- JaxLib version: 0.4.13
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 3060 Laptop GPU





Comments:
Hi @jzaba123, thanks for raising this issue! 

Would you like to open a PR to update the instructions to use `"`? 

cc @Rocketknight1 regarding the TF-CPU install 

Hi @jzaba123, yes, thanks for the issue! `pip install 'transformers[torch]'` works on Linux but presumably not on Windows. Can you open a PR to fix it? The relevant file is `installation.md`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
AutoTokenizer for XGLM model not working properly
Author: amirrezamirzaei
Pull request: no
Labels: name: Core: Tokenization

name: Fast Tokenizers

name: bug

### System Info

- `transformers` version: 4.44.2
- Platform: Linux-6.1.85+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.24.7
- Safetensors version: 0.4.5
- Accelerate version: 0.34.2
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.4.1+cu121 (False)
- Tensorflow version (GPU?): 2.17.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.8.4 (cpu)
- Jax version: 0.4.26
- JaxLib version: 0.4.26
- Using distributed or parallel set-up in script?: no

### Who can help?

@ArthurZucker @itazap

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
from transformers import XGLMTokenizer, AutoTokenizer

xglm_tok = XGLMTokenizer.from_pretrained("facebook/xglm-2.9B")
auto_tok = AutoTokenizer.from_pretrained("facebook/xglm-2.9B")

print(xglm_tok.encode('a ')) # [2, 11]
print(auto_tok.encode('a ')) # [2, 11, 6]
```

### Expected behavior

both the tokenizer should output the same ids.

Comments:
Hi @amirrezamirzaei , it happens because both statements load different tokenizer. `xglm_tok` loads `transformers.models.xglm.tokenization_xglm.XGLMTokenizer` and `auto_tok` loads the fast version`transformers.models.xglm.tokenization_xglm_fast.XGLMTokenizerFast`

@RUFFY-369 I see thank you. Closing this issue.
Load PreTrainedTokenizerFast if tokenizer.json exists and fast=True 
Author: itazap
Pull request: yes

Current status for `AutoTokenizer` with `fast=True`:

1.  checks `tokenizer_config.json` if `tokenizer_class` name ends with `Fast`
2. if not, load a slow tokenizer

(This PR):
(unchanged) 1.  checks `tokenizer_config.json` if `tokenizer_class` name ends with `Fast`
2. if not, check if repo has a `tokenizer.json` file
2.1 if yes, load `PreTrainedTokenizerFast`
3. if not, load a slow tokenizer

Comments:
Doc and config mismatch for DeBERTa
Author: fkrasnov2
Pull request: yes

Fixes # (issue)
[Docs mismatch config](https://github.com/huggingface/transformers/issues/33645)

Before submitting
[+] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[+] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
Pull Request section?
[+] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
to it if that's the case.
[+] Did you make sure to update the documentation with your changes? Here are the
[documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
[here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).

@molbap

Comments:
add sdpa and flash_attention2 support to speech2text
Author: avishaiElmakies
Pull request: yes

I added support for sdpa and flash_attention2 to speech2text model (I started with sdpa but beacuse of "#copied" thought it would just be easier to add flash_attentnion2 as well). copy from bart. 
addresses #26350 and #28005 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@amyeroberts @fxmarty 

## Notes:
1. there was a bug in the class ```Speech2TextSinusoidalPositionalEmbedding``` which caused ```test_eager_matches_sdpa_generate``` to fail. I fixed it. The class was copied in ```SpeechT5```. so I fixed the bug there as well
2.  the test ```test_flash_attn_2_generate_reuse_cache``` was failing because he wasn't meant for speech2text model. I copied and tweaked the test from ```whisper```.
3. the only test currently failing on my machine is ```test_flash_attn_2_from_config``` with the error ``` ValueError: Unrecognized configuration class <class 'transformers.models.speech_to_text.configuration_speech_to_text.Speech2TextConfig'> for this kind of AutoModel: AutoModelForCausalLM.``` . speech2text doesn't have a model for causalLM but i couldn't find what other models that have the same situation and what they do with the test. bart has  ```BartForCausalLM```. so would love some guidance here 

Comments:
cc @ylacombe for first review
Stable dropout show drop prob in model print
Author: fkrasnov2
Pull request: yes

What does this PR do?
Fixes # [issue](https://github.com/huggingface/transformers/issues/33646)

Before submitting
[-] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[+ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
Pull Request section?
[+] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
to it if that's the case.
[+] Did you make sure to update the documentation with your changes? Here are the
[documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
[here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
[-] Did you write any new necessary tests?
Who can review?
[@LysandreJik](https://twitter.com/LysandreJik)

Comments:
There are some quality issues, do you mind taking a look? Here's the error:

```
Exception: Found the following copy inconsistencies:
- src/transformers/models/deberta_v2/modeling_deberta_v2.py: copy does not match models.deberta.modeling_deberta.StableDropout at line 219
- src/transformers/models/sew_d/modeling_sew_d.py: copy does not match models.deberta.modeling_deberta.StableDropout at line 625
Run `make fix-copies` or `python utils/check_copies.py --fix_and_overwrite` to fix them.
```

In general, for the code quality tools, you can install them with the following, from the root of your clone:
```
pip install -e ".[quality]"
```
And then run them with:
```
make fixup
```
[`clean_up_tokenization_spaces`] Pl bart was failing, updating
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Update clean_up_tokenization_spaces after #31938

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33735). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
A mismatch in the docs
Author: fkrasnov2
Pull request: no
Labels: name: Documentation

### System Info

Hi,
Everything's going great with DeBERTa, thanks a lot! 

There's a tiny bit of a mismatch in the docs.

![image](https://github.com/user-attachments/assets/3c56c65d-51f9-48e2-9663-0e90d92a1a53)

Hey, no biggie, but if it's important, I can fix it. Just let me know.

Best,
Fedor


### Who can help?

@stevhliu

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://github.com/huggingface/transformers/blob/78b2929c0554b79e0489b451ce4ece14d265ead2/src/transformers/models/deberta/configuration_deberta.py#L43

### Expected behavior

https://github.com/huggingface/transformers/blob/78b2929c0554b79e0489b451ce4ece14d265ead2/src/transformers/models/deberta/configuration_deberta.py#L105

Comments:
Hi @fkrasnov2 ! For sure, feel free to open a doc PR :) 

I believe this can be closed as #33713 was just merged, but feel free to reopen in case you'd like to do additional fixes!
fix: Issue #32917, fixed _prune_heads in T5Model 
Author: anaghasid
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes issue #32917
Changed the `_prune_heads()` method in T5Model. This can now be used with the generic `model.prune_heads(heads_to_prune)` found in modeling_utils.py


Comments:
Ping @anaghasid in case you'd still want to get this merged!
add support for loading a pretrainedfast tokenizer if fast=true and t…
Author: itazap
Pull request: yes

Current status for `AutoTokenizer` with `fast=True`:

1.  checks `tokenizer_config.json` if `tokenizer_class` name ends with `Fast`
2. if not, load a slow tokenizer

(This PR):
(unchanged) 1.  checks `tokenizer_config.json` if `tokenizer_class` name ends with `Fast`
2. if not, check if repo has a `tokenizer.json` file
2.1 if yes, load `PreTrainedTokenizerFast`
3. if not, load a slow tokenizer

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33750). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.