noamaanMulla-03/transformers-issues huggingface dataset:
[i18n-DE] Complete first toc chapter
Author: flozi00
Pull request: yes

# What does this PR do ?
This PR translates all the file of the first toc chapter to german
It continues https://github.com/huggingface/transformers/issues/18564

- [x] add new model
- [x] add new pipeline
- [x] add tensorflow model
- [x] llm tutorial
- [x] peft
- [x] run scripts
- [x] transformers agents
- [x] re-read at the live documentation preview from PR

Comments:
@ArthurZucker or who is the correct one to ping ?

I just wondered why the doc builder job did not started as the last times
Could you approve the workflows to start ?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26311). All of your documentation changes will be reflected on that endpoint.
🌐 [i18n-KO] Translated `debugging.md` to Korean
Author: wonhyeongseo
Pull request: yes

# What does this PR do?

@kj021 translated the `debugging.md` file of the documentation to Korean. I made sure hanging suggestions were resolved.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
Moved from https://github.com/huggingface/transformers/pull/24869

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
<!-- Team OSSCA, may you please review this PR? @bolizabeth, @nuatmochoi, @heuristicwave, @mjk0618, @keonju2, @harheem, @HongB1, @junejae, @54data, @Sunmin0520, @seank021, @augustinLib, @sronger, @TaeYupNoh, @kj021, @eenzeenee -->
Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- May you please review this PR? @sgugger, @ArthurZucker, @stevhliu -->

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26246). All of your documentation changes will be reflected on that endpoint.
🌐 [i18n-KO] Translated `perf_train_gpu_many.md` to Korean
Author: wonhyeongseo
Pull request: yes

# What does this PR do?

@hyunhp translated the `perf_train_gpu_many.md` file of the documentation to Korean. I made sure hanging suggestions were resolved.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
Moved from https://github.com/huggingface/transformers/pull/24983

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
<!-- Team OSSCA, may you please review this PR? @bolizabeth, @nuatmochoi, @heuristicwave, @mjk0618, @keonju2, @harheem, @HongB1, @junejae, @54data, @Sunmin0520, @seank021, @augustinLib, @sronger, @TaeYupNoh, @kj021, @eenzeenee -->
Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- May you please review this PR? @sgugger, @ArthurZucker, @stevhliu -->

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26244). All of your documentation changes will be reflected on that endpoint.
Fix `cos_sin` device issue in Falcon model
Author: ydshieh
Pull request: yes

# What does this PR do?

We should update the device accordingly. See the comments along the changes.

So far, running the model on GPU, then change model to CPU and running CPU inputs, the cached tensors (like `cos_cached`) will be on GPU and fail to run with CPU tensors.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
AttributeError: 'CTCTrainer' object has no attribute 'scaler'
Author: aprzez
Pull request: no

When attempting to fine-tune Wav2Vec2 XLSR on my own dataset I get this error once training starts: "AttributeError: 'CTCTrainer' object has no attribute 'scaler'"

I followed the steps outlined here and as far as I can tell have all the necessary packages installed in my conda environment: https://github.com/huggingface/transformers/blob/main/examples/research_projects/wav2vec2/FINE_TUNE_XLSR_WAV2VEC2.md#local-machine
The only change to the run_common_voice.py file is that I am loading my own data from a CSV.

Comments:
cc @sanchit-gandhi but for research projects in `examples` you should make sure you are using the pinned version of `transformers`. (also share the output of `transformers-cli env` when you submit an issue

Hey @aprzez - the examples script referenced is quite outdated and no longer maintained. It uses the Hugging Face Trainer **prior** to the recent `accelerate` upgrade in v4.30.0. As such, it is not guaranteed to work out of the box, since it requires updating for the new HF Trainer internals.

I would recommend you either:
1. Downgrade your `transformers` version to <4.30 (less optimal, you won't get the latest features from the most recent releases)
2. Use the latest version of `transformers`, but with the **updated** and **maintained** examples script [`run_speech_recognition_ctc.py`](https://github.com/huggingface/transformers/tree/main/examples/pytorch/speech-recognition#connectionist-temporal-classification) (more optimal - we maintain this and ensure it works with the latest `transformers` versions!)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
optimize VRAM for calculating pos_bias in LayoutLM v2, v3
Author: NormXU
Pull request: yes

# What does this PR do?

The current implementation of calculating 1d_pos_bias/2d_pos_bias in LayoutLMv2, v3 is VRAM-consuming due to the large one-hot matrix. 

Considering the idea of 1d_pos_bias/2d_pos_bias is to categorize all relative positions into several buckets, assign each position id to a specific bucket based on its relative distance to another token, and embed the position id into a feature, we can drop the large one-hot matrix and directly use the Linear weight features like an nn.Embedding. 

In my tests, as for an input sequence of $[10, 1024]$ (bz, nseq), this can save 3 Gb VRAM for 2d_pos_bias calculations


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?  # This PR can reuse previous tests


## Who can review?
@ArthurZucker and @younesbelkada



Comments:
Hey! Thanks for opening a PR, pinging @rafaelpadilla for a review here 😉 

@rafaelpadilla I've reformatted the codes. It's ready to be merged.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26139). All of your documentation changes will be reflected on that endpoint.
change mention of decoder_input_ids to input_ids and same with decode_inputs_embeds
Author: tmabraham
Pull request: yes

The arguments to Llama2 is `input_ids` and `inputs_embeds` but yet in several places it is mentioned as `decoder_input_ids` and `decoder_inputs_embeds` including in some error messages. This PR fixes that.

Comments:
Could you just run the code quality tool to ensure that the code quality passes? You can install them with the following, from the root of your clone:
```
pip install -e ".[quality]"
```
And then run them with:
```
make fixup
``` 

Hmm looks like something is wrong? Not sure if I am doing something wrong...
```
(base) tmabraham@DESKTOP-TPBRMQJ:/mnt/c/Users/tmabraham/Documents/GitHub/transformers$ make fixup
make: Warning: File 'Makefile' has modification time 70901 s in the future
No library .py files were modified
python utils/custom_init_isort.py
python utils/style_doc.py src/transformers docs/source --max_len 119
running deps_table_update
updating src/transformers/dependency_versions_table.py
python utils/class_mapping_update.py
python utils/check_copies.py
Traceback (most recent call last):
  File "/mnt/c/Users/tmabraham/Documents/GitHub/transformers/utils/check_copies.py", line 353, in <module>
    check_copies(args.fix_and_overwrite)
  File "/mnt/c/Users/tmabraham/Documents/GitHub/transformers/utils/check_copies.py", line 186, in check_copies
    new_diffs = is_copy_consistent(filename, overwrite)
  File "/mnt/c/Users/tmabraham/Documents/GitHub/transformers/utils/check_copies.py", line 164, in is_copy_consistent
    theoretical_code = blackify(lines[start_index - 1] + theoretical_code)
  File "/mnt/c/Users/tmabraham/Documents/GitHub/transformers/utils/check_copies.py", line 104, in blackify
    result = black.format_str(code, mode=black.FileMode([black.TargetVersion.PY35], line_length=119))
  File "<string>", line 3, in __init__
TypeError: set object expected; got list
make: *** [Makefile:38: extra_quality_checks] Error 1
```

Hmm indeed this seems like a weird error! I pushed a fix directly to your branch, I'll merge this PR as soon as it's green.

Thanks for your PR!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26406). All of your documentation changes will be reflected on that endpoint.
docs: change assert to raise and some small docs
Author: pphuc25
Pull request: yes

Hi,
I create a concise document outlining the process of improving code readability by replacing 'assert' statements with 'raise' statements, along with the addition of documentation and 'logger.info' statements.
I would like cc @stevhliu to review my PR.

Comments:
Hi, in general we like to use `raise` statements instead of `assert` because the latter are considered more like debugging statements that can be toggled on or off in certain setups.

So to catch more debugging information, I switch from using assert to raise. I believe the code I am working on is correct, @stevhliu.

Hi @stevhliu, can you help me, I cannot run this command

Yes, it looks like you're missing `black` which can be fixed by installing it first:

```bash
pip install -e ".[quality]"
```

Then you should be able to run the `make style` command :)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26232). All of your documentation changes will be reflected on that endpoint.

Hi @LysandreJik, I have reverted them back. However, in run_mlm_no_trainer the logging do not have eval_loss, this seem the important one so user can know how well loss of their model is, I think we should keep add the eval_loss logging in that file.
[`PEFT`] introducing `adapter_kwargs` for loading adapters from different Hub location (`subfolder`, `revision`) than the base model
Author: younesbelkada
Pull request: yes

# What does this PR do?

Currently on main branch the script below fails:

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "andrewrreed/falcon-7b-guanaco-qlora-arr"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id,
    torch_dtype=torch.bfloat16, 
    load_in_4bit=True,
)

print(model)
```

This is because of falcon models the revision parameter gets overriden here: https://github.com/huggingface/transformers/blob/main/src/transformers/models/falcon/modeling_falcon.py#L779 for BC and to make sure users will load transformers model 

This introduced an error in the logic of loading adapters as the same `revision` argument was used all over the place. As this scenario might occur more often, I propose to introduce a new argument `adapter_revision` for users that want to load a base model and an adapter from different revisions.

cc @ArthurZucker @LysandreJik 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

I think that I was not clear in the PR description, the PR is not a patch for Falcon but globally for users that want to load a base model from a specific revision and the adapter on the default revision (or another revision), which by luck occured for Falcon because of the patch introduced recently on main. 
Currently on main it is not possible to load a base model from a revision, let's say `"revision-1"` and load the adapter from the default revision `"main"`, unless if users syncronises the revision between both repos - usually adapter weights are not stored in the same repository as the base model.
Let me know if this is clearer and if this makes sense

As seen offline, I would rather pack kwargs such as this one within a single `adapter_kwargs` dict to be consumed by a specific method, rather than adding a (potentially) large amount of kwargs as we add more functionality

The PR should be ready for review! I have added a test, in the past when modifying those parts of the code it caused some issues [see here](https://huggingface.slack.com/archives/C01NE71C4F7/p1692864347980869) I made sure that test pass also with this PR.
'tiiuae/falcon-7b' is not supported using flash attention 2
Author: nirdoshrawal009
Pull request: no

ValueError: The current architecture does not support Flash Attention 2.0. Please open an issue on GitHub to request support for this architecture: https://github.com/huggingface/transformers/issues/new

Comments:
cc @younesbelkada  :)

I believe this is actually a duplicate of https://github.com/huggingface/transformers/issues/26443, closing in favor of this one

Hi @nirdoshrawal009 
Please take a look at my comment here: https://github.com/huggingface/transformers/issues/26443#issuecomment-1738789689
For more details
Do not warn about unexpected decoder weights when loading T5EncoderModel and LongT5EncoderModel
Author: fleonce
Pull request: yes

# What does this PR do?

Adds `[r"decoder"]` to both `T5EncoderModel` and `LongT5EncoderModel`, as both models do not have any decoder layers and loading pretrained model checkpoints like `t5-small` will give warnings about keys found in the checkpoint but not in the model itself. To prevent this issue, `r"decoder"` has been added to `_keys_to_ignore_on_load_unexpected` for both model classes.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@ArthurZucker @younesbelkada 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26211). All of your documentation changes will be reflected on that endpoint.

Thank you for the follow-up question!
The problem can be reproduced when the log level for the whole library has been set to INFO:
```python3
import transformers
transformers.logging.set_verbosity_info()
m = transformers.T5EncoderModel.from_pretrained('t5-small')
```

I was not using the latest version of `transformers` when I was encountering this issue initially, now it is hidden by default because verbosity info is required to show those kinds of warnings, unless `T5EncoderModel` is contained in `model.config.architectures`, but only `T5ForConditionalGeneration` is in there. This seems to be caused by commit 096f2cf12664bb7da41f89897d3a22966baee9b4 (https://github.com/huggingface/transformers/blob/bc7ce1808f6c30df87fd9dff871a53ef510ccf77/src/transformers/modeling_utils.py#L3674)

The problem persists however, but we wont see the warning by default
feat: add trainer label to wandb run upon initialization
Author: parambharat
Pull request: yes

# What does this PR do?
This PR adds a telemetry label to the wandb run making it possible to identify W&B usage from the trainer class.


## Before submitting
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?

## Who can review?
@sgugger


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25590). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Thanks, @ArthurZucker 

@muellerzr , could you please take a look and let me know if this PR is good to merge or is anything is needed from my end here.


Hey @parambharat the PR looks good but it seems GitHub didn't appreciate your merge/rebase, unfortunately (it happens sometimes!), and it shows 311 commits.

Do you mind just closing this PR and opening one from the same branch once again? Ping me on it and I'll be happy to take another look. Thanks!
Uninitialized token embeddings MBART when using device_map
Author: BramVanroy
Pull request: no

### System Info

Current master

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Loading a finetuned model's safetensors with device_map=auto, I get a warning that the tied embeddings are not initialized.

> Some weights of MBartForConditionalGeneration were not initialized from the model checkpoint and are newly initialized: ['model.decoder.embed_tokens.weight', 'model.encoder.embed_tokens.weight']

I finetuned an MBART model with the trainer, use_safetensors was set to True. The model's vocabulary (embedding size) was extended, maybe that matters.

```python
from transformers import AutoModelForSeq2SeqLM
# Works:
model = AutoModelForSeq2SeqLM.from_pretrained("BramVanroy/mbart_test")
# Does not work (tied embeddings not loaded correctly - triggers a warning)
model = AutoModelForSeq2SeqLM.from_pretrained("BramVanroy/mbart_test", device_map="auto")
```

It is not just the warning, it seems actually the case that the weights are not loaded correctly (random output).

### Expected behavior

Correctly loaded safetensors

Comments:
I've traced this to the definition of the ` find_tied_parameters` function from `accelerate`.

In `transformers` we have a control flow to identify tied parameters that does not work for `meta` tensors; if we identify a `meta` tensor, we then rely on the `find_tied_parameters` function from `accelerate`. There seems to be a discrepancy in the number of layers returned by these two methods here, depending on whether we'reusing a `device_map` or not:

https://github.com/huggingface/transformers/blob/0ac3875011d32dc85e0e83970507e3afe8f0febb/src/transformers/modeling_utils.py#L3471-L3481

@SunMarc, would you like to investigate what might be going on here?

Yes @LysandreJik, I will check that is going on.