noamaanMulla-03/transformers-issues huggingface dataset:
🌐[i18n-KO] Translated `llm_tutorial.md` to Korean
Author: harheem
Pull request: yes

# What does this PR do?

Translated the `llm_tutorial.md` file of the documentation to Korean 😄 
Thank you in advance for your review!

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)
@bolizabeth, @nuatmochoi, @heuristicwave, @mjk0618, @keonju2, @harheem, @HongB1, @junejae, @54data, @seank021, @augustinLib, @sronger, @TaeYupNoh, @kj021, @eenzeenee

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)
 May you please review this PR?
@sgugger, @ArthurZucker, @eunseojo

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25791). All of your documentation changes will be reflected on that endpoint.
[`core`] Import tensorflow inside relevant methods in `trainer_utils`
Author: younesbelkada
Pull request: yes

# What does this PR do?

Imports tensorflow inside relevant methods in `trainer_utils`. 
When Deepspeed and accelerate are available in your env, it will try to import transformers all the way down to https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_utils.py#L49 

Leading to considerable slowdowns in import time

![Screenshot 2023-09-12 at 10 48 50](https://github.com/huggingface/transformers/assets/49240599/b03a4f28-dafb-4b06-89d2-2f9516f49b58)

This PR imports tensorflow inside the relevant methods so that it will never get globally imported by transformers anymore.
If this patch gets applied, there is no speedup, however this can be fixed by lazy loading TF in accelerate itself:
 
![Screenshot 2023-09-12 at 10 49 26](https://github.com/huggingface/transformers/assets/49240599/c61e1eda-ea96-4940-970f-49ccea6ee50b)

cc @amyeroberts @patrickvonplaten @pacman100 @muellerzr @SunMarc 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[`Persimmon`] Add support for persimmon
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Add support for the `Persimmon` models by Adept.

cc @Rocketknight1 

Comments:
Looks good so far, and the outputs are definitely better than purely random, so I suspect the port is mostly correct except for one detail somewhere.

I don't want to double up on work, but let me know if you want me to take any part of the port!

_The documentation is not available anymore as the PR was closed or merged._

The Fast Tokenizer requires `tokenizers == 0.14.0` which will be added in #23909 . 
I left the logic for now as it can work without! 

Thank you for setting this up @ArthurZucker and folks! This is awesome work, we're amazed by how quickly y'all spun this up ⚡  Moving over a comment from the other PR over on our repo:

> @ArthurZucker thank you for putting this together!!! A really quick note: We'd recommend adding the prompt structure for the chat-finetuning to the chat inference, since performance really degrades without it! It looks something like this:
```
human: <some query>

adept:
```

It should hopefully be an easy add here (but happy to help do this in a subsequent PR as well). Thanks so much again!!

The follow up PR is #25323 which introduces chat templating! 
The only thing I have left to do is make sure I have 1 to 1 outputs and will merge. Just double checking 


That sounds great @ArthurZucker. Just to confirm, would chat templating be the default for the chat model? If not, what steps can we take to make that happen? Thanks!

The chat template will be uploaded on the hub as it will be part of the tokenizer. It won't be the default for the tokenizer (as it uses the `LlamaTokenizer`) neither will it be the default for normal tasks. But this seems align with your codebase, as users have the freedom to use `Conversational` pipeline vs normal text generation 🤗
[BUG] `ExponentialDecayLengthPenalty` decreases negative scores
Author: pokjay
Pull request: no

### System Info

- `transformers` version: 4.31.0
- Platform: Linux-5.15.109+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.2
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.0.1+cu118 (False)
- Tensorflow version (GPU?): 2.12.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.7.1 (cpu)
- Jax version: 0.4.14
- JaxLib version: 0.4.14
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@gante

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

https://colab.research.google.com/drive/1_Ur0sQhrUan68OXDlFK0SGQfDhwlb8Pc?usp=sharing

### Expected behavior

#### Issue Explanation
`ExponentialDecayLengthPenalty` is intended to exponentially increase the score of the eos_token_id after start_index has been reached, allowing generating shorter sequences without having a hard cutoff. (Original PR: https://github.com/huggingface/transformers/pull/15245 )

When working with shorter sequences it doesn't necessarily cut the sequence, no matter how large the decay factor is.
In the line below, the processor attempts to increase the score of EOS. However when EOS score is negative, this actually decreases the score, as the exponent will be positive.
As I understand, giving a negative decay factor won't work as well due to the power. Due to this it will only succeed if EOS becomes positive.
https://github.com/huggingface/transformers/blob/f1732e1374a082bf8e43bd0e4aa8a2da21a32a21/src/transformers/generation/logits_process.py#L982


#### Proposed solution
In the attached [Colab notebook](https://colab.research.google.com/drive/1_Ur0sQhrUan68OXDlFK0SGQfDhwlb8Pc?usp=sharing), I added a proposed solution. If the score is negative, we can compute the expected penalty added to EOS if the score was positive, and add that to the original negative score.

If this is acceptable, I'd like to create a PR to fix the issue!

Comments:
cc @gante

Hey @pokjay! It is indeed a bug, and one that is not caught in its test (`test_exponential_decay_length_penalty`) because the test uses positive logits.

I am in favor of the fix, and I'd also like to ask to add a case with negative logits to the test :) 

@gante Great, I'll fix it and add the test case! I'll add to the same PR the documentation examples from https://github.com/huggingface/transformers/issues/24783

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[WIP] Add Grounding DINO
Author: EduardoPach
Pull request: yes

# What does this PR do?

This PR adds Grounding DINO

Fixes #25423 

To-Do's:

- [x]  Port vision backbone
- [x]  Port Text backbone
- [ ]  Port Feature Enhancer
- [ ]  Port Cross-Modality Decoder

Comments:
cc @amyeroberts for information. Please let us know when your PR is ready for review!

@amyeroberts Hey, I've actually started a new branch to port the model because in the branch from this PR, I've done `add-new-model-like` and then `swin`, but talking to @NielsRogge he mentioned that starting from `deformable-detr` would be easier (which was a SUPER helpful tip btw). Should I close this PR and open a new one with the correct branch? (still WIP, but almost there)

@EduardoPach Do whatever is easiest for you in terms of code management. If you open a new PR, just make sure to link to it in a comment on this one so it can be easily tracked. 

@amyeroberts opened a new PR in https://github.com/huggingface/transformers/pull/26087 as mentioned. Should I close this one?

@EduardoPach I can close it for you :)
fix FSDP model resume optimizer & scheduler
Author: pkumc
Pull request: yes

# What does this PR do?

This PR is to fix FSDP model training when resume from checkpoint. All changes are made in [Trainer](https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py).

1. In [save_fsdp_optimizer](https://github.com/huggingface/accelerate/blob/a16b843a1b8626d571e62f7301e3a478be2c04b4/src/accelerate/utils/fsdp_utils.py#L142) method,  optimizer is saved with the name **optimizer.bin**. So I modify [_load_optimizer_and_scheduler](https://github.com/huggingface/transformers/blob/07998ef39926b76d3f6667025535d0859eed61c3/src/transformers/trainer.py#L2442) method to match it.
2. In [_save_checkpoint](https://github.com/huggingface/transformers/blob/07998ef39926b76d3f6667025535d0859eed61c3/src/transformers/trainer.py#L2311) method, scheduler is not saved when FSDP is enabled. I think it could be a careless mistake. I refactor the code to fix it.

## Who can review?
- trainer: @muellerz and @pacman100


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25852). All of your documentation changes will be reflected on that endpoint.

Could you resolve the code quality issues, once that is done we are good to merge

> Could you resolve the code quality issues, once that is done we are good to merge

@pacman100 The code quality has been resolved, thx.
Add an efficient vision transformer backbone in ICLR 2022: CrossFormer
Author: cheerss
Pull request: no
Labels: name: New model

### Model description

The CrossFormer has three new things that does not exist in other ViTs (such as Swin):

1. The cross-scale embedding layer(CEL) that generate cross-scale embeddings as ViT's input.
2. The long-short distance attention (LSDA) mechanism, which is an efficient replacement of the vanilla self-attention and shows better performance than Swin
3. A dynamic relative position bias, a kind of relative position bias that support dynamic group size.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

The open source website: https://github.com/cheerss/CrossFormer
The paper was accepted in ICLR 2022: https://openreview.net/forum?id=_PHymLIxuI

Comments:
I can pick this up. 

@raghavanone Are you still working on this? If so, would you have an estimation of when it will be ready for review? This would be a great addition to the library, if you don't have enough bandwidth then we can open it up for someone else in the community to pick up :) 

cc @rafaelpadilla 

I had paused this for a while , I have bandwidth now , will continue to work on it . 

@raghavanone  Thanks for your great work. I checked the merge workflow and found the error is due to the model not being added into `_import_structure` like [this](https://github.com/huggingface/transformers/blob/main/src/transformers/__init__.py#L535). Hope that may help you.
LoRA A and B param.requires_grad showing as false during training
Author: RonanKMcGovern
Pull request: no

### System Info

I'm running the latest version of transformers in a Colab notebook.

I'm simply printing out the param.requires_grad in the model's named parameters to see if it is set True or false, and I'm finding that all have requires_grad set to false. If there is some other list showing the LoRA params with required gradients, how can I access that?

```
base_model.model.model.embed_tokens.weight: False
base_model.model.model.layers.0.self_attn.q_proj.weight: False
base_model.model.model.layers.0.self_attn.q_proj.lora_A.default.weight: False
base_model.model.model.layers.0.self_attn.q_proj.lora_B.default.weight: False
base_model.model.model.layers.0.self_attn.k_proj.weight: False
base_model.model.model.layers.0.self_attn.k_proj.lora_A.default.weight: False
base_model.model.model.layers.0.self_attn.k_proj.lora_B.default.weight: False`
...etc.
```

### Who can help?

@SunMarc @younesbelkada 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
        model = PeftModel.from_pretrained(base_model, path)
        model.gradient_checkpointing_enable()
        model = prepare_model_for_kbit_training(model)
        model.to(input_ids.device)

        # Print trainable parameters here
        for name, param in model.named_parameters():
            print(f"{name}: {param.requires_grad}")
```

### Expected behavior

I would have expected that all of the LoRA A and B matrixes would show that they require gradiants...

Comments:
Hi @RonanKMcGovern, this happens because you used `prepare_model_for_kbit_training` after loading the adapters. `prepare_model_for_kbit_training` should be used before adding the adapters to the quantized model. Another solution would be to fix `prepare_model_for_kbit_training` so that we only freeze the base model layers. 

ah thanks @SunMarc , I re-ran and now am just seeing the base layer frozen and True for the LoRA A and B
Place AMD GPU tests in a separate workflow (correct branch)
Author: fxmarty
Pull request: yes

As suggested https://github.com/huggingface/transformers/pull/26007#discussion_r1318684244 and discussed offline cc @mfuntowicz

Note: this PR merges into a branch, not main.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

LGTM waiting from transformers folks feedback ✌🏻

cc @ydshieh
🌐 [i18n-KO] Translated `llama2.md` to Korean
Author: mjk0618
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다! -->
# What does this PR do?

Translated the `llama2.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
<!-- May you please review this PR? @member1 @member2 ... -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
May you please review this PR? @sgugger, @ArthurZucker, @eunseojo

Comments:
> 좋은 번역 감사합니다! 😆 일부 리뷰를 해봤습니다. 검토 부탁드려요!

리뷰 감사합니다! 말씀해주신 부분 반영하여 커밋하였습니다.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26047). All of your documentation changes will be reflected on that endpoint.

> Uhhh hellooo? This PR should unexist.

Hi! I didn't understand what you meant, why shouldn't this PR exist?

May you please review this PR? @sgugger, @ArthurZucker, @eunseojo
[docs] Updates to TTS task guide with regards to the new TTS pipeline 
Author: MKhalusova
Pull request: yes

This PR adds: 

- An example of inference with the newly added text-to-speech pipeline
- A necessary training code update to make sure the fine-tuned checkpoint is usable via the pipeline (saving the processor)
- An introduction example of inference with Bark and a link to the course for more examples



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Generate: text generation pipeline no longer emits `max_length` warning when it is not set
Author: gante
Pull request: yes

# What does this PR do?

Fixes #22636 

In the `text-generation` pipeline, `max_length` is updated to take into account the prefix (which defaults to the BOS token). When `max_new_tokens` was set, it meant that `.generate` received both parameters, triggering the warning.

This PR defers the `max_length` update to right before generation, in case `max_new_tokens` is set at call time, and only updates it if `max_new_tokens` is not set -- avoiding triggering the warning if the user has not set `max_length` while keeping the same behavior.

____________________________________
Test script, which was triggering the warning before this change:
```py
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline, GenerationConfig

device = "cuda:0"
model_name = "facebook/opt-1.3b"

# tokenizer, model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    low_cpu_mem_usage=True,
    pad_token_id=tokenizer.eos_token_id
).to(device)

# pipeline
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device=device)

# generate text
text = "Hello "
result = pipe(
    text,
    generation_config=GenerationConfig(
        max_new_tokens=70,
        num_beams=1,
        do_sample=False
    )
)

# print result
print(result)
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@Narsil added test 👍 (precisely as suggested, using a small model checks that the warning is raised only when it should)

Sorry am I wrong or this is still an issue on `text2text_generation`?
I'm asking this because i keep seeing these warnings on stdout during a summarization task:

> Your max_length is set to 1024, but your input_length is only 197. Since this is a summarization task, where outputs shorter than the input are typically wanted, you might consider decreasing max_length manually, e.g. summarizer('...', max_length=98)
> Both `max_new_tokens` (=100) and `max_length`(=1024) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)

And I can see these lines in `src/transformers/pipelines/text2text_generation.py:184-186`:
```python
        generate_kwargs["min_length"] = generate_kwargs.get("min_length", self.model.config.min_length)
        generate_kwargs["max_length"] = generate_kwargs.get("max_length", self.model.config.max_length)
        self.check_inputs(input_length, generate_kwargs["min_length"], generate_kwargs["max_length"])
```

@ndricca it's possible, they are different pipelines. I will check whether we can sort this one out :) 

@ndricca the PR linked above sorts the (incorrect) warning you were seeing :)
Text2text pipeline: don't parameterize from the config
Author: gante
Pull request: yes

# What does this PR do?

Fixes the warning raised in [this comment](https://github.com/huggingface/transformers/pull/23139#issuecomment-1674143603).

Similar to other recent PRs, it removes a few lines that attempt to parameterize `.generate()` using values from the model config, when they are absent. The `GenerationConfig` class already handles these default values, so it's redundant. Moreover, when an unused kwarg is explicitly passed to `.generate()`, warnings are triggered (which was the issue in the comment).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@amyeroberts To answer your top-level question: the main change in the PR is that we do not set `max_length` when the user does not pass it. The warning was raised when the user passed `max_new_tokens`, which would clash with the (removed) `max_length` set from the default values. If the user passed `max_length`, no warning was raised, as it should.

Note: in `generate()`, because we have the old default value of `max_length=20`, we rely on detecting whether `max_length` was passed to throw relevant length-related warnings (like `max_length` and `max_new_tokens` being mutually exclusive).
chore: correct update_step and correct gradient_accumulation_steps
Author: pphuc25
Pull request: yes

Hi,
As discussed in issue #25691, it seems that the update step is incorrect. I have made corrections to all the files to ensure synchronization. However, during my edits, I noticed that there were multiple instances of incorrect variable name `args.gradient_accumulation_steps` so I have rectified these issues as well.

I would like cc @ArthurZucker to review my code.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26068). All of your documentation changes will be reflected on that endpoint.
Typo in the doc of AutoTokenizer? Two parameters for slow_tokenizer_class
Author: xukp20
Pull request: no

**Is your feature request related to a problem? Please describe.**
The parameters [here](https://huggingface.co/docs/transformers/v4.33.0/en/model_doc/auto#transformers.AutoTokenizer.register.slow_tokenizer_class) contain two slow_tokenizer_class which should be a typo I think.

**Additional context**
Add any other context or screenshots about the feature request here.
![image](https://github.com/huggingface/hub-docs/assets/71265541/ff5e9bbf-4baa-4bac-a82a-035daa44cc2c)


Comments:
Hi! i think this issue should be moved to the `transformers` repo

@xukp20 Indeed, thanks for pointing out! Would you like to open a PR to fix the typo? This way you get the github contribution. 

@amyeroberts Thanks, but I've never opened a PR for transformers before and never learned about writing the doc of transformers, so it's hard to find where to fix it quickly. I believe this minor change does not worth the trouble, so it would be nice if you can fix it at your convenience or remind anyone else to do so. Never mind the contribution! :)


@xukp20, ok no worries. I've opened a PR to fix it. Thanks again for highlighting!

If you ever fancy opening a PR in this repo on day, here's a guide: https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md/#create-a-pull-request
Update logits_process.py docstrings
Author: larekrow
Pull request: yes

This PR fixes points 1 and 2 of #25970 by correcting the docstrings for `RepetitionPenaltyLogitsProcessor` and `EncoderRepetitionPenaltyLogitsProcessor`.

Point 3 is left untouched as it can be "fixed" by multiple ways: either enforcing `penalty > 1` or clarifying further in the docstrings.

@gante 


Comments: