noamaanMulla-03/transformers-issues huggingface dataset:
Adding padding token GPTj config for the tokenizer.
Author: jojivk73
Pull request: yes

# What does this PR do?

This PR adds pad_tokens to GPTJ  tokenizer. This was seen as needed when GPTJ uses GPT2 or other tokenizer for GLUE fine-tuning tasks. The changes are in line with ONNX GPTJ Config setting in the same file. If there is an alternate fix that would solve it, that can be added as well.

Fixes # (issue)
Adding padding for GLUE fine-tuning Tasks.

## Before submitting
## Who can review?

Models:
GPTJ

Library:
Integrations:

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:
- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
@sgugger , The GPTJOnnxConfig uses it though. Is there a better resolution for this in run_glue.py as it requires padding.

The `GPTJOnnxConfig` is not used anymore and only there for backward compatibility. Like I said, you can manually add that `pad_token_id` in the config to suit your needs, but it shouldn't be there by default.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[`RWKV`] Add Gradient Checkpointing support for RWKV
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/24831

As stated by the issue above, technically there is no reason to not support GC for RWKV model

cc @sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

This snippet worked on my end:

```python
import torch
from transformers import RwkvForCausalLM

model = RwkvForCausalLM.from_pretrained("RWKV/rwkv-4-169m-pile").to(0)
model.train()
model.gradient_checkpointing_enable()

dummy_input = torch.LongTensor([[1, 2, 3, 4]]).to(0)

outputs = model(dummy_input)
logits = outputs.logits

loss = logits.mean()
loss.backward()
```
I will assume things are working (+ the CI should be triggered once the model has `supports_gradient_checkpointing = True`) , merging!
Change logic for logging in the examples
Author: muellerzr
Pull request: yes

# What does this PR do?

Changes logic to check if we are doing distributed training by reading the `parallel_mode` instead, since `local_rank` will always (for the most part) be `>-1` with Accelerate now

Fixes # (issue)

Solves https://github.com/huggingface/transformers/issues/24924

Verified that it now shows as "distributed" when using multi-gpu, and `False` if not.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24956). All of your documentation changes will be reflected on that endpoint.
Fallback for missing attribute `Parameter.ds_numel`
Author: apoorvkh
Pull request: yes

#22193 added a parameter count for Deepspeed sharded models (i.e. using the `Parameter` attribute `ds_numel`). However, `Parameter` Tensors don't always have the `ds_numel` attribute (even when the model is sharded with Zero stage 3).

We can see how this is [alternatively handled in Deepspeed](https://github.com/microsoft/DeepSpeed/blob/ceccfa3ef68182384c6db1349fab43b9af3ed7f3/deepspeed/runtime/engine.py#L3220), by falling back to `Parameter.numel()` if `ds_numel` is not an attribute. I've added this fix to the function in question.

Fixes #24792

@stas00 @pacman100

Comments:
Sure -- imo my rewrite was more readable but I guess that's subjective 👍 

Anyway, I've reverted the other changes and added the fix. Lmk, thanks!

> Sure -- imo my rewrite was more readable but I guess that's subjective +1
> 
> Anyway, I've reverted the other changes and added the fix. Lmk, thanks!

If I may share why I wrote it this way:

From the perspective of a user who doesn't know anything about deepspeed the current version is easier to understand since they don't need to go into that branch. 

From the perspective of a deepspeed user your original version is easier to read.

As there are many more non-deepspeed users the former version is preferred.

We could actually rewrite all of this code into a single line of 

```
p.ds_numel if hasattr(p, "ds_numel") else p.numel()
```
and not even need to check if it's running under deepspeed zero-3, but then the reader will wonder what in the world `ds_numel` is ;)

Thank you again for your contribution, @apoorvkh 


_The documentation is not available anymore as the PR was closed or merged._
🌐 [i18n-KO] Translated `perf_infer_gpu_one.md` to Korean
Author: eenzeenee
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다! -->
# What does this PR do?

Translated the `perf_infer_gpu_one.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [ ] Check for missing / redundant translations (번역 누락/중복 검사)
- [ ] Grammar Check (맞춤법 검사)
- [ ] Review or Add new terms to glossary (용어 확인 및 추가)
- [ ] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
May you please review this PR? @sronger @TaeYupNoh @HanNayeoniee @sim-so

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- May you please review this PR? @sgugger, @ArthurZucker, @eunseojo -->

Comments:
🌐 [i18n-KO] Translated perf_infer_gpu_one.md to Korean
Author: eenzeenee
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다! -->
# What does this PR do?

Translated the `perf_infer_gpu_one.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
May you please review this PR? @sronger, @TaeYupNoh, @kj021, @HanNayeoniee, @eenzeenee, @sim-so

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- May you please review this PR? @sgugger, @ArthurZucker, @eunseojo -->

Comments:
RwkvForCausalLM does not support gradient checkpointing.
Author: jonataslaw
Pull request: no

### System Info

Is there some reason for RwkvForCausalLM does not support gradient checkpointing, since  RWKV-LM supports it?

@ArthurZucker and @younesbelkada

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

`model.gradient_checkpointing_enable()`

```
ValueError(f"{self.__class__.__name__} does not support gradient checkpointing.")
ValueError: RwkvForCausalLM does not support gradient checkpointing.
```

### Expected behavior

No errors, as long as RWKV-LM supports it.

Comments:
Thanks for reporting @jonataslaw 
https://github.com/huggingface/transformers/pull/24955 has introduced the GC support for RWKV models, can you try that out by installing `transformers` from source and let us know how it goes?

```bash
pip uninstall transformers
pip install git+https://github.com/huggingface/transformers.git
```

Thanks for the quick update!
I tested your PR, and it works like a charm.
However it stops with an error about 10% of training (during eval):

  File "/home/jonataslaw/miniconda3/lib/python3.9/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/home/jonataslaw/miniconda3/lib/python3.9/site-packages/transformers/models/rwkv/modeling_rwkv.py", line 645, in forward
    self._rescale_layers()
  File "/home/jonataslaw/miniconda3/lib/python3.9/site-packages/transformers/models/rwkv/modeling_rwkv.py", line 738, in _rescale_layers
    block.attention.output.weight.quant_state[0].div_(
RuntimeError: result type Float can't be cast to the desired output type Byte


Edit: NVM, it is a unrelated problem about inference.

Ohh, I get it, it is the nested quantization problem related in https://github.com/huggingface/transformers/issues/23848

Yes, sadly nested quantization is not supported for RWKV, please use the un-nested one ! 

Thanks for the update.
I will change my code and stay alert in case it changes in the future.

Thanks again for fixing the GC, it helps a lot.

Thanks very much @jonataslaw  !
🌐 [i18n-KO] Translated `generation_strategies.md` to Korean
Author: wonhyeongseo
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `generation_strategies.md` file of the documentation to Korean 😄 
Thank you in advance for your review!

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [ ] Check for missing / redundant translations (번역 누락/중복 검사)
- [ ] Grammar Check (맞춤법 검사)
- [ ] Review or Add new terms to glossary (용어 확인 및 추가)
- [ ] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 OSSCA 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- Team OSSCA, may you please review this PR?  -->
<!-- @wonhyeongseo, @keonju2, @harheem, @HongB1, @junejae -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- May you please review this PR? -->
<!-- @sgugger, @ArthurZucker, @eunseojo -->

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24977). All of your documentation changes will be reflected on that endpoint.
🌐 [i18n-KO] Translated `<perf_train_gpu_many>.md` to Korean
Author: hyunhp
Pull request: yes

# What does this PR do?

Translated the `<your_file>.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

May you please review this PR? @nuatmochoi, @bolizabeth, @heuristicwave, @mjk0618, @jungnerd, @hyunhp

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- May you please review this PR? @sgugger, @ArthurZucker, @eunseojo -->

Comments:
미리보기 문서 전 ready for request 진행하여 취소 후, reopen
[New model] Bark for realistic text-to-speech
Author: xenova
Pull request: no
Labels: name: New model

### Model description

As stated in their [README](https://github.com/suno-ai/bark/blob/main/README.md):

> Bark is a transformer-based text-to-audio model created by [Suno](https://suno.ai/). Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects. The model can also produce nonverbal communications like laughing, sighing and crying. To support the research community, we are providing access to pretrained model checkpoints ready for inference.


Some of their demos are quite amazing (albeit slightly creepy), being able to add "uhms" and "ahhs" in the synthesized audio. For example:

```
Hello, my name is Suno. And, uh — and I like pizza. [laughs] 
But I also have other interests such as playing tic tac toe.
```

https://user-images.githubusercontent.com/34592747/238155864-cfa98e54-721c-4b9c-b962-688e09db684f.webm


### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

GitHub repo: https://github.com/suno-ai/bark
Author: @gkucsko
Demo: https://huggingface.co/spaces/suno/bark
Model weights: Although not very well documented, [here](https://github.com/suno-ai/bark/blob/2c12023eb22868a633b76357b69d657b374736d9/bark/generation.py#L92-L119) is the portion of the code which links to the model weights. @Vaibhavs10 also looks to have uploaded them to the HF Hub [here](https://huggingface.co/reach-vb/bark-small) 🔥 







Comments:
Hello, if no one else is working on this, I would love to take a look and try to add this model to HuggingFace! 

https://github.com/huggingface/transformers/pull/23375

see https://github.com/huggingface/transformers/pull/24086
fix llama2 chat system prompt
Author: jphme
Pull request: yes

# What does this PR do?

Fixes the Llama 2 System Prompt so its consistent with METAs version.

When testing my finetuning script, I found that the official LLAMA Code and the Huggingface code returned different tokens for the same code, apparently due to different linebreaks.

Code to test:

```python
#from https://github.com/facebookresearch/llama/blob/6c7fe276574e78057f917549435a2554000a876d/llama/generation.py#L46C11-L46C11
DEFAULT_SYSTEM_PROMPT = """\
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information."""

tokenizer = LlamaTokenizer.from_pretrained("/Users/jph/dev2/models/Llama-2-7b-chat-hf")
token_llama=tokenizer.encode(DEFAULT_SYSTEM_PROMPT)
from transformers.models.llama.tokenization_llama import DEFAULT_SYSTEM_PROMPT as DEFAULT_SYSTEM_PROMPT_TRANSFORMERS
token_hf=tokenizer.encode(DEFAULT_SYSTEM_PROMPT_TRANSFORMERS_FAST)
for i in range(100):
    print (f"llama: {tokenizer.decode(token_llama[i])} , transformers: {tokenizer.decode(token_hf[i])}")
    if token_llama[i]!=token_hf[i]:
        print('!!!')
```
which returns:
```
...
llama: being , transformers: being
llama: safe , transformers: safe
llama: . , transformers: .
llama: Your , transformers: Your
llama: answers , transformers: ans
!!!
llama: should , transformers: wers
...
```

I don't know how big the difference is, but it surely doesn't make sense to deviate here (even if there is no performance issue and it's just for debugging reasons).

## Who can review?

@ArthurZucker @sgugger 


Comments:
> Hey! Thanks for catching this. There seems to be a missing space in our prompt, here: `Youranswers should not include...`.
> 
> I can't accept the PR in the current state as the formating is there to comply with our linters. If you can just add the missing space would be great! Thanks for catching this. We have a test in the conversational pipeline, which did not detect this!

sure; tried to fix the formatting and make it more readable....

Hey! Sorry, #24930 is ready, we'll merge it in favor of this one! Thanks a lot for pointing out and contributing! 🤗 

> Hey! Sorry, #24930 is ready, we'll merge it in favor of this one! Thanks a lot for pointing out and contributing! 🤗

sure, didnt see that there was another PR fixing this. 
But just for next time - why did the checks still fail? I have absolutely no idea from the CircleCI messages how to check myself or what the reason is, lines should be short enough?

You have to run the linter! The lines are probably too short this time and can be optimised haha! Use `make style` again after each changed.
fsdp fixes and enhancements
Author: pacman100
Pull request: yes

# What does this PR do?
1. Fixes #24724. Should be merged after https://github.com/huggingface/accelerate/pull/1753
2. Fixes #24568.  Should be merged after https://github.com/huggingface/accelerate/pull/1753


Comments:
_The documentation is not available anymore as the PR was closed or merged._
🌐 [i18n-KO] Fixed Korean and English `quicktour.md`
Author: wonhyeongseo
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Updated and fixed some issues on the `quicktour.md` file for the Korean and English documentation.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
=Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
@sgugger, @ArthurZucker, @eunseojo May you please review this PR?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger, @ArthurZucker, @eunseojo May you please review this PR?
Thank you so much for your support!
Fix missing spaces in system prompt of Llama2 tokenizer
Author: chenjoya
Pull request: yes

# What does this PR do?

This PR fixes a typo  in system prompt of Llama2 tokenizer. There are missing spaces compared to https://github.com/facebookresearch/llama/blob/main/llama/generation.py

Thank you.

Comments:
cc @ArthurZucker 

Hey thanks, a similar PR is also #24935. Same comment would apply here, make sure `make style` is green and should be good! 

Hi, thank you so much for your help. Seems it still cannot go green. I am not very familiar with it ... Could you give me some guidance? ;)

_The documentation is not available anymore as the PR was closed or merged._