noamaanMulla-03/transformers-issues huggingface dataset:
🌐 [i18n-KO] Translated `language-modeling.mdx`
Author: wonhyeongseo
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `language-modeling.mdx` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `language-modeling.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
@sgugger, @ArthurZucker, @eunseojo May you please review this PR?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger, @ArthurZucker, @eunseojo May you please review this PR?
Possible bug in `forced_decoder_ids` in `modeling_whisper.py`
Author: akuzeee
Pull request: no

### System Info

N/A

### Who can help?

@sanchit-gandhi @connor-henderson

Hi, thank you for the great work!

https://github.com/huggingface/transformers/blob/796162c51298547c357b20cc33d64cbcf77d0241/src/transformers/models/whisper/modeling_whisper.py#L1649 might have a bug (which is introduced in this [PR](https://github.com/huggingface/transformers/pull/22496)).

According to a comparison with openAI's whisper, I think the order of the elements should be as follows:
```python
            forced_decoder_ids = [
                # Slicing the text prompt ids in a manner consistent with the OpenAI implementation
                # to accomodate context space for the prefix (see https://github.com/openai/whisper/blob/c09a7ae299c4c34c5839a76380ae407e7d785914/whisper/decoding.py#L599)
                generation_config.decoder_start_token_id,
                *text_prompt_ids[-self.config.max_length // 2 - 1 :],
                *[token for _rank, token in non_prompt_forced_decoder_ids],
            ]
```


### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

N/A

### Expected behavior

N/A

Comments:
Hey @akuzeee! Cool to see you've been going through the Transformers Whisper codebase! We actually update the decoder start token id prior to setting the forced decoder ids:
https://github.com/huggingface/transformers/blob/796162c51298547c357b20cc33d64cbcf77d0241/src/transformers/models/whisper/modeling_whisper.py#L1636-L1637

The decoder start token id is set to the `
` token (or `self.tokenizer.sot_prev` in the Whisper codebase), and is always forced as the beginning of sequence (BOS) token during generation (i.e. the first token in our sequence). So the decoder start token id is handled outside of the forced decoder ids. 

We then add the `
` token after the prompts, which indicates the end of the prompt and start of the generation. It seems like the OpenAI codebase has this `
` token included as part of their `tokens`, i.e. looking at the code from https://github.com/openai/whisper/blob/c09a7ae299c4c34c5839a76380ae407e7d785914/whisper/decoding.py#L597:
```python
            tokens = (
                [self.tokenizer.sot_prev]  # this is equivalent to us setting the sot prev start token id as the decoder start token id
                + prompt_tokens[-(self.n_ctx // 2 - 1) :]  # this is the same as our code
                + tokens  # these tokens include the bos decoder start token id
            )
```

Thank you for the detailed and kind explanation! I understand now.
Generate: increase left-padding test atol
Author: gante
Pull request: yes

# What does this PR do?

As stated in #23437, `GPTBigCode` was failing the left padding test. Upon further investigation:
1. `GPTBigCode` is left padding compatible, it accepts `position_ids` and uses them correctly
2. The test is only failing on CPU
3. The diff extremely small...

...so I've raised `atol` to `1e-7` (instead of the default `1e-8`) 👀 With `1e-7`, we can still detect failing cases, like the ones skipped in #23437 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add support for non-rust implemented tokenization for `__getitem__` method.
Author: jacklanda
Pull request: yes

# Overview
This PR is going to add a support for the usage scenario of "getting a slice from the batch-tokenized sequences".

Without this PR, it seems to raise KeyError with the following message KeyError: 'Indexing with integers (to access backend Encoding for a given batch index) is not available when using Python based tokenizers'

P.S. The above scenario could be reproduced by using some models new uploaded but not support to Rust-implemented tokenization, such as fnlp/moss-moon-003-sft. Also we can run the following examplar script for reproducing this issue:

```python
# test script `/home/workspace/test.py` for this PR. 
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("fnlp/moss-moon-003-sft", trust_remote_code=True)
tok.add_special_tokens({"pad_token": "[PAD]"})

texts = ["Today is a good day!", "It's a good idea!", "How's going?"]
batch_tok = tok(texts, padding=True)
print(batch_tok[0:3])  # report `KeyError` here
```
# Error Message
```txt
Traceback (most recent call last):
  File "/home/workspace/test.py", line 8, in <module>
    print(batch_tok[0:3])  # report `KeyError` here
  File "/home/app/anaconda3/envs/test/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 242, in __getitem__
    raise KeyError(
KeyError: 'Indexing with integers (to access backend Encoding for a given batch index) is not available when using Python based tokenizers'
```

All in all, I think it seems useful to implement __getitem__ method behind it in Python side :)

Note that this PR is associative with the previous closed one. 
https://github.com/huggingface/transformers/pull/23645

Comments:
It seems that failed in workflow due to `Read Timeout` on T5-relevant testing.
How can I rerun for this?

![图片](https://github.com/huggingface/transformers/assets/54089835/f9922d94-440f-42d6-92b7-69f38e604d81)
![图片](https://github.com/huggingface/transformers/assets/54089835/4a1eec18-f2cf-4509-93aa-1c254286fe2e)


We can re-run that for you 😉 


_The documentation is not available anymore as the PR was closed or merged._

Request for review :)

@jacklanda Could you update the error message as requested by @ArthurZucker? 

> @jacklanda Could you update the error message as requested by @ArthurZucker?

@amyeroberts Have updated the mentioned error messages by @ArthurZucker 
Thanks.

Ask for final review :)
Error with pip install in Colab Notebook
Author: sonnyarora
Pull request: no

### System Info

Google Colab

### Who can help?

@younesbelkada 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1) Copy the colab notebook from the page https://huggingface.co/docs/transformers/perf_infer_gpu_one linked at https://colab.research.google.com/drive/1qOjXfQIAULfKvZqwCen8-MoWKGdSatZ4?usp=sharing.
2) Run First Cell to pip install packages. I get the following error:
ERROR: Could not find a version that satisfies the requirement bitsandbyte (from versions: none)
ERROR: No matching distribution found for bitsandbyte

### Expected behavior

I expect the cell to run without error. When I replace
```
!pip install --quiet bitsandbyte
```
with
```
!pip install --quiet bitsandbytes
```
I get the desired behavior.

Comments:
Thank you so much for flagging, just fixed the notebook. Closing the issue, feel free to re-open it if you see more issues!
[`Hub`] Add `safe_serialization` in push_to_hub
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR adds the possibility to directly push safetensors weight on the Hub, as the `save_pretrained` method was called with default args and it is currently not possible to pass kwargs to use `safe_serialization=True`.

cc @sgugger @Narsil 

Related: https://github.com/huggingface/peft/pull/553

also cc @pacman100 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
BitsAndBytesConfig llm_int8_skip_modules does not work in the new version
Author: AntoineBlanot
Pull request: no

### System Info

- `transformers` version: 4.30.0.dev0
- Platform: Linux-5.4.0-148-generic-x86_64-with-glibc2.17
- Python version: 3.8.16
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


### Who can help?

@ArthurZucker @younesbelkada

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import RobertaForSequenceClassification, BitsAndBytesConfig
quantization_config = BitsAndBytesConfig(load_in_8bit=True, llm_int8_skip_modules=['classifier'])
model = RobertaForSequenceClassification.from_pretrained('roberta-large-mnli', quantization_config=quantization_config)
```

### Expected behavior

The 'classifier' layer should be in Float16 but is actually loaded in 8bit. This is problematic because it drastically lower the performance of the model. It also make it impossible to train it (using peft library for example).

Comments:
Hi @AntoineBlanot 
Thanks for the issue and flagging! 
https://github.com/huggingface/transformers/pull/24043 should fix the issue!
[`bnb`] Fix bnb skip modules
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/24037

https://github.com/huggingface/transformers/pull/23479 removed by mistake the logic introduced in https://github.com/huggingface/transformers/pull/21579 to deal with modules that are not needed to be converted

The PR also adds a nice test to make sure this will never happen again



Comments:
_The documentation is not available anymore as the PR was closed or merged._
device_map issue with M1_M2 MacOS 
Author: phdykd
Pull request: no

### System Info

Hi,
I am on M2 MAX CHIP MACOS that has 12 CPU, 38 GPU. I am having issue with ever modification of this code snippet. Would you please tell me how I can correct it?


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction


from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-40b-instruct", trust_remote_code=True)
model = model.to('mps')

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
torch_dtype=torch.bfloat16,
trust_remote_code=True,
# device = torch.device('mps'),
# device_map="auto",
)

### Expected behavior



from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-40b-instruct", trust_remote_code=True)
model = model.to('mps')

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
torch_dtype=torch.bfloat16,
trust_remote_code=True,
# device = torch.device('mps'),
# device_map="auto",
)

Comments:
This does not work yet indeed., this is known and is on our roadmap.

Hi,

Thanks for the reply.

I hope this will come into play. I purchased very expensive MacOS to continue on ML, but apparently I can not use it for now to implement this type of things. 

Is there a way to import pdf and then chat with it using this falcon-40b model?





Thanks
Regards

> On May 31, 2023, at 1:29 PM, Sylvain Gugger ***@***.***> wrote:
> 
> 
> This does not work yet indeed., this is known and is on our roadmap.
> 
> —
> Reply to this email directly, view it on GitHub <https://github.com/huggingface/transformers/issues/23916#issuecomment-1570629684>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARF6VBUWSXW76NC2SMHKAI3XI55VZANCNFSM6AAAAAAYVX5XTI>.
> You are receiving this because you authored the thread.
> 



Hi @phdykd , we have added the support with M1/M2 MacOS. Let us know if works. 

Hi @SunMarc,
Please share the link with me.
Thanks for your support.

Hi @phdykd , you just need to install the latest version of accelerate (main branch).
Pop
Author: jamesthesnake
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
🌐 [i18n-KO] Translated `tasks_explained.mdx` to Korean
Author: 0525hhgus
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.mdx` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `tasks_explained.mdx` file of the documentation to Korean 😄 
~~*Reference documents I added: `generation_strategies.mdx`, `task_summary.mdx`~~
Thank you in advance for your review!

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- Team PseudoLab, may you please review this PR?  -->
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
May you please review this PR?
@sgugger, @ArthurZucker, @eunseojo 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

특별히 수정할 만한 곳을 찾지 못했습니다. 고생하셨습니다!

> 무척 길고 알찬 문서였네요! 고생 많으셨습니다. 몇 가지 수정 의견을 아래와 같이 제안 드립니다 😄

꼼꼼한 리뷰 감사합니다! 리뷰 주신 사항 반영하여 커밋하였습니다 👍 

May you please review this PR? 😄 
@sgugger, @ArthurZucker, @eunseojo
Be nice to TF
Author: ydshieh
Pull request: yes

# What does this PR do?

... and to @Rocketknight1 

To be serious: to avoid OOM issue introduced in #23234. Note `torch_job` use `pytest_num_workers=3`.

See [this comment](https://github.com/huggingface/transformers/pull/24071#issuecomment-1580778679). 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

LGTM! If the torch value is `3` we could probably reduce the TF value even lower, but let's try this first.

`7` avoids the issue, but still 96~98% memory. Change it to `6` and will merge.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24076). All of your documentation changes will be reflected on that endpoint.
Convnext flax
Author: Shubhamai
Pull request: yes

# Flax Implementation of `facebook/convnext-tiny-224`

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Flax: @sanchit-gandhi

## TODO

Last Updated : 10 Feb, 2023

- [x] Fixing tests failed in `ci/circleci: tests_flax` actions.   
- [ ] Uploading [Shubhamai/convnext-tiny-224](https://huggingface.co/Shubhamai/convnext-tiny-224) flax weights to [facebook/convnext-tiny-224](https://huggingface.co/facebook/convnext-tiny-224).
- [x] Depends on merge of [_Improve flax test_attention_outputs_ PR](https://github.com/huggingface/transformers/pull/21486) to pass (or technically skip) `test_attention_outputs` test.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21485). All of your documentation changes will be reflected on that endpoint.

@sanchit-gandhi this PR is also ready for your review in case it was missed. And again, thank you so much for taking the time to review the PR.

@sanchit-gandhi Reminder incase my previous message got missed! Also the https://github.com/huggingface/transformers/pull/21472 ( previous reviews implemented ) and https://github.com/huggingface/transformers/pull/21867 PR are awaiting review. Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix expected value in tests of the test fetcher
Author: sgugger
Pull request: yes

# What does this PR do?

#24051 broke a test in the test suite of the test fecther. The PR did not run the CI because the modification was detected as a docstring modification. This is due to a """ in the middle of the file that this PR also fixes.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24077). All of your documentation changes will be reflected on that endpoint.