noamaanMulla-03/transformers-issues huggingface dataset:
[i18n-<languageCode>] Translating docs to <languageName>中文
Author: Amyao855
Pull request: no
Labels: name: WIP

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @stevhliu and @MKhalusova for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [ ] [index.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.md) https://github.com/huggingface/transformers/pull/20180
- [ ] [quicktour.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.md) (waiting for initial PR to go through)
- [ ] [installation.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.md).

## Tutorial section
- [ ] [pipeline_tutorial.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.md)
- [ ]  [autoclass_tutorial.md](https://github.com/huggingface/transformers/blob/master/docs/source/autoclass_tutorial.md)
- [ ]  [preprocessing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.md)
- [ ]  [training.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.md)
- [ ]  [accelerate.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.md)
- [ ]  [model_sharing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.md)
- [ ]  [multilingual.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.md)

<!--
Keep on adding more as you go 🔥
-->

Comments:
Hi, thanks for your interest in translating the docs! There's already an ongoing translation effort for Chinese at https://github.com/huggingface/transformers/issues/26803. Feel free to coordinate your translation efforts there 🤗
Stop throwing cache warning
Author: LysandreJik
Pull request: yes

The cache warning is thrown at every `transformers` install in a new env because the file isn't found.

If the cache is empty, it's a new env, so the file created should already have a '1' instead of a '0'

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31694). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Dependencies: fix typo
Author: gante
Pull request: yes

# What does this PR do?

#32374: `speech` is not a known dependency, `torch-speech` is ([source](https://github.com/huggingface/transformers/blob/083e13b7c47f674b11c74d1b7c7ee7cd1241b406/setup.py#L303)) 🙃 

Note: CI is not broken, but the [daily build for this specific image](https://github.com/huggingface/transformers/actions/runs/10207817526/job/28243277190) is. The fact that this CI update is either manual or prone to errors is a great example of the need to update it :)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32389). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Wav2Vec2ProcessorWithLM doesn't handle unknown token well for BPE
Author: amritregmi26
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.42.3
- Platform: Linux-6.5.0-41-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.23.4
- Safetensors version: 0.4.3

### Who can help?
@ArthurZucker 

### Information
My own modified scripts

### Tasks
My own task or dataset (give details below)

### Reproduction

1. Create and train a BPE tokenizer with continuing_word_prefix = "▁"
2. Create decoder with the same tokenizer labels
3. Initialize Wav2Vec2ProcessorWithLM with feature_extractor, bpe tokenizer and decoder
4. After the third step, it throws missing token error for this token '⁇'
5. When I checked the code for pyctcdecode they have UNK_BPE_TOKEN = "▁⁇▁" but in processor with lm this token is not handled correctly. 

### Expected behavior

For BPE tokenizer (identified by decoder._alphabet.is_bpe), the unk token should be ▁⁇▁ and not ⁇

Comments:
Hey! Did you try simply setting the tokenizer.unk_token? 

Thank you for your response! I was curious why there is no dynamic handling for this part of the code. Is there any specific reason for not using the unk_bpe_token from pyctcdecoder? While I understand that the CTC architecture performs best with a character-level tokenizer, is that the reason for not incorporating this token?

I changed the source code and it worked perfectly but I was just curious about the exact reason behind this code:
```python
from pyctcdecode.alphabet import BLANK_TOKEN_PTN, UNK_TOKEN, UNK_TOKEN_PTN
if UNK_TOKEN_PTN.match(token):
                tokenizer_vocab_list[i] = UNK_TOKEN
```
Can't we just add an if condition and handle the token accordingly? I might be wrong in this case but I just want to understand it from the root level.

> dynamic handling for this part of the code 

We can't always expect people to use the default unk token for pyctcdecode no? 
If you feel like this can be easily done, feel free to open a PR! 

I got my answer. Thanks !!!
Add support for non-CUDA architectures at the same time Bitsandbytes is doing it
Author: sealad886
Pull request: no
Labels: name: Feature request

### Feature request

Currently, the helper/setup functions explicitly check for CUDA support:
https://github.com/huggingface/transformers/blob/8685b3c5d2dd2550527773d2a02499495a759e31/src/transformers/quantizers/quantizer_bnb_4bit.py#L60-L63

BNB is currently doing a project to enable support for other GPU backends:
[ALPHA TESTERS WANTED](https://github.com/TimDettmers/bitsandbytes?tab=readme-ov-file#alpha-testers-wanted-multi-backend-refactor-amd-gpu--intel-cpugpu-specific-bnb-backend-implementations)

### Motivation

Apple MPS support is being added for so many major players, it'd be great for the biggest one of all to support it as its dependencies do. Also would be good to not hard-code this kind of limitation so that code updates aren't necessary as dependent libraries update themselves...

### Your contribution

idea done

Comments:
cc @younesbelkada @SunMarc 

Thanks ! We could instead use an environment variable to temporarly bypass this check to easy let users experiment with transformers + bnb multi-backend refactor

See also: https://github.com/huggingface/transformers/pull/31098

WDYT @Titus-von-Koeller ?

cc for visibility @pnunna93 @Xia-Weiwen @jianan-gu @matthewdouglas for others involved in the multi backend refactor

I think we're probably going to need some changes in `accelerate` as well where there's similar device checks.

Something we could consider is the addition of an 'a' suffix on the version number in the refactor branch and check against that. An environment variable sounds reasonable too.

Will be resolved by #31098. However, Apple Silicon support is still not implemented. For now we will work on enabling use of BNB AMD and Intel backends and once the Apple Silicon implementation will be provided by the community, we might add further necessary tweaks in a separate PR to Transformers.
🌐 [i18n-KO] Translated `<hqq>.md` to Korean
Author: mjk0618
Pull request: yes

# What does this PR do?

Translated the `<hqq>.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
<!-- @jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave -->
<!-- @win2dvp21, @enchantee00, @fabxoe, @1kmmk1, @junejae -->
<!-- @SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd -->
@mjk0618, @cjfghk5697, @ahnjj, @JinukHong, @nuatmochoi
<!-- @010kim, @chhaewxn, @boyunJang, @jeongiin, @harheem -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
<!-- @stevhliu May you please review this PR? -->

Comments:
한번 간단하게 변경해보았습니다..! 좋은 리뷰 감사합니다!
Fix get large model config for Switch Transformer encoder only tester
Author: JuanFKurucz
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

The current provided model `switch_base_8` does not exist in huggingface, the test code uses `google/switch-base-8` in the class above.


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Generate: fix end to end compilation
Author: gante
Pull request: yes

# What does this PR do?

As @zucchini-nlp noticed, `RUN_SLOW=1 py.test tests/models/llama/test_modeling_llama.py::LlamaModelTest::test_generate_compile_fullgraph` was failing on `main`

This PR fixes it.

Ran locally:
- slow llama tests
- slow cache tests [NOTE: the torch.export test is failing, and was failing since it was introduced.]

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32465). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Correct, was caught by the slow ci :)
support copies
Author: ArthurZucker
Pull request: yes
Labels: name: run-slow

# What does this PR do?
We can't copy the cache 😢 inheriting from module fixes this easily
This renders us unable to re-use prompts / system prompt like this:
```python 
import os, torch, copy
from transformers import AutoModelForCausalLM, AutoTokenizer, DynamicCache
device = "cuda"
ckpt = "path-to-ckpt"
INITIAL_PROMPT = "From now on, you are going to answer all my questions with historical details. Make sure to always add a bit of french here and there, for style."

model = AutoModelForCausalLM.from_pretrained(ckpt, torch_dtype=torch.float16)
model.to(device)

tokenizer = AutoTokenizer.from_pretrained(ckpt)

prompt_cache = DynamicCache()
inputs = tokenizer(INITIAL_PROMPT, return_tensors="pt").to("cuda")
prompt_cache = model(**inputs, past_key_values = prompt_cache).past_key_values


prompt = "Why are french people obsessed with french?"
new_inputs = tokenizer(INITIAL_PROMPT + prompt, return_tensors="pt").to("cuda")
past_key_values = copy.deepcopy(prompt_cache)
outputs = model.generate(**new_inputs, past_key_values=past_key_values,max_new_tokens=20) 
response = tokenizer.batch_decode(outputs)[0]
print(response)
"""
"""

prompt = "What is the best city to swim in?"
new_inputs = tokenizer(INITIAL_PROMPT + prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**new_inputs, past_key_values=copy.deepcopy(prompt_cache),max_new_tokens=20) 
response = tokenizer.batch_decode(outputs)[0]
print(response)
"""


```

Comments:
> We can't copy the cache 😢

What kind of copying are we talking about here? Like cache.copy? 

@amyeroberts `copy.deepcopy`

On main, without the fix, we get 
```
RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment.  If you were attempting to deepcopy a module, this may be because of a torch.nn.utils.weight_norm usage, see https://github.com/pytorch/pytorch/pull/103001
```

Cache copying is needed to reuse the cache from the prompt. E.g. to run new prompts on top of the system prompt without spending compute on the system prompt.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32159). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I'm sorry if it's not the right place to ask this question, but.

In Llama.cpp it's trivial to save and load state to/from disk to maintain the cache between sessions. Is it currently possible with Transformers, and if yes, could you please provide a minimal example or point to docs?

Cheers,

@vladfaust yes it is possible, but it requires custom code (i.e. you would need to store and restore the cache's tensors).

We will add a user-friendly API for that in the future :)

Ps this was actually already merged in #32168 so I'll close this one!
Experimental TFLite fix for Whisper
Author: Rocketknight1
Pull request: yes
Labels: name: run-slow

Don't merge, this will probably break some tests!

Fixes #32125 
Fixes #19691

Comments:
Hi @sanchit-gandhi @amyeroberts, this PR fixes an issue with TFLite conversion of TFWhisper models. However, the cause of the bug was that our forced decoding logits processor sets unwanted logits to `-inf`, and that breaks TFLite. This PR uses `-10000.` instead, which should be within the `float16` representable range.

The output of the sampling before and after this PR should be the same, because a logit of `-10000` will yield a probability of 0 after softmax at any precision. However, it does create a difference in the logits between Torch and TF. This doesn't seem to break any of our cross-tests, though, even when I ran the slow ones locally.

As such, I think this is safe to merge, and I'll double-check with @dsame that it fixes the issue. Let me know if you're opposed, though!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32301). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Ah! Given @sanchit-gandhi's comment, we probably shouldn't merge this PR - instead we should try to refactor the TF generation to not use forced decoder IDs, which avoids the problem entirely.
`pip install accelerate` (and similar) error messages should specify min version
Author: mikelmcdaniel
Pull request: no

### System Info

- `transformers` version: 4.41.2
- Platform: Linux-5.15.0-97-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.23.4
- Safetensors version: 0.4.3
- Accelerate version: 0.31.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 1.12.1+cu102 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

@muellerzr

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Have accelerate installed at a low version (e.g. 0.10.0)
2. Run the following code:
```python
import torch
from transformers import pipeline

pipe = pipeline("text-generation", model="TinyLlama/TinyLlama-1.1B-Chat-v1.0", torch_dtype=torch.bfloat16, device_map="auto")
```
3. Error message "ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate`" pops up


Ideally, the error message would be "ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install 'accelerate>=0.21.0'`"

---

### Expected behavior

Error message would specify the `utils.import_utils.ACCELERATE_MIN_VERSION`:

"ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install 'accelerate>=0.21.0'`"

Comments:
Potential fix (that I didn't test): https://github.com/mikelmcdaniel/transformers/commit/157e20f9a57b097b4c756558d12834696527ebb3

cc @muellerzr @SunMarc this seems like valuable feedback

Yes agreed, I'm reworking the imports today so I'll make sure to run through any accelerated-related errs like this to specify min versions :)
MLU Check Breaks Trainer at Import Time for Older Accelerate Versions
Author: alex-jw-brooks
Pull request: no
Labels: name: trainer

name: bug

name: Accelerate

### System Info

- `transformers` version: 4.44.0.dev0
- Platform: Linux-4.15.0-206-generic-x86_64-with-glibc2.27
- Python version: 3.9.16
- Huggingface_hub version: 0.24.5
- Safetensors version: 0.4.4
- Accelerate version: 0.29.0
- PyTorch version (GPU?): 2.4.0+cu121 (True)
- Tensorflow version (GPU?): 2.15.1 (True)
- Flax version (CPU?/GPU?/TPU?): 0.7.0 (cpu)
- Jax version: 0.4.13
- JaxLib version: 0.4.13
- GPU type: Tesla V100-PCIE-32GB

### Who can help?

@muellerzr @SunMarc

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

- Install the latest version of transformers with `accelerate==0.21.0`, the current minimum supported version.
- Try `from transformers import trainer`

For older versions of Accelerate, this import will throw a runtime error caused by:
```
ImportError: cannot import name 'is_mlu_available' from 'accelerate.utils' /Users/alexanderjbrooks/transformers/venv/lib/python3.9/site-packages/accelerate/utils/__init__.py)
```



### Expected behavior

We should be able to import the trainer with the minimum supported version of Accelerate! I will submit a PR to this effect.

Comments:
Hey @alex-jw-brooks, thanks for the report ! This should be fixed by this PR https://github.com/huggingface/transformers/pull/32292 that was created a few days ago. Let us know this this fixes your issue ! 

Looks good to me, thanks @SunMarc! I'll close my PR and this issue