noamaanMulla-03/transformers-issues huggingface dataset:
🌐 [i18n-KO] Translated `bitsandbytes.md` to Korean
Author: gongsoonyee
Pull request: yes

# What does this PR do?

Translated the `bitsandbytes.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
<!-- @jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave -->
<!-- @win2dvp21, @enchantee00, @fabxoe, @1kmmk1, @junejae -->
@SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd
<!-- @mjk0618, @cjfghk5697, @ahnjj, @JinukHong, @nuatmochoi -->
<!-- @010kim, @chhaewxn, @boyunJang, @jeongiin, @harheem -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR? 

Comments:
좋은 번역 감사합니다! 
PR 서식 맨 첫 줄 `Translated the <your_file>.md file of the documentation to Korean.` 부분의 `<your_file>.md`를 번역하신 문서로 수정해주세요!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32408). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Nice job, thanks for the translation! Can you check if whether removing the "?" in some of the sentences alters the meaning? Thanks! 🤗

Thank you so much for checking! Your suggestion was spot on, so I've made the changes accordingly.
Set ffmpeg_microphone input to system default on Darwin platform
Author: ruisilvestre
Pull request: yes

# What does this PR do?

Currently, the device index used for audio capture using ffmpeg_microphone is 0, which does not always work as expected. This PR sets the default audio capturing device on the Darwin platform to `:default` which is the ffmpeg supported way to do this.

Fixes #28154

There is no documentation or tests referencing ffmpeg_microphone so I did not update any of it.

I have created a local small test (similar to the [voice assistant tutorial](https://huggingface.co/learn/audio-course/chapter7/voice-assistant) to prove that switching between external mic (airpods) and Mac audio capture works ok.
```
from transformers.pipelines.audio_utils import ffmpeg_microphone_live, ffmpeg_microphone
from transformers import pipeline
import sys
import torch

device = "cuda:0" if torch.cuda.is_available() else "cpu"
transcriber = pipeline(
    "automatic-speech-recognition", model="openai/whisper-small.en", device=device
)

def transcribe(chunk_length_s=15.0, stream_chunk_s=1.5):
    sampling_rate = transcriber.feature_extractor.sampling_rate

    mic = ffmpeg_microphone_live(
        sampling_rate=sampling_rate,
        chunk_length_s=chunk_length_s,
        stream_chunk_s=stream_chunk_s,
    )

    print("Start speaking...")
    for item in transcriber(mic, generate_kwargs={"max_new_tokens": 2048}):
        sys.stdout.write("\033[K")
        print(item["text"], end="\r")
        if not item["partial"][0]:
            break

    return item["text"]

if __name__ == "__main__":
    transcribe()
```

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ylacombe @Narsil 



Comments:
Hi @ruisilvestre, thanks for opening this PR! 

I can see that the device selection changes here were incorporated in #31846. Can you confirm the behaviour is now as expect on `main`?
🌐 [i18n-KO] Translated `trainer.md` to Korean
Author: cjfghk5697
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `trainer.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [ ] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
<!-- @jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave -->
<!-- @win2dvp21, @enchantee00, @fabxoe, @1kmmk1, @junejae -->
<!-- @SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd -->
@mjk0618, @cjfghk5697, @ahnjj, @JinukHong, @nuatmochoi
<!-- @010kim, @chhaewxn, @boyunJang, @jeongiin, @harheem -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32260). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@stevhliu Thank you for reviewing! I have revised the document to resolve the conflict. Could you please re-review this PR?
Fixed test `test_static_cache_exportability` with torch 2.4.0
Author: guangy10
Pull request: yes

# What does this PR do?

A workaround to run test `test_static_cache_exportability` with torch 2.4.0.



## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. Issue discussed in the channel.
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@gante 

Comments:
Fixed import compatible version. Should be good to go.
Translation docs claim two different tokenizers are used in translation for input and output language, which is incorrect
Author: frereit
Pull request: no
Labels: name: Documentation

https://github.com/huggingface/transformers/blob/b51d4145bbfa9bd90dc878e552d5cdc5133dabc2/docs/source/en/tasks/translation.md?plain=1#L93

<https://huggingface.co/docs/transformers/en/tasks/translation#preprocess> claims:

> Tokenize the input (English) and target (French) separately because you can’t tokenize French text with a tokenizer pretrained on an English vocabulary.

However, in the code below (`preprocess_function`), clearly there's only one tokenizer in use:

```
model_inputs = tokenizer(inputs, text_target=targets, max_length=128, truncation=True)
```

Comments:
cc @stevhliu
🌐 [i18n-KO] Translated `image_to_image.md` to Korean
Author: shinhyunji36
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `image_to_image.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
@jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave
<!-- @win2dvp21, @enchantee00, @fabxoe, @1kmmk1, @junejae -->
<!-- @SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd -->
<!-- @mjk0618, @cjfghk5697, @ahnjj, @JinukHong, @nuatmochoi -->
<!-- @010kim, @chhaewxn, @boyunJang, @jeongiin, @harheem -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR?

Comments:
번역하느라 고생하셨습니다!😄 @shinhyunji36

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32327). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
🌐 [i18n-KO] Translated `ko-llm_tutorial_optimization.md` to Korean
Author: 010kim
Pull request: yes

# What does this PR do?

Translated the `ko-llm_tutorial_optimization.md` file of the documentation to Korean. Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
<!-- @jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave -->
<!-- @win2dvp21, @enchantee00, @fabxoe, @1kmmk1, @junejae -->
<!-- @SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd -->
<!-- @mjk0618, @cjfghk5697, @ahnjj, @JinukHong, @nuatmochoi -->
@010kim, @chhaewxn, @boyunJang, @jeongiin, @harheem

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32372). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@stevhliu No worries! I'm quite comfortable with English. Thank you for the approval.
Gemma2: fix FA2 generation
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As reported in https://github.com/huggingface/transformers/issues/32309#issuecomment-2277201220 Gemma2 FA2 is broken in the latest release due to applying 4D mask when preparing inputs. FA2 expects a 2D mask so we can skip that step if FA2 is set

Ran the FA2 generate tests, everything passing, except for the Phi3 which is already fixed in another PR for patch release

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32553). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Yes, I'll add link to this PR in the internal slack thread for patches :)

Already added 😉
Change Phi3 `_supports_sdpa` to True
Author: pocca2048
Pull request: yes

# What does this PR do?
Change Phi3 model's sdpa support from False to True.

Fixes #31863 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ArthurZucker



Comments:
@ArthurZucker I think it is all green now~!

Thanks!
Can I use "attn_implementation" in model config file 
Author: hanwen-sun
Pull request: no

hi, I want to use the examples/pytorch/language-modeling/run_clm.py to train my model. But I find that the only way to use flash_attention is to modify the code in run_clm.py like:
```python
         model = AutoModelForCausalLM.from_pretrained(
             model_args.model_name_or_path,
             from_tf=bool(".ckpt" in model_args.model_name_or_path),
             trust_remote_code=model_args.trust_remote_code,
             torch_dtype=torch_dtype,
             low_cpu_mem_usage=model_args.low_cpu_mem_usage,
+            attn_implementation="flash_attention_2",
         )
     else:
-        model = AutoModelForCausalLM.from_config(config, trust_remote_code=model_args.trust_remote_code)
+        model = AutoModelForCausalLM.from_config(
+            config,
+            trust_remote_code=model_args.trust_remote_code,
+            attn_implementation="flash_attention_2",
+        )
```

So can I use "attn_implementation" in model config like:
```json
{
  ...
  "attn_implementation": "flash_attention_2"
}
```
I found a discussion here(in model_utils.py):
https://github.com/huggingface/transformers/pull/26572/files/3f06a3a0aec8cc1ec3ad6bf66ebe277392c5ab37#r1502470664
It seems that the "attn_implementation" will not be exposed in the config file.

Can any one give me an answer? thanks.

Comments:
Hi @hanwen-sun, thanks for opening this issue! 

If you're modifying the config file, the parameter to add is `_attn_implementation` i.e.:

```
{
  ...
  "_attn_implementation": "flash_attention_2"
}
```

If you're instantiating a config class, then you can pass in `attn_implementation` directly e.g.:

```
config = AutoConfig.from_pretrained(checkpoint, attn_implementation="flash_attention_2")
```

We should likely permit the `attn_implementation` config value to change it, rather than relying on the private attribute (cc @ArthurZucker)

@amyeroberts thank you for reply, I add the "_attn_implementation" in config and it don't work. I think the config._attn_implementation in transformers model come from the "attn_implementation" outside.

@hanwen-sun Could you share how you're creating the model? If `attn_implementation` is being passed into the `from_pretrained` argument then this will override the value in the config. You should be able to do:

```py
model = AutoModelForXxx.from_pretrained(checkpoint, attn_implementation="flash_attention_2")
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[Apple Intelligence Foundation Model]
Author: gyin94
Pull request: yes

# What does this PR do?

This PR adds minor change in Llama model implementation for Apple Foundation Model architecture used for Apple Intelligence, mentioned in this [paper](https://arxiv.org/pdf/2407.21075), for open research.

```python
from transformers import LlamaModel, LlamaConfig

# Apple Intelligence on device 2.7B model config.
afm_on_device_config = LlamaConfig(
    vocab_size=49152,
    hidden_size=3072,
    intermediate_size=8064,
    num_attention_heads=24,
    num_hidden_layers=26,
    num_key_value_heads=8,
    tie_word_embeddings=True,
    layer_norm_epsilon=1e-05,
    max_position_embeddings=32768,
    rms_norm_eps=1e-05,
    pad_token_id=0,
    bos_token_id=1,
    eos_token_id=1,
    rope_theta=500000.0,
    qk_norm=True,
)

# Initialize the model with the specified configuration.
afm_on_device_model = LlamaModel(config=afm_on_device_config)
```

Notes:
- Weights are not open sourced.
- After adding this change, we will add the parameter converter (from jax to safetensors) into AFM training framework, [axlearn](https://github.com/apple/axlearn/) for internal use cases. [Converter example](https://github.com/apple/axlearn/blob/a962a839130c1e9d63bf0b86b9be2dd6637b547f/axlearn/common/param_converter_test.py#L643)

Open questions:
- Will leave it to huggingface maintainers to decide whether it would be preferred to create a standalone `models/afm/` repo for apple foundation models. We may have future architecture change for research. If so, would appreciate if huggingface team can help add it to avoid duplicate code and tests

Models:
- text models: @ArthurZucker

Contact: gyin@apple.com


Comments:
cc @ArthurZucker 

Hi @gyin94, thanks for opening a PR! 

We only add models with open-source weights available for the community to use. Would Apple be open to releasing some weights? This would be jointly beneficial as we can add the model so that you can utilise transformers and any future features and would serve as a strong promotion for the work and research you're doing. 

> Hi @gyin94, thanks for opening a PR!
> 
> We only add models with open-source weights available for the community to use. Would Apple be open to releasing some weights? This would be jointly beneficial as we can add the model so that you can utilise transformers and any future features and would serve as a strong promotion for the work and research you're doing.

Hi @amyeroberts , unfortunately we will not open source the weight so far. This PR can be useful for our internal usage and open research on top of huggingface/transformers. If this is not aligned with the current repo's design. We will try to add a reference change in axlearn then. Thanks.

😭 

😭
Fix a bug in Qwen2Audio
Author: faychu
Pull request: yes

Found a bug after merging `Add Qwen2audio` into main branch.
@ylacombe @amyeroberts 

Comments:
Slow tests passed locally according to @faychu, is it ok if I merge @ArthurZucker ?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32552). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
add Gemma2 for causal LM to test classes
Author: molbap
Pull request: yes

# What does this PR do?

Generate tests depend on the attribute `all_generative_model_classes`, which was empty for Gemma2, causing all generate tests to be skipped. 
Q additional maybe @gante : do you think it'd be useful to use the lists in `modeling_auto.py` instead?


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32558). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Right, I'm seeing the CI go redder and redder :sweat_smile: , might not be relevant to add then - I'll close this PR and we can rely on the other specific test from #32493
Quickfix generate tests
Author: molbap
Pull request: yes

# What does this PR do?

(Should) fix broken tests on main for generate testing, tests broken not caught in https://github.com/huggingface/transformers/pull/32493 .

cc @ArthurZucker 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32546). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

The tests that do run work, but the fetcher doesn't it seems :grimacing: other than that seems to pass