noamaanMulla-03/transformers-issues huggingface dataset:
🌐 [i18n-KO] Translated `bertology.mdx` to Korean
Author: wonhyeongseo
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `bertology.mdx` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `bertology.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- @sgugger, @ArthurZucker, @eunseojo May you please review this PR? -->

Comments:
Missed the first header in my first commit. Closing in favor of https://github.com/huggingface/transformers/pull/23968

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23967). All of your documentation changes will be reflected on that endpoint.
Auto-download is a security hole.
Author: freckletonj
Pull request: no

### System Info

I just ran a project and it decided to download a completely unrelated dataset, which I didn't want or need. The extraneous download was https://huggingface.co/datasets/allenai/c4, which upon inspection contains 800+ trojan viruses. Are these false positives? I shouldn't have to care unless I'm interested in this specific dataset.

I think any network calls should be strictly opt-in, eg pehaps `HF_NETWORK_ALLOWED=True python whatever.py`

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Steps to reproduce:

1. Run any HF model for the first time. It will make network calls, and download datasets and weights. 

### Expected behavior

0 network calls are made, unless opted in to.

Comments:
Hi @freckletonj, thanks for raising this issue. 

Without knowing which code you're running, it's hard to know what specifically triggered the dataset download (or how unrelated it is). Typically, a dataset would be downloaded if it's requested through the `load_dataset` functionality.  However, I see that allenai/c4 dataset [needs to be downloaded through `git clone`](https://huggingface.co/datasets/allenai/c4#how-do-i-download-this). In general, if you've spotted malicious content within a dataset, I'd recommend flagging on the repo (there's already an [open discussion here](https://huggingface.co/datasets/allenai/c4/discussions/2))

You can run transformers in a firewalled or offline mode setting `TRANSFORMERS_OFFLINE=1` in your environment. For datasets, this is `HF_DATASETS_OFFLINE=1`. See: https://huggingface.co/docs/transformers/v4.28.1/en/installation#offline-mode.  

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
whisper identified the wrong language
Author: LYPinASR
Pull request: no

### Feature request

When I follow the example of long-form transcription for whisper-large with Korean, the result is English. But after finetuning the whisper-large model with some Korean data, the checkpoint can output Korean. I also test other model size, but all the models output English. 
I was confused about it. How should I do to output Korean with the original model? 
Thank you!

### Motivation

Test whisper in Korean.

### Your contribution

Test whisper in Korean.

Comments:
Hi there. Questions like this are better suited on the [forums](https://discuss.huggingface.co/) or a discussion on the model page as we keep issues for bugs and feature requests only.

If you use pipeline, you should add option like 
generate_kwargs = {"task":"transcribe", "language":"
"}


ref1: https://colab.research.google.com/drive/1rS1L4YSJqKUH_3YxIQHBI982zso23wor#scrollTo=dPD20IkEDsbG
ref2: https://github.com/huggingface/transformers/issues/22331

however, I think default task should be "transcribe" not "translate". I insist It's an error.

I have solved the problem.
Step 1: Upgrade transformers, unfixed.
Step 2: Add option like "generate_kwargs = {"task":"transcribe", "language":"
"}", unfixed.
Step 3: Add a line like "pipe.model.config.forced_decoder_ids = pipe.tokenizer.get_decoder_prompt_ids(language="ko", task="transcribe")", fixed.

However, I still don't understand why the original model output is English but the fine-tuned model output is in Korean.

maybe you can checked your fine-tuned model's config.json  or generation_config.json, double check the default task type, I think it's null or "transcribe"

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Help on Firewalled installation
Author: SupreethRao99
Pull request: no

Hello,

I would like to install huggingface in a firewalled environment. I have a Git-lfs repository with all my models stored, I would like to use them with the transformers libraries' `.from_pretrained()` feature, I referred to this (https://github.com/huggingface/huggingface_hub/tree/main/src/huggingface_hub) to set up the git-repo, could someone help me use the models stored on my git instead of the huggingface hub. 

Thanks 

Comments:
That is not possible. You can only use models from the Hub or locally downloaded.

Thanks @sgugger 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Added tokenizer kwargs for fill mask pipeline
Author: sajeedmehrab
Pull request: yes

Added tokenizer kwargs for the fill mask pipeline, which enables to truncate/pad/specify max length etc for the tokenizer. Pipeline can be used as follows following the edit:

`from transformers import pipeline`
`fill_mask_pipeline = pipeline(`
`    'fill-mask', `
`    model=model, `
`    tokenizer=tokenizer, `
`    device=0`
`)`

`tokenizer_kwargs = {'truncation':True, 'max_length':2048}`
`output = fill_mask_pipeline("Text to predict <mask>", **tokenizer_kwargs)`


Comments:
cc @Narsil 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22995). All of your documentation changes will be reflected on that endpoint.

Can we refactor in order to do :

```python
 output = fill_mask_pipeline("Text to predict <mask>", tokenizer_kwargs=tokenizer_kwargs)
 ```
 Instead ? Accepting kwargs directly is very hard to maintain down the line because of clashing arguments (for instance `max_length` is one that pops up often enough).
  
 We can also whiteliste some parameters like `truncation` or `padding` to make them more convenient. but enabling all the kwargs directly is really not something we want I think.
 
 Thanks for the contribution though, it's a step in the good direction ! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Baai glm
Author: shunxing1234
Pull request: yes
Labels: name: WIP

add glm model

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17113). All of your documentation changes will be reflected on that endpoint.

As requested on slack, I just had a quick look at your PR. It's in good shape! :blush: 

Regarding the fast tokenizer part, to check its validity, I suggest we take a first stab by looking at the results of the tests you created. For the moment, it seems to me that we can't leverage the tests files you added -at least with the CI. To fix it, I think you need to add the model you are creating to the main init file (this one: https://github.com/huggingface/transformers/blob/main/src/transformers/__init__.py). To know what to add maybe the easiest way as Lysander mentioned earlier is to see on another branch the additions that are made when you add a model from another model with the cli command `transformers-cli add-new-model-like` (cf [this discussion](https://huggingface.slack.com/archives/C036A8ECY79/p1647333073740149)).

Then, concerning the tokenizer fast, I already see 2 small things: 1. it will be necessary to add a converter (cf the discussion we had [here](https://huggingface.slack.com/archives/C036A8ECY79/p1650009413440189?thread_ts=1649381592.438409&cid=C036A8ECY79)) and 2. I have the impression that there are new special tokens added - like `ENC_token`, `sop_token`, `eop_token`, `gMASK_token` etc (but I am not sure to see where they are used in the code). 

Don't hesitate to ask for more details! I'm here to help
change to suitable with half precision in tpu
Author: pphuc25
Pull request: yes

I have made changes to your statement to make it suitable for a pull request on GitHub. Please review the following:

Description:
I am utilizing TPU for training and have observed that when setting the half_precision_backend to 'auto', it automatically assigns it as 'cuda_amp'. However, this causes a bug since there is no torch.cuda available on TPUs. To resolve this issue, I have implemented a conditional check where if the device is XLA (TPU), it will switch to 'cpu_amp' instead, and then avoid bug that call torch.cuda when using TPU.

However, it appears that this change has led to a decrease in TPU speed. I would appreciate it if you could review my modification and provide suggestions for improvements.

Comments:
cc @sgugger
Fix typo in doc comment of BitsAndBytesConfig
Author: ledyba
Pull request: yes

# What does this PR do?

Fix a typo in doc comments of `BitsAndBytesConfig` class.

It must be `nf4` instead of `fn4`.

Here is a comparison code in current main branch:

https://github.com/huggingface/transformers/blob/539e2281cd97c35ef4122757f26c88f44115fa94/src/transformers/utils/quantization_config.py#L166-L167

`qlora` implementations also uses `nf4`.

https://github.com/artidoro/qlora/blob/bdc655dfa71e5ef1553a078980fb5083c346a4cf/qlora.py#L138-L141

So I think it is better to fix doc, not a code.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger, @stevhliu @MKhalusova @younesbelkada and @sourabh112

Or @TimDettmers?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Skip `test_multi_gpu_data_parallel_forward` for `MobileViTV2ModelTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

Skip `test_multi_gpu_data_parallel_forward` for `MobileViTV2ModelTest`.

This passes on CI, but the other 2x tests running after this one are all failing with `CUDA error: misaligned address`.

(Similar to #21991)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Yes, see #21991. That PR mentioned torch 2.0, but this test are skipped for some other modes even before that PR for other reasons.
The 3D attention mask in the LongFormer is wrong
Author: syiswell
Pull request: no

### Feature request

I used a 3d attention mask in the LongFormer, but also failed. I find that the code
attention_mask = nn.functional.pad(
attention_mask, (0, padding_len), value=0
) # no attention on the padding tokens
in line 1626 in modeling_longformer may not support the 3D attention mask. Please correct me if I am wrong

### Motivation

I want to input a 3d attention mask in the LongFormer to conduct the visible field for different tokens.

### Your contribution

posting a code snippet example

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Adding XLA support for greedy sampling
Author: aashiqmuhamed
Pull request: yes

# What does this PR do?

This CR enables greedy sampling in model.generate on XLA devices such as Trainium and TPU. This addresses issues such as  https://github.com/huggingface/transformers/issues/18661 and https://github.com/huggingface/transformers/issues/12322. 

The implementation is inspired by the corresponding Tensorflow generate function in transformers. The CR uses conditional statements to support greedy sampling, and the user can switch between GPU implementation or XLA implementation depending on the state of `is_torch_tpu_available`. The CR also implements kv-cache functionality that is XLA compatible.

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@gante Feel free to suggest appropriate tests/refactors for this PR. We have tested generation locally using a trn1.32xlarge instance and matched Rouge scores for T5-small summarization.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23021). All of your documentation changes will be reflected on that endpoint.

Hey @aashiqmuhamed! In general the PR looks positive, but before diving deeper, let us (`transformers` team) have a discussion about adding this type of PRs (new HW-oriented optimizations). `generate` is very complex ATM, and we want to make it more manageable -- if we merge all PRs of this kind in the current `generate` state, it will become maintenance hell for everyone. 

I will get back to you in this PR within a week.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Saving prediction for --do_predict and --predict_with_generate in  transormers/examples/pytorch/question-answering /run_seq2seq_qa.py
Author: venkatesh-das
Pull request: no

### Feature request

The feature for saving predictions for `--do_predict` and `--predict_with_generate` was not functional for `run_seq2seq_qa.py` module.
Missing code file -> `transformers/examples/pytorch/question-answering/trainer_seq2seq_qa.py`
Image of the section of code which should handle this.
![hugging_face error1](https://user-images.githubusercontent.com/47103482/234679772-efecc017-8efa-445d-a55f-04b6b6ef0732.png)


### Motivation

Some other modules like run_summarization.py have that feature.
Motivation from `transformers/examples/pytorch/summarization/run_summarization.py`
![hugging_face error2](https://user-images.githubusercontent.com/47103482/234680113-5cbb7030-99e3-428a-adbd-b171088237c0.png)


### Your contribution

Adding a code snippet would help to save the predictions for `--do_predict` and `--predict_with_generate`.
Code changes to be done here -> `transformers/examples/pytorch/question-answering/trainer_seq2seq_qa.py`
```python
   # Prediction
    if training_args.do_predict:
        logger.info("*** Predict ***")
        results = trainer.predict(predict_dataset, predict_examples)
        metrics = results.metrics

        max_predict_samples = (
            data_args.max_predict_samples if data_args.max_predict_samples is not None else len(predict_dataset)
        )
        metrics["predict_samples"] = min(max_predict_samples, len(predict_dataset))

        trainer.log_metrics("predict", metrics)
        trainer.save_metrics("predict", metrics)
        
        # Added code section for saving predictions
        if trainer.is_world_process_zero():
            if training_args.predict_with_generate:
                predictions = predict_results.predictions
                predictions = [pred['prediction_text'] for pred in predictions]
                output_prediction_file = os.path.join(training_args.output_dir, "generated_predictions.txt")
                with open(output_prediction_file, "w") as writer:
                    writer.write("\n".join(predictions))

``` 
![hugging_face error](https://user-images.githubusercontent.com/47103482/234679511-45055aba-c21b-45b3-bc66-02cbd68711d5.png)


Comments:
Do you want to open a PR with those changes?

Yeah Sure I can do that.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
aml vision benchmark
Author: prathikr
Pull request: yes

aml vision benchmark

Comments:
Thanks for your PR, but we are not interested in this modification of this example.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22574). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
expose safe_serialization argument in the pipeline API
Author: yessenzhar
Pull request: yes

# What does this PR do?
expose safe_serialization argument of PreTrainedModel and TFPreTrainedModel in the save_pretrained of the pipeline API


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@Narsil


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Pix2Struct: wrong attention mask broadcasting in the visual encoder
Author: affjljoo3581
Pull request: no

### System Info

- `transformers` version: 4.29.2
- Platform: Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.10
- Python version: 3.8.16
- Huggingface_hub version: 0.14.1
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): 0.6.10 (cpu)
- Jax version: 0.4.10
- JaxLib version: 0.4.10
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker @younesbelkada @amyeroberts

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The Pix2Struct visual attention layer masks the attention scores (logits) by adding negative infinities from the attention mask.
https://github.com/huggingface/transformers/blob/539e2281cd97c35ef4122757f26c88f44115fa94/src/transformers/models/pix2struct/modeling_pix2struct.py#L200-L220

And the below line is to broadcast the boolean attention mask of which shape is `[batch_size, seq_len]` to make a shape of `[batch_size, num_heads, query_len, key_len]`. But it seems the mask tensor is broadcasted on wrong axes.
https://github.com/huggingface/transformers/blob/539e2281cd97c35ef4122757f26c88f44115fa94/src/transformers/models/pix2struct/modeling_pix2struct.py#L213

Here is the common way to extend the mask:

https://github.com/huggingface/transformers/blob/539e2281cd97c35ef4122757f26c88f44115fa94/src/transformers/modeling_utils.py#L896

Because of that, the model will attend mask tokens as well while the original implementation uses `flaxformer` as follows:
https://github.com/google/flaxformer/blob/9adaa4467cf17703949b9f537c3566b99de1b416/flaxformer/components/attention/dense_attention.py#L2019


### Expected behavior

Fix the order of new axes on the attention mask.

Comments: