noamaanMulla-03/transformers-issues huggingface dataset:
🌐 [i18n-KO] Translated `community.md` to Korean
Author: sim-so
Pull request: yes

# What does this PR do?

Translated the `community.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

Team OSSCA, may you please review this PR? 
@bolizabeth, @nuatmochoi, @heuristicwave, @mjk0618, @keonju2, @harheem, @junejae, @54data, @Sunmin0520, @seank021, @augustinLib, @sronger, @TaeYupNoh, @kj021

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)
@sgugger, @ArthurZucker, @stevhliu May you please review this PR?

Comments:
리뷰 하실 때 참고하실 점 공유 드립니다.
  - [`Nlp`](https://huggingface.co/docs/datasets/v0.3.0/installation.html)는 `Datasets` 이전에 사용된 라이브러리 이름입니다. v1.0.0부터 `Datasets`로 바뀌었습니다. (현재 가장 최신 Datasets 버전은 v2.14입니다!) 모든 노트북에서 잘 실행되지는 않는 것 같지만 여전히 `Nlp` 이름으로 모듈을 다운로드 받을 수 있고, `Datasets`로 바꿀 때 관련 노트북도 수정이 필요할 것 같아 원문의 이름을 유지했습니다. 
  - 모델 이름인 `transformer`는 한글로, Hugging Face 라이브러리 이름인 `Transformers`는 영어로 옮겼습니다.
  - Hugging Face 라이브러리 앞에 쓰는 🤗 이모지는 원문에서 쓴 그대로 옮겼습니다. 원문에서도 일관되지 않게 붙이고 있는데요, 전부 붙일지 고민하다 일단 그대로 두었습니다.

위 내용 관련하여 의견 있으시면 편히 남겨주세요! 리뷰 잘 부탁 드립니다 😊

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25674). All of your documentation changes will be reflected on that endpoint.

@sgugger, @ArthurZucker, @stevhliu 
May you please review this PR? 😊
Documentation issue, tokenization_llama.py, legacy = True
Author: dpaleka
Pull request: no

### System Info

v4.32.1

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The error message in tokenization_llama.py asks the user to provide a flag which does nothing:

https://github.com/huggingface/transformers/blob/77713d11f6656314fb06c217cf43c4b8f5c64df8/src/transformers/models/llama/tokenization_llama.py#L140-L148 seems wrong:


(Or the documentation is faulty. Does `legacy = True` turn the legacy behavior on or off?)

### Expected behavior

Not sure, but possibly, the error message should ask the user to set `legacy=False` to get the new behavior.

Comments:
No, not everyone needs / should be using the new behaviour. `legacy = True` means `True, I want to use legacy`. 
However there is indeed a type, `legacy = False` should be set for the new behaviour !
[AutoTokenizer] Add data2vec to mapping
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Currently, loading the `pipeline` class with data2vec returns an empty tokenizer:

```python
from transformers import pipeline

pipe = pipeline("automatic-speech-recognition", model="facebook/data2vec-audio-base-960h")
print(pipe.tokenizer is None)
```

**Print Output:**
```
True
```

This is because data2vec is missing from the auto-tokenizer mapping list.

cc @Vaibhavs10 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Mask2Former after 4.32 release use more memory
Author: Emilon1928
Pull request: no

### System Info

transfomers 4.32.0
ubuntu 22.04
python 3.9

@amyeroberts 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Run the script in the colab notebook twice (with T4 GPU):
1. with version 4.31.0 (everything should work fine)
2. with version 4.32.0 (OOM in inference)

https://colab.research.google.com/drive/1xq54l9a2AQLIHT5jw63btifbOrvSqzts?usp=sharing

### Expected behavior

GPU memory consumption should be constant between versions 

Comments:
`CUDA out of memory. Tried to allocate 56.25 GiB`. It looks a lot.

@amyeroberts Could you take a look? I tried the notebook and it shows the issue.
Generate: models with custom `generate()` return `True` in `can_generate()`
Author: gante
Pull request: yes

# What does this PR do?

See discussion in #25532 

This PR updates `GenerationMixin.can_generate()` to also return `True` when a model has a custom `generate()` -- which means that the model is allowed to call `generate()`. While at it, removes a few overloads, as they are redundant with the change in the PR ✂️ 

Additional context: ALL models have a `generate()` function defined, as the mixin is added to all models. This function is a helper to detect when the model class is compatible with `generate()` or not.

______________________________________________
Double-checking with code:
```py
from transformers import Blip2ForConditionalGeneration, BertForQuestionAnswering, BarkFineModel, BarkModel, SpeechT5ForTextToSpeech

print(Blip2ForConditionalGeneration.can_generate())
print(BarkFineModel.can_generate())
print(BarkModel.can_generate())
print(SpeechT5ForTextToSpeech.can_generate())
print(BertForQuestionAnswering.can_generate())  # sanity check: this one can't generate
```
After this PR, this prints `True` in all cases except the last one, as expected. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Usage of FusedRMSNorm with T5 model
Author: sh0tcall3r
Pull request: no

Hello!
In HF T5 model description I found that if I had installed Nvidia apex package, T5 model would automatically use FusedRMSNorm instead of T5LayerNorm and hence inference would increase.
I just installed apex and using mt0-xxl model in 8bit loading mode, which I've already fine-tuned. But I see no difference neither in layers description (there are still T5LayerNorm layers in model named children list), nor in inference speed. 
I know that in case of really big models speed increase not dramatically, but am I doing everything right?

@stas00 As far as I know you have some experience with that, could you please try to help me?

I have cuda 11.3
pytorch 1.12.1
apex 0.1


Comments:
Hi @sh0tcall3r 

I assume you use https://huggingface.co/bigscience/mt0-xxl/blob/main/config.json

It is `MT5ForConditionalGeneration` whose definition is in `src/transformers/models/mt5/modeling_mt5.py`. In that file, `FusedRMSNorm` is not used.

However, I think what you see should be `MT5LayerNorm` rather than `T5LayerNorm`. Maybe it is a typo in your description ?

> However, I think what you see should be `MT5LayerNorm` rather than `T5LayerNorm`. Maybe it is a typo in your description ?

Yes, indeed. I see `MT5LayerNorm`. And yes, you're absolutely right, it's `MT5ForConditionalGeneration`.
Is there any option to use `apex` for speed up of inference in this case?
Or is there any option of inference speed up without `apex`, but with some other methods/libraries/packages?  

You can try to modify the modeling file of mt5 (see what has been done in t5 file).
Or you can use `load_in_8bit` or `load_in_4bit`, see `docs/source/en/main_classes/quantization.md`.

Or see if [this](https://twitter.com/younesbelkada/status/1696478075721302143) works for mt5 :-)

> You can try to modify the modeling file of mt5 (see what has been done in t5 file). Or you can use `load_in_8bit` or `load_in_4bit`, see `docs/source/en/main_classes/quantization.md`.
> 
> Or see if [this](https://twitter.com/younesbelkada/status/1696478075721302143) works for mt5 :-)

Got it, will do, thanks!

Let me close this issue - I think what could be answered are all given :-)

Sure, I agree
Add ViTMatte
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds the ViTMatte model, an elegant approach to image matting, entirely relying on the Vision Transformer backbone doing the heavy work, with a lightweight head on top.

Here's a Colab notebook showcasing inference: https://colab.research.google.com/drive/1pWTn3Iur-NR2xUIyDE31dBgng_hXjSsn?usp=sharing. 

The model leverages [VitDet](https://arxiv.org/abs/2203.16527) as backbone, hence this PR adds VitDet as a standalone model as well. It then leverages the AutoBackbone class to use this model as a backbone for image matting.

Fixes #25040.

Comments:
@sgugger apart from the toctree issue which I'm still investigating, all comments are addressed.



Before I start reviewing - could you separate out the addition of VitDet and VitMatte? They should have their own respective PRs.
[BUG] `current_segment_id` should increment from largest segment id.
Author: jimmysue
Pull request: no

Code below increment segment id from last iteration segment id, which is wrong. Correct segment id should increment from the latest(which is the largest so far) segment id.
https://github.com/huggingface/transformers/blob/a7920065f2cfd2549b838f9a30afd7c265fcdd88/src/transformers/models/mask2former/image_processing_mask2former.py#L234-L237

Comments:
Hi @jimmysue, thanks for raising this issue! 

For the segment ids, they increment from 0 -> total number of segments for each image. This is expected for segmentation like panoptic, as we want each individual instance to have its own segment ID assigned e.g. segment 0 -> car 0, segment 1 -> car 1, segment 3 -> sky etc. 


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Update README.md
Author: NinoRisteski
Pull request: yes

deleted unnecessary comma in the Adding a new model section.



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25832). All of your documentation changes will be reflected on that endpoint.

This PR (and the commit merged to `main`) triggered the whole CI, which is strange as this only changed a `.md` file. I am going to check, but @amyeroberts let me know if you already have a clear idea in mind why is this 🙏 .
[`Sentencepiece`] make sure `legacy` do not require `protobuf`
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes the `get_spm_processor()` function to make sure that if `protobuf` is not installed, we can still initialize the model when `legacy=False`.  
Just realized that `make fixup` fails without protobuf :sweat: 
fixes #25753
```python 
>>> import tensorflow as tf
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[2], line 1
----> 1 import tensorflow as tf

File /opt/conda/envs/py39/lib/python3.9/site-packages/tensorflow/__init__.py:37
     34 import sys as _sys
     35 import typing as _typing
---> 37 from tensorflow.python.tools import module_util as _module_util
     38 from tensorflow.python.util.lazy_loader import LazyLoader as _LazyLoader
     40 # Make sure code inside the TensorFlow codebase can use tf2.enabled() at import.

File /opt/conda/envs/py39/lib/python3.9/site-packages/tensorflow/python/__init__.py:37
     29 # We aim to keep this file minimal and ideally remove completely.
     30 # If you are adding a new file with @tf_export decorators,
     31 # import it in modules_with_exports.py instead.
     32 
     33 # go/tf-wildcard-import
     34 # pylint: disable=wildcard-import,g-bad-import-order,g-import-not-at-top
     36 from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow
---> 37 from tensorflow.python.eager import context
     39 # pylint: enable=wildcard-import
     40 
     41 # Bring in subpackages.
     42 from tensorflow.python import data

File /opt/conda/envs/py39/lib/python3.9/site-packages/tensorflow/python/eager/context.py:28
     25 from absl import logging
     26 import numpy as np
---> 28 from tensorflow.core.framework import function_pb2
     29 from tensorflow.core.protobuf import config_pb2
     30 from tensorflow.core.protobuf import coordination_config_pb2

File /opt/conda/envs/py39/lib/python3.9/site-packages/tensorflow/core/framework/function_pb2.py:7
      5 import sys
      6 _b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
----> 7 from google.protobuf import descriptor as _descriptor
      8 from google.protobuf import message as _message
      9 from google.protobuf import reflection as _reflection

ModuleNotFoundError: No module named 'google.protobuf'
```
fixes #25667

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Unable to follow Object Detection Task Example due to ImageProcessor error
Author: govindrai
Pull request: no

### System Info

transformers 4.3.2
python 3.10
using google colab



### Who can help?

@amyeroberts, @sgugger, @stevhliu

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction
Follow [official tutorial](https://huggingface.co/docs/transformers/v4.32.0/en/tasks/object_detection) instructions:
After applying a transform function to the CPPE5 dataset and trying to load a single element fails with the following error:

```
TypeError                                 Traceback (most recent call last)
[<ipython-input-75-e2a7222a96ab>](https://localhost:8080/#) in <cell line: 22>()
     20 
     21 cppe5["train"] = cppe5["train"].with_transform(transform_aug_ann)
---> 22 cppe5["train"][15]

9 frames
[/usr/local/lib/python3.10/dist-packages/transformers/models/detr/image_processing_detr.py](https://localhost:8080/#) in <listcomp>(.0)
   1285             if annotations is not None:
   1286                 annotations = [
-> 1287                     self.normalize_annotation(
   1288                         annotation, get_image_size(image, input_data_format), input_data_format=input_data_format
   1289                     )

TypeError: DetrImageProcessor.normalize_annotation() got an unexpected keyword argument 'input_data_format'
```

Here's a colab notebook so you test/run it easily. Go to the last step to view the error:
https://colab.research.google.com/drive/1WtwFcW7_N5N7VLsoswx2U4O8CBkCRNc4?usp=sharing

### Expected behavior

I expect to not encounter a failure and continue the tutorial. 

Comments:
cc @amyeroberts seems to come from #25464

Hi @govindrai, thanks for raising this issue! 

Yes, this was a bug introduced in #25464. A fix was merged in yesterday in #25643 on main. Installing from source should resolve the issue. 

Thank you, all! That fixes it :). 

For others, run `!pip install git+https://github.com/huggingface/transformers` to install from source. 
 

@govindrai The fix has now been included as part of a [patch release](https://github.com/huggingface/transformers/releases/tag/v4.32.1) and can be directly installed from pypi with `pip install transformers`
Train_DataLoader's sampler changed from RandomSampler -> SequentialSampler
Author: dumpmemory
Pull request: no

### System Info

- `transformers` version: 4.32.1
- Platform: Linux-5.4.119-19.0009.28-x86_64-with-glibc2.35
- Python version: 3.10.6
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.3
- Accelerate version: 0.22.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.1.0a0+b5021ba (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I found after https://github.com/huggingface/transformers/blob/ccb92be23def445f2afdea94c31286f84b89eb5b/src/transformers/trainer.py#L1569 

the output of 

```
 logger.info(
            f"{type(train_dataloader)}, {type(train_dataloader.sampler)},{type(train_dataloader.batch_sampler)}")
```

is 

```
<class 'accelerate.data_loader.DataLoaderShard'>, <class 'torch.utils.data.sampler.SequentialSampler'>,<class 'accelerate.data_loader.BatchSamplerShard'>
```

The train_dataloader dataargs is 

```
{'batch_size': 4, 'collate_fn': <function default_data_collator at 0x7f404cf33520>, 'num_workers': 0, 'pin_memory': True, 'sampler': <torch.utils.data.sampler.RandomSampler object at 0x7f404cbd26e0>, 'drop_last': False, 'worker_init_fn': <function seed_worker at 0x7f4061da8820>}
```


why sample changed from RandomSampler -> SequentialSampler

### Expected behavior

The sampler should be same 

Comments:
cc @muellerzr @pacman100 

it might be relate to https://github.com/huggingface/accelerate/blob/69e4c3c54da3201eda288b500d138761e7a5221c/src/accelerate/data_loader.py#L709 

I am checking train_dataloader.batch_sampler.batch_sampler

train_dataloader.batch_sampler.batch_sampler.sampler is torch.utils.data.sampler.RandomSampler
Add flax installation in daily doctest workflow
Author: ydshieh
Pull request: yes

# What does this PR do?

#25763 enables the doctesting against a few more files, including some flax related files. As the docker image doesn't have flax/jax installed, the pytest failed to collect the test to run, see [this doctest run](https://github.com/huggingface/transformers/actions/runs/6019814019/job/16330174475):
```bash
src/transformers/generation/flax_utils.py - ModuleNotFoundError: No module named 'flax'
```

Let's install jax/flax in the doctest workflow, and see what this gives us.

Note we don't want to install jax/flax in the docker image, as the same image is used for daily (non-doctest) CI, and we don't want to have them to (potentially) interfere the CI.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

As Flax and Jax break our environments frequently because of (large) breaking changes in their releases and it's a pain to managed, could we pin the install to the highest version currently allowed in setup.py?

Hi @amyeroberts , with 

```bash
python3 -m pip install -e .[flax]
```
it should already respect the versions we specify in `setup.py`. Correct me otherwise 🙏 


@amyeroberts jax, jaxlib, and flax are pinned in the setup, so it should be okay :D (e.g. [here](https://github.com/huggingface/transformers/blob/52574026b6740a3882d6dd1cbf1e1663d4cea27b/setup.py#L116))

@gante @ydshieh 🤦‍♀️ d'oh, my bad, sorry I just skimmed the pip command
Fix imports
Author: ydshieh
Pull request: yes

# What does this PR do?

So far test fetcher fails to fetching and all tests are collected. This PR fix a few import issues introduced in #25599, so we won't have all tests being collected and run.

Comments:
@younesbelkada  No, it's not about doctest - it's the usual `code` tests

_The documentation is not available anymore as the PR was closed or merged._
fix max_memory for bnb
Author: SunMarc
Pull request: yes

# What does this PR do ? 
Fixes #24965 . For bnb models, we leave some space (10%) for the buffers that are created during the quantization. This fixes cases where we uses `device_map = "sequential"` with bnb. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks a lot @SunMarc!
`speed_metrics` may cause ambiguity under multi-nodes scenario
Author: CokeDong
Pull request: no

### System Info

- `transformers` version: 4.32.0
- Platform: Linux-3.10.0-957.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.8.16
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.3
- Accelerate version: 0.22.0
- Accelerate config:    not found
- PyTorch version (GPU?): 1.13.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@patrickvonplaten 


### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

e.g. any training programs with `torchrun` running with multi-nodes.

```bash
#master nodes
WORLD_SIZE=4  CUDA_VISIBLE_DEVICES=0,1 torchrun --nnodes=2 --node_rank=0 --nproc_per_node=2 --master_port=1234 train.py ...

# slave nodes
WORLD_SIZE=4  CUDA_VISIBLE_DEVICES=2,3 torchrun --nnodes=2 --node_rank=1 --nproc_per_node=2 --master_port=1234 train.py ...
```

### Expected behavior

the speed_metrics logs will got printed twice, which cause ambiguity.
e.g this is one of my testing results

```
#master nodes

{'train_runtime': 318.7822, 'train_samples_per_second': 16.061, 'train_steps_per_second': 0.031, 'train_loss': 1.8525979161262511, 'epoch': 0.1}
# slave nodes
   {'train_runtime': 318.7808, 'train_samples_per_second': 16.061, 'train_steps_per_second': 0.031,  'train_loss': 1.8525979161262511, 'epoch': 0.1}
```

maybe the log is better to be printed **only** by master modes.

Comments:
Hi @CokeDong!

Since the output is identical, I would say it's not a big issue :-)

cc @muellerzr @pacman100 

yes, except the `train_runtime` is not the same. 

@CokeDong train runtime will be slightly different as it comes from each node, and each one can take slightly longer depending on the connection speed, etc. I don't believe it's actually truly an issue, nor something we can really do anything about. It's a minute difference. 

got that, thx~