noamaanMulla-03/transformers-issues huggingface dataset:
fix Whisper tests on GPU
Author: hollance
Pull request: yes

# What does this PR do?

The daily CI showed that Whisper has new test failures, related to the recent merge of the prompting feature. This PR fixes those test failures.

The tests ran OK on CPU but failed on GPU because the input data wasn't moved to the GPU.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Also skipping a few tests in `WhisperModelTest` that were previously skipped in `WhisperEncoderModelTest`, see https://github.com/huggingface/transformers/pull/22060

Although I just saw there's another open PR dealing with the same issue, so maybe none of these should be skipped: https://github.com/huggingface/transformers/pull/22803

Thanks again!
[i18n-<languageCode>] Translating docs to <languageName>
Author: rezz90
Pull request: no
Labels: name: WIP

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @ArthurZucker, @sgugger for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [ ] [index.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.mdx) https://github.com/huggingface/transformers/pull/20180
- [ ] [quicktour.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.mdx) (waiting for initial PR to go through)
- [ ] [installation.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.mdx).

## Tutorial section
- [ ] [pipeline_tutorial.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.mdx)
- [ ]  [autoclass_tutorial.mdx](https://github.com/huggingface/transformers/blob/master/docs/source/autoclass_tutorial.mdx)
- [ ]  [preprocessing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.mdx)
- [ ]  [training.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.mdx)
- [ ]  [accelerate.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.mdx)
- [ ]  [model_sharing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.mdx)
- [ ]  [multilingual.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.mdx)

<!--
Keep on adding more as you go 🔥
-->

Comments:
🌐 [i18n-KO] Translated `fast_tokenizers.mdx` to Korean
Author: kihoon71
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.mdx` to Korean" 으로 부탁드립니당 -->
# What does this PR do?

Translated the `fast_tokenizers.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

<!-- 제출 전 체크리스트로, 가짜연구소만의 체크리스트도 <details>로 감싸서 만들어두면 더 좋을 것 같아요. -->

## Who can review?
<!-- 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- @sgugger, @ArthurZucker, @eunseojo May you please review this PR? -->

Team PseudoLab, may you please review this PR?

@0525hhgus, @wonhyeongseo , @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd




Comments:
_The documentation is not available anymore as the PR was closed or merged._

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@sgugger, @ArthurZucker, @eunseojo May you please review this PR?
[i18n-KO] Translated video_classification.mdx to Korean
Author: kihoon71
Pull request: yes

# What does this PR do?

Translated the video_classification.mdx file of the documentation to Korean.
Thank you in advance for your review.


## Before reviewing
 - [x] Check for missing / redundant translations (번역 누락/중복 검사)
 - [x] Grammar Check (맞춤법 검사)
 - [x] Review or Add new terms to glossary (용어 확인 및 추가)
 - [x] Check Inline TOC (e.g. [[lowercased-header]])
 - [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review(initial)?

Team PseudoLab, may you please review this PR? @0525hhgus, @HanNayeoniee, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review(initial)?

@sgugger, @ArthurZucker, @eunseojo May you please review this PR?



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey! Sorry for the long delay. There seems to be 2 suggestions not adresses, should we wait for these? 🤗 

@ArthurZucker the suggestions that i didn't accept were about same sentences or ealier version of our glossary so you don't need to wait for the other suggestions to be accepted!! Thank you!!
🌐 [i18n-KO] Translated `troubleshooting.mdx` to Korean
Author: 0525hhgus
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.mdx` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `troubleshooting.mdx` file of the documentation to Korean.
Thank you in advance for your review 😄 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
Team PseudoLab, may you please review this PR? 
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
May you please review this PR? 
@sgugger, @ArthurZucker, @eunseojo  

Comments:
_The documentation is not available anymore as the PR was closed or merged._

May you please review this PR? 😃 
@sgugger, @ArthurZucker, @eunseojo

@0525hhgus you need to put it out of draft mode if it's ready for review.

> @0525hhgus you need to put it out of draft mode if it's ready for review.

I changed to ready for review status! Thank you for your review.
Space CPU Basic and  Nvidia T4 - small should be FREE FREE FREE
Author: pure-rgb
Pull request: no

### Feature request

Look at this horrible chart, from [this page](https://huggingface.co/pricing#spaces).

![image](https://github.com/huggingface/transformers/assets/45315076/7b781ad5-f5f6-4469-a845-c408fb126bd7)

The CPU basic and T4 small should be completely free. 

### Motivation

- I have a code example (gradio web app) and I want to upload it in huggingface space. The code is minimal but contains some API level issue to run on CPU and I need minimum GPU to run it. Now I couldn't run it on space due to no GPU for free.
- But I easily run this app using google Colab. Which provides free T4.

In summary, I git clone the huggingface space repo to google colab and run my code there. 

- How is it good for huggingface businessnes?
- How is it good for end-user demand?

Please reconsider.

### Your contribution

Don't ask me for sponsor ;)

Comments:
Wrong RoBERTa configuration
Author: amariucaitheodor
Pull request: no

https://github.com/huggingface/transformers/blob/c2393cad085e3875ee2206d917d46d15e50602a3/src/transformers/models/roberta/configuration_roberta.py#L108

Must match the tokenizer vocab. size, i.e., `50265`.

Others also mentioned this: https://discuss.pytorch.org/t/hugging-faces-roberta-config-and-tokenizer-do-not-have-matching-vocabulary/134868

Comments:
cc @ArthurZucker 

Thanks for pointing this out and opening a PR! 

Hey @ArthurZucker @amyeroberts , as i still see that  #23389  is not merged , so can I fix this and create a new PR? 

Sure! Would be great if you can checkout his branch to include the work he has done 😉 

fix #23863
[from_pretrained] imporve the error message when `_no_split_modules` is not defined
Author: ArthurZucker
Pull request: yes

# What does this PR do?
As a lot of issues seem to have appeared related to this, the warning is improved. 
Adresses #23816

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks, though black would format it! 

Tests are again unrealted to the PR, will merge once the doc is built
[wip: testing doc-builder]
Author: mishig25
Pull request: yes

testing https://github.com/huggingface/doc-builder/pull/373

Comments:
@mishig25 Should be fixed by [this](https://github.com/huggingface/doc-builder/pull/373/commits/4895cfbf332994c5c5d73e83f990fe017184ecc5)

Rerunning the doc-build https://github.com/huggingface/transformers/actions/runs/5036878423 

Re running the doc-build: https://github.com/huggingface/transformers/actions/runs/5036878423

Latest error message is:
```
<img> is a void element and cannot have children, or a closing tag
```
which appears to be an error by the person who made the docs? In the original version, did you attempt to fix those for the user?

Confirmed (https://huggingface.co/docs/transformers/v4.29.1/pt/index) - see trailing "&lt;/img>" after image:
![image](https://github.com/huggingface/transformers/assets/26504141/28e90431-72d9-4bf1-b10a-935fa7d5005e)


I made a PR to fix the closing tag https://github.com/huggingface/transformers/pull/23646

Thanks a lot for [this PR](https://github.com/huggingface/transformers/pull/23625#issuecomment-1556623634).

Despite the img tag was wrong before, previous doc-builds were not failing. Does it mean that a change in https://github.com/huggingface/doc-builder/pull/373 is making closing tag of img fail ?

Well yes since `<img></img>` is invalid, svelte complains.

Previously, the error was not being detected because </img> was being encoded as `&lt;/img>`

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23625). All of your documentation changes will be reflected on that endpoint.
Add InternImage
Author: Weiyun1025
Pull request: no
Labels: name: New model

### Model description

InternImage is a new large-scale CNN-based foundation model, which can obtain the gain from increasing parameters and training data like ViTs. Different from the recent CNNs that focus on large dense kernels, InternImage takes deformable convolution as the core operator, so that this model not only has the large effective receptive field required for downstream tasks such as detection and segmentation, but also has the adaptive spatial aggregation conditioned by input and task information. InternImage-H achieved a new record 65.4 mAP on COCO test-dev and 62.9 mIoU on ADE20K, outperforming current leading CNNs and ViTs.

It is worth noting that InternImage relies on a custom cuda operator, so if this causes problems for model addition, you can replace [the cuda operator](https://github.com/OpenGVLab/InternImage/blob/master/classification/ops_dcnv3/modules/dcnv3.py#L218) with [a pytorch implementation](https://github.com/OpenGVLab/InternImage/blob/master/classification/ops_dcnv3/modules/dcnv3.py#L91).

In fact, we have already submitted [a version of the code on transformers](https://huggingface.co/OpenGVLab/internimage_t_1k_224/tree/main), however, due to security reasons, the code we submitted cannot call your web inference api, so we would like you to add InternImage to transformers.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

https://github.com/OpenGVLab/InternImage

Comments:
Can I take it up?

> Can I take it up?

Of course, thank you!

@souravpy Are you currently working on this? If not, I would love to take a look to see if I could help in adding this model to HF Transformers!

The [modeling code and weights](https://huggingface.co/OpenGVLab/internimage_s_1k_224/blob/main/intern_image.py) for Intern Image are already on the hub, and so the model can already be used directly with the `AutoModel` API. 

cf. https://github.com/huggingface/transformers/pull/23782#issuecomment-1568459737
Possible pickle issues
Author: Natyren
Pull request: no

https://github.com/huggingface/transformers/blob/af2aac51fc1c59237ff7228908ace2cd8fc0d9a6/src/transformers/optimization.py#L49
When attempting to pickle this function, there is a potential for an error to occur due to the lambda function that cannot be pickled.
I suggest the following solution for this problem.


```
def get_constant_lambda(_):
    return 1

def get_constant_schedule(optimizer: Optimizer, last_epoch: int = -1):
    """
    Create a schedule with a constant learning rate, using the learning rate set in optimizer.

    Args:
        optimizer ([`~torch.optim.Optimizer`]):
            The optimizer for which to schedule the learning rate.
        last_epoch (`int`, *optional*, defaults to -1):
            The index of the last epoch when resuming training.

    Return:
        `torch.optim.lr_scheduler.LambdaLR` with the appropriate schedule.
    """

    return LambdaLR(optimizer, get_constant_lambda, last_epoch=last_epoch)
```





Comments:
Would like to open a PR with your fix?

yes, is it okay?
Editing issue with pickle def with lambda function
Author: Natyren
Pull request: yes

# What does this PR do?
In this PR, I address the problem of pickling the constant LR scheduler, which fails during the process (potentially during multi-GPU training, as observed in my case) due to the presence of a lambda function within it.

Fixes  #23865    (issue)


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Pytorch BigBird random attention
Author: Bearnardd
Pull request: no

### Reproduction

`Pytorch->Flax` and `Flax->Pytorch` equivalence tests were failing. At the moment they are skipped by https://github.com/huggingface/transformers/pull/23040

### Expected behavior

During working on https://github.com/huggingface/transformers/pull/21023 I have found out that there is a bug in pytorch's implementation of `BigBird`. Namely random attention is used no matter whether we are in training/eval mode. Corect behaviour is that during inference (eval) we should not introduce any randomness, hence we random attention should not be used.

Comments:
Hi @sanchit-gandhi @ydshieh! I have opened [PR](https://github.com/huggingface/transformers/pull/23056) that fixes failing tests. I am wondering if the changes in the PR are okay (usage of random attention based on current mode) or do we want to have some more control over usage of random attention e.g. add `deterministic` argument for `__call__` of `BigBirdPreTrainedModel`. Secondly I was wondering what is the advantage of marking `_bigbird_block_rand_mask` as a `staticmethod` and then calling it with `self._bigbird_block_rand_mask` and passing it arguments from `self` like `self.max_seqlen` instead of treating it as a regular method. It looks kinda weird to me. Am I missing something?

Closed via https://github.com/huggingface/transformers/pull/23056.
Why don't we set use_cache=False in default when training?
Author: Coldog2333
Pull request: no

### Feature request

Let's take GPT-2 as an example, in the current implementation (modeling_gpt2.py: Line 856~861):
```
if self.gradient_checkpointing and self.training:
    if use_cache:
        logger.warning_once(
            "`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`..."
        )
        use_cache = False
```
Why don't we set like this:
```
if self.training:    # As long as it is being trained, we set use_cache=True. 
    if use_cache:
        logger.warning_once(
            "`use_cache=True` makes no sense when training. Setting `use_cache=False`..."
        )
        use_cache = False
```
Because when training, `use_cache=True` makes no sense (at least for decoder-only auto-regressive model) and if you use gradient_checkpointing, it should be under training instead of inference.

### Motivation

Hello contributors, I realize that we set `use_cache=False` in default for almost all the transformer-based models. I understand that it can speed up the generation when using the cache from the previous step. However, when training (pertaining or fine-tuning), we don't need it and it consumes many memories when processing a very long sequence, especially when the model is very large. But it cannot provide us any advantage.

### Your contribution

If needed, I can help to correct this for all models in the transformers library.

Comments:
We can't change the models as it would be a breaking change, sadly.
move fsdp handling to accelerate
Author: pacman100
Pull request: yes

### What does this PR do?
1. Moves PyTorch FSDP handling to Accelerate
2. Should be merged after #23151 
3. No user-facing change. Now, users can use `accelerate launch` for fsdp in Trainer, e.g.:
```
accelerate launch --num_processes=2 --use_fsdp --mixed_precision=bf16 --fsdp_auto_wrap_policy=TRANSFORMER_BASED_WRAP  --fsdp_transformer_layer_cls_to_wrap="BertLayer" --fsdp_sharding_strategy=1 --fsdp_state_dict_type=FULL_STATE_DICT ./examples/pytorch/text-classification/run_glue.py   --model_name_or_path bert-base-cased   --task_name $TASK_NAME   --do_train   --do_eval   --max_seq_length 128   --per_device_train_batch_size 16   --learning_rate 5e-5   --num_train_epochs 3   --output_dir /tmp/$TASK_NAME/ --overwrite_output_dir
```

Continue to use torchrun with trainer args as usual.
```
torchrun --nnodes 1 --nproc-per-node 2 ./examples/pytorch/text-classification/run_glue.py   --model_name_or_path bert-base-cased   --task_name $TASK_NAME   --do_train   --do_eval   --max_seq_length 128   --per_device_train_batch_size 16   --learning_rate 5e-5   --num_train_epochs 3   --output_dir /tmp/$TASK_NAME/ --overwrite_output_dir --fsdp "full_shard auto_wrap" --fsdp_transformer_layer_cls_to_wrap BertLayer --bf16
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hello Sylvain, we can't do that as FSDP XLA integration uses it and that isn't supported yet in accelerate
shift torch dynamo handling to accelerate
Author: pacman100
Pull request: yes

### What does this PR do?

1. Shits the torch dynamo handling to accelerate
2. Should be merged after #23158 
3. No user facing change. Now, users can use `accelerate launch` for torch dynamo, e.g.,
```
accelerate launch --dynamo_backend=inductor ./examples/pytorch/text-classification/run_glue.py   --model_name_or_path bert-base-cased   --task_name $TASK_NAME   --do_train   --do_eval   --max_seq_length 128   --per_device_train_batch_size 32   --learning_rate 2e-5   --num_train_epochs 3   --output_dir ~/temp/$TASK_NAME/ --fp16 --overwrite_output_dir --pad_to_max_length --dataloader_drop_last
```

Current usage like below is unimpacted:
```
python ./examples/pytorch/text-classification/run_glue.py   --model_name_or_path bert-base-cased   --task_name $TASK_NAME   --do_train   --do_eval   --max_seq_length 128   --per_device_train_batch_size 32   --learning_rate 2e-5   --num_train_epochs 3   --output_dir ~/temp/$TASK_NAME/ --fp16 --overwrite_output_dir --torch_compile --pad_to_max_length --dataloader_drop_last
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._