noamaanMulla-03/transformers-issues huggingface dataset:
Use main in conversion script
Author: ydshieh
Pull request: yes

# What does this PR do?

Put the code under `if __name__ == "__main__":` so it won't affect other stuff (for example, pytest collection).

Doctest is currently failing due to this, see [here](https://github.com/huggingface/transformers/actions/runs/6079493829/job/16492061986)

Comments:
Update RAG README.md with correct path to examples/seq2seq
Author: tleyden
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

Update RAG README.md with correct path to examples/seq2seq


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).



## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Documentation: @stevhliu and @MKhalusova


<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

Documentation: @stevhliu and @MKhalusova


 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25953). All of your documentation changes will be reflected on that endpoint.
Trainer: delegate default generation values to `generation_config`
Author: gante
Pull request: yes

# What does this PR do?

Thank you @nick-maykr for raising the related issue.

In a nutshell, at some point in the seq2seq trainer, if `max_length` and `num_beams` were not set through the legacy arguments, we were fetching them from `model.config`. This is wrong -- all the logic about defaults is now handled in `model.generation_config`, and doesn't need any explicit value setting.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Show failed tests on CircleCI layout in a better way
Author: ydshieh
Pull request: yes

# What does this PR do?

I am not sure if this will helps the navigation (much), but at least for the team members to try.

A new step is added to show the (only faiiled) test results, and it looks like
<img width="650" alt="Screenshot 2023-08-31 184247" src="https://github.com/huggingface/transformers/assets/2521628/5f77833d-d859-43ce-8636-7db428641fe4">


The `Run tests` step won't fail now (otherwise we can't have the new `Check test results` step), but the new step will (if there is indeed a failed test).

For a job run page, see [here](https://app.circleci.com/pipelines/github/huggingface/transformers/71854/workflows/8151a27f-4617-4781-9d0d-80bcb4847edf/jobs/905042)

WDYT?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

No problem to close if you maintainers don't feel it makes navigation (even slightly)  better/faster.

I'm actually in favor if we also remove tee 😉 

@amyeroberts  BTW, the `tests_output.txt` will be removed to make @ArthurZucker  happier. But want to have your confirmation too.

No more tea Sir, sorry.

I need a run (page) that already has such colors and format.

Personally, I don't feel it's necessary to have it - as we (mostly if not always) care the failing ones. The new step already give ONLY the failed tests without the passing and skipped one - so the color doesn't matter (at least for me).

I would move on to merge this PR as you and amy is OK with the changes. For the color thing, if you can provide a run page, I could probably give it a try in the future.

Sure 😉 feel free to merge! 

We do have color (on the test run step, not the new test result showing step), but it only show the failing ones. See [here](https://app.circleci.com/pipelines/github/huggingface/transformers/72192/workflows/c446000b-bf65-4531-a4ea-13934ef594ec/jobs/910150).

On current main, no RED color at all.

<img width="478" alt="Screenshot 2023-09-05 152009" src="https://github.com/huggingface/transformers/assets/2521628/5823276e-18ff-482c-8fc7-e9f6cf133a17">




Happy for `tests_output.txt` to go - all the info can be found in the other artefacts 

Perfect!
Patch with accelerate xpu
Author: abhilash1910
Pull request: yes

Patch for Accelerate XPU support.
cc @sgugger @muellerzr 

Comments:
@sgugger  seems like ruff is flagging errors on other files (which are not edited in this commit ). Could you take a look ? Thanks

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25714). All of your documentation changes will be reflected on that endpoint.

@abhilash1910 I am no longer working at Hugging Face, so you should ping @muellerzr to help :-)

@amyeroberts thanks for the suggestions. However when I try to run  ```make style``` then many unrelated files are getting edited and ruff test triggers failure on some unmodified files.  One such example was the markupml model test script (make style changed the formatting).Could you suggest how to go about this ?  

@abhilash1910 Are you running the ruff and black versions expected in the library? They can be installed using `pip install -Ue .[quality]`

> @abhilash1910 Are you running the ruff and black versions expected in the library? They can be installed using `pip install -Ue .[quality]`

Yes I used the same versions , make style is causing some other files to change (the ones added below). Should I add that in the commit? @amyeroberts  @muellerzr  could you suggest.
```     modified:   src/transformers/__init__.py
        modified:   src/transformers/models/code_llama/tokenization_code_llama.py
        modified:   src/transformers/models/code_llama/tokenization_code_llama_fast.py
        modified:   src/transformers/models/idefics/modeling_idefics.py
        modified:   src/transformers/models/llama/tokenization_llama.py
        modified:   src/transformers/models/llama/tokenization_llama_fast.py
```


I think the initial tests pass now; I have not added the style fixes of the other mentioned files ( I used ```pip install -Ue .[quality]``` for black/ruff/isort etc ) . No files are open in my workspace IDE. Seems when I run make style on markuplm , it is formatting seq list (without it tests fail on make style). 
@amyeroberts  @muellerzr  could you trigger the slow tests(if any) and re-review?   Thanks. 

@amyeroberts @muellerzr  could you please re-review (retrigger tests)? Since this integration is a little urgent for us ; thanks again for all suggestions.
Add TFDebertaV2ForMultipleChoice
Author: raghavanone
Pull request: yes

# What does this PR do?

Fixes #25537 

Add TFDebertaV2ForMultipleChoice model

@ArthurZucker 

Comments:
cc @Rocketknight1 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25932). All of your documentation changes will be reflected on that endpoint.

@Rocketknight1 Not sure why this test fails.


That test is an issue with the CI rather than this PR, you can ignore it! Are you ready for me to merge now?

> That test is an issue with the CI rather than this PR, you can ignore it! Are you ready for me to merge now?

Yes 

Ugh, it won't let me merge, which means we'll need to rebase to get the tests working. Can you:

1) [Pull the latest changes to your fork's main branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork)
2) In your local working repo, `git checkout main` and `git pull` to ensure your local main branch is up to date
3) In your local repo, `git checkout fix_issue_25537` and then `git pull` and `git rebase main` to rebase onto the latest `main` branch
4) Finally, `git push --force` to upload the rebased branch to Github

After that, tests should pass!

> Ugh, it won't let me merge, which means we'll need to rebase to get the tests working. Can you:
> 
> 1. [Pull the latest changes to your fork's main branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork)
> 2. In your local working repo, `git checkout main` and `git pull` to ensure your local main branch is up to date
> 3. In your local repo, `git checkout fix_issue_25537` and then `git pull` and `git rebase main` to rebase onto the latest `main` branch
> 4. Finally, `git push --force` to upload the rebased branch to Github
> 
> After that, tests should pass!

Done, Lets hope the test passes.

Looks like everything's passing now - sorry about the CI issues, and thanks for a very clean and useful PR!
seq2seq speech recognition example: use forward attention mask if apply spec augment is True
Author: sorgfresser
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

The example for speech recognition for `seq2seq` did not forward the attention mask for using SpecAugment on using whisper. I strongly believe this is a bug in the example and it should be resolved now.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@patrickvonplaten 
@bofenghuang 


<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.


 -->


Comments:
cc @sanchit-gandhi
[bug] `facebook/wav2vec2-conformer-rope-large-960h-ft` refuses to work in `fp16`
Author: Vaibhavs10
Pull request: no

### System Info

- `transformers` version: 4.32.1
- Platform: Linux-5.15.109+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.3
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): 2.12.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.7.2 (gpu)
- Jax version: 0.4.14
- JaxLib version: 0.4.14
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@sanchit-gandhi @Patrick

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`facebook/wav2vec2-conformer-rope-large-960h-ft` throws a rather cryptic error (`RuntimeError: mat1 and mat2 must have the same dtype`) when loaded in half-precision.
repro: https://github.com/Vaibhavs10/scratchpad/blob/main/conformer_wav2vec2_repro.ipynb
model: https://huggingface.co/facebook/wav2vec2-conformer-rope-large-960h-ft
Note: It works fine on `fp32`

### Expected behavior

It should work without any issues!

Comments:
[Wav2Vec2 Conformer] Fix inference float16
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Fixes #25964 - the Wav2Vec2 conformer model with rotary embeddings now works when we load it `from_pretrained` with float16. The issue was originating in the rotary embedding layer, which was returning the positional embeddings in float32 always

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix Mega chunking error when using decoder-only model
Author: tanaymeh
Pull request: yes

# What does this PR do?

This PR aims to fix the error caused by `MegaModel` when the `is_decoder` setting is used in conjunction with `use_chunking` and `chunk_size` settings.

The error is described in detail [here](https://github.com/huggingface/transformers/issues/23331#issuecomment-1693729295).

Fixes #23331

## Who can review?
@ArthurZucker 

Comments:
Hey! Feel free to ping me when the PR is ready for review 😉 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25765). All of your documentation changes will be reflected on that endpoint.

@ArthurZucker You may review it now!

Will do. Two questions regarding this:
1. For the test of chunking, I should only check the expected sizes of output or something else?
2. Conceptually, do you see any troubles with this solution in its current state? I am a little suspicious because of how simple the solution was 😅

1. Yes you could check expected sizes on a smaller model (using the small configs) 
2. I don't really see a problem since the attention mask created use the newly defined sequence length, and the code was already very clean so probably juste a typo! 

Thanks for confirming @ArthurZucker. I have added a test that checks [here](https://github.com/tanaymeh/transformers/blob/bb6c5eaff2421766daa79d5b489318a9a2c54726/tests/models/mega/test_modeling_mega.py#L316-L340) if the `attentions` returned in the `CausalLMOutputWithCrossAttentions` have their last dimension (`shape[-1]`), which is supposed to be `sequence_length` is equal to the `chunk_size` or not.

If the checks I added in `modeling_mega.py` are correct, it will use `chunk_size` instead of the actual `sequence_length`.

Is this correct, or shall I make any changes?

Update: The tests are failing because of an error in Wav2Vec2 model, here: `test_modeling_wav2vec2.py::Wav2Vec2RobustModelTest::test_model_for_pretraining`

A Github Pull should fix it.

Added your suggested changes @ArthurZucker! With `input_mask`, the mega tests now pass.

Perfect! If you can just rebase on main to make sure the CIs are green? 

> Perfect! If you can just rebase on main to make sure the CIs are green?

Done @ArthurZucker!

Congrats on the PR 🚀 thanks for fixing! 

Thanks a lot for helping @ArthurZucker!
use clone() method before save tensor when converting Hf model to Megatron model
Author: flower-with-safe
Pull request: no

### System Info

when converting a huggingface model to megatron model, we need to split the model into TP * PP parts.
when TP is 1 and PP is X, each part of splited model occupies 1 / X storage space of the original model.
However, when TP is x and pp is 1, each part of the splited model occupies exactly the sample storage space as the original model.
This is due to when using torch.chunk() method to split tensor and then use torch.save method to save tensor, each saved tenor is a view of the original tensor and occupies the same storage space.

using the clone() method to clone tensor can solve this problem and save storage space.



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction


```python
import torch

a = torch.rand(1024, 512)
torch.save(a, "./full.pt")
print(a.size())
out = torch.chunk(a, 4, dim=0)

for i in range(4):
    print(out[i].size())
    torch.save(out[i], "./" + str(i) + "sub.pt")
    torch.save(out[i].clone(), "./" + str(i) + "sub_clone.pt")`
```
### Expected behavior

the size of the pt file explains.

Comments:
Sounds good, could you share an actual example with a model from `transformers`?

would you like to open a draft PR to show the potential changes? If this saves spaces in general and not only for megatron it's also a good addition
can not speicfy gpu in trainner
Author: kimkeithvn
Pull request: no

### System Info

- `transformers` version: 4.31.0
- Platform: Linux-4.15.0-213-generic-x86_64-with-glibc2.27
- Python version: 3.10.11
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: true
- Using distributed or parallel set-up in script?: false

### Who can help?

/

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hi I’m trying to fine-tune model with Trainer in transformers,

Well, I want to use a specific number of GPU in my server.
My server has two GPUs and I want to train my model with GPU index 1.

I’ve read the `Trainer` and `Seq2SeqTrainingArguments` documents, and I’ve tried the `CUDA_VISIBLE_DEVICES` env variable already. I found `torch.cuda.current_device()` was correct before defining training_args object by `Seq2SeqTrainingArguments`.

transformers=4.31.0
torch=2.0.0
CUDA Version: 11.8

```python
gpu_idx = 1
os.environ["CUDA_VISIBLE_DEVICES"] = str(gpu_idx)
torch.cuda.set_device(gpu_idx)

# torch.cuda.current_device = 1
...
training_args = Seq2SeqTrainingArguments(output_dir=output_dir, do_train=True, save_strategy='no',
                                         evaluation_strategy='no', logging_strategy='epoch', do_eval=False,
                                         load_best_model_at_end=True,
                                         greater_is_better=True,
                                         num_train_epochs=train_epoch, learning_rate=lr, seed=seed,
                                         per_device_train_batch_size=batch_size, per_device_eval_batch_size=batch_size,
                                         warmup_steps=warmup_steps, weight_decay=0.01, logging_dir=logging_dir,
                                         fp16=True, report_to=['tensorboard'])

# torch.cuda.current_device = 0
```


```shell
nvidia-smi
```
![image](https://github.com/huggingface/transformers/assets/41425845/5f472d30-c6ed-4bd3-8a0e-e005d5c8e188)

I will appreciate your suggestions for this problem. Thanks.

### Expected behavior

I want to specify gpu in my script with `Trainer` and `Seq2SeqTrainingArguments`
```
# torch.cuda.current_device = 1
```

Comments:
cc @muellerzr @pacman100 

Same comment as https://github.com/huggingface/transformers/issues/25321, you need to do this outside the trainer/during the call to launch the script. 

I use `CUDA_VISIBLE_DEVICES=1 acclerate launch script.py` and it works. Thanks for your advice.
Feature request: support more than device keyword arg when calling .to() on BatchEncoding
Author: NielsRogge
Pull request: no
Labels: name: Good First Issue

name: Feature request

### Feature request

I see that the `.to()` [method](https://github.com/huggingface/transformers/blob/f1660d7e23d4432513fe060bde4f9b7b29f05204/src/transformers/tokenization_utils_base.py#L751) of `BatchEncoding` returned by tokenizers only supports the `device` keyword argument.

However, the `BatchFeature` returned by image processors/audio feature extractors supports [more keyword arguments](https://github.com/huggingface/transformers/blob/f1660d7e23d4432513fe060bde4f9b7b29f05204/src/transformers/feature_extraction_utils.py#L187), most importantly `dtype`.

### Motivation

This is handy as it allows to do:
```
from transformers import ViTImageProcessor
import torch
from PIL import Image
import requests

processor = ViTImageProcessor()

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

encoding = processor(image, return_tensors="pt").to(device="cuda", dtype=torch.float16)

for k,v in encoding.items():
    print(k,v.dtype)
```
which returns
```
pixel_values torch.float16
```


### Your contribution

I could submit a PR but if anyone has the bandwidth for this, would be great to add it :D 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@NielsRogge @amyeroberts Hey I am interested on working on this issue. It is my first one so I'll need some guidance. I'll try my best and ping you if i need help!

@amannagarkar Great! :D Feel free to ask questions on the PR once it's opened! 

I can help with this too @amyeroberts
Would that be okay? @amannagarkar 

@Rishab26 I am already working on the issue! I will let you know in a couple of days? I am testing the code for errors and will open a pr shortly! 

@amannagarkar Sure, no worries. Happy to collaborate together too. I've taken a shot at it 👍

@NielsRogge I saw in the PR that you said "instead update multimodal processors in the library that return a BatchEncoding instead of a BatchFeature." What do you mean by update the multimodal processors? Also the problem addressed by this issue does not exist (i think, by the discussion in the PR) so maybe we could close it. If not, I'm willing to work on any remaining issue. 

@Lorenzobattistela you're right in that this issue can be closed, since BatchEncoding is only meant for tokenizers, which always return LongTensors, making the `dtype` irrelevant.