noamaanMulla-03/transformers-issues huggingface dataset:
ValueError: operands could not be broadcast together with shapes (60,4) (24,2,60,16,1024,64)
Author: Luke-4
Pull request: no

I am trying to fine-tune a model using the trainer API but I am getting this error: 
![image](https://github.com/huggingface/transformers/assets/138615931/22a5ce44-a0ee-4fba-83f4-9d31c9ce76d4)

I have looked online and I can't find anything similar to this, at least related to transformers and NPL.

here is the code that I am using:
https://colab.research.google.com/drive/1hdAG3rC1LHp7tJ4DCKbt90IDFsGcubHf?usp=sharing

Comments:
You should use the [forums](https://discuss.huggingface.co/) to debug your code as we keep issues for feature requests and bugs in the library only. Here it seems your `compute_metrics` function does not take the logits from the result of the model, which contains two arrays at least (the logits and some kind of hidden state).
Add support for HYBRID_SHARD and _HYBRID_SHARD_ZERO2 in the trainer
Author: raghavanone
Pull request: yes

#21156 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger   Requesting for review.

@pacman100 Gentle remainder.

Hello @raghavanone, after PR #23158, the FSDP logic is being handled by Accelerate. So, these changes should reflect in Accelerate if they aren't being supported by FSDP-XLA 

> Hello @raghavanone, after PR #23158, the FSDP logic is being handled by Accelerate. So, these changes should reflect in Accelerate if they aren't being supported by FSDP-XLA

Sure, I can make the changes there too, any pointers on things to do in accelerate will of great help.

Hello @raghavanone, the above PR adds this functionality to the Accelerate which now powers Trainer—also made you the co-author of it. Thank you for all the effort and helping us keep up-to-date with the PyTorch FSDP.

> Hello @raghavanone, after PR #23158, the FSDP logic is being handled by Accelerate. So, these changes should reflect in Accelerate if they aren't being supported by FSDP-XLA

Hi @pacman100, I have a question. Does this PR #23158 only work for accelerating launch? I am trying to use hybrid shared by using torchrun with trainer args "torchrun --fsdp hybrid shared ...". It seems that I need to merge #23812. Is there a better solution without modifying the transformers code?
🌐 [i18n-KO] Translated `perf_infer_gpu_one.md` to Korean
Author: eenzeenee
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다! -->
# What does this PR do?

Translated the `perf_infer_gpu_one.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
May you please review this PR? @sronger, @TaeYupNoh, @kj021, @HanNayeoniee, @eenzeenee, @sim-so

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
May you please review this PR? @sgugger, @ArthurZucker, @eunseojo

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Could you review this PR? 😃
@sgugger, @ArthurZucker, @eunseojo

Thanks for fixing! 🔥
Issues with SpeechT5ForTextToSpeech docs
Author: osanseviero
Pull request: no

### System Info

N/A https://huggingface.co/docs/transformers/main/model_doc/speecht5#transformers.SpeechT5ForTextToSpeech

### Who can help?

@sanchit-gandhi @vaibh

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://huggingface.co/docs/transformers/main/model_doc/speecht5#transformers.SpeechT5ForTextToSpeech

The code example does not work. In particular, `model.generate` errors out with

```
TypeError: The current model class (SpeechT5ForTextToSpeech) is not compatible with `.generate()`, as it doesn't have a language model head. Please use one of the following classes instead: {'SpeechT5ForSpeechToText'}
```

The correct way to run inference would be using `model.generate_speech` instead, but `generate_speech` does not show up in the docs.

### Expected behavior

-

Comments:
Hi @osanseviero,
Thanks for raising this issue, would you have more information on how to reproduct this issue, notably regarding the transformers version ?
Many thanks,
Yoach
Fix SpeechT5 docs
Author: osanseviero
Pull request: yes

# What does this PR do?

Fixes #25335


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @osanseviero, 

This is closely linked to [this PR](https://github.com/huggingface/transformers/pull/25233), that adds a `generate` methode to SpeechT5ForTextToSpeech, so the docs change is an expected behavior.

I'm open to add back `generate_speech` to it though, if you think it makes sense to have `generate` and `generate_speech`!

Closing after offline discussion. Having `generate` is intended as it's supported in main :)
[GPT-2] Add docs
Author: NielsRogge
Pull request: yes

# What does this PR do?

Lots of people don't seem to know about batched generation with GPT-2 and friends. Hence this PR adds a section to the docs, similar to the T5 docs.

It also fixes an issue in the T5 docs.

Comments:
cc @stevhliu 

_The documentation is not available anymore as the PR was closed or merged._

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@stevhliu can we proceed by adding this, or would you like to have it on a separate page?

I think it'll be best to put this in its own section on the [Text generation strategies](https://huggingface.co/docs/transformers/generation_strategies#text-generation-strategies) page :)

cc @gante will close this PR in favor of adding it to the text generation strategies page.

Would it be possible to add a section on batched generation?

@NielsRogge absolutely 👍
Update TF pin in docker image
Author: ydshieh
Pull request: yes

# What does this PR do?

2.12 -> 2.13

(It has been 2.13 on CircleCI for some time)
(and with the new `tensorflow_probablity` a few days ago, we need TF 2.13 otherwise tests can't be collected by pytest)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Generate: remove Marian hack
Author: gante
Pull request: yes

# What does this PR do?

Removes `adjust_logits_during_generation`, clearing a temporary hack for `Marian` that has been long fixed.

- In TF: `adjust_logits_during_generation` is defined but not called anywhere, and is not a public function, so it can be deleted.
- In PT: `adjust_logits_during_generation` is redundant with the `NoBadWordsLogitsProcessor` applied to the pad token, both set the log probability of the pad token to `-inf`. Marian models set the `bad_word_ids` to their pad token in their generation config file (e.g. [here](https://huggingface.co/Helsinki-NLP/opus-mt-en-zh/blob/main/generation_config.json#L5)), so `adjust_logits_during_generation` is redundant. It is also not a public function, so all traces can be safely removed.
- No further traces of `adjust_logits_during_generation` exist in the codebase, after these changes.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Issue from recent versions: Unexpected in state_dict: embeddings.position_ids
Author: yangheng95
Pull request: no

### System Info

- `transformers` version: 4.31.0
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.10.9
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.0.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No



### Who can help?

@sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I am sorry I encounter this issue in a this repo:
https://github.com/yangheng95/PyABSA/blob/v2/examples-v2/aspect_polarity_classification/inference.py

Maybe you can install this package: 
pip install pyabsa

Or I guess it is possible to understand this issue without an installation

### Expected behavior

I think the old checkpoints are supposed to work for latest transformers versions. But recent update make the loading fail. The update happens to many models. e.g., 
https://github.com/huggingface/transformers/blob/8e5d1619b3e57367701d74647e87b95f8dba5409/src/transformers/models/albert/modeling_albert.py#L211

I have no idea about the context of the purpose of this modification. As this is a minor issue, it will be ok to ignore and close this issue. 

Thank you for your great work!

Comments:
> I think the old checkpoints are supposed to work for latest transformers versions.

Only if use the `from_pretrained` method. You cannot use `torch.load_state_dict` without using `strict=False` since they contain key that we do not use. In general, `from_pretrained` is the fully supported way to load models for Transformers.

Thanks for your clear explanation! @sgugger
fsdp support bool type in trainArgs, use len(args.fsdp) would evoke TypeError when set fsdp=True
Author: duanzhenyu001
Pull request: no

https://github.com/huggingface/transformers/blob/2dc5e1a120176594ed2dcb7d2f02a5dd62266232/src/transformers/trainer.py#L433

Comments:
No, because it has been converted [here](https://github.com/huggingface/transformers/blob/2dc5e1a120176594ed2dcb7d2f02a5dd62266232/src/transformers/training_args.py#L1461).

Please do not open issue without a code reproducer of the problem.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Addition of test code for GPTNeoX Flax support
Author: gojiteji
Pull request: yes

@sanchit-gandhi

I have added a test code for the GPTNeoX Flax support #22950. 

I implemented it based on a fork at https://github.com/OhadRubin/transformers the above PR and [Flax GPT-Neo](https://github.com/gojiteji/transformers/blob/fix_flax_gpt_neox/tests/models/gpt_neo/test_modeling_flax_gpt_neo.py) test code.

During the execution of the tests based on [the doc](
https://huggingface.co/docs/transformers/add_new_model#2-next-prepare-your-environment), the log displayed the following output:

```
platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
rootdir: /myhomedir/transformers
configfile: setup.cfg
plugins: anyio-3.6.2
collected 43 items

tests/models/gpt_neox/test_modeling_flax_gpt_neox.py sssssssssssssssssssssssssssssssssssssssssss [100%]

=================================================== 43 skipped in 1.97s ===================================================
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24002). All of your documentation changes will be reflected on that endpoint.

Hey @gojiteji! Thanks for picking-up the Flax GPT Neo PR! Would you mind rebasing onto main:
```
git fetch upstream
git rebase upstream main
```

And then force pushing the changes:
```
git push -f origin fix_flax_gpt_neox
```

This will then isolate the changes from your PR amongst the other ones

Hey @gojiteji - not sure if you pushed or force pushed? See previous comment: https://github.com/huggingface/transformers/pull/24002#issuecomment-1577141369

Let's see if we can revive the commit history here. In the case that we can't, we probably need to open a new PR for this

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hey @gojiteji - feel free to open a new PR for this if you still want to continue the integration. Currently not sure which bits are new since the commit history is broken, but am more than happy to help with any questions / queries on a fresh PR!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix more offload edge cases
Author: ydshieh
Pull request: yes

# What does this PR do?

With this PR, only one failed tests (disk offload for `Longformer`).

~~I haven't checked `test_model_parallelism` yet~~ fix for it are required (44 failures)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

For `test_model_parallelism`, in order to pass `self.assertSetEqual(set(new_model.hf_device_map.values()), {0, 1})`, I need to set a higher ratio (`0.5` --> `0.6`), but then I get errors for some models:

```bash
        position_embeddings = self.position_embedding(position_ids)
>       embeddings = inputs_embeds + position_embeddings
E       RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!
```

It looks to me we  should keep the higher values but have a fix in the modeling file to ensure the 2 embeddings will be on the same device. Is this correct?

> For `test_model_parallelism`, in order to pass `self.assertSetEqual(set(new_model.hf_device_map.values()), {0, 1})`, I need to set a higher ratio (`0.5` --> `0.6`), but then I get errors for some models:
> 
> ```shell
>         position_embeddings = self.position_embedding(position_ids)
> >       embeddings = inputs_embeds + position_embeddings
> E       RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!
> ```
> 
> It looks to me we should keep the higher values but have a fix in the modeling file to ensure the 2 embeddings will be on the same device. Is this correct?

BTW, any comment for this, @sgugger ?
Migrate Trainer from `Repository` to `upload_folder`
Author: sgugger
Pull request: yes

# What does this PR do?

This PR migrates the internal of the push to Hub strategy inside the Trainer to the upload methods instead of the old git approach. The goal is to benefit from the improvements in terms of speed (for instance `upload_folder` is faster than `Repository` on Colab) and avoid some weird sync errors we have seen happen in the past.

This migration breaks a few things. I'm listing them for the sake of completeness but I don't think those are important breaking changes.
1. the return of `Trainer.push_to_hub` will change: in the blocking case it's the url of the repo instead of the URL of the commit. In the non-blocking case, it's the Future object returned by the `huggingface_hub` libreary (instead of a tuple commit address, job in progress)
2. before the migration, it was instant to cancel the push in progress when at the end of the training, to then push the final version of the Hub (but we also had some weird bugs). After the migration we will have to wait for the last push to be completed before pusing the final version of the model.
3. the `output_dir` of the `Trainer` won't be a git repository which is an exact clone of the model repo anymore.

Note that I chose to push checkpoint jobs in two `upload_folder`: one for the model and one for the optimizer state (roughly). This way I can cancel the second one if it hasn't started when at the end of training.

Comments:
Note that the Hub tests are failing because `upload_folder` seems very slow on moon-staging (with 503 errors and multiple retries for each push). Might be worth putting everything in blocking mode for those tests (though we won't test the fact that jobs properly run in the background then)

_The documentation is not available anymore as the PR was closed or merged._
[DOCS] Add `NoRepeatNGramLogitsProcessor` Example for `LogitsProcessor` class
Author: Rishab26
Pull request: yes

# What does this PR do?

This PR adds an example to the docstring of `NoRepeatNGramLogitsProcessor` and edits the docstring description of the same.
This is with reference to #24783 and is part of the #24575 

Fixes # (issue)
#24783

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).


## Who can review?

@gante @sgugger 

### Additional Notes:

In comparison with [TFNoRepeatNGramLogitsProcessor](https://github.com/huggingface/transformers/blob/05cda5df3405e6a2ee4ecf8f7e1b2300ebda472e/src/transformers/generation/tf_logits_process.py#L388), I noticed that the required functions `_get_ngrams`, `_get_generated_ngrams`, and `_calc_banned_ngram_tokens` were outside the class. Is this expected?



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey @gante Thanks for your feedback. I've made the requested changes. Let me know what you think. 

@gante I've made the recommended changes. Does this look better?

@amyeroberts I've made some changes. Let me know what you think. 

> Thanks for iterating!
> 
> Just a few small notes on the correctness of the docstrings / styling nits.
> 
> Note: What's being used in these functions are token ids not tokens. When a sentence is tokenized, it typically means splitting the words into word pieces e.g. `"This is a tokenized sentence"` -> `['this', 'is', 'a', 'token', '##ized', 'sentence']` i.e. it is a list of strings. These are then mapped to ints, which are token ids and that we pass to the model.
> 
> We're not completely strict everywhere with using token ids vs tokens and someone will understand if you say `list of generated tokens`. However `tokenized list of next words observed` is wrong - the list isn't tokenized and it's not next words but ints that map to word-pieces. We should say something like `list of next token ids`.

Of course, I understand that :) I was only trying to make that function simpler to understand. Having said that, you're absolutely right,  `list of next token ids` is the right framing. I'll make the change. 

@amyeroberts Thanks for your feedback. Let me know if the changes look good now 👍
Docs: Added benchmarks for `torch.compile()` for vision models
Author: merveenoyan
Pull request: yes

As discussed with @amyeroberts & @sayakpaul, this PR adds `torch.compile()` benchmarks to our documentation. I mainly benchmarked for latency, I can add throughput as well. 

I got it with doc-builder on my local, it looks like below. 

<img width="928" alt="Screenshot 2023-07-11 at 18 05 42" src="https://github.com/huggingface/transformers/assets/53175384/fecf12e0-750b-4085-8224-2fe91705bbfd">
<img width="529" alt="Screenshot 2023-07-11 at 18 04 39" src="https://github.com/huggingface/transformers/assets/53175384/abe6a3f5-0e20-4d76-9e89-8724c7b1cb45">

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sayakpaul can you check this out when you get time? after that, we can merge. 

@sayakpaul I added more models.

I thought I committed more models but apparently haven't committed the work, just did now

@sayakpaul @amyeroberts @stevhliu I added more models and visualizations, can you give another round of review?

I made a mistake when doing visualizations for T4 batch=4 ViT, I did replace the image in HF documentation-images but since it's once uploaded for md preview on GitHub, GitHub doesn't update that (so everything's actually fine, it's just GitHub)

_Note_: this PR will be stale until the benchmarks are improved. 

Hey @amyeroberts I added `nightly` + `nightly`/`reduce-overhead` comparisons.