noamaanMulla-03/transformers-issues huggingface dataset:
Generate: force caching on the main model, in assisted generation
Author: gante
Pull request: yes

# What does this PR do?

Fixes #23686 

Caching is a requirement in assisted generation -- we even check for it ([here](https://github.com/huggingface/transformers/blob/8f093fb799246f7dd9104ff44728da0c53a9f67a/src/transformers/generation/utils.py#L1485)).

However, it was still possible for the main model to run without cache. This PR fixes it.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix device issue in `OpenLlamaModelTest::test_model_parallelism`
Author: ydshieh
Pull request: yes

# What does this PR do?

See the comments in the changes.

Currently, CI has a failure 

```bash
src/transformers/models/open_llama/modeling_open_llama.py:740: in forward
    logits = torch.einsum("blh,vh->blv", hidden_states, self.model.embed_tokens.weight)
   ...
   ...
   RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument mat2 in method wrapper_CUDA_bmm)
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update `GPTNeoXLanguageGenerationTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

Due to the changes in [EleutherAI/pythia-410m-deduped](https://huggingface.co/EleutherAI/pythia-410m-deduped/commits/main), we have to update the output value, despite the new value doesn't look great 😭 

(The previous revisions seem to disappear ...I  guess they rewrite the commit history on their Hub repo) 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24193). All of your documentation changes will be reflected on that endpoint.
mac m2 max data collator issue
Author: sd3ntato
Pull request: no

### System Info

- `transformers` version: 4.28.1
- Platform: macOS-13.2.1-arm64-arm-64bit
- Python version: 3.9.6
- Huggingface_hub version: 0.13.4
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help?

@sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

data_collator = DataCollatorForSeq2Seq(
    tokenizer,
    model=model,
    label_pad_token_id=label_pad_token_id,
    pad_to_multiple_of=8,
    return_tensors='pt'
)


### Expected behavior

I'd expect normal behaviour but get
TypeError: can't convert mps:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

Comments:
No one can help without a clear reproducer of the issue.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Better prompt error messages
Author: lucasjinreal
Pull request: no

I got error when AutoTokenizer.from_pretrained:

```
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': './dist/models/vicuna-v1-7b'. Use `repo_type` argument if needed.
```

this was caused local path doesn't exist actually, but the error messsages makes me confused,

maybe add some prompt if path does not exist to aware users you should put right path rather than keep telling my using right repo id??

Comments:
Hi @lucasjinreal , I'm transferring this issue to `transformers` as this is not really related to `huggingface_hub` itself (hfh is the underlying library making calls to the HF Hub but is not responsible if a path is provided as repo_id when downloading a file).

cc @sgugger I'm not an expert on how files are loaded in `transformers` but I think a "catch `HfValidationError`" statement in [this try/except](https://github.com/huggingface/transformers/blob/main/src/transformers/utils/hub.py#L423) (`from huggingface_hub.utils import HFValidationError`) would allow a better error message.

Mmm that's actually a bit tricky since this error can come from multiple causes.

Yeah, but I think they might can be detect in priority order, some obviously pattern can handled more properly

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Generate: detect special architectures when loaded from PEFT
Author: gante
Pull request: yes

# What does this PR do?

Fixes #23686 

As identified in [this comment](https://github.com/huggingface/transformers/issues/23686#issuecomment-1587285715), a PEFT-loaded BLOOM can't be used as an assistant with assisted generation.

BLOOM (and GPTBigCode) need special handling due to their different cache API, and the architecture detection code was incompatible with PEFT models. This PR adds the logic to detect these special architectures when loaded with PEFT.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24198). All of your documentation changes will be reflected on that endpoint.
🌐 [i18n-KO] Translated tasks_summary.mdx to Korean
Author: kihoon71
Pull request: yes

# What does this PR do?

Translated task_summary.mdx file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing

- [x]  Check for missing / redundant translations (번역 누락/중복 검사)
- [x]  Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x]  Check Inline TOC (e.g. [[lowercased-header]])
- [x]  Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?(Final)
@sgugger, @ArthurZucker, @eunseojo May you please review this PR?
<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix `_load_pretrained_model`
Author: SunMarc
Pull request: yes

# What does this PR do ? 
Fixes the following test from my old PR Add check for tied parameters (#24029): 
`RUN_SLOW=1 python3 -m pytest -v tests/models/marian/test_modeling_marian.py::TestMarian_FI_EN_V2::test_batch_generation_en_fr`

Comments:
This line what something I added in my previous PR thinking that we forgot to tie the weights. But I guess it was done intentionally as I see that one test failed and generated garbage value. Here the error that we get from the test: 
```
tests/models/marian/test_modeling_marian.py:433: in _assert_generated_batch_equal_expected
    self.assertListEqual(self.expected_text, generated_words)
E   AssertionError: Lists differ: ['I like to read books', 'I like watching football'] != ['obliterat obliterat obliterat obliterat o[1345 chars]ɰɰɰ']
E   
E   First differing element 0:
E   'I like to read books'
E   'obliterat obliterat obliterat obliterat o[1214 chars]erat'
E   
E   Diff is 1556 characters long. Set self.maxDiff to None to see it.
```



> Could you explain a bit more why we need to remove this line? Thanks!



_The documentation is not available anymore as the PR was closed or merged._

BTW, let's make the PR title a bit more precise, sth like `fix _load_pretrained_model` or anything you think more precise, @SunMarc .
Thanks!

Yeah, I will explore a little bit more why we have this weird behavior before merging.
In examples, complete steps not correct when load from checkpoint
Author: Ethan-yt
Pull request: no

### System Info

https://github.com/huggingface/transformers/blob/8f093fb799246f7dd9104ff44728da0c53a9f67a/examples/pytorch/language-modeling/run_clm_no_trainer.py#L575
Should be:
```python
completed_steps = resume_step // args.gradient_accumulation_steps
```

### Who can help?

@sgugger

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Enable grad acc.
1. Save the checkpoint.
1. Load from it.

### Expected behavior

The `completed_steps` should be divided by grad acc steps.

Comments:
Would you like to make a PR with a fix?

> Would you like to make a PR with a fix?


Done!
https://github.com/huggingface/transformers/pull/24197
Fix steps bugs in no trainer examples
Author: Ethan-yt
Pull request: yes

# What does this PR do?


Fixes #24186


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->
@sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._
typo: fix typos in CONTRIBUTING.md and deepspeed.mdx
Author: zsj9509
Pull request: yes

# What does this PR do?
Fix the following two typos
+ Missing line break after the third item in the Pull Request Checklist section of the CONTRIBUTING.md.
+ A typo error in the deepspeed.mdx file

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24184). All of your documentation changes will be reflected on that endpoint.
Remove unnecessary aten::to overhead in llama
Author: fxmarty
Pull request: yes

As per title, in `LlamaRotaryEmbedding` the [`cos_cached` and `sin_cached` buffers](https://github.com/huggingface/transformers/blob/08ae37c820395e91fc3aa8b801696de5002481d2/src/transformers/models/llama/modeling_llama.py#L94-L104) are not initialized on the right dtype, because `inv_freq` is always initialized on fp32 no matter the default pytorch dtype in use.

The result is that `cos_cached` and `sin_cached` do not obey `torch_dtype=torch.float16` or `torch_dtype=torch.float32`, resulting in unnecessary overheads:

![image](https://github.com/huggingface/transformers/assets/9808326/e1aa5b64-9344-45cf-90be-f3dbf6017e1d)

I leave the `to()` in the `forward`, but it may be removed, WDYT? It is also fine as is, as it is now a no-op:

![image](https://github.com/huggingface/transformers/assets/9808326/83319770-1b7d-4dc6-b7a6-ff056376c1c7)




Comments:
woops wrong branch

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24202). All of your documentation changes will be reflected on that endpoint.
Skip RWKV test in past CI
Author: ydshieh
Pull request: yes

# What does this PR do?

In the past CI with torch 13.1 (and older), RWKV tests fail.

- The first failure is

  ```bash
  test_model_parallelism

  (line 114)  RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasSgemm( handle, opa, opb, m, n, k,   &alpha, a, lda, b, ldb, &beta, c, ldc)
  ```
  - this affects 1x more subsequential tests
  - skip this particular test will reduce a lot the number of failures.

- The docker env. of that past CI uses a base docker image shipped with `cuda 11.6`

- **In a docker image with cuda `11.8` but installing `torch==1.13+cu116`, no failure.**

Let's not take too much time on identifying what exactly cause the failure here, and just skip RWKV tests on past CI with torch < 2.0.

The goal is just to have a clean past CI report.

Comments:
Remove unnecessary aten::to overhead in llama
Author: fxmarty
Pull request: yes

As per title, in `LlamaRotaryEmbedding` the [`cos_cached` and `sin_cached` buffers](https://github.com/huggingface/transformers/blob/08ae37c820395e91fc3aa8b801696de5002481d2/src/transformers/models/llama/modeling_llama.py#L94-L104) are not initialized on the right dtype, because `inv_freq` is always initialized on fp32 no matter the default pytorch dtype in use.

The result is that `cos_cached` and `sin_cached` do not obey `torch_dtype=torch.float16` or `torch_dtype=torch.float32`, resulting in unnecessary overheads when running in fp16:

![image](https://github.com/huggingface/transformers/assets/9808326/e1aa5b64-9344-45cf-90be-f3dbf6017e1d)

I leave the `to()` in the `forward`, but it may be removed, WDYT? It is also fine as is, as it is now a no-op:

![image](https://github.com/huggingface/transformers/assets/9808326/83319770-1b7d-4dc6-b7a6-ff056376c1c7)


Comments:
Thanks @sgugger good catch, edited as suggested.
Whisper feature extraction: tiny condition check error
Author: ozancaglayan
Pull request: no

Hi

The `frame_width` below should be probably compared against `n_fft`, otherwise this `if` will always execute for nothing but not necessarily apply a padding as the frame width is already OK

https://github.com/huggingface/transformers/blob/04ab5605fbb4ef207b10bf2772d88c53fc242e83/src/transformers/models/whisper/feature_extraction_whisper.py#L163

Comments:
cc @sanchit-gandhi 

Should be fixed by https://github.com/huggingface/transformers/pull/21998

Closed via #21998
Add bark
Author: sanchit-gandhi
Pull request: yes

## What does this PR do?

Fixes #

## TODO

- [x] Add autoregressive text model
- [x] Add autoregressive coarse model
- [x] Add non-autoregressive fine model
- [x] Check text weights
- [ ] Check coarse weights
- [ ] Check fine weights
- [ ] Add Bark model / config -> what design for concatenating the three models?
- [ ] Generation code
- [ ] Update with transformers Encodec checkpoint
- [ ] Docs


Comments:
Superseded by #24086
Update `WhisperForAudioClassification` doc example
Author: ydshieh
Pull request: yes

# What does this PR do?

After the config file change in Hub commit `a7a63ecc2bd1015783dead844fced2af7531edd2` on `sanchit-gandhi/whisper-medium-fleurs-lang-id`, the doc example for `WhisperForAudioClassification` has to be updated.

Currently the test fails  due to a different expected output value.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Finish dataloader integration
Author: muellerzr
Pull request: yes
Labels: name: trainer

# What does this PR do?

Follow up to https://github.com/huggingface/transformers/pull/24028, which removes the TPU-specific dataloader bits.

The `MpDeviceLoader` does already what `Trainer` was doing before, it's just wrapped:

```python
class MpDeviceLoader(object):
  """Wraps an existing PyTorch DataLoader with background data upload.

  This class should only be using with multi-processing data parallelism.

  Args:
    loader (:class:`torch.utils.data.DataLoader`): The PyTorch DataLoader to be
      wrapped.
    device (`torch.device`...): The device where the data has to be sent.
    kwargs: Named arguments for the `ParallelLoader` constructor.
  """

  def __init__(self, loader, device, **kwargs):
    self._loader = loader
    self._device = device
    self._parallel_loader_kwargs = kwargs

  def __iter__(self):
    parallel_loader = ParallelLoader(self._loader, [self._device],
                                     **self._parallel_loader_kwargs)
    return parallel_loader.per_device_loader(self._device)

  def __len__(self):
    return len(self._loader)
```

So the native Accelerate integration will work just fine

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add the number of `model` test failures to slack CI report
Author: ydshieh
Pull request: yes

# What does this PR do?

We decided to add deepspeed (nightly version) CI job to past CI in #22393. Also `accelerate` is installed with the `main` branch.
This makes the deepspeed CI job have quite a lot of failures most of the time. Sometimes we need to wait DS team for a fix, and sometimes a fix is required from HF.

**Let's add an information regarding `number of model test failures` (i.e. not counting deepspeed CI job's failures) to the Slack CI report**, so we have a number indicating the progress in past CI and make me feel a bit more of fulfillment 🙏 .

Comments:
The screenshot below shows the issue that motivates this PR

<img width="506" alt="Screenshot 2023-06-12 205537" src="https://github.com/huggingface/transformers/assets/2521628/f6b322e2-a51e-48e9-a438-d1d431470637">


_The documentation is not available anymore as the PR was closed or merged._