noamaanMulla-03/transformers-issues huggingface dataset:
Error when fine-tuning RWKV using HuggingFace Trainer: 2 positional arguments but 3 were given
Author: breadbrowser
Pull request: no

### System Info

i am using Kaggle and I am using 2 t4 gpus.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

steps to reproduce:

1. run my code
2. it just happens

### Expected behavior

this is my code https://www.kaggle.com/code/lostgoldplayer/fine-tuning using the huggingface trainer,
i am trying to fine-tune "RWKV/rwkv-4-430m-pile" using my dataset "breadlicker45/musenet-encoders-12k".

Comments:
cc @younesbelkada @ArthurZucker 

I think this has been fixed in https://github.com/huggingface/transformers/pull/23774 
Can you try to install transformers with:
```bash
pip install git+https://github.com/huggingface/transformers.git
```

> I think this has been fixed in #23774 Can you try to install transformers with:
> 
> ```shell
> pip install git+https://github.com/huggingface/transformers.git
> ```

thanks, it works now
Fix `is_optimum_neuron_available`
Author: michaelbenayoun
Pull request: yes

# What does this PR do?

#23163 introduced a new way for checking which seems to not work in this case. 
I get this when running `optimum-neuron`:

```
Please use the TrainiumTrainer from optimum[neuron] instead of the Transformers library to perform training on AWS Trainium instances. More information here: https://github.com/huggingface/o
ptimum-neuron  
```

It happens [here](https://github.com/huggingface/transformers/blob/main/src/transformers/training_args.py#L82).
Basically it means that `optimum-neuron` is found to not be available, this PR fixes that.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

No, here the check for `_is_package_available("optimum.neuron")` is done at "runtime" not "import time". 
For some reason `optimum.neuron` is not available at import time of this file but is available later... 
Maybe this is due to the fact that `optimum` is a namespace package, not sure.

Yes, I will investigate. If I find a fix on the other side, will revert back to the original version here.

I'm fine with having the workaround :-) Just flagging this for you to look at when you have some time.

Failure is due to a library pinned on main, so merging.
fix executable batch size issue
Author: pacman100
Pull request: yes

# What does this PR do?
1. Fixes #24050 
2. Context: We weren't handling properly the `auto_find_batch_size=True` case. Here, we need to free all of the stored model references in the Accelerator each time as mentioned in the https://github.com/huggingface/accelerate/blob/main/examples/by_feature/automatic_gradient_accumulation.py
3. This PR does that.


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Not to merge before 2023/06/28] Time to say goodbye to py37 😭 
Author: ydshieh
Pull request: yes

# What does this PR do?

Byebye!

EOL of python 3.7 is `2023/06/27`.

https://endoflife.date/python



Comments:
_The documentation is not available anymore as the PR was closed or merged._

test failures will be addressed soon by related authors. Not related to this PR.
Up pinned accelerate version
Author: muellerzr
Pull request: yes

# What does this PR do?

Increases pinned accelerate version, and also let's the `is_accelerate_available` check see for a specific version, since now we care much more about just if `PartialState` is available.

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Not to merge before 2023/06/28] Time to say goodbye to py37
Author: ydshieh
Pull request: yes

# What does this PR do?

Same as #24075, but that PR got freezed after I force pushed (after rebase), and my changes to address the comments were not able to appear)

----

Byebye!

EOL of python 3.7 is `2023/06/27`.

https://endoflife.date/python




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24087). All of your documentation changes will be reflected on that endpoint.
Do not prepare lr scheduler as it as the right number of steps
Author: sgugger
Pull request: yes

# What does this PR do?

Right now the LR scheduler is prepared by the Accelerate but it has a number of steps that already accounts for the number of processes. This results in the LR scheduler being stepp through num_processes too fast. This PR thus removes the lr_scheduler from the prepared objects.

Should fix #23986 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
fix accelerator prepare during eval only mode
Author: pacman100
Pull request: yes

# What does this PR do?
1. As mentioned in https://github.com/huggingface/transformers/pull/23957#issuecomment-1573810900, currently the accelerate `prepare` method is happening only during training loop. If the user is directly doing `eval`/`predict` without the training loop, the model isn't prepared leading to wrong behaviour. This PR is aimed at fixing it.
2. Should be merged after https://github.com/huggingface/accelerate/pull/1540

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The thing is that mixed precision application for eval only mode won't work unless we prepare model
Up pinned accelerate version
Author: muellerzr
Pull request: yes

# What does this PR do?

Increases pinned accelerate version, and also let's the `is_accelerate_available` check see for a specific version, since now we care much more about just if `PartialState` is available.

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger

re-opened again so tests can pass and we can merge 😅 

Comments:
Let's wait for the patch and pin 0.20.1

_The documentation is not available anymore as the PR was closed or merged._
Support Azure OpenAI in transformer agents
Author: huajianmao
Pull request: no

### Feature request

Support it by setting the openai properties.

``` python
openai.api_type = "azure"
openai.api_version = "2023-03-15-preview"
openai.api_base = "https://THE_NAME.openai.azure.com/"
openai.api_key = "AZURE_OPENAI_API_KEY"

# response = openai.Completion.create(engine="xxxx", ...)
```

### Motivation

Azure OpenAI is also used by a lot of people.
It would be great to support it in the transformer agents.

### Your contribution

I may contribute to test it.

Comments:
@huajianmao thanks for opening this issue, 

As a user can enable azure open AI support by adding these lines to their own code, there isn't a requirement to add support in the library. 

> @huajianmao thanks for opening this issue,
> 
> As a user can enable azure open AI support by adding these lines to their own code, there isn't a requirement to add support in the library.

That's not entirely correct. You'll get an InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 
'openai.api_resources.completion.Completion'>

Because you would have to support some way of specifying the Azure-deployment_id in your code.

https://huggingface.co/docs/transformers/v4.29.1/en/main_classes/agent#transformers.OpenAiAgent
agent = OpenAiAgent(model="text-davinci-003", api_key=pswd)
ideally would be changed. The Azure API is a different type and different enough to allow the relevant parameters to be passed in.
It doesn't sound "clean" to me to access the openai API parameters directly outside the OpenAiAgent.
The Azure based version is rapidly gaining in importance as people can use their corporate or personal Azure credits vs having to pay OpenAI separately.

Also the call needs to be changed to something like that currently:
        result = openai.Completion.create(
            #model=self.model,
            deployment_id=self.model,
            prompt=prompts,
            temperature=0,
            stop=stop,
            max_tokens=200,
        )

btw. something to easily stumble over that the version is api_version="2022-12-01" even though the portal shows "1". Best is to go to the playground and have it generate example code currently it seems.

for GPT4 you need to use 
openai.api_version = "2023-03-15-preview" # used for GPT-4 - see https://learn.microsoft.com/en-gb/azure/cognitive-services/openai/how-to/chatgpt?pivots=programming-language-chat-completions

Be aware the chat message pattern changed nd as well the json for the response, which is the new mode for all new models after GPT4 as well.

Sure, but we were not talking about GPT-4. The examples only covered DaVinci-003

Anyway the api is changed - no matter what model you use - you need to adopt to the new prompt style.
Add AzureOpenAiAgent
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds an AzureOpenAiAgent, superceding #23355 since the contributor there does not seem to want to finish the PR.

Fixes #23324 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24058). All of your documentation changes will be reflected on that endpoint.
Add FlaxWhisperForAudioClassification model
Author: raghavanone
Pull request: yes

Fixes #21779 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @sanchit-gandhi 

The test failures are appearing on this one. Let's fix them and re-merge!

We need to make two changes following the updates in #22954! First, we need to assign the attribute `gradient_checkpointing` to the class `FlaxWhisperForAudioClassificationModule`, similar to what we do for `FlaxWhisperForConditionalGeneration`:
https://github.com/huggingface/transformers/blob/a5741d7cb59f8a81d1f5fc7a6b106056d34f9969/src/transformers/models/whisper/modeling_flax_whisper.py#L1176
We then need to forward `self.gradient_checkpointing` to the encoder:
```diff
-         self.encoder = FlaxWhisperEncoder(config=self.config, dtype=self.dtype)
+         self.encoder = FlaxWhisperEncoder(config=self.config, dtype=self.dtype, gradient_checkpointing=self.gradient_checkpointing)
```
This will facilitate gradient checkpointing for the module!

@sgugger @sanchit-gandhi Done, all tests pass !
Fix a tiny typo in `WhisperForConditionalGeneration::generate` docstring
Author: sadra-barikbin
Pull request: yes

- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


@sanchit-gandhi 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sadra-barikbin Thanks for fixing this! 

It seems there is an issue with your CircleCI permissions, the tests won't run.
Could you try refreshing your permissions as shown [here](https://support.circleci.com/hc/en-us/articles/360048210711-How-to-Refresh-User-Permissions-)?

CircleCI has banned my account. Feel free to make another PR.

@sadra-barikbin OK - as the changes are small and don't affect code logic, the ci checks aren't critical, so I'm going to merge.
reset accelerate env variables after each test
Author: pacman100
Pull request: yes

# What does this PR do?

## Context:
For the failing tests in `test_trainer_ext`, the reason is given below:
1. These tests are run via following command:
```
python -m pytest -v --make-reports=multi-gpu_tests_torch_cuda_extensions_gpu tests/deepspeed tests/extended
```

2. Now, as DeepSpeed tests are run first, even though the AcceleratorState  is reset during teardown, the env variable set by Accelerate ACCELERATE_USE_DEEPSPEED  isn't deleted (if the test isn't a script run as a subprocess) and as such Accelerator object initialization in extended tests create DeepSpeedPlugin leading to them failing with - HFDeepSpeedPlugin raises config mismatch error

3. Simple reproducer:
```
cd transformers
export CUDA_VISIBLE_DEVICES="0,1"
export RUN_SLOW="yes"
pytest -sv tests/deepspeed/test_deepspeed.py::TrainerIntegrationDeepSpeed::test_hf_ds_config_mismatch tests/extended/test_trainer_ext.py::TestTrainerExt::test_run_seq2seq
```

This PR fixes it by deleting all the env variables having `ACCELERATE` in them during test `tearDown`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[`Trainer`] Correct behavior of `_load_best_model` for PEFT models
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/24096

This PR fixes the bugs related with PEFT models and `load_best_model_at_end`. It also refactors a bit the current logic to extend it generally to all LoRA models, not only 8-bit base models + LoRA.

<details><summary>Repro script</summary>

```python
from datasets import load_dataset
from trl import SFTTrainer
from peft import LoraConfig
from transformers import TrainingArguments

dataset = load_dataset("imdb", split="train")

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)

args = TrainingArguments(
    max_steps=1,
    save_steps=1,
    eval_steps=1,
    evaluation_strategy="steps",
    per_device_train_batch_size=1,
    resume_from_checkpoint=True,
    output_dir="test_trainer",
    load_best_model_at_end=True,
)

trainer = SFTTrainer(
    "EleutherAI/gpt-neo-125m",
    train_dataset=dataset,
    eval_dataset=dataset,
    dataset_text_field="text",
    peft_config=peft_config,
    max_seq_length=128,
    args=args,
)
trainer.train()
```
</details>

cc @sgugger @pacman100 



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Flyte Callback
Author: peridotml
Pull request: no

### Feature request

Hi! I am an OSS contributor to [Flyte](https://flyte.org/), a workflow orchestration tool. I am working on a HuggingFace plugin to integrate better with Hugging Face.

I am working on a `FlyteCallback` that will automatically integrate with Flyte's checkpointing, visualizations, and eventually logging! 

I was hoping that we could add it to the `integrations.py` similar to other ML Tools, but I wanted to check with you all before making a PR.

### Motivation

A good way to help Flyte users working with Hugging Face and Hugging Face users who end up using Flyte for orchestration.

### Your contribution

I would clean up and extend the callback in this [gist](https://gist.github.com/peridotml/68f376f0f4fd1926fb0746daaeea09f8) and create a PR.

Comments:
Hi there! You can definitely open a PR. `integrations.py` is open to the community and callbacks defined there are maintained by their authors and not by us. So as long as you're fine getting pinged by users who get a problem with this callback in the future, please share your new callback :-)
fix get_keys_to_not_convert function
Author: SunMarc
Pull request: yes

# What does this PR do ? 

Fix the behavior of the get_keys_to_not_convert function for the following cases: 
- If the lm_head is tied, we won't be able to see it using the method named_parameters() and  the last visible module was added instead -> using `named_children() `instead
- Fix tied_params variable that we should not crop.(  Example of what was happening : `[['lm_head.weight', 'model.decoder.embed_tokens.weight']] -> ['model.decoder.embed_tokens.weight']`)



Comments:
_The documentation is not available anymore as the PR was closed or merged._
[`GPT2`] Add correct keys on `_keys_to_ignore_on_load_unexpected` on all child classes of `GPT2PreTrainedModel`
Author: younesbelkada
Pull request: yes

# What does this PR do?

as per title

forgot to add them in https://github.com/huggingface/transformers/pull/23256

Currently this snippet:
```python
from transformers import GPT2Model

model = GPT2Model.from_pretrained("gpt2")
```
Gives a big warning:
```bash
Some weights of the model checkpoint at gpt2 were not used when initializing GPT2Model: ['h.10.attn.bias', 'h.5.attn.bias', 'h.7.attn.bias', 'h.0.attn.bias', 'h.11.attn.bias', 'h.8.attn.bias', 'h.1.attn.bias', 'h.9.attn.bias', 'h.2.attn.bias', 'h.4.attn.bias', 'h.6.attn.bias', 'h.3.attn.bias']
- This IS expected if you are initializing GPT2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing GPT2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
```
This PR fixes it by adding the correct regex expressions on `_keys_to_ignore_on_load_unexpected` for all child classes that inherit from `GPT2PreTrainedModel`

cc @sgugger @patrickvonplaten 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24113). All of your documentation changes will be reflected on that endpoint.
Trainer doesn't run `compute_metrics` when a `torch.compile` model is passed.
Author: davidgilbertson
Pull request: no

### System Info

- `transformers` version: 4.28.1
- Platform: Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.31
- Python version: 3.10.8
- Huggingface_hub version: 0.12.1
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Run training with evaluation that has a `compute_metrics` function defined.

When passing the model to `Trainer`, pass a `torch.compile()` wrapped model.

In `Trainer.__init__()` there's the line `default_label_names = find_labels(self.model.__class__)` but the model class is `torch._dynamo.eval_frame.OptimizedModule` so no labels are assigned and this has the side effect of `compute_metrics` not being run.

It would be great if this checked for this case and got the correct model, or just threw a warning. I'm guessing lots of people are going to come across this as Torch 2.0 gains traction.

I only realised _after_ chasing down the cause of evaluation not running that I can pass `torch_compile=True`, so this bug no longer affects me.

### Expected behavior

Works the same as passing a non-wrapped model.

Comments:
Another thing I just discovered, it also doesn't complete `save_pretrained()` correctly. Is saves _something_, but there isn't a `config.json` file in there. I'm guessing it's the line in `_save()` starting with `if not isinstance(self.model, PreTrainedModel)...`

Again, I know now I can do `torch_compile`, but I reckon this is going to sting lots of users as they try to pass in a compile model with the understanding that it "just works" without any code changes.

You shouldn't pass a `torch.compile`-d model to the Trainer, but let the Trainer do the compilation itself.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
add trust_remote_code option to CLI download cmd
Author: radames
Pull request: yes


# What does this PR do?

Add option to allow trust remote code download via CLI command

Address #24063 

Fixes # (issue)


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case). #24063 
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. #24063 
- [] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).

Could't find much documentation about `transformers-cli download`

- [X] Did you write any new necessary tests?

Added two test, one for a simple download pointing to `tmp` and check if folders `blobs,snapshots,refs` are present, and a test for `--trust-remote-code`.
Duplicated testing model https://huggingface.co/hf-internal-testing/test_dynamic_model_with_tokenizer adding a tokenizer to work as expected

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger add the decorator, waiting to se if the tests here are passing. on my local env all the 4 test passes successfully 🤔 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24097). All of your documentation changes will be reflected on that endpoint.