noamaanMulla-03/transformers-issues huggingface dataset:
Adds `TRANSFORMERS_TEST_DEVICE`
Author: vvvm23
Pull request: yes

Adds support for environment variable `TRANSFORMERS_TEST_DEVICE` to set device in use for running the test suite. This pattern is already in use in `diffusers`.

# What does this PR do?

Adds support for environment variable `TRANSFORMERS_TEST_DEVICE` to set device in use for running the test suite. This is a pattern already in use in [`diffusers`](https://github.com/huggingface/diffusers/blob/d93ca268938940839118fdb8d0cf4c1ca7a9fead/src/diffusers/utils/testing_utils.py#L45) which would be useful to have in transformers.

Additionally, I would like to propose removing the check on available backends, as is found in the diffusers version. I included it here to match diffusers, but it would be useful to remove it (for example, if testing new backends and the like). Let me know if that is okay and I will amend the PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger, git blame says you! :hugs: 


Comments:
Hi thanks, it is a good suggestion to simply try creating the device. Perhaps `diffusers` can be updated to do the same?

Doing this approach makes the list of backends obsolete, so I removed. I just let it throw an unhandled error as I feel the message is informative enough, but let me know if you want that to be caught.

Updated the error message, let me know if it works for you~

> Thanks a lot! Can you just run make style on your branch to fix the quality issue?

Woops, I ran this but forgot to commit the changes.

> Could you also add some documentation in testing.md to not make it hidden?

Will do 👍 where do you think the best place is this for this in the file?

I should say, to all the suggested changes here, should these also be mirrored in `diffusers`? My initial PR was basically a direct copy from there.

You can probably add it to `docs/source/en/testing.md`.
Not sure about diffusers but documenting a new env variable should always be good 😉 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25506). All of your documentation changes will be reflected on that endpoint.

@ArthurZucker added a small section following the "To GPU or not to GPU" section – in my head this fits well. Let me know how you feel about the language there 🤗 I ran `make style` also which I believe also formats the docs.
Skip `test_beam_search_xla_generate_simple` for `T5`
Author: ydshieh
Pull request: yes

# What does this PR do?

TF 2.13 breaks this test on GPU 😢 😭 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25566). All of your documentation changes will be reflected on that endpoint.
install from source dont work
Author: IdoTal120
Pull request: no

### System Info

ms server 2019 

PS C:\Users\a_ital> pip install git+https://github.com/huggingface/transformers
Collecting git+https://github.com/huggingface/transformers
  Cloning https://github.com/huggingface/transformers to c:\users\a_ital\appdata\local\temp\6\pip-req-build-no6t74od

  ERROR: Error [WinError 2] The system cannot find the file specified while executing command git version
ERROR: Cannot find command 'git' - do you have 'git' installed and in your PATH?


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

PS C:\Users\a_ital> pip install git+https://github.com/huggingface/transformers
Collecting git+https://github.com/huggingface/transformers
  Cloning https://github.com/huggingface/transformers to c:\users\a_ital\appdata\local\temp\6\pip-req-build-no6t74od

  ERROR: Error [WinError 2] The system cannot find the file specified while executing command git version
ERROR: Cannot find command 'git' - do you have 'git' installed and in your PATH?
PS C:\Users\a_ital>

### Expected behavior

I need to install transformers 


Comments:

Hello @IdoTal120 ! Welcome to Github, 👋
Taking this into account
```
ERROR: Error [WinError 2] The system cannot find the file specified while executing command git version
ERROR: Cannot find command 'git' - do you have 'git' installed and in your PATH?
```


The error message is telling you that `pip `doesn't know where you have installed `git`. Can you confirm that you have `git` installed? It can be installed for Windows [here](https://git-scm.com/download/win).

Then you need to add it to the PATH environment variable. Currently, `pip` is checking for `git.exe `in all the locations listed in PATH, but the location for the` git `executable is not there. I believe during installation of git you have the option to update PATH automatically, but you can do it at any time: find the filepath to git.exe on your local machine (for example, `C:\Program Files\...\git\bin`) and then add it to PATH ([instructions here for your operating system I think](https://www.opensourceforu.com/2021/01/how-to-install-and-configure-git-on-a-windows-server/)).

Taken from [here](https://github.com/stefmolin/Hands-On-Data-Analysis-with-Pandas-2nd-edition/issues/3)
Let me know how it goes and welcome to the Open! 


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
NLLB MoE router_state referenced before assignment
Author: drunkcoding
Pull request: no

### System Info

- `transformers` version: 4.29.2
- Platform: Linux-5.15.0-69-generic-x86_64-with-glibc2.17
- Python version: 3.8.17
- Huggingface_hub version: 0.15.1
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker @youn

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python

model = AutoModelForSeq2SeqLM.from_pretrained("facebook/nllb-moe-54b")
model(
    input_ids=input_ids,
    attention_mask=attenstion_mask,
    decoder_input_ids=decoder_input_ids,
    decoder_attention_mask=decoder_attention_mask,
    output_router_logits=True,
    return_dict=True,
)
```

```bash
transformers/models/nllb_moe/modeling_nllb_moe.py", line 720, in forward
outputs += (router_states,)
UnboundLocalError: local variable 'router_states' referenced before assignment
```

### Expected behavior

return encoder_router_logits and decoder_router_logits rather than error. The error happens on the dense layers where no router_state is returned.

Comments:
cc @ArthurZucker 

Hey! Thanks for reporting! I remember working on a bug where NLLB-MoE was not being torch compiled because None values were returned. Will push a fix! 
Glad to see that Nllb-MoE is being used 🤗
[`NllbMoe`] Update code to properly support loss computation
Author: ArthurZucker
Pull request: yes

# What does this PR do?
The code works in `Switch` because we decided to only return none `None` outputs, but it makes more sens to return None when the layers is not sparse. Otherwise finding the index of the layer that produced this might be impossible. 

fixes #24898
- TODO: 
- [x] Update Switch Transformers code as well
- [x] Add some tests

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25429). All of your documentation changes will be reflected on that endpoint.
[`Tests`] Fix failing 8bit test
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes two failing tests in https://github.com/huggingface/transformers/actions/runs/5873964880/job/15928072870 

- `tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_get_keys_to_not_convert` & `tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_get_keys_to_not_convert`

Context: https://github.com/huggingface/transformers/pull/25105 added stronger checks to enable the correct quantization of models on the Hub. [Therefore it added a test that checks if `mpt-7b` is correctly quantized](https://github.com/huggingface/transformers/blob/main/tests/quantization/bnb/test_mixed_int8.py#L141). Since that model requires einops to be added as a dependency I propose to simply add einops in the docker image

cc @ydshieh 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @younesbelkada ! Could you tell a bit more why MPT needs `einops`?

Also you probably need refresh your CircleCI token?

for MPT as you can see from the remote model, einops is used in files that are imported in `modeling_mpt.py` such as `attention.py` here: https://huggingface.co/mosaicml/mpt-7b/blob/main/attention.py#L7 it seems to be a strong dependency (hence the error on the slow test attached)
It is also listed as a dependency in the requirements file together with their custom fork of triton: https://huggingface.co/mosaicml/mpt-7b/blob/main/requirements.txt#L2 but the triton dependency is optional

OK, I think I got confused by the fact we have a modeling file and we also use the same checkpoint name in our model tests. Now I get it, but a question: why we want to use the remote code rather than the code in `transformers` for this quantization test ..? Is it really necessary to use the remote code the quantization?

Thanks! 
Regarding your question, yes I think so, #25105 added the support for correct quantization for most remote code models, therefore it is crucial to test the remote code model + non remote model in the test
Revert "change version (#25387)"
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR revert  #25387. As @younesbelkada  pointed [out](https://github.com/huggingface/transformers/pull/25387#pullrequestreview-1582170053), the training still works as expected if we don't use 8-bit optimizers. I will propose a fix in a follow-up PR. 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
ONNX test suite is slow - run in 5.5 hours
Author: ydshieh
Pull request: no
Labels: name: Tests

name: Benchmarks

name: WIP

### Who can help?

@lewtun  @LysandreJik 

As shown in [this job](https://github.com/huggingface/transformers/runs/8074936385?check_suite_focus=true) run, the ONNX tests now run in 5.5 hours.

From https://github.com/huggingface/transformers/blob/73c6273d481f9052098a2a5a5f001fa75daaace9/tests/onnx/test_onnx_v2.py#L182
we see that the tests use real model checkpoints. As ONNX graph compile is known to be slow, and the models are quite big, it makes the tests very slow.

The whole scheduled CI test suite now run in 14.5 hours, and we have 2 test suites to run each day, so it requires 29 hours and can't finish in one day. This causes the test suite and their reports being delayed a lot.

We are wondering if it is possible to use tiny models from [hf-internal-testing](https://huggingface.co/hf-internal-testing) for ONNX tests.


Comments:
Thanks for raising this - it's an issue we've also faced with `optimum`'s test suite. Let me take a look and see if it's possible to use the tiny models as you suggest

cc @echarlaix @philschmid 

Keep the issue alive :-)

Thanks for the ping - on my TODO list this week!
Support `text-to-speech` in `pipeline` function and in Optimum
Author: josephrocca
Pull request: no
Labels: name: Core: Pipeline

name: Feature request

### Feature request

SpeechT5 was recently added to Transformers:

* **Blog post**: https://huggingface.co/blog/speecht5
* **Spaces demo**: https://huggingface.co/spaces/Matthijs/speecht5-tts-demo
* **Models**: https://huggingface.co/mechanicalsea/speecht5-tts

It would be great if `text-to-speech` could be supported across the Transformers stack.

### Motivation

@xenova [bumped into this as an issue](https://github.com/xenova/transformers.js/issues/59) when trying to get SpeechT5 working in the browser (Transformers.js).

### Your contribution

Probably unable to help with this at the moment.

Comments:
cc @sanchit-gandhi 

Indeed, a TTS pipeline would be super helpful to run SpeechT5. We're currently planning on waiting till we have 1-2 more TTS models in the library before pushing ahead with a TTS pipeline, in order to verify that the pipeline is generalisable and gives a benefit over loading a single model + processor.

cc @hollance

Any viable contenders for the other 1-2 models? https://paperswithcode.com/task/text-to-speech-synthesis

Hey, I'd be more than happy to take up this task if we can decide on the other 1-2 models

> Hey, I'd be more than happy to take up this task if we can decide on the other 1-2 models

We can probably just select the most popular models from the hub: https://huggingface.co/models?pipeline_tag=text-to-speech&sort=downloads

There is an [open PR](https://github.com/huggingface/transformers/pull/15773) for FastSpeech2. I think this is a good new model to add. If anyone is interested in taking that PR to completion, that would be awesome!

> Hey, I'd be more than happy to take up this task if we can decide on the other 1-2 models

Let me know if you need any help! I’m excited for this to be added 🔥

Here's another model which could fall into the `text-to-speech` category: https://github.com/huggingface/transformers/issues/23036

Just added one more https://github.com/huggingface/transformers/issues/23050

Please add support for the mms-tts model as mentioned in above [issue](https://github.com/xenova/transformers.js/issues/209)  to the TTS pipeline.

Good news! This is currently being worked on: https://github.com/huggingface/transformers/pull/24952 🚀🔥
Skip `test_contrastive_generate` for `TFXLNet`
Author: ydshieh
Pull request: yes

# What does this PR do?

This model has special cache mechanism and `test_contrastive_generate` fails for more than 8 month. When I remove the argument `penalty_alpha` from the kwargs, it is able to generate something.



Comments:
_The documentation is not available anymore as the PR was closed or merged._

For the record, it starts to fail on commit `a6c850e4` (#20994) but with a different error `TypeError: prepare_inputs_for_generation() got multiple values for keyword argument 'use_cache'`. Then in #21149, we get the current error.
add warning for 8bit optimizers
Author: SunMarc
Pull request: yes

# What does this PR do ? 
This PR adds a log when 8-bit optimizers are used with a version of bitsandbytes < 0.41.1. We do that because a major bug was fixed for 8-bit optimizers as reported by [Tim](https://twitter.com/Tim_Dettmers/status/1687458541643390976). 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
#24028 seems to break the last epoch for a logging integration
Author: franz101
Pull request: no

### System Info

Hey @muellerzr,

thanks for your lightning fast (accelerated) reply ;) regarding #24028, I'm currently debugging what's causing the issue

Setup:
- A custom callback to log embeddings, the data collator in the Trainer is wrapped to extract ids of each sample in a batch

Error:
- The wrapped data collation works fine except in the last step

How to reproduce?
See reproduction tab

Currently this is the example I can show for reproduction.
My first guess, it's related to multiprocessing. It seems like the custom collator is not called in the last step.
But will give more details or a possible solution soon.


### Who can help?

@muellerzr

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
git clone https://github.com/rungalileo/dataquality.git
cd dataquality
python -m venv .venv
source .venv/bin/activate
pip install invoke
inv all
pip install --upgrade transformers
pytest tests/integrations/hf/test_text_classification_hf.py -s -k test_remove_unused_columns
```

### Expected behavior

Test should finish collating each step

Comments:
Reproducible in [this colab example](https://colab.research.google.com/drive/1NObtjxY37VEx2u3SBFDDa_zwTILbX3B2?usp=sharing) currently

Found the issue, it seems like the on step end is called only after two data points have been collated.
Load T5 model in 8 bit fails
Author: PansaLegrand
Pull request: no

### System Info

I use free google Colab with T4 GPU

### Who can help?

@ArthurZucker 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I am running the colab created for [DeepFloyd](https://huggingface.co/blog/if),

and I cannot load the T5 encoder in 8 bits.

```
from transformers import T5EncoderModel

text_encoder = T5EncoderModel.from_pretrained(
    "DeepFloyd/IF-I-XL-v1.0",
    subfolder="text_encoder", 
    device_map="auto", 
    load_in_8bit=True, 
    variant="8bit"
)
```

The error message is :
```
You are loading your model in 8bit or 4bit but no linear modules were found in your model. Please double check your model architecture, or submit an issue on github if you think this is a bug.

RuntimeError                              Traceback (most recent call last)
[<ipython-input-66-186cdabda356>](https://localhost:8080/#) in <cell line: 3>()
      1 from transformers import T5EncoderModel
      2 
----> 3 text_encoder = T5EncoderModel.from_pretrained(
      4     "DeepFloyd/IF-I-XL-v1.0",
      5     subfolder="text_encoder",

4 frames
[/usr/local/lib/python3.10/dist-packages/torch/nn/parameter.py](https://localhost:8080/#) in __new__(cls, data, requires_grad)
     34             # For ease of BC maintenance, keep this path for standard Tensor.
     35             # Eventually (tm), we should change the behavior for standard Tensor to match.
---> 36             return torch.Tensor._make_subclass(cls, data, requires_grad)
     37 
     38         # Path for custom tensors: set a flag on the instance to indicate parameter-ness.

RuntimeError: Only Tensors of floating point and complex dtype can require gradients
```



I cannot see where the problem come from.

Comments:
cc @SunMarc and @younesbelkada 

I am facing the same issue as well, anybody found a solution to this?

Thanks for reporting this issue @leejielong and @PansaLegrand . To solve this, please install the main branch of transformers `pip install git+https://github.com/huggingface/transformers.git`. I was able to reproduce the issue with the latest release v4.31.0 but the issue is solved in the main branch. 

Hi @SunMarc, installing transformers from source worked for me too. Thanks!
Fix typo in example code
Author: ameliereymond
Pull request: yes

# What does this PR do?
Fixes a typo in the docs

`lang_code_to_id("en_XX")` => `lang_code_to_id["en_XX"]`

lang_code_to_id is a dict

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

 @sgugger

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25583). All of your documentation changes will be reflected on that endpoint.
more and more difficult to use
Author: Misoknisky
Pull request: no

### System Info

huggingface is going the same way as TensorFlow, version control is getting worse and worse. In particular, the versions between different packages need to be strictly adapted, which is more troublesome than TensorFlow!

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Extremely bad experience! When I used the datasets library to load the glue dataset, I found that the datasets library had a chain reaction with hub, evaluate, pandas, etc., and various conflicts did not match. The data formats provided by different hubs and other libraries are not compatible, and it is really a headache to adapt them.

### Expected behavior

Extremely bad experience! When I used the datasets library to load the glue dataset, I found that the datasets library had a chain reaction with hub, evaluate, pandas, etc., and various conflicts did not match. The data formats provided by different hubs and other libraries are not compatible, and it is really a headache to adapt them.

Comments:
Hey! Thanks for providing feedback, though this is the `transformers` repository, `datasets` has its own repo [here](https://github.com/huggingface/datasets) if you want to give them your thoughts!