noamaanMulla-03/transformers-issues huggingface dataset:
DeBERTa/DeBERTa-v2/SEW Support for torch 1.11
Author: LysandreJik
Pull request: yes

The internal `torch` method `_softmax_backward_data` changed API between 1.10 and 1.11, from requiring a tensor as its last output to requiring a size.

This PR updates the concerned models so that they are correctly supported.

Torch 1.11: https://github.com/pytorch/pytorch/blame/e47a5a64bbf4d388b70397e3237f9d5710ee4c9c/tools/autograd/derivatives.yaml#L1861
Before: https://github.com/pytorch/pytorch/blame/768cfaa8f86bf7c7b0af441d1536f060274c27a0/tools/autograd/derivatives.yaml#L1704

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Addressed your comment @sgugger, could you do a second review? 

As seen with Sylvain offline, I've moved out the `packaging.version.parse` operation of the methods, as otherwise they would be called inside the methods themselves, which are called multipled times in forward passes. @patrickvonplaten could you check if that's fine with you?

@sgugger  @LysandreJik  Thanks for your awesome work on building this immensely valuable ecosystem (and community!). 
I'm waiting to release a package that requires this post-4.17 commit and it would be great to avoid pointing to a specific commit for packaging purposes. Is a 4.17.1 patch release planned? I asked on Discord but this was indicated to be a better forum for the question. Thanks again for helping lead this great community!

I don't think we have a patch planned. We will have 4.18 released probably next week instead :-) 

> I don't think we have a patch planned. We will have 4.18 released probably next week instead :-)

AWESOME! looking forward to it!

Note that this evaluates True for pre-releases such as '1.11.0a0+b6df043'. So the the error is still present.
`is_torch_less_than_1_11 = version.parse(version.parse(torch.__version__).base_version) < version.parse("1.11")` may help.
Am I missing something?
Number of BART layers is confusing in the Pretrained models page
Author: mnskim
Pull request: no

In the [Pretrained models](https://huggingface.co/transformers/pretrained_models.html) page, facebook/bart-large is described as 24-layer, 1024-hidden, 16-heads, 406M parameters, and facebook/bart-base as 12-layer, 768-hidden, 16-heads, 139M parameters.

I think this is a little confusing because bart-base consists of 6 encoder layers and 6 decoder layers, and bart-large consists of 12 layers each. (You can check in [the fairseq repo](https://github.com/pytorch/fairseq/tree/master/examples/bart)) 

I think specifying the layers as: "N encoder layers, N decoder layers" would be a lot clearer

Comments:
Hi @mnskim 

Usually, we specify the total number of layers in a model instead of an individual components layer (i.e encoder or decoder). And seq2seq models like BART, T5 usually use the same number of layers in both encoder-decoder i.e 6-6 or 12-12. We could specify the encoder layer and decoder layer if they are different.

But yeah, if you think specifying encoder and decoder layer is clearer then feel free to open a PR :) We could write 12 layer (6 encoder and 6 decoder layers). Thanks.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

There is another issue with this page. facebook/bart-base actually contains 12 heads, not 16 heads. Unfortunately, the file for this page has been removed, so it cannot be updated anymore. The wrong page is still available on Google, though.

![](https://user-images.githubusercontent.com/68557794/160236798-5fb25524-5851-48a1-b6c6-605b9afb197d.png)

![](https://user-images.githubusercontent.com/68557794/160236815-40cd6335-cc5b-40f4-93a5-59a2f7d6788f.png)
Don't use `store_xxx` on optional bools
Author: sgugger
Pull request: yes

# What does this PR do?

Optional bool fields in `TrainingArguments` are given the `store_true` attribute by `HFArgumentParser` which can yield to bugs (as highlighted in #7755). This PR fixes this and to avoid breaking existing scripts, removes the optional in the `evaluate_during_training` argument.

It also fixes a few instances with the right argument called (since that argument is deprecated).

Fixes #7755

Comments:
IndexError while applying stride for ASR
Author: IGWPark
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.16.2
- Platform: colab
- Python version: 3.7.12

## Information

Hi :)

`new_stride` from `rescale_stride()` can cause an error at `apply_stride()` when `token_n` and `left` have the same value. I got an error while running ASR through `pipeline() + chunk_length_s=10` with 15 secs of audio.

https://github.com/huggingface/transformers/blob/db7d6a80e82d66127b2a44b6e3382969fdc8b207/src/transformers/pipelines/automatic_speech_recognition.py#L69-L90

https://github.com/huggingface/transformers/blob/db7d6a80e82d66127b2a44b6e3382969fdc8b207/src/transformers/pipelines/automatic_speech_recognition.py#L92-L105

## To reproduce

Steps to reproduce the behavior:

```python
import numpy as np

from transformers import pipeline

mock_duration = 15
mock_sample_rate = 16_000
mock_audio = np.random.rand(mock_duration*mock_sample_rate,)

pipe = pipeline(model="facebook/wav2vec2-base-960h")

output = pipe(mock_audio, chunk_length_s=10)
```

```python
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
[<ipython-input-9-6be82957f842>](https://localhost:8080/#) in <module>()
      9 pipe = pipeline(model="facebook/wav2vec2-base-960h")
     10 
---> 11 output = pipe(mock_audio, chunk_length_s=10)

5 frames
[/usr/local/lib/python3.7/dist-packages/transformers/pipelines/automatic_speech_recognition.py](https://localhost:8080/#) in apply_stride(tokens, stride)
     98         # next letter, and last letter
     99 
--> 100         first_letter = tokens[i, left_token]
    101         tokens[i, :left_token] = first_letter
    102 

IndexError: index 83 is out of bounds for dimension 1 with size 83
```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
About `decoder_input_ids` in BART doc
Author: ShaneTian
Pull request: no

> For translation and summarization training, decoder_input_ids should be provided. If no decoder_input_ids is provided, the model will create this tensor by shifting the input_ids to the right for denoising pre-training following the paper.
https://huggingface.co/docs/transformers/v4.16.2/en/model_doc/bart#transformers.BartForConditionalGeneration.forward.decoder_input_ids

This is inappropriate. If `labels` is provided, the model will create this tensor by shifting the `labels`:
https://github.com/huggingface/transformers/blob/db7d6a80e82d66127b2a44b6e3382969fdc8b207/src/transformers/models/bart/modeling_bart.py#L1320-L1324

Comments:
Indeed, similar to #11357 and duplicate of #14328.

Can you open a PR to fix this? Otherwise I'll do it.

I have submit a PR to fix it. Can you review it?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
one of the variables needed for gradient computation has been modified by an inplace operation
Author: bjmajic
Pull request: no

i use  the model of rbt3 to run a pretrain model

self.model = AutoModel.from_pretrained("/workspace/wanglei/data_set/rbt3")
self.model.train()

nodes = self.model(**rnn_nodes_input_gpu).pooler_output

but got a error
"
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [287, 768]], which is output 0 of TanhBackward, is at version 2; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
"


Comments:
Hi could please a few more details, like what model are you using (bert, roberta etc) and code snippet so we can reproduce this ?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Wav2Vec2 for long audio with N-gram Language model
Author: Omarnabk
Pull request: no

I'm trying to use Wav2Vec2 for transcribing a long audio file following this blog post here (https://huggingface.co/blog/asr-chunking). 

The mentioned example is this :

```
from transformers import pipeline
pipe = pipeline(model="facebook/wav2vec2-base-960h")
output = pipe("very_long_file.mp3", chunk_length_s=10, stride_length_s=(4, 2))
```

The author said that it is possible to use N-GRAM LM but it is not clear to me as the pipeline produces only the "text", not the logits. Som my question is **how to let the pipeline produces logits instead of pure text**? 



Comments:
according to this (https://huggingface.co/blog/wav2vec2-with-ngram) and source for pipeline if the model has an N GRAM  LM as the decoder it should be used automatically used for decoding if pyctcdecode and kenlm are installed

As @Dharisd said, I will help you out with a little bit of code snippet. First thing to do is to install pyctcdecode and kenlm. Use:

`!pip install pyctcdecode`
`!pip install https://github.com/kpu/kenlm/archive/master.zip`

Next make the changes in the file _transformers/pipelines/automatic_speech_recognition.py_ . Add the following code in the line after 315.
`return logits` (This will return the logits instead of text when you use pipeline).

Finally, make sure you are using the N GRAM LM instead of _"facebook/wav2vec2-base-960h"_. The following snippet will give you the desired output:

```
from transformers import pipeline
pipe = pipeline(model="patrickvonplaten/wav2vec2-base-100h-with-lm")
output = pipe("very_long_file.mp3", chunk_length_s=10, stride_length_s=(4, 2))
```

Hope this helps! :D





This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
how can I download T5-11B pretrained model?
Author: shamanez
Pull request: no
Labels: name: wontfix

It gives me the following error. 

**OSError: Can't load weights for 't5-11b'. Make sure that:

- 't5-11b' is a correct model identifier listed on 'https://huggingface.co/models'

- or 't5-11b' is the correct path to a directory containing a file named one of pytorch_model.bin, tf_model.h5, model.ckpt.**


Comments:
I get an error when trying to use the hosted inference API too.


> ⚠️ Error loading model Can't load weights for 't5-11b'. Make sure that: - 't5-11b' is a correct model identifier listed on 'https://huggingface.co/models' - or 't5-11b' is the correct path to a directory containing a file named one of pytorch_model.bin, tf_model.h5, model.ckpt. OSError("Can't load weights for 't5-11b'. Make sure that:\n\n- 't5-11b' is a correct model identifier listed on 'https://huggingface.co/models'\n\n- or 't5-11b' is the correct path to a directory containing a file named one of pytorch_model.bin, tf_model.h5, model.ckpt.\n\n")


This is a known issue, and is due to the fact that the [checkpoint size is 42GB](https://huggingface.co/t5-11b#list-files) while the max supported file size for Cloudfront is [20GB](https://aws.amazon.com/blogs/aws/amazon-cloudfront-support-for-20-gb-objects/).

You should use the `use_cdn=False` flag in `AutoModel.from_pretrained(use_cdn=False)` to work around this for now.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Forgot to update this issue at the time, but note that this now works transparently

Hi. I am having trouble when downloading t5-11b model too but with bit different issue. I am keep getting 'connection rest by peer' error and failing to download the pretrained weights. Is there other way to download the model parameters?

Hi @nobellant215 you could try downloading the weights file directly: `https://huggingface.co/t5-11b/resolve/main/pytorch_model.bin`
question about the the specific text generation question
Author: xinli2008
Pull request: no

Hello, i am a tiro in transformers, and i have a question.
if i want to use serval concepts like:  I question english ,and i want to the output like this: i have a english question, or i have a question rather than english question.
Does any project in the hub can handle this? If not ,can you give me some useful advice about how to use the current model and modify it to finish my above mentioned question?
Thank you very much

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!
[Flax] Improve Robustness of Back-Prop Tests
Author: sanchit-gandhi
Pull request: yes

This PR makes several modifications to the `test_freeze_feature_encoder` tests for two Flax models, FlaxWav2Vec2 and FlaxSpeechEncoderDecoderModel. This test is in place to verify that feature encoder of the FlaxWav2Vec2 model is correctly frozen when the keyword argument `freeze_feature_encoder` is set to `True`. This is achieved through one dummy forward and backward pass, in which the losses and gradients are compared for the case when the feature encoder is frozen and the case when it is not.

The modifications made include:

- Implementing a check that asserts the outputs of the unfrozen and frozen feature encoder models are _precisely equal_;

- Asserting that the losses of the unfrozen and frozen models are _precisely equal_, instead of _almost equal_ as was previously;

- Asserting that the gradients of the unfrozen feature encoder layers contain at least one non-zero entry, instead of asserting that at least one entry that surpasses a specified tolerance;

- Asserting that the gradients of all unfrozen layers remain _precisely equal_, instead of _almost equal_ as was previously.

The statements that assert unfrozen and frozen gradient values are _precisely equal_ instead of _almost equal_ provides more rigour to the back-propagation test. It verifies that `jax.stop_gradient` has no impact on the forward or backward pass, other than skipping computation of the gradients of any frozen layers. **Crucially**, these changes bypass the need to set a specified tolerance (`tol`) in the `assert_almost_equal` or `assert_difference` functions. Even with seemingly appropriate values for this tolerance (`1e-5`), the tests were prone to failing a small proportion of the time (<2.5%). With this PR, the tests should now pass 100% of the time.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Providing a easier way to download pre-trained models.
Author: zhaowei-wang-nlp
Pull request: no

# 🚀 Feature request
I tried to download T5-base manually because of my special environment. I tried to git clone T5-base from https://huggingface.co/t5-base. The model in PyTorch (which I am using) is only about 800M. However, I need to download files used in other DL frameworks (like TensorFlow). Also, I need to download the git files (.git directory). Finally, I need to download about 7G files, which is much larger than the model itself.

So, could there be a way to manually download pre-trained models in a specific DL framework only?

## Motivation
I need to download much more files than the model itself, which is too slow

## Your contribution


Comments:
Hello! I recommend using `snapshot_download`, from the `huggingface_hub` package:
```
from huggingface_hub import snapshot_download
```

See https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/snapshot_download.py#L21, especially the `allow_regex` and `ignore_regex` which should allow you to specify which files exactly you'd like to get.

@LysandreJik is 100% right, or an alternative if it's just a one-off download is you could just download individual files from the `Files and versions` tab (note that depending on your browser you might need to rename the files):

<img width="1833" alt="Screenshot 2022-03-28 at 12 04 16" src="https://user-images.githubusercontent.com/326577/160375553-4f920b72-8644-4300-a0b6-d782b0bafd91.png">


Thank you guys a lot!
Text2Speech classes
Author: avacaondata
Pull request: no

Hey, it would be very cool if there were classes already developed for carrying out Text2Speech tasks. We do have many classes capable of doing Speech2Text, but for the opposite task there are no transformer classes available, although there are starting to be models in the hub: https://huggingface.co/facebook/tts_transformer-es-css10

It would be great if instead of using fairseq for loading those models, we could load them and fine-tune them further with Transformers. 

Comments:
Pinging @anton-l for knowledge :)

Hi @alexvaca0, thanks for your interest!
We're currently working on implementing FastSpeech2 natively: https://github.com/huggingface/transformers/pull/15773
And looking into FastPitch as well: https://github.com/huggingface/transformers/issues/16349

Feel free to subscribe to those PRs/issues to follow the progress :)
Fix Respect output device in the Pipeline base impl.
Author: prassanna-ravishankar
Pull request: yes

changes hardcoded `"cpu"` device in return in the forward method to `self.device`

# What does this PR do?
The base class in pipeline surprisingly returns a tensor on the right device, and this is then moved to the "cpu" for some reason. This PR makes sure that the input and the output to the `forward(..)` method remain on the same device. 

Fixes #16315 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)?  https://github.com/huggingface/transformers/issues/16315
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@LysandreJik


Comments:
_The documentation is not available anymore as the PR was closed or merged._

1/ Is my fix correct? Was there any other reason to keep this on CPU?
2/ Do I need to write any tests?
3/ Could someone rerun the Model templates runner / run_tests_templates (pull_request) task for me?

cc @Narsil 

@LysandreJik The discussion happened in the original issue: https://github.com/huggingface/transformers/issues/16315
[FlaxSpeechEncoderDecoderModel] Ensure Input and Output Word Embeddings Are **Not** Tied
Author: sanchit-gandhi
Pull request: yes

In a seq2seq Speech-Encoder-Text-Decoder Model, the word embeddings of the text decoder should  **not** be tied to the word embeddings of the speech encoder. The embedding matrices lie in two completely different vector spaces that have no affiliation. The embedding matrix for the decoder should be specific to the `XXXForCausalLM` decoder model used. Thus, the encoder and decoder word embeddings should not be constrained to being equal. This is observed in the PyTorch script for the `SpeechEncoderDecoderModel`:

https://github.com/huggingface/transformers/blob/e02f95b2298997016cd01fdb182442093b34e8d2/src/transformers/models/speech_encoder_decoder/modeling_speech_encoder_decoder.py#L437-L438

This PR makes adds the same change to the `config` for the `FlaxSpeechEncoderDecoderModel`, as well as adding tests to ensure that the `tie_word_embeddings` config is set correctly for both the `SpeechEncoderDecoderModel` and `FlaxSpeechEncoderDecoderModel`.



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Is there a possible way to moniter the realtime training  in the example?
Author: SadGare
Pull request: no

In the `transformers/examples/pytorch/translation/run_translation.py`, it seems it only produce `the runs` log file after stopping training. Is it possible to generate the runs log while training through certain steps in that script? I fail to see any readme talk about it. Any advice will be appreciated.

Comments:
Hi, please consider using the forum to ask such general questions as we use issues for for bug reports and feature requests, thanks!

The `Trainer` should automatically log during training, you can control the logging using `--logging_strategy` and `--logging_steps` argument. You can also use TensorBoard or w&B for logging, refer to this [doc](https://github.com/huggingface/transformers/tree/main/examples/pytorch#logging--experiment-tracking).

Thank you, I will use forum in the future.😊
Translation from english to spanish of file pipeline_tutorial.mdx
Author: FernandoLpz
Pull request: yes

## Translation from English to Spanish of file pipeline_tutorial.mdx

Fixes #15947 
@omarespejel

I added the translation of the `pipeline_tutorial.mdx` file

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks @FernandoLpz! Will review it :) 

Your welcome!

I just fix the typos. 

@omarespejel 

Hi @sgugger! It seems to me that the failed checks are due to something else and not this doc. WDYT?
Remove kwargs argument from IBERT MLM forward pass
Author: lewtun
Pull request: yes

# What does this PR do?

This PR fixes a small bug that was introduced in #16389 where a `kwargs` argument was added to the `forward` pass of the MLM IBERT class.

Once this is merged, the slow ONNX test for this model should also pass. 


Comments:
_The documentation is not available anymore as the PR was closed or merged._