noamaanMulla-03/transformers-issues huggingface dataset:
Fix `AlignModelTest` tests
Author: ydshieh
Pull request: yes

# What does this PR do?

- Fix `AlignModelTest` torchscript tests.  This model has non-persistent buffer, and need a bit more care (same as in common test)
```python
        self.register_buffer(
            "token_type_ids", torch.zeros(self.position_ids.size(), dtype=torch.long), persistent=False
        )
```

- Fix `AlignModelTest.test_multi_gpu_data_parallel_forward`: same as CLIP, we need a even number for `batch_size`, as this model have `logits_per_image` and `logits_per_text` which is of shape `(batch_size, batch_size)`, and in order to gather across devices, we need the second dim being the same. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Avoid failure in `check_repo.py` due to missing backends
Author: ydshieh
Pull request: yes

# What does this PR do?

My 2 PRs #21903 and #21909 forgot to check missing backends just like `check_all_models_are_auto_configured`, and it could fail on users env where some backends are missinig.

This PR fixes this by checking missing backends. Sorry about that.

**Remark**

```
from transformers.models.auto.modeling_flax_auto import FLAX_MODEL_MAPPING_NAMES
```
this worked even without backend jax/flax. That's why I missed.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add FLAN-UL2
Author: shermansiu
Pull request: no
Labels: name: New model

### Model description

UL2 is a unified framework for pretraining models that are universally effective across datasets and setups. UL2 uses Mixture-of-Denoisers (MoD), a pre-training objective that combines diverse pre-training paradigms together. UL2 introduces a notion of mode switching, wherein downstream fine-tuning is associated with specific pre-training schemes.

FLAN-UL2 has the same configuration as the original UL2 20B model, except that it has been instruction tuned with Flan. 

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

The model architecture (UL2) is already in Huggingface Transformers.
The 20B model weights are here: https://github.com/google-research/google-research/tree/master/ul2#checkpoints

Comments:
@DanielHesslow (since you ported the original UL2 weights). I would like to contribute, but I'm not too sure how to convert the weights from JAX to PyTorch.

I had a dirty dirty script which unfortunately lives on my old dev machine that I don't have with me at the moment 😅 

I basically just loaded the t5 weights and went through and renamed every thing to match the HF format. 

Hey! Thanks for opening, they will be available on the hub soon! We are converting them with @younesbelkada 

The model is already out! (https://huggingface.co/google/flan-ul2)
@younesbelkada has a space comparing Flan-T5-XXL and Flan-UL2 here: https://huggingface.co/spaces/ybelkada/i-like-flan-ul2
[Flan-UL2] Add-flan-ul2
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Adds the documentation for the FLan-UL2 model
cc @younesbelkada 
Fixes #21917

Comments:
Also, why are the international README entries in the wrong language?

This should not be the case! Will fix this. 

@sgugger it seems that the README translations are not correct for several languages (korean, jp, etc) due to the fact that they do not follow the correct structure ("released with the repository", "released with the blogpost", ..), As this requires a bit of work I think that we can address a solution in a follow-up PR! Wdyt?

Btw, I opened a new issue for the README translations to keep things nice and organized (#21932).

_The documentation is not available anymore as the PR was closed or merged._

Thanks a lot!
Fix wrong documentation about DataCollator padding defaults
Author: substanc3-dev
Pull request: yes

# What does this PR do?

Fixes some documation for DataCollatorForTokenClassification and DataCollatorForSeq2Seq, which previously mentioned no padding is the default, which is false.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR (@sgugger).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[CLAP] Support batched inputs for CLAP. Fixes pipeline issues
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Support batching the `is_longer` part of the input for `zero_shot_audio_classification`. The model was not able to run batched inputs. Added a test to support this. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Just need to update the expected values for the doctest and will merge

Pipeline, CI and doctests are all green 😉
Fixed gradient_checkpointing/use_cache bug in blenderbot
Author: Batese2001
Pull request: yes

# What does this PR do?

Fixes #21737 for blenderbot

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@younesbelkada

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@younesbelkada Thanks for your help! It should be all good
HF <-> megatron checkpoint reshaping and conversion for GPT
Author: pacman100
Pull request: yes

# What does this PR do? 
With respect to GPT model,
1. Now, user would be able to convert Megatron-LM GPT model having different tensor parallel and pipeline parallel sizes to a universal transformers checkpoint for `gpt2` model. This checkpoint is also sharded by default (10GB per shard) or the value which user provides. Sample command to run to convert from Megatron-LM to Transformers checkpoint (below command is run for a checkpoint having tp-size 2 and pp-size 1):
```
python checkpoint_reshaping_and_interoperability.py \
--convert_checkpoint_from_megatron_to_transformers \
--load_path "megatron_lm_gpt/iter_0005000" \
--save_path "hf_checkpoint" \
--max_shard_size "200MB" \
--tokenizer_name "/home/sourab/code-parrot-minimal" \
--print-checkpoint-structure
```
Ouput logs: https://gist.github.com/pacman100/eedce29f084f3efdac76456bd407f978#file-megatron_to_trfs-log

2. Reverse conversion from transformers checkpoint to Megatron checkpoint with variable TP and PP sizes is supported. Sample command for it is given below (converting to a checkpoint with `target_tensor_model_parallel_size`=2 and `target_pipeline_model_parallel_size`=1. `target_data_parallel_size` is used when `--use_distribured_optimizer` is passed):
```
python checkpoint_reshaping_and_interoperability.py \
--load_path "hf_checkpoint" \
--save_path "megatron_lm_checkpoint" \
--target_tensor_model_parallel_size 2 \
--target_pipeline_model_parallel_size 1 \
--target_data_parallel_size 2 \
--target_params_dtype "bf16" \
--make_vocab_size_divisible_by 128 \
--print-checkpoint-structure 
```
Output logs: https://gist.github.com/pacman100/eedce29f084f3efdac76456bd407f978#file-trfs_to_megatron-log

A quick test to make sure everything is working properly:
Code as well as output logs: https://gist.github.com/pacman100/eedce29f084f3efdac76456bd407f978#file-gistfile1-txt


Comments:
_The documentation is not available anymore as the PR was closed or merged._

I can't use this from  transformers to megatron  for accelerate-megatron-plugin to continue finetune the checkpoint. is there any thing i missing? then load the converted checkpoint with resume_from_checkpoint args, it seems that the gpus are reinit and only one gpu is 100% utility. 

Hello, thank you for the converting tools.
By the way, is there any plan that the script of T5 HF model <-> Megatron model will be supported and when?
Unable to use GPU during wav2vec2 decoding 
Author: manjuke
Pull request: no

### System Info

Hi All,
I have built a finetuned model for Tamil using facebook/wav2vec2-xls-r-300m. I could do inferencing successfully using CPU. However, Wav2vec2 decoding (with pyctcdecoder) using GPU is not working. I have tried enabling device='gpu' in the decoding script, and also running the decoding script as “python -m torch.distributed.launch --nproc_per_node=<num of GPUs> <Decoding_Script.py>”. But, on monitoring nvidia-smi output during decoding, none of these methods are using GPU for decoding. Pls suggest @sanchit-gandhi . Thanks

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Use 'cuda' as device for decoding

        audio_name=pd.DataFrame(test_data).audio[i]
        text_org = pd.DataFrame(test_data).text[i]
        audio_input, sample_rate = sf.read(audio_name)
        
        #withLM
        input_values = processor_with_lm(audio_input, sampling_rate=16000, return_tensors="pt").input_values
        logits = model(input_values).logits
        hypothesis = processor_with_lm.batch_decode(logits.detach().numpy()).text
        text_with_lm = hypothesis[0]
        #withoutLM
        input_values_wo = processor(audio_input, sampling_rate=16000, return_tensors="pt").input_values
        logits_wo = model(input_values_wo).logits
        predicted_ids = torch.argmax(logits_wo, dim=-1)
        hypothesis_wo_lm = processor.decode(predicted_ids[0])
        text_wo_lm = hypothesis_wo_lm.replace('[PAD]','')
        


### Expected behavior

GPU decoding should have happened and nvidia-smi output to be shown accordingly.

Comments:
`pyctcdecode` doesn't support GPU

Indeed, `pyctcdecode` is a CPU only decoding method. PyTorch recently released a fast beam search decoder with a Flashlight backend: https://pytorch.org/blog/fast-beam-search-decoding-in-pytorch-with-torchaudio-and-flashlight-text/

We could look at integrating this into transformers for faster CTC + LM decoding!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
pip install error
Author: devjwsong
Pull request: no

Hi, I tried to install transformers library via `pip install transformers` and I got tokenizer install error.

The error logs are as follows.
```
ERROR: Could not find a version that satisfies the requirement tokenizers==0.8.1.rc1 (from transformers) (from versions: 0.0.2, 0.0.3, 0.0.4, 0.0.5, 0.0.6, 0.0.7, 0.0.8, 0.0.9, 0.0.10, 0.0.11, 0.0.12, 0.0.13, 0.1.0, 0.1.1, 0.2.0, 0.2.1, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.5.0, 0.5.1, 0.5.2, 0.6.0, 0.7.0, 0.8.0)
ERROR: No matching distribution found for tokenizers==0.8.1.rc1 (from transformers)
```

I googled about it but I couldn't find the way to solve it.
Is there I can do to handle this issue?

Thank you.

Comments:
@devJWSong, how did you solve this?

Oh, actually I didn't solve it.
I just installed downgraded version which is 2.11.0. and it worked.
I still don't know the reason but I think it is the problem from my virtual environment setting since when I tried to install the recent version in the different environment, it worked...

its error occurs to me too.... could  you give me another solution about that problems? ...

Any updates on this problem?

Try changing index-url and trusted-host in pip config.
I had same issue with the environment with index-url='http://ftp.daumkakao.com/pypi/simple' and trusted-host='ftp.daumkakao.com', but everything worked well with the environment without such config.

+)
try `pip install transformers -i https://pypi.python.org/simple`

Thank you! It worked. :)
T5 model for classification doesn't work properly for large number of classes.
Author: HiteshVamshi
Pull request: no
Labels: name: wontfix

T5 model works properly for 50 class classification but when we try for 100 classes it outputs empty string ("") for a large number of test data. Is this model suitable for multi-class classification with a large number of classes? If yes, what do you think might be the problem with what I am doing. 

Comments:
Hi, @HiteshVamshi , what you are trying is highly experimental, I haven't seen anyone using T5 for 100 class classification. So you'll probably need to experiment with it.

I would like to know few more details
1) What is the size of your dataset
2) which version of t5 are you using (t5-small, t5-base, t5-large etc)
3) how many epochs  

Hi, @patil-suraj , I was also experimenting with the T5 model. I got a good performance for 50 classes so tried with more.
-The size of the dataset used was 65k.
-I used T5-small.
-I tested for 10 epochs. But there was no significant improvement after 3 epochs.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Can we use the T5 model with only 2000 samples for classifications (not a lot of classes, just around 10)? what about binary classification?
about the opt model KeyError: 'opt'
Author: DericZhao
Pull request: no
Labels: name: bug

### System Info

```shell
I just copy the code from the model card on the https://huggingface.co/facebook/opt-30b, then there is an error, can you help me to find what's the problem of it?

Traceback (most recent call last):
  File "D:\SentenceRewrite\opt.py", line 4, in <module>
    model = AutoModelForCausalLM.from_pretrained("facebook/opt-30b", torch_dtype=torch.float16).cuda()
  File "D:\Anaconda\envs\torch\lib\site-packages\transformers\models\auto\auto_factory.py", line 382, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
  File "D:\Anaconda\envs\torch\lib\site-packages\transformers\models\auto\configuration_auto.py", line 517, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
  File "D:\Anaconda\envs\torch\lib\site-packages\transformers\models\auto\configuration_auto.py", line 266, in __getitem__
    raise KeyError(key)
KeyError: 'opt'
```


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained("facebook/opt-30b", torch_dtype=torch.float16).cuda()

# the fast tokenizer currently does not work correctly
tokenizer = AutoTokenizer.from_pretrained("facebook/opt-30b", use_fast=False)

prompt = "Hello, I'm am conscious and"


input_ids = tokenizer(prompt, return_tensors="pt").input_ids.cuda()

generated_ids = model.generate(input_ids)

tokenizer.batch_decode(generated_ids, skip_special_tokens=True)


### Expected behavior

```shell
KeyError: 'opt' how to solve it?
```


Comments:
I met the same problem. Have you fixed this?

> 

You can download the latest version of transformers, have a good day.

@DericZhao  I installed the latest version, I am still getting the error. Any help?

I met the same problem. Any solutions?

me too!my transformers version is 4.4.1,I still have the problem.

`pip install --upgrade transformers`

Check the versions of python(>=3.8 required) and pytorch , they may not match transformers
Update expected values in `XLMProphetNetModelIntegrationTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

After #21870, we also need to update some expected values in `XLMProphetNetModelIntegrationTest` (as has been done for `ProphetNetModelIntegrationTest` in that PR)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasSgemmStridedBatched (handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)
Author: phosseini
Pull request: no

## Environment info

NVIDIA-SMI 495.46
Driver Version: 460.32.03
CUDA Version: 11.2   

- `transformers` version: 4.16.2
- Platform: Google Colab Pro +
- Python version: 3.7
- PyTorch version (GPU?): 1.10.0+cu111
- Using GPU in script?: Yes

Error:

```
Saving model checkpoint to ../models/checkpoints/checkpoint-287
Configuration saved in ../models/checkpoints/checkpoint-287/config.json
Model weights saved in ../models/checkpoints/checkpoint-287/pytorch_model.bin
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-3-f7f41780dc73> in <module>()
    157 )
    158 
--> 159 trainer.train()
    160 
    161 trainer.save_model()

14 frames
/usr/local/lib/python3.7/dist-packages/transformers/models/roberta/modeling_roberta.py in forward(self, hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask, past_key_value, output_attentions)
    273             attention_probs = attention_probs * head_mask
    274 
--> 275         context_layer = torch.matmul(attention_probs, value_layer)
    276 
    277         context_layer = context_layer.permute(0, 2, 1, 3).contiguous()

RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)`
```

Models:
- RoBERTa

Library:
- Trainer: @sgugger

## To reproduce

Steps to reproduce the behavior:
 I'm calling the trainer with `trainer.train()` and training a model for 10 epochs. I'm %99 percent sure I don't have a memory issue since I'm using a high ram node. The model successfully gets trained for one epoch and checkpoints are saved (as seen in the log,) but as soon as the second epoch starts, I see the error.

## Expected behavior
Completing the train without error!

P.S. Just to add, the very script in the very environment with the same GPU works perfectly fine with `bert-large-*` models. So I wonder if this error could have anything to do with the RoBERTa model?

Comments:
With no information on how to reproduce the error, there is little we can do to help.

> With no information on how to reproduce the error, there is little we can do to help.

Thanks for your response. Unfortunately, the data needed for reproducibility is proprietory and not public, so I can't share it for reproducibility. I thought this might be a common or known issue.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I meet similar problem on an A100 GPU with 11.7 cuda.
Add SpikeGPT model
Author: gsarti
Pull request: no
Labels: name: New model

### Model description

**Abstract:**
>As the size of large language models continue to scale, so does the computational resources required to run it. Spiking neural networks (SNNs) have emerged as an energy-efficient approach to deep learning that leverage sparse and event-driven activations to reduce the computational overhead associated with model inference. While they have become competitive with non-spiking models on many computer vision tasks, SNNs have also proven to be more challenging to train. As a result, their performance lags behind modern deep learning, and we are yet to see the effectiveness of SNNs in language generation. In this paper, inspired by the RWKV language model, we successfully implement `SpikeGPT', a generative language model with pure binary, event-driven spiking activation units. We train the proposed model on three model variants: 45M, 125M and 260M parameters. To the best of our knowledge, this is 4x larger than any functional backprop-trained SNN to date. We achieve this by modifying the transformer block to replace multi-head self attention to reduce quadratic computational complexity to linear with increasing sequence length. Input tokens are instead streamed in sequentially to our attention mechanism (as with typical SNNs). Our preliminary experiments show that SpikeGPT remains competitive with non-spiking models on tested benchmarks, while maintaining 5x less energy consumption when processed on neuromorphic hardware that can leverage sparse, event-driven activations.

Concretely, it is a GPT model using Receptance Weighted Key Value (RWKV) instead of regular attention, and an adapted FFN layer.

### Open source status

- [X] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

[Paper](https://arxiv.org/abs/2302.13939) | [Code](https://github.com/ridgerchu/SpikeGPT)

Author: @ridgerchu

Comments:
Thanks for your interest to our work! The checkpoint weights of 120M spike GPT has available [now](https://huggingface.co/ridger/SpikeGPT-BookCorpus/blob/main/BookCorpus-SpikeGPT.pth), but just for debug and playing with the model.

I've read the paper, this model looks really cool 👍
[CI] Fix ci 
Author: ArthurZucker
Pull request: yes

# What does this PR do?
A typo made during the full cleanup of the code was making 2 test fail. 
2 other slow tests are failing, but they were also failing prior to #20211 .

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> 2 other slow tests are failing, but they were also failing prior to #20211 .

Could you share the names of these 2 tests.
I can check on CI runners. I did it once on Friday for one test you mentioned, but let's make sure.


There is :
- `tests/models/deformable_detr/test_modeling_deformable_detr.py::DeformableDetrModelIntegrationTests::test_inference_object_detection_head_with_box_refine_two_stage`
- `tests/models/deformable_detr/test_modeling_deformable_detr.py::DeformableDetrModelIntegrationTests::test_inference_object_detection_head_equivalence_cpu_gpu`
- `tests/models/deformable_detr/test_modeling_deformable_detr.py::DeformableDetrModelIntegrationTests::test_inference_object_detection_head`
Ran them on the CI runner an they are all green so LGTM.