noamaanMulla-03/transformers-issues huggingface dataset:
Error when loading a model cloned without git-lfs is quite cryptic
Author: thomwolf
Pull request: no
Labels: name: Feature request

# 🚀 Error message request

If you forget to install git-LFS (e.g. on Google Colab) and you just do:

```python
!git clone https://huggingface.co/facebook/bart-base
from transformers import AutoModel
model = AutoModel.from_pretrained('./bart-base')
```

The cloning seems to work well but the model weights are not downloaded. The error message is then quite cryptic and could probably be tailored to this (probably) common failure case:

```
loading weights file ./bart-large-cnn/pytorch_model.bin
---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    950             try:
--> 951                 state_dict = torch.load(resolved_archive_file, map_location="cpu")
    952             except Exception:

4 frames
UnpicklingError: invalid load key, 'v'.

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    952             except Exception:
    953                 raise OSError(
--> 954                     f"Unable to load weights from pytorch checkpoint file for '{pretrained_model_name_or_path}' "
    955                     f"at '{resolved_archive_file}'"
    956                     "If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. "

OSError: Unable to load weights from pytorch checkpoint file for './bart-large-cnn' at './bart-large-cnn/pytorch_model.bin'If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. 
```

git-LFS specification files are pretty simple to parse and typically look like this:
```
version https://git-lfs.github.com/spec/v1
oid sha256:097417381d6c7230bd9e3557456d726de6e83245ec8b24f529f60198a67b203a
size 440473133
```

The first *key* is always `version`: https://github.com/git-lfs/git-lfs/blob/master/docs/spec.md

Comments:
Yep, they way I would go about this would be to programmatically check whether the file is text-only (non-binary) and between 100 and 200 bytes. If it is (and we expected a weights file), it's probably a lfs pointer file.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


unstale
Better error raised when cloned without lfs
Author: LysandreJik
Pull request: yes

Raise a better error when cloning a repository without having git-lfs installed.

The pointer files contain data such as the following:

```
version https://git-lfs.github.com/spec/v1
oid sha256:c393f632913cdc64c13fcd1b039a74f17dd83cc3029c556802c0f2f8792b46f9
size 557941479
```

This tries to parse the file and check that the first string is `version`; if so, it detects it as a pointer file.

Closes https://github.com/huggingface/transformers/issues/8497

Comments:
Fix Tensorflow T5 with int64 input
Author: Rocketknight1
Pull request: yes

Our Tensorflow T5 model uses constant values that are initialized with `tf.constant()` or just from Python int literals. Tensorflow guesses the dtype in both of these cases as `int32`, which causes the operations using them to fail if `int64` input is supplied.

This PR adds checks that cast the constants to the dtype of the inputs, to ensure that the operations succeed for both `int32` and `int64` inputs.

Comments:
Using `model.sample()` and increasing the `max_length` leads to CUDA OOM crash
Author: moinedgylabs
Pull request: no

## Environment info

- `transformers` version: 4.6.0.dev0
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.11
- PyTorch version (GPU?): 1.9.0+cu102 (True)
- Tensorflow version (GPU?): 2.5.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

I think @patrickvonplaten and @LysandreJik

## Information

Model I am using GPT-NEO:

The problem arises when using:
* my own modified scripts: Simply using `.sample()` method with `GPTNeoForCausalLM`

The tasks I am working on is:

* my own task: Simple Text generation.

## To reproduce

Steps to reproduce the behavior:

1. Visit this colab and run GPU runtime: https://colab.research.google.com/drive/1VjVUrptwgUx3TxdlcVqPNwXyX6YJYSAK
2. Execute Runtime -> Run All
3. Note the `nvidia-smi` output.
4. In the last cell, increase the `max_length` from 30 to 350. And run again.
4. Even if the crash doesn't occur, check `nvidia-smi` again.

## Expected behavior

To not consume that much GPU memory. It is understood that some tasks may take some memory, but using **~10 GB** for an increase of 300 tokens is... something wrong.

Comments:
Related to https://github.com/huggingface/transformers/issues/11320

Cc @patil-suraj 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
huggingface-hub version conflict
Author: CraigMiloRogers
Pull request: no

`transformers` requires `huggingface-hub==0.0.12` in `setup.py` in the `master` branch. 

`huggingface-hub` has just released version `0.0.15`. 

Other projects that use `huggingface-hub`, such as `https://github.com/UKPLab/sentence-transformers`, are happy with the latest version of `huggingface-hub`.

Depending upon the order of installation, a version conflict may result.  Here's a sample message from our project, `kgtk`, which requires both `transformers` and `sentence-transformers`:

```pkg_resources.ContextualVersionConflict: (huggingface-hub 0.0.15 (/opt/anaconda3/envs/kgtk-env/lib/python3.8/site-packages/huggingface_hub-0.0.15-py3.8.egg), Requirement.parse('huggingface-hub==0.0.12'), {'transformers'})```


Comments:
This is discussed in #12961 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fixed the MultilabelTrainer document, which would cause a potential bug when executing the code originally documented.
Author: Mohan-Zhang-u
Pull request: yes

if train with the MultilabelTrainer documented in the original document 

```
from torch import nn
from transformers import Trainer

class MultilabelTrainer(Trainer):
    def compute_loss(self, model, inputs, return_outputs=False):
        labels = inputs.pop("labels")
        outputs = model(**inputs)
        logits = outputs.logits
        loss_fct = nn.BCEWithLogitsLoss()
        loss = loss_fct(logits.view(-1, self.model.config.num_labels),
                        labels.float().view(-1, self.model.config.num_labels))
        return (loss, outputs) if return_outputs else loss
```

a bug that looks like this would appear:
`
  File "~/anaconda3/lib/python3.8/site-packages/transformers/trainer.py", in train
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
  File "~/anaconda3/lib/python3.8/site-packages/transformers/trainer.py", in _maybe_log_save_evaluate
    metrics = self.evaluate()
  File "~/anaconda3/lib/python3.8/site-packages/transformers/trainer.py", in evaluate
    output = self.prediction_loop(
  File "~/anaconda3/lib/python3.8/site-packages/transformers/trainer.py", in prediction_loop
    loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only, ignore_keys=ignore_keys)
  File "~/anaconda3/lib/python3.8/site-packages/transformers/trainer.py", in prediction_step
    labels = nested_detach(tuple(inputs.get(name) for name in self.label_names))
  File "~/anaconda3/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", in nested_detach
    return type(tensors)(nested_detach(t) for t in tensors)
  File "~/anaconda3/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", in <genexpr>
    return type(tensors)(nested_detach(t) for t in tensors)
  File "~/anaconda3/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", in nested_detach
    return tensors.detach()
AttributeError: 'NoneType' object has no attribute 'detach'
`

change the original code to below would effectively avoid this bug.
```
from torch import nn
from transformers import Trainer

class MultilabelTrainer(Trainer):
    def compute_loss(self, model, inputs, return_outputs=False):
        labels = inputs.get("labels")
        outputs = model(**inputs)
        logits = outputs.get('logits')
        loss_fct = nn.BCEWithLogitsLoss()
        loss = loss_fct(logits.view(-1, self.model.config.num_labels),
                        labels.float().view(-1, self.model.config.num_labels))
        return (loss, outputs) if return_outputs else loss
```


Comments:
Fix integration tests for `TFWav2Vec2` and `TFHubert`
Author: anton-l
Pull request: yes

**Issue**
Errors for integration tests on `TFWav2Vec2` and `TFHubert`
```
E           ValueError: When setting ``truncation=True``, make sure that ``max_length`` is defined and ``padding='max_length'
```
CI logs: https://github.com/huggingface/transformers/runs/3539314355?check_suite_focus=true

**Solution**
Remove `truncation=True` from feature extractors since the logic changed recently https://github.com/huggingface/transformers/pull/12804/
Also some other small fixes because the tests now reach those lines :slightly_smiling_face: 

Comments:
Fix scheduled tests for `SpeechEncoderDecoderModel`
Author: anton-l
Pull request: yes

```
FAILED tests/test_modeling_speech_encoder_decoder.py::Wav2Vec2BertModelTest::test_real_model_save_load_from_pretrained
FAILED tests/test_modeling_speech_encoder_decoder.py::Speech2TextBertModelTest::test_real_model_save_load_from_pretrained
FAILED tests/test_modeling_speech_encoder_decoder.py::Wav2Vec2Speech2Text2::test_real_model_save_load_from_pretrained
```

**Problem**
`test_real_model_save_load_from_pretrained` was using a missing `self.get_inputs()` and failing for all three flavors of encoder-decoder models. CI logs: https://github.com/huggingface/transformers/actions/runs/1199765292

**Solution**
Pass pretrained models together with suitably-shaped inputs.

Comments:
Add the `AudioClassificationPipeline`
Author: anton-l
Pull request: yes

# What does this PR do?

This adds the audio classification pipeline needed for `Wav2Vec2ForSequenceClassification` and others (see #13153).

The implementation is mostly based on `ImageClassificationPipeline` with `ffmpeg` audio file loading borrowed from `AutomaticSpeechRecognitionPipeline` by @Narsil 

Once merged, model cards like https://hf.co/superb/hubert-base-superb-ks should be able to have an `audio-classification` inference widget.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@patrickvonplaten @Narsil 

Comments:
Add Wav2Vec2 & Hubert ForSequenceClassification
Author: anton-l
Pull request: yes

# What does this PR do?

This adds a Hubert extension for sequence classification.
Ultimately this classification head should be compatible with s3prl `UtteranceLevel` [implementation](https://github.com/s3prl/s3prl/blob/master/s3prl/downstream/model.py#L35) to support classification tasks from SUPERB, such as [Keyword Spotting](https://huggingface.co/datasets/superb#ks) and transfer their pretrained models.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@patrickvonplaten @patil-suraj 


Comments:
Accuracy evaluation on SUPERB tasks:

- **KS** has uniform-length samples, so no padding
- **ER** has non-uniform padded batches 
- **SID** is evaluated with batch_size=1 as in `s3prl`

| Task | Model         | normalize=True | normalize=False | Paper  |
| ---- | ------------- | -------------- | --------------- | ------ |
| **KS**   | Wav2Vec2-base | 0.9627         | 0.9643          | 0.9623 |
|      | Hubert-base   | 0.9669         | 0.9672          | 0.9630  |
| **ER**   | Wav2Vec2-base | 0.5281         | 0.6258          | 0.6343 |
|      | Hubert-base   | 0.5502         | 0.6359          | 0.6492 |
| **SID**  | Wav2Vec2-base | 0.7360         | 0.7518          | 0.7518 |
|      | Hubert-base   | 0.8071         | 0.8071          | 0.8142 |

So far `normalize=False` is always better, as expected (`s3prl` never used normalization during eval).
There's also some slight variation with the official results, but it's of the same magnitude as `s3prl` vs `paper` results. 

- [x] Passed integration test for all 4 tasks on both models
- [x] Added `Copied from` where possible (the script just inserts a full copy of `W2V2.forward()` before `End copy`, so I didn't use it there)
- [x] Added dummy examples to `forward()` docs
- [x] Moved the models to `https://huggingface.co/superb`

@patrickvonplaten everything should be ready to merge now :) 

Awesome job @anton-l ! Feel free to merge the PR whenever you want
[Errno 13] Permission denied: '/.cache'
Author: RachelDong
Pull request: no

Hello,

I'm using the Dialo-GPT model:
```
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "microsoft/DialoGPT-small"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
```
When I run the above code in server, the error throws:
```
[Errno 13] Permission denied: '/.cache'
```
Does anyone know how to solve this problem? Thanks a lot.

Comments:
It seems like the `/.cache` directory is not writable, you could try changing permissions or you could tell transformers to use a different dir for cache by setting the env variable `HF_HOME`


Yes thank you for the answer:)
MarianMT int dtype fix
Author: Rocketknight1
Pull request: yes

Found the same integer dtype issue in MarianMT as in T5 - I'll probably need to find a proper way of dealing with this across the codebase at some point, but this'll do to get the notebook examples working at least.

Comments:
Update: I found multiple Seq2Seq models were copying the problematic `shift_tokens_right` method from BART, so I fixed the BART method and ran `make fix-copies`. That should hopefully fix this issue for a lot of our models.
Add m2m100
Author: patil-suraj
Pull request: yes
Labels: name: PR for Model Addition

# What does this PR do?

Adds the M2M100 model
https://github.com/pytorch/fairseq/tree/master/examples/m2m_100

Fixes #8054

Comments:
Sure, Patrick !

I’ve addressed all the review comments, and all the slow/fast tests are now passing.

I didn’t add fast tokenizer because `M2M100` is `sentencepiece`  based tokenizer, but it uses `sentencepiece`  for just tokenizing and then uses a vocab file to convert the tokens to ids and ids to tokens. So our current `SpmConverter` doesn’t work for this. I’ll try to add fast tokenizer in a follow-up PR.

Merging!

> I’ve addressed all the review comments, and all the slow/fast tests are now passing.
> 
> I didn’t add fast tokenizer because `M2M100` is `sentencepiece` based tokenizer, but it uses `sentencepiece` for just tokenizing and then uses a vocab file to convert the tokens to ids and ids to tokens. So our current `SpmConverter` doesn’t work for this. I’ll try to add fast tokenizer in a follow-up PR.
> 
> Merging!

Hey, I was wondering if there's any progress on a Fast Tokenizer for M2M or if any help can be needed?
Thanks :)
Refactor internals for Trainer push_to_hub
Author: sgugger
Pull request: yes

# What does this PR do?

This PR refactors the internals of the `Trainer` integration with the hub to use a `Repository` object instead of relying on the `PushToHubMixin` (which will get simplified drastically soon, see https://github.com/huggingface/huggingface_hub/pull/321).

In passing, to get closer to the way `Repository` works, the `push_to_hub_model_id` and `push_to_hub_organization` are both deprecated in favor of `hub_model_id`, which should contain the full ID of the model (either username/model_name or organization/model_name). `push_to_hub_token` is deprecated in favor of `hub_token`, which is easier.

Tests are adapted to use those new arg names, but otherwise there is no breaking change.

Comments:
add non auto regressive model?
Author: nlpcat
Pull request: no
Labels: name: New model

This research area is developing very fast in the past two years, which could improve latency  significantly with quality on par. Will huggingface team be interested into the implementation of it? Thanks.

Comments:
Can you point to certain papers, perhaps with corresponding code + weights?

This paper announced to have quality on par with 10x speed up. https://github.com/FLC777/GLAT

Another similar nat paper https://github.com/tencent-ailab/ICML21_OAXE

Most nat models are based on fairseq implementations.
https://github.com/pytorch/fairseq/blob/master/fairseq/models/nat/nonautoregressive_transformer.py
https://github.com/pytorch/fairseq/blob/master/fairseq/models/nat/levenshtein_transformer.py
examples: minor fixes in flax example readme
Author: stefan-it
Pull request: yes

Hi,

this PR introduces some minor fixes in the FLAX LM doc.

It mainly prevents an:

```
In [6]: tokenizer.save("./")
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-6-7d2c76b50951> in <module>
----> 1 tokenizer.save("./")

~/dev/lib/python3.8/site-packages/tokenizers/implementations/base_tokenizer.py in save(self, path, pretty)
    334                 A path to the destination Tokenizer file
    335         """
--> 336         return self._tokenizer.save(path, pretty)
    337 
    338     def to_str(self, pretty: bool = False):

Exception: Is a directory (os error 21)
```

error, because `.save()` function of the `ByteLevelBPETokenizer` instance expects a file name.

Comments:
[Wav2Vec2] Fix normalization for non-padded tensors
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR fixes a problem with normalization when the input is a list of different length that is not numpified - see: https://github.com/huggingface/transformers/issues/13504

Just noticed that this bug is pretty severe actually as it affects all large-Wav2Vec2 fine-tuning  :-/.
It was introduced by me in this PR: https://github.com/huggingface/transformers/pull/12804/files - I should have written more and better tests for this. 

=> This means that from transformers 4.9.0 to until this PR is merged the normalization for all large Wav2Vec2 models was way off when fine-tuning the model.
 
@LysandreJik - do you think it might be possible to do a patched release for this? 


Comments:
TF multiple choice loss fix
Author: Rocketknight1
Pull request: yes

The `TFMultipleChoiceLoss` inherits from `TFSequenceClassificationLoss` and can get confused when some of the input dimensions (specifically, the number of multiple choices) are not known at compile time. This can cause a compilation failure or other misbehaviour. Ensuring that the `TFMultipleChoiceLoss` never attempts to do regression fixes the problem.

Comments: