noamaanMulla-03/transformers-issues huggingface dataset:
Tokenization in run_mlm is not correct (maybe)?
Author: gpucce
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 4.17.0.dev0
- Platform: Linux-5.13.0-28-generic-x86_64-with-glibc2.17
- Python version: 3.8.12
- PyTorch version (GPU?): 1.10.2 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: True
```


### Who can help?

@sgugger 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In the `run_mlm.py` unless I am missing something which is completely possible, the two functions `tokenize_function` and `group_texts` end up tokenizing sentences in this way (for BERT for example):

The sentence "One sentence. Another sentence. One last sentence." is tokenized as 
[101, id_One, id_sentence, 102, 101, id_Another, id_sentece, 102, 101, id_One, id_last, id_sentence, 102]
instead of 
[101, id_One, id_sentence, 102, id_Another, id_sentece, 102, id_One, id_last, id_sentence, 102]

I believe this ends up being in conflict (not erroring but behaving differently) with the default tokenizer behaviour on fine tuning tasks with pairs of sentences such as mnli. Is this intended?

### Expected behavior

```shell
Maybe this is the expected tokenization
[101, id_One, id_sentence, 102, id_Another, id_sentece, 102, id_One, id_last, id_sentence, 102]?
```


Comments:
This is just an example, as mentioned in the main example README, and you are free to adapt the data processing to your needs.
This script is not reproducing the BERT pretraining (which also has some next sentence prediction objective), it just gives an example of masked language modeling.

Ok I thought maybe it would make sense pointing it out since it did trick me :)
AttributeError: 'DataParallel' object has no attribute 'save_pretrained'
Author: bilalghanem
Pull request: no
Labels: name: bug

### System Info

```shell
torch==1.10.2+cu113
transformers==4.18.0
Python 3.6.9
Linux "18.04.6 LTS (Bionic Beaver)"
```

I am training a T5 transformer (T5ForConditionalGeneration.from_pretrained(model_params["MODEL"])) to generate text. The model works well when I train it on a single GPU. But when I want to parallel the data across several GPUs by doing `model = nn.DataParallel(model)`, I can't save the model.

The error is:

> File "run.py", line 288, in T5Trainer
>     model.save_pretrained(path)                                                                                                
>   File "/home/USER_NAME/venv/pt_110/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1178, in __getattr__
>     type(self).__name__, name))
> AttributeError: 'DataParallel' object has no attribute 'save_pretrained'



### Reproduction

Wrap the model with `model = nn.DataParallel(model)`.

### Expected behavior

```shell
The model should be saved without any issues.
```


Comments:
`DataParallel` wraps the model. To access the underlying module, you can use the `module` attribute:
```py
>>> from torch.nn import DataParallel
>>> model = nn.DataParallel(model)
>>> model.module.save_pretrained(<directory>)
```

> `DataParallel` wraps the model. To access the underlying module, you can use the `module` attribute:
> 
> ```python
> >>> from torch.nn import DataParallel
> >>> model = nn.DataParallel(model)
> >>> model.module.save_pretrained(<directory>)
> ```

Thanks @LysandreJik!
Docs link to example scripts is broken
Author: ftnext
Pull request: no

This is about **documentation link error**.

## Error

At https://huggingface.co/docs/transformers/main/en/main_classes/trainer#trainer
>It’s used in most of the [example scripts](https://huggingface.co/docs/transformers/main/en/examples).

The link *example scripts* https://huggingface.co/docs/transformers/main/en/examples is broken, got 404.

<img width="549" alt="image" src="https://user-images.githubusercontent.com/21273221/166940684-aa6489cb-d47a-4c94-beab-c197d0b588f0.png">

## Source

https://github.com/huggingface/transformers/blob/dd16a113a48cc2117953397e9611383577d6a70d/docs/source/en/main_classes/trainer.mdx#L15

## Question

The *example scripts* link means https://github.com/huggingface/transformers/tree/main/examples ?


Comments:
Hi, a PR has been opened to fix this issue. Thanks for reporting! :)

The problem was `examples` had been removed from the `toctree` which caused the link to be broken.
Remove torchhub test
Author: sgugger
Pull request: yes

# What does this PR do?

The torchhub test is failing with a cryptic error. This PR removes the test but keeps the integration intact. As the Hugging Face Hub becomes the main marketplace for `transformers` models, it should be the integration users use and while torchhub remains supported we will not test it going forward.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17097). All of your documentation changes will be reflected on that endpoint.

## 😭😭
Is gradient checkpointing really needed when fine-tuning LED on 16384 tokens?
Author: caesar-one
Pull request: no

Hi,

The [documentation of LED](https://huggingface.co/docs/transformers/model_doc/led) states: 
> To fine-tune LED on all 16384, it is necessary to enable gradient checkpointing by executing model.gradient_checkpointing_enable().

Moreover, @patrickvonplaten in [this notebook](https://colab.research.google.com/drive/12LjJazBl7Gam0XBPy_y0CTOJZeZ34c2v?usp=sharing) initializes the model using `AutoModelForSeq2SeqLM.from_pretrained("allenai/led-base-16384", gradient_checkpointing=True, use_cache=False)`.

I have a 24GB GPU, but the base model fits perfectly without the need of activating gradient checkpointing, which instead slows down performance by a large margin. I am using a batch size = 1 with 16 steps of gradient accumulation.

So my question is: is my approach (just gradient accumulation) enough for fine-tuning LED, or it is _really_ necessary to use gradient checkpointing (for reasons I may have missed)? If so, what are the implications of using gradient checkpointing in this case, besides saving memory?

I was also wondering what exactly does the `use_cache=False` parameter in the `from_pretrained` method.

Thank you very much

Comments:
Hey @caesar-one,

Sorry, the documentation of LED might not have been super clear in this case. It should probably be rewritten to:

```
Gradient check-pointing is only needed if training leads to out-of-memory (OOM) errors
```

Would you like to improve the docs here maybe? :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Sure :)
Fix self-push CI report path in cat
Author: ydshieh
Pull request: yes

# What does this PR do?

`self-push.yml` has lines like

```
run: cat reports/tests_torch_gpu_failures_short.txt
```
which should be 
```
run: cat reports/tests_torch_gpu/failures_short.txt
```

Currently, we get errors like
```
cat: reports/tests_torch_multi_gpu_failures_short.txt: No such file or directory
Error: Process completed with exit code 1.
```

(see for example [this job](https://github.com/huggingface/transformers/runs/6307735900?check_suite_focus=true)).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> If this change is needed, it should also be done in the other GitHub action files, 

The other occurrences are in `self-nightly-scheduled.yml"`. ~~I can update it too.~~ **Done**

> and also in the report part of the job? Not sure if they will push properly nested files in subdirectories.

I guess you are talking about files like `notification_service_deprecated.py` and `notification_service.py`. They are not using `failures_short.txt`. 

Note that:
 - this PR doesn't change any artifact directory location
 - `notification_service_deprecated.py` does have issues with the report.
   - For example, `"run_all_tests_tf_gpu_test_reports/[X].txt"` should be `run_all_tests_tf_gpu_test_reports/tests_torch_gpu/[X].txt`
     - (if we don't change the artifact directory in `self-push.yml`)  
   - I am still on the task of the self-push report format, which might still take some time.
     - But think I can already fix a few small things in the meantime.
Export longformer to ONNX
Author: anasilvasd
Pull request: no

Hi :) I need to convert a pre-trained pytorch model to tensorflow 2. I'm trying to follow these instructions https://huggingface.co/docs/transformers/serialization#exporting-a-model-to-onnx (export to onnx first).

But I'm having this error:

KeyError: "longformer is not supported yet. Only ['albert', 'bart', 'mbart', 'bert', 'ibert', 'camembert', 'distilbert', 'marian', 'm2m-100', 'roberta', 't5', 'xlm-roberta', 'gpt2', 'gpt-neo', 'layoutlm', 'electra'] are supported. If you want to support longformer please propose a PR or open up an issue."

Is it possible to add support to longformer, please? Thanks



Comments:
> Is it possible to add support to longformer, please? Thanks

Hey @anasilvasd we have created a group recently that aims to add ONNX support for all models on the hub. Maybe you could help us by doing the implementation yourself?

Here is the link of the issue relating all the models that need help #16308
And here is the HF org if you want to join us: [ONNXConfig for all](https://huggingface.co/OWG)

You will find help on implementation by reading issue post and looking at others implementation linked in the issue below. Have a great week-end

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
DeBERTa from official example return random logits that change every time I run the same example.
Author: Oxi84
Pull request: no

EDIT: This happens each time the model  is loaded again from disk. Once model is loaded the logits  are the same each run for the same input but when I reload the model the logits change.

I tried to run the official example of Deberta and found that the output logins are just random numbers that change every time i run the model? Is this a bug?

Transformers version is: 4.9.2 also in 4.15.0 is the same and i tried the current latest version and it is the same.

     from transformers import DebertaTokenizer, DebertaForMaskedLM
    import torch
    tokenizer = DebertaTokenizer.from_pretrained("microsoft/deberta-base")
    model = DebertaForMaskedLM.from_pretrained("microsoft/deberta-base")
    model.eval()


    inputs = tokenizer("The capital of France is [MASK].", return_tensors="pt")
    labels = tokenizer("The capital of France is Paris.", return_tensors="pt")["input_ids"]

    with torch.no_grad():
        outputs = model(**inputs, labels=labels)

    loss = outputs.loss

    logits = outputs.logits

    print("logits",logits)`


example output:
1) tensor([[[-0.3574, -0.4279
2) tensor([[[-1.0341,  0.5181

I also get this as warning:
Some weights of the model checkpoint at microsoft/deberta-base were not used when initializing DebertaForMaskedLM: ['deberta.embeddings.position_embeddings.weight', 'lm_predictions.lm_head.LayerNorm.weight', 'lm_predictions.lm_head.LayerNorm.bias', 'lm_predictions.lm_head.bias', 'lm_predictions.lm_head.dense.weight', 'lm_predictions.lm_head.dense.bias']
- This IS expected if you are initializing DebertaForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing DebertaForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of DebertaForMaskedLM were not initialized from the model checkpoint at microsoft/deberta-base and are newly initialized: ['cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.




@LysandreJik

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] When I try to use my model
Author: MLDovakin
Pull request: no

an error occurred while importing my model from a folder. I cloned my repository and wanted to use the model, I got an error
https://huggingface.co/Fidlobabovic/beta-kvantorium-simple-small
Do I need to change files in my repository? What should I fix in code or model files?
#7370 
#9667 
````
from transformers import pipeline

nlp = pipeline("question-answering",  model='/content/beta-kvantorium-simple-small', tokenizer='/content/beta-kvantorium-simple-small')

context = r"""
   Цель текущего контроля и аттестаций - выявление уровня обученности, развития способностей обучающихся, приобретенных компетенций и их соответствие прогнозируемым результатам дополнительной общеобразовательной программы.
"""

print(nlp(question="Какая Цель текущего контроля аттестаций?", context=context))

-> 1046                 state_dict = torch.load(resolved_archive_file, map_location="cpu")
   1047             except Exception:

9 frames
UnpicklingError: invalid load key, 'v'.

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
OSError: Unable to load weights from pytorch checkpoint file for '/content/beta-kvantorium-simple-small' at '/content/beta-kvantorium-simple-small/pytorch_model.bin'If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. 

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1183                     raise EnvironmentError(
   1184                         "Error no file named {} found in directory {} or `from_pt` set to False".format(
-> 1185                             [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path
   1186                         )
   1187                     )

OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory /content/beta-kvantorium-simple-small or `from_pt` set to False



Comments:
Please check if the `pytorch_model.bin` file is available in your cloned repo. I can see that file on the hub, so there might have been some mistake when cloning the repo.

@patil-suraj  when cloning the repository, this file is located in the folder


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

> Please check if the `pytorch_model.bin` file is available in your cloned repo. I can see that file on the hub, so there might have been some mistake when cloning the repo.

Thanks!🥰 It solves my problem, some files' name were changed after downloaded🤕
is it possible to split the embedding layers from bert during fine tuning and inference
Author: zhufeijuanjuan
Pull request: no
Labels: name: bug

### System Info

```shell
in my bert model, parameters in embedding layers are huge since the vocabulary size is very big. So I plan to split the embedding layer from the entire model and use it as a independent module to calculate word embeddings. Is it possible to do like this? thanks
```


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

NONE

### Expected behavior

```shell
NONE
```


Comments:
Deberta gives completelly wrong and random output tested on multiple machines and multiple versions of transformers
Author: Oxi84
Pull request: no

I do not understand why would you add this model here write lengthy documentation and let it not work at all. 

**Its prediction is: The capital of France is plunge**

I have tried on multiple machines and multiple version of transformers and the results are just random.

    from transformers import pipeline
    unmasker = pipeline('fill-mask', model='deberta-base')
    the_out = unmasker("The capital of France is [MASK].")
    print("the_out",the_out)

As you can see the deberta results is completely wrong, there is some big error in porting it to transformers. 

 the_out [{'score': 0.001861382625065744, 'token': 18929, 'token_str': 'ABC', 'sequence': 'The capital of France isABC.'}, {'score': 0.0012871784856542945, 'token': 15804, 'token_str': ' plunge', 'sequence': 'The capital of France is plunge.'}, {'score': 0.001228992477990687, 'token': 47366, 'token_str': 'amaru', 'sequence': 'The capital of France isamaru.'}, {'score': 0.0010126306442543864, 'token': 46703, 'token_str': 'bians', 'sequence': 'The capital of France isbians.'}, {'score': 0.0008897537481971085, 'token': 43107, 'token_str': 'insured', 'sequence': 'The capital of France isinsured.'}]


    from transformers import pipeline
    unmasker = pipeline('fill-mask', model='bert-base-uncased')
    the_out = unmasker("The capital of France is [MASK].")
    print("the_out",the_out)

The bert result is good:

   the_out [{'score': 0.41678911447525024, 'token': 3000, 'token_str': 'paris', 'sequence': 'the capital of france is paris.'}, {'score':    0.07141649723052979, 'token': 22479, 'token_str': 'lille', 'sequence': 'the capital of france is lille.'}, {'score': 0.06339272856712341, 'token': 10241, 'token_str': 'lyon', 'sequence': 'the capital of france is lyon.'}, {'score': 0.04444753751158714, 'token': 16766, 'token_str': 'marseille', 'sequence': 'the capital of france is marseille.'}, {'score': 0.030297178775072098, 'token': 7562, 'token_str': 'tours', 'sequence': 'the capital of france is tours.'}]




@LysandreJik




Comments:
I don't think the pre-trained language model weights are available for deberta-base, so the output looks completely random. See here for more info: https://github.com/huggingface/transformers/issues/15216

I see but it does not also work for base-v3 as well I will check other models and see.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
cannot import name 'RegNetModel' from 'transformers'
Author: MorningStarOvO
Pull request: no
Labels: name: bug

### System Info

```shell
python                    3.8 
transformers              4.18.0
```


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import RegNetModel

### Expected behavior

```shell
how to import RegNetModel ?
```


Comments:
RegNet is currently only available from the main branch, it will be included it in the next release. You can install it as follows:

`pip install git+https://github.com/huggingface/transformers.git`
Token classification pipeline results different with tokenizers==0.11.6 vs tokenizers==0.12.0
Author: davidmezzetti
Pull request: no

I'm not sure if this is an issue with transformers, an issue with tokenizers or expected behavior. But when running the token classification pipeline with an aggregation_strategy="simple" the results are slightly different with tokenizers==0.12.0.

The following code produces different results (both examples use transformers==4.17.0).

```python
from transformers import pipeline
nlp = pipeline("token-classification")
nlp("Hugging Face Inc. is a company based in New York City", aggregation_strategy="simple")
```

With tokenizers==0.11.6:

```
[{'entity_group': 'ORG', 'score': 0.99305606, 'word': 'Hugging Face Inc', 'start': 0, 'end': 16}, {'entity_group': 'LOC', 'score': 0.9988098, 'word': 'New York City', 'start': 40, 'end': 53}]
```

With tokenizers==0.12.0:

```
[{'entity_group': 'ORG', 'score': 0.99305606, 'word': ['Hu', 'gging', ' Face', ' Inc'], 'start': 0, 'end': 16}, {'entity_group': 'LOC', 'score': 0.9988098, 'word': ['New', ' York', ' City'], 'start': 40, 'end': 53}]
```



Comments:
cc @SaulLu @Narsil 

There's a proposed PR here : https://github.com/huggingface/transformers/pull/16537

Tokenization tests were run before releasing `0.12` but not the pipeline tests :(

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
run_clm.py crashes server.
Author: ShivamSharma2705
Pull request: no

I was running the run_clm.py script with deepspeed for finetuning gpt2-small on a 300GB data. I had provided 5 GPUs for the process, but as soon as the preprocessing of the data was completed the server crashed and I had to get it restarted. Can you point me to where we need to make changes to the script to support finetuning with such large datasets?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.