noamaanMulla-03/transformers-issues huggingface dataset:
Wav2vec2 finetuning shared file system
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

Make wav2vec2 fine-tuning script more robust when dealing with multi-node / shared file systems (interesting edge case :sweat_smile: )

Fixes #17412


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Adding the Portuguese version of the tasks/sequence_classification.mdx documentation
Author: jonatasgrosman
Pull request: yes

# What does this PR do?

Adding the Portuguese version of the tasks/sequence_classification.mdx documentation

Work on #16824

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@omarespejel 



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Muito obrigado pelo seu PR @jonatasgrosman! If you wish to contribute with the translation of another doc please let me know in the Issue 🤗.

@sgugger I find it ready to merge :)

Thanks a lot for your help!
Added es version of bertology.mdx doc
Author: jQuinRivero
Pull request: yes

Fixes #15947

Added spanish version of language_modeling.mdx documentation file.

@omarespejel @sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Muchas gracias @jQuinRivero for the PR! 🤗 Please let me know if you wish to translate another one. 

@sgugger LGTM :)
Spanish translation of the file preprocessing.mdx
Author: yharyarias
Pull request: yes

Hi, I made the translation of the **preprocessing.mdx** file to add it to the Spanish documentation (transformers/docs/source_es/). 
I will be waiting for any correction, thanks.
@omarespejel @LysandreJik @sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._

All good to me from the syntax point of view. Be careful though, as you PR files include the `training.mdx` so we can't know if there are small changes in it.

You're right, it was a mistake, It's just the preprocessing file

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @yharyarias! Could you please review what Sylvain[ commented above](https://github.com/huggingface/transformers/pull/16299#issuecomment-1074021882)? We are almost done.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Muchas gracias @yharyarias for the PR! 🤗 Please let me know if you wish to translate another one. Fixes issue #15947.

@sgugger LGTM :)
Pretraining code of LayoutLMv2
Author: TejasDuseja
Pull request: no

I was wondering is pre-training code of LayoutLMv2 model publicly available. @NielsRogge 

Comments:
Hi, 

Microsoft hasn't open-sourced any pretraining code, unfortunately. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[new model] `GPTMeg`  (a clone of `GPT2` with a few tiny changes)
Author: sIncerass
Pull request: yes

# What does this PR do?

This PR tries to address [this issue](https://github.com/bigscience-workshop/Megatron-DeepSpeed/issues/138) by cloning `HF's GPT2` to create `GPTMeg` with a few tiny changes for fp16 adaptation. 

The 3 sources of divergence are:

1. `layer_norm` override to be forced to be done in fp32 - as `MixedFusedLayerNorm` (meg) performs it in fp32 and then casts back to fp16
2. overrides `gelu_fast` to use meg's version which uses `torch.jit.script` which under fp16 returns diverging output (the bwd function was needed to support jit and it's not the issue)
3. `_attn` override to use `torch.baddbmm` instead of `torch.matmul` - the divergence happens due to the alpha factor as it gets applied differently in the 2 ways.


Questions:
- need to decide whether `GPTMeg` is a good name - as BigScience is going to release several more variations of GPT2 - this particular model is however a straight Megatron-LM gpt2 without any changes from BigScience. We discovered that the HF gpt2 model is an almost exact fit wrt to producing the same logits under fp32, but quite a large divergence happens under fp16. So this variant does things differently under fp16.
- I'm not sure whether the tokenizer should be cloned as it's the same? Do we have models that re-use tokenizers from other models and don't provide their own? here GPT2Tok* is just fine


Comments:
Hey @sIncerass! If the tokenizer is identical, then there is no need to implement it once again: models may use different tokenizers by specifying the `tokenizer_name` value in their `config.json` file. If using the `GPT2Tokenizer`, you would put the following:
```
"tokenizer_name": "GPT2Tokenizer"
```
in either the `config.json` or the `tokenizer_config.json`.

@sIncerass, as this PR didn't get merged because of the delays with the licensing and naming, and meanwhile it has become too stale, we wanted to make sure you get credits for all the efforts you have invested into this PR, so we [added you as a contributor](https://github.com/huggingface/transformers/pull/17202/commits/e1db789c9348b90b2e7bc2c207f6e10b2b3a13c5) to the final incarnation of the BigScience architecture PR https://github.com/huggingface/transformers/pull/17202

and we can now safely close this PR.
improve no-trainer examples
Author: pacman100
Pull request: yes

# What does this PR do?
1. In no-trainer examples, `train_loss` being logged wasn't normalized and as such wasn't intuitive to understand. This also made it difficult to compare train loss between different tools such as comparing train loss from Trainer with that of Accelerate. This PR normalizes the train_loss per epoch to make is more intuitive and comparable.
2. Replaces HF AdamW with torch AdamW for NLP no-trainer examples. This prevents corresponding warning being displayed.
3. Fixing no-trainer examples so that tracker run is created only for the main process else wandb will create num_processes runs with no data.
4. converting `train_loss` from tensor to float so that it gets logged in tensorboard tracker
5. Fixing `run_ner_no_trainer.py` to correctly log `train_loss` in `all_results.json`
6. Adding `report_to` arg to enable users to specify preferred tracker instead of all available trackers which is default option. This prevents logging to the trackers that user doesn't want.
7. In many no-trainer NLP tasks one can train model from scratch, this means that user can bypass `model_name_or_path` arg. However, it is set as required for all scripts which throws error when it isn't specified. Setting this arg `required=False` in corresponding examples to resolve the error when training from scratch.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Opened a new PR on the branch of main repo instead of fork #17449
Pin protobouf that breaks TensorBoard in PyTorch
Author: sgugger
Pull request: yes

# What does this PR do?

The recent release of Protobuf (4.21) has broken TensorBoard in PyTorch and thus multiple tests. This PR pins protobuf to fix said tests.

Comments:
Yes to rebuliding a docker image ASAP. I will merge this once all tests pass :-) 

_The documentation is not available anymore as the PR was closed or merged._

Python dependencies 🤦‍♀️
❓ How to run pipeline (summarization) in FP16 mode ?
Author: astariul
Pull request: no

# ❓ Questions & Help

I couldn't find on the documentation any parameter that allow running a pipeline in FP16 mode. **Did I miss it or it's not a feature yet ?**

Comments:
@sshleifer - do you have more info on that? 

You can't without editing the code, unfortunately.


Running a pipeline in FP16 mode would be really useful for optimizing the GPU RAM usage. Can this be turned into a feature request?

**Edit:** I just found out that the following works:

```
pipeline.model.half()
```
Improve notrainer examples
Author: pacman100
Pull request: yes

# What does this PR do?
1. In no-trainer examples, `train_loss` being logged wasn't normalized and as such wasn't intuitive to understand. This also made it difficult to compare train loss between different tools such as comparing train loss from Trainer with that of Accelerate. This PR normalizes the train_loss per epoch to make is more intuitive and comparable.
2. Replaces HF AdamW with torch AdamW for NLP no-trainer examples. This prevents corresponding warning being displayed.
3. Fixing no-trainer examples so that tracker run is created only for the main process else wandb will create num_processes runs with no data.
4. converting `train_loss` from tensor to float so that it gets logged in tensorboard tracker
5. Fixing `run_ner_no_trainer.py` to correctly log `train_loss` in `all_results.json`
6. Adding `report_to` arg to enable users to specify preferred tracker instead of all available trackers which is default option. This prevents logging to the trackers that user doesn't want.
7. In many no-trainer NLP tasks one can train model from scratch, this means that user can bypass `model_name_or_path` arg. However, it is set as required for all scripts which throws error when it isn't specified. Setting this arg `required=False` in corresponding examples to resolve the error when training from scratch.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Supress long sequence tokenization warning
Author: eladbitton
Pull request: no

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

While tokenizing some sequences longer than 512 i get this error. I am aware that bert can't handle sequences longer than 512 so i split it later.

> Token indices sequence length is longer than the specified maximum sequence length for this model (619 > 512). Running this sequence through the model will result in indexing errors

Is there a way to supress this warning?

Comments:
This is simply a warning, it won't change your results. I think it's important we keep it for people that are unaware that sequences have a max length of 512, so there's currently no option to suppress that warning.

Just a note: if you want to avoid displaying the warning, you can raise the level of the logger with `logging.getLogger("pytorch_pretrained_bert.tokenization").setLevel(logging.ERROR)`.

We did that in the  Transfer Learning tutorial code (see [here](https://github.com/huggingface/naacl_transfer_learning_tutorial/blob/master/utils.py#L134))

This will avoid display all the logging message under the error level though, so use it with care ;-)

> Just a note: if you want to avoid displaying the warning, you can raise the level of the logger with `logging.getLogger("pytorch_pretrained_bert.tokenization").setLevel(logging.ERROR)`.
> 
> We did that in the Transfer Learning tutorial code (see [here](https://github.com/huggingface/naacl_transfer_learning_tutorial/blob/master/utils.py#L134))
> 
> This will avoid display all the logging message under the error level though, so use it with care ;-)

Just a small fix. Because the names have changed, use this instead:
`logging.getLogger("pytorch_transformers.tokenization_utils").setLevel(logging.ERROR)`

I encountered this problem too. As of Oct. 16, 2019, the correct way to suppress warning is:
`logging.getLogger("transformers.tokenization_utils").setLevel(logging.ERROR)`

I'm not sure from which version, but the above doesn't work anymore.
If any one stumbles on the same problem, try this:
`logging.getLogger("transformers.tokenization_utils_base").setLevel(logging.ERROR)`

See the global solution here: https://github.com/huggingface/transformers/issues/3050#issuecomment-682167272

from transformers.utils import logging
logging.set_verbosity(40)

This will set your logger to only display Errors (no warnings).

See more here -> https://huggingface.co/docs/transformers/main_classes/logging
Quick tour_ AutoModel Introduction
Author: Clearloveyuan
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 3.4.0
- Platform: Linux-4.15.0-144-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoModelForSequenceClassification
model_name = "nlptown/bert-base-multilingual-uncased-sentiment"
pt_model = AutoModelForSequenceClassification.from_pretrained(model_name,return_dict=True)


pt_outputs = pt_model(**pt_batch)


from torch import nn
pt_predictions = nn.functional.softmax(pt_outputs.logits, dim=-1)
print(pt_predictions)

### Expected behavior

```shell
from transformers import AutoModelForSequenceClassification
model_name = "nlptown/bert-base-multilingual-uncased-sentiment"
pt_model = AutoModelForSequenceClassification.from_pretrained(model_name)


pt_outputs = pt_model(**pt_batch)


from torch import nn
pt_predictions = nn.functional.softmax(pt_outputs.logits, dim=-1)
print(pt_predictions)

I think pt_outputs.logits is wrong, since from_pretrained(model_name) without return_dict=True.
```


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Module import takes too long
Author: StefanHeng
Pull request: no

### Feature request

Make
```python
import transformers
```

faster 



### Motivation

When I import the transformers package, it always takes > 5 seconds on my MacBook Pro machine. 

I'm not sure if it's just me but how can I make this faster? 

This can get a bit annoying when my debug script is pretty fast to run. 

MWE to illustrate the problem

```python
from datetime import datetime
start = datetime.now()
import transformers
print(datetime.now() - start)
```


### Your contribution

NA

Comments:
Hey @StefanHeng, it should not take that long. What version of `transformers` do you have installed?

Interesting. 

I have python version `3.9.7` and `transformers` version 4.18.0 in a conda environment. 

You may have tools that you do not directly need that slow down your instantiation: are you using torch, tensorflow or flax? If you're only using one, can you try to uninstall the other two if they're installed and checking if it speeds up the install?

I'm using `torch`. Now that you mentioned it, I do also have `tensorflow` installed in the environment too. 

Because I use `torch` for the architectures but I need the `tensorboard` under `tensorflow` for visualizations. 

When I uninstall `tensorflow`, I'm seeing ~1 seconds for the `transformers` import. Nice! 

but is it possible to speed up loading 2 backends? cos I do need the `tensorboard` functionalities under `tensorflow`... 

Unfortunately, it isn't `transformers` taking all that time, it's likely `tensorflow`. Loading `tensorflow` on its own will likely take most of the time that you saw when you had both frameworks installed, as we load a tiny subset of the items when doing `import transformers`. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Spanish docs - Links don't work
Author: omarespejel
Pull request: no

## TLDR
Links to docs that don't exist (eg ./main_classes/pipelines) don't work. Issue first mentioned in #17349.

## Description
Currently, the links lead to an error if the doc is not yet translated. For example, this fragment in [`autoclass_tutorial`](https://huggingface.co/docs/transformers/main/es/autoclass_tutorial) leads a to an error for not having [model_doc/auto.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/auto.mdx)
 translated yet.
> Finalmente, las clases AutoModelFor te permiten cargar un modelo preentrenado para una tarea dada (revisa [aquí](https://huggingface.co/docs/transformers/main/es/model_doc/auto) para conocer la lista completa de tareas disponibles).

## Possible solution
Linking to the English docs while the Spanish versions become available.

## Possible reviewers
@sgugger @mishig25 

Comments:
@omarespejel thanks a lot for notifying. I'm working on a fix