noamaanMulla-03/transformers-issues huggingface dataset:
static masking for BERT or RoBERTa model
Author: sgonzaloc
Pull request: no

I would like to use static masking for Roberta and also BERT.

What I saw here is that the collector is always implmeneted like dynamic masking. https://github.com/huggingface/transformers/issues/5979

There're 2 issues with this. First, BERT is static masking so to be able to reproduce and run BERT like the original paper, we need to have it. Second, in MNLI the paper of Roberta says that is better the static option, so in some cases would be better and interesting to run it in static way. 

So in short, I really would like if somebody adds the missing option to be able to run static masking, and not like now that always is dynamic and we don't have any option for changing this

cc @sgugger 

Comments:
You can do static masking in the function that preprocesses your dataset. The `data_collator` are only here for dynamic operations to apply at batching (dynamic paddings, dynamic masking etc.)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.15.1/metrics/sacrebleu/sacrebleu.py
Author: xiuzhilu
Pull request: no

when I run python examples/pytorch/translation/run_translation.py     --model_name_or_path t5-small     --do_train     --do_eval     --source_lang en     --target_lang de     --source_prefix "translate English to German: "     --dataset_name stas/wmt14-en-de-pre-processed     --output_dir /tmp/tst-translation     --per_device_train_batch_size=4     --per_device_eval_batch_size=4     --overwrite_output_dir     --predict_with_generate. I get the issue "ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.15.1/metrics/sacrebleu/sacrebleu.py". But I can open it on the website. What should I do to solve the problem.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
TF and PT code confuse in the documentation 
Author: LucienShui
Pull request: no

Hi, @sgugger, at [quicktour.mdx#L337](https://github.com/huggingface/transformers/blame/5eca742f6c0e513d8c7d8085fd14b7a754ea96f7/docs/source/quicktour.mdx#L337) and [quicktour.mdx#L345](https://github.com/huggingface/transformers/blame/5eca742f6c0e513d8c7d8085fd14b7a754ea96f7/docs/source/quicktour.mdx#L345), TF code was written in PT block, so do PT.

Comments:
Thanks again for fixing it!
XLNetLMHeadModel has no attribute 'from_config'
Author: josutk
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.8.0
- Platform: Linux-4.15.0-20-generic-x86_64-with-LinuxMint-19-tara
- Python version: 3.6.9
- PyTorch version (GPU?): 1.8.0+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes

### Who can help
@patrickvonplaten

## Information

Model I am trying to train a XLNet model from scratch to portuguese but when i run this script: https://github.com/huggingface/transformers/blob/v4.8.2-release/examples/pytorch/language-modeling/run_plm.py i get this error:

Traceback (most recent call last):
  File "run_plm.py", line 498, in <module>
    main()
  File "run_plm.py", line 330, in main
    model = XLNetLMHeadModel.from_config(config)
AttributeError: type object 'XLNetLMHeadModel' has no attribute 'from_config'

The problem arises when using:
* [x] the official example scripts: (give details below)

The tasks I am working on is:
* [x] my own task or dataset: (give details below)
portuguese wikipedia dataset




Comments:
Good catch @josutk,

I think we should change this line to:

```
model = XLNetLMHeadModel(config)
```

would you like to open a PR for this? :-)

@patrickvonplaten I have open PR.
Improve perceiver
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR removes the need for the hard-coded `d_model` attribute of the Perceiver. Instead, one calculates the dimensionality (i.e. number of channels) of the inputs based on the `num_channels` property of the preprocessor.

Note that `d_model` is still relevant for users 1) when embedding text using `PerceiverTextProcessor` 2) when no preprocessor is provided.

Comments:
Fix docs pointers for DeepSpeed
Author: marctorsoc
Pull request: no

In this [section](https://huggingface.co/docs/transformers/master/en/main_classes/trainer#deepspeed), the reader is pointed to several links that are broken:

* https://huggingface.co/docs/transformers/master/en/main_classes/trainer#deepspeed-trainer-integration
* https://huggingface.co/docs/transformers/master/en/main_classes/trainer#deepspeed-installation
* https://huggingface.co/docs/transformers/master/en/main_classes/trainer#deepspeed-multi-gpu
* https://huggingface.co/docs/transformers/master/en/main_classes/trainer#deepspeed-one-gpu
*https://huggingface.co/docs/transformers/master/en/main_classes/trainer#deepspeed-config-passing
* etc

All of them should be changed like this:

`https://huggingface.co/docs/transformers/master/en/main_classes/trainer#(.*)` --> `https://huggingface.co/docs/transformers/master/en/main_classes/deepspeed#(.*)`

I would do it myself, but I see that the code uses e.g. `:ref:`deepspeed-notebook` and not sure how to point to a section in another page 🤷‍♂️  It looks like a trivial change though


Comments:
If I'm not mistaken this is patched on `master` and will be reflected soon on the documentation cc @sgugger  

@LysandreJik At least in this line: https://github.com/huggingface/transformers/blame/master/docs/source/main_classes/trainer.rst#L506, it seems to be still outdated. Are you sure this is patched?

I will push a patch later today.
Mention no images added to repository
Author: LysandreJik
Pull request: yes

Mention that images shouldn't be added to the repository as it will otherwise significantly weigh down the repository.

Comments:
WIll merge this as a first step!

Merged too soon? We need a concrete destination for images that we all share.

Here it is: https://huggingface.co/datasets/huggingface/documentation-images

but it's not part of this PR that was merged. i.e. @NielsRogge's suggestion wasn't integrated into the doc.

additionally please please keep these instructions in `docs/README.md` - thank you!
SacreBLEU uses incorrect tokenizer for Japanese
Author: ekoenitz
Pull request: no

[SacreBLEU](https://github.com/mjpost/sacreBLEU) needs to be told to use a different tokenizer in order to properly evaluate Japanese text, but as far as I can tell the HuggingFace framework has no means with which to do this.  I imagine other languages that need special tokenizers are affected as well, but I haven't tested them.

## Environment info
- `transformers` version: 4.13.0.dev0
- Platform: Linux-5.4.0-84-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.10.0+cu113 (True)
- Tensorflow version (GPU?): 2.6.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
- Text generation: @patrickvonplaten @narsil

## Information

Model I am using: mBART-50 and mT5

The problem arises when using:
* [ ] the official example scripts: /transformers/examples/pytorch/run_translation.py

The tasks I am working on is:
* [ ] my own task or dataset: [JESC](https://nlp.stanford.edu/projects/jesc/)

## To reproduce

Steps to reproduce the behavior:

1. Run a translation model as described in /transformers/examples/pytorch/README.md, with **Japanese as the target language**
2. Use the --do_eval, --do_predict, and --predict_with_generate flags
3. Note the low BLEU score
4. Run the [SacreBLEU](https://github.com/mjpost/sacreBLEU) command on detokenized target vs. generated_predictions.txt with Japanese tokenizer (example: sacrebleu dev_ja_detokenized.txt -i generated_predictions.txt --tokenize ja-mecab -b)
5. Note the different, probably higher, BLEU score

## Expected behavior

I would expect HuggingFace's SacreBLEU evaluation score to be the same as SacreBLEU proper.

To give an example of how big the gap gets: After 3 epochs of fine-tuning mT5-large on JESC, HF evaluated the BLEU score as only **5.17**.  Running the output through the SacreBLEU command, however, caused the score to jump to **11.90**, which is much more in-line with the ja-en score of 16.8.


Comments:
Hi @ekoenitz,

I am by no means an expert in this example.
However, looking at it, it seems you are adding an argument to help `sacrebleu` figuring out the japanese tokenizer `--tokenizer ja-meca` right ?

If that's the case, the bleu score is actually calculated by Sacrebleu too and you would need to send that option too., https://github.com/huggingface/transformers/blob/master/examples/pytorch/translation/run_translation.py#L515 Adding `metrics.compute(...., tokenizer="ja-meca")`  should behave the same as your command.

Could that be the explanation ?

If that's it, this could probably be added to the README.md ?

Ah, I didn't know you could send metric-specific arguments like that.  That does solve the issue, yes, thank you.

Along with readme updates, perhaps adding something like an eval_tokenizer argument would be a good idea?  I was 50/50 on whether I should file this issue as a bug or a feature request.
Length penalty for beam search
Author: tuvuumass
Pull request: no

Hey everyone!

I want to reproduce T5's result on the CNN/DM summarization task. As described in the paper, T5 uses beam search with a beam width of 4 and a length penalty of α = 0.6 ([Wu et al., 2016](https://arxiv.org/abs/1609.08144)). However, I couldn't find a specific argument to set the length penalty α (which is a scaling factor for brevity penalty). Is it available somewhere?

Comments:
Tagging @patil-suraj as it involves summarization.

Hi @tuvuumass , there's the `length_penalty` argument in the `generate` method, which you can use for this.
https://huggingface.co/docs/transformers/master/en/main_classes/model#transformers.generation_tf_utils.TFGenerationMixin.generate.length_penalty

@patil-suraj: Great, thanks Suraj! Is it also available for a pre-trained Pytorch model, e.g., BART?

it's in the `generate` method, which is available for all causal and seq2seq models, so yes for BART as well.

Great, thanks!
[doc] performance: groups of operations by compute-intensity
Author: stas00
Pull request: yes

I found this really good summary of 3 groups of operations in transformers by their compute-intensity from https://arxiv.org/abs/2007.00072

I adapted it to be generic as it was originally written to be specific to data movements.

I think it'd make a great addition to the performance doc.

@sgugger 

Comments:
Importing problem 
Author: abetatos
Pull request: no

- `transformers` version: 4.5.1
- It just cannot import the version., . cannot import name 'PegasusTokenizer' from 'transformers', python 3.8

Comments:
try the newest version 4.6.0.dev0

![image](https://user-images.githubusercontent.com/54096137/116375420-d7b16580-a841-11eb-8d0e-e562d5dfb263.png)



Could you install `sentencepiece` and try again? The `PegasusTokenizer` is based on the `sentencepiece` library.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Iam also facing the same issue
Fix the doc_build_test job
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes the `build_doc_test` job by installing Transformers from the branch instead of master. This then triggers a problem if we do an editable install because the `pull-request` environment of GitHub actions is read-only and a Python package. Thankfully just removing the `-e` solves the issue!

Comments:
Nan when training LayoutLM_V2 Model
Author: Asocsar
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.13.0
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyTorch version (GPU) : 1.10.0+cu111
- Tensorflow version (GPU): 2.7.0
- Flax version: not installed
- Jax version: not installed
- JaxLib version: not installed


### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people. -->

@NielsRogge




## Information

The model used is LayoutLMv2:

The problem arises when using:
* [x] my own modified scripts:


The tasks I am working on is:
* [x] Document streaming segmentation
In my script, I try to determine when starts a new document with the objective to divide into segments a streaming of folders, so each segment can be interpreted as an independent document.

## To reproduce

Steps to reproduce the behavior:

1. Access to the colab notebook created to train LayoutLM_V2 (https://colab.research.google.com/drive/1MsEkj_WlGYDOs3vFcm1JxmMNLWj_Se78?usp=sharing)
2. Execute every cell in order
3. In the training loop, Accuracy, loss, and output will be printed, and there will be a moment when the output, Accuracy and Loss will become Nan.

## Expected behavior

The model trains, and despite if it accomplishes its task or not, the training loop ends without any Nan.


Comments:
Possible simplification in T5 docs
Author: xhluca
Pull request: no

I noticed the following nested list comprehension being recommended in the [T5 docs:](https://huggingface.co/docs/transformers/model_doc/t5)

```python
# replace padding token id's of the labels by -100
labels = [
           [(label if label != tokenizer.pad_token_id else -100) for label in labels_example] for labels_example in labels
] 
labels = torch.tensor(labels)
```

I believe it's possible to simplify above to:
```python
labels = torch.tensor(labels)
labels[labels == tokenizer.pad_token_id] = -100
```

I would be happy to make a PR if the change is deemed relevant, though I'm not sure where the docs are located so please let me know if needed :)

Comments:
Hey,

that's indeed a way more readable way of replacing padding tokens by -100. 

The docs of T5 is located [here](https://github.com/huggingface/transformers/blob/master/docs/source/model_doc/t5.rst). Note that we are migrating all docs from .rst files to Markdown. But for now, you can add just update the code example.

Thanks for sharing! I made a PR
TF model cards
Author: Rocketknight1
Pull request: yes

- Creating model cards for models trained with Keras!
- Model cards will automatically be created by the `PushToHubCallback` - the callback will peek at the metrics and accumulate information during the training run to enable this.
- Calling `model.push_to_hub()` will not, by default, create a model card. This method is cross-platform (it comes from the `PushToHubMixin`), and changing its behaviour would negatively affect `Trainer` users.
- Instead, Keras models now have a `create_model_card()` method if users want to create a model card outside of the `PushToHubCallback`. Because this method can't peek at the training history like the callback can, you need to pass the `History` object returned by `model.fit()` to create the card.

Comments:
@sgugger all comments should be addressed and testing looks good. My main remaining concern is that `model.push_to_hub` does not generate a model card by default, because that method comes from the cross-platform `PushToHubMixin`. If you want, I can edit that method so that it creates a model card if you pass it a Keras model history? This would leave the behaviour for `Trainer` unchanged.

I'm fine with both options, so it's really up to what you think is best @Rocketknight1 
Also, if you could rebase on master to fix the CI that would be great :-)

Will do both!

@LysandreJik I'm hoping to post an example with it when it's ready, but right now I'm having some issues with the method generating malformed YAML and they're a pain to track down! 

@sgugger @LysandreJik This should be ready to go now - some tests are failing even after rebasing but they have nothing to do with this PR. Okay if I merge?

Yes you can!
Update examples/documentation for torch.distributed.run
Author: BramVanroy
Pull request: no

Most examples now read local rank from a CLI argument `--local_rank`. This ensures for it to work when using the torch launch utility `torch.distributed.launch`. However, since recent torch versions, `launch` is deprecated. The new suggested way to run distributed code from CLI is with `torch.distributed.run`. The difference from before is that instead of automatically passing CLI arguments, it sets the rank **as an environment variable**. The examples therefore need to be updated to read the ENV variable instead of the CLI argument.

See https://pytorch.org/docs/stable/elastic/run.html#launcher-api for more.

Comments:
I missed that the training arguments already have this covered:

https://github.com/huggingface/transformers/blob/bcc3f7b6560c1ed427f051107c7755956a27a9f2/src/transformers/training_args.py#L681-L685

Documentation may need an update though (see below).

Hi @BramVanroy! Let us know if this isn't sufficiently documented so that we may patch the issue

@LysandreJik I think the code in the TrainingArgs is fine. Documentation may need an update as it refers to `launch` in a couple of places. A quick search gives me mentions of `torch.distributed.launch` in these documents that may need to be replaced by `torch.distributed.run`. You'd also need to check whether `--nproc_per_node` is still an accepted argument for that command because I am not sure. So it is not only a find-and-replace update!

- https://huggingface.co/transformers/performance.html
- https://huggingface.co/transformers/main_classes/deepspeed.html
- https://huggingface.co/transformers/debugging.html

Maybe of interest to @sgugger 

I wouldn't update the documentation as it won't work for older versions of PyTorch and I would prefer to only put one command vs If your PyTorch version is this then xxx else yyy. Let's wait a bit more :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

My previous comment still stands :-)

Definitely. I think it's a good idea to keep bumping this to open, so that others can also find this issue easily, and it can serve as a reminder that as soon as `launch` is obsolete, the docs need an update. It's a tiny thing and easy to forget, so an open issue may help to remember as time goes on.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Segfault on python 3.9 exit
Author: LoicGrobol
Pull request: no

Ok, that's a weird one

## Environment info
     
- `transformers` version: 4.1.1
- Platform: Linux-5.8.0-34-generic-x86_64-with-glibc2.32
- Python version: 3.9.0+
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

Steps to reproduce the behavior:

1. Run the following script

    ```python
    import torch
    import transformers
   
    loss = torch.tensor([1.0], requires_grad=True)
    loss.backward()
    ```

The script runs correctly but exits with

```text
[1]    46823 segmentation fault (core dumped)  python testcase.py
```

Which doesn't happen if `import transformers` is commented out.

Only happens when on Python 3.9, it works as expected in 3.8.

## Full env

```text
certifi==2020.12.5
chardet==4.0.0
click==7.1.2
filelock==3.0.12
idna==2.10
joblib==1.0.0
numpy==1.19.4
packaging==20.8
pyparsing==2.4.7
regex==2020.11.13
requests==2.25.1
sacremoses==0.0.43
six==1.15.0
tokenizers==0.9.4
torch==1.7.1
tqdm==4.54.1
transformers==4.1.1
typing-extensions==3.7.4.3
urllib3==1.26.2
```


Comments:
I encountered a similar issue, in a context where `transformers` was not imported. I've [reported](https://github.com/pytorch/pytorch/issues/50858) the issue to the PyTorch project.

I too have similar problem when running unittests for a Python package on travis, using Python 3.9. Specifically, all unit tests run fine, but the final outcome is segfault. See: https://travis-ci.org/github/LoryPack/abcpy/builds/755321073



This is probably fixed by https://github.com/pytorch/pytorch/pull/50998
Unable to load mT5 with MT5Model.from_pretrained("google/mt5-small") 
Author: Xuanfang1121
Pull request: no

## Environment info
- `transformers` version: 4.2.2
- Platform: win10
- Python version: 3.6
- PyTorch version (GPU?): pytorch==1.8.1
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?: 

### Who can help
 @LysandreJik@patrickvonplaten, @patil-suraj
## Information
Model I am using (mt5): code

from transformers import MT5Model
from transformers import T5Tokenizer

tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")
model = MT5Model.from_pretrained("google/mt5-small")

When I use the above code, the following error appears：
Some weights of the model checkpoint at google/mt5-small were not used when initializing MT5Model: ['lm_head.weight']
- This IS expected if you are initializing MT5Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing MT5Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

https://huggingface.co/docs/transformers/model_doc/mt5

How to load the mt5-base(or mt5-small) model correctly? thanks

Comments:
Hi @Xuanfang1121 !

You are loading `MT5Model` , which loads the base model without the `lm_head`, so when loading the pre-trained weights the `lm_head.weight` is ignored. 

If you want to load the model for conditional generation training or inference then you should use the `MT5ForConditionalGeneration` class which has the `lm_head`.

> Hi @Xuanfang1121 !
> 
> You are loading `MT5Model` , which loads the base model without the `lm_head`, so when loading the pre-trained weights the `lm_head.weight` is ignored.
> 
> If you want to load the model for conditional generation training or inference then you should use the `MT5ForConditionalGeneration` class which has the `lm_head`.

ok,thanks