noamaanMulla-03/transformers-issues huggingface dataset:
Update README.txt
Author: bino282
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
Closed in favor of #8957
Update README.txt
Author: bino282
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
Create README.md
Author: wietsedv
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Add model card.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
Can't upload the larger model file(9GB)
Author: mymusise
Pull request: no

Hey, guy, I got a problem when I upload my new model file, it raises an error.
```
/data2/CPM-TF/CPM-Third-Party on  main ⌚ 23:30:47
$ git push 
Username for 'https://huggingface.co': mymusises                                                                                        
Password for 'https://mymusise@huggingface.co': 
LFS: Client error: https://s3.amazonaws.com/lfs.huggingface.co/mymusise/CPM-Third-Party/d853c...?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AK...%2F20201119%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20201119T153653Z&X-Amz-Expires=900&X-Amz-Signature=84...&X-Amz-SignedHeaders=host
error: failed to push some refs to 'https://huggingface.co/mymusise/CPM-Third-Party'
(env) 
/data2
```
Then, I delete the folder and try it again, and I noted there's an error when I add the model file:
```
/data2/CPM-TF/CPM-Third-Party on  main! ⌚ 23:24:04
$ git add --all
Encountered 1 file(s) that may not have been copied correctly on Windows:
	tf_model.h5
```

And I sure I have installed `glf` with `git lfs install` before doing this.
What should I do?

## system info

system: ubuntu 18.04.4
git version: 2.17.1
git-lfs version: 2.12.1


 Model Cards: @julien-c

Comments:
This is a known issue that's being tracked at #8663
Copyright
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds a copyright in any file missing, or fixes the copyright in some of the files to include HuggingFace when missing. We should be vigilant when new files are added to make sure they get one @LysandreJik and @patrickvonplaten 

I've excluded the examples folder as I'll do the copyright addition along with the cleaning.

Comments:
Merging to avoid merge conflicts. Can address comments in a follow-up PR.
Fix bad import with PyTorch <= 1.4.1
Author: sgugger
Pull request: yes

# What does this PR do?

`trainer_pt_utils` imports `SAVE_STATE_WARNING` from PyTorch, which only exists in 1.5.0 or later. This fixes that problem.

Fixes #8232

Comments:
FYI `SAVE_STATE_WARNING` has been removed 3 days ago: pytorch/pytorch#46813

So `transformers` needs to be recoded not to use that constant.

Looking at its use, this probably would suffice:

```
--- a/src/transformers/trainer_pt_utils.py
+++ b/src/transformers/trainer_pt_utils.py
@@ -34,7 +34,7 @@ from .utils import logging
 if is_torch_tpu_available():
     import torch_xla.core.xla_model as xm

-if version.parse(torch.__version__) <= version.parse("1.4.1"):
+if version.parse(torch.__version__) <= version.parse("1.4.1") or version.parse(torch.__version__) > version.parse("1.7.0"):
     SAVE_STATE_WARNING = ""
 else:
     from torch.optim.lr_scheduler import SAVE_STATE_WARNING
```

and perhaps adding a note why this was needed in first place.
BertForMaskedLM train
Author: juyunsang
Pull request: no

I have a question
When training using BertForMaskedLM, is the train data as below correct?
 - token2idx
```
<pad> : 0, <mask>: 1, <cls>:2, <sep>:3
```
 - max len : 8
 - input token
```
 <cls> hello i <mask> cats <sep>
```
 - input ids
```
 [2, 34,45,1,56,3,0,0]
```
 - attention_mask
```
 [1,1,1,1,1,1,0,0]
```
 - labels
```
 [-100,-100,-100,64,-100,-100,-100,-100]
```

I wonder if I should also assign -100 to labels for padding token.

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
ImportError: cannot import name 'SAVE_STATE_WARNING' from 'torch.optim.lr_scheduler'
Author: yuchenlin
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform: ubuntu 18.04
- Python version: 3.7
- PyTorch version (GPU?): 1.4.0
- Tensorflow version (GPU?): 
- Using GPU in script?: no
- Using distributed or parallel set-up in script?:no

### Who can help

 Trainer: @sgugger
 



## Information

This import is not compatible with PyTorch 1.4.0 

The problem arises when using:
* [ *] the official example scripts: (give details below) 

The tasks I am working on is:
* [ *] an official GLUE/SQUaD task: (give the name) 
## To reproduce

Steps to reproduce the behavior:

```python
>>> from transformers import PreTrainedTokenizer, is_tf_available, is_torch_available
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/yuchenlin/anaconda3/envs/mcqa/lib/python3.7/site-packages/transformers/__init__.py", line 611, in <module>
    from .trainer import Trainer
  File "/home/yuchenlin/anaconda3/envs/mcqa/lib/python3.7/site-packages/transformers/trainer.py", line 69, in <module>
    from .trainer_pt_utils import (
  File "/home/yuchenlin/anaconda3/envs/mcqa/lib/python3.7/site-packages/transformers/trainer_pt_utils.py", line 26, in <module>
    from torch.optim.lr_scheduler import SAVE_STATE_WARNING
ImportError: cannot import name 'SAVE_STATE_WARNING' from 'torch.optim.lr_scheduler' (/home/yuchenlin/anaconda3/envs/mcqa/lib/python3.7/site-packages/torch/optim/lr_scheduler.py)
```
 

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Oh I didn't check when they added this. Do you know if PyTorch 1.4.0 is the last version without it? Will add a fix this morning.

thank you for the quick fix.

`SAVE_STATE_WARNING` has been removed 3 days ago: https://github.com/pytorch/pytorch/pull/46813

Need to update https://github.com/huggingface/transformers/pull/8237 to reflect this change.
Why is the XLM-RoBERTa sometimes producing a standalone start of the word character (the special underscore with ord = 9601)
Author: konstantinmiller
Pull request: no

I'm using the `transformers` library 3.4.0.

The XLM-RoBERTa tokenizer produces in certain cases standalone start of the word characters. Is that intended?

For example:
```
tokenizer.tokenize('amerikanische')
['▁', 'amerikanische']
```
while 
```
tokenizer.tokenize('englische')
['▁englische']
```


Comments:
Ah, I guess I figured that out. Does it happen when the training data set of the tokenizer never had this token at the beginning of a word but only inside a word?
Error: Asking to return token_type_ids while setting add_special_tokens to False
Author: chrk623
Pull request: no

In the code below, while using `batch_encode_plus`, i get the error saying that i asked ***“to return token_type_ids while setting add_special_tokens to False”***, when `return_token_type_ids` is `False`. I'm not sure if i am comprehending the error message correctly. For this specific case, I also found that the behaviour between `BertTokenizer` and `BertTokenizerFast` is different.

```python
from transformers import AutoTokenizer
t = AutoTokenizer.from_pretrained("bert-base-uncased", add_special_tokens=False)

txt = ["huggingface", "transformers"]
t.batch_encode_plus(
    txt,
    add_special_tokens=False,
    return_attention_mask=False,
    return_token_type_ids=False,
)
```
Error message
```
Traceback (most recent call last):
  File "/Users/charcohui/Desktop/env/mlenv/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3417, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-30-226d8ccb1c88>", line 5, in <module>
    return_token_type_ids=False,
  File "/Users/charcohui/Desktop/env/mlenv/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2399, in batch_encode_plus
    **kwargs,
  File "/Users/charcohui/Desktop/env/mlenv/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 567, in _batch_encode_plus
    verbose=verbose,
  File "/Users/charcohui/Desktop/env/mlenv/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 617, in _batch_prepare_for_model
    verbose=verbose,
  File "/Users/charcohui/Desktop/env/mlenv/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2681, in prepare_for_model
    "Asking to return token_type_ids while setting add_special_tokens to False "
ValueError: Asking to return token_type_ids while setting add_special_tokens to False results in an undefined behavior. Please set add_special_tokens to True or set return_token_type_ids to None.
```


However, it works when `BertTokenizerFast` is used:
```python
tfast = AutoTokenizer.from_pretrained("bert-base-uncased", use_fast=True, add_special_tokens=False)
tfast.batch_encode_plus(
    txt,
    add_special_tokens=False,
    return_attention_mask=False,
    return_token_type_ids=False,
)
# {'input_ids': [[17662, 12172], [19081]]}
```

Comments:
Hi, indeed this seems to be an error. Fixing this in #8854 if it is so.
Removed unused `encoder_hidden_states` and `encoder_attention_mask` 
Author: guillaume-be
Pull request: yes

# What does this PR do?

This PR removes unused `encoder_hidden_states` and `encoder_attention_mask` from MobileBert forward methods. These are use for decoder models, but MobileBert does not include a cross-attention mechanism.

Fixes https://github.com/huggingface/transformers/issues/8969


## Who can review?
 albert, bert, XLM: @LysandreJik


Comments:
@LysandreJik I had to remove some tests that were testing the decoder mode for MobileBert. 

One test still fails (flax), the error seems unrelated to this PR unless I am missing something?
MobileBERT decoder capabilities
Author: guillaume-be
Pull request: no

The current input parameters for MobileBERT indicate that the model may be used in a decoder setting. However, the model architecture does not contain a cross-attention mechanism and several inputs to the model are effectively never used: `encoder_hidden_states` and `encoder_attention_mask`.

This can be seen in:
- https://github.com/huggingface/transformers/blob/de6befd41f3986c68f4af302761b627cb6519eb7/src/transformers/models/mobilebert/modeling_mobilebert.py#L247, where these 2 inputs are not used
- https://github.com/huggingface/transformers/blob/de6befd41f3986c68f4af302761b627cb6519eb7/src/transformers/models/mobilebert/modeling_mobilebert.py#L330, where these inputs are just passed to the previous forward function (where they have no impact)
- https://github.com/huggingface/transformers/blob/de6befd41f3986c68f4af302761b627cb6519eb7/src/transformers/models/mobilebert/modeling_mobilebert.py#L496, where these parameters are not used (not even passed to the `MobileBertAttention`)
- https://github.com/huggingface/transformers/blob/de6befd41f3986c68f4af302761b627cb6519eb7/src/transformers/models/mobilebert/modeling_mobilebert.py#L552 where they are passed to the `MobileBertLayer` described above (therefore without impact)
- https://github.com/huggingface/transformers/blob/de6befd41f3986c68f4af302761b627cb6519eb7/src/transformers/models/mobilebert/modeling_mobilebert.py#L847 where they will trigger some reshaping of the attention mask, but eventually not get used.

I believe these unused inputs make the code more difficult to follow and potentially misleading (I don't believe the model can actually be used as a decoder).

Would you be generally supportive of a cleanup of the MobileBERT architecture to reflect its current capabilities? I'd be happy to share a PR but I wanted to check your general thoughts on this.

Thank you,

### Who can help
 albert, bert, GPT2, XLM: @LysandreJik 
(did not find anyone for MobileBert? But this is relevant for package maintenance I believe you may be the right person for this)


Comments:
You are right, there are never used. These seem to have been added by mistake during the original implementation. I would be all for a cleanup, as long as it doesn't touch to anything other than these attributes.