noamaanMulla-03/transformers-issues huggingface dataset:
Import Error: cannot import name 'warmup_linear'
Author: GBR-613
Pull request: no

I get the following error:

```
  File "/Users/gregory/PROJECTS/MyML/MLClassification/TrainAndTest/Models/controller.py", line 11, in <module>
    from Models.bert import BertModel
  File "/Users/gregory/PROJECTS/MyML/MLClassification/TrainAndTest/Models/bert.py", line 9, in <module>
    from pytorch_pretrained_bert.optimization import BertAdam, warmup_linear
ImportError: cannot import name 'warmup_linear'
```

This problem was already noted in comments to the following issue:
[https://github.com/huggingface/pytorch-pretrained-BERT/issues/499](https://github.com/huggingface/pytorch-pretrained-BERT/issues/499)
The response said that it was fixed with PR #506, and suggested to clone this Git repository and install the package from it.
However @goyalsaransh97 already mentioned that the problem persists for him (or her).

So it does for me too. 

BTW, please notice, the code works fine on machine where all packages were installed about an year ago.


Comments:
I probably was wrong and the issue was supposed to be fixed with PR #518... Anyway, the latest was merged a while ago and did not help. 

You have to wait for the next release or use the master branch

@thomwolf ,
> You have to wait for the next release or use the master branch

I cloned the git repo directly from here:

`git clone https://github.com/huggingface/pytorch-pretrained-BERT.git`

I think it takes master branch by default, isn't it?


Explicit "git checkout master" changes nothing.

The latest master branch gives the same issue. Version 0.5.1 works as of now without this issue

The version 0.4.0 doesn't give this issue.
pip install pytorch_pretrained_bert==0.4.0

> The version 0.4.0 doesn't give this issue.
> pip install pytorch_pretrained_bert==0.4.0

Downgrading to 0.4.0 solved my problem.

This issue has popped up again in 0.6.2 for me. Downgrading to 0.6.1 solved it
```bash
pip install pytorch-pretrained-bert==0.6.1
```
Padding Strategy Code missing an else case (maybe?)
Author: amanpreet692
Pull request: no

## Environment info
- `transformers` version: 3.0.2
- Platform: macOS 10.15.5 
- Python version: 3.7
- PyTorch version (GPU?): 1.5 GPU-Yes
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
 tokenizers: @mfuntowicz
 Summarization: @sshleifer
 T5: @patrickvonplaten
 
## Information

Model I am using (T5 via Autotokenizer):

The problem arises when using:
`tokenizer([line], max_length=max_length, padding='max_length' if pad_to_max_length else False,
                     truncation=True, return_tensors=return_tensors, **extra_kw)`

In batch encoding, the latest code decides on a padding strategy:
`_get_padding_truncation_strategies(
        self, padding=False, truncation=False, max_length=None, pad_to_multiple_of=None, verbose=True, **kwargs
    ):`

       ` elif padding is not False:
            if padding is True:
                padding_strategy = PaddingStrategy.LONGEST  # Default to pad to the longest sequence in the batch
            elif not isinstance(padding, PaddingStrategy):
                padding_strategy = PaddingStrategy(padding)`

While calling the tokenizer, instead of 'max_length' I first gave the actual PaddingStrategy.MAX_LENGTH Enum as argument,
but the above code throws an error as 'padding_strategy' is not defined.
## To reproduce
Call the tokenizer as:
`tokenizer([line], max_length=max_length, padding=PaddingStrategy.MAX_LENGTH if pad_to_max_length else False,
                     truncation=True, return_tensors=return_tensors, **extra_kw)`

## Expected behavior

The PaddingStrategy enum should be assigned no issue.

##Suggested Solution
               
                    ` elif padding is not False:
                              if padding is True:
                                 padding_strategy = PaddingStrategy.LONGEST  # Default to pad to the longest sequence in the batch
                      elif not isinstance(padding, PaddingStrategy):
                             padding_strategy = PaddingStrategy(padding)
                      else:
                          padding_strategy = padding`        
It's a one line fix basically, I can raise a PR for the same, unless PaddingStrategy wasn't designed to be used directly?


Comments:
This issue also applies to the `truncation` parameter.

I assumed the enums are supposed to be used directly because the release notes (https://github.com/huggingface/transformers/releases/tag/v3.0.0) explicitly mention the `TensorType` enum, which is defined right below the `PaddingStrategy` and `TruncationStrategy` enums.

I agree that this is a problem that should be fixed, if the enums are meant to be used.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I think we should fix this, wdyt @LysandreJik ?

I believe this was already fixed by https://github.com/huggingface/transformers/pull/7610

Nice, thanks!
Make Trainer evaluation handle dynamic seq_length
Author: sgugger
Pull request: yes

# What does this PR do?

In token classification problems or language modeling predictions, logits and labels are usually of shape `[batch_size, seq_length, ...]` and if one uses dynamic padding, the `seq_length` might change from batch to batch. This made the evaluate and predict methods of the `Trainer` fail, because the Trainer wants to concatenate all predictions in one array.

This PR fixes this, by potentially padding up to the maximum seq length when needed, using the -100 pad index (since it's the marker for ignored labels in PyTorch). It add tests that were failing before the PR, to check the fix actually solves the issue.



Comments:
[s2s] test_distributed_eval
Author: stas00
Pull request: yes

This PR:

* [x] adds a helper function `get_gpu_count()`, which returns the number of available gpus (regardless of whether torch or tf is used) (otherwise we have to do all the `if _torch_available: import torch` rigmarole. 
* [x] adds a basic test for `run_distributed_eval.py`

Things I wasn't sure about:
- I put the new test into `test_seq2seq_examples_multi_gpu.py`, but it doesn't require multigpu. should I create a new test file  `test_seq2seq_examples_gpu.py` or just leave it here for now and refactor once we have more tests like this?
- I made it `@slow`, but wasn't 100% sure - it takes some 25s - probably too slow.

Fixes: #8297

@sshleifer 

Comments:
Fyi the example tests don't currently run on the multi-gpu setup, as pretty much none of the examples tests were made to run on a multi-gpu setup.

We can look into adding them to the CI.

> Fyi the example tests don't currently run on the multi-gpu setup, as pretty much none of the examples tests were made to run on a multi-gpu setup.
> 
> We can look into adding them to the CI.

Absolutely!

Do you prefer to start with a CI job that explicitly lists tests that were converted to run on multi-gpu (3 at the moment) or just run them all? It's all that `require_torch_mutigpu` plus ` that `require_torch_gpu` as it's flexible.

> Do you prefer to start with a CI job that explicitly lists tests that were converted to run on multi-gpu (3 at the moment) or just run them all? It's all that require_torch_mutigpu plus that `require_torch_gpu` as it's flexible.

I would send a PR with whatever you think makes most sense. Or wait until the correct answer is clearer. At this point you know pretty much as much about CI tradeoffs as I do :)

I will prepare a new PR for github CIs.

otherwise I think this PR is complete.
[s2s] 1 GPU test for run_distributed_eval
Author: sshleifer
Pull request: no
Labels: name: Help wanted

Add test coverage for run_distributed_eval.py that can run on 1 GPU.
The command:
```bash
python -m torch.distributed.launch --nproc_per_node=1 run_distributed_eval.py --model_name Helsinki-NLP/opus-mt-en-ro --save_dir opus_wmt_en_ro_gens --data_dir wmt_en_ro
```

works on 1 GPU.


After adding test coverage, we could try to improve API consistency between run_distributed_eval.py and run_eval.py .

Comments:
wdyt @stas00 

I will work on that, thank you.

A minor correction to the command (corrected `data_dir`):
```
python -m torch.distributed.launch --nproc_per_node=2 run_distributed_eval.py --model_name Helsinki-NLP/opus-mt-en-ro --save_dir test_data/opus_wmt_en_ro_gens --data_dir test_data/wmt_en_ro
```

Question: why only 1 gpu? we currently don't have it tested at all.


I thought 1 GPU test coverage would be runnable in current CI/by more users.
But if much easier to test 2 gpu/easy to add test for 2 GPU that is great!

Bottom line - run with as many GPUs as available. 

Thank you for clarifying.
[s2s] test_bash_script.py - actually learn something
Author: stas00
Pull request: yes

As discussed in https://github.com/huggingface/transformers/issues/6049 this PR replaces the original `test_train_mbart_cc25_enro_script` test with mostly new guts doing the following:

* [x] switches to downloading and caching a bigger custom dataset, which is a subset of the full wmt_en_ro.  I created https://cdn-datasets.huggingface.co/translation/wmt_en_ro-tr40k-va0.5k-te0.5k.tar.gz - hope the name is intuitive - self-documenting. It's just 3.6M (vs 56M original). I made it using this script: https://github.com/stas00/porting/blob/master/transformers/translation/make-wmt_en_ro-subset.md

* [x] performs qualitative checks on the eval/test results:
   - Minimum learning requirement 1: BLEU improves over the course of training by more than 2 pts
   - Minimum learning requirement 2: BLEU finishes above 17
   - Minimum learning requirement 3: test bleu and val bleu within ~1 pt. 

also makes some extra improvements:
* [x] uses `require_torch_gpu` decorator
* [x] removes hardcoded paths

Runtime is about ~2min on unoptimized rtx-3090.

Fixes: #6049

@sshleifer 

Comments:
It's ready to merge
Model card: GPT-2 fine-tuned on CommonGen
Author: mrm8488
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Comments:
german medbert model details
Author: smanjil
Pull request: yes
Labels: name: model card

# What does this PR do?

- added details for German MedBERT


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Comments:
Create README.md
Author: yfpeng
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
Create README.md
Author: kuppulur
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Comments:
Muti GPU training with torch==1.7.0 not working
Author: PhilipMay
Pull request: no

Hi,
muti GPU training with torch==1.7.0 is not working.

I have the following configuration:
- CUDA 11.0
- V100 GPU (1 or 4)
- Torch 1.7.0 (1.6.0)
- transformers 3.4.0
- Linux

When I train with one GPU all works. When I run it with 4 GPUs the CPU load goes up to 100% on one core like 1,5 MB memory is used from each GPU but the training does not start. When I downgrade torch to Version 1.6.0 everything works.

Might be an upstream problem but you might want to warn users from using torch 1.7.0

PS: Do you think I should report this at the torch repo?

Comments:
I haven't noticed anything strange on my side, but didn't play much with 1.7 yet. Could you post the code you're using?

> Could you post the code you're using?

That is not possible - it is some company code. If you have some "toy" code maybe you could try it.

I am using an electra base model. Pretokenize everything. Create a torch Dataset for classification (binary).
Then I am using the normal Trainer to train...



On AWS Sagemaker it works with Torch 1.7.0.
I think this might be an NVIDIA Driver probelm on my side...
I am closing this for now. Will reopen if needed.
... sorry ...
torch 1.4.0 transformers segment fault
Author: namenotexist
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->
torch 1.4.0 transformers segment fault
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
Hi, please complete the template, otherwise we cannot help you.