noamaanMulla-03/transformers-issues huggingface dataset:
Why does ALBERT use einsum in PyTorch implementation while in TF one it does not?
Author: marrrcin
Pull request: no

# ❓ Questions & Help
I wanted to learn internals of ALBERT model from your implementation (which is BTW really clean in comparison to the original one - good job!), but I've stumbled upon weird looking part in the `AlbertAttention`: https://github.com/huggingface/transformers/blob/6af3306a1da0322f58861b1fbb62ce5223d97b8a/src/transformers/modeling_albert.py#L258

Why does PyTorch version use `einsum`-based notation while calculating hidden state (with manual usage of `dense` layer's weights), while the TensorFlow version just reshapes the `context_layer` and does standard "forward" on dense layer?

https://github.com/huggingface/transformers/blob/6af3306a1da0322f58861b1fbb62ce5223d97b8a/src/transformers/modeling_tf_albert.py#L296

I would really like to know the explanation of this implementation - @LysandreJik cloud you shed some light here?


Comments:
The two implementations are equivalent, but the  Pytorch version  is cumbersome.
I think  the code

```python
        context_layer = context_layer.permute(0, 2, 1, 3).contiguous()

        # Should find a better way to do this
        w = (
            self.dense.weight.t()
            .view(self.num_attention_heads, self.attention_head_size, self.hidden_size)
            .to(context_layer.dtype)
        )
        b = self.dense.bias.to(context_layer.dtype)

        projected_context_layer = torch.einsum("bfnd,ndh->bfh", context_layer, w) + b
```
should be rewritten by

```python
        context_layer = context_layer.permute(0, 2, 1, 3).contiguous()
        new_shape = context_layer.size()[:-2] + (-1,)
        context_layer = context_layer.view(*new_shape)

        projected_context_layer = self.dense(context_layer)
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Ping @LysandreJik 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Bump

Ping @LysandreJik again

For no particular reason, but it might not have been the best choice according to [this thread on performance](https://github.com/huggingface/transformers/issues/6771).

OK, thanks.
Free Inference API Not Accessible
Author: ncoop57
Pull request: no

Hi, I attempted to use the free version of the Model Hub's Inference API. However, it is not working for me anymore:

![image](https://user-images.githubusercontent.com/7613470/95273777-e7b81800-0811-11eb-8090-800188be768f.png)

I do have an account and I am signed in when I get the above message. I also have done my email verification as well. Also, when I try to send a POST request using curl with my API token, I get a 503 error.

I was able to successfully use the free version on October 4th, 2020 and so I was wondering, is this a bug or is the free version no longer available?

Comments:
Hi Nathan, can you send us a quick email to api-enterprise@huggingface.co? 

The free version of the Hub's Inference API is still up, but maybe you've hit the rate limiting?

Resolved. Ended up being a misinterpretation of my part on error codes that HF's API produces.
Trainer callbacks
Author: sgugger
Pull request: yes

# What does this PR do?

This PR does two things: clean up a bit the files supporting `Trainer` and the `Trainer` class, and add callbacks to `Trainer`.

### Callbacks

This PR introduces a new class called `TrainerCallback` that can access the current state of the training loop and make some decisions (shown in the `TrainerControl` object). This allows us to isolate the pieces of code that do log-reporting on the various ML platforms or report progress in another file and clean up the code of the main `train` method of the `Trainer`.

This way, any new platform we want to integrate with for log-reporting or new behavior (like early stopping) can be implemented in a Callback while `Trainer` focuses on the main aspects of actual training, with or without mixed precision, on one or several GPUs/TPUs.

As an example, integrations with TensorBoard, Wandb and ComeML are moved to the `integrations` module in clean callbacks, while the control flow of logs/saves/evaluations as well as progress reporting are moved to the `trainer_callback` file.

Most of the behavior stays the same as this PR essentially moves code around, but there are a few API changes:
- deprecating the `tb_writer` argument in `Trainer` (with full backward compatibility), people should now use the `TensorBoardCallback`.
- a new `callbacks` argument in the `Trainer` init and new `add_callback`, `pop_callback` and `remove_callback` for the `Trainer`. For all of those, you can either pass an instance of a callback or a callback class.
- Cleaned up the progress bars a bit with only one main progress bar over all the steps we will do for training and evaluation bars that disappear after being done

### Progress bars

Here is the new progress bar behavior in console mode (checked in single and multi GPU envs, to make sure only one progress bar is displayed/logs are only printed once):

![](https://i.ibb.co/Fq60bFS/console-progress.png)

and in a jupyter notebook:

![](https://i.ibb.co/kSzs1fC/notebook-progress.png)

### General cleanup

Not directly related to this PR, but related to the general cleanup of `Trainer`, I moved a bit of stuff around: moved the utils at the start of `Trainer` to a new `trainer_utils_pt`. This way `trainer_utils` can be about the general training utils that work on both PyTorch and TensorFlow, and I moved the ones specific to PyTorch to `trainer_utils_pt`.

Also in `Trainer`, the code for logs, save and evaluation ended being duplicated between the end of a training step and the end of an epoch, so I put it in its private method to improve readability.

Comments:
Finally! Can't wait for this PR to be merged.
I've briefly looked at the code and from my understanding it should support this case, but correct me if I'm wrong:

_When saving checkpoints (including model weights as well as scheduler and optimizer states), I will be able to attach to this process and store the checkpoint in some external repository (i.e GCS / W&B artifact)_,

right?

Yes, you will be able to inject custom behavior to the saved checkpoint with the `on_save` event.
Update model card - Fix arxiv link
Author: iliaschalkidis
Pull request: yes
Labels: name: model card

Minor changes: Add arxiv link + Layout improvement + fix typos

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
Update README.md
Author: dartrevan
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
[model_card] bert-base-5lang-cased
Author: amineabdaoui
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR adds the model card of [amine/bert-base-5lang-cased](https://huggingface.co/amine/bert-base-5lang-cased). A smaller version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handles only 5 languages (en, fr, es, de and zh) instead of 104 while reducing its size by 30%.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
Added model cards for Tagalog BERT models
Author: jcblaisecruz02
Pull request: yes
Labels: name: model card

# What does this PR do?
Adds model cards for five Tagalog BERT models:
* jcblaise/bert-tagalog-base-cased
* jcblaise/bert-tagalog-base-uncased
* jcblaise/bert-tagalog-base-cased-WWM
* jcblaise/bert-tagalog-base-uncased-WWM
* jcblaise/distilbert-tagalog-base-uncased


Comments:
Thanks!
Fix Failing Slow tests
Author: sshleifer
Pull request: no

```
FAILED tests/test_modeling_bart.py::BartHeadTests::test_tokenization - Assert...
FAILED tests/test_modeling_bart.py::BartModelIntegrationTests::test_mnli_inference
FAILED tests/test_modeling_blenderbot.py::Blenderbot3BIntegrationTests::test_generation_from_short_input_same_as_parlai_3B
FAILED tests/test_modeling_blenderbot.py::Blenderbot90MIntegrationTests::test_90_generation_from_long_input
```

Comments:
Fix 3 failing slow bart/blender tests
Author: sshleifer
Pull request: yes

3 of these were simple fixes.
+ 1 typo in blenderbot
+ 2 BART failures changed by new `assert_tensors_close` helper fn checking shapes more aggressively. Output shapes have not changed.

The fourth failure is a bit harder to verify
Blenderbot 3b was OOMing
before fix: 11.4 GB
After: 6.4 GB
Why:  going to fp16 before going to cuda

Comments:
I don't think the change can possibly harm, so I will merge without review. cc @sgugger @LysandreJik
Create Model Card For "abhilash1910/french-roberta" Model
Author: abhilash1910
Pull request: yes
Labels: name: model card

# Model Card (Roberta MLM on French News Corpus)

Model Card for [abhilash1910/french-roberta](https://huggingface.co/abhilash1910/french-roberta). Contains the model specification and important links which helped me create this . It uses the Roberta MLM on French New corpus (extracted from Leipzig).



Comments:
Thanks for sharing!

Thank you @julien-c !
Green tests: update torch-hub test dependencies (add protobuf and pin tokenizer 0.9.0-RC2)
Author: thomwolf
Pull request: yes

# What does this PR do?

Update the torch-hub CI test dependencies to add protobuf and pin tokenizer on 0.9.0-rc2 until final release.

## Who can review?

@sgugger @n1t0 

Comments:
Thanks a lot!
fail to download vocabulary behind proxy server
Author: jingjingli01
Pull request: no

## ❓ Questions & Help
I work behind a proxy server. Following this [issue](https://github.com/huggingface/pytorch-transformers/issues/856), I manually download the `config.json` and `pytorch_model.bin` and the model can successfully load config and model weights.

However, in running `tokenizer = BertTokenizer.from_pretrained('bert-base-cased', do_lower_case=False)`,

I get:
INFO:pytorch_transformers.file_utils:https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-vocab.txt not found in cache, downloading to /tmp/tmpkat40bei
ERROR:pytorch_transformers.tokenization_utils:Couldn't reach server to download vocabulary.

If I download it manually, where should I put this vocab.txt? 
Thanks!


Comments:
Put it in the same directory as your config.json file

Which directory is it by default?

> Which directory is it by default?

Have you found the solution?
T5 Beam search num_beans always equals 1
Author: marcoabrate
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.3.1
- Platform: Debian 10.6
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6
- Tensorflow version (GPU?): N.A.
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.  -->
TextGeneration: @TevenLeScao 
 T5: @patrickvonplaten

## To reproduce

Steps to reproduce the behavior:

1. load T5 model and tokenizer
```
from transformers import T5ForConditionalGeneration, T5Tokenizer

# initialize the model architecture and weights
model = T5ForConditionalGeneration.from_pretrained("t5-base")

tokenizer = T5Tokenizer.from_pretrained("t5-base")
```
2. prepare input for summarization. i guess the error persists for any given generation task but did not try.
```
article = """article etc etc"""
inputs = tokenizer.encode("summarize: " + article,
                          return_tensors = "pt",
                          max_length = 512, truncation = True)
```
3. attempt beam search
```
model.config.update({"num_beans": 4})
print(model.config.num_beans)
# output is 4 as expected

outputs = model.generate(inputs,
                         max_length = 200,
                         min_length = 100,
                         length_penalty = 5,
                         num_return_sequences = 2,
                         early_stopping = True)
```
or
```
outputs = model.generate(inputs,
                         max_length = 200,
                         min_length = 100,
                         length_penalty = 5,
                         num_beams = 4,
                         num_return_sequences = 2,
                         early_stopping = True)
```

error:

> AssertionError: Greedy decoding will always produce the same output for num_beams == 1 and num_return_sequences > 1. Please set num_return_sequences = 1

as if num_beans == 1, but we set num_beans to 4.

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
the generate function should execute beam search with 4 beams without errors


Comments:
Hey @marcoabrate, 

please make sure that you use the correct parameter name `num_beams` instead of `num_beans`. 
When using `num_beams`, I cannot reproduce your error.

of course it was that!

thank you
Seq2SeqTrainer: missing features
Author: sshleifer
Pull request: no

These could all be separate issues, if you want to tackle 1 feel free to make a new issue to link to your PR, or not!

1. Configure lr scheduler from the command line https://github.com/huggingface/transformers/blob/master/examples/lightning_base.py#L119
2. Configure dropout, layerdrop from the command line: https://github.com/huggingface/transformers/blob/master/examples/lightning_base.py#L92
3. Logging to wandb seems much less frequent than with the PL integration, which sends train loss to wandb every step.
4. Losses printed out are different than with PL, (seem to be normalized in some way). This merits investigation.
cc @patil-suraj 

Comments:
@sshleifer 

2. Done #7532
3. logging can be controlled using `--logging_steps`, default is 500
4. I've also observed this, PL does something different I guess, final metrics should be same IMO