noamaanMulla-03/transformers-issues huggingface dataset:
New TF loading weights
Author: jplu
Pull request: yes

# What does this PR do?

This PR improves the way we load the TensorFlow weights. Before we had to go through the instantiated model + the checkpoint twice:
- once for loading the weights from the checkpoints into the instantiated model
- once for computing the missing and unexpected keys

Now both are done simultaneously which makes the loading faster.

Comments:
I have added a lot of comments in the method to make it clearer, I removed a small part of the code that was due to the moment where I was updating to the new names in same time. @LysandreJik @sgugger it should be easier to understand now.

It's a lot clearer, thanks. There are still unaddressed comments however, and I can't comment on line 259 but it should be removed now (since the dict is create two lines below).

What is missing now?

There is Lysandre's comments at line 283 and mine about the loop line 277. Like I said in my previous comments, doing the two functions in one is great, I just don't get the added complexity of the new `model_layers_name_value` variable when we could stick to the previous loop in the function `load_tf_weights` while adding the behavior of `detect_tf_missing_unexpected_layers`.

The comments are a great addition, thanks a lot for adding those!

I have addressed the Lysandre's comment at line 283 and yours for the loop at line 277. Do you see anything else?

The typos should be fixed now. Sorry for that.

Good to merge for me too!
Model Card for abhilash1910/financial_roberta
Author: abhilash1910
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
Add model card for ai4bharat/indic-bert
Author: divkakwani
Pull request: yes
Labels: name: model card

# What does this PR do?

This PR adds model card for IndicBERT model (shortcut name: `ai4bharat/indic-bert`)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Also:  @julien-c (model cards)


Comments:
really cool, thanks for sharing
[s2s] distillation apex breaks return_dict obj
Author: stas00
Pull request: yes

This is a continuation of https://github.com/huggingface/transformers/pull/8612 for `distillation.py` - this PR is switching from `.property` to `["property"]`.

Unfortunately, the script itself doesn't seem to work under apex even after the fix - perhaps it never was. 

But it's probably still OK to merge, since it no longer fails with the #8530-related symptoms and is in sync with `finetune.py` now.

I filed a separate issue about it: https://github.com/huggingface/transformers/issues/8632

@sgugger 

Comments:
grammar
Author: cronoik
Pull request: yes

# What does this PR do?
Fixes a typo in the pull request template.

<!-- Remove if not applicable -->

## Before submitting
- [x ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

 documentation: @sgugger



Comments:
Small formatting fix
Author: timpal0l
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Just adding the bash formatting for the markdown in the run_mlm_wwm.py snippet


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 --> documentation: @sgugger

Comments:
Updated the Extractive Question Answering code snippets
Author: cronoik
Pull request: yes

# What does this PR do?
The Extractive Question Answering code snippets do not work anymore since the models return task-specific output objects. This commit fixes the pytorch and tensorflow examples but adding `.values()` to the model call.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

documentation: @sgugger


Comments:
Thanks for flagging this! I think it would be better to show how to use the attributes, so something like:
```
outputs = model(**inputs)
answer_start_scores = outputs.start_logits
answer_end_scores = outputs.end_logits
```

Yes, you are right. @sgugger
Create README.md
Author: fajri91
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Adding model card for `indolem/indobert-base-uncased` 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
Thanks for sharing, looks really cool!
RAG: OSError: Can't load tokenizer for 'facebook/rag-sequence-nq/question_encoder_tokenizer'
Author: shamanez
Pull request: no

Hi I am trying to run [use_own_knowledge_dataset.py] (https://github.com/huggingface/transformers/blob/master/examples/rag/use_own_knowledge_dataset.py)) with **Transformers  Version: 3.5.1** ((from your latest [PR](https://github.com/huggingface/transformers/pull/8585)).  But it gives the following error.

```

OSError: Can't load tokenizer for 'facebook/rag-sequence-nq/question_encoder_tokenizer'. Make sure that:

- 'facebook/rag-sequence-nq/question_encoder_tokenizer' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'facebook/rag-seq
```uence-nq/question_encoder_tokenizer' is the correct path to a directory containing relevant tokenizer files


Comments:
Can you try installing transformers from master?

Thanks for working perfectly when loading from sources.
Add cards for all Geotrend models
Author: amineabdaoui
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Adds model card


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
Awesome, thank you!
Fix missing return_dict in RAG example to use a custom knowledge source
Author: lhoestq
Pull request: yes

We did some changes regarding the output of models but didn't change the return_dict parameter of this RAG example script.
It works as expected now

Comments:
@lhoestq 

What the version of transformers you have used in this PR? 

Hi !
The one on the master branch

> Hi !
> The one on the master branch

So you have installed from sources, right **(Version: 4.0.0.dev0)**? I tried to execute **use_own_knowledge_dataset.py** with your [previous PR](https://github.com/huggingface/transformers/pull/8585).  But I got the following error.  Seems like **question_enc_outputs** is not a dict but just the tensor. 

`/transformers/src/transformers/models/rag/modeling_rag.py", line 628, in forward
    question_enc_hidden_states = question_enc_outputs.hidden_states
AttributeError: 'tuple' object has no attribute 'hidden_states'`




Thanks for letting me know ! I'll fix that as well :) 

Perfect. Btw it works perfectly for version 3.4.0.

On Thu, Nov 19, 2020, 23:20 Quentin Lhoest <notifications@github.com> wrote:

> Thanks for letting me know ! I'll fix that as well :)
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/pull/8653#issuecomment-730274786>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AEA4FGQVTDUZUCDJWDMROG3SQTWQBANCNFSM4T3EZTTA>
> .
>
`disable_ngram_loss` doesn't work correctly in ProphetNetForConditionalGeneration
Author: Zhylkaaa
Pull request: no

When I am using ProphetNet with `disable_ngram_loss=True` I am getting loss that is greater than with `disable_ngram_loss=False`. It seems to me that this is the problem of setting `fill_(self.padding_idx)` in `_compute_loss` instead of -100 so that ngram part is omitted in loss calculation 

Also I think that `loss = F.nll_loss(lprobs, expend_targets.view(-1), reduction="sum")` reduce should be set to `mean` so that model loss is comparable between models working on the same task (like `mbart`). Can somebody tell me if it's a good point or should I leave it as it is? I am planning to add PR with this changes.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.5.0 from source
- Platform: macOS Catalina
- Python version: 3.7.5
- PyTorch version (GPU?): 1.6.0

### Who can help

I can't figure out whom to tag.

## Information

Model I am using (Bert, XLNet ...): ProphetNetForConditionalGeneration

## To reproduce

```
from transformers import XLMProphetNetTokenizer, XLMProphetNetForConditionalGeneration
tokenizer = XLMProphetNetTokenizer.from_pretrained('microsoft/xprophetnet-large-wiki100-cased')
model = XLMProphetNetForConditionalGeneration.from_pretrained('microsoft/xprophetnet-large-wiki100-cased')
inputs = tokenizer('Hi my name is', return_tensors='pt').input_ids
targets = tokenizer('Hi my name is John', return_tensors='pt').input_ids

model_loss = model(input_ids=inputs, labels=targets, return_dict=True).loss

model.disable_ngram_loss = True
model_disable_loss = model(input_ids=inputs, labels=targets, return_dict=True).loss

from torch.nn import CrossEntropyLoss

loss_fct = CrossEntropyLoss(reduction='sum')
logits = model(input_ids=inputs, labels=targets, return_dict=True).logits
loss_cross_entropy = loss_fct(logits.view(-1, model.config.vocab_size), targets.view(-1))
```
the problem is `model_loss < model_disable_loss` and `model_disable_loss != loss_cross_entropy` which it should be I think.

Note: 
`CrossEntropyLoss(reduction='sum')` is used to match implementation in `_compute_loss` (`loss = F.nll_loss(lprobs, expend_targets.view(-1), reduction="sum")`) but other models use default reduction which makes outputs incomparable (at least directly)

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
when `model.disable_ngram_loss=True` `CrossEntropyLoss` should be equal to `model(input_ids=inputs, labels=targets, return_dict=True).loss`

Comments: