noamaanMulla-03/transformers-issues huggingface dataset:
Create README.md
Author: ghost
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
Update model card
Create README.md
Author: ganeshkharad2
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Comments:
Fix PreTrainedTokenizer.pad when first inputs are empty
Author: sgugger
Pull request: yes

# What does this PR do?

Currently, `PreTrainedTokenizer.pad` errors when the first `input_ids` are empty (because it tries to guess the type of the tokens by looking at the first element). This PR slightly changes the behavior to loop until we find a non empty list.

Fixes #8674 (not the initial issue but the one mentioned at the end)

Comments:
Remove docs only check
Author: LysandreJik
Pull request: yes

Remove the docs only check as it can result in [crashes](https://app.circleci.com/pipelines/github/huggingface/transformers/17209/workflows/d3807ea6-9697-4699-a114-98e6b4d2c4d0/jobs/135576).

Will revert if you disagree @stas00. 

Comments:
Bump notebook from 6.1.4 to 6.1.5 in /examples/research_projects/movement-pruning/lxmert
Author: dependabot[bot]
Pull request: yes
Labels: name: dependencies

Bumps [notebook](https://github.com/jupyter/jupyterhub) from 6.1.4 to 6.1.5.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/jupyter/jupyterhub/commits">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=notebook&package-manager=pip&previous-version=6.1.4&new-version=6.1.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/huggingface/transformers/network/alerts).

</details>

Comments:
Make ProphetNetModel really compatible with EncoderDecoder
Author: patrickvonplaten
Pull request: yes

The interesting part of ProphetNet is its decoder which can do n-gram causal language modeling. So it could be very interesting to load a pre-trained prophetnet decoder model into an encoder-decoder design with - let's say - a longformer encoder for long-range sequence modeling. 

Due to some narrow-minded thinking on my part, this didn't work previously.

```python
from transformers import EncoderDecoderModel

EncoderDecoderModel.from_encoder_decoder_pretrained("allenai/longformer-large-4096", "microsoft/prophetnet-large-uncased")
```

As one can see none of pre-trained **decoder** weights are loaded into the model. The reason is because `ProphetNetForCausalLM` was badly modularized in `ProphetNetForCausalLM`.

Merging this PR would make it possible to load any prophetnet decoder into an encoder-decoder model and fine-tuning an "build-it-yourself" encoder decoder would become much easier, *e.g.*:

```python
from transformers import EncoderDecoderModel
import torch

model = EncoderDecoderModel.from_encoder_decoder_pretrained("allenai/longformer-large-4096", "microsoft/prophetnet-large-uncased")

input_ids = torch.tensor([10 * [1]])
labels = torch.tensor([10 * [0]])
loss = model(input_ids, decoder_input_ids=labels, labels=labels).loss
loss.backward()
```

The above use-case might also be interesting for @ibeltagy actually.

## Breaking changes

This does introduce a pretty heavy breaking change to `ProphetNetForCausalLM`. However, the only reason this class was created was to make it useable with `EncoderDecoderModel` and this arguably failed a bit the first time since it made it way too difficult to load pretrained ProphetNet models into the `EncoderDecoderModel`. I guess I see this more of solving a bug then "new design". Also there are no pre-trained `ProphetNetForCausalLM` models on the model hub and I highly doubt anybody has really used this class.

I want to use the same pattern for BartForCausalLM and T5ForCausalLM, so it'd be great to get this merged even though there are some breaking changes.

Comments:
Update quicktour docs to showcase the use of truncation
Author: navjotts
Pull request: yes

# What does this PR do?
Currently, running the tokenizer batch example on https://huggingface.co/transformers/quicktour.html gives an error 

```
Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.
```

This PR fixes the above by passing the `max_length` param explicitly (instead of depending on it having a default, which might not be the case for all models).

The fix also adds clarity to the statement in the docs above this example
> If your goal is to send them through your model as a batch, you probably want to pad them all to the same length, truncate them to the maximum length the model can accept and get tensors back

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@sgugger 


Comments:
Having to specify too many `ignore_keys` in `Trainer.prediction_step`
Author: Fraser-Greenlee
Pull request: no

Since all model output dicts that have logits give it with the key `logits` I think this code could be simplified to just use the `logits` key. (Rather than having to specify a bunch of `ignore_keys`.)

from:
https://github.com/huggingface/transformers/blob/e20ac6611df97f66148ce8b7886f01ffe9d17484/src/transformers/trainer.py#L1471-L1473
to:
```python
 if isinstance(outputs, dict): 
     loss = outputs["loss"].mean().detach() 
     logits = (outputs.get('logits', None),)
```

This prevents other keys for being sent to `nested_concat` causing an error:
https://github.com/huggingface/transformers/blob/e20ac6611df97f66148ce8b7886f01ffe9d17484/src/transformers/trainer.py#L1367-L1368

I'd be happy to make this change, let me know if I'm missing something here.

Comments:
No, this is too simplistic. First of all, all question-answering  models return two logits called `start_logits` and `end_logits`. Then a user might want to get the predictions for all their `all_hidden_states` or `all_attentions` when the model has the proper config keys, which is why the `Trainer` gather all the tensors different from the loss.

Ahh I see, guess there's no simple answer here. Thanks for the info!
Using BERT for spelling correction
Author: naturecreator
Pull request: no
Labels: name: wontfix

I am currently working on the task of spelling correction. I used BERT by masking the misspelled word to get predictions with their probability score. However, the results are not so good. Hence, I want to fine-tune the BERT. For this spelling correction task, I would like to know which is the suitable method to fine-tune the BERT. 

I would be glad if anyone could help me in this regard. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
RAG - reproducing RAG-Sequence QA score 
Author: acslk
Pull request: no
Labels: name: wontfix

I'm trying to reproduce RAG-Sequence NQ score of 44.5 presented in Table 1 of the paper at https://arxiv.org/abs/2005.11401.

I used the command in the examples/rag readme
```bash
python examples/rag/eval_rag.py \
    --model_name_or_path facebook/rag-sequence-nq \
    --model_type rag_sequence \
    --evaluation_set path/to/test.source \
    --gold_data_path path/to/gold_data \
    --predictions_path path/to/e2e_preds.txt \
    --eval_mode e2e \
    --gold_data_mode qa \
    --n_docs 5 \
    --print_predictions \
    --recalculate \
```

For gold_data_path I used data.retriever.qas.nq-test from DPR repo, consisting of 3610 questions and answers: https://github.com/facebookresearch/DPR/blob/master/data/download_data.py#L91-L97

For evaluation_set, my understanding it should be the questions, so I extracted just the questions from the qas.nq-test csv file.

I tried the above command with n_docs 5 and 10, with the following results:

n_docs 5
INFO:__main__:F1: 49.67
INFO:__main__:EM: 42.58

n_docs 10
INFO:__main__:F1: 50.62
INFO:__main__:EM: 43.49

With n_docs 10 it's still 1 point below the score in paper. What would be the proper setup to reproduce the number, is the pretrained model loaded different, higher n_docs, or different test data?

Thanks in advance!

Comments:
Gently pinging @ola13 here, she probably knows best which command to run to reproduce the eval results :-) 

Hi @acslk, thanks for your post!

You should be able to reproduce paper results for the RAG Token model (44.1 EM on NQ) by evaluating `facebook/rag-token-nq` with 20 docs.

As for the RAG Sequence model - we have lost some quality when translating the checkpoint from `fairseq` (the experimentation framework we used to obtain the original paper results) to HuggingFace. We are now working on replicating the paper numbers in HF and we'll update the official `facebook/rag-sequence-nq` model weights once we have that so stay tuned!

Thanks for the response, I tried the command above with RAG Token model and n_docs 20 on NQ test set and can confirm it matches paper results:
INFO:__main__:F1: 51.44
INFO:__main__:EM: 44.10

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Loading saved model not working
Author: irocks04
Pull request: no
Labels: name: wontfix

Do we know do to load the saved model pipeline back up and make predictions again locally? The from_pretrained()is not working.
Pls provide few instructions how to load the model using from pretrained

Comments:
What do you mean it's not working? Could you provide all the information required in the template? What is your environment? What code are you running? What is the error shown? How do you use `from_pretrained`?

What is meant is how do we utilize the “from_pretrained” functionality after saving the pipeline? I want to load the pipeline back up to start making predictions locally the saved file(s) below:


import transformers

ner_original = pipeline("ner")
ner = pipeline("ner",grouped_entities=True)

path = 'path to folder'

ner.save_pretrained(path)

I am currently running transformers==2.11.0 and python 3.7.4. I have tried:

pipe = transformers.pipeline(task="ner", model="pytorch_model.bin",
tokenizer="tokenizer_config.json")

This gave an error: ValueError: Unrecognized model in tokenizer_config.json. Should have a `model_type` key in its config.json

pipe = transformers.TokenClassificationPipeline(model="pytorch_model.bin",
tokenizer="tokenizer_config.json")

This gave an error: AttributeError: 'str' object has no attribute 'config'

Changing the tokenizer to "config.json" yielded the following error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte


This is the code which gives the error

from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(path)
model = AutoModel.from_pretrained(path)

label_list = [
    "O",       # Outside of a named entity
    "B-MISC",  # Beginning of a miscellaneous entity right after another miscellaneous entity
    "I-MISC",  # Miscellaneous entity
    "B-PER",   # Beginning of a person's name right after another person's name
    "I-PER",   # Person's name
    "B-ORG",   # Beginning of an organisation right after another organisation
    "I-ORG",   # Organisation
    "B-LOC",   # Beginning of a location right after another location
    "I-LOC"    # Location
]

sequence = "Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO, therefore very" \
           "close to the Manhattan Bridge."

# Bit of a hack to get the tokens with the special tokens
tokens = tokenizer.tokenize(tokenizer.decode(tokenizer.encode(sequence)))
inputs = tokenizer.encode(sequence, return_tensors="pt")

outputs = model(inputs)[0]
predictions = torch.argmax(outputs, dim=2)

print([(token, label_list[prediction]) for token, prediction in zip(tokens, predictions[0].tolist())])


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Turning the SQuAD dataset class into an iterator to save ram and redistribute time
Author: mariusjohan
Pull request: no
Labels: name: wontfix

# 🚀 Feature request
When I'm using the SquadDataset class I sometimes run out of RAM in my Colab session. It also takes a lot of time to process the data, so if we could just preprocess the data when using the __getitem__. I think we should turn the examples into an iterator rather than a list. And when using the __getitem__ we should then convert the example to a feature

## Motivation
As I mentioned, this can save a lot of ram and redistribute the time, so we do the preprocessing simultaneously.

When I run this snippet in Google Colab it runs out of memory even though I've 12 GB of ram
```python
train_dataset = SquadDataset(
    args=data_args,
    tokenizer=tokenizer,
    cache_dir=model_args.cache_dir)
```

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->



## Your contribution
I can pretty much code all of it we you believe it would be a necessary feature
<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Uber AI plug and play language model (PPLM)
Author: ajay01994
Pull request: no

Hi Team,

Thanks for the hugging face repo and appreciate  your great efforts towards adding datasets and models, I was trying to find PPLM model in model page but it was shown as 404 error, could you please check the model  if it's available and let me know 

Thanks 

Ajay  

Comments:
I don't think we host a specific pplm model in the model hub (cc @julien-c). 

Also we don't really plan on continuing to support PPLM in the future

@ajay01994 are you looking for the code? https://github.com/huggingface/transformers/tree/master/examples/text-generation/pplm

Or you might be referring to https://transformer.huggingface.co/model/pplm – but indeed we don't host the inference anymore (cc @LysandreJik) as it was a bit costly to support.

Thanks for your quick reply.Actually the PPLM model is having issues due to use of transformers lib - 3.1.0 which is old and thus need certain changes to upgrade. Do you know any better or similar  model than PPLM for controlling text in GPT-2 ? that would be of great help 

Regards 

Ajay 

not at the top of my head, but maybe @mimosavvy or @w4nderlust knows!

ok....closing this for now ,thanks for your help :)

> Thanks for your quick reply.Actually the PPLM model is having issues due to use of transformers lib - 3.1.0 which is old and thus need certain changes to upgrade. Do you know any better or similar model than PPLM for controlling text in GPT-2 ? that would be of great help
> 
> Regards
> 
> Ajay

Can you be more specific about the issues? There has been a PR that should have solved the change to dictionary as returns to the model.
[Transfo-XL] Impossible to pass `attention_mask` to model
Author: patrickvonplaten
Pull request: no
Labels: name: wontfix

It is not possible to forward an `attention_mask` to transfo-xl because Transfo-XL's forward function does not accept an `attention_mask`. This makes it impossible to do batch generation with Transfo-XL for example.

IMO, this could be implemented quite easily.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.