noamaanMulla-03/transformers-issues huggingface dataset:
MPNet: Masked and Permuted Pre-training for Natural Language Understanding
Author: StillKeepTry
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
New TF model inputs
Author: jplu
Pull request: yes

# What does this PR do?

This PR improves the way the inputs are handled in the TensorFlow models:

- It should be easier now to write a TensorFlow model
- The order of the inputs are now better handled, mostly when using Keras Tensors
- Replace all the occurrences of `inputs` by `input_ids` to make easier to understand what this parameter is for and to align with PyTorch input names and the Tokenizers outputs.

@LysandreJik @sgugger @patrickvonplaten let me know what you think about this new input processing, it is not finished yet but any comment will be helpful for me :)

Comments:
@sgugger Thanks for these useful comments, I should have addressed them as you proposed. All the modification have been directly integrated in the new `input_processing` function. can you please also check the `modeling_tf_bart.py`, I have updated the documentation, let me know if I did something wrong.

@patrickvonplaten can you please check the same BART file as I have done some updates in order to make it able to properly handle the `return_dict=True`  and `return_dict=False` my updates don't affect the usual behaviour of the model even if the tests are ok.

@LysandreJik @sgugger @patrickvonplaten Do you think it would be interesting to have the same thing for the outputs? knowing that for graph mode compliance we will need to have such or such output behaviour depending the state we are (eager or not). 

> Remember to do separate PRs for separate issues please, a lot of changes in TFBart here are unrelated to the main focus of the PR.

Do you prefer that I move the Bart changes into another PR and keep only the inputs changes? I don't mind to do this :)

I have also updated the TF template.

> Do you prefer that I move the Bart changes into another PR and keep only the inputs changes? I don't mind to do this :)

Since @patrickvonplaten approved, I think they're okay here for this time (unless he says otherwise ;-) ) 

I would prefer the same, it looks a bit too big to be added at this point of the release. I should have addressed all the comments. Let me know if I missed some.
Big model table
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds a big table in the first page of the doc, indicating whether each of our models has support for a slow/fast tokenizer, PyTorch, TensorFlow and Flax. Result can be found [here](https://125258-155220641-gh.circle-artifacts.com/0/docs/_build/html/index.html) (scroll a bit down).
It is updated automatically via `make fix-copies` and checked for updates in `make quality`, being built form the content of the auto models module.

There were a few issues with the imports on the flax side that I fixed in passing, and I renamed a constant to add the `FLAX` prefix. @mfuntowicz this doesn't really change anything but pinging you just so you're aware.

Comments:
Prophetnet - predicted n-future tokens
Author: nsankar
Pull request: no

Hi,

How can we get the predicted n-future tokens **as a string** data from the model output. I couldn't find it in the API doc and sample code. Could you please guide / provide the code snippet for ProphetNet and XLM-Prophetnet ? Thanks in advance.

Comments:
Hey @nsankar - that's a good question for the forum https://discuss.huggingface.co/ I think. Currently we don't really support sampling from the "n-future" tokens, but it should not be too difficult - just sample from each of the 1...n logit vectors and then you can use the tokenizer to decode the sampled tokens. So in short, you should do this operation: https://github.com/huggingface/transformers/blob/42111f1d56947797d9dfb0908908f42a22ca9823/src/transformers/generation_utils.py#L843 for all n-future logit vectors and then pass it to the tokenizer.
Attempt to get a better fix for QA
Author: Narsil
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
Fix QA argument handler
Author: LysandreJik
Pull request: yes

The QA argument handler does not handle multiple sequences at a time anymore. This was not tested, so I added it to the tests.

Fix https://github.com/huggingface/transformers/issues/8759

To test it out run the following on `master`:

```py
from transformers import pipeline

nlp = pipeline("question-answering")

context = r"""
Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune
a model on a SQuAD task, you may leverage the `run_squad.py`.
"""

print(
    nlp(
        question=["What is extractive question answering?", "What is a good example of a question answering dataset?"],
        context=[context, context],
    )
)
```

Comments:
The errors are due to connection errors.
Possible to add additional features as input to TFBertForSequenceClassification?
Author: brandonbell11
Pull request: no

Say I have a binary classification problem, but in addition to the sentence I'd like to also input some scalar value as well. Is it possible to just tack on this scalar as input to the last linear layer of BERT?

For example, I'd like to detect if a particular sentence is from my source data or generated. And I know that many instances of a repeated word increases the likelihood that it is a generated sentence. So I'd like to pass the sentence itself into BERT as well as a scalar feature such as the number of unique words in the sentence. 

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
Create README.md
Author: ghost
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
Confidence score prediction of pretrained models in extractive QA - similar to pipeline
Author: govindraj513
Pull request: no
Labels: name: wontfix

How can we calculate the confidence of a single sentence predicted from extractive question answering using autotokenizer. we will get a score using the pipeline method for a sentence, but what we get from the extractive qa are answer_Start_scores and answer_end_scores. How can we get one single score ?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Have you got your answer? I got the same question..
Running squad_convert_examples_to_features causes warnings.
Author: dominichillier
Pull request: no
Labels: name: wontfix

Running function squad_convert_examples_to_features from data/processors/squad.py causes warning. Very annoying to disable warnings when running function many times :)

Warned that tokenization_base_utils.py, max_len is depricated. 
`tokenization_utils_base.py:1320: FutureWarning: The max_len attribute has been deprecated and will be removed in a future version, use model_max_length instead.`

Code to reproduce.
```python
from transformers import AutoTokenizer
from transformers.data import SquadExample, squad_convert_examples_to_features
tokenizer =  AutoTokenizer.from_pretrained('a-ware/roberta-large-squadv2')
example = SquadExample(None,'what is test','test is good',None,None,None)
features = squad_convert_examples_to_features(
            examples=[example],
            tokenizer=tokenizer,
            max_seq_length=512,
            doc_stride=128,
            max_query_length=64,
            is_training=False,
            tqdm_enabled=False
        )
```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Found this issue looking for the same problem.

I'm using a `question_answering` pipeline (which call the `squad_convert_examples_to_features` under the hood) and it's really annoying when doing an information retrieval system to get the warning each time the QnA infrence is made... 

I tried to disable it with `logging.getLogger("transformers.tokenization_utils_base").setLevel(logging.ERROR)` without success.

Do you have any clues how to disable it or to change the code so it's not deprecated anymore ? 

Thanks in advance

The version v4.0.0-rc-1 that will be released today or tomorrow will not have this warning anymore.

You can also disable this warning with

```python
import warnings
warnings.simplefilter("ignore")
```
Sentiment Analysis Pipeline is predicting incorrect sentiment in 2.9
Author: mabu-dev
Pull request: no
Labels: name: wontfix

# 🐛 Bug
Sentiment Analysis Pipeline is predicting incorrect sentiment. Everything seems to be NEGATIVE. 

## To reproduce

Steps to reproduce the behavior:
Left a Colab Notebook here for reproducing: https://colab.research.google.com/drive/1hIwyBOGFpcLMUNmaKf5UYb0wPeYb4yKS?usp=sharing

`pip install -qU transformers`

`from transformers import pipeline`
`sentiment_model29 = pipeline('sentiment-analysis')`
`sentiment_model29('I hate this movie')`
Returns: [{'label': 'NEGATIVE', 'score': 0.703369677066803}]



Comments:
This PR should fix the problem:
https://github.com/huggingface/transformers/pull/4264

A mismatched tokenizer is hooked up with the pipeline.

yes, you are right. 

I also tested all sentences both positive and negative. But it is showing NEGATIVE all the time.

> This PR should fix the problem:
> #4264
> 
> A mismatched tokenizer is hooked up with the pipeline.

This PR didn't resolve the issue. I pulled from master but behaviour still the same. Have you tested?

Yes, I'm always getting NEGATIVE class regardless of the text

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Same issue...incorrect sentiment for "nlp("i hate you")" shows [{'label': 'POSITIVE', 'score': 0.536677360534668}]

in 3.1, I always get label="POSITIVE". Must be a bug somewhere

I get Positive label with high score(0.99) for neutral statements like "This is just a statement" or "My name is this".
I don't know whats wrong. I used the base example.
from transformers import pipeline
sentiment_model = pipeline('sentiment-analysis')
sentiment_model('This is just a statement.')
Add MPNet
Author: StillKeepTry
Pull request: yes


Comments:
@StillKeepTry Don't forget to ping me if you need another round of review!

By the way, I haven't seen any refactorization for code like this:
```python
from .modeling_bert import (
    BertEmbeddings, 
    BertPreTrainedModel, 
    BertIntermediate, 
    BertOutput, 
    BertSelfAttention,
    BertSelfOutput, 
    BertAttention, 
    BertModel
)
```
I recommend you copy the code like `BertEmbeddings` and paste them into your own modeling file and rename it to `MPNetEmebedding`.

Would you please add tests for MPNet? You can do it following other models.

Also there are several CI fails, please fix it and we can move to the next step. Thanks for your great contribution!

Hey @StillKeepTry - there seems to have been a problem with a git rebase. We sadly cannot review 905 changed files. Can you open a new PR that only has the necessary changes? 

> Hey @StillKeepTry - there seems to have been a problem with a git rebase. We sadly cannot review 905 changed files. Can you open a new PR that only has the necessary changes?

ok

The new PR is submitted at [https://github.com/huggingface/transformers/pull/8804](https://github.com/huggingface/transformers/pull/8804)

@patrickvonplaten
Create README.md
Author: hailabpucpr
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments: