noamaanMulla-03/transformers-issues huggingface dataset:
Do I need to apply the softmax function to my logit before calculating the CrossEntropyLoss?
Author: h56cho
Pull request: no
Labels: name: wontfix

Hello, I am trying to compute the CrossEntropyLoss directly by using this code:
```
loss_fct = CrossEntropyLoss()
mc_loss = loss_fct(reshaped_logits, mc_labels)
```
If the reshaped_logits contain the logit values before softmax, should I apply `nn.softmax` function before I do `loss_fct(reshaped_logits, mc_labels)`? Thank you,

Comments:
There is no need to add  the nn.softmax function. Pls. refer to the [pytorch document](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html?highlight=crossentropyloss#torch.nn.CrossEntropyLoss).

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[bert_generation] enable cache by default
Author: patil-suraj
Pull request: yes

# What does this PR do?

- add `use_cache`  to `BertGenerationConfig` with default to `True`
- in `BertGenerationEncoder` if `use_cache` is `None` (this is the default behaviour) then set it using the `config`.

This will enable caching by default in inference for `BertGenerationEncoder`

Comments:
Proposed Fix : [RagSequenceForGeneration] generate "without" input_ids 
Author: ratthachat
Pull request: yes

# What does this PR do?
Proposal of option (2) mentioned in the issue `RagSequenceForGeneration.generate()` without `input_ids`
https://github.com/huggingface/transformers/issues/9184

###  Re-paste here:
In `RagSequenceForGeneration` method `generate()` function, the doc said that both `input_ids` and `context_input_ids` are optional (one of them must be specified) . 

However, in the code https://github.com/ratthachat/transformers/blob/ragseq_context_id/src/transformers/models/rag/modeling_rag.py#L907
It specifically need `input_ids` in all cases.

Not sure which option is the best

(1) simply said `input_ids` is always needed , OR

(2) add code to calculate `nll` if only `context_input_ids` is provided , but in this case `doc_scores` and `context_attention_mask` have to be provided as well (similar to RagModel requirement ) : https://github.com/ratthachat/transformers/blob/ragseq_context_id/src/transformers/models/rag/modeling_rag.py#L588

I think option (2) should be reasonable since  `RagTokenForGeneration` method `generate()` also requires the same.

### who can review?
@lhoestq or @patrickvonplaten 

### Make style applied but failed quality check
I already applied `make style` and passed, but somehow still fails the "code quality check" sorry :( 

Comments:
@ratthachat - thanks a lot for your PR! I agree that option 2) is the better option and think the general approach is exactly right. I left a couple of statements making that improve readability a bit IMO. Also could we add one test here: 
https://github.com/huggingface/transformers/blob/f38c4ad302dd34faef1137b13e9636f4408b0462/tests/test_modeling_rag.py#L502
called `test_model_generate_from_context_input_ids` that verifies that using `input_ids` and `context_input_ids` yields the same result?

@ratthachat Don't worry about `make style` I can fix that later!

Thanks for the review, Patrick! I pretty much agree on every point, esp. the test case. I will do it. :)

Hi Patrick, I think I addressed every suggestion :)
[ Still fails on code-quality even already applied `make style` again :D ] @patrickvonplaten 

Great job @ratthachat - I'm currently running all RAG slow tests to make sure nothing was unintentionally broken. Will merge once they all pass 

Slow tests all passing

Nice @patrickvonplaten !
[RagSequenceForGeneration] generate "without" input_ids
Author: ratthachat
Pull request: no

Hi guys,

In `RagSequenceForGeneration` method `generate()` function, the doc said that both `input_ids` and `context_input_ids` are optional (one of them must be specified) . 

However, in the code https://github.com/ratthachat/transformers/blob/ragseq_context_id/src/transformers/models/rag/modeling_rag.py#L907
It specifically need `input_ids` in all cases.

Not sure which option is the best

(1) simply said `input_ids` is always needed , OR

(2) add code to calculate `nll` if only `context_input_ids` is provided , but in this case `doc_scores` and `context_attention_mask` have to be provided as well (similar to RagModel requirement ) : https://github.com/ratthachat/transformers/blob/ragseq_context_id/src/transformers/models/rag/modeling_rag.py#L588

I think option (2) should be reasonable since  `RagTokenForGeneration` method `generate()` also requires the same.
Proposed fix in https://github.com/huggingface/transformers/pull/9220 


Comments:
fix typo in modeling_encoder_decoder.py
Author: daniele-sartiano
Pull request: yes

Fixed typo.

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes a typo


## Before submitting
- [ x ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
[Bart doc] Fix outdated statement
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #9298 . Bart docs should be slightly updated. Thank @forest1988 !

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
allow integer device for BatchEncoding
Author: jethrokuan
Pull request: yes

# What does this PR do?

Fixes #9244

I'm not fully aware of the details behind the Apex guard, in the method, so maybe this is not the solution.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

 tokenizers: @mfuntowicz

Comments:
Metrics calculate error: can only calculate the mean of floating types. Got Bool instead
Author: lucadiliello
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.3.1
- Platform: Ubuntu 18.04
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 GPU
- Tensorflow version (GPU?): -
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes

### Who can help
Someone working on the examples/GLUE benchmarks 

## Information

Model I am using (Bert, XLNet ...): BERT

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: (GLUE
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run QNLI experiment

Error:
```bash
File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/transformers/data/metrics/__init__.py", line 73, in glue_compute_metrics
    return {"acc": simple_accuracy(preds, labels)}
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/transformers/data/metrics/__init__.py", line 34, in simple_accuracy
    return (preds == labels).mean()
RuntimeError: Can only calculate the mean of floating types. Got Bool instead.
Exception ignored in: <function tqdm.__del__ at 0x7f8d1d09ff28>
Traceback (most recent call last):
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/tqdm/std.py", line 1086, in __del__
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/tqdm/std.py", line 1293, in close
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/tqdm/std.py", line 1471, in display
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/tqdm/std.py", line 1089, in __repr__
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/tqdm/std.py", line 1433, in format_dict
TypeError: cannot unpack non-iterable NoneType object
```

A simple fix is to modify `transformers/data/metrics/__init__.py", line 34, in simple_accuracy` to:
```python
return (preds == labels).to(dtype=torch.float32).mean()
```

## Expected behavior

Metric should be computed without problems

Comments:
Great, this looks like the correct fix! Do you want to open a PR?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
RAM MemoryError
Author: shizhediao
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

I was wondering is the RAM MemoryError issue solved?
I encountered this issue because my 128 GB RAM memory could not load all 48 GB data.
There are some discussions before, such as #6636, #3388, #1290, #4009 
However, I don't see there is lazyDataLoader right now.
Could you provide any hints about how to deal with a large dataset?
Thanks!

Comments:
Do you solve the problem?......

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
DPR retriever module
Author: mchari
Pull request: no
Labels: name: wontfix

I see https://github.com/huggingface/transformers/pull/5279 that describes the DPR flow.

Just checking to see when the retriever module will be available.
Many thanks for making DPR available !

Comments:
Pinging @lhoestq!

Hi @mchari 

The retriever is now part of the `nlp` library.
You can install it with

```
pip install nlp
```

and load the retriever:
```python
from nlp import load_dataset

wiki = load_dataset("wiki_dpr", with_embeddings=False, with_index=True, split="train")
```

The retriever is basically a dense index over wikipedia passages.
To query it using the DPR question encoder you can do:
```python
from transformers import DPRQuestionEncoderTokenizer, DPRQuestionEncoder

question_tokenizer = DPRQuestionEncoderTokenizer.from_pretrained('facebook/dpr-question_encoder-single-nq-base')
question_encoder = DPRQuestionEncoder.from_pretrained('facebook/dpr-question_encoder-single-nq-base')

question = "What is love ?"
question_emb = question_encoder(**question_tokenizer(question, return_tensors="pt"))[0].detach().numpy()
passages_scores, passages = wiki.get_nearest_examples("embeddings", question_emb, k=20)  # get k nearest neighbors
```

Shall we make a blog post or something to show how to use it with `transformers` @thomwolf ?

EDIT: `nlp` is now renamed to `datasets`

Hi @lhoestq ,
Which metric does the `FaissIndex` use to compute vector similarity? (i.e. how `passages_scores` values are computed?)

It uses the _inner product_ (as described in DPR paper) or something else?
Thank you

It uses inner product.
You can see the code that creates the index here  https://github.com/huggingface/nlp/blob/6655008c738cb613c522deb3bd18e35a67b2a7e5/datasets/wiki_dpr/wiki_dpr.py#L171

Thanks for the retriever functionality !  Not sure how it works if I want to use it on my own documents.


@lhoestq , any guidance for fine tuning the retriever module on another set of documents ?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
SequenceSummary class in modeling_utils.py
Author: h56cho
Pull request: no
Labels: name: wontfix

Hello,
I have a question about the documentation strings provided for the forward function of the SequenceSummary class from modeling_utils.py:
https://github.com/huggingface/transformers/blob/dc552b9b7025ea9c38717f30ad3d69c2a972049d/src/transformers/modeling_utils.py#L1484

So when `cls_index` is not specified as the argument in SequenceSummary() statement, is the last token of the sequence used for the classification task? The entire sentence for the description is somewhat awkward...

Thanks,

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
I'm getting "nan" value for loss, while following a tutorial from the documentation
Author: sunnyville01
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help


## Details
Hi,

I’m following the “Fine-tuning with Custom Dataset” tutorial for Question Answering on the SQuaD dataset tutorial: [available here](https://huggingface.co/transformers/custom_datasets.html#question-answering-with-squad-2-0). I’ve copy-pasted all the code shown in the tutorial step by step. However, when my model starts training, I don’t get the expected metric values for loss as I normally would, instead I get “nan”. Here is the code for training
`model.fit(train_dataset.shuffle(1000).batch(16), epochs=3, batch_size=16)`

Here is the output with the "nan" values for the losses.
```
Epoch 1/3
5427/5427 [==============================] - 4604s 848ms/step - loss: nan - output_1_loss: nan - output_2_loss: nan
Epoch 2/3
365/5427 [=>…] - ETA: 1:11:28 - loss: nan - output_1_loss: nan - output_2_loss: nan

```
I don’t know what is wrong, and I don’t think this output is what is supposed to be.

Would appreciate any help with this regard.
Thank you.


**A link to original question on the forum/Stack Overflow**:
https://discuss.huggingface.co/t/im-getting-nan-value-for-loss-while-following-a-tutorial-from-the-documentatin/1530

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@sunnyville01 
hi, facing the same issue, did you manage to solve this?
T5 supervised denoising task
Author: ghost
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Hi everyone!

I'm experimenting with T5 and I would like to fine-tune a specific pre-trained model of mine for tackling the 'filling the mask' task. To be clear, I have the following: I love \<mask> and Mario. Where \<mask> can be a single token or span.
At the moment I framed the problem in this way:
   - input: I love <extra_id_0> and Mario.
   - output/label: luca

The task I want to tackle is different from the canonical unsupervised one where I was able to perform it correctly. 
Do you think that the discussed framing presented above is enough?

From the result that I got,  it doesn't seem so.


Comments:
@patrickvonplaten Any news?

Yeah this looks good to me: 

input_ids: I love <extra_id_0> and Mario.
decoder_input_ids: `decoder_start_token_id`
output/label: luca <EOS>

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
can't set evaluation_strategy to "epoch"
Author: yuvalkirstain
Pull request: no
Labels: name: wontfix

## Environment info

- `transformers` version: 3.3.1

### Who can help
@sgugger 

## Information
Seems like we can't use the evaluation_strategy=epoch option. The default value of evaluate_during_training is boolean (False). In transformers/training_args.py, __post_init__ function, there is a check - `if self.evaluate_during_training is not None:` (line 326). If I'm not mistaken, it will always result in True. I think that if you change the line into `if self.evaluate_during_training:` the problem will be solved.

Thank you so much for your awesome work :)  

Comments:
This has been done already, see [here](https://github.com/huggingface/transformers/blob/master/src/transformers/training_args.py#L335). But you need to install from source for that, since the change is recent,

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
GPT2Tokenizer.add_tokens() didnt change tokenizer.vocab_size
Author: Makigumoe
Pull request: no
Labels: name: wontfix

follow the instruction in add_tokens()

code:

special_tokens = ['<pad>', '<go_r>', '<unk>', '<go_b>', '<go_a>', '<go_u>', ]
num_added_toks = tokenizer.add_tokens(special_tokens)
print('We have added', num_added_toks, 'tokens')

it can successfully encode and decode the newly added tokens, but the GPT2Tokenizer.vocab_size is still 50257. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.