noamaanMulla-03/transformers-issues huggingface dataset:
Fix embeddings for PyTorch 1.8
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes several embeddings layer with the recent breaking change introduced in PyTorch 1.8. Up until PyTorch 1.7, the `padding_idx` passed to an embedding layer was used to initialize the corresponding row in the weights to 0 but ignored afterwards.

Now, this `padding_idx` is used at every forward pass and ignores the potential weights of the padding index (spoiler alert, all pretrained models I checked have a nonzero one).

To solve this, this PR removes all `padding_idx` passed to embeddings layer. As we were in any case re-initializing them in the `_init_weights` function, the zero weight for that index was ignored in any case. This PR thus introduces no breaking change on our side while dealing with the breaking change in PyTorch.


Comments:
Truncate max length if needed in all examples
Author: sgugger
Pull request: yes

# What does this PR do?

As pointed out in #10015, most examples will let the tokenzation and training run when `tokenizer.model_max_length < max_seq_length` and the default value is sometimes bigger than the max length for some models (like BertTweet). This PR addresses that.

Fixes #10015

Comments:
@sgugger Thanks for fixing this. I made this mistake again (running many jobs at once with different models) and my jobs didn't crash (meanwhile I had upgraded to the latest transformers 🚀  )
Stale Bot
Author: LysandreJik
Pull request: yes

Adds a stale bot based on GitHub Actions.

This bot is slightly different than the previous one in that it comments that it is closing the issue and closes it immediately, rather than waiting 7 days. From what I've seen up to now, this shouldn't be an issue at all.

I've commented out the code for now so that it doesn't actually close issues, but you can take a look at which would be closed [here](https://github.com/huggingface/transformers/runs/2027394567?check_suite_focus=true). Will un-comment the code before merging.

Comments:
The `GITHUB_TOKEN` got rate-limited, unfortunately. The PR should be good to go, I'll try to run it again tomorrow night.
Make use of our copy-consistency script for task-specific models
Author: sgugger
Pull request: no
Labels: name: Good First Issue

name: Good Second Issue

This is an intermediate issue, which is why it gets both the good first issue and good second issue tags.

We have an automated script to check when copies of the same code are consistent inside the library, which allows us to avoid subclassing and keep all code for one model's forward pass inside one file (see our [philosophy]() for more details on this).

The XxxModelForYyy are very similar to one another and should be able to leverage that functionality, so we can easily change only one file when there is a bug/docstring to tweak and all the others are updated. More precisely, models that have a pooler layer could probably base themselves on BERT and models that don't could be based on ELECTRA. The Seq2Seq models that are a bit particular could be based on BART.

To enable this, the checkpoint use in the decorator `@add_code_sample_docstrings` needs to be defined in a constant (otherwise it will end up being copied which we don't want) so to tackle this issue, your mission, should you accept it, will have two steps:

1. Define in all modeling files a `_CHECKPOINT_FOR_DOC` at the beginning (with `_TOKENIZER_FOR_DOC` and `_CONFIG_FOR_DOC`) that should then be used in all the XxxModelForYyy. 
2. Adds the relevant `# Copied from xxx with Xxx -> Yyy` whenever possible.

Comments:
Hi @sgugger I'm up for the mission. After looking though the code I think I have a basic understanding of what you mean. However, would you be able to provide an example on one model just for clarification?  I'm a bit confused on the second step


>modeling_bert.py

```
_CONFIG_FOR_DOC = "BertConfig"
_TOKENIZER_FOR_DOC = "BertTokenizer"
_CHECKPOINT_FOR_DOC = "bert-base-uncased"
```
...

```
class BertForSequenceClassification(BertPreTrainedModel):
...
...
    @add_code_sample_docstrings(
        tokenizer_class=_TOKENIZER_FOR_DOC,
        checkpoint=_CHECKPOINT_FOR_DOC,
        output_type=SequenceClassifierOutput,
        config_class=_CONFIG_FOR_DOC,
    )


```

Thank you

I've given an example of the first step in the issue above. Sadly the second step can't fully work with our utils just yet, I need to make some adjustments to our internal tooling. If you want to begin on step 1 though, don't hesitate!
Can't make sense of encoding for a downloadable AutoTokenizer
Author: wickedsprout
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.2

### Who can help

Probably @n1t0 or @LysandreJik (AutoTokenizer)

## To reproduce

Steps to reproduce the behavior:

1. Boot up an AutoTokenizer using `AutoTokenizer.from_pretrained("sberbank-ai/rugpt3small_based_on_gpt2")`
2. Execute `tokenizer.get_vocab()`

The vocabulary contains gibberish instead of Russian tokens (yet the model works fine):

![image](https://user-images.githubusercontent.com/31738272/106778421-03fc8200-6657-11eb-8f24-48daad35ff16.png)

How do I decode and read the actual tokens?

Comments:
This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
Disk memory management
Author: marsupialtail
Pull request: no
Labels: name: wontfix

I am wondering if you guys could add support for disk memory management when running large transformer models. At least when running on my laptop with limited DRAM, it is not feasible to fully materialize some of the larger models (T5-3b e.g. or even T5-large) in DRAM, especially if there are other memory intensive tasks running (like the IDE). I'm wondering if it's possible for the Huggingface library to not materialize the entire model in DRAM as a Python object for these larger models and instead re-materialize them layer by layer from the disk.

Comments:
This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
Model Save/Load Fails for Hadoop File Server
Author: Rmsharks4
Pull request: no
Labels: name: wontfix

## Environment info
- `transformers` version: 4.2.2
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.3
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): 2.4.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@n1t0, @LysandreJik @sgugger

## Information

Model I am using (Bert, XLNet ...): bert-base-uncased

The problem arises when using:
* [X] the official example scripts: 
```python
config = pipeline.get_common_model_file(model_name, Constants.CONFIG)
model = AutoModelForSequenceClassification.from_pretrained(config=config, pretrained_model_name_or_path='http://192.168.0.61:50070/webhdfs/v1/user/root/NLPEngine/models/bert-base-uncased/pytorch_model.bin?op=OPEN')
```
* [ ] my own modified scripts: 

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: 
A Conversational (Multi-Turn Dialog) Dataset for Task of Knowledge Selection. Dataset is raising no issues.

## To reproduce

Steps to reproduce the behavior:

1. Install Hadoop File Server
2. Setup WebHDFS to use cURL commands to load/save files on HDFS or directly use hadoop for that. Just make sure that all your servers are listed in your system's hosts configurations.
3. Place bert-base-uncased model on Hadoop (anywhere).
4. Try and access it from the code mentioned above.

## Expected behavior

The model should be loaded from the file, as it would be loaded locally or from a server that returns an E-Tag, but Hadoop is not configured / built for returning E-Tags. It first returns a temporary-redirect URL, and then from that the actual object retrieved off of one of the servers in its cluster(s). 
If I turn off the E-Tag validation in the source code, then it starts working perfectly, but as of now, its part of the source-code, and thats causing this code to crash. 
Here is the change I made to get it to work (in my copy of the library's code):

```python
File "C:\Users\rsiddiqui\Anaconda3\Lib\site-packages\transformers\file_utils.py", line 1182, in get_from_cache
etag = ''
File "C:\Users\rsiddiqui\Anaconda3\Lib\site-packages\transformers\file_utils.py", line 1187, in get_from_cache
etag = r.headers.get("X-Linked-Etag", '') or r.headers.get("ETag", '')
```


Comments:
This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
What is the correct way to use Adafactor?
Author: avacaondata
Pull request: no
Labels: name: wontfix

Hi, from the papers I've seen that Adafactor is typically used with no learning rate (as in Pegasus paper), however, when I try to execute run_seq2seq.py or seq2seq/finetune_trainer.py from your examples, and set --adafactor parameter, without specifying learning rate (for no learning rate), it uses the default 3e-05. Is there a way to use Adafactor without learning rate?

Comments:
This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
tokenizer is slow when adding new tokens
Author: davidnarganes
Pull request: no
Labels: name: wontfix

Hi,
The tokenizer is slow when adding new tokens even with the Fast class:

```
from transformers import GPT2Config, TFGPT2LMHeadModel, GPT2TokenizerFast, GPT2Tokenizer

# Maybe this url for the files:
# https://huggingface.co/transformers/v3.1.0/_modules/transformers/tokenization_gpt2.html
paths = dict()
paths["tokenizer"] = "whatever/is/the/path/to/pretrained/vocab.json/merges.txt"

# They have to be sorted in reverse by length, otherwise the tokens arent 
newtokens = range(0, 20000)
newtokens = list(newtokens)
newtokens.sort(reverse=True)
newtokens = ["new_" + str(x) for x in newtokens]

# loading tokenizer from the saved model path
tokenizers = dict()
tokenizers["fast"] = GPT2TokenizerFast.from_pretrained(paths["tokenizer"])
tokenizers["fast_custom"] = GPT2TokenizerFast.from_pretrained(paths["tokenizer"])
tokenizers["slow_custom"] = GPT2Tokenizer.from_pretrained(paths["tokenizer"])
tokenizers["slow"] = GPT2Tokenizer.from_pretrained(paths["tokenizer"])

tokenizer.add_special_tokens({
  "eos_token": "
",
  "bos_token": "
",
  "unk_token": "<unk>",
  "pad_token": "<pad>",
  "mask_token": "<mask>"
})

# Add new vocab
# https://huggingface.co/transformers/v2.11.0/main_classes/tokenizer.html
# https://github.com/deepset-ai/FARM/issues/157
for k in tokenizers:
    if "custom" in k:
        print(k)
        print("Vocab length before:", len(tokenizers[k].get_vocab()))
        tokenizers[k].add_tokens(newtokens)
        print("Vocab length after:", len(tokenizers[k].get_vocab()))

# creating the configurations from which the model can be made
config = GPT2Config(
  vocab_size=len(tokenizer),
  bos_token_id=tokenizer.bos_token_id,
  eos_token_id=tokenizer.eos_token_id
)

# creating the model
# https://huggingface.co/transformers/_modules/transformers/configuration_gpt2.html
model = TFGPT2LMHeadModel(config)

# Differences when tokenising the text...
text = "this is a sentence containing new_200"
for k,v in tokenizers.items():
    print(k, v.tokenize(text))
```

and then profiling the speed in jupyter:
```
for k in tokenizers:
    print(k)
    %timeit tokenizers[k].tokenize(text)
```

any ideas why this may be happening? I understand that the vocab size could increase by ~20% and that may slow things down but in this code there's a performance difference of 1000 fold in the speed. That doesn't seem right?

Comments:
Hi @davidnarganes,
Someone from HF correct me if I am wrong, but you'll probably get a faster response posting this issue in the Tokenizer repo:
https://github.com/huggingface/tokenizers

Best of luck

This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
RAG + DPR model performance issues
Author: krishanudb
Pull request: no
Labels: name: wontfix

Hi.
I am trying to reproduce the results obtained in the Retriever Augmented Generation paper for Question Answering on the Natural Questions (NQ) Dataset (Exact Match accuracy 44%).
However, I am not able to reproduce them.

Can someone kindly let me know which DPR model, DPR dataset and RAG dataset was used to obtain the 44% EM Accuracy on NQ dataset?

Thanks.

Comments:
This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
prediction_step() is not using compute_loss()
Author: hadifar
Pull request: no
Labels: name: wontfix

Hi @sgugger , I think there is an issue with `prediction_step` in `trainer.py`. The problem arises when implementing a custom loss function that requires reshaping input labels. In `training_step`, for loss calculation, it calls `compute_loss()` which is totally fine but in the `prediction_step` it calculates loss without `compute_loss()` function. This inconsistency causes some issues. Would you think it would be better to call `compute_loss()` in both cases in order to avoid this problem? 

Update: As I see the code, the loss is tightly coupled with `LabelSmoother` which makes it hard to do them in a single function. If you have any suggestion, it makes me happy to contribute to Huggingface ;)

[prediction_step()](https://huggingface.co/transformers/_modules/transformers/trainer.html#Trainer.prediction_step)

[training_step()](https://huggingface.co/transformers/_modules/transformers/trainer.html#Trainer.training_step)

Comments:
Mmmm probably. This is a bit tricky to make sure it doesn't break anything but makes more sense. I'll try to look at this on Monday.

This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
How to add more fields in TrainingArguments
Author: g-jing
Pull request: no
Labels: name: wontfix

I am using ```from transformers import TrainingArguments```. However, there are more training arguments in my own project. How can I add more fields (parameters) in to the ```args```? Besides, if I have some other ```Arguments Class``` that is similar to ```TrainingArguments```, how to merge them into one ```args```?

Comments:
You could subclass the `TrainingArguments` class and add more fields to it. You could refer to `https://github.com/huggingface/transformers/blob/master/src/transformers/training_args_seq2seq.py` for an example https://github.com/huggingface/transformers/blob/master/src/transformers/training_args_seq2seq.py

This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
Add the ability to skip runtime version check
Author: Guitaricet
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Add the ability to skip version check / raise a warning instead of an error if the checks failed.

## Motivation

Currently, Transformers perform a runtime libraries version check and raise an error if some of the requirements has a version that is different from the specified.

While version check is a reasonable thing to do, an error seems like an overkill. As [I mentioned](https://github.com/huggingface/transformers/pull/8073#issuecomment-765632175) in #8073, this, for example, prevents from using the latest version of Tokenizers without modifying the source code.

## Your contribution

I suggest creating an environment variable (e.g. `TRANSFORMERS_VERSION_CHECK_STRICT`) that would control the reaction to a version mismatch. If it is set to `true`, the current behavior remains. If `false`, we log a warning instead of raising an error. Personally, I think that having this variable `false` by default will make it easier for the users to work in such cases.

What's your opinion in that? Probably we can find an even better solution together.

Comments:
If anyone else comes here, until this is fixed you can find the version in `transformers/dependency_versions_check.py` - e.g. change `"tokenizers": "tokenizers==0.9.4"` to `"tokenizers": "tokenizers>=0.9.4"`

This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
Strange start token in MT5 generation
Author: tomdzh
Pull request: no
Labels: name: wontfix

## Environment info

- `transformers` version: 4.2.2
- Platform: Linux
- Python version: 3.6
- PyTorch version (GPU?):1.7.1

### Who can help
 Text Generation: @patrickvonplaten @TevenLeScao
 T5: @patrickvonplaten
 
## Information

Model I am using (Bert, XLNet ...): MT5

## To reproduce

Steps to reproduce the behavior:
```python
from transformers import MT5ForConditionalGeneration, MT5Tokenizer

model = MT5ForConditionalGeneration.from_pretrained("google/mt5-small")
tokenizer = MT5Tokenizer.from_pretrained("google/mt5-small")

text = 'summarize: Bidirectional Encoder Representations from Transformers is a Transformer-based machine learning technique for natural language processing pre-training developed by Google'
inputs = tokenizer([text], max_length=512, truncation=True, return_tensors='pt')
summary_ids = model.generate(inputs['input_ids'])
print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True) for g in summary_ids])
```
The output I got is ['<extra_id_0>.']
## Expected behavior
I tried a few input texts. The generated output always start with <extra_id_0>, which doesn't happen in t5 generation. Anyone knows how to solve it?


Comments:
One more thing: this behavior still persists after I fine tuned the model on my own dataset

Hi @tomdzh 

first of all, unlike the original T5, mT5 is not pre-trained on any supervised downstream task (like summarization, translation etc), so generation work without fine-tuning it.

Also, it would be hard to answer why it's happening in fine-tuned model without looking at any code.

This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
RagRetriever question_hidden_states shape
Author: aqweteddy
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.2
- Platform: Linux-5.4.0-60-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?:no

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 -->

 RAG: @patrickvonplaten, @lhoestq

## Information

The shape of RagRetriever's argument question_hidden_states mentioned in the document is `(batch_size, vector_size)`.
However, when RagModel calls self.retriever in forward function, it passes question_encoder_last_hidden_state`(batch_size, seq_len, vector_size)` to RagRetriever. Maybe question_encoder_last_hidden_state should be replaced with question_encoder_pooler_output?

The problem arises when using:
* [v] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [v] my own task or dataset: (give details below)

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
maybe a small bug?

Comments:
I'm pretty sure it's just a bad naming of that variable. It should be question_encoder_pooler_output.

Indeed this variable is computed as the first output of a `DPRQuestionEncoder`. The DPR encoders don't return the hidden states (they basically skip it) to return the pooler output (aka DPR embeddings) of shape `(batch_size, vector_size)` instead.

If you want to contribute, feel free to open a PR to rename this variable :) 

This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
Fix an efficiency related bug the "prediction_loop" of trainer_tf.py
Author: zhangzhenyu13
Pull request: yes
Labels: name: wontfix

# the "numpy.append" method is not suitable for large evaluation/test dataset
It will cause "memory ops blocked issue" (i.e. system wait for a large area of continuous memory to expand the array) because numpy array requires continuous memory, and will try to re-allocate a ram space when the current capacity is not enough
### this procedure is quite slow(as if the prediction loop is blocked): tested when data set size  is larger than 10K

### we need to use the concatenate strategy(for efficiency) or batch generator strategy(when the data size is quite large).
### The method I provided here will significantly boost the prediction speed.

Comments:
This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.