internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: huggingface/transformers

Issue:
weights partially missing for CamembertForMaskedLM 
## Environment info

- `transformers` version: 3.1.0
- Platform: Darwin-18.7.0-x86_64-i386-64bit
- Python version: 3.7.5
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
 
@louismartin 

## Information

When loading "camembert-base" with `CamembertForMaskedLM` with:

    from transformers import CamembertForMaskedLM
    model = CamembertForMaskedLM.from_pretrained("camembert-base")

the bias of the LM head decoder is not loaded:

    Some weights of CamembertForMaskedLM were not initialized from the model checkpoint at camembert-base and are newly initialized: ['lm_head.decoder.bias']
    You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

As I understand `lm_head.decoder.bias` is therefore initialized randomly.
I checked the original `camembert-base` model as published by the author, and the lm_head decoder bias is missing too, which is not discussed in the camembert or roberta publication.


Discussion:
Hi @raphael0202 , we converted the fairseq checkpoints using [this script](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_roberta_original_pytorch_checkpoint_to_pytorch.py), maybe the language model head was not transfered during the conversion. 
You can either use the model in fairseq as per [our website](https://camembert-model.fr/) or try to understand if and why the LM head hasn't transfered from fairseq to transformers.
Check this tutorial as well for huggingface: https://huggingface.co/camembert-base
After a bit of search I found out the warning comes from the way the decoder bias is defined in the code, the same warning is issued when loading the roberta model (see issue 6193). I checked that the LM head decoder bias is the same as the one in the original pytorch checkpoint.
Thank you for your help!

Resolving patch:
diff --git a/src/transformers/modeling_roberta.py b/src/transformers/modeling_roberta.py
--- a/src/transformers/modeling_roberta.py
+++ b/src/transformers/modeling_roberta.py
@@ -303,6 +303,7 @@ def prepare_inputs_for_generation(self, input_ids, attention_mask=None, **model_
 class RobertaForMaskedLM(BertPreTrainedModel):
     config_class = RobertaConfig
     base_model_prefix = "roberta"
+    authorized_missing_keys = [r"position_ids", r"lm_head\.decoder\.bias"]
 
     def __init__(self, config):
         super().__init__(config)


Test patch:
Repository: huggingface/transformers

Issue:
Trainer Evaluates at each step (Not of epoch end) , indentation bug
## Environment info
   
- `transformers` version: 3.2.0
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: NA

### Who can help
 
 Trainer: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...): BERT

The problem arises when using:
- [ ] the official example scripts:
- [x] my own modified scripts:

The tasks I am working on is:
- [ ] an official GLUE/SQUaD task:
- [x] my own task or dataset: 

Basic Single Sentence Classification Dataset loaded via a Dataset class

## To reproduce

Steps to reproduce the behavior:

1. Use the trainer training function with `training_args.evaluation_strategy = EvaluationStrategy.EPOCH`


## Expected behavior

Evaluation should happen after each epoch ends, but instead it happens after each step (batch).

Indentation bug

## Suggested Fix


Move the if condition on line https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L829 to before line https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L834 and remove 1 indentation block



Discussion:

Resolving patch:
diff --git a/src/transformers/trainer.py b/src/transformers/trainer.py
--- a/src/transformers/trainer.py
+++ b/src/transformers/trainer.py
@@ -826,13 +826,15 @@ def train(self, model_path: Optional[str] = None, trial: Union["optuna.Trial", D
                             torch.save(self.lr_scheduler.state_dict(), os.path.join(output_dir, "scheduler.pt"))
 
                 epoch_pbar.update(1)
-                if self.args.evaluation_strategy == EvaluationStrategy.EPOCH:
-                    metrics = self.evaluate()
-                    self._report_to_hp_search(trial, epoch, metrics)
                 if self.args.max_steps > 0 and self.global_step >= self.args.max_steps:
                     break
             epoch_pbar.close()
             train_pbar.update(1)
+
+            if self.args.evaluation_strategy == EvaluationStrategy.EPOCH:
+                metrics = self.evaluate()
+                self._report_to_hp_search(trial, epoch, metrics)
+
             if self.args.tpu_metrics_debug or self.args.debug:
                 if is_torch_tpu_available():
                     # tpu-comment: Logging debug metrics for PyTorch/XLA (compile, execute times, ops, etc.)


Test patch:
Repository: huggingface/transformers

Issue:
RAG: error in outputs = model(input_ids=input_ids, labels=input_dict["labels"])
I tried the following code given in the documentation. 

```
from transformers import RagTokenizer, RagRetriever, RagModel
import torch
tokenizer = RagTokenizer.from_pretrained("facebook/rag-token-base")
retriever = RagRetriever.from_pretrained("facebook/rag-token-base", index_name="exact", use_dummy_dataset=True)
# initialize with RagRetriever to do everything in one forward call
model = RagModel.from_pretrained("facebook/rag-token-base", retriever=retriever)
input_dict = tokenizer.prepare_seq2seq_batch("How many people live in Paris?", "In Paris, there are 10 million people.", return_tensors="pt")
input_ids = input_dict["input_ids"]
outputs = model(input_ids=input_ids, labels=input_dict["labels"])

```
In the last step, it gives the following error. 


**/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'labels'**



Discussion:

Resolving patch:
diff --git a/src/transformers/modeling_rag.py b/src/transformers/modeling_rag.py
--- a/src/transformers/modeling_rag.py
+++ b/src/transformers/modeling_rag.py
@@ -537,7 +537,7 @@ def forward(
 
             >>> input_dict = tokenizer.prepare_seq2seq_batch("How many people live in Paris?", "In Paris, there are 10 million people.", return_tensors="pt")
             >>> input_ids = input_dict["input_ids"]
-            >>> outputs = model(input_ids=input_ids, labels=input_dict["labels"])
+            >>> outputs = model(input_ids=input_ids)
 
         """
         use_cache = use_cache if use_cache is not None else self.config.use_cache


Test patch:
Repository: huggingface/transformers

Issue:
ELECTRA - some weights are not loaded
## Environment info
- `transformers` version: 3.3.1
- Platform: Linux-4.15.0-118-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


### Who can help
 albert, bert, GPT2, XLM: @LysandreJik 
 Model Cards: @julien-c 

## Information

Model I am using: ELECTRA
I am getting a warning:

> Some weights of the model checkpoint at google/electra-large-discriminator were not used when initializing ElectraModel: ['electra.embeddings_project.weight', 'electra.embeddings_project.bias']
> - This IS expected if you are initializing ElectraModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
> - This IS NOT expected if you are initializing ElectraModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

when using the AutoModel.from_pretrained for google/electra-base-discriminator or google/electra-large-discriminator. There is no warning for google/electra-small-discriminator.

The problem remains the same when directly using the ElectraModel.from_pretrained method.


## To reproduce
```
import transformers
m=transformers.AutoModel.from_pretrained("google/electra-large-discriminator")
# or
m=transformers.AutoModel.from_pretrained("google/electra-base-discriminator")
```


## Expected behavior
no warning


Discussion:

Resolving patch:
diff --git a/src/transformers/modeling_electra.py b/src/transformers/modeling_electra.py
--- a/src/transformers/modeling_electra.py
+++ b/src/transformers/modeling_electra.py
@@ -535,6 +535,7 @@ class ElectraPreTrainedModel(PreTrainedModel):
     load_tf_weights = load_tf_weights_in_electra
     base_model_prefix = "electra"
     authorized_missing_keys = [r"position_ids"]
+    authorized_unexpected_keys = [r"electra\.embeddings_project\.weight", r"electra\.embeddings_project\.bias"]
 
     # Copied from transformers.modeling_bert.BertPreTrainedModel._init_weights
     def _init_weights(self, module):


Test patch:
Repository: huggingface/transformers

Issue:
Functionality to pass first few tokens as input to the decoder in T5 model
I am finetuning the T5 model on a downstream sequence to sequence task. I wanted to know if it is possible to pass first few tokens as input to the T5 decoder during inference (like it is done in text generation models) apart from the input provided to the encoder?


Discussion:

Resolving patch:
diff --git a/src/transformers/generation_utils.py b/src/transformers/generation_utils.py
--- a/src/transformers/generation_utils.py
+++ b/src/transformers/generation_utils.py
@@ -111,6 +111,7 @@ def postprocess_next_token_scores(
     def generate(
         self,
         input_ids: Optional[torch.LongTensor] = None,
+        decoder_input_ids: Optional[torch.LongTensor] = None,
         max_length: Optional[int] = None,
         min_length: Optional[int] = None,
         do_sample: Optional[bool] = None,
@@ -151,6 +152,9 @@ def generate(
             input_ids (:obj:`torch.LongTensor` of shape :obj:`(batch_size, sequence_length)`, `optional`):
                 The sequence used as a prompt for the generation. If :obj:`None` the method initializes
                 it as an empty :obj:`torch.LongTensor` of shape :obj:`(1,)`.
+            decoder_input_ids (:obj:`torch.LongTensor` of shape :obj:`(batch_size, sequence_length)`, `optional`):
+                initial input_ids for the decoder of encoder-decoder type models. If :obj:`None` then only
+                decoder_start_token_id is passed as the first token to the decoder.
             max_length (:obj:`int`, `optional`, defaults to 20):
                 The maximum length of the sequence to be generated.
             min_length (:obj:`int`, `optional`, defaults to 10):
@@ -417,14 +421,19 @@ def generate(
             )  # shape: (batch_size * num_return_sequences * num_beams, cur_len)
 
         if self.config.is_encoder_decoder:
-            # create empty decoder input_ids
-            input_ids = torch.full(
-                (effective_batch_size * num_beams, 1),
-                decoder_start_token_id,
-                dtype=torch.long,
-                device=next(self.parameters()).device,
-            )
-            cur_len = 1
+            device = next(self.parameters()).device
+            if decoder_input_ids is not None:
+                # give initial decoder input ids
+                input_ids = decoder_input_ids.repeat(effective_batch_size * num_beams, 1).to(device)
+            else:
+                # create empty decoder input_ids
+                input_ids = torch.full(
+                    (effective_batch_size * num_beams, 1),
+                    decoder_start_token_id,
+                    dtype=torch.long,
+                    device=device,
+                )
+            cur_len = input_ids.shape[-1]
 
             assert (
                 batch_size == encoder_outputs.last_hidden_state.shape[0]


Test patch:
Repository: huggingface/transformers

Issue:
Tokenizer: UnboundLocalError with PaddingStrategy MAX_LENGTH
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.1.0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


### Who can help
 
 tokenizers: @mfuntowicz

## Information

Model I am using: <transformers.tokenization_bert.BertTokenizer>

The problem arises when using:
* the official example scripts: 
using the `encode_plus`

The tasks I am working on is:
* Tokenizing

## To reproduce

Steps to reproduce the behavior:

1. tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
2. tokenizer.encode_plus("hello word", max_length=128, padding=PaddingStrategy.MAX_LENGTH)

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

`UnboundLocalError: local variable 'padding_strategy' referenced before assignment`

## Expected behavior

Return Tokenizer output


Discussion:

Resolving patch:
diff --git a/src/transformers/tokenization_utils_base.py b/src/transformers/tokenization_utils_base.py
--- a/src/transformers/tokenization_utils_base.py
+++ b/src/transformers/tokenization_utils_base.py
@@ -1781,6 +1781,8 @@ def _get_padding_truncation_strategies(
                 padding_strategy = PaddingStrategy.LONGEST  # Default to pad to the longest sequence in the batch
             elif not isinstance(padding, PaddingStrategy):
                 padding_strategy = PaddingStrategy(padding)
+            elif isinstance(padding, PaddingStrategy):
+                padding_strategy = padding
         else:
             padding_strategy = PaddingStrategy.DO_NOT_PAD
 
@@ -1806,6 +1808,8 @@ def _get_padding_truncation_strategies(
                 )  # Default to truncate the longest sequences in pairs of inputs
             elif not isinstance(truncation, TruncationStrategy):
                 truncation_strategy = TruncationStrategy(truncation)
+            elif isinstance(truncation, TruncationStrategy):
+                truncation_strategy = truncation
         else:
             truncation_strategy = TruncationStrategy.DO_NOT_TRUNCATE
 


Test patch:
Repository: huggingface/transformers

Issue:
SquadProcessor: Wrong reference name/filename in docstring
As the docstring of the function `get_train_examples()` refers to  `train-v1.1.json`/`train-v2.0.json`, I guess `get_dev_examples()` should refer to `dev-v1.1.json`/`dev-v2.0.json` (but refers to `train-v1.1.json`/`train-v2.0.json`):

https://github.com/huggingface/transformers/blob/aa6c3c14b4ff8fbd5d40b100f4aae71fb359d6ae/src/transformers/data/processors/squad.py#L610-L617


Discussion:
Indeed! Do you want to open a PR fixing the doc?
Yes, I can do that. :)

Resolving patch:
diff --git a/src/transformers/data/processors/squad.py b/src/transformers/data/processors/squad.py
--- a/src/transformers/data/processors/squad.py
+++ b/src/transformers/data/processors/squad.py
@@ -614,7 +614,7 @@ def get_dev_examples(self, data_dir, filename=None):
         Args:
             data_dir: Directory containing the data files used for training and evaluating.
             filename: None by default, specify this if the evaluation file has a different name than the original one
-                which is `train-v1.1.json` and `train-v2.0.json` for squad versions 1.1 and 2.0 respectively.
+                which is `dev-v1.1.json` and `dev-v2.0.json` for squad versions 1.1 and 2.0 respectively.
         """
         if data_dir is None:
             data_dir = ""


Test patch:
Repository: huggingface/transformers

Issue:
SqueezBert link gives a 404 error
The main Readme.md file (https://github.com/huggingface/transformers/blob/master/README.md), the SqueezeBert link (https://huggingface.co/transformers/model_doc/squeezebert.html) gives a "404 - Not Found" Error.


Discussion:
Yes, unfortunately that link will only be live at the next release (for now squeezeBERT is only in master, so only in the master documentation).
@LysandreJik not sure if there is a way to properly fix this unless we add "Check all the links in the README to remove the master" in our release check list.
You're right, I don't think there's any other way without over-engineering a feature.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -3,7 +3,9 @@
 
 To create the package for pypi.
 
-1. Change the version in __init__.py, setup.py as well as docs/source/conf.py.
+1. Change the version in __init__.py, setup.py as well as docs/source/conf.py. Remove the master from the links in
+   the new models of the README:
+   (https://huggingface.co/transformers/master/model_doc/ -> https://huggingface.co/transformers/model_doc/)
 
 2. Unpin specific versions from setup.py that use a git install.
 
diff --git a/utils/check_copies.py b/utils/check_copies.py
--- a/utils/check_copies.py
+++ b/utils/check_copies.py
@@ -24,6 +24,7 @@
 # python utils/check_copies.py
 TRANSFORMERS_PATH = "src/transformers"
 PATH_TO_DOCS = "docs/source"
+REPO_PATH = "."
 
 
 def find_code_in_transformers(object_name):
@@ -175,7 +176,7 @@ def get_model_list():
     # If the introduction or the conclusion of the list change, the prompts may need to be updated.
     _start_prompt = "🤗 Transformers currently provides the following architectures"
     _end_prompt = "1. Want to contribute a new model?"
-    with open(os.path.join("README.md"), "r", encoding="utf-8") as f:
+    with open(os.path.join(REPO_PATH, "README.md"), "r", encoding="utf-8") as f:
         lines = f.readlines()
     # Find the start of the list.
     start_index = 0
@@ -219,7 +220,17 @@ def split_long_line_with_indent(line, max_per_line, indent):
 def convert_to_rst(model_list, max_per_line=None):
     """ Convert `model_list` to rst format. """
     # Convert **[description](link)** to `description <link>`__
-    model_list = re.sub(r"\*\*\[([^\]]*)\]\(([^\)]*)\)\*\*", r"`\1 <\2>`__", model_list)
+    def _rep_link(match):
+        title, link = match.groups()
+        # Keep hard links for the models not released yet
+        if "master" in link or not link.startswith("https://huggingface.co/transformers"):
+            return f"`{title} <{link}>`__"
+        # Convert links to relative links otherwise
+        else:
+            link = link[len("https://huggingface.co/transformers/") : -len(".html")]
+            return f":doc:`{title} <{link}>`"
+
+    model_list = re.sub(r"\*\*\[([^\]]*)\]\(([^\)]*)\)\*\*", _rep_link, model_list)
 
     # Convert [description](link) to `description <link>`__
     model_list = re.sub(r"\[([^\]]*)\]\(([^\)]*)\)", r"`\1 <\2>`__", model_list)


Test patch:
Repository: huggingface/transformers

Issue:
Trainer callback breaks old code
https://github.com/huggingface/transformers/blob/ba4bbd92bcb55febbfa06aaa1551738388ec7eb0/src/transformers/trainer_callback.py#L438-L447

Currently it depends on the fact that evaluate() will first call `self.prediction_loop` https://github.com/huggingface/transformers/blob/ba4bbd92bcb55febbfa06aaa1551738388ec7eb0/src/transformers/trainer.py#L1181

which will then call `.callback_handler.on_prediction_step `
https://github.com/huggingface/transformers/blob/ba4bbd92bcb55febbfa06aaa1551738388ec7eb0/src/transformers/trainer.py#L1270

But in my old code (3.1.0), I subclass Trainer and overwrite evaluate(), without calling self.prediction_loop.

And results in this error:
```
self.prediction_bar.close() 
AttributeError: 'NoneType' object has no attribute 'close'
```

I propose we add `on_predict_begin`  and `on_predict_end`.


Discussion:
Or add this check in `on_evaluate`
```python
if self.prediction_bar is not None:
    self.prediction_bar.close()
```

I think the second solution is better (since it should be there in any case). We can add the events you suggest in the first one if we find other use cases that need them.

Do you want to tackle this in a PR?
I was thinking we can move the opening and closing of `prediction_bar` into separate events, so we don't need the `if` statements. 
But it can be hard to debug if someone miss the opening, so it's probably unnecessary.

I will create a PR with the second solution.

Resolving patch:
diff --git a/src/transformers/trainer_callback.py b/src/transformers/trainer_callback.py
--- a/src/transformers/trainer_callback.py
+++ b/src/transformers/trainer_callback.py
@@ -443,7 +443,8 @@ def on_prediction_step(self, args, state, control, eval_dataloader=None, **kwarg
 
     def on_evaluate(self, args, state, control, **kwargs):
         if state.is_local_process_zero:
-            self.prediction_bar.close()
+            if self.prediction_bar is not None:
+                self.prediction_bar.close()
             self.prediction_bar = None
 
     def on_log(self, args, state, control, logs=None, **kwargs):


Test patch:
Repository: huggingface/transformers

Issue:
BlenderbotSmallTokenizer throws tuple index out of range error for stopword
Using transformers==3.4.0

Script used:
```

from transformers import BlenderbotSmallTokenizer, BlenderbotForConditionalGeneration
mname = 'facebook/blenderbot-90M'
tokenizer = BlenderbotSmallTokenizer.from_pretrained(mname)

sentence = "."
tokenizer(sentence)['input_ids']
```

This throws `IndexError: tuple index out of range`


Discussion:

Resolving patch:
diff --git a/src/transformers/tokenization_blenderbot.py b/src/transformers/tokenization_blenderbot.py
--- a/src/transformers/tokenization_blenderbot.py
+++ b/src/transformers/tokenization_blenderbot.py
@@ -166,6 +166,9 @@ def bpe(self, token: str) -> str:
         tokens = token.split(" ")
         words = []
         for token in tokens:
+            if not len(token):
+                continue
+
             token = token.lower()
             word = tuple(token)
             word = tuple(list(word[:-1]) + [word[-1] + "</w>"])


Test patch:
diff --git a/tests/test_tokenization_blenderbot.py b/tests/test_tokenization_blenderbot.py
--- a/tests/test_tokenization_blenderbot.py
+++ b/tests/test_tokenization_blenderbot.py
@@ -75,6 +75,15 @@ def test_special_tokens_small_tok(self):
         assert src_text != decoded  # I wish it did!
         assert decoded == "i am a small frog ."
 
+    def test_empty_word_small_tok(self):
+        tok = BlenderbotSmallTokenizer.from_pretrained("facebook/blenderbot-90M")
+        src_text = "I am a small frog ."
+        src_text_dot = "."
+        encoded = tok(src_text)["input_ids"]
+        encoded_dot = tok(src_text_dot)["input_ids"]
+
+        assert encoded[-1] == encoded_dot[0]
+
 
 class Blenderbot3BTokenizerTests(unittest.TestCase):
     @cached_property