tf.saved_model.save is not worked on TFElectra* series.

# 🐛 Bug

## Information

Model I am using TFElectraModel:

Language I am using the model on English:

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
import tensorflow as tf
from pathlib import Path
from transformers import TFAutoModel

dump_dir = Path("test_electra")

if not dump_dir.exists():
    dump_dir.mkdir(parents=True)

model = TFAutoModel.from_pretrained("google/electra-base-discriminator")
tf.saved_model.save(model, export_dir=str(dump_dir))
```
```bash
Traceback (most recent call last):
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-a8a019d74543>", line 11, in <module>
    tf.saved_model.save(model, export_dir=str(dump_dir))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 951, in save
    obj, export_dir, signatures, options, meta_graph_def)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 1008, in _build_meta_graph
    checkpoint_graph_view)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/saved_model/signature_serialization.py", line 75, in find_function_to_export
    functions = saveable_view.list_functions(saveable_view.root)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 143, in list_functions
    self._serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 1656, in _list_functions_for_serialization
    Model, self)._list_functions_for_serialization(serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 2750, in _list_functions_for_serialization
    .list_functions_for_serialization(serialization_cache))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/base_serialization.py", line 87, in list_functions_for_serialization
    fns = self.functions_to_serialize(serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 77, in functions_to_serialize
    serialization_cache).functions_to_serialize)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 92, in _get_serialized_attributes
    serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/model_serialization.py", line 53, in _get_serialized_attributes_internal
    serialization_cache))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 101, in _get_serialized_attributes_internal
    functions = save_impl.wrap_layer_functions(self.obj, serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 153, in wrap_layer_functions
    original_fns = _replace_child_layer_functions(layer, serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 272, in _replace_child_layer_functions
    serialization_cache).functions)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 92, in _get_serialized_attributes
    serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/model_serialization.py", line 53, in _get_serialized_attributes_internal
    serialization_cache))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 101, in _get_serialized_attributes_internal
    functions = save_impl.wrap_layer_functions(self.obj, serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 163, in wrap_layer_functions
    '{}_layer_call_and_return_conditional_losses'.format(layer.name))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 503, in add_function
    self.add_trace(*self._input_signature)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 418, in add_trace
    trace_with_training(True)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 416, in trace_with_training
    fn.get_concrete_function(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 547, in get_concrete_function
    return super(LayerCall, self).get_concrete_function(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 959, in get_concrete_function
    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 865, in _get_concrete_function_garbage_collected
    self._initialize(args, kwargs, add_initializers_to=initializers)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 506, in _initialize
    *args, **kwds))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2446, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2777, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2667, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 981, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 441, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 524, in wrapper
    ret = method(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 170, in wrap_with_training_arg
    lambda: replace_training_and_call(False))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 65, in smart_cond
    pred, true_fn=true_fn, false_fn=false_fn, name=name)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/framework/smart_cond.py", line 54, in smart_cond
    return true_fn()
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 169, in <lambda>
    lambda: replace_training_and_call(True),
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 165, in replace_training_and_call
    return wrapped_call(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 566, in call_and_return_conditional_losses
    return layer_call(inputs, *args, **kwargs), layer.get_losses_for(inputs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/transformers/modeling_tf_electra.py", line 292, in call
    hidden_states = self.embeddings([input_ids, position_ids, token_type_ids, inputs_embeds], training=training)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 968, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 71, in return_outputs_and_add_losses
    outputs, losses = fn(inputs, *args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 170, in wrap_with_training_arg
    lambda: replace_training_and_call(False))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 65, in smart_cond
    pred, true_fn=true_fn, false_fn=false_fn, name=name)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/framework/smart_cond.py", line 54, in smart_cond
    return true_fn()
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 169, in <lambda>
    lambda: replace_training_and_call(True),
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 165, in replace_training_and_call
    return wrapped_call(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 541, in __call__
    self.call_collection.add_trace(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 418, in add_trace
    trace_with_training(True)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 416, in trace_with_training
    fn.get_concrete_function(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 547, in get_concrete_function
    return super(LayerCall, self).get_concrete_function(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 959, in get_concrete_function
    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 865, in _get_concrete_function_garbage_collected
    self._initialize(args, kwargs, add_initializers_to=initializers)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 506, in _initialize
    *args, **kwds))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2446, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2777, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2667, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 981, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 441, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 524, in wrapper
    ret = method(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 170, in wrap_with_training_arg
    lambda: replace_training_and_call(False))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 65, in smart_cond
    pred, true_fn=true_fn, false_fn=false_fn, name=name)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/framework/smart_cond.py", line 54, in smart_cond
    return true_fn()
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 169, in <lambda>
    lambda: replace_training_and_call(True),
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 165, in replace_training_and_call
    return wrapped_call(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 566, in call_and_return_conditional_losses
    return layer_call(inputs, *args, **kwargs), layer.get_losses_for(inputs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1627, in get_losses_for
    reachable = tf_utils.get_reachable_from_inputs(inputs, losses)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 140, in get_reachable_from_inputs
    raise TypeError('Expected Operation, Variable, or Tensor, got ' + str(x))
TypeError: Expected Operation, Variable, or Tensor, got None
```
## Expected behavior
```python
import tensorflow as tf
from pathlib import Path
from transformers import TFAutoModel

dump_dir = Path("test_distilbert")

if not dump_dir.exists():
    dump_dir.mkdir(parents=True)

model = TFAutoModel.from_pretrained("distilbert-base-uncased")
tf.saved_model.save(model, export_dir=str(dump_dir))
```
```bash
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x16538b0d0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x165347250>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x16531c2d0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x165e22f90>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x16528bd90>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x16526d850>, because it is not built.
INFO:tensorflow:Assets written to: test_distilbert/assets
```

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Darwin-19.6.0-x86_64-i386-64bit
- Python version: 3.7.7
- PyTorch version (GPU?): 1.5.1 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


ValueError raises when load Flaubert from pre-train with Transformers >=3.0.0

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): **Flaubert**

Language I am using the model on (English, Chinese ...): **French**

The problem arises when using: my own modified scripts

## To reproduce

**Just load Flaubert model with from_pretrained:**
```
MODEL = "flaubert/flaubert_large_cased"
transformer_layer = TFAutoModel.from_pretrained(MODEL, from_pt=True)
```

**Then a ValueError is raised:**

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-22-4235ae313fbd> in <module>()
      1     MODEL = "flaubert/flaubert_large_cased"
----> 2     transformer_layer = TFAutoModel.from_pretrained(MODEL, from_pt=True)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    421         for config_class, model_class in TF_MODEL_MAPPING.items():
    422             if isinstance(config, config_class):
--> 423                 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
    424         raise ValueError(
    425             "Unrecognized configuration class {} for this kind of TFAutoModel: {}.\n"

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    479         if from_pt:
    480             # Load from a PyTorch checkpoint
--> 481             return load_pytorch_checkpoint_in_tf2_model(model, resolved_archive_file, allow_missing_keys=True)
    482
    483         model(model.dummy_inputs, training=False)  # build the network with dummy inputs

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_pytorch_utils.py in load_pytorch_checkpoint_in_tf2_model(tf_model, pytorch_checkpoint_path, tf_inputs, allow_missing_keys)
     91
     92     return load_pytorch_weights_in_tf2_model(
---> 93         tf_model, pt_state_dict, tf_inputs=tf_inputs, allow_missing_keys=allow_missing_keys
     94     )
     95

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_pytorch_utils.py in load_pytorch_weights_in_tf2_model(tf_model, pt_state_dict, tf_inputs, allow_missing_keys)
    123
    124     if tf_inputs is not None:
--> 125         tf_model(tf_inputs, training=False)  # Make sure model is built
    126
    127     # Adapt state dict - TODO remove this and update the AWS weights files instead

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_xlm.py in call(self, inputs, **kwargs)
    635             heads.
    636         """
--> 637         outputs = self.transformer(inputs, **kwargs)
    638         return outputs
    639

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_flaubert.py in call(self, inputs, attention_mask, langs, token_type_ids, position_ids, lengths, cache, head_mask, inputs_embeds, training, output_attentions, output_hidden_states)
    267                 tensor_normalized = self.layer_norm1[i](tensor)
    268                 attn_outputs = self.attentions[i](
--> 269                     [tensor_normalized, attn_mask, None, cache, head_mask[i]], training=training
    270                 )
    271                 attn = attn_outputs[0]

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_xlm.py in call(self, inputs, training)
    139         Self-attention (if kv is None) or attention over source sentence (provided by kv).
    140         """
--> 141         input, mask, kv, cache, head_mask, output_attentions = inputs
    142         # Input is (bs, qlen, dim)
    143         # Mask is (bs, klen) (non-causal) or (bs, klen, klen)

ValueError: not enough values to unpack (expected 6, got 5)
```
## Expected behavior
Load the model successfully.

- It can be achieved with Transformers version <= 2.11.0

- I also try CamemBERT, and Bert-like models in other languages like tr, ru, es, pt, it. None of them have this problem.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: **>=3.0.0 ( 3.0.0, 3.0.1, 3.0.2 all have this problem.)**

(I try this on google colab with GPU and TPU. The environment does not matter here. But I still provide the one with GPU.)
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1+cu101 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Error when using np.where() during squad tokenization

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):Bert

Language I am using the model on (English, Chinese ...):English

The problem arises when using:
* [x] the official example scripts: squad1.1
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: SQUaD task

* [ ] my own task or dataset: (give details below)

## To reproduce
 I'm confused about this line:
https://github.com/huggingface/transformers/blob/896300177bf9f35feac4698370212a80a5ab6138/src/transformers/data/processors/squad.py#L230

Should this be
```
pad_token_indices = np.where(np.array(span["input_ids"]) == tokenizer.pad_token_id)
```
Cause "p_mask: mask with 1 for token than cannot be in the answer (0 for token which can be in an answer)", and the padding positions should not be answers

Steps to reproduce the behavior:
```python
span["input_ids"] = [1,1,3,0,0,0]
np.where([1,1,3,0,0,0] == tokenizer.pad_token_id)
# return (array([], dtype=int64),)

np.where(np.array([1,1,3,0,0,0]) == tokenizer.pad_token_id)
# return (array([3, 4, 5]),)
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
As described above.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 7a68d401388bc68f10dfeb591709352736a6c0b6
- Platform:
- Python version: python3.6.8
- PyTorch version (GPU?):1.5.0 CPU
- Tensorflow version (GPU?):
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?:NO


🐛 Inconsistencies between BartTokenizer and BartTokenizerFast

# 🐛 Bug

## Description

It's possible to use the argument `add_prefix_space` with `BartTokenizer` :

```
from transformers import BartTokenizer

t = BartTokenizer.from_pretrained("facebook/bart-large")
x = t("This is an example.", add_prefix_space=True)
```

But when doing the same with `BartTokenizerFast` :
```
from transformers import BartTokenizerFast

t = BartTokenizerFast.from_pretrained("facebook/bart-large")
x = t("This is an example.", add_prefix_space=True)
```

It throws the following error :

```
ValueError: Keyword arguments {'add_prefix_space': True} not recognized.
```

## To reproduce

[Colab notebook](https://colab.research.google.com/drive/1f0W2llsJfVIkXsYk0XK1C3oiy2xSqYOx?usp=sharing)

## Work-around

It seems working if the argument is specified in the constructor :
```
from transformers import BartTokenizerFast

t = BartTokenizerFast.from_pretrained("facebook/bart-large", add_prefix_space=True)
x = t("This is an example.")
```

@sshleifer


[s2s] Delete useless method, log tokens_per_batch

- trimming is already performed in `collate_fn`, there is no need to call it again.
- val_summ_len can be called gen_len so that it makes sense for a translation model.
- new metric: tpb = how many non pad tokens per batch.  Useful for debugging/verifying speedups.
cc @patil-suraj


Proposal: seq2seq tokenizers expose a prepare_seq2seq_batch method

Core: Tokenization

Internals of the library; Tokenization.

This has been useful in `MarianTokenizer.prepare_translation_batch` and `MBartTokenizer.prepare_translation_batch`.

and would also be a useful addition for `T5Tokenizer`, and `BartTokenizer`.
@LysandreJik mentioned this in a PR so I wanted to see what others thought before starting work.
@patrickvonplaten @sgugger @mfuntowicz @patil-suraj ?

Proposed signature (I would not add this to PretrainedModel, just rename the Marian/MBart implems and add implems for Bart, T5. When pegasus, blenderbot get merged, I would also add implems there consider a helper function.)

```python
def prepare_seq2seq_batch(
        self,
        src_texts: List[str],
        tgt_texts: Optional[List[str]] = None,
        max_length: Optional[int] = None,
		max_target_length:Optional[int] = None,
        padding: str = "longest",
        **kwargs,
    ) -> BatchEncoding:
        if max_length is None:
            max_length = self.max_len
		if max_target_length is None: # default to max_length
			max_target_length =  max_length# no need to specify twice for translation
        model_inputs: BatchEncoding = self(
            src_texts,
            add_special_tokens=True,
            return_tensors=return_tensors,
            max_length=max_length,
            padding=padding,
            truncation=True,
            **kwargs,
        )
        if tgt_texts is None:
            return model_inputs
		# Here classes can implement logic to put decoder_start_token_id at the front if they want,
		# or deal with language codes for multilingual models.
        decoder_inputs: BatchEncoding = self(
            tgt_texts,
            add_special_tokens=True,
            return_tensors=return_tensors,
            padding=padding,
            max_length=max_target_length,
            truncation=True,
            **kwargs,
        )
        for k, v in decoder_inputs.items():
            model_inputs[f"decoder_{k}"] = v
        return model_inputs # still a BatchEncoding
```


[s2s] Don't mention packed data in README


model.roberta.from_pretrained() fails to change the parameters

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Hi all, my problem is "how to use a pretrained 3-way sequence classification model to fine-tune on a 2-way classification task". I am using "https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_glue.py";
in the code I change

```
    model_args.model_name_or_path = 'roberta-large'
    model = AutoModelForSequenceClassification.from_pretrained(
        model_args.model_name_or_path,
        from_tf=bool(".ckpt" in model_args.model_name_or_path),
        config=config,
        cache_dir=model_args.cache_dir,
    )
    model.roberta.from_pretrained(path_to_my_3way_pretrained_model)
```
This is what I tried. Pls note that I can not used the "AutoModelForSequenceClassification.from_pretrained()" to load "path_to_my_3way_pretrained_model" directly because there will be class mismatch (i.e., 3-way model does not apply to 2-way task); but I think no matter it is 3-way or 2-way sequence classification, what their architectures share is the roberta part; so I used the model.roberta to load the parameters from my pretrained model.

However, I found the roberta parameters do not change, I checked as follows
```
    for name, param in model.named_parameters():
        if param.requires_grad and name == 'roberta.encoder.layer.16.attention.self.value.weight':
            print('new:', name, param.data)
```

Any clue why it doesn't work? Or any solution how to use a fine-tune a pretrained 3-way model on a 2-way task? Thanks


[s2s] dont document packing because it hurts performance


Create README.md

model card

Related to pretrained model cards


Cannot use the RobertaForMultipleChoice model for processing multiple choice questions with 4 options

Hello,

When I try to use the `RobertaForMultipleChoice` pretrained model with the code below, it generates an error:

```python
from transformers import RobertaTokenizer, RobertaForMultipleChoice
import torch
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
model = RobertaForMultipleChoice.from_pretrained('roberta-base')
prompt = "In Italy, pizza served in formal settings, such as at a restaurant, is presented unsliced."
choice0 = "It is eaten with a fork and a knife."
choice1 = "It is eaten while held in the hand."
choice2 = "It is eathen with a napkin."
choice3 = "It is not eatable."
labels = torch.tensor(0).unsqueeze(0)  # choice0 is correct (according to Wikipedia ;)), batch size 1
encoding = tokenizer([[prompt, prompt,prompt,prompt], [choice0, choice1, choice2, choice3]], return_tensors='pt', return_token_type_ids=True,padding=True)
```

The error message is:

```python

  File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 534, in _batch_encode_plus
    ids, pair_ids = ids_or_pair_ids

ValueError: too many values to unpack (expected 2)
```

What am I doing wrong here? Thank you.


Error in the RobertaTokenizer?

Hello,

I think I spotted an error in the `RobertaTokenizer` (`AutoTokenizer`).

I tested the `tokenizer` function by using the code below:

```python
import torch
from torch.nn import CrossEntropyLoss
from matplotlib import pyplot as plt
from transformers import RobertaTokenizer, RobertaForMultipleChoice, AdamW, get_constant_schedule
from transformers import AutoTokenizer
import numpy as np
import pandas as pd
import pickle
import dill
from matplotlib.pyplot import plot, savefig, xlim, figure, ylim, legend, boxplot, setp, axes, xlabel, ylabel, xticks
import gc
import math
import time
from random import seed
from random import randint
import sys
import statistics
from numpy import nan
import scipy.stats as ss
from statistics import mode
from pylab import *
from numpy import arange

# import the pre-trained HuggingFace RobertaTokenizer
tokenizer = AutoTokenizer.from_pretrained('roberta-base')

# get the pre-trained HuggingFace RobertaForMultipleChoice and resize the token embeddings after adding the special token
best_model_roberta = RobertaForMultipleChoice.from_pretrained('roberta-base', output_hidden_states = True)

sequence_a = 'china is a very large country .'
sequence_b = 'indeed'

tokenizer(sequence_a, sequence_b, padding = True, return_token_type_ids=True, return_tensors="pt")
```
and below is the corresponding output:

```
{
'input_ids': tensor([[ 0,  611,  1243,  16,  10,  182,  739,  247,  479, 2,  2, 2028, 10247, 2]]),
 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]),
'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])
}
```

The output above is not what I was expecting to get from the `tokenizer` function. Since I am inputting `sequence_a` along with `sequence_b`, I was expecting my `token_type_ids` to be: `tensor([[0,0,0,0,0,0,0,0,0,0,0,1,1,1]])`, but the output does not include any 1's...everything is 0. Perhaps this is the error within the `RobertaTokenizer`?

Thank you,


Loading and running on CPU, the RoBERTa model traced/saved on GPU.

# ❓ Questions & Help

## Details
I have installed the transformers from source and I am trying to trace/save roberta_base model for sequence classification on GPU and load it on CPU, similar to the issue [#5664](https://github.com/huggingface/transformers/issues/5664) that was addressed [PR #5773](https://github.com/huggingface/transformers/pull/5773)  and I am facing similar issues with position_embeddings. I was wondering is any step is missing or RoBERTa is also covered in [PR #5773](https://github.com/huggingface/transformers/pull/5773). Thanks.

## Sample Script

```python
import transformers
from pathlib import Path
import os
import json
import torch
from transformers import (AutoModelForSequenceClassification, AutoTokenizer, AutoModelForQuestionAnswering,
 AutoModelForTokenClassification, AutoConfig)

print('Transformers version',transformers.__version__)
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
dummy_input = "This is a dummy input for torch jit trace"
max_length=20
config = AutoConfig.from_pretrained('roberta-base',num_labels=2,torchscript=True)
model = AutoModelForSequenceClassification.from_pretrained('roberta-base', config=config)
tokenizer = AutoTokenizer.from_pretrained('roberta-base',do_lower_case=True)
inputs = tokenizer.encode_plus(dummy_input,max_length = max_length,pad_to_max_length = True,truncation=True, add_special_tokens = True, return_tensors = 'pt')

print(inputs.keys())
input_ids = inputs["input_ids"].to(device)
attention_mask = inputs["attention_mask"].to(device)
model.to(device).eval()
traced_model = torch.jit.trace(model, (input_ids,attention_mask))
torch.jit.save(traced_model, "Roberta_cuda.pt")

print(traced_model.graph)
print("\n")
print("Load model onto CPU")
loaded = torch.jit.load("Roberta_cuda.pt", map_location=torch.device("cpu"))
print("\n")
print(loaded.graph)
outputs = loaded(input_ids.to("cpu"),attention_mask.to("cpu"))
print(outputs)
```

## Error

Traceback (most recent call last):
  File "test.py", line 100, in <module>
    outputs = loaded(input_ids.to("cpu"),attention_mask.to("cpu"))
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
  File "code/__torch__/transformers/modeling_roberta.py", line 10, in forward
    attention_mask: Tensor) -> Tuple[Tensor]:
    _0 = self.classifier
    _1 = (self.roberta).forward(input_ids, attention_mask, )
          ~~~~~~~~~~~~~~~~~~~~~ <--- HERE
    return ((_0).forward(_1, ),)
class RobertaModel(Module):
  File "code/__torch__/transformers/modeling_roberta.py", line 32, in forward
    _9 = torch.to(extended_attention_mask, 6, False, False, None)
    attention_mask0 = torch.mul(torch.rsub(_9, 1., 1), CONSTANTS.c0)
    _10 = (_3).forward((_4).forward(input_ids, input, ), attention_mask0, )
                        ~~~~~~~~~~~ <--- HERE
    _11 = (_2).forward(_10, )
    return _10
  File "code/__torch__/transformers/modeling_roberta.py", line 59, in forward
    input0 = torch.to(_19, dtype=4, layout=0, device=torch.device("cuda:0"), pin_memory=False, non_blocking=False, copy=False, memory_format=None)
    _20 = (_16).forward(input_ids, )
    _21 = (_15).forward(input0, )
           ~~~~~~~~~~~~ <--- HERE
    _22 = (_14).forward(input, )
    input1 = torch.add(torch.add(_20, _21, alpha=1), _22, alpha=1)
  File "code/__torch__/torch/nn/modules/sparse/___torch_mangle_0.py", line 7, in forward
  def forward(self: __torch__.torch.nn.modules.sparse.___torch_mangle_0.Embedding,
    input: Tensor) -> Tensor:
    position_embeddings = torch.embedding(self.weight, input, 1, False, False)
                          ~~~~~~~~~~~~~~~ <--- HERE
    return position_embeddings

Traceback of TorchScript, original code (most recent call last):
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/functional.py(1724): embedding
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/modules/sparse.py(114): forward
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/modules/module.py(534): _slow_forward
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/modules/module.py(548): __call__
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/transformers/modeling_bert.py(202): forward
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/transformers/modeling_roberta.py(76): forward
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/modules/module.py(534): _slow_forward
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/modules/module.py(548): __call__
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/transformers/modeling_bert.py(792): forward
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/modules/module.py(534): _slow_forward
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/modules/module.py(548): __call__
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/transformers/modeling_roberta.py(344): forward
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/modules/module.py(534): _slow_forward
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/nn/modules/module.py(548): __call__
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/jit/__init__.py(1027): trace_module
/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/torch/jit/__init__.py(875): trace
test.py(91): <module>
RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_index_select


lightning_base: new clarg: lr_scheduler=polynomial_decay

lr_scheduler should be a string with many options, including polynomial_decay, like this [command](https://github.com/pytorch/fairseq/blob/master/examples/mbart/README.md#finetune-on-en-ro)


lightning_base: new clarg: adam_betas

should be able to pass `adam_betas` through command line like this [command](https://github.com/pytorch/fairseq/blob/master/examples/mbart/README.md#finetune-on-en-ro)


link to README.md


fix typo

wrong filename passed to `tar`


Add fire to setup.cfg to make isort happy


Make all data collators accept dict

+ one file changed by make style for some reason.

I'm only keeping the inputs, for more evolved data collation, I think it's best to let the user write their own subclass.


[Performance improvement] "Bad tokens ids" optimization

Running some benchmarks I noticed that the generation pipeline was varying quite a bit in terms of execution time. Especially the banned token masking seems to be fairly expensive (I ran some experiments where up to 30% of the time for an entire generation process was spent in this step - which seems too high considering its expected simplicity).

This PR accelerates the entire generation pipeline for models using a `bad_words_ids` in their configuration by around 20% on a GPU-enabled node (this includes for example translation using the Marian models).

The following changes contribute to the performance improvement:
- Single conversion from tensor to list. Previous approach was accessing the GPU buffer for every banned token and every batch element, causing this operation to be slower than the entire forward pass through the model
- Vectorized update of the banned tokens using a masked fill
- Skipping the EOS token for the banned tokens (avoiding a potential duplicate masking)


[fix] no warning for position_ids buffer

Fixes [#6044](https://github.com/huggingface/transformers/issues/6044)

The failing tests check that there are no missing keys for bart, but morgan's recent change started registering a position_ids buffer in `__init__` for four models, with more expected.

Since we do not expect weights to be saved for position_ids, this PR adds the pattern `position_ids` to authorized_missing_keys for all PT models.


Add new AutoModel classes in pipeline

This PR removes the about to be deprecated `AutModelWithLMHead` class and uses the new `AutoModelForSeq2SeqLM`, `AutoModelForCausalLM` and `AutoModelForMaskedLM` for `translation`, `text-generation` and `fill-mask` pipelines respectively.

Regrading issue #6060

@sgugger


Pipelines should use tuples instead of namedtuples

Fix https://github.com/huggingface/transformers/issues/5713

The ONNX conversion cannot handle other objects than tuples, lists and variables. Since calling the conversion script via the command line uses a pipeline, and pipelines cannot be configured with specific model kwargs, the change is so that pipelines manage tuples instead of namedtuples (not a breaking change).


new AutoModel classes in pipeline

The `translation`, `text-generation` and `fill-mask` pipelines still use the `AutoModelWithLMHead` class but from the warning it seems that it's going to be deprecated. Is there any specific reason for this or can we use the new `AutoModelForSeq2SeqLM` for translation, `AutoModelCausalLM` for text-generation and `AutoModelForMaskedLM` for fill-mask pipeline ? If yes then I'll be happy to open a PR.

@sgugger


Errors while using TFAutoModelForMultipleChoice and TFTrainer on winogrande dataset

# ❓ Questions & Help
I'm using TFAutoModelForMultipleChoice and TFTrainer to train on winogrande dataset.
The dataset is from [here](https://leaderboard.allenai.org/winogrande/submissions/get-started)

And I got an error while I starting training.
```python
Traceback (most recent call last):
  File "run_wsc.py", line 233, in <module>
    main()
  File "run_wsc.py", line 208, in main
    trainer.train()
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/transformers/trainer_tf.py", line 358, in train
    for step, training_loss in enumerate(self._training_steps(train_ds, optimizer)):
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/transformers/trainer_tf.py", line 401, in _training_steps
    for i, loss in enumerate(self._accumulate_next_gradients(ds)):
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/transformers/trainer_tf.py", line 434, in _accumulate_next_gradients
    yield _accumulate_next()
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 568, in __call__
    result = self._call(*args, **kwds)
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 615, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 497, in _initialize
    *args, **kwds))
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 2389, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 2703, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 2593, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py", line 978, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 439, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py", line 968, in wrapper
    raise e.ag_error_metadata.to_exception(e)
TypeError: in converted code:

    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/transformers/trainer_tf.py:430 _accumulate_next  *
        return self._accumulate_gradients(per_replica_features, per_replica_labels)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/transformers/trainer_tf.py:440 _accumulate_gradients  *
        per_replica_loss = self.args.strategy.experimental_run_v2(
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/distribute/one_device_strategy.py:180 experimental_run_v2
        return super(OneDeviceStrategy, self).experimental_run_v2(fn, args, kwargs)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/transformers/trainer_tf.py:453 _forward  *
        per_example_loss, _ = self._run_model(features, labels, True)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/transformers/trainer_tf.py:474 _run_model  *
        loss, logits = self.model(features, labels=labels, training=training)[:2]
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py:778 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/transformers/modeling_tf_bert.py:1114 call  *
        loss = self.compute_loss(labels, reshaped_logits)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/transformers/modeling_tf_utils.py:134 compute_loss  *
        if shape_list(logits)[1] == 1:
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/autograph/operators/control_flow.py:918 if_stmt
        basic_symbol_names, composite_symbol_names)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/autograph/operators/control_flow.py:956 tf_if_stmt
        error_checking_orelse)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py:507 new_func
        return func(*args, **kwargs)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/ops/control_flow_ops.py:1174 cond
        return cond_v2.cond_v2(pred, true_fn, false_fn, name)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/ops/cond_v2.py:83 cond_v2
        op_return_value=pred)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py:983 func_graph_from_py_func
        expand_composites=True)
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py:568 map_structure
        structure[0], [func(*x) for x in entries],
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py:568 <listcomp>
        structure[0], [func(*x) for x in entries],
    /Users/qixinli/anaconda/envs/a2/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py:943 convert
        (str(python_func), type(x)))

    TypeError: To be compatible with tf.contrib.eager.defun, Python functions must return zero or more Tensors; in compilation of <function tf_if_stmt.<locals>.error_checking_body at 0x158e993b0>, found return value of type <class 'tensorflow.python.keras.losses.MeanSquaredError'>, which is not a Tensor.
```

## Details

```python
import logging
import os
from dataclasses import dataclass, field
from enum import Enum
from typing import Dict, Optional

import numpy as np
import tensorflow as tf

from transformers import (
    AutoConfig,
    AutoTokenizer,
    EvalPrediction,
    HfArgumentParser,
    PreTrainedTokenizer,
    TFTrainer,
    TFTrainingArguments,
    TFAutoModelForMultipleChoice
)

from wsc_processor import (
    WinograndeProcessor,
    convert_multiple_choice_examples_to_features,
    compute_metrics
)

class Split(Enum):
    train = "train"
    dev = "validation"
    test = "test"

def get_wsc_ds(
    tokenizer: PreTrainedTokenizer, max_seq_length: Optional[int] = None, mode: Split = Split.train, data_dir:str=None
):
    processor = WinograndeProcessor()
    if mode == Split.train:
        examples = processor.get_train_examples(data_dir)
    elif mode == Split.dev:
        examples = processor.get_dev_examples(data_dir)
    else:
        examples = processor.get_test_examples(data_dir)

    features = convert_multiple_choice_examples_to_features(examples, label_list=processor.get_labels(), tokenizer=tokenizer, max_seq_length=max_seq_length, output_mode="multiple_choice")

    def gen():
        for ex in features:
            inputs = []
            masks = []
            segments = []
            for feature in ex.option_features:
                inputs.append(feature["input_ids"])
                masks.append(feature["input_mask"])
                segments.append(feature["segment_ids"])
            yield (
                {
                    "input_ids": inputs,
                    "attention_mask": masks,
                    "token_type_ids": segments,
                },
                ex.label,
            )

    ds = tf.data.Dataset.from_generator(
        gen,
        ({"input_ids": tf.int32, "attention_mask": tf.int32, "token_type_ids": tf.int32}, tf.int64),
        (
            {
                "input_ids": tf.TensorShape([None,None]),
                "attention_mask": tf.TensorShape([None,None]),
                "token_type_ids": tf.TensorShape([None,None]),
            },
            tf.TensorShape([]),
        ),
    )
    return ds

logger = logging.getLogger(__name__)

@dataclass
class GlueDataTrainingArguments:
    task_name: str = field(default="wsc")
    data_dir: str = field(default="./data/WSC")
    max_seq_length: int = field(
        default=128,
        metadata={
            "help": "The maximum total input sequence length after tokenization. Sequences longer "
            "than this will be truncated, sequences shorter will be padded."
        },
    )
    overwrite_cache: bool = field(
        default=False, metadata={"help": "Overwrite the cached training and evaluation sets"}
    )

@dataclass
class ModelArguments:
    model_name_or_path: str = field(
        default="roberta-base",
        metadata={"help": "Path to pretrained model or model identifier from huggingface.co/models"}
    )
    config_name: Optional[str] = field(
        default=None, metadata={"help": "Pretrained config name or path if not the same as model_name"}
    )
    tokenizer_name: Optional[str] = field(
        default=None, metadata={"help": "Pretrained tokenizer name or path if not the same as model_name"}
    )
    use_fast: bool = field(default=False, metadata={"help": "Set this flag to use fast tokenization."})
    cache_dir: Optional[str] = field(
        default="./model/", metadata={"help": "Where do you want to store the pretrained models downloaded from s3"}
    )

def main():
    parser = HfArgumentParser((ModelArguments, GlueDataTrainingArguments, TFTrainingArguments))
    model_args, data_args, training_args = parser.parse_args_into_dataclasses()

    if (
        os.path.exists(training_args.output_dir)
        and os.listdir(training_args.output_dir)
        and training_args.do_train
        and not training_args.overwrite_output_dir
    ):
        raise ValueError(
            f"Output directory ({training_args.output_dir}) already exists and is not empty. Use --overwrite_output_dir to overcome."
        )

    # Setup logging
    logging.basicConfig(
        format="%(asctime)s - %(levelname)s - %(name)s -   %(message)s",
        datefmt="%m/%d/%Y %H:%M:%S",
        level=logging.INFO,
    )

    logger.info("Training/evaluation parameters %s", training_args)

    output_mode = "classification"

    config = AutoConfig.from_pretrained(
        model_args.config_name if model_args.config_name else model_args.model_name_or_path,
        # num_labels=1,
        finetuning_task=data_args.task_name,
        cache_dir=model_args.cache_dir
    )
    tokenizer = AutoTokenizer.from_pretrained(
        model_args.tokenizer_name if model_args.tokenizer_name else model_args.model_name_or_path,
        cache_dir=model_args.cache_dir
    )

    # with training_args.strategy.scope():
    model = TFAutoModelForMultipleChoice.from_pretrained(
        model_args.model_name_or_path,
        from_pt=bool(".bin" in model_args.model_name_or_path),
        config=config,
        cache_dir=model_args.cache_dir,
    )

    # Get datasets
    train_dataset = (
        get_wsc_ds(tokenizer=tokenizer, max_seq_length=data_args.max_seq_length,data_dir=data_args.data_dir)
        if training_args.do_train
        else None
    )
    eval_dataset = (
        get_wsc_ds(tokenizer=tokenizer, max_seq_length=data_args.max_seq_length,mode=Split.dev, data_dir=data_args.data_dir)
        if training_args.do_eval
        else None
    )

    def metric(p: EvalPrediction) -> Dict:
        if output_mode == "classification":
            preds = np.argmax(p.predictions, axis=1)
        else:
            preds = np.squeeze(p.predictions)
        return compute_metrics("winogrande", preds, p.label_ids)

    # Initialize our Trainer
    trainer = TFTrainer(
        model=model,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=eval_dataset,
        compute_metrics=metric,
    )

    # Training
    if training_args.do_train:
        trainer.train()
        trainer.save_model()
        tokenizer.save_pretrained(training_args.output_dir)
    return results

if __name__ == "__main__":
    main()
```
the wsc_processor is from [here](https://github.com/allenai/winogrande/blob/master/scripts/utils.py)

I wonder what is wrong in the code (am I wrong in the get_wsc_ds() method ?) , and how to fix it?


Create README.md


Transformer layers + Functional API is failed but subclass is successful

# 🐛 Bug

## Information

Model I am using (Bert):

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce
**successful**
```python
# define model via subclassing
class MyModel(tf.keras.Model):
    def __init__(self, item_dim, num_layers, num_heads, max_len, **kwargs):
        super(MyModel, self).__init__(**kwargs)
        self.item_dim = item_dim
        self.num_layers = num_layers
        self.num_heads = num_heads
        self.max_len = max_len
        self.config = BertConfig(hidden_size=item_dim, num_hidden_layers=num_layers,
                            num_attention_heads=num_heads, intermediate_size=item_dim*4,
                            max_position_embeddings=max_len)
        self.bert = TFBertMainLayer(config=self.config)
        self.dense = Dense(1, activation='sigmoid')
    def call(self, inputs):
        seq_emb = self.bert(inputs=None, inputs_embeds=inputs)[0]
        last_token_emb = seq_emb[:, -1, :]
        outputs = self.dense(last_token_emb)
        return outputs
```
**failed**
```python
def build_model(item_dim, num_layers, num_heads, max_len):
    config = BertConfig(hidden_size=item_dim, num_hidden_layers=num_layers,
                        num_attention_heads=num_heads, intermediate_size=item_dim*4,
                        max_position_embeddings=max_len)
    bert = TFBertMainLayer(config=config)
    inputs = Input(shape=(max_len, item_dim), dtype=tf.float32, name='inputs')
    # pre-training vectors to bert
    seq_emb = bert(inputs=None, inputs_embeds=inputs)[0]
    last_token_emb = seq_emb[:, -1, :]
    outputs = Dense(1, activation='sigmoid')(last_token_emb)
    model = Model(inputs=inputs, outputs=outputs)
    return model
```
Errors with
`ValueError: It appears you are trying to construct a functional model, but not all of the inputs in the first positional argument of your layer call are symbolic tensors. (Input objects, or the output of another layer) Functional models cannot correctly track custom layers unless all values in the first call argument are symbolic.`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
* Run the code successfully when use transformer layers and functional api.
<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
https://colab.research.google.com/gist/Douboo/80dfa91917c176c530ef64be244a99a6/untitled2.ipynb
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.1.0
- Platform: google colab
- Python version: 3.7
- Tensorflow version (GPU?): 2.3.0
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


Empty assert hunt

Empty asserts are bad for debugging. I tried to remove them all and to add helpful Pytorch-style messages with the shapes of the corresponding objects when they were mismatched-lengths checks + the file paths when they were file-not-found checks.


add another e.g. to avoid confusion


Errors while creating a subclass of BertForTokenClassification in run_ner.py file

 Question 1) I want to skip calling AutoModelForTokenClassification ( https://github.com/huggingface/transformers/blob/f7f03b22dc15543317635770f312adf4513303d0/examples/token-classification/run_ner.py#L158 ) and directly use  BertForTokenClassification. When I am directly using BertForTokenClassification in run_ner.py using the same parameters which were fed to AutoModelForTokenClassification, it does not load the weights from pretrained BERT checkpoint.
Can you suggest what changes I have to make in the input parameters (most likely in the "config") such that it loads the weights from pretrained BERT checkpoint.

Question 2) I also tried to create a subclass of BertForTokenClassification and call it within AutoModelForTokenClassification. In this case, it throws an error - AttributeError: 'BertConfig' object has no attribute 'use_return_tuple'   .
Can anyone please suggest something to fix this error which is caused while just using a subclass of BertForTokenClassification instead of the original BertForTokenClassification class. Please note that the subclass is exactly same as BertForTokenClassification.

Thanks.


Errors when creating a subclass of "BertForTokenClassification"


add gpt2 padding for tflite

# ❓ Questions & Help

Since tflite requires a fixed input length, how can we correctly add padding for gpt2 input with a fix length so that it won't affect the final result?

```
import tensorflow as tf
from transformers import *
gpt2_model = TFGPT2LMHeadModel.from_pretrained('distilgpt2')
class WrapModel(tf.keras.models.Model):
    def __init__(self, transformer):
        super(WrapModel, self).__init__()
        self.transformer = transformer

    def call(self, input_ids, **kwargs):
        inputs = {"inputs": input_ids}
        outputs = self.transformer(**inputs)
        next_token_logits = outputs[0][:, -1, :]

        # Greedy decoding
        next_token = tf.math.argmax(next_token_logits, axis=-1, output_type=tf.int32)
        return {"decoded_ids": next_token}

w = WrapModel(gpt2_model)
input_layer = tf.keras.layers.Input(shape=(1, 10), dtype=tf.int32, name='input_ids')
prediction_model = w(input_layer)
tf_model = tf.keras.models.Model(inputs=input_layer, outputs=prediction_model)
export_dir = "./gpt2_tmp/"
tf.keras.models.save_model(tf_model, export_dir)

saved_model_dir = "./gpt2_tmp/"
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.experimental_new_converter = True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()
```

I am not clear how we can use attention_mask to solve this concern based on this one. https://github.com/huggingface/transformers/issues/2630


examples/seq2seq:  add a dataloader that supports dynamic batch size

Should follow the logic of `load_langpair_dataset` in fairseq, roughly.
Batches should be created such that they include N(default 1024) tokens of source documents, however many examples that requires.


CI: run tests against torch=1.6

Help wanted

Extra attention is needed, help appreciated

Tests

Related to tests

Through github or circleci.
If github actions:
copy `.github/self-scheduled.yml` to `.github/torch_future.yml` and modify the install steps.


examples/seq2seq/test_bash_script.py :: actually learn something

Related to tests

At the moment validation bleu barely gets above zero in the tests, so they don't really prove much about our code.

we could use a larger model like sshleifer/student_marian_6_3, and more data, and train for 10 minutes . This would allows us to test whether changing default parameters/batch techniques obviously degrades performance.

The github actions CI reuses it's own disk, so this will only run there and hopefully not have super slow downloads.


examples/seq2seq/test_bash_script.py covers summarization

Related to tests


Feed to forward new parameters as computed manually by update rule

Hi,

I would like to update the parameters of BertModel manually while training without using automatic optimizer to do it. I am trying meta-learning on BertModel and if I use automatic optimization on both inner and outer loops I get in-place operation errors. I also tried deepcopying but I cannot use deepcopy with DataParallel. So, I have opted for configuring my own manual optimization mechanism. how can I feed new parameters I computed from autograd of loss with respect to old parameters and update rule into the forward pass of BertModel manually? Is it a feature that exists? Otherwise is there a workaround?

Thanks,


is_pretokenized seems to work incorrectly

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): roberta

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
*  the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

I use `RobertaTokenizerFast` on pretokenized text, but problem arises when I switch to slow version too

The tasks I am working on is:
*  an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

I am trying to implement sliding window for roberta

## To reproduce
I use `tokenizer.tokenize(text)` method to tokenize whole text (1-3 sentences), when I divide tokens into chunks and try to use `__call__` method (I also tried `encode`) with `is_pretokenized=True` argument, but this creates additional tokens (like 3 times more then should be). I worked this around by using `tokenize` -> `convert_tokens_to_ids` -> `prepare_for_model` -> `pad` pipeline, but I believe that batch methods should be faster and more memory efficient
Steps to reproduce the behavior:

0. `tokenizer = AutoTokenizer.from_pretrained('roberta-base', add_prefix_space=True, use_fast=True)`
1. `ex_text = 'long text'`
2. `tokens = tokenizer.tokenize(ex_text)`
3. `examples = [tokens[i:i+126] for i in range(0, len(tokens), 100)]`
4. `print(len(tokenizer(examples, is_pretokenized=True)['input_ids'][0])) # this prints more than 128`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

I would expect to get result similar to result I get when I use

```
tokens = tokeniser.tokenize(ex_text)
inputs = tokenizer.convert_tokens_to_ids(tokens)
inputs = [inputs[i:i+126] for i in range(0, len(tokens), 100)]
inputs = [tokenizer.prepare_for_model(example) for example in inputs]
inputs = tokenizer.pad(inputs, padding='longest')
```
Am I doing something wrong or it's unexpected behaviour?

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: MacOs
- Python version: 3.8.3
- PyTorch version (GPU?): 1.5.1 (no GPU)
- Tensorflow version (GPU?): NO
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO

EDIT:
I see that when I use `__call__` it actually treat ` Ġ` as 2 tokens:
`tokenizer(tokenizer.tokenize('How'), is_pretokenized=True)['input_ids']`
`out: [0, 4236, 21402, 6179, 2]` where 4236, 21402 is ` Ġ`


Test BART's memory  consumption

Benchmarks

Issues related to Memory regressions in tests and scripts

Tests

Related to tests

WIP

Label your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress

- this can run on GPU only and be marked `@slow`
- check how much memory bart is using at `__init__`
- assert that it doesn't use more than 110% of that.
- check how much memory bart uses on a single forward pass. (optionally test this in fp16).
- assert that it doesn't use more than 110% of that.
- check how much memory bart uses on a single forward and backward pass.
- assert that it doesn't use more than 110% of that.

### Bonus:
- add similar asserts for timing!
- let the test run and check memory on CPU (make sure that if pytest is run with `-n -8` the test still passes!
- add a test to `test_modeling_common.py` to make it easy for all models to test this.
- add a test to `test_modeling_common_tf.py` to make it easy for all TF models to test this.

The benchmarking utilities may help.
It may also help to use `torch.cuda.max_memory...`

@patrickvonplaten may have further thoughts!


slow from_pretrained failures

github actions has [full traceback](https://github.com/huggingface/transformers/runs/910644181?check_suite_focus=true)

Failures are all like
```python
        for model_name in BERT_PRETRAINED_MODEL_ARCHIVE_LIST[:1]:
            config = BertConfig.from_pretrained(model_name)
            self.assertIsNotNone(config)
            self.assertIsInstance(config, PretrainedConfig)

            model = BertModel.from_pretrained(model_name)
            model, loading_info = BertModel.from_pretrained(model_name, output_loading_info=True)
            self.assertIsNotNone(model)
            self.assertIsInstance(model, PreTrainedModel)
            for value in loading_info.values():
>               self.assertEqual(len(value), 0)
E               AssertionError: 1 != 0
```

@sgugger is that from your change?


docs(pretrained_models): fix num parameters

@julien-c  This PR corrects the number of parameters of BERT based models.

Sometimes the difference between a given model and its pairs is important. For instance:
`bert-base-uncased` has **110M parameters** but `bert-base-multilingual-cased` has more than **178M parameters**, even if both models share the same architecture (12-layer, 768-hidden, 12-heads).

The difference is due to the vocabulary size:
`bert-base-uncased` uses a vocabulary of **30k** entries while `bert-base-multilingual-cased` uses a vocabulary of **119k** entries.

To compute the number of parameters:
``` python
from transformers import AutoModelForMaskedLM

bert_base = AutoModelForMaskedLM.from_pretrained('bert-base-uncased')
print(bert_base.num_parameters())

bert_multiling = AutoModelForMaskedLM.from_pretrained('bert-base-multilingual-cased')
print(bert_multiling.num_parameters())
```


Update README.md of my model

model card

Related to pretrained model cards


Correct the number of parameters of BERT based models.

Sometimes the difference is important. For instance:
`bert-base-uncased` has **110M parameters** but `bert-base-multilingual-cased` has more than **178M parameters**, even if both models share the same architecture (12-layer, 768-hidden, 12-heads).

The difference is due to the vocabulary size:
`bert-base-uncased` uses a vocabulary of 30k entries while `bert-base-multilingual-cased` uses a vocabulary of 119k entries.


Draft Etalab QA model

Related to pretrained model cards

Card for the model https://huggingface.co/etalab-ia/camembert-base-squadFR-fquad-piaf


MarianMT - How to find out the actual names of the languages? - Only language-codes are available

# Question

## Information

I want to use the [MarianMT](https://huggingface.co/transformers/model_doc/marian.html) library for creating a translation application. There are lots of languages available with their language-codes mentioned such as "en" and "es", but no where can I find the actual names of the languages for each code. I know "en" is for English and "fr" for French, but what "bg" and "lu" and so many others that are available. [This resource](https://huggingface.co/Helsinki-NLP) doesn't mention the actual names as well.

Is there any place where I can find the corresponding names for these codes. I tried using some online source, but there seems to be a miss-match and I can't be sure if I got the match correct.


Rework TF trainer

This PR brings several improvements to the TensorFlow Trainer:

-  make it more intuitive with less multiple small functions.
- should also make the trainer compliant with TPU.
- Fix the TensorFlow version to at least 2.2 for the trainer
- increase speed of the dataset preprocessing

@Colanim Can you please test this PR with TPU to see if it looks ok.


Model card for Vamsi/T5_Paraphrase_Paws

Related to pretrained model cards

Creating a model card for my uploaded model on the transformers hub.


don't complain about missing W&B when WANDB_DISABLED=true

I get a lot of crashes with W&B in transformers examples: https://github.com/huggingface/transformers/pull/5835 so I have to use `WANDB_DISABLED=true` - this PR removes a complaint that shouldn't be there when this env var is used.


add a summary report flag for run_examples on CI

Currently, it's hard to derive which example tests were run on CI, and which weren't. Adding `-rA` flag to `pytest`, will now include a summary like:

```
==================================================================== short test summary info =====================================================================
PASSED examples/test_examples.py::ExamplesTests::test_generation
PASSED examples/test_examples.py::ExamplesTests::test_run_glue
PASSED examples/test_examples.py::ExamplesTests::test_run_language_modeling
PASSED examples/test_examples.py::ExamplesTests::test_run_squad
FAILED examples/test_examples.py::ExamplesTests::test_run_pl_glue - AttributeError: 'Namespace' object has no attribute 'gpus'
============================================================ 1 failed, 4 passed, 8 warnings in 42.96s ============================================================
```
which makes it easier to validate whether some example is being covered by CI or not.

The PR came about following on the discussion at https://github.com/huggingface/transformers/pull/6027#issuecomment-663894148


add pl_glue example test

Currently PL glue example isn't being tested by CI. This PR fixes that.
https://github.com/huggingface/transformers/pull/6027

the added test currently fails (goodness) and fixes are being PR'ed in
1. https://github.com/huggingface/transformers/pull/6027
2. https://github.com/huggingface/transformers/pull/6028

At the moment this is a basic 'can-run' test - I will add an actual validation once the `pl_glue_run.py` script is made to work.


Is there an easy way to access the multiple choice head of the RobertaForMultipleChoice?

Hello,

This question is specifically for the ```RobertaForMultipleChoice``` model.
I know how to extract the hidden state vectors from each layer of the ```RobertaForMultipleChoice``` model, but is there any way that I can directly feed the hidden state vectors of layer `j` as an input to the multiple-choice head of the model?

In other words, I would like to know if there is any way that I can control the input of the multiple-choice head of the ```RobertaForMultipleChoice``` model (or if there is any easy way that I can access the multiple-choice head directly).

Thank you,


Create README.md

Related to pretrained model cards

Adding model card - readme


Error with batch_encode_plus

# 🐛 Bug

## Information
When using the tokenizer method `batch_encode_plus` on the new `transformers` version, I run into an error

Model I am using (Bert, XLNet ...): BertTokenizer

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Here is a sample script to reproduce the error:
```python
import torch
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
text = [['sample text 1'], ['sample text 2']]
encode = tokenizer.batch_encode_plus(text)
```

The stack trace is:
```
Traceback (most recent call last):
  File "bug.py", line 8, in <module>
    encode = tokenizer.batch_encode_plus(text)
  File "/home/nithinh/debug/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 1832, in batch_encode_plus
    **kwargs,
  File "/home/nithinh/debug/lib/python3.6/site-packages/transformers/tokenization_utils.py", line 534, in _batch_encode_plus
    ids, pair_ids = ids_or_pair_ids
ValueError: not enough values to unpack (expected 2, got 1)
```
## Expected behavior

Successfully tokenize without errors. The script runs fine with `transformers` v2.11.0.

## Environment info

- `transformers` version: 3.0.2
- Platform: Linux-4.19.0-9-amd64-x86_64-with-debian-10.4
- Python version: 3.6.3
- PyTorch version (GPU?): 1.5.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


create model-card for lordtt13/emo-mobilebert

Related to pretrained model cards


tensorflow bert model can‘t  return all hidden_states

```
import tensorflow as tf
import tensorflow_datasets
from transformers import *
from tensorflow.keras import layers

configuration = BertConfig.from_pretrained('bert-base-cased', output_hidden_states=True)
# Load dataset, tokenizer, model from pretrained model/vocabulary
tokenizer = BertTokenizer.from_pretrained('bert-base-cased', config=configuration)
# BERT encoder
encoder = TFBertModel.from_pretrained('bert-base-cased', config=configuration)
# Model
input_ids = layers.Input(shape=(100,), dtype=tf.int32)
token_type_ids = layers.Input(shape=(100,), dtype=tf.int32)
attention_mask = layers.Input(shape=(100,), dtype=tf.int32)
outputs = encoder(
    input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask
)
print(outputs)

_, _, hidden_states = outputs[0], outputs[1], outputs[2]
```

output:

```
If your task is similar to the task the model of the ckeckpoint was trained on, you can already use TFBertModel for predic
(<tf.Tensor 'tf_bert_model_2/Identity:0' shape=(None, 100, 768) dtype=float32>, <tf.Tensor 'tf_bert_model_2/Identity_1:0' shape=(None, 768) dtype=float32>)

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-7-63944e137dcd> in <module>()
     17 )
     18 print(outputs)
---> 19 _, _, hidden_states = outputs[0], outputs[1], outputs[2]

IndexError: tuple index out of range
```

You can check it:
[colab code](https://colab.research.google.com/drive/1zZo2SwBuGoCHsbvL3_fVR5rygWiuR16-?usp=sharing)


examples/text-classification/run_pl.sh multiple problems

Fixing this sequence of errors - each fix required for the next error

running:
```
cd examples/text-classification
./run_pl.sh
```

error 1:
```
Traceback (most recent call last):
  File "run_pl_glue.py", line 183, in <module>
    trainer = generic_train(model, args)
  File "/mnt/nvme1/code/huggingface/transformers-issue-1/examples/lightning_base.py", line 289, in generic_train
    if args.gpus > 1:
AttributeError: 'Namespace' object has no attribute 'gpus'
```
solution: added `--n_gpus` arg

error 2:
```
Traceback (most recent call last):
  File "run_pl_glue.py", line 183, in <module>
    trainer = generic_train(model, args)
  File "/mnt/nvme1/code/huggingface/transformers-issue-1/examples/lightning_base.py", line 300, in generic_train
    **train_params,
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 853, in from_argparse_args
    return cls(**trainer_kwargs)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 468, in __init__
    self.tpu_cores = _parse_tpu_cores(tpu_cores)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_parts.py", line 526, in _parse_tpu_cores
    raise MisconfigurationException("`tpu_cores` can only be 1, 8 or [<1-8>]")
pytorch_lightning.utilities.exceptions.MisconfigurationException: `tpu_cores` can only be 1, 8 or [<1-8>]
```
solution: removed `default=0` for `tpu_cores`

error 3:
```
Traceback (most recent call last):
  File "run_pl_glue.py", line 183, in <module>
    trainer = generic_train(model, args)
  File "/mnt/nvme1/code/huggingface/transformers-issue-1/examples/lightning_base.py", line 304, in generic_train
    trainer.fit(model)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1038, in fit
    model.setup('fit')
  File "/mnt/nvme1/code/huggingface/transformers-issue-1/examples/lightning_base.py", line 125, in setup
    dataloader = self.get_dataloader("train", train_batch_size)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/torch/nn/modules/module.py", line 594, in __getattr__
    type(self).__name__, name))
AttributeError: 'GLUETransformer' object has no attribute 'get_dataloader'
```
solution: added a wrapper - but it's incomplete - what to do with the `shuffle` arg?

error 4:
```
Traceback (most recent call last):
  File "run_pl_glue.py", line 187, in <module>
    trainer = generic_train(model, args)
  File "/mnt/nvme1/code/huggingface/transformers-issue-1/examples/lightning_base.py", line 306, in generic_train
    trainer.fit(model)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1044, in fit
    results = self.run_pretrain_routine(model)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1213, in run_pretrain_routine
    self.train()
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 370, in train
    self.run_training_epoch()
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 452, in run_training_epoch
    batch_output = self.run_training_batch(batch, batch_idx)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 632, in run_training_batch
    self.hiddens
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 776, in optimizer_closure
    hiddens)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 956, in training_forward
    output = self.model.training_step(*args)
  File "run_pl_glue.py", line 44, in training_step
    tensorboard_logs = {"loss": loss, "rate": self.lr_scheduler.get_last_lr()[-1]}
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/torch/nn/modules/module.py", line 594, in __getattr__
    type(self).__name__, name))
AttributeError: 'GLUETransformer' object has no attribute 'lr_scheduler'
```
solution: I'm not sure how it used to work, but there is no `self.lr_scheduler` in pytorch-lightning (PL), I found one here: `self.trainer.lr_schedulers[0]["scheduler"]` and set this attribute. I have no idea whether this always works. Someone who wrote this script would probably know better where the missing attribute has gone. It's set inside `def fit`/CPU but inside the `trainer` object and not `nn.Module`.

Further notes:

`run_pl.sh` invokes PL in CPU mode, despite available GPU. I haven't tested this on gpu yet - I just saw during debug that PL [inits optimizers](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/trainer/trainer.py#L1096) just before it runs `run_pretrain_routine`, so I didn't find an easy PL predefined method where one could preset `self.lr_scheduler`.

Perhaps PL API has changed and caused this issue?

error 5:
```
Traceback (most recent call last):
  File "run_pl_glue.py", line 218, in <module>
    trainer = generic_train(model, args)
  File "/mnt/nvme1/code/huggingface/transformers-issue-1/examples/lightning_base.py", line 305, in generic_train
    trainer.fit(model)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1044, in fit
    results = self.run_pretrain_routine(model)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1213, in run_pretrain_routine
    self.train()
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 370, in train
    self.run_training_epoch()
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 452, in run_training_epoch
    batch_output = self.run_training_batch(batch, batch_idx)
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 671, in run_training_batch
    self.on_batch_end()
  File "/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/pytorch_lightning/trainer/callback_hook.py", line 82, in on_batch_end
    callback.on_batch_end(self, self.get_model())
  File "/mnt/nvme1/code/huggingface/transformers-issue-1/examples/lightning_base.py", line 198, in on_batch_end
    lrs = {f"lr_group_{i}": lr for i, lr in enumerate(self.lr_scheduler.get_lr())}
AttributeError: 'LoggingCallback' object has no attribute 'lr_scheduler'
```
solution: see notes for error 4.

with these fixes the code at least starts training, I didn't test further, since clearly there is a better way. Only the fixes for the first 2 errors are obviously correct to merge.

All the fixes are in one PR, as one can't move to the next error, before fixing the previous ones.


[Fix] text-classification PL example

Examples

Which is related to examples in general

The text-classification example needed to have several edits to get it to working. The main one was that the hparams are loaded as a dict instead of a Namespace object from the checkpoint so this needed to be fixed with recasting the hparams to a Namespace object.

Though this is not the ideal solution, it works for now.

I also have some other fixes such as gpus argument which needed to be added to the generic arguments list in the `lightning_base.py` and removing the default value for `n_tpu_cores`. And the lr_scheduler was not accessed by the logging callback correctly. These have been all fixed and the example works correctly.


Fix tokenizer saving and loading error

This PR is to address #6025


Failed to save tokenizer with AddedToken in additional_special_tokens

# 🐛 Bug

I tried to add new special tokens to some tokenizer. I wanted them to be `AddedToken` in order to handle specific whitespace striping. But I got the following error when saving the tokenizer.

## To reproduce

Steps to reproduce the behavior:

```python
>>> from transformers import BertTokenizer
>>> from tokenizers import AddedToken
>>> tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
>>> new_token = AddedToken('new_token', lstrip=True)
>>> tokenizer.add_special_tokens({'additional_special_tokens': [new_token]})
1
>>> tokenizer.save_pretrained('.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/junyuan/anaconda3/envs/deep/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1371, in save_pretrained
    f.write(json.dumps(write_dict, ensure_ascii=False))
  File "/home/junyuan/anaconda3/envs/deep/lib/python3.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/home/junyuan/anaconda3/envs/deep/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/home/junyuan/anaconda3/envs/deep/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/home/junyuan/anaconda3/envs/deep/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type AddedToken is not JSON serializable
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->


Feed forward chunking

Official PR for #5928


Remove unused file

This seems to be the old script to deploy the docs, new one is `.circleci/deploy.sh`.


Fix the return documentation rendering for all model outputs

All PyTorch model outputs are documented from their output types. A problem is that just using the docstrings of the output class doesn't render properly on sphinx (this was also the case before the new model outputs were introduced).

This PR adds a function that converts the args part of the docstrings of the output class to render properly on our doc. You can see the transformation by looking [here](https://huggingface.co/transformers/master/model_doc/bert.html#transformers.BertModel.forward) for the docs before this PR and [here](https://64423-155220641-gh.circle-artifacts.com/0/docs/_build/html/model_doc/bert.html#transformers.BertModel.forward) for after it's merged (it's one example, but it will give the same result for all models). Scroll a bit to get to the return part of the doc.


[CI] Don't test apex


Create README.md

model card

Related to pretrained model cards

README.md for my model

https://huggingface.co/rdenadai/BR_BERTo


Update the new model template

The model template files were old and did not use any of the recent improvements (output attentions, output hidden states, docstrings refactored...)

This PR fixes them and adds a few more instructions.

Tagging @LysandreJik, @patrickvonplaten, @thomwolf and @julien-c : We have to make sure that whenever we make a substantial change that applies to all model, this template is also updated.


seq2seq/finetune.py can take config train_params through command line

- this might belong in lightning_base
- check how/if trainer does it.

Desired API:
```bash
python finetune.py --encoder_layerdrop 0.1 --decoder_layerdrop 0.1 --dropout 0.1 --attention_dropout 0.1
```


convert_bart script should support mbart through command line.

Desired command:

```convert_mbart_checkpoint.py --checkpoint_dir --pytorch_dump_path```

This should probably also convert the tokenizer.


Create model card for RuPERTa-base

Related to pretrained model cards


Tf trainer cleanup

This follows up from #5982 and does the same thing for `TFTrainer`.
Basically no code is changed, just moved around and the customization points are made public and explicit to the user. Starting to make the docs better but I will do it properly (for both Trainer and TFTrainer) after this PR is merged.


Fix question template


Cannot import DPRReader

# 🐛 Bug

You have DPRReader included in your v3.0.2 documentation, but it's in the Master branch, it's not included in v3.0.2 tag; therefore it crashes and cannot import it. I think it would be convenient to either eliminate that piece of documentation until the models are added in the tag, or update the transformers version with the Master branch.


When i train a GPT-2 which uses BPE, the computed perplexity is per sub-word right?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


[Benchmark]

# 🖥 Benchmarking `transformers`

## Benchmark

Which part of `transformers` did you benchmark?

## Set-up

What did you run your benchmarks on? Please include details, such as: CPU, GPU? If using multiple GPUs, which parallelization did you use?

## Results

Put your results here!


install sentence-transformers on Linux by python error

# ❓ Questions & Help

When I install sentence-transformers on Linux by python
![error](https://user-images.githubusercontent.com/15052580/88359057-da0efc80-cda3-11ea-98fb-01deddb09acd.png)
, I got an error message:

ERROR: Could not find a version that satisfies the requirement transformers>=3.0.2 (from sentence-transformers) (fr
om versions: 0.1, 2.0.0, 2.1.0, 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.4.0, 2.4.1, 2.5.0, 2.5.1)
ERROR: No matching distribution found for transformers>=3.0.2 (from sentence-transformers)

system: Linux 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1+deb9u1 (2020-06-07) x86_64 on GCP VM instance.
Is there any suggestion?


[model_cards] roberta-base-finetuned-yelp-polarity

Related to pretrained model cards

Model card for [roberta-base-finetuned-yelp-polarity](https://huggingface.co/VictorSanh/roberta-base-finetuned-yelp-polarity).


fix: model card readme clutter

Related to pretrained model cards

this removes the clutter line in the readme.md of model card `csarron/roberta-base-squad-v1`. It also fixes the result table.


Fine tune T5 for paraphrase generation

# ❓ Questions & Help
Dear all,
I am new to NLP and has a lot of questions. Sorry to ask this long list here. I tried asking on huggingface's forum but as a new user, I can only put 2 lines there.

My goal is to fine-tuned t5-large for paraphrase generation. I found [this code](https://github.com/ramsrigouthamg/Paraphrase-any-question-with-T5-Text-To-Text-Transfer-Transformer-/blob/master/t5-pretrained-question-paraphraser.ipynb) which is based on [this code](https://github.com/patil-suraj/exploring-T5). So I just modified to further fine tune on my dataset. My questions( I also asked some of them on [the github code mentioned above](https://github.com/patil-suraj/exploring-T5/issues/3) but I feel these question may be better address here):

1. I saw for 2 epoches and the paraphrases generated looks good. When I trained for 11 epochs and the model seems overfitted (the paraphrases generated is similar to the original sentence). Do you have any recommendation for further improving the performance beside decrease the number of epochs?

2. For paraphrase generation using T5 as a text-to-text task, I don't know how to utilize the negative examples (pairs that are not paraphrases) directly here. Any recommendation?

3. One idea I have to include the negative examples is : I plan to first further fine tune T5-large's paraphrase identification with my data set (with positive and negative examples) and then used this fine tuned version to further fine tune on paraphrase generation. My assumption is the information learned in paraphrase identification task will help improve paraphrase generation. Is this correct?

4. I am also a little confused about the prefix.
On huggingface'T5 works well on a variety of tasks out-of-the-box by prepending a different prefix to the input corresponding to each task, e.g.: for translation: translate English to German: …, summarize: …. For more information about which prefix to use, it is easiest to look into Appendix D of the paper '.
Thus, I think that prefix notifies the T5 which task he should be performing. [The answer of this thread](https://github.com/huggingface/transformers/issues/4092 ) agrees with my understanding. However, in the first two examples [here](https://github.com/patil-suraj/exploring-T5), the code seems only add </s> at the end of the sentence   "< / s >" but no 'prefix'. Could you tell me why? Do that mean these fine-tuned model will not do pretrain tasks of T5 but only their specific trained task so we don't need a prefix?

5.Also, MRPC and QQP are both paraphrase identification. If I want to fine tune, should I use my data set to fine with both of their prefix, or fine tune with one of their prefix or create my own prefix?

6. The loss function of the code is cross entropy, which is not the best for this task.  I am thinking if I can use the paraphrase identification result (like the probability of being paraphrase) as the target function. Is this OK? I feel it maybe suuuupppeer slow. And I am not really sure how to implement it.

7. I have 3 paraphrase datasets (let's call them A,B,C) from different sources. Previously, I first train the model on A for 2 epoch, and then load this model as pretrained model to further train B for 2 epoches, and then C.
I then Combine the A,B,C into one dataset and directly trained for 2 epoches. The resulting two models has different results and the second one is worse. I have the same random seeds for them. Any idea?

8. I set  early_stop_callback= True and set max_epochs=32, then it stops at epoch 11. But if I set max_epochs = 6, it stops at epoch 3. I don't understand, as I thought it will stop at epoch 6. I have the same random seed.

9. Another strange thing during training, I saw this on the screen:
Epoch 10: 100%............(time, loss et al)...
INFO:_main_:avg_train_loss = tensor(..)
INFO:_main_:epoch = 8
........
Why the epoch number is not the same?!
10. what is the correct way to evaluate on testing set? I saw several different examples.

[In this example, ](https://www.kaggle.com/enzoamp/t5-for-q-a-training-tutorial-pytorch)
 t5 = T5ForConditionalGeneration.from_pretrained('output/')
    input_ids = tokenizer.encode(text, return_tensors="pt", add_special_tokens=True)  # Batch size 1
    **t5.eval()**
    generated_ids = t5.generate(
        input_ids=input_ids,
        num_beams=1,
        max_length=80,
        #repetition_penalty=2.5
    ).squeeze()
    predicted_span = tokenizer.decode(generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True)
    return predicted_span

[This code has two examples:](https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb)
First directly:
outs = **model.model.generate**(input_ids=batch['source_ids'].cuda(),
                              attention_mask=batch['source_mask'].cuda(),
                              max_length=2)
He also has :
loader = DataLoader(dataset, batch_size=32, num_workers=4)
**model.model.eval()**
outputs = []
targets = []
for batch in tqdm(loader):
  outs = model.model.generate(input_ids=batch['source_ids'].cuda(),
                              attention_mask=batch['source_mask'].cuda(),
                              max_length=2)

 Is eval() necessary? From[ Huggingface's doc,](https://huggingface.co/transformers/model_doc/auto.html) it seems not necessary when the model is 'from pretrain':
"The model is set in evaluation mode by default using model.eval() (Dropout modules are deactivated) To train the model, you should first set it back in training mode with model.train()"
On the other side, all the examples above, none of them uses 'model.train()' to set the mode. They directly train the model. I am confused.

 Is model.model necessary?

Thanks!!


Model cards: add roberta-base-squad-v1 and bert-base-uncased-squad-v1

Related to pretrained model cards