convert_graph_to_onnx not working as expected.

# ❓ Questions & Help

not sure if there is a bug.

when running
```python
from transformers.convert_graph_to_onnx import convert
convert(framework="tf", model = my_fine_tuned_bert_model, output="onnx-fine-tuned/model.onnx", opset=11, tokenizer=tokenizer)
```
I got the following log/output
```
ONNX opset version set to: 11
Loading pipeline (model: <__main__.TFBertForMultiClassification object at 0x7f2c37ba9b50>, tokenizer: <transformers.tokenization_bert.BertTokenizerFast object at 0x7f2c37ba9ad0>)
Creating folder onnx-fine-tuned
/!\ Please note TensorFlow doesn't support exporting model > 2Gb /!\
Using framework TensorFlow: 2.1.0, keras2onnx: 1.7.0
Found input input_ids with shape: {0: 'batch', 1: 'sequence'}
Found input token_type_ids with shape: {0: 'batch', 1: 'sequence'}
Found input attention_mask with shape: {0: 'batch', 1: 'sequence'}
Found output output_0 with shape: {0: 'batch'}
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertForMultiClassification.call of <__main__.TFBertForMultiClassification object at 0x7f2c37ba9b50>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: module 'gast' has no attribute 'Num'
WARNING: AutoGraph could not transform <bound method TFBertForMultiClassification.call of <__main__.TFBertForMultiClassification object at 0x7f2c37ba9b50>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: module 'gast' has no attribute 'Num'
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertMainLayer.call of <transformers.modeling_tf_bert.TFBertMainLayer object at 0x7f2c3dc34910>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: module 'gast' has no attribute 'Num'
WARNING: AutoGraph could not transform <bound method TFBertMainLayer.call of <transformers.modeling_tf_bert.TFBertMainLayer object at 0x7f2c3dc34910>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: module 'gast' has no attribute 'Num'
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c371ffe90>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c371ffe90>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c3769bb50>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c3769bb50>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c3769bed0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c3769bed0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37682e10>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37682e10>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c8ca90>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c8ca90>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37c9b050>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37c9b050>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37ca6c90>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37ca6c90>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37cae910>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37cae910>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37caee90>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37caee90>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c3783eb10>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c3783eb10>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37834790>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37834790>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37834d10>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37834d10>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c3781e990>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c3781e990>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37814610>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37814610>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37814b90>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37814b90>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37778850>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37778850>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c377704d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c377704d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37770a50>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37770a50>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c3775b6d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c3775b6d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37751350>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37751350>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c377518d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c377518d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37c42550>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37c42550>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c4b1d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c4b1d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37c4b750>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37c4b750>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37c5f3d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37c5f3d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c5ffd0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c5ffd0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37c685d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37c685d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37c7c210>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37c7c210>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c7ce50>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c7ce50>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37c07410>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37c07410>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37c1a0d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37c1a0d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c1acd0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c1acd0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37c24290>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37c24290>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37c2ded0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertSelfOutput.call of <transformers.modeling_tf_bert.TFBertSelfOutput object at 0x7f2c37c2ded0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c35b50>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertIntermediate.call of <transformers.modeling_tf_bert.TFBertIntermediate object at 0x7f2c37c35b50>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37bc3110>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertOutput.call of <transformers.modeling_tf_bert.TFBertOutput object at 0x7f2c37bc3110>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:AutoGraph could not transform <bound method TFBertPooler.call of <transformers.modeling_tf_bert.TFBertPooler object at 0x7f2c37bc3cd0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
WARNING: AutoGraph could not transform <bound method TFBertPooler.call of <transformers.modeling_tf_bert.TFBertPooler object at 0x7f2c37bc3cd0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Bad argument number for Name: 3, expecting 4
tf executing eager_mode: True
tf.keras model eager_mode: False
The ONNX operator number change on the optimization: 2579 -> 1674
```

should I ignore the warning?

The shape of the exported onnx model is
```
graph_name: tf_bert_for_multi_classification
domain: onnxmltools
description:
input 0: "attention_mask" ["N", 7] Int32
input 1: "input_ids" ["N", 7] Int32
input 2: "token_type_ids" ["N", 7] Int32
output 0: "output_1" ["N", 4404, 1] Float
```

I don't think that's correct. where are "N" and 7 from?
when I try to run the model on input
```
{'input_ids': array([ 101,  146, 1169, 1631, 1103, 3974,  117, 1169, 1128,  136,  102]),
 'token_type_ids': array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]),
 'attention_mask': array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])}
```
I got error
```
>>> results = session.run(None, inputs_onnx)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/xws61xnjc03fjiwfh7ci5cwgg1chmp3l-python3.7-onnxruntime-1.4.0/lib/python3.7/site-packages/onnxruntime/capi/session.py", line 110, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Unexpected input data type. Actual: (N11onnxruntime17PrimitiveDataTypeIlEE) , expected: (N11onnxruntime17PrimitiveDataTypeIiEE)
```


Error: 'GPT2Model' object has no attribute '_step' when converting tf-based checkpoint into pytorch

I'm trying to convert a tensorflow-based GPT-2 checkpoint into pytorch using `convert_gpt2_checkpoint_to_pytorch`, and get errors like:
```
INFO:transformers.modeling_gpt2:Converting TensorFlow checkpoint from /content/model.ckpt-220000
INFO:transformers.modeling_gpt2:Loading TF weight global_step with shape []
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/LayerNorm_embed_norm/beta with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/LayerNorm_embed_norm/beta/adafactor_v with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/LayerNorm_embed_norm/gamma with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/LayerNorm_embed_norm/gamma/adafactor_v with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/pos_embed with shape [1024, 1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/pos_embed/adafactor_vc with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/pos_embed/adafactor_vr with shape [1024]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/word_embed with shape [8021, 1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/word_embed/adafactor_vc with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/word_embed/adafactor_vr with shape [8021]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer00/LayerNorm_mlp_ln0/beta with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer00/LayerNorm_mlp_ln0/beta/adafactor_v with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer00/LayerNorm_mlp_ln0/gamma with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer00/LayerNorm_mlp_ln0/gamma/adafactor_v with shape [1536]
...
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer47/value_layer/bias/adafactor_v with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer47/value_layer/kernel with shape [1536, 1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer47/value_layer/kernel/adafactor_vc with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer47/value_layer/kernel/adafactor_vr with shape [1536]
---------------------------------------------------------------------------
ModuleAttributeError                      Traceback (most recent call last)
<ipython-input-38-45b704eacf86> in <module>()
      1 from transformers.convert_gpt2_original_tf_checkpoint_to_pytorch import convert_gpt2_checkpoint_to_pytorch
----> 2 convert_gpt2_checkpoint_to_pytorch('./model.ckpt-220000', '', 'pytorch')

2 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __getattr__(self, name)
    770                 return modules[name]
    771         raise ModuleAttributeError("'{}' object has no attribute '{}'".format(
--> 772             type(self).__name__, name))
    773
    774     def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:

ModuleAttributeError: 'GPT2Model' object has no attribute '_step'
```
It seems that the program cannot convert `global_step` layer into pytorch. Is there any solution to this?


add custom datasets tutorial

A tutorial showing examples for working with custom datasets on several tasks. Goals:

1. Keep it general. The point is to show people how to use their own datasets, so don't use any processors or utilities that are dataset-specific.
2. Show several tasks with different data formats. I include sequence classification with IMDb, token classification with W-NUT NER, and question answering with squad 2.0. Also link to how to train a language model blog post.
3. Prepare the data in a way that works with Trainer, TFTrainer, native PyTorch, and native TensorFlow with keras's `fit` method.


Longformer convert error

When i install transformers from source and convert bert to "long vesion", [failed.](https://colab.research.google.com/github/allenai/longformer/blob/master/scripts/convert_model_to_long.ipynb)


[BartTokenizerFast] add BartTokenizerFast in AutoTokenizer

This PR adds BartTokenizerFast in AutoTokenizer.
@sshleifer


[LongformerTokenizerFast] add LongformerTokenizerFast in AutoTokenizer

This PR adds LongformerTokenizerFast in AutoTokenizer. Fixes #6459

@patrickvonplaten


minor typo fix in modeling_utils


Sort unique_no_split_tokens to make it deterministic

The `unique_no_split_tokens` attribute of tokenizers is not deterministic, and it makes the hashing in the `nlp` lib return different hashes for the same tokenizer over different sessions.

To fix that I changed its type to a `set` instead of a `list`.

Fix #6460


Hashing a tokenizer using the 🤗 nlp lib is not deterministic

In the `nlp` library it is common to use a tokenizer on a dataset.
The library takes care of caching the results, so that if you run the tokenization twice, it will reuse the previous results.
To make the caching work, we compute a hash of the tokenizer.

However the `unique_no_split_tokens` attribute of tokenizers is not deterministic, and it makes the hashing return different hashes for the same tokenizer over different sessions.

`unique_no_split_tokens` can be a list like `['[CLS]', '[MASK]', '[PAD]', '[SEP]', '[UNK]']` for example. But it happens that re-loading a tokenizer in another session shuffles the tokens in the list.

For example this code doesn't always return the same output over different sessions:

```python
from transformers import AutoTokenizer

model_name = "distilbert-base-uncased-finetuned-sst-2-english"
tokenizer = AutoTokenizer.from_pretrained(model_name)

print(tokenizer.unique_no_split_tokens)
```

Reproduce on google colab: https://colab.research.google.com/drive/1nyskaLavcTCkXibZBlYX71bkG476uSzz?usp=sharing


Autotokenizer not returning instance of LongformerTokenizerFast

## Environment info

- `transformers` version: 3.0.2
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: Google Colab

## Information

Model I am using :  Longformer
Path: **'allenai/longformer-base-4096'** and **'allenai/longformer-large-4096'**

The problem arises when trying to load 'Fast' version for Longformer using Autotokenizer, the returned tokenizer instance is an object of LongformerTokenizer and not LongformerTokenizerFast.

![Annotation 2020-08-13 160156](https://user-images.githubusercontent.com/20542313/90124665-68ace300-dd7e-11ea-8cc7-fdd80f070bec.png)

I require the offset mappings for a sub task of extracting word embeddings.

## To reproduce
Just as in the screenshot i am adding the code below to instantiate the tokenizer object:
```
longformer_tokenizer = AutoTokenizer.from_pretrained(
    pretrained_model_name_or_path = 'allenai/longformer-base-4096', use_fast=True)
print(longformer_tokenizer.is_fast)
print(longformer_tokenizer)
```
And since its not an instance of  transformers.LongformerTokenizerFast, I cannot `return_offsets_mapping=True`
As in the below code throws `NotImplementedError`

```
longformer_encoded_dict = longformer_tokenizer.encode_plus(text=sequence_3,
                                                    add_special_tokens = True,
                                                    max_length = 75,
                                                    truncation = True,
                                                    pad_to_max_length = False,
                                                    return_token_type_ids = False,
                                                    return_attention_mask = True,
                                                    return_overflowing_tokens = False,
                                                    return_special_tokens_mask = False,
                                                    return_offsets_mapping=True)
```
**Error**
`
NotImplementedError: return_offsets_mapping is not available when using Python tokenizers.To use this feature, change your tokenizer to one deriving from transformers.PreTrainedTokenizerFast.`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

@mfuntowicz
@patrickvonplaten


Unknown task zero-shot-classification

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Ubuntu 18
- Python version: 3.7

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 tensorflow: @jplu
documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. I downloaded transformer version 3.0.2
2. From transformer, I imported pipeline
3. And from the pipeline, I was trying to load this task `zero-shot-classification` and then I got the error.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

```
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-12-1f0825594ce1> in <module>
----> 1 classifier = pipeline("zero-shot-classification")

~/anaconda3/lib/python3.7/site-packages/transformers/pipelines.py in pipeline(task, model, config, tokenizer, framework, **kwargs)
   1819     # Retrieve the task
   1820     if task not in SUPPORTED_TASKS:
-> 1821         raise KeyError("Unknown task {}, available tasks are {}".format(task, list(SUPPORTED_TASKS.keys())))
   1822
   1823     framework = framework or get_framework(model)

KeyError: "Unknown task zero-shot-classification, available tasks are ['feature-extraction', 'sentiment-analysis', 'ner', 'question-answering', 'fill-mask', 'summarization', 'translation_en_to_fr', 'translation_en_to_de', 'translation_en_to_ro', 'text-generation']"
```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Add POS tagging and Phrase chunking token classification examples

This PR adds POS tagging and Phrase chunking examples to token classification examples. The current example (NER) is minimally adjusted to allow users to experiment with their token classification model training easily. Although experimenting with token classifications other than NER token classification is already possible for skilled developers, this PR lowers the barrier to entry even further and demonstrates HF extensibility.

The adjustments made consist of:

- extracting [TokenClassificationTask](https://github.com/vblagoje/transformers/blob/6caa8c1946fb9e3fb76fad081833805b25b182df/examples/token-classification/utils_ner.py#L69) superclass
- implementing the specific task particulars (reading of InputExample etc.) in task [subclasses](https://github.com/vblagoje/transformers/blob/6caa8c1946fb9e3fb76fad081833805b25b182df/examples/token-classification/tasks.py)
- "dynamic loading" of a task [subclass](https://github.com/vblagoje/transformers/blob/6caa8c1946fb9e3fb76fad081833805b25b182df/examples/token-classification/run_ner.py#L118) depending on the token classification task trained

I also noticed that:

- [NER dataset](https://github.com/vblagoje/transformers/blob/6caa8c1946fb9e3fb76fad081833805b25b182df/examples/token-classification/run.sh#L1) used is unavailable and should be replaced. I didn't replace it in this PR
- PL training needs to be slightly retrofitted to adjust for the latest PL's BaseTransformer master changes. I made the change to make sure my changes work for these new examples

If you think adding one rather than two token task classification example is enough (say POS tagging) let me know - I'll remove the other. Also, please let me know if any additional adjustments are needed.


Open-Retrieval Question Answering (ORQA)

# 🌟 New model addition

Open-Retrieval Question Answering system (ORQA) was introduced in the paper https://arxiv.org/abs/1906.00300. This approach is very useful for those who work on Open Domain Question Answering.

<!-- Important information -->

## Open source status

* [x] the model implementation is available: All the implementation code has been released in https://github.com/google-research/language/tree/master/language/orqa
* [x] the model weights are available: `gs://orqa-data/`
* [x] who are the authors: @kentonl et al.


MASS : A generalization of BERT and GPT

# 🌟 New model addition

## Model description

MASS is a novel pre-training method for sequence to sequence based language generation tasks. It randomly masks a sentence fragment in the encoder, and then predicts it in the decoder. In this way, MASS can jointly train the encoder and decoder to develop the capability of representation extraction and language modeling. This pre-training is very helpful when the encoder and decoder are shared between multiple languages.

## Open source status

- [x] the model implementation is available :  The model is implemented upon fair-seq [here.](https://github.com/microsoft/MASS)
- [x] the model weights are available: Pre-trained model on various language pairs, for unsupervised translation, supervised translation and abstractive summarization are provided on the GitHub repo itself.
- [x] Kaitao Song, Xu Tan,  Tao Qin,  Jianfeng Lu,  Tie-Yan Liu  are the authors: ( @StillKeepTry , @tobyoup , @xutaatmicrosoftdotcom )

This is my first time contributing to this repository, so forgive me for any mistake . Please let know whether I should do it or not. Also, if anyone wants to come along and help, please let me know that too ! 😀


Memory Issue while following LM tutorial

(Didn't get answer)
https://stackoverflow.com/questions/63387831/memory-issue-while-following-lm-tutorial

SPECS:
torch==1.5.0
transformers==3.0.2
OS: Windows 10
CUDA: 10.1
GPU: RTX 2060 6G VRAM (x2)
RAM: 32GB
tutorial: https://huggingface.co/blog/how-to-train

Hello I am trying to train my own language model and I have had some memory issues. I have tried to run some of this code in Pycharm on my computer and then trying to replicate in my Collab Pro Notebook.

## First, my code
```
from transformers import RobertaConfig, RobertaTokenizerFast, RobertaForMaskedLM, LineByLineTextDataset
from transformers import DataCollatorForLanguageModeling, Trainer, TrainingArguments

config = RobertaConfig(vocab_size=60000, max_position_embeddings=514, num_attention_heads=12, num_hidden_layers=6,
                       type_vocab_size=1)

tokenizer = RobertaTokenizerFast.from_pretrained("./MODEL DIRECTORY", max_len=512)

model = RobertaForMaskedLM(config=config)

print("making dataset")

dataset = LineByLineTextDataset(tokenizer=tokenizer, file_path="./total_text.txt", block_size=128)

print("making c")

data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=True, mlm_probability=0.15)

training_args = TrainingArguments(output_dir="./MODEL DIRECTORY", overwrite_output_dir=True, num_train_epochs=1,
                                  per_gpu_train_batch_size=64, save_steps=10000, save_total_limit=2)
print("Building trainer")
trainer = Trainer(model=model, args=training_args, data_collator=data_collator, train_dataset=dataset,
                  prediction_loss_only=True)
trainer.train()

trainer.save_model("./MODEL DIRECTORY")

```
`"./total_text.txt"` being a 1.7GB text file.

## PyCharm Attempt

This code on pycharm builds the dataset and then would throw an error saying that my preferred gpu was running out of memory, and that Torch was already using 3.7GiB of memory.

I tried:
* import gc doing a gc clear to try to flush what ever was going on my gpu
* Decreasing my batch size for my gpu (training only happened on a batch size of 8 resulting in 200,000+ epochs that all took 1.17 seconds)
* Setting my `os.environ["CUDA_VISIBLE_OBJECTS"] =""` so that torch would have to use my CPU and not my GPU. Still threw same gpu memory error...

So succumbing to the fact that torch, for the time being, was forcing itself to use my gpu, I decided to go to Collab.

## Collab Attempt

Collab has different issues with my code. It does not have the memory to build the dataset, and crashes due to RAM shortages. I purchased a Pro account and then increased the usable RAM to 25GB, still memory shortages.

Cheers!


Clean directory after script testing

#6421 #6433

This PR cleans the directory after each script testing to prevent bugs like this.


getting error while training bert language model. "ValueError: Expected input batch_size (8) to match target batch_size (1024)."

from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
from transformers import BertForSequenceClassification
model = BertForSequenceClassification.from_pretrained('bert-base-uncased')

%%time
from transformers import LineByLineTextDataset,TextDataset

paths = '/content/drive/My Drive/MyFile.txt'
dataset = TextDataset(
    tokenizer=tokenizer,
    file_path=paths,
    block_size=128,
)

from transformers import DataCollatorForLanguageModeling

data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer, mlm=True, mlm_probability=0.15
)

from transformers import Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir="./EsperBERTo",
    overwrite_output_dir=True,
    num_train_epochs=1,
    save_total_limit=2,
)

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=dataset,
    prediction_loss_only=True,
)

%%time
trainer.train()

**'**''''''''''''''''''''''''''''''''''''''Getting error after executing trainer.train() ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''****

ValueError                                Traceback (most recent call last)
<ipython-input-12-0c647bc3a8b8> in <module>()
----> 1 get_ipython().run_cell_magic('time', '', 'trainer.train()')

10 frames
<decorator-gen-60> in time(self, line, cell, local_ns)

<timed eval> in <module>()

/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   2214     if input.size(0) != target.size(0):
   2215         raise ValueError('Expected input batch_size ({}) to match target batch_size ({}).'
-> 2216                          .format(input.size(0), target.size(0)))
   2217     if dim == 2:
   2218         ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

**ValueError: Expected input batch_size (8) to match target batch_size (1024).**

please help me to resolve this issue.


```
ERROR: Could not find a version that satisfies the requirement tokenizers==0.8.1.rc1 (from transformers) (from versions: 0.0.2, 0.0.3, 0.0.4, 0.0.5, 0.0.6, 0.0.7, 0.0.8, 0.0.9, 0.0.10, 0.0.11, 0.0.12, 0.0.13, 0.1.0, 0.1.1, 0.2.0, 0.2.1, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.5.0, 0.5.1, 0.5.2, 0.6.0, 0.7.0, 0.8.0, 0.8.1)
ERROR: No matching distribution found for tokenizers==0.8.1.rc1 (from transformers)
```

The error above occurs when I pip install transformers from an anaconda environment of

![image](https://user-images.githubusercontent.com/8081512/90089485-7b162500-dd5c-11ea-9e0e-16f8044b490e.png)


Error in PyTorch Trainer when used with TPU

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0a0+d6149a7 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 tensorflow: @jplu
documentation: @sgugger
 --> @sgugger

## Information

Model I am using (Bert, XLNet ...): BERT

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: SQUaD
* [ ] my own task or dataset: (give details below)

The following error arises when using the `run_squad_trainer.py` script with TPU:
```python
Epoch:   0% 0/2 [00:00<?, ?it/s]
Iteration: 0it [00:00, ?it/s]Exception in device=TPU:0: 'NoneType' object cannot be interpreted as an integer
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 119, in _start_fn
    fn(gindex, *args)
  File "/content/transformers/examples/question-answering/run_squad_trainer.py", line 156, in _mp_fn
    main()
  File "/content/transformers/examples/question-answering/run_squad_trainer.py", line 145, in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 584, in train
    self.epoch = epoch + (step + 1) / len(epoch_iterator)
TypeError: 'NoneType' object cannot be interpreted as an integer
```

## To reproduce

Steps to reproduce the behavior:

1. install transformers from the master branch
2. install pytorch-xla using the following command:
```shell
VERSION = "20200325"
curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py
python pytorch-xla-env-setup.py --version $VERSION
```
3. run the training script (I'm using 1 tpu core merely to simplify the logs. The error is the same (for each core) when using 8 cores):
```shell
cd transformers/examples/

python ./xla_spawn.py --num_cores 1 \
question-answering/run_squad_trainer.py \
    --model_name_or_path bert-base-multilingual-cased \
    --model_type bert \
    --data_dir $DATA_DIR \
    --do_train \
    --per_device_train_batch_size 64 \
    --learning_rate 3e-5 \
    --num_train_epochs 2.0 \
    --max_seq_length 384 \
    --doc_stride 128 \
    --output_dir $OUT_DIR \
    --overwrite_output_dir
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The script runs and trains the model
<!-- A clear and concise description of what you would expect to happen. -->


Trainer automatically drops unused columns in nlp datasets

Here is a basic example of use for evaluation on SST-2:
```
from nlp import load_dataset, load_metric
from transformers import AutoModelForSequenceClassification, AutoTokenizer, Trainer, TrainingArguments
dataset = load_dataset('glue', 'sst2')
metric = load_metric('glue', 'sst2')

model_name = "distilbert-base-uncased-finetuned-sst-2-english"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

encoded_dataset = dataset.map(lambda examples: tokenizer(examples['sentence'], padding=True), batched=True)
args = TrainingArguments(output_dir = "test")

def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    return metric.compute(predictions.argmax(axis=-1), labels)

trainer = Trainer(
    model,
    args,
    train_dataset=encoded_dataset["train"],
    eval_dataset=encoded_dataset["validation"],
    compute_metrics=compute_metrics,
)
trainer.evaluate()
```

The goal is to then refine this new API by trying to use it in all examples.


[DO NOT SUBMIT] Run TPU examples for PR commits.

Trying out the CircleCI flow. I'll delete this PR after testing.


[TF Longformer] Improve Speed for TF Longformer

This PR:
- adds a simple test for all tf models to verify that the forward function can be used in graph mode
- optimizes TF Longformer, by removing unnecessary calculation, such as `tf.transpose()` (In contrast to PyTorch, `tf.transpose()` allocates a new tensor and thus should be avoided). This also cleans up the code IMO.

=> These changes lead to a speed-up of 1.03 which is actually not that much...more details in benchmark below.

After a lot of digging TF XLA will not be very easy to implement as a lot of kernels that are highly used in this model `tf.where` are not implemented for XLA (yet). So TF Longformer TPU will not work sadly for the moment @ibeltagy

### Conclusion

For me the PR was also a good exercise to see whether TF can significantly sped up by removing unnecessary tensor allocations. It seems like it's not really worth it go through all the tf models if the improvement in speed is only around 2,3%.


Get GKE logs via kubectl logs instead of gcloud logging read.

This should be a much faster method of getting logs from GKE back to the CircleCI machine.


Test model outputs equivalence

Adds a test to check that the model outputs keep the same values and order as the tuple output.


Can't download  'Helsinki-NLP/opus-mt-hye-eng' model

## Environment info

- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-51-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.3.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: not sure

## Information

Model I am using: MarianMTModel, AutoModelWithLMHead

The problem arises when using the official example scripts (https://huggingface.co/Helsinki-NLP/opus-mt-hye-eng):

```python
from transformers import AutoTokenizer, AutoModelWithLMHead
tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-hye-eng")
model = AutoModelWithLMHead.from_pretrained("Helsinki-NLP/opus-mt-hye-eng")
```

Gives error

```
/home/sonja/.local/lib/python3.6/site-packages/transformers/modeling_auto.py:798: FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed in a future version. Please use `AutoModelForCausalLM` for causal language models, `AutoModelForMaskedLM` for masked language models and `AutoModelForSeq2SeqLM` for encoder-decoder models.
  FutureWarning,
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~/.local/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    654                 if resolved_archive_file is None:
--> 655                     raise EnvironmentError
    656             except EnvironmentError:

OSError:

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<ipython-input-2-04055899a280> in <module>
      1 from transformers import AutoTokenizer, AutoModelWithLMHead
      2 tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-hye-eng")
----> 3 model = AutoModelWithLMHead.from_pretrained("Helsinki-NLP/opus-mt-hye-eng")

~/.local/lib/python3.6/site-packages/transformers/modeling_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    804         for config_class, model_class in MODEL_WITH_LM_HEAD_MAPPING.items():
    805             if isinstance(config, config_class):
--> 806                 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
    807         raise ValueError(
    808             "Unrecognized configuration class {} for this kind of AutoModel: {}.\n"

~/.local/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    660                     f"- or '{pretrained_model_name_or_path}' is the correct path to a directory containing a file named one of {WEIGHTS_NAME}, {TF2_WEIGHTS_NAME}, {TF_WEIGHTS_NAME}.\n\n"
    661                 )
--> 662                 raise EnvironmentError(msg)
    663
    664             if resolved_archive_file == archive_file:

OSError: Can't load weights for 'Helsinki-NLP/opus-mt-hye-eng'. Make sure that:

- 'Helsinki-NLP/opus-mt-hye-eng' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'Helsinki-NLP/opus-mt-hye-eng' is the correct path to a directory containing a file named one of pytorch_model.bin, tf_model.h5, model.ckpt.
```

Tried to download the model manually from link I got while debugging (https://cdn.huggingface.co/Helsinki-NLP/opus-mt-hye-eng/pytorch_model.bin) but it doesn't return anything relatable. Although for 'hye-rus' model (https://cdn.huggingface.co/Helsinki-NLP/opus-mt-hye-rus/pytorch_model.bin) I can easily download the file. Works fine for "eng-hye" and "rus-hye" too.

Hjälp, @sshleifer (sorry if mistagged)


Simple train from the start for translation transformer

Hi, sorry to bother. I am trying to train a translation transformer, I have seen the documentation but I am still really lost.

I have two datasets, the original message and the translated message.

Example:
dataset_x.txt
This is the message.
This is another message.
Another one.

dataset_y.txt
This<&>is the message<^>.
This is another<&> message.
Another one<%>.

I wanted a simple script which could tokenize these datasets and train any suited model from scratch.
Could anyone help me? Thanks a bunch!


Adding PaddingDataCollator

New version of #6398


MBartForConditionalGeneration

This PR adds MBartForConditionalGeneration. Regarding #6416

@sshleifer


Getting Error from Default Data Collator while training Bert on SQUAD 2.0

Hello, I'm a fresher playing with the transformers. I suppose to train the BERT model `bert-base-cased` on SQUAD 2.0, but having an error in `data_collator`: It shows an error of `TypeError: an integer is required`

Here is the detail:
```
Epoch:   0%|                                              | 0/2 [00:00<?, ?it/s]
Iteration:   0%|                                       | 0/4135 [00:00<?, ?it/s]
Epoch:   0%|                                              | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/scratch/yyu/codebase/odqa/odqa/reader/reader_trainer.py", line 138, in <module>
    trainer.train()
  File "/scratch/yyu/codebase/odqa/odqa/reader/reader_trainer.py", line 119, in train
    self._trainer.train()
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/transformers/trainer.py", line 456, in train
    for step, inputs in enumerate(epoch_iterator):
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/tqdm/std.py", line 1130, in __iter__
    for obj in iterable:
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 363, in __next__
    data = self._next_data()
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 403, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/transformers/data/data_collator.py", line 62, in default_data_collator
    batch[k] = torch.tensor([f[k] for f in features], dtype=torch.long)
TypeError: an integer is required (got type dict)
```

Here is my trainer configuration:
```
training_args = TrainingArguments(
            output_dir=self.output_dir,
            overwrite_output_dir=True,
            num_train_epochs=2,
            per_gpu_train_batch_size=32,
            # per_device_eval_batch_size=64,
            warmup_steps=500,
            weight_decay=0.01,
            # evaluate_during_training=True,
            save_steps=10_000,
            logging_dir='./logs',
        )
        self._trainer = Trainer(
            model=self._model,
            args=training_args,
            compute_metrics=self.compute_metrics,
            train_dataset=self.train_dataset,
            eval_dataset=self.test_dataset
        )
```

Does anyone know why it happens? And how can I fix this error? I don't know if this is one mistake in my code or from the transformers :-( Thanks for all help.

Best


TrainingArguments are ignored?!

## Environment info
- `transformers` version: 3.0.2
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
Trainer: @sgugger

## Information

Hey, I'm using `01_how-to-train.ipynb` to get a feeling for the object oriented way of training a bert model from scratch. Until now I've been using the scripts offered by offical bert repository. My target is to train all of my future Transformer models with your Huggingface interface (from scratch and of course fine tuning too).

I used `max_steps = 500_000` but it gets completely ignored. After training is started the output says:
```
Iteration: 11639/422095 [1:52:03<70:16:42, 1.62it/s]
Epoch 0/2 [00:00<?, ?it/s]
```
**Two epochs and 422095 iterations seems wrong!?** Official docs say _"max_steps = the total number of training steps to perform"_ Am I misinterpreting something?

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset:
  * line by line dataset
  * training a bert language model from scratch (generating vocab, setting a config, ...)

## To reproduce

Use colab "01_how-to-train.ipynb" (https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb
) and change TrainingArguments to the following:
```
training_args = TrainingArguments(
    output_dir="./smallBERTa",
    overwrite_output_dir=True,
    do_train=True,
    warmup_steps=5000,
    max_steps=500000,
    per_gpu_train_batch_size=64,
    save_steps=10_000,
    save_total_limit=2,
)
```
Yes, I am passing the `training_args` to the `Trainer()` object.

## Expected behavior

I'm expecting to get 500.000 global training steps and just one epoch.


Training GPT2 and Reformer from scratch.

Hello, I am looking, for example, script/notebook to train GPT2 and Reformer model from scratch in German.
Something similar to :
https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb

I am trying to modify the same notebook but GPT2 doesn't seem to accept LinebyLineDataset or padding.


Fix #6428

The `HfArgumentParser` was failing on arguments type-annoted with `Optional[bool]`. This fixes that (and issue #6428 in the process) so we don't have to remember to not put `Optional` around bools.


Epoch iterator for run_pl_ner.py

Dear all,

While training the run_pl_ner.py code, with increasing documents added the epoch iterator increases. But with this increase, it also prints a new line making the progress bar for the epoch span on multiple lines as shown below. I was wondering if there is a way to restrict this multi-line progress bar to one line?

Thank You.

![image](https://user-images.githubusercontent.com/45199062/90014027-a6d6d380-dca6-11ea-86d0-1ccc5f820d18.png)


Update README.md

model card

Related to pretrained model cards


Centralize logging

The goal of this PR is to offer a better way to manage logging to the HuggingFace/transformers users. It's a very simple proposal: implement a single logger that is shared across all files, and implement three helper methods that can be used across the library and by users:

```py
def get_logger():
    '''
    Returns the logger instance for the library, that can be managed as a traditional `logging` logger.
    '''

def get_verbosity():
    '''
    Returns the logger instance verbosity level. Used to manage what is printed, for example with tqdm loading bars.
    Same as doing:
        hf_logging.get_logger().getEffectiveLevel()
    '''

def set_verbosity(level: int):
    '''
    Sets the logger instance verbosity level. Used to set the desired verbosity level across the library.
    Same as doing:
        hf_logging.get_logger().setLevel(level)
    '''
```

Users can use these methods as such:

```py
from transformers import hf_logging

logger = hf_logging.get_logger()

hf_logging.set_verbosity(hf_logging.INFO)
# same as doing
logger.setLevel(hf_logging.INFO)
```

The noteworthy additions/changes are shown below.


Fix PABEE & PL CI failure


TF2 implementation of LineByLineTextDataset?

Hi, I have a text file which I want to use as input for trainer_tf.py. Since it requires a dataset object as input, is there any implementation of something like the LineByLineTextDataset module in TF2 as well?


Disabled pabee test

@JetRunner


[WIP] QA Loss refactoring

This is a refactoring experiment as suggested at https://github.com/huggingface/transformers/issues/6204

10 models and 1 template have been refactored - I will check for more if this looks promising (the doc and new function's signature are incomplete). Let me know whether to continue or not.

the refactoring was done with:

```
perl -0777 -pi -e '
$in = <<END;
logits = self.qa_outputs(sequence_output)
        start_logits, end_logits = logits.split(1, dim=-1)
        start_logits = start_logits.squeeze(-1)
        end_logits = end_logits.squeeze(-1)

        total_loss = None
        if start_positions is not None and end_positions is not None:
            # If we are on multi-GPU, split add a dimension
            if len(start_positions.size()) > 1:
                start_positions = start_positions.squeeze(-1)
            if len(end_positions.size()) > 1:
                end_positions = end_positions.squeeze(-1)
            # sometimes the start/end positions are outside our model inputs, we ignore these terms
            ignored_index = start_logits.size(1)
            start_positions.clamp_(0, ignored_index)
            end_positions.clamp_(0, ignored_index)

            loss_fct = CrossEntropyLoss(ignore_index=ignored_index)
            start_loss = loss_fct(start_logits, start_positions)
            end_loss = loss_fct(end_logits, end_positions)
            total_loss = (start_loss + end_loss) / 2
END
s/\Q$in\E/start_logits, end_logits, total_loss = self.calc_qa_loss(sequence_output, start_positions, end_positions)\n/msg
' \
./templates/adding_a_new_model/modeling_*    ./src/transformers/modeling_*

```

@sshleifer, I'm not sure how you're going to judge the coverage change as the coverage data is unreliable at the moment: https://github.com/huggingface/transformers/issues/6317


[test schedulers] adjust to test the first step's reading

As I was working on a new scheduler, it was difficult to match numbers since the first step's reading was dropped in `unwrap_schedule` wrappers (they were taking the measurement after stepping). This PR adjusts the wrappers to first take a reading and then step.

This PR also makes a small refactoring to move all the unwrapping into the script, so the test just compares 2 lists. (avoiding multiple `[l[0] for l in lrs_1]`)

The updated table is:
```
        scheds = {
            get_constant_schedule: ({}, [10.0] * self.num_steps),
            get_constant_schedule_with_warmup: (
                {"num_warmup_steps": 4},
                [0.0, 2.5, 5.0, 7.5, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0],
            ),
            get_linear_schedule_with_warmup: (
                {**common_kwargs},
                [0.0, 5.0, 10.0, 8.75, 7.5, 6.25, 5.0, 3.75, 2.5, 1.25],
            ),
            get_cosine_schedule_with_warmup: (
                {**common_kwargs},
                [0.0, 5.0, 10.0, 9.61, 8.53, 6.91, 5.0, 3.08, 1.46, 0.38],
            ),
            get_cosine_with_hard_restarts_schedule_with_warmup: (
                {**common_kwargs, "num_cycles": 2},
                [0.0, 5.0, 10.0, 8.53, 5.0, 1.46, 10.0, 8.53, 5.0, 1.46],
            ),
            get_polynomial_decay_schedule_with_warmup: (
                {**common_kwargs, "power": 2.0, "lr_end": 1e-7},
                [0.0, 5.0, 10.0, 7.656, 5.625, 3.906, 2.5, 1.406, 0.625, 0.156],
            ),
        }
```
Unrelated to the changes suggestion in this PR, it exposes 2 minor issues:

1. We definitely have a one off problem there, as the last step's reading is one reading too early (which this change exposes) - it doesn't complete the intended cycle. This is probably unimportant for 100s of steps, but it definitely stands out when developing a new scheduler.

To illustrate, see this change in reported number for `get_polynomial_decay_schedule_with_warmup`:

```
-                [5.0, 10.0, 7.656, 5.625, 3.906, 2.5, 1.406, 0.625, 0.156, 1e-07],
+                [0.0, 5.0, 10.0, 7.656, 5.625, 3.906, 2.5, 1.406, 0.625, 0.156],
```
the expected last step of `1e-07` is not there. It never was.

2. Also the first step's reading is `0.0` in all schedulers, except in `get_constant_schedule`, so the first step does nothing. This can be fixed with a potentially added `min_lr=1e-7` to all schedulers, as it was suggested by @sshleifer in one of the recent scheduler-related PRs.

Let me know if this better fits into its own issue, as these issues have nothing to do with the PR itself. Or perhaps the 2 issues are just unimportant...


Error in run_tf_squad.py script

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 tensorflow: @jplu
documentation: @sgugger
 --> @sgugger

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: SQUaD
* [ ] my own task or dataset: (give details below)

I'm simply trying to train a new question answering model using the TF trainer script, and I get the following error:
```python
Traceback (most recent call last):
  File "run_tf_squad.py", line 244, in <module>
    main()
  File "run_tf_squad.py", line 123, in main
    parser = HfArgumentParser((ModelArguments, DataTrainingArguments, TFTrainingArguments))
  File "/usr/local/lib/python3.6/dist-packages/transformers/hf_argparser.py", line 40, in __init__
    self._add_dataclass_arguments(dtype)
  File "/usr/local/lib/python3.6/dist-packages/transformers/hf_argparser.py", line 72, in _add_dataclass_arguments
    elif hasattr(field.type, "__origin__") and issubclass(field.type.__origin__, List):
  File "/usr/lib/python3.6/typing.py", line 1154, in __subclasscheck__
    return super().__subclasscheck__(cls)
  File "/usr/lib/python3.6/abc.py", line 209, in __subclasscheck__
    ok = cls.__subclasshook__(subclass)
  File "/usr/lib/python3.6/typing.py", line 890, in __extrahook__
    if cls.__extra__ and issubclass(subclass, cls.__extra__):
TypeError: issubclass() arg 1 must be a class
```

## To reproduce

Steps to reproduce the behavior:

1.install transformers from the master branch
2.run the example script in question-answering:
```
python run_tf_squad.py \
    --model_name_or_path bert-base-uncased \
    --output_dir model \
    --max_seq_length 384 \
    --num_train_epochs 2 \
    --per_gpu_train_batch_size 8 \
    --per_gpu_eval_batch_size 16 \
    --do_train \
    --logging_dir logs \
    --logging_steps 10 \
    --learning_rate 3e-5 \
    --doc_stride 128
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The script should run normally and train the model
<!-- A clear and concise description of what you would expect to happen. -->


Activate check on the CI

The check of modules documented and tested was only in `make quality`, not circleCI


Move prediction_loss_only to TrainingArguments

It didn't make sense to me to have that boolean flag in the init of `Trainer` when all the other ones are in `TrainingArguments` so I deprecated it and moved it.

Let me know if you think it's a wrong move.

Unrelated changes: had to fix `make quality` that was complaining about non-documented or tested models and it was easier to fix them than change my setup (which doesn't let me push if make quality fails).


[examples] add pytest dependency

Not really a new dependency, since already installed by `pip install -e .[testing]` but some examples users just run:
```
pip install -r examples/requirements.txt
```
so they don't have it, and tests break.


actions CI self-scheduled: run_examples torch even if run_torch_tests fails


Fixes to make life easier with the nlp library

This PR adds two things to make the interface easier with the `nlp` library:
- `BatchEncoding` stops enforcing a 2-dim for every tensor, which causes problems for labels (which should be one vector of shape `[batch_size]`).
- `PreTrainedTokenizerBase.pad` accepts tensors as inputs, which makes it easy to use this function for data collation.

Added proper documentation and tests from @thomwolf initial work.


[test] replace capsys with the more refined CaptureStderr/CaptureStdout

Now https://github.com/huggingface/transformers/pull/6231 has been merged, we can now do a more refined, more tightly scoped std stream captures as shown [here](https://github.com/huggingface/transformers/pull/6231#issuecomment-671789424)

otherwise no test functionality change.

Any CI fails are unrelated.


test_run_glue_with_pabee failing

examples/bert-loses-patience/test_run_glue_with_pabee.py::PabeeTests::test_run_glue

https://app.circleci.com/pipelines/github/huggingface/transformers/10373/workflows/0c9f2e61-2732-4857-84f0-71b59ddf10a9/jobs/71646

@JetRunner


Experiment: ROUGE impact of using pegasus length-penalty implementation

Help wanted

Extra attention is needed, help appreciated

code is under `length_normalization` in the pegasus [tf repo](https://github.com/google-research/pegasus)


Add pegasus model cards


All learning rates are 0 warning

- `transformers` version: 3.0.2
- Platform: Linux
- Python version: 3.6
- PyTorch version (GPU?): 1.4 (GPU)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@sshleifer

## Information

Model I am using (Bert, XLNet ...): BART

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Running the example script in https://github.com/huggingface/transformers/tree/master/examples/seq2seq (finetune_bart_tiny.sh), I'm getting this warning in the beginning of training. However, the training process is continuing after that.

Warning:
```
finetune.py:245: UserWarning: All learning rates are 0
  warnings.warn("All learning rates are 0")
Epoch 1:   0%|
/home/sajad/anaconda3/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:224: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.
  warnings.warn("To get the last learning rate computed by the scheduler, "
```

## Expected behavior
While the training seemingly goes well, I'm wondering if this warning would cause problems, leading to deteriorate model's final performance? As a add-on, I've also incorporated the gradient checkpointing to some computational blocks of BART (modifying `modelling_bart.py` script a bit). But even w/o incorporating this module, I'm still getting this warning message? Any thoughts of how to solve it?


how to fine tune t5 model for summarization task using tensorflow2?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


Docs: Separate documentation for mbart

Currently all mbart documentation is stuffed into docs/
https://github.com/huggingface/transformers/blob/353b8f1e7a7361c0afd9e391381bc226b4a5ca8f/docs/source/model_doc/bart.rst#L42

mbart should have it's own `model_doc/mbart.rst` and entry in `pretrained_models.rst`.

Optionally you can also create a new `src/transformers/modeling_mbart.py` with roughly these contents:

```python
from .modeling_bart import BartForConditionalGeneration
from .configuration_bart import MbartConfig
class MBartForConditionalGeneration(BartForConditionalGeneration):
    config_class = MbartConfig
	# this model fully inherits its implementation from bart
```


[EncoderDecoder] Add Cross Attention for GPT2

This PR implements **Bert2GPT2** by adding cross-attention layers to GPT2.

Note that currently it is not possible to speed up decoder generation with the encoder-decoder framework (by using GPT2's past tensors) since it has to be implemented for all models that are compatible with the encoder/decoder framework (Bert, Roberta) before it can be used within the framework.

All GPT2 `RUN_SLOW` tests are verified to pass.

**Future PRs TODO**:
- [ ] Verify that Bert2GPT2 works by training on CNN Daily Mail summarization
- [ ] Add smart caching to Bert and add it to the encoder-decoder framework
- [ ] Update encoder-decoder docs
- [ ] Add a notebook explaining how to use encoder-decoder models.


## Environment info

- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-53-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: Distributed

Hey there,

I've run into this issue and not sure how to fix it:

    TypeError: forward() got an unexpected keyword argument 'labels'

I'm running transformers v3.0.2 installed via pip

Please see my code below. There is nothing fancy going on, I'm just trying to train RobertaMLM for a few more epochs on a different dataset.

```python
import os
import argparse
import datetime
from torch.utils.tensorboard import SummaryWriter
from transformers import RobertaModel, RobertaConfig, RobertaTokenizerFast, LineByLineTextDataset, DataCollatorForLanguageModeling, Trainer, TrainingArguments
from configs import model_directory, tensorboard_directory
from logger import get_logger

log = get_logger(__name__)

args = argparse.Namespace(
	seed=42,
	model_id="Roberta2",
	pretrained_model_name_or_path="roberta-base",
	vocab_file="/data/nlp/roberta_vocabulary/roberta-base-vocab.json",
	merges_file="/data/nlp/roberta_vocabulary/roberta-base-merges.txt",
	filename="/data/nlp/trc2.txt",
	block_size=2**7,
	epochs=25,
)

output_directory = os.path.join(model_directory, args.model_id)
os.makedirs(output_directory, exist_ok=True)
os.environ["TOKENIZERS_PARALLELISM"] = "false"

def build_model():
	tokenizer = RobertaTokenizerFast.from_pretrained(pretrained_model_name_or_path=args.pretrained_model_name_or_path, lowercase=True, add_prefix_space=True, max_len=512)

	config = RobertaConfig.from_pretrained(args.pretrained_model_name_or_path)
	config.output_hidden_states = False

	model = RobertaModel.from_pretrained(pretrained_model_name_or_path=args.pretrained_model_name_or_path, config=config, cache_dir=output_directory)

	dataset = LineByLineTextDataset(
		tokenizer=tokenizer,
		file_path=args.filename,
		block_size=args.block_size,
	)

	data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=True, mlm_probability=0.15)

	training_args = TrainingArguments(
		seed=args.seed,
		output_dir=output_directory,
		overwrite_output_dir=True,
		num_train_epochs=args.epochs,
		per_device_train_batch_size=128,
		save_steps=10_000,
		# save_total_limit=2,
		fp16=True,
		fp16_opt_level="O1"
	)

	trainer = Trainer(
		model=model,
		args=training_args,
		data_collator=data_collator,
		train_dataset=dataset,
		prediction_loss_only=True,
	)

	trainer.train()
	trainer.save_model(output_directory)

```

tag: @sgugger


Create README.md

model card

Related to pretrained model cards

Model card for https://huggingface.co/akhooli/gpt2-small-arabic


Create model card T5-base fine-tuned on event2Mind for Intent Prediction

Related to pretrained model cards

More funny examples: https://twitter.com/mrm8488/status/1292952742395367424


[EncoderDecoder] Add encoder-decoder for roberta/ vanilla longformer

This PR adds Roberta to the Encoder Decoder framework. Thus, it automatically makes it possible to use both `Roberta2Roberta` models and `Longformer2Roberta` model:

```python
from transformers import EncoderDecoderModel
model = EncoderDecoderModel.from_pretrained("roberta-base", "roberta-base")
input_ids = torch.tensor([10 * [0]])
model(input_ids=input_ids, decoder_input_ids=input_ids)
```

and

```python
from transformers import EncoderDecoderModel
model = EncoderDecoderModel.from_pretrained("allenai/longformer-base-4096", "roberta-base")
input_ids = torch.tensor([10 * [0]])
model(input_ids=input_ids, decoder_input_ids=input_ids)
```

Also pinging @ibeltagy and @patil-suraj


Cannot unzip the XNLI-MT 1.0 zip file.

# ❓ Questions & Help

Is there anyone who succeeding the unzip of XNLI-MT 1.0 zip file?

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->


TF2 TPU slow?

## Environment info
- `transformers` version: 3.0.2
- Platform: ubuntu 18.04
- Python version: 3.6
- Tensorflow version (GPU?): TF 2.3.0

### Who can help
@jplu

## Information

Model I am using (Bert, XLNet ...): BERT

The problem arises when using:
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] task: MLM

## To reproduce
I am using the TFTrainer from `transformers` for MLM pretraining. However, it seems that even for TPU training each batch is fed separately to the TPU, while it's usually more common to feed a bunch of batches to the TPU for efficiency (see https://github.com/tensorflow/models/blob/master/official/nlp/bert/model_training_utils.py#L226).

I am not sure that's the only problem, but MLM pretraining BERT is around 3x slower on TPU with the TFTrainer compared to the official implementation (https://github.com/google-research/bert).

For better TPU usage, we probably need sth like here:
https://github.com/tensorflow/models/blob/master/official/nlp/bert/model_training_utils.py#L345-L361


i have used t5_base for abstractive summarization but it is not giving good results,Could you please give me solution for this

# 🖥 Benchmarking `transformers`

## Benchmark

Which part of `transformers` did you benchmark?

## Set-up

What did you run your benchmarks on? Please include details, such as: CPU, GPU? If using multiple GPUs, which parallelization did you use?

## Results

Put your results here!


Slow Decoding Speed when using BertForLMModel

I set the BertLMHeadModel as Decoder in my Seq2Seq model. It seems to work well in training. But when decoing, it decodes very slowly. I think there is no layer_past used in GPT2, XLNet in BertLMHeadModel and many attentions are computed repetitively?