Add support for `encoder_hidden_states` and `encoder_attention_mask` in modeling_longformer

# 🚀 Feature request

In order to use the longformer with the `EncoderDecoderModel` structure, it needs to accept `head_mask`, `encoder_hidden_states`, and `encoder_attention_mask` in the `forward()` function. Currently, `LongformerSelfAttention` asserts that `encoder_hidden_states` and `encoder_attention_mask` are `None`. I attempted to implement this functionality by comparing the bert implementation, which accepts these values, to the longformer implementation.

I can create an `EncoderDecoderModel` like so: `model = EncoderDecoderModel.from_encoder_decoder_pretrained("allenai/longformer-base-4096", "allenai/longformer-base-4096")` and the model does train using my changes. Even though the model trains, I am not confident that the changes I made are correct. I think the cross attention mechanism already exists within the `LongformerModel` since it is a subclass of `RobertaModel` which is a subclass of `BertModel`. `BertModel` uses `BertLayer`s which use cross attention if `self.is_decoder` is True.

Furthermore, I get the below error messages, which have something to do with the inheritance hierarchy. It fails to initialize the cross attention layers, which makes sense, but I am not sure about the other ones.

```
2020-06-21 18:00:53,533|transformers.modeling_encoder_decoder|INFO> Initializing allenai/longformer-base-4096 as a decoder model. Cross attention layers are added to allenai/longformer-base-4096 and randomly initialized if allenai/longformer-base-4096's architecture allows for cross attention layers.
2020-06-21 18:00:53,609|transformers.modeling_utils|INFO> loading weights file https://cdn.huggingface.co/allenai/longformer-base-4096/pytorch_model.bin from cache at /root/.cache/torch/transformers/dfc92dbbf5c555abf807425ebdb22b55de7a17e21fe1c48cbaa5764982c1d9c0.cd65234711d2e83d420aa696eb9186cdec6ab79ef8bf090b442cf249443dfa92
2020-06-21 18:00:58,792|transformers.modeling_utils|INFO> Weights of BertLMHeadModel not initialized from pretrained model: ['embeddings.word_embeddings.weight', 'embeddings.position_embeddings.weight', 'embeddings.token_type_embeddings.weight', 'embeddings.LayerNorm.weight', 'embeddings.LayerNorm.bias', 'encoder.layer.0.attention.self.query.weight', 'encoder.layer.0.attention.self.query.bias', 'encoder.layer.0.attention.self.key.weight', 'encoder.layer.0.attention.self.key.bias', 'encoder.layer.0.attention.self.value.weight', 'encoder.layer.0.attention.self.value.bias', 'encoder.layer.0.attention.output.dense.weight', 'encoder.layer.0.attention.output.dense.bias', 'encoder.layer.0.attention.output.LayerNorm.weight', 'encoder.layer.0.attention.output.LayerNorm.bias', 'encoder.layer.0.crossattention.self.query.weight', 'encoder.layer.0.crossattention.self.query.bias', 'encoder.layer.0.crossattention.self.key.weight', 'encoder.layer.0.crossattention.self.key.bias', 'encoder.layer.0.crossattention.self.value.weight', 'encoder.layer.0.crossattention.self.value.bias', 'encoder.layer.0.crossattention.output.dense.weight', 'encoder.layer.0.crossattention.output.dense.bias', 'encoder.layer.0.crossattention.output.LayerNorm.weight', 'encoder.layer.0.crossattention.output.LayerNorm.bias', 'encoder.layer.0.intermediate.dense.weight', 'encoder.layer.0.intermediate.dense.bias', 'encoder.layer.0.output.dense.weight', 'encoder.layer.0.output.dense.bias', 'encoder.layer.0.output.LayerNorm.weight', 'encoder.layer.0.output.LayerNorm.bias', 'encoder.layer.1.attention.self.query.weight', 'encoder.layer.1.attention.self.query.bias', 'encoder.layer.1.attention.self.key.weight', 'encoder.layer.1.attention.self.key.bias', 'encoder.layer.1.attention.self.value.weight', 'encoder.layer.1.attention.self.value.bias', 'encoder.layer.1.attention.output.dense.weight', 'encoder.layer.1.attention.output.dense.bias', 'encoder.layer.1.attention.output.LayerNorm.weight', 'encoder.layer.1.attention.output.LayerNorm.bias', 'encoder.layer.1.crossattention.self.query.weight', 'encoder.layer.1.crossattention.self.query.bias', 'encoder.layer.1.crossattention.self.key.weight', 'encoder.layer.1.crossattention.self.key.bias', 'encoder.layer.1.crossattention.self.value.weight', 'encoder.layer.1.crossattention.self.value.bias', 'encoder.layer.1.crossattention.output.dense.weight', 'encoder.layer.1.crossattention.output.dense.bias', 'encoder.layer.1.crossattention.output.LayerNorm.weight', 'encoder.layer.1.crossattention.output.LayerNorm.bias', 'encoder.layer.1.intermediate.dense.weight', 'encoder.layer.1.intermediate.dense.bias', 'encoder.layer.1.output.dense.weight', 'encoder.layer.1.output.dense.bias', 'encoder.layer.1.output.LayerNorm.weight', 'encoder.layer.1.output.LayerNorm.bias', 'encoder.layer.2.attention.self.query.weight', 'encoder.layer.2.attention.self.query.bias', 'encoder.layer.2.attention.self.key.weight', 'encoder.layer.2.attention.self.key.bias', 'encoder.layer.2.attention.self.value.weight', 'encoder.layer.2.attention.self.value.bias', 'encoder.layer.2.attention.output.dense.weight', 'encoder.layer.2.attention.output.dense.bias', 'encoder.layer.2.attention.output.LayerNorm.weight', 'encoder.layer.2.attention.output.LayerNorm.bias', 'encoder.layer.2.crossattention.self.query.weight', 'encoder.layer.2.crossattention.self.query.bias', 'encoder.layer.2.crossattention.self.key.weight', 'encoder.layer.2.crossattention.self.key.bias', 'encoder.layer.2.crossattention.self.value.weight', 'encoder.layer.2.crossattention.self.value.bias', 'encoder.layer.2.crossattention.output.dense.weight', 'encoder.layer.2.crossattention.output.dense.bias', 'encoder.layer.2.crossattention.output.LayerNorm.weight', 'encoder.layer.2.crossattention.output.LayerNorm.bias', 'encoder.layer.2.intermediate.dense.weight', 'encoder.layer.2.intermediate.dense.bias', 'encoder.layer.2.output.dense.weight', 'encoder.layer.2.output.dense.bias', 'encoder.layer.2.output.LayerNorm.weight', 'encoder.layer.2.output.LayerNorm.bias', 'encoder.layer.3.attention.self.query.weight', 'encoder.layer.3.attention.self.query.bias', 'encoder.layer.3.attention.self.key.weight', 'encoder.layer.3.attention.self.key.bias', 'encoder.layer.3.attention.self.value.weight', 'encoder.layer.3.attention.self.value.bias', 'encoder.layer.3.attention.output.dense.weight', 'encoder.layer.3.attention.output.dense.bias', 'encoder.layer.3.attention.output.LayerNorm.weight', 'encoder.layer.3.attention.output.LayerNorm.bias', 'encoder.layer.3.crossattention.self.query.weight', 'encoder.layer.3.crossattention.self.query.bias', 'encoder.layer.3.crossattention.self.key.weight', 'encoder.layer.3.crossattention.self.key.bias', 'encoder.layer.3.crossattention.self.value.weight', 'encoder.layer.3.crossattention.self.value.bias', 'encoder.layer.3.crossattention.output.dense.weight', 'encoder.layer.3.crossattention.output.dense.bias', 'encoder.layer.3.crossattention.output.LayerNorm.weight', 'encoder.layer.3.crossattention.output.LayerNorm.bias', 'encoder.layer.3.intermediate.dense.weight', 'encoder.layer.3.intermediate.dense.bias', 'encoder.layer.3.output.dense.weight', 'encoder.layer.3.output.dense.bias', 'encoder.layer.3.output.LayerNorm.weight', 'encoder.layer.3.output.LayerNorm.bias', 'encoder.layer.4.attention.self.query.weight', 'encoder.layer.4.attention.self.query.bias', 'encoder.layer.4.attention.self.key.weight', 'encoder.layer.4.attention.self.key.bias', 'encoder.layer.4.attention.self.value.weight', 'encoder.layer.4.attention.self.value.bias', 'encoder.layer.4.attention.output.dense.weight', 'encoder.layer.4.attention.output.dense.bias', 'encoder.layer.4.attention.output.LayerNorm.weight', 'encoder.layer.4.attention.output.LayerNorm.bias', 'encoder.layer.4.crossattention.self.query.weight', 'encoder.layer.4.crossattention.self.query.bias', 'encoder.layer.4.crossattention.self.key.weight', 'encoder.layer.4.crossattention.self.key.bias', 'encoder.layer.4.crossattention.self.value.weight', 'encoder.layer.4.crossattention.self.value.bias', 'encoder.layer.4.crossattention.output.dense.weight', 'encoder.layer.4.crossattention.output.dense.bias', 'encoder.layer.4.crossattention.output.LayerNorm.weight', 'encoder.layer.4.crossattention.output.LayerNorm.bias', 'encoder.layer.4.intermediate.dense.weight', 'encoder.layer.4.intermediate.dense.bias', 'encoder.layer.4.output.dense.weight', 'encoder.layer.4.output.dense.bias', 'encoder.layer.4.output.LayerNorm.weight', 'encoder.layer.4.output.LayerNorm.bias', 'encoder.layer.5.attention.self.query.weight', 'encoder.layer.5.attention.self.query.bias', 'encoder.layer.5.attention.self.key.weight', 'encoder.layer.5.attention.self.key.bias', 'encoder.layer.5.attention.self.value.weight', 'encoder.layer.5.attention.self.value.bias', 'encoder.layer.5.attention.output.dense.weight', 'encoder.layer.5.attention.output.dense.bias', 'encoder.layer.5.attention.output.LayerNorm.weight', 'encoder.layer.5.attention.output.LayerNorm.bias', 'encoder.layer.5.crossattention.self.query.weight', 'encoder.layer.5.crossattention.self.query.bias', 'encoder.layer.5.crossattention.self.key.weight', 'encoder.layer.5.crossattention.self.key.bias', 'encoder.layer.5.crossattention.self.value.weight', 'encoder.layer.5.crossattention.self.value.bias', 'encoder.layer.5.crossattention.output.dense.weight', 'encoder.layer.5.crossattention.output.dense.bias', 'encoder.layer.5.crossattention.output.LayerNorm.weight', 'encoder.layer.5.crossattention.output.LayerNorm.bias', 'encoder.layer.5.intermediate.dense.weight', 'encoder.layer.5.intermediate.dense.bias', 'encoder.layer.5.output.dense.weight', 'encoder.layer.5.output.dense.bias', 'encoder.layer.5.output.LayerNorm.weight', 'encoder.layer.5.output.LayerNorm.bias', 'encoder.layer.6.attention.self.query.weight', 'encoder.layer.6.attention.self.query.bias', 'encoder.layer.6.attention.self.key.weight', 'encoder.layer.6.attention.self.key.bias', 'encoder.layer.6.attention.self.value.weight', 'encoder.layer.6.attention.self.value.bias', 'encoder.layer.6.attention.output.dense.weight', 'encoder.layer.6.attention.output.dense.bias', 'encoder.layer.6.attention.output.LayerNorm.weight', 'encoder.layer.6.attention.output.LayerNorm.bias', 'encoder.layer.6.crossattention.self.query.weight', 'encoder.layer.6.crossattention.self.query.bias', 'encoder.layer.6.crossattention.self.key.weight', 'encoder.layer.6.crossattention.self.key.bias', 'encoder.layer.6.crossattention.self.value.weight', 'encoder.layer.6.crossattention.self.value.bias', 'encoder.layer.6.crossattention.output.dense.weight', 'encoder.layer.6.crossattention.output.dense.bias', 'encoder.layer.6.crossattention.output.LayerNorm.weight', 'encoder.layer.6.crossattention.output.LayerNorm.bias', 'encoder.layer.6.intermediate.dense.weight', 'encoder.layer.6.intermediate.dense.bias', 'encoder.layer.6.output.dense.weight', 'encoder.layer.6.output.dense.bias', 'encoder.layer.6.output.LayerNorm.weight', 'encoder.layer.6.output.LayerNorm.bias', 'encoder.layer.7.attention.self.query.weight', 'encoder.layer.7.attention.self.query.bias', 'encoder.layer.7.attention.self.key.weight', 'encoder.layer.7.attention.self.key.bias', 'encoder.layer.7.attention.self.value.weight', 'encoder.layer.7.attention.self.value.bias', 'encoder.layer.7.attention.output.dense.weight', 'encoder.layer.7.attention.output.dense.bias', 'encoder.layer.7.attention.output.LayerNorm.weight', 'encoder.layer.7.attention.output.LayerNorm.bias', 'encoder.layer.7.crossattention.self.query.weight', 'encoder.layer.7.crossattention.self.query.bias', 'encoder.layer.7.crossattention.self.key.weight', 'encoder.layer.7.crossattention.self.key.bias', 'encoder.layer.7.crossattention.self.value.weight', 'encoder.layer.7.crossattention.self.value.bias', 'encoder.layer.7.crossattention.output.dense.weight', 'encoder.layer.7.crossattention.output.dense.bias', 'encoder.layer.7.crossattention.output.LayerNorm.weight', 'encoder.layer.7.crossattention.output.LayerNorm.bias', 'encoder.layer.7.intermediate.dense.weight', 'encoder.layer.7.intermediate.dense.bias', 'encoder.layer.7.output.dense.weight', 'encoder.layer.7.output.dense.bias', 'encoder.layer.7.output.LayerNorm.weight', 'encoder.layer.7.output.LayerNorm.bias', 'encoder.layer.8.attention.self.query.weight', 'encoder.layer.8.attention.self.query.bias', 'encoder.layer.8.attention.self.key.weight', 'encoder.layer.8.attention.self.key.bias', 'encoder.layer.8.attention.self.value.weight', 'encoder.layer.8.attention.self.value.bias', 'encoder.layer.8.attention.output.dense.weight', 'encoder.layer.8.attention.output.dense.bias', 'encoder.layer.8.attention.output.LayerNorm.weight', 'encoder.layer.8.attention.output.LayerNorm.bias', 'encoder.layer.8.crossattention.self.query.weight', 'encoder.layer.8.crossattention.self.query.bias', 'encoder.layer.8.crossattention.self.key.weight', 'encoder.layer.8.crossattention.self.key.bias', 'encoder.layer.8.crossattention.self.value.weight', 'encoder.layer.8.crossattention.self.value.bias', 'encoder.layer.8.crossattention.output.dense.weight', 'encoder.layer.8.crossattention.output.dense.bias', 'encoder.layer.8.crossattention.output.LayerNorm.weight', 'encoder.layer.8.crossattention.output.LayerNorm.bias', 'encoder.layer.8.intermediate.dense.weight', 'encoder.layer.8.intermediate.dense.bias', 'encoder.layer.8.output.dense.weight', 'encoder.layer.8.output.dense.bias', 'encoder.layer.8.output.LayerNorm.weight', 'encoder.layer.8.output.LayerNorm.bias', 'encoder.layer.9.attention.self.query.weight', 'encoder.layer.9.attention.self.query.bias', 'encoder.layer.9.attention.self.key.weight', 'encoder.layer.9.attention.self.key.bias', 'encoder.layer.9.attention.self.value.weight', 'encoder.layer.9.attention.self.value.bias', 'encoder.layer.9.attention.output.dense.weight', 'encoder.layer.9.attention.output.dense.bias', 'encoder.layer.9.attention.output.LayerNorm.weight', 'encoder.layer.9.attention.output.LayerNorm.bias', 'encoder.layer.9.crossattention.self.query.weight', 'encoder.layer.9.crossattention.self.query.bias', 'encoder.layer.9.crossattention.self.key.weight', 'encoder.layer.9.crossattention.self.key.bias', 'encoder.layer.9.crossattention.self.value.weight', 'encoder.layer.9.crossattention.self.value.bias', 'encoder.layer.9.crossattention.output.dense.weight', 'encoder.layer.9.crossattention.output.dense.bias', 'encoder.layer.9.crossattention.output.LayerNorm.weight', 'encoder.layer.9.crossattention.output.LayerNorm.bias', 'encoder.layer.9.intermediate.dense.weight', 'encoder.layer.9.intermediate.dense.bias', 'encoder.layer.9.output.dense.weight', 'encoder.layer.9.output.dense.bias', 'encoder.layer.9.output.LayerNorm.weight', 'encoder.layer.9.output.LayerNorm.bias', 'encoder.layer.10.attention.self.query.weight', 'encoder.layer.10.attention.self.query.bias', 'encoder.layer.10.attention.self.key.weight', 'encoder.layer.10.attention.self.key.bias', 'encoder.layer.10.attention.self.value.weight', 'encoder.layer.10.attention.self.value.bias', 'encoder.layer.10.attention.output.dense.weight', 'encoder.layer.10.attention.output.dense.bias', 'encoder.layer.10.attention.output.LayerNorm.weight', 'encoder.layer.10.attention.output.LayerNorm.bias', 'encoder.layer.10.crossattention.self.query.weight', 'encoder.layer.10.crossattention.self.query.bias', 'encoder.layer.10.crossattention.self.key.weight', 'encoder.layer.10.crossattention.self.key.bias', 'encoder.layer.10.crossattention.self.value.weight', 'encoder.layer.10.crossattention.self.value.bias', 'encoder.layer.10.crossattention.output.dense.weight', 'encoder.layer.10.crossattention.output.dense.bias', 'encoder.layer.10.crossattention.output.LayerNorm.weight', 'encoder.layer.10.crossattention.output.LayerNorm.bias', 'encoder.layer.10.intermediate.dense.weight', 'encoder.layer.10.intermediate.dense.bias', 'encoder.layer.10.output.dense.weight', 'encoder.layer.10.output.dense.bias', 'encoder.layer.10.output.LayerNorm.weight', 'encoder.layer.10.output.LayerNorm.bias', 'encoder.layer.11.attention.self.query.weight', 'encoder.layer.11.attention.self.query.bias', 'encoder.layer.11.attention.self.key.weight', 'encoder.layer.11.attention.self.key.bias', 'encoder.layer.11.attention.self.value.weight', 'encoder.layer.11.attention.self.value.bias', 'encoder.layer.11.attention.output.dense.weight', 'encoder.layer.11.attention.output.dense.bias', 'encoder.layer.11.attention.output.LayerNorm.weight', 'encoder.layer.11.attention.output.LayerNorm.bias', 'encoder.layer.11.crossattention.self.query.weight', 'encoder.layer.11.crossattention.self.query.bias', 'encoder.layer.11.crossattention.self.key.weight', 'encoder.layer.11.crossattention.self.key.bias', 'encoder.layer.11.crossattention.self.value.weight', 'encoder.layer.11.crossattention.self.value.bias', 'encoder.layer.11.crossattention.output.dense.weight', 'encoder.layer.11.crossattention.output.dense.bias', 'encoder.layer.11.crossattention.output.LayerNorm.weight', 'encoder.layer.11.crossattention.output.LayerNorm.bias', 'encoder.layer.11.intermediate.dense.weight', 'encoder.layer.11.intermediate.dense.bias', 'encoder.layer.11.output.dense.weight', 'encoder.layer.11.output.dense.bias', 'encoder.layer.11.output.LayerNorm.weight', 'encoder.layer.11.output.LayerNorm.bias', 'pooler.dense.weight', 'pooler.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.decoder.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.bias']
2020-06-21 18:00:58,816|transformers.modeling_utils|INFO> Weights from pretrained model not used in BertLMHeadModel: ['longformer.embeddings.word_embeddings.weight', 'longformer.embeddings.position_embeddings.weight', 'longformer.embeddings.token_type_embeddings.weight', 'longformer.embeddings.LayerNorm.weight', 'longformer.embeddings.LayerNorm.bias', 'longformer.encoder.layer.0.attention.self.query.weight', 'longformer.encoder.layer.0.attention.self.query.bias', 'longformer.encoder.layer.0.attention.self.key.weight', 'longformer.encoder.layer.0.attention.self.key.bias', 'longformer.encoder.layer.0.attention.self.value.weight', 'longformer.encoder.layer.0.attention.self.value.bias', 'longformer.encoder.layer.0.attention.self.query_global.weight', 'longformer.encoder.layer.0.attention.self.query_global.bias', 'longformer.encoder.layer.0.attention.self.key_global.weight', 'longformer.encoder.layer.0.attention.self.key_global.bias', 'longformer.encoder.layer.0.attention.self.value_global.weight', 'longformer.encoder.layer.0.attention.self.value_global.bias', 'longformer.encoder.layer.0.attention.output.dense.weight', 'longformer.encoder.layer.0.attention.output.dense.bias', 'longformer.encoder.layer.0.attention.output.LayerNorm.weight', 'longformer.encoder.layer.0.attention.output.LayerNorm.bias', 'longformer.encoder.layer.0.intermediate.dense.weight', 'longformer.encoder.layer.0.intermediate.dense.bias', 'longformer.encoder.layer.0.output.dense.weight', 'longformer.encoder.layer.0.output.dense.bias', 'longformer.encoder.layer.0.output.LayerNorm.weight', 'longformer.encoder.layer.0.output.LayerNorm.bias', 'longformer.encoder.layer.1.attention.self.query.weight', 'longformer.encoder.layer.1.attention.self.query.bias', 'longformer.encoder.layer.1.attention.self.key.weight', 'longformer.encoder.layer.1.attention.self.key.bias', 'longformer.encoder.layer.1.attention.self.value.weight', 'longformer.encoder.layer.1.attention.self.value.bias', 'longformer.encoder.layer.1.attention.self.query_global.weight', 'longformer.encoder.layer.1.attention.self.query_global.bias', 'longformer.encoder.layer.1.attention.self.key_global.weight', 'longformer.encoder.layer.1.attention.self.key_global.bias', 'longformer.encoder.layer.1.attention.self.value_global.weight', 'longformer.encoder.layer.1.attention.self.value_global.bias', 'longformer.encoder.layer.1.attention.output.dense.weight', 'longformer.encoder.layer.1.attention.output.dense.bias', 'longformer.encoder.layer.1.attention.output.LayerNorm.weight', 'longformer.encoder.layer.1.attention.output.LayerNorm.bias', 'longformer.encoder.layer.1.intermediate.dense.weight', 'longformer.encoder.layer.1.intermediate.dense.bias', 'longformer.encoder.layer.1.output.dense.weight', 'longformer.encoder.layer.1.output.dense.bias', 'longformer.encoder.layer.1.output.LayerNorm.weight', 'longformer.encoder.layer.1.output.LayerNorm.bias', 'longformer.encoder.layer.2.attention.self.query.weight', 'longformer.encoder.layer.2.attention.self.query.bias', 'longformer.encoder.layer.2.attention.self.key.weight', 'longformer.encoder.layer.2.attention.self.key.bias', 'longformer.encoder.layer.2.attention.self.value.weight', 'longformer.encoder.layer.2.attention.self.value.bias', 'longformer.encoder.layer.2.attention.self.query_global.weight', 'longformer.encoder.layer.2.attention.self.query_global.bias', 'longformer.encoder.layer.2.attention.self.key_global.weight', 'longformer.encoder.layer.2.attention.self.key_global.bias', 'longformer.encoder.layer.2.attention.self.value_global.weight', 'longformer.encoder.layer.2.attention.self.value_global.bias', 'longformer.encoder.layer.2.attention.output.dense.weight', 'longformer.encoder.layer.2.attention.output.dense.bias', 'longformer.encoder.layer.2.attention.output.LayerNorm.weight', 'longformer.encoder.layer.2.attention.output.LayerNorm.bias', 'longformer.encoder.layer.2.intermediate.dense.weight', 'longformer.encoder.layer.2.intermediate.dense.bias', 'longformer.encoder.layer.2.output.dense.weight', 'longformer.encoder.layer.2.output.dense.bias', 'longformer.encoder.layer.2.output.LayerNorm.weight', 'longformer.encoder.layer.2.output.LayerNorm.bias', 'longformer.encoder.layer.3.attention.self.query.weight', 'longformer.encoder.layer.3.attention.self.query.bias', 'longformer.encoder.layer.3.attention.self.key.weight', 'longformer.encoder.layer.3.attention.self.key.bias', 'longformer.encoder.layer.3.attention.self.value.weight', 'longformer.encoder.layer.3.attention.self.value.bias', 'longformer.encoder.layer.3.attention.self.query_global.weight', 'longformer.encoder.layer.3.attention.self.query_global.bias', 'longformer.encoder.layer.3.attention.self.key_global.weight', 'longformer.encoder.layer.3.attention.self.key_global.bias', 'longformer.encoder.layer.3.attention.self.value_global.weight', 'longformer.encoder.layer.3.attention.self.value_global.bias', 'longformer.encoder.layer.3.attention.output.dense.weight', 'longformer.encoder.layer.3.attention.output.dense.bias', 'longformer.encoder.layer.3.attention.output.LayerNorm.weight', 'longformer.encoder.layer.3.attention.output.LayerNorm.bias', 'longformer.encoder.layer.3.intermediate.dense.weight', 'longformer.encoder.layer.3.intermediate.dense.bias', 'longformer.encoder.layer.3.output.dense.weight', 'longformer.encoder.layer.3.output.dense.bias', 'longformer.encoder.layer.3.output.LayerNorm.weight', 'longformer.encoder.layer.3.output.LayerNorm.bias', 'longformer.encoder.layer.4.attention.self.query.weight', 'longformer.encoder.layer.4.attention.self.query.bias', 'longformer.encoder.layer.4.attention.self.key.weight', 'longformer.encoder.layer.4.attention.self.key.bias', 'longformer.encoder.layer.4.attention.self.value.weight', 'longformer.encoder.layer.4.attention.self.value.bias', 'longformer.encoder.layer.4.attention.self.query_global.weight', 'longformer.encoder.layer.4.attention.self.query_global.bias', 'longformer.encoder.layer.4.attention.self.key_global.weight', 'longformer.encoder.layer.4.attention.self.key_global.bias', 'longformer.encoder.layer.4.attention.self.value_global.weight', 'longformer.encoder.layer.4.attention.self.value_global.bias', 'longformer.encoder.layer.4.attention.output.dense.weight', 'longformer.encoder.layer.4.attention.output.dense.bias', 'longformer.encoder.layer.4.attention.output.LayerNorm.weight', 'longformer.encoder.layer.4.attention.output.LayerNorm.bias', 'longformer.encoder.layer.4.intermediate.dense.weight', 'longformer.encoder.layer.4.intermediate.dense.bias', 'longformer.encoder.layer.4.output.dense.weight', 'longformer.encoder.layer.4.output.dense.bias', 'longformer.encoder.layer.4.output.LayerNorm.weight', 'longformer.encoder.layer.4.output.LayerNorm.bias', 'longformer.encoder.layer.5.attention.self.query.weight', 'longformer.encoder.layer.5.attention.self.query.bias', 'longformer.encoder.layer.5.attention.self.key.weight', 'longformer.encoder.layer.5.attention.self.key.bias', 'longformer.encoder.layer.5.attention.self.value.weight', 'longformer.encoder.layer.5.attention.self.value.bias', 'longformer.encoder.layer.5.attention.self.query_global.weight', 'longformer.encoder.layer.5.attention.self.query_global.bias', 'longformer.encoder.layer.5.attention.self.key_global.weight', 'longformer.encoder.layer.5.attention.self.key_global.bias', 'longformer.encoder.layer.5.attention.self.value_global.weight', 'longformer.encoder.layer.5.attention.self.value_global.bias', 'longformer.encoder.layer.5.attention.output.dense.weight', 'longformer.encoder.layer.5.attention.output.dense.bias', 'longformer.encoder.layer.5.attention.output.LayerNorm.weight', 'longformer.encoder.layer.5.attention.output.LayerNorm.bias', 'longformer.encoder.layer.5.intermediate.dense.weight', 'longformer.encoder.layer.5.intermediate.dense.bias', 'longformer.encoder.layer.5.output.dense.weight', 'longformer.encoder.layer.5.output.dense.bias', 'longformer.encoder.layer.5.output.LayerNorm.weight', 'longformer.encoder.layer.5.output.LayerNorm.bias', 'longformer.encoder.layer.6.attention.self.query.weight', 'longformer.encoder.layer.6.attention.self.query.bias', 'longformer.encoder.layer.6.attention.self.key.weight', 'longformer.encoder.layer.6.attention.self.key.bias', 'longformer.encoder.layer.6.attention.self.value.weight', 'longformer.encoder.layer.6.attention.self.value.bias', 'longformer.encoder.layer.6.attention.self.query_global.weight', 'longformer.encoder.layer.6.attention.self.query_global.bias', 'longformer.encoder.layer.6.attention.self.key_global.weight', 'longformer.encoder.layer.6.attention.self.key_global.bias', 'longformer.encoder.layer.6.attention.self.value_global.weight', 'longformer.encoder.layer.6.attention.self.value_global.bias', 'longformer.encoder.layer.6.attention.output.dense.weight', 'longformer.encoder.layer.6.attention.output.dense.bias', 'longformer.encoder.layer.6.attention.output.LayerNorm.weight', 'longformer.encoder.layer.6.attention.output.LayerNorm.bias', 'longformer.encoder.layer.6.intermediate.dense.weight', 'longformer.encoder.layer.6.intermediate.dense.bias', 'longformer.encoder.layer.6.output.dense.weight', 'longformer.encoder.layer.6.output.dense.bias', 'longformer.encoder.layer.6.output.LayerNorm.weight', 'longformer.encoder.layer.6.output.LayerNorm.bias', 'longformer.encoder.layer.7.attention.self.query.weight', 'longformer.encoder.layer.7.attention.self.query.bias', 'longformer.encoder.layer.7.attention.self.key.weight', 'longformer.encoder.layer.7.attention.self.key.bias', 'longformer.encoder.layer.7.attention.self.value.weight', 'longformer.encoder.layer.7.attention.self.value.bias', 'longformer.encoder.layer.7.attention.self.query_global.weight', 'longformer.encoder.layer.7.attention.self.query_global.bias', 'longformer.encoder.layer.7.attention.self.key_global.weight', 'longformer.encoder.layer.7.attention.self.key_global.bias', 'longformer.encoder.layer.7.attention.self.value_global.weight', 'longformer.encoder.layer.7.attention.self.value_global.bias', 'longformer.encoder.layer.7.attention.output.dense.weight', 'longformer.encoder.layer.7.attention.output.dense.bias', 'longformer.encoder.layer.7.attention.output.LayerNorm.weight', 'longformer.encoder.layer.7.attention.output.LayerNorm.bias', 'longformer.encoder.layer.7.intermediate.dense.weight', 'longformer.encoder.layer.7.intermediate.dense.bias', 'longformer.encoder.layer.7.output.dense.weight', 'longformer.encoder.layer.7.output.dense.bias', 'longformer.encoder.layer.7.output.LayerNorm.weight', 'longformer.encoder.layer.7.output.LayerNorm.bias', 'longformer.encoder.layer.8.attention.self.query.weight', 'longformer.encoder.layer.8.attention.self.query.bias', 'longformer.encoder.layer.8.attention.self.key.weight', 'longformer.encoder.layer.8.attention.self.key.bias', 'longformer.encoder.layer.8.attention.self.value.weight', 'longformer.encoder.layer.8.attention.self.value.bias', 'longformer.encoder.layer.8.attention.self.query_global.weight', 'longformer.encoder.layer.8.attention.self.query_global.bias', 'longformer.encoder.layer.8.attention.self.key_global.weight', 'longformer.encoder.layer.8.attention.self.key_global.bias', 'longformer.encoder.layer.8.attention.self.value_global.weight', 'longformer.encoder.layer.8.attention.self.value_global.bias', 'longformer.encoder.layer.8.attention.output.dense.weight', 'longformer.encoder.layer.8.attention.output.dense.bias', 'longformer.encoder.layer.8.attention.output.LayerNorm.weight', 'longformer.encoder.layer.8.attention.output.LayerNorm.bias', 'longformer.encoder.layer.8.intermediate.dense.weight', 'longformer.encoder.layer.8.intermediate.dense.bias', 'longformer.encoder.layer.8.output.dense.weight', 'longformer.encoder.layer.8.output.dense.bias', 'longformer.encoder.layer.8.output.LayerNorm.weight', 'longformer.encoder.layer.8.output.LayerNorm.bias', 'longformer.encoder.layer.9.attention.self.query.weight', 'longformer.encoder.layer.9.attention.self.query.bias', 'longformer.encoder.layer.9.attention.self.key.weight', 'longformer.encoder.layer.9.attention.self.key.bias', 'longformer.encoder.layer.9.attention.self.value.weight', 'longformer.encoder.layer.9.attention.self.value.bias', 'longformer.encoder.layer.9.attention.self.query_global.weight', 'longformer.encoder.layer.9.attention.self.query_global.bias', 'longformer.encoder.layer.9.attention.self.key_global.weight', 'longformer.encoder.layer.9.attention.self.key_global.bias', 'longformer.encoder.layer.9.attention.self.value_global.weight', 'longformer.encoder.layer.9.attention.self.value_global.bias', 'longformer.encoder.layer.9.attention.output.dense.weight', 'longformer.encoder.layer.9.attention.output.dense.bias', 'longformer.encoder.layer.9.attention.output.LayerNorm.weight', 'longformer.encoder.layer.9.attention.output.LayerNorm.bias', 'longformer.encoder.layer.9.intermediate.dense.weight', 'longformer.encoder.layer.9.intermediate.dense.bias', 'longformer.encoder.layer.9.output.dense.weight', 'longformer.encoder.layer.9.output.dense.bias', 'longformer.encoder.layer.9.output.LayerNorm.weight', 'longformer.encoder.layer.9.output.LayerNorm.bias', 'longformer.encoder.layer.10.attention.self.query.weight', 'longformer.encoder.layer.10.attention.self.query.bias', 'longformer.encoder.layer.10.attention.self.key.weight', 'longformer.encoder.layer.10.attention.self.key.bias', 'longformer.encoder.layer.10.attention.self.value.weight', 'longformer.encoder.layer.10.attention.self.value.bias', 'longformer.encoder.layer.10.attention.self.query_global.weight', 'longformer.encoder.layer.10.attention.self.query_global.bias', 'longformer.encoder.layer.10.attention.self.key_global.weight', 'longformer.encoder.layer.10.attention.self.key_global.bias', 'longformer.encoder.layer.10.attention.self.value_global.weight', 'longformer.encoder.layer.10.attention.self.value_global.bias', 'longformer.encoder.layer.10.attention.output.dense.weight', 'longformer.encoder.layer.10.attention.output.dense.bias', 'longformer.encoder.layer.10.attention.output.LayerNorm.weight', 'longformer.encoder.layer.10.attention.output.LayerNorm.bias', 'longformer.encoder.layer.10.intermediate.dense.weight', 'longformer.encoder.layer.10.intermediate.dense.bias', 'longformer.encoder.layer.10.output.dense.weight', 'longformer.encoder.layer.10.output.dense.bias', 'longformer.encoder.layer.10.output.LayerNorm.weight', 'longformer.encoder.layer.10.output.LayerNorm.bias', 'longformer.encoder.layer.11.attention.self.query.weight', 'longformer.encoder.layer.11.attention.self.query.bias', 'longformer.encoder.layer.11.attention.self.key.weight', 'longformer.encoder.layer.11.attention.self.key.bias', 'longformer.encoder.layer.11.attention.self.value.weight', 'longformer.encoder.layer.11.attention.self.value.bias', 'longformer.encoder.layer.11.attention.self.query_global.weight', 'longformer.encoder.layer.11.attention.self.query_global.bias', 'longformer.encoder.layer.11.attention.self.key_global.weight', 'longformer.encoder.layer.11.attention.self.key_global.bias', 'longformer.encoder.layer.11.attention.self.value_global.weight', 'longformer.encoder.layer.11.attention.self.value_global.bias', 'longformer.encoder.layer.11.attention.output.dense.weight', 'longformer.encoder.layer.11.attention.output.dense.bias', 'longformer.encoder.layer.11.attention.output.LayerNorm.weight', 'longformer.encoder.layer.11.attention.output.LayerNorm.bias', 'longformer.encoder.layer.11.intermediate.dense.weight', 'longformer.encoder.layer.11.intermediate.dense.bias', 'longformer.encoder.layer.11.output.dense.weight', 'longformer.encoder.layer.11.output.dense.bias', 'longformer.encoder.layer.11.output.LayerNorm.weight', 'longformer.encoder.layer.11.output.LayerNorm.bias', 'longformer.pooler.dense.weight', 'longformer.pooler.dense.bias', 'lm_head.bias', 'lm_head.dense.weight', 'lm_head.dense.bias', 'lm_head.layer_norm.weight', 'lm_head.layer_norm.bias', 'lm_head.decoder.weight']
2020-06-21 18:00:58,818|transformers.configuration_encoder_decoder|INFO> Set `config.is_decoder=True` for decoder_config
```

## Motivation

I'd like to use the longformer as an `EncoderDecoderModel` and these changes are required for the `EncoderDecoderModel` to work properly. My end goal is to use a encoder-decoder architecture for long documents. The better solution appears to be #4406. However, I still wonder if my changes are correct.

## Your contribution

My changes are in the `longformer-encoder_hidden_states` branch of [HHousen/transformers](https://github.com/HHousen/transformers/tree/longformer-encoder_hidden_states). Link to modified file: [modeling_longformer.py](https://github.com/HHousen/transformers/blob/longformer-encoder_hidden_states/src/transformers/modeling_longformer.py).


[Tokenizer] batch_encode_plus method cannot encode List[Tuple[str]] with is_pretokenized=True

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

## Information

Model I am using: BERT

Language I am using the model on: English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Load a tokenizer
2. Create a generic List[Tuple[List[int], List[int]]], for example [([2023, 2573], [2023, 2573, 2205])]
3. Encode the list using the method batch_encode_plus with is_tokenized=True to encode the pairs together
4. Error
```python
from transformers import BertTokenizer
import torch
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
text = [('This works','this works too')]
print(tokenizer.batch_encode_plus(text, add_special_tokens=False)['input_ids'])  # This works
input_ids = [([2023, 2573], [2023, 2573, 2205])]
tokenizer.batch_encode_plus(input_ids, is_pretokenized=True) # This raises error

/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py in get_input_ids(text)
   1715             else:
   1716                 raise ValueError(
-> 1717                     "Input is not valid. Should be a string, a list/tuple of strings or a list/tuple of integers."
   1718                 )
   1719

ValueError: Input is not valid. Should be a string, a list/tuple of strings or a list/tuple of integers.
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

batch_encode_plus would be able to encode List[Tuple[List[int], List[int]]], as described in the function description, hence, the example's input_ids would be:
`[[101, 2023, 2573, 102, 2023, 2573, 2205, 102]]`

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0+cu101 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: probably not, but irrelevant
- Using distributed or parallel set-up in script?: probably not


Summarization Examples: Support label_smoothed_cross_entropy

Examples

Which is related to examples in general

Help wanted

Extra attention is needed, help appreciated

This seems to be used by nearly all papers, but the HF implementation in `examples/summarization/finetune.py` uses the standard cross entropy loss.

Ideally, we could expose a command line arg to allow this choice to be toggled, like:

```python
parser.add_argument('--loss', type=str, choices=['cross_entropy', 'label_smoothed_cross_entropy'], default='cross_entropy')
```


results for wikitext-2 clm using GPT-2 differ between paper and example code

Hi,

The example code [here](https://github.com/huggingface/transformers/tree/master/examples/language-modeling) to train causal LM on wikitext-2 uses a gpt2 base version (117M parameters). This code claims and also yields a perplexity of ~20 on the test data.

As you can see, the example script uses ```gpt``` as the ```model_name_or_path``` param which is actually the 117M model.
```python
python run_language_modeling.py \
    --output_dir=output \
    --model_type=gpt2 \
    --model_name_or_path=gpt2 \
    --do_train \
    --train_data_file=$TRAIN_FILE \
    --do_eval \
    --eval_data_file=$TEST_FILE
```

![image](https://user-images.githubusercontent.com/6441313/85228483-3c6d8980-b401-11ea-86f7-7f18e16b83a7.png)

However, the GPT-2 [paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf) [page 5, top table] shows that they got a perplexity of 29.41 with the said model.

![image](https://user-images.githubusercontent.com/6441313/85228388-d254e480-b400-11ea-87ac-29ab9064e794.png)

Did I miss something? Can you please explain me the mismatch?

Thanks in advance.


Cannot import AutoModelForSeq2SeqLM

# 🐛 Bug

Has the `AutoModelForSeq2SeqLM` class changed?

I am trying to run transformer examples, basically the [token-classification](https://github.com/huggingface/transformers/blob/master/examples/token-classification/run_pl.sh) with pytorch-lightning, which calls [AutoModelForSeq2SeqLM](https://github.com/huggingface/transformers/blob/master/examples/lightning_base.py#L18). However, I am getting an import error. See below.

## Information

Model I am using (Bert, XLNet ...):

`bert-base-multilingual-cased`

Language I am using the model on (English, Chinese ...):

`English`

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```
cd transformers/examples/token-classification
./run_pl.sh
```

```
Traceback (most recent call last):
  File "run_pl_ner.py", line 12, in <module>
    from lightning_base import BaseTransformer, add_generic_args, generic_train
  File "/transformers/examples/lightning_base.py", line 12, in <module>
    from transformers import (
ImportError: cannot import name 'AutoModelForSeq2SeqLM' from 'transformers' (/anaconda3/envs/transformers/lib/python3.8/site-packages/transformers/__init__.py)
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

Reproduce the example.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: Linux
- Python version: Python 3.8.3
- PyTorch version (GPU?): 1.5.1
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?:


Create README.md

model card

Related to pretrained model cards

Creating README.md file for model on Community contribution.


output generate scores per hypothesis/token

# 🚀 Feature request
Thanks for doing such an awesome work.
i'm interested in the hypothesis score when running generate.
This could be done per hypothesis, or preferably per token in the hypothesis.

## Motivation

The motivation is to gain confidence for my generated text,

I suggest:
1.  adding flag in modeling_utils.py to  generate to return_scores
2. in _generate_beam_search :
  if return_scores:
   return also
 for _generate_beam_search:

`        best_scores = []

        # retrieve best hypotheses
        for i, hypotheses in enumerate(generated_hyps):
            sorted_hyps = sorted(hypotheses.beams, key=lambda x: x[0])
            for j in range(output_num_return_sequences_per_batch):
                effective_batch_idx = output_num_return_sequences_per_batch * i + j
                hyp_score, best_hyp = sorted_hyps.pop()
                sent_lengths[effective_batch_idx] = len(best_hyp)
                best.append(best_hyp)
                best_scores.append(hyp_score)

        # shorter batches are filled with pad_token
        if sent_lengths.min().item() != sent_lengths.max().item():
            assert pad_token_id is not None, "`Pad_token_id` has to be defined"
            sent_max_len = min(sent_lengths.max().item() + 1, max_length)
            decoded = input_ids.new(output_batch_size, sent_max_len).fill_(pad_token_id)

            # fill with hypothesis and eos_token_id if necessary
            for i, hypo in enumerate(best):
                decoded[i, : sent_lengths[i]] = hypo
                if sent_lengths[i] < max_length:
                    decoded[i, sent_lengths[i]] = eos_token_id
        else:
            # none of the hypotheses have an eos_token
            assert (len(hypo) == max_length for hypo in best)
            decoded = torch.stack(best).type(torch.long).to(next(self.parameters()).device)

        if return_scores:
            return decoded, best_scores  `

for _generate_no_beam_search:
`         output_score = 0

        while cur_len < max_length:
            model_inputs = self.prepare_inputs_for_generation(
                input_ids, past=past, attention_mask=attention_mask, use_cache=use_cache, **model_specific_kwargs
            )

            outputs = self(**model_inputs)
            next_token_logits = outputs[0][:, -1, :]

            # if model has past, then set the past variable to speed up decoding
            if self._use_cache(outputs, use_cache):
                past = outputs[1]

            # repetition penalty from CTRL paper (https://arxiv.org/abs/1909.05858)
            if repetition_penalty != 1.0:
                self.enforce_repetition_penalty_(next_token_logits, batch_size, 1, input_ids, repetition_penalty)

            if no_repeat_ngram_size > 0:
                # calculate a list of banned tokens to prevent repetitively generating the same ngrams
                # from fairseq: https://github.com/pytorch/fairseq/blob/a07cb6f40480928c9e0548b737aadd36ee66ac76/fairseq/sequence_generator.py#L345
                banned_tokens = calc_banned_ngram_tokens(input_ids, batch_size, no_repeat_ngram_size, cur_len)
                for batch_idx in range(batch_size):
                    next_token_logits[batch_idx, banned_tokens[batch_idx]] = -float("inf")

            if bad_words_ids is not None:
                # calculate a list of banned tokens according to bad words
                banned_tokens = calc_banned_bad_words_ids(input_ids, bad_words_ids)

                for batch_idx in range(batch_size):
                    next_token_logits[batch_idx, banned_tokens[batch_idx]] = -float("inf")

            # set eos token prob to zero if min_length is not reached
            if eos_token_id is not None and cur_len < min_length:
                next_token_logits[:, eos_token_id] = -float("inf")

            if do_sample:
                # Temperature (higher temperature => more likely to sample low probability tokens)
                if temperature != 1.0:
                    next_token_logits = next_token_logits / temperature
                # Top-p/top-k filtering
                next_token_logits = top_k_top_p_filtering(next_token_logits, top_k=top_k, top_p=top_p)
                # Sample
                probs = F.softmax(next_token_logits, dim=-1)
                next_token = torch.multinomial(probs, num_samples=1).squeeze(1)
            else:
                # Greedy decoding
                next_token = torch.argmax(next_token_logits, dim=-1)
            from IPython import embed; embed()
            next_score = torch.gather(next_token_logits, -1, next_tokens)  # (batch_size, num_beams * 2)
            # update generations and finished sentences
            if eos_token_id is not None:
                # pad finished sentences if eos_token_id exist
                tokens_to_add = next_token * unfinished_sents + (pad_token_id) * (1 - unfinished_sents)
            else:
                tokens_to_add = next_token

            # add token and increase length by one
            input_ids = torch.cat([input_ids, tokens_to_add.unsqueeze(-1)], dim=-1)
            output_score+=next_score
            cur_len = cur_len + 1

            if eos_token_id is not None:
                eos_in_sents = tokens_to_add == eos_token_id
                # if sentence is unfinished and the token to add is eos, sent_lengths is filled with current length
                is_sents_unfinished_and_token_to_add_is_eos = unfinished_sents.mul(eos_in_sents.long()).bool()
                sent_lengths.masked_fill_(is_sents_unfinished_and_token_to_add_is_eos, cur_len)
                # unfinished_sents is set to zero if eos in sentence
                unfinished_sents.mul_((~eos_in_sents).long())

            # stop when there is a </s> in each sentence, or if we exceed the maximul length
            if unfinished_sents.max() == 0:
                break

            # extend attention_mask for new generated input if only decoder
            if self.config.is_encoder_decoder is False:
                attention_mask = torch.cat(
                    [attention_mask, attention_mask.new_ones((attention_mask.shape[0], 1))], dim=-1
                )

        # if there are different sentences lengths in the batch, some batches have to be padded
        if sent_lengths.min().item() != sent_lengths.max().item():
            assert pad_token_id is not None, "`Pad_token_id` has to be defined if batches have different lengths"
            # finished sents are filled with pad_token
            decoded = input_ids.new(batch_size, sent_lengths.max().item()).fill_(pad_token_id)
        else:
            decoded = input_ids

        for hypo_idx, hypo in enumerate(input_ids):
            decoded[hypo_idx, : sent_lengths[hypo_idx]] = hypo[: sent_lengths[hypo_idx]]

        if return_scores:
              return decoded,  output_score
        return decoded `

In the next step we could save the score per token to allow the user to decide where he wants to truncate the generated text as function of confidence


Why doesn't stride in squad_convert_example_to_features‘s encode_plus set to doc_stride?

Core: Tokenization

Internals of the library; Tokenization.

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


How to  save the created embedding of the text corpus.

After converting the text corpus into word embedding how can we save the embeddings into a file. It is not practical and feasible to create word embedding each and every time. Is there ant work around to save the embedding.


Keras model created from individual Bert Layers has weights not shown in trainable_weights nor non_trainable_weights. model.summary() / utils.plot_model shows those weights as part of graph though

# 🐛 Bug

## Information

I created a model which takes 2 layers from from a bert model, and creates a mini-bert model.

However, not all the weights/layers from the component layers are in this model. But These weights are shown was connected in `model.summary()` and in

```
tf.keras.utils.plot_model(
    model, to_file='model.png', show_shapes=False, show_layer_names=True,
    rankdir='TB', expand_nested=False, dpi=96
)
```

Model I am using (Bert, XLNet ...): bert-base-uncased

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ X] my own modified scripts: (give details below)

## To reproduce

Steps to reproduce the behavior:

Here is a Colabnotebook with a minimal example that reproduced the issue.

https://colab.research.google.com/drive/1n3_XNhdgH6Qo7GT-M570lIKWAoU3TML5?usp=sharing

And here is the code

```
!pip install transformers --q
%tensorflow_version 2.x

from transformers import TFBertModel, AutoModel, TFRobertaModel, AutoTokenizer
import tensorflow as tf
import tensorflow_addons as tfa

tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)

from tensorflow import keras
from tensorflow.keras import layers
from copy import deepcopy

logger = tf.get_logger()
logger.info(tf.__version__)

def get_mini_models():
    tempModel = TFRobertaModel.from_pretrained('bert-base-uncased', from_pt=True)

    layer9 = deepcopy(tempModel.layers[0].encoder.layer[8])
    layer10 = deepcopy(tempModel.layers[0].encoder.layer[9])

    inputHiddenVals = tf.keras.Input(shape=[None, None], dtype=tf.float32, name='input_Q',
                                    batch_size=None)

    hidden1 = layer9((inputHiddenVals, None, None))
    hidden2 = layer10((hidden1[0], None, None))
    modelNew = tf.keras.Model(inputs=inputHiddenVals, outputs=hidden2)

    del tempModel

    return modelNew

@tf.function
def loss_fn(_, probs):
    bs = tf.shape(probs)[0]
    labels = tf.eye(bs, bs)
    return tf.losses.categorical_crossentropy(labels,
                                              probs,
                                              from_logits=True)

model = get_mini_models()
model.compile(loss=loss_fn,
                optimizer=tfa.optimizers.AdamW(weight_decay=1e-4, learning_rate=1e-5,
                                                epsilon=1e-06))

# Get model and layers directly to compare
tempModel = TFRobertaModel.from_pretrained('bert-base-uncased', from_pt=True)
layer9 = deepcopy(tempModel.layers[0].encoder.layer[8])
layer10 = deepcopy(tempModel.layers[0].encoder.layer[9])

# Only one layer, and that layer also has missing weights.
for i, var in enumerate(model.weights):
    print(model.weights[i].name)

# Full weights for one layer
for i, var in enumerate(layer9.weights):
    print(layer9.weights[i].name)

# Test what correct output should be

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
inputt = tokenizer.encode('This is a sentence', return_tensors='tf')
outt = tempModel(inputt)[0]

# Test model output. Not the same.

model(outt)

# Model summary somehow lists the weights
model.summary()

# Model diagram shows the correct connections between all the layers.

tf.keras.utils.plot_model(
    model, to_file='model.png', show_shapes=False, show_layer_names=True,
    rankdir='TB', expand_nested=False, dpi=96
)

```

Edit: I also tried making the layers from scratch, and setting the weights directly, same result. Here's a colab notebook that does htis. https://colab.research.google.com/drive/1EC_fObSp9lUsj_PFaYgFtRI93ErPYmU9?usp=sharing

## Expected behavior

The model should contain all the weights from the component layers, and the output of the model should be the same as executing the layers in the same way outside the model.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0+cu101 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?:no
- Using distributed or parallel set-up in script?: no


Create README.md

model card

Related to pretrained model cards


Transformer pipeline loading model and tokenizer on every prediction request

# 🐛 Bug

## Information
I'm trying to build API which accept question and context, response should be the answer. I'm trying to use Transformer Question Answering pipeline task with BERT model.
I'm initializing the model and pipeline object on __init__ method and trying to use pipeline object in another method, so that on server startup the model is loaded to memory and prediction will be fast.
But on every prediction request, __init__method is called, when i debugged the flow, i think its because of feature calculation is happening using Pool threads.
https://github.com/huggingface/transformers/blob/68e19f1c228c92d5d800533f558faff24b57127a/src/transformers/data/processors/squad.py#L318

Can you please to resolve the issue?

Model I am using (Bert, XLNet ...): Bert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] my own modified scripts: Own script

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: SQUaD

## To reproduce

Steps to reproduce the behavior:

Source Code:
```
from transformers import BertTokenizer, BertForQuestionAnswering
import torch
import time
from transformers import pipeline

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using {device} for QNA Model prediction")

class QNAModel:
    def __init__(self):
        start_time = time.time()
        self.tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
        self.model = BertForQuestionAnswering.from_pretrained(
            'bert-large-uncased-whole-word-masking-finetuned-squad')
        self.pipeline = pipeline("question-answering", model=self.model, tokenizer=self.tokenizer)
        print("--- %s seconds model load time ---" % (time.time() - start_time))

    def get_answers(self, passages, question):
        start_time = time.time()
        answers = []
        for passage in passages:
            context = passage["passage"]
            print(self.pipeline(question=question, context=context))
        print("--- %s seconds Total QNA model prediction---" % (time.time() - start_time))
        return answers
```

Testing
```
qna_model = QNAModel()
qna_model.get_answers(passages1, question1)
qna_model.get_answers(passages1, question2)
```

Logs,
```
Using cpu for QNA Model prediction
--- 16.12714695930481 seconds model load time ---
convert squad examples to features: 100%|██████████| 1/1 [00:00<00:00, 58.58it/s]
add example index and unique id: 100%|██████████| 1/1 [00:00<?, ?it/s]
{'score': 0.39260570705747, 'start': 364, 'end': 380, 'answer': '<answer1>'}
Using cpu for QNA Model prediction
--- 14.559933185577393 seconds model load time ---
convert squad examples to features: 100%|██████████| 1/1 [00:00<00:00, 187.58it/s]
add example index and unique id: 100%|██████████| 1/1 [00:00<?, ?it/s]
{'score': 0.2136769815853059, 'start': 0, 'end': 84, 'answer': '<answer2>'}
--- 51.28193521499634 seconds Total QNA model prediction---
```

## Expected behavior
Model should load once so that prediction is fast

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.7.0
- Platform: Windows
- Python version: 3.7
- PyTorch version (GPU?): 1.5.0+cpu
- Tensorflow version (GPU?): NA
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Fix PABEE's result table


[examples] fixes arguments for summarization finetune scripts

This PR fixes arguments in bash scripts for finetuning bart/bart_tiny/t5 models in summarization examples.

After changes that were merged in #4951 i noticed some small problems. That includes:
* missing default arguments for `finetune.py` (--data_dir, output_dir)
* invalid argument name for number of gpu to use (`n_gpu` instead of `gpus`)
* argument `model_type` that is not present in list of arguments and crashes the script


BertForMaskedLM "labels" is an unexpected keyword

# 🐛 Bug

## Information

The official BertForMaskedLM example: https://huggingface.co/transformers/model_doc/bert.html#bertformaskedlm
 has a bug. When running: outputs = model(input_ids, labels=input_ids), it alerts: TypeError: forward() got an unexpected keyword argument 'labels'

Model I am using (Bert, XLNet ...):
BERT

Language I am using the model on (English, Chinese ...):
English

The problem arises when using:
* [ ] the official example scripts: (give details below)
from transformers import BertTokenizer, BertForMaskedLM
import torch

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForMaskedLM.from_pretrained('bert-base-uncased')

input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0)  # Batch size 1
outputs = model(input_ids, labels=input_ids)

* [ ] my own modified scripts: (give details below)
The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. https://huggingface.co/transformers/model_doc/bert.html#bertformaskedlm
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
should work with labels??
<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: Linux
- Python version: 3.7
- PyTorch version (GPU?): 1.5.1
- Tensorflow version (GPU?): 2.2.0
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


new tokenizer backend breaks old code

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): any model

Language I am using the model on (English, Chinese ...): english

The problem arises when using:
* [x] the official example scripts: (give details below) QA script from official repo

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name) Squad 1 and 2

## To reproduce

Steps to reproduce the behavior:

Just try to run the example script for QA task

The error output is too large to copy in here.
The new tokenizer backend is not backward compatible. The truncation is set to false by default.
With the new tokenizer backend the truncation need to get initialized with True value, but in example or old codes there is often no possibility to set it while calling.
I think the feature conversion code is not updated to the new backend, right ?

## Expected behavior
It should train the model correctly

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: any
- Python version: 3.7-3.8
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


Lite transformer

# 🌟 New model addition

## Model description
Transformer has become ubiquitous in natural language processing (e.g., machine translation, question answering); however, it requires enormous amount of computations to achieve high performance, which makes it not suitable for mobile applications that are tightly constrained by the hardware resources and battery. In this paper, we present an efficient mobile NLP architecture, Lite Transformer to facilitate deploying mobile NLP applications on edge devices. The key primitive is the Long-Short Range Attention (LSRA), where one group of heads specializes in the local context modeling (by convolution) while another group specializes in the long-distance relationship modeling (by attention). Such specialization brings consistent improvement over the vanilla transformer on three well-established language tasks: machine translation, abstractive summarization, and language modeling. Under constrained resources (500M/100M MACs), Lite Transformer outperforms transformer on WMT'14 English-French by 1.2/1.7 BLEU, respectively. Lite Transformer reduces the computation of transformer base model by 2.5x with 0.3 BLEU score degradation. Combining with pruning and quantization, we further compressed the model size of Lite Transformer by 18.2x. For language modeling, Lite Transformer achieves 1.8 lower perplexity than the transformer at around 500M MACs. Notably, Lite Transformer outperforms the AutoML-based Evolved Transformer by 0.5 higher BLEU for the mobile NLP setting without the costly architecture search that requires more than 250 GPU years. Code has been made available at https://github.com/mit-han-lab/lite-transformer
<!-- Important information -->

## Open source status

* [x] the model implementation is available: (give details)https://github.com/mit-han-lab/lite-transformer
* [x] the model weights are available: (give details) linked in readme of GitHub repo
* [x] who are the authors: (mention them, if possible by @gh-username)Zhanghao Wu • Zhijian Liu • Ji Lin • Yujun Lin • Song Han


Create README.md

model card

Related to pretrained model cards

electra_large_discriminator_squad2_512 LM for Question Answering


TFBertForSequenceClassification: TypeError: call() got an unexpected keyword argument 'labels'

# 🐛 Bug

## Information

Model I am using TFBertForSequenceClassification

Language I am using the model on: English

The problem arises when using:
* [X] the official example scripts: (give details below)

The tasks I am working on is:
* [X] my own task or dataset: (give details below)
To classify text and learn the model and package.

## To reproduce

Steps to reproduce the behavior:

1. pip install transformers (currently 2.11.0)
2. run default code on website: https://huggingface.co/transformers/model_doc/bert.html#tfbertforsequenceclassification
3. I tried to follow this: https://github.com/huggingface/transformers/issues/4848; and the issue remains the same.

```
import tensorflow as tf
from transformers import BertTokenizer, TFBertForSequenceClassification

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased')
input_ids = tf.constant(tokenizer.encode("Hello, my dog is cute"))[None, :]  # Batch size 1
labels = tf.reshape(tf.constant(1), (-1, 1)) # Batch size 1
outputs = model(input_ids, labels=labels)
loss, logits = outputs[:2]
```

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.7.7
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


[MobileBert] fix dropout

This PR fixes dropout in class TFMobileBertOutput.


Update glossary

Update the glossary to the new tokenizer API. I also added a first section about general terms that might confuse a beginner (which anyone is welcome to expand when they see a term that's not necessarily obvious and that we use a lot) and proper references to the subsections about the model inputs to make sure none of the links break.

There was a bad indent introduced since my cleanup of sphinx warnings, so fixed that in passing.


Typo in Reformer model card

Related to pretrained model cards


Upgrade examples to pl=0.8.1

This upgrades to the latest PL and follows the pl docs to avoid Deprecation warnings.

### Known Issue
`trainer.test` fails in multigpu, this is not a new issue, but here we add a test that will pass when the bug is fixed upstream.

Failure discussed [here](https://github.com/PyTorchLightning/pytorch-lightning/issues/2267) can be invoked on a multigpu machine with

```bash
pytest examples/summarization

```
This failure is not new, it also existed in previous code.
Traceback:
```bash
examples/summarization/finetune.py:322: in main
    trainer.test(model)  # this breaks in DDP, known lightning issue. See evaluate_ch
eckpoint to recover metrics.
../.conda/envs/nb/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py:11
55: in test
    self.barrier('test_setup')
../.conda/envs/nb/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py:12
60: in barrier
    torch_distrib.barrier()
../.conda/envs/nb/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py:1
484: in barrier
    _check_default_pg()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _check_default_pg():
        """
        Helper that checks if the default ProcessGroup has been initialized, with
        assertion

        """
        assert _default_pg is not None, \
>           "Default process group is not initialized"
E       AssertionError: Default process group is not initialized

../.conda/envs/nb/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py:1
87: AssertionError
```


Quick tour

This PR replaces the old quickstart guide by splitting it in two:
- a quick tour that is before the installation page,
- the philosophy page, after the installation.

It also renames Usage to Task summary since it's really what it is, and renames the files usage to task_summary, summary to model_summary.

The quick tour tries to go over all the API introduced in Transformers at a high level, pointing out tutorials (existing or upcoming) for each of them.


Scoring each word from the sentence using Pretrained LM

# ❓ Questions & Help

## Details
I am interested in scoring each word  from the sentence using the LM. Detect how likely possible is the word in the given context by scoring it using the LM.

Is there a way out for this ?


Passing in own embeddings for image input

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

I am trying to use your pretrained BERT/RoBERTa models to write multimodal classifiers such as the bitransformer (see diagram at top of page 2):
https://arxiv.org/pdf/1909.02950.pdf

and Visual BERT:
https://arxiv.org/pdf/1908.03557.pdf

It would be nice to be able to change the input layer of the transformer models to handle custom input embeddings rather than just input_ids. If there is already a way to do this, it would be great to get some instructions on how. I have tried printing the layers of the pretrained RoBERTa model, but it only gives:
`<transformers.modeling_tf_roberta.TFRobertaMainLayer object at 0x7f79fc04dcf8>`

Edit: I actually see that there is a MMBT model mentioned in the documentation, but I cannot find the pre-trained model or any examples with it. Please help. Thank you.

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

I would like to be able to use your pretrained models to handle image embeddings as well. I am working on a project with this multimodal dataset:
https://gombru.github.io/2019/10/09/MMHS/

Any help would be greatly appreciated. Thanks!


T5 special tokens not mapped to unique indices in vocabulary

Core: Tokenization

Internals of the library; Tokenization.

The docs recommend adding the special eos_token `<\s>` to the end of each string when encoding/decoding with `T5Tokenizer`. However, this (and the other special tokens e.g. `unk_token`, `pad_token` aren't assigned unique ids in the lookup vocabulary (they are mapped to `{0,1,2}`, which are indices for other common words in the vocab). In practice, I find my model fails to properly produce the `eos_token` since it is associated with blank spaces, so the model produces run-ons during generation

## To reproduce
```
>>> from transformers import T5Tokenizer
>>> tokenizer = T5Tokenizer.from_pretrained('t5-base')
>>> tokenizer.pad_token
'<pad>'
>>> tokenizer.pad_token_id
0
>>> tokenizer.eos_token
'</s>'
>>> tokenizer.eos_token_id
1
>>> tokenizer.unk_token
'<unk>'
>>> tokenizer.unk_token_id
2
```
```
>>> tokenizer.decode([0])
''
>>> tokenizer.decode([1])
''
>>> tokenizer.decode([2])
' ⁇ '
```
## Expected behavior

```
>>> tokenizer.decode([0])
'<pad>'
>>> tokenizer.decode([1])
'</s>'
>>> tokenizer.decode([2])
'<unk>'
```

## Environment info
- `transformers` version: 2.9.1


[bart-mnli] Fix class flipping bug

Previously, `eval_mnli/acc` was 0.35, now .9. Same issue as other models ported from fairseq.
New results of Victor's command.
```bash
wandb:   eval_mnli-mm/acc 0.9001220504475184
wandb:           _runtime 274.7172155380249
wandb:          eval_loss 0.3496225342093929
wandb:      eval_mnli/acc 0.9011716760061131
wandb:         _timestamp 1592585765.2334569
```


[pl_examples] deprecate BaseTransformer.is_logger

- proc_rank is deprecated in new PL
- updates pl to 0.8.1

@williamFalcon suggestions more than welcome!

- The test that is failing is `test_bdc_multigpu`


`facebook/bart-large-mnli` - random accuracy on MNLI

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): BART

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: MNLI
* [ ] my own task or dataset: (give details below)

## To reproduce

There might be something wrong with the `facebook/bart-large-mnli` checkpoint (https://huggingface.co/facebook/bart-large-mnli). I can only get a random accuracy (0.35) on MNLI:
`python examples/text-classification/run_glue.py --data_dir <GLUE DATA DIR/MNLI> --task MNLI --model_name_or_path facebook/bart-large-mnli --output_dir ./dbg/ --max_seq_length 128 --overwrite_cache --do_eval`

I checked and the checkpoint has a classification head (`'classification_head.dense.bias', 'classification_head.out_proj.weight', 'classification_head.out_proj.bias'`) i.e. the classification head is not initialized randomly.

cc @sshleifer as discussed


Fix in Reformer Config documentation

Fixed a variable name in the Reformer Config docstring (`lsh_chunk_length` -> `lsh_attn_chunk_length`)


xlm-mlm-17-1280: after run model to get embeddings shape 20000

i want to get embeddings for :ru: text by `xlm-mlm-17-1280`
at the end get embeddings with shape 2k

example code (using the last version transformers & torch on ubuntu):
```
from transformers import AutoTokenizer, AutoModelWithLMHead

xlm_mlm = 'xlm-mlm-17-1280'
tokenizer_xlm_mlm = AutoTokenizer.from_pretrained(xlm_mlm)
model_xlm_mlm =  AutoModelWithLMHead.from_pretrained(xlm_mlm)

input_ids = torch.tensor([tokenizer_xlm_mlm.encode(my_input_text)]) # batch size of 1
print(f'{input_ids.shape=}')
# input_ids.shape=torch.Size([1, 373])

lang_id_ru = tokenizer_xlm_mlm.lang2id['ru']

langs_ru = torch.tensor([lang_id_ru] * input_ids.shape[1])  # torch.tensor([0, 0, 0, ..., 0])
print(f'{langs_ru.shape=}')
# langs_ru.shape=torch.Size([373])

langs_ru = langs_ru.view(1, -1) # is now of shape [1, sequence_length]
print(f'{langs_ru.shape=}')
# langs_ru.shape=torch.Size([1, 373])

outputs = model_xlm_mlm(input_ids, langs=langs_ru)
outputs[0].shape
# torch.Size([1, 373, 200000])
```

so it's a bug or my bad?


Transformer-XL tokenizer cannot properly tokenize brackets

Internals of the library; Tokenization.

# 🐛 Bug

## Information
The `TransfoXLTokenizer` is not able to tokenize words with surrounding brackets correctly. I compared it with the `BertTokenizer` from `bert-base-uncased` which gives the expected result. Example text is: `"Hello (bracket)"`

Model I am using: **Transformer-XL**

Language I am using the model on: **English**

The problem arises when using:
* [x] my own modified scripts

## To reproduce

Steps to reproduce the behavior:
```python
from transformers import BertTokenizer, TransfoXLTokenizer

bert = BertTokenizer.from_pretrained('bert-base-uncased')
transfoxl = TransfoXLTokenizer.from_pretrained('transfo-xl-wt103')

def test_bracket(tokenizer):
    enc = tokenizer.encode("Hello (bracket)")
    dec = tokenizer.decode(enc)
    print(f"ORG: Hello (bracket)\nENC: {enc}\nDEC: {dec}")
```
Results:
`test_bracket(bert)` gives the following output:
```
ORG: Hello (bracket)
ENC: [101, 7592, 1006, 21605, 1007, 102]
DEC: [CLS] hello ( bracket ) [SEP]
```

`test_bracket(transfoxl)` gives the following output:
```
ORG: Hello (bracket)
ENC: [14049, 24]
DEC: Hello <unk>
```
If the parameter `add_space_before_punct_symbol=True` is passed, then the result is:
```
ORG: Hello (bracket)
ENC: [14049, 24, 21]
DEC: Hello <unk> )
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The `TransfoXLTokenizer` should detect the punctuation symbols, e.g. `(`, separately and thus give the same result as the `BertTokenizer` (except the special tokens of course): `hello ( bracket )`

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.6.10
- PyTorch version (GPU?): 1.4.0 (False)
- Tensorflow version (GPU?): 2.1.0 (False)


src/transformers/trainer.py relies on path to infer global training steps, skips training for glue example

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): bert-base-uncased that has been retrained stored somewhere else, with a timestamp in the path

Language I am using the model on (English, Chinese ...):EN

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name) MRPC
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. store a pretrained model with a different name, say /funny/path/2020-06-19_120000/retrained ,
with the tokenizer and stuff inside.
2. Run the glue example run_glue on that path with --do_train, --do_eval
3. The training will be skipped because there's [this](https://github.com/huggingface/transformers/blob/84be482f6698fac822a5113735f2242c6d3abc76/src/transformers/trainer.py#L431) funny line in the trainer that will bump the global steps because of the parsing of the date..

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
I would not expect the trainer to rely on an arbitrary internal convention for naming checkpoints or any kind of path to yield a spurious global step count.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: Linux-5.4.0-29-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.5
- PyTorch version (GPU?): 1.4.0 (False)
- Tensorflow version (GPU?):
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


What does adjust_logits_during_generation (formerly prepare_logits_for_generation) do?

Discussion

Discussion on a topic (keep it focused or open a new issue though)

https://github.com/huggingface/transformers/blob/49c5202522bdaf66e45df505b3a3c566e56134c3/src/transformers/modeling_utils.py#L794

Hi. Anyone can share why there is this method here?


How to find and use fine-tuned model for GLUE-CoLA?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:

I am trying to use fine-tuned model for CoLA from GLUE. Is there a quick guide? Thanks


fix bart doc

fix bart doc
change the model name in the example from "bart-large-cnn" to "facebook/bart-large-cnn"


Update BERT-of-Theseus model card to avoid confusion

model card

Related to pretrained model cards

@VictorSanh said this checkpoint cannot be evaluated directly on MNLI. Indeed this model is for intermediate task transfer (in BERT-of-Theseus [arxiv version](https://arxiv.org/abs/2002.02925), it's called a "general-purpose" model) so it doesn't (and shouldn't) contain a classification head. This note helps prevent confusion.


added subtitle for recent contributors in readme


Add mbart-large-cc25, support translation finetuning

translation

machine translation utilities and models

TODO:

- [x] fix config (max_length=100)
- [x] split test_modeling_mbart.py to new file
- [x] decoder_input_ids should start with decoder_start_token_id
- [x] AutoTokenizer
- [x] test finetuning. Requires adding `--src_lang` and `--tgt_lang` clargs.
- [x] Documentation
- [ ] Model Card
- [x] fix mbart tokenizer tests
- [x] tokenizer docs


Pin `sphinx-rtd-theme`


Cached feature files - Naming introduces confusion/model mixing

# 🐛 Bug

The problem arises when using:
* [X] the official example scripts: glue.py
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: any example of GLUE
* [ ] my own task or dataset: (give details below)

In `run_glue.py` (and possibly in other run_* scripts interfaced with the new Trainer), the cache naming for the pre-processed features introduces confusion between models.

https://github.com/huggingface/transformers/blob/3d3e605affb792b78c918aac48f6bc82cfbf7e3e/src/transformers/data/datasets/glue.py#L87

`tokenizer.__class__.__name__` is `BertTokenizer`, `DistilBertTokenizer`, etc. which removes the information of which tokenization is used. For instance, `bert-base-uncased` and `bert-base-cased` would end up with the same file name while being potentially two really different files.

In scripts not interfaced yet with the Trainer, the `model_name_or_path` is used which prevents conflicting files to have the same name:
https://github.com/huggingface/transformers/blob/3d3e605affb792b78c918aac48f6bc82cfbf7e3e/examples/text-classification/run_xnli.py#L315


[fix] Move _adjust_logits above postprocess to fix Marian.generate

Fixes slow gpu test failures caused by #5031

This logic was already discussed in a previous PR, but, to summarize, the marian model loves to predict `pad_token_id` and if we allow pad_token_id's high score to enter the softmax we get very low probabilities for everything else.

I also rename `prepare_logits_for_generation` -> `_adjust_logits`


[tokenizers] Fix #5081 and improve backward compatibility

Adds a generic fallback for `get_special_tokens_mask()` when there is only one input sentence and `already_has_special_tokens=True`.

It's still recommended to use `return_special_mask=True` in any encoding method for efficiency but this should work as well the above indicated cases.

Removed one test related to `get_special_tokens_mask()` which didn't test its operation in a reliable manner across tokenizers.


SLOW GPU test Failures

```bash
=========================== short test summary info ============================
FAILED tests/test_modeling_marian.py::TestMarian_RU_FR::test_batch_generation_ru_fr
FAILED tests/test_modeling_marian.py::TestMarian_MT_EN::test_batch_generation_mt_en
==== 2 failed, 1110 passed, 392 skipped, 361 warnings in 1347.28s (0:22:27) ====
```

https://github.com/huggingface/transformers/runs/782480196?check_suite_focus=true


Fine-Tuning GPT2

This post references a page that no longer exists (the run_lm_finetuning.py). Where is the updated page?

"You can look into GPT-2's training on the CLM task, which is done on WikiText-2 in this example."

_Originally posted by @LysandreJik in https://github.com/huggingface/transformers/issues/1145#issuecomment-526322616_


Fix #5114

This fixes #5114, with a test to make sure there is no regression again.


AutoTokenizer supports mbart-large-en-ro

- adds `MBartConfig` so that `MBartTokenizer` can have it's own key in `TOKENIZER_MAPPING`.
- I will remove all mentions of `sshleifer/mbart-large-en-ro` right before this is merged.
It's only difference with `facebook/mbart-large-en-ro` is that `config.model_type='mbart'`. Since I didn't want to break existing Mbart callers, I will do the s3 migration right before I merge.


AutoTokenizer.from_pretrained('facebook/mbart-large-en-ro') fails

It is not in TOKENIZER_MAPPING because it has same config as Bart.
Need to make separate MBartConfig.


[cleanup] remove redundant code in SummarizationDataset


UnboundLocalError: local variable 'next_tokens' referenced before assignment when using Generate()

# 🐛 Bug

I have pre-trained GPT2 on a summarisation dataset such that summarisation is a language modelling task i.e. input = concat( padded_to_max_len(body , "TL;DR:" , summary)).

For some reason, this error occurs when I try to generate via beam search using GPT2's with language modelling head. Here is my code:

```python
from transformers import AutoTokenizer, GPT2LMHeadModel
import torch

# define tokenizer
tokenizer_kwargs = {"bos_token": "<|startoftext|>", "eos_token": "<|endoftext|>", "pad_token": "<|pad|>"}
tokenizer = AutoTokenizer.from_pretrained("gpt2", **tokenizer_kwargs)

# define model
model = GPT2LMHeadModel.from_pretrained("gpt2", pad_token_id=tokenizer.eos_token_id)

# define input
input_ids = torch.tensor(tokenizer.encode("some text that ends in TL;DR:")).unsqueeze(0)

# attempt to generate
y_pred_tensor = model.generate(input_ids=input_ids,
                                                num_beams=5,
                                                early_stopping=True,
                                                no_repeat_ngram_size=2,
                                                max_length=100
                                                )
```
```bash
File "/Users/christopherdoyle/cp_projects/scribbl-ai/Scribbl/Scribbl/summarizer/models.py", line 147, in summarize
    y_pred_tensor = self.model.generate(input_ids=input_tensor,
  File "/Users/christopherdoyle/cp_projects/scribbl-ai/Scribbl/venv/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/Users/christopherdoyle/cp_projects/scribbl-ai/Scribbl/venv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 1100, in generate
    output = self._generate_beam_search(
  File "/Users/christopherdoyle/cp_projects/scribbl-ai/Scribbl/venv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 1499, in _generate_beam_search
    (token_id % vocab_size).item() is not eos_token_id for token_id in next_tokens[batch_idx]
UnboundLocalError: local variable 'next_tokens' referenced before assignment
```

Model I am using: ```GPT2LMHeadModel```

Language I am using the model on (English, Chinese ...): ```en```

Rather strangely, it works when ```max_length = 1024```, but not with smaller values.

## To reproduce
Running on CPU
python==3.8, transformers==2.11.0, torch==1.5.0


An Implementation of ERNIE For Language Understanding (including Pre-training models and Fine-tuning tools)

# 🌟 New model addition
ERNIE 2.0 is a continual pre-training framework for language understanding in which pre-training tasks can be incrementally built and learned through multi-task learning. ERNIE 2.0 builds a strong basic for nearly every NLP tasks: Text Classification, Ranking, NER, machine reading comprehension, text genration and so on.
## Model description

<!-- Important information -->
https://github.com/PaddlePaddle/ERNIE

## Open source status

* [ ] the model implementation is available: (give details)
* [ ] the model weights are available: (give details)
* [ ] who are the authors: (mention them, if possible by @gh-username)


support local_files_only option for tf models


[cleanup] generate_beam_search comments

This PR does two small things:

1) try to make `generate_beam_search`'s inline comments work with the code, and fix some typos.
2) make the `cur_len` argument to `BeamHypothesis.is_done` mandatory, since it is always specified. This simplifies the logic a tiny bit.


data_collator.py does not allow NoneType labels for test set predictions on Glue

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Distilbert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Open the  example Colab for text-classification from here https://huggingface.co/transformers/examples.html
2. Try to run the prediction function found in run_glue.py to predict on the official Glue test set.
3. The error is as shown below.

Earlier, this has worked with the exact same program, ever since an update recently, this error shows up.

`TypeError                                 Traceback (most recent call last)
<ipython-input-16-9eecdd4d48b1> in <module>()
      2 output_mode = "classification"
      3
----> 4 predictions = trainer.predict(test_dataset=test_dataset).predictions
      5 if output_mode == "classification":
      6     predictions = np.argmax(predictions, axis=1)

7 frames
/usr/local/lib/python3.6/dist-packages/transformers/data/data_collator.py in default_data_collator(features)
     45     if "label" in first:
     46         dtype = torch.long if type(first["label"]) is int else torch.float
---> 47         batch["labels"] = torch.tensor([f["label"] for f in features], dtype=dtype)
     48     elif "label_ids" in first:
     49         if isinstance(first["label_ids"], torch.Tensor):

TypeError: must be real number, not NoneType`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The error can be seen in the Colab notebook here https://colab.research.google.com/drive/1H_92qdsOOql2hS210qNrfMEEMRcAoHD_?usp=sharing

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: Colab
- Python version: NA
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?:


GPU out of memory with Reformer enwik8 model

# ❓ Questions & Help

I'm trying to run the pretrained model `google/reformer-enwik8` but I'm getting CUDA out of memory errors unless I limit the sequences to one-fourth of the model capacity (~16k instead of the 65k).

This happens with a Titan Xp with 12GB RAM; I expected all the tricks of the Reformer to make the model with the original sequence size fit.

The code I'm running:
```
model = ReformerModelWithLMHead.from_pretrained('google/reformer-enwik8')
model.cuda()

config = model.config
max_len = config.max_position_embeddings
dataset = Enwik8Dataset(
    path, max_len, pad_id=config.pad_token_id,
    eos_id=config.eos_token_id)
loader = DataLoader(dataset, batch_size=1, shuffle=False)

acc_loss = 0
for batch in loader:
    with torch.no_grad():
        batch_loss = model(input_ids=batch, labels=batch)[0]
    acc_loss += batch_loss.mean().item()

acc_loss /= len(dataset)
```

The Enwik8Dataset inherits from Dataset and does the basic data preprocessing, I can post the code if necessary.

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: https://stackoverflow.com/questions/62373033/gpu-out-of-memory-with-enwik8-reformer-from-huggingface


Strange exception

# ❓ Questions & Help

Hi everybody,
I'm keeping experiment with the following colab:
https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb#scrollTo=X9_Go99fvW-z

however I got a new error after the fine-tuning starts:

Exception in thread Thread-20:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/parallel_loader.py", line 141, in _loader_worker
    _, data = next(data_iter)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 354, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 394, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "<ipython-input-6-cc30f67d4a07>", line 34, in collate_batch
    input_ids = torch.stack([example['input_ids'] for example in batch])
  File "<ipython-input-6-cc30f67d4a07>", line 34, in <listcomp>
    input_ids = torch.stack([example['input_ids'] for example in batch])
TypeError: string indices must be integers

Is this due to the new changes introduced yesterday for the DataCollate ?


[Marian] Run predictions on GPU? RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select`

Is it possible to run the Marian NMT model predictions on GPU, should not it be faster? Once I try to move the model to GPU I get the error described below. Is there any quick fix to the code to achieve this?
```
torch.__version__
Out[15]: '1.5.0'

```

The code to reproduce:
```
from transformers import MarianMTModel, MarianTokenizer
model_name = 'Helsinki-NLP/opus-mt-fr-en'
model = MarianMTModel.from_pretrained(model_name).cuda()
tokenizer = MarianTokenizer.from_pretrained(model_name)
```
Then I check the current device:
```
model.device
Out[33]: device(type='cuda', index=0)
```
Then I try to predict following the examples from documentation:
```
batch = tokenizer.prepare_translation_batch(list_of_sents)
gen = model.generate(**batch, num_beam=1, early_stopping = True, no_repeat_ngram_size = 3)

  File "<ipython-input-9-54f44d46fb14>", line 1, in <module>
    gen = model.generate(**batch, num_beam=1, early_stopping = True, no_repeat_ngram_size = 3)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1146, in generate
    encoder_outputs: tuple = encoder(input_ids, attention_mask=attention_mask)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/transformers/modeling_bart.py", line 292, in forward
    inputs_embeds = self.embed_tokens(input_ids) * self.embed_scale

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/modules/sparse.py", line 114, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/functional.py", line 1724, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)

RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select`

```


XLMForMultipleChoice

This PR adds `XLMForMultipleChoice`. One of the missing models in this [project](https://github.com/huggingface/transformers/projects/17)

@sgugger


Flaky tests sometimes caused by S3 failures

# 🐛 Bug

## Information
The test: `tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_fill_mask` is flaky on circle ci: https://app.circleci.com/pipelines/github/huggingface/transformers/7734/workflows/7c51cfcf-5425-4172-aa15-ed677e37f7fc/jobs/49891/steps

## To reproduce

Not sure. It pops up from time to time on circle ci.
Need to investigate more

## Expected behavior

The test should not be flaky

## Environment info
On circle ci


Create README.md

model card

Related to pretrained model cards


Create README.md

Related to pretrained model cards

@julien-c check out that dataset meta tag is right


How can I initialize RobertaForSequenceClassification empty?

I am having my own custom dataset which is completely different from the one with roberta. Now using ByteLevelBPETokenizer, I am creating vocab.json and merges.txt . I am using these two to files to initialize RobertaTokenizerFast for encoding my corpus. Now I am training a RobertaForSequenceClassification for a binary classification problem. When I am initializing RobertaForSequenceClassification with any of the pre-trained models I am getting
`IndexError: index out of range in self` on the RAM while on GPU I am getting `RuntimeError: CUDA error: an illegal memory access was encountered` . I have followed other [issues ](https://github.com/pytorch/pytorch/issues/21819) but of no help. My understanding since I am creating my own vocabulary, some of the tokens are not in pre-trained model. So is there a way to initialize RobertaForSequenceClassification empty or to train this classification model for my dataset?


Is there a helper script to randomly mask spans of text for T5 pretraining?

Hello team,

 If I have an input text like

```The cute dog walks in the park```

Is there a library within HuggingFace that can output the following for me by masking random spans of text by (optional) taking an input masking probability?

```
masked input: The <extra_id_1> walks in <extra_id_2> park
target: <extra_id_1> cute dog <extra_id_2> the <extra_id_3> </s>

```

This is for preparing data for T5 within my own codebase.