Repository: huggingface/transformers

Issue:
ValueError raises when load Flaubert from pre-train with Transformers >=3.0.0
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): **Flaubert**

Language I am using the model on (English, Chinese ...): **French**

The problem arises when using: my own modified scripts

## To reproduce

**Just load Flaubert model with from_pretrained:**
```
MODEL = "flaubert/flaubert_large_cased"
transformer_layer = TFAutoModel.from_pretrained(MODEL, from_pt=True)
```

**Then a ValueError is raised:**

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-22-4235ae313fbd> in <module>()
      1     MODEL = "flaubert/flaubert_large_cased"
----> 2     transformer_layer = TFAutoModel.from_pretrained(MODEL, from_pt=True)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    421         for config_class, model_class in TF_MODEL_MAPPING.items():
    422             if isinstance(config, config_class):
--> 423                 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
    424         raise ValueError(
    425             "Unrecognized configuration class {} for this kind of TFAutoModel: {}.\n"

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    479         if from_pt:
    480             # Load from a PyTorch checkpoint
--> 481             return load_pytorch_checkpoint_in_tf2_model(model, resolved_archive_file, allow_missing_keys=True)
    482 
    483         model(model.dummy_inputs, training=False)  # build the network with dummy inputs

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_pytorch_utils.py in load_pytorch_checkpoint_in_tf2_model(tf_model, pytorch_checkpoint_path, tf_inputs, allow_missing_keys)
     91 
     92     return load_pytorch_weights_in_tf2_model(
---> 93         tf_model, pt_state_dict, tf_inputs=tf_inputs, allow_missing_keys=allow_missing_keys
     94     )
     95 

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_pytorch_utils.py in load_pytorch_weights_in_tf2_model(tf_model, pt_state_dict, tf_inputs, allow_missing_keys)
    123 
    124     if tf_inputs is not None:
--> 125         tf_model(tf_inputs, training=False)  # Make sure model is built
    126 
    127     # Adapt state dict - TODO remove this and update the AWS weights files instead

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_xlm.py in call(self, inputs, **kwargs)
    635             heads.
    636         """
--> 637         outputs = self.transformer(inputs, **kwargs)
    638         return outputs
    639 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_flaubert.py in call(self, inputs, attention_mask, langs, token_type_ids, position_ids, lengths, cache, head_mask, inputs_embeds, training, output_attentions, output_hidden_states)
    267                 tensor_normalized = self.layer_norm1[i](tensor)
    268                 attn_outputs = self.attentions[i](
--> 269                     [tensor_normalized, attn_mask, None, cache, head_mask[i]], training=training
    270                 )
    271                 attn = attn_outputs[0]

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_xlm.py in call(self, inputs, training)
    139         Self-attention (if kv is None) or attention over source sentence (provided by kv).
    140         """
--> 141         input, mask, kv, cache, head_mask, output_attentions = inputs
    142         # Input is (bs, qlen, dim)
    143         # Mask is (bs, klen) (non-causal) or (bs, klen, klen)

ValueError: not enough values to unpack (expected 6, got 5)
```
## Expected behavior
Load the model successfully. 

- It can be achieved with Transformers version <= 2.11.0

- I also try CamemBERT, and Bert-like models in other languages like tr, ru, es, pt, it. None of them have this problem.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: **>=3.0.0 ( 3.0.0, 3.0.1, 3.0.2 all have this problem.)**

(I try this on google colab with GPU and TPU. The environment does not matter here. But I still provide the one with GPU.)
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1+cu101 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No



Discussion:
Hi @wzmJimmy,
Did you get any solution for this problem? I am also facing the same problem. Please let me know if you are able to make it run. Thanks.
Well, my solution is inside the expected behavior part. Downgrade transformers to version lower than 3.0.0 and it just works. I post the bug report just to ask what's wrong and how to fix it in the newer version.

Resolving patch:
diff --git a/src/transformers/modeling_tf_flaubert.py b/src/transformers/modeling_tf_flaubert.py
--- a/src/transformers/modeling_tf_flaubert.py
+++ b/src/transformers/modeling_tf_flaubert.py
@@ -296,7 +296,7 @@ def call(
             else:
                 tensor_normalized = self.layer_norm1[i](tensor)
                 attn_outputs = self.attentions[i](
-                    tensor_normalized, attn_mask, None, cache, head_mask[i], training=training
+                    tensor_normalized, attn_mask, None, cache, head_mask[i], output_attentions, training=training
                 )
                 attn = attn_outputs[0]
                 if output_attentions:


Test patch: