Repository: huggingface/transformers

Issue:
Fail to convert the Funnel Transformer tensorflow version to transformer one when use the official script
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:3.5.1
- Platform:Centos
- Python version:3.7
- PyTorch version (GPU?):1.6.0
- Tensorflow version (GPU?):2.3.2
- Using GPU in script?:yes
- Using distributed or parallel set-up in script?:yes


## Information

Model I am using (Bert, XLNet ...):Funnel Transformer


## To reproduce

Steps to reproduce the behavior:

1.use the script `convert_funnel_original_tf_checkpoint_to_pytorch.py`@sgugger @LysandreJik
raise error
```
Traceback (most recent call last):
  File "run_pretraining.py", line 158, in <module>
    convert_tf_checkpoint_to_pytorch(args.tf_checkpoint_path, args.config_file, args.pytorch_dump_path)
  File "run_pretraining.py", line 40, in convert_tf_checkpoint_to_pytorch
    load_tf_weights_in_funnel(model, config, tf_checkpoint_path)
  File "run_pretraining.py", line 122, in load_tf_weights_in_funnel
    pointer = getattr(pointer, _layer_map[m_name])
  File "/root/miniconda3/envs/transformers/lib/python3.7/site-packages/torch/nn/modules/module.py", line 772, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'FunnelForPreTraining' object has no attribute 'embeddings'
```
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->



Discussion:
Hi! Could you explain the full procedure? Where did you obtain the Funnel transformer TensorFlow version? Is it a model you trained yourself using another framework? (like this one: https://github.com/laiguokun/Funnel-Transformer)
just use the official ones(like this one: https://github.com/laiguokun/Funnel-Transformer) @LysandreJik 
the layer map "input" -> "embedding", raise error
Could you provide the configuration you used, as well as which Funnel Transformer (which identifier? Is it the TensorFlow or the TensorFlow-Full) you tried to convert? Thank you
@LysandreJik I was train my funnel with the official code, I think my pretrain tensorflow is Tensorflow-Full with the adam weight. May be I need to transform my pretrain model to the TensorFlow or the TensorFlow-Full one first, then use the convert script to change to the transformer one?

Resolving patch:
diff --git a/src/transformers/models/funnel/convert_funnel_original_tf_checkpoint_to_pytorch.py b/src/transformers/models/funnel/convert_funnel_original_tf_checkpoint_to_pytorch.py
--- a/src/transformers/models/funnel/convert_funnel_original_tf_checkpoint_to_pytorch.py
+++ b/src/transformers/models/funnel/convert_funnel_original_tf_checkpoint_to_pytorch.py
@@ -19,18 +19,18 @@
 
 import torch
 
-from transformers import FunnelConfig, FunnelForPreTraining, load_tf_weights_in_funnel
+from transformers import FunnelBaseModel, FunnelConfig, FunnelModel, load_tf_weights_in_funnel
 from transformers.utils import logging
 
 
 logging.set_verbosity_info()
 
 
-def convert_tf_checkpoint_to_pytorch(tf_checkpoint_path, config_file, pytorch_dump_path):
+def convert_tf_checkpoint_to_pytorch(tf_checkpoint_path, config_file, pytorch_dump_path, base_model):
     # Initialise PyTorch model
     config = FunnelConfig.from_json_file(config_file)
     print("Building PyTorch model from configuration: {}".format(str(config)))
-    model = FunnelForPreTraining(config)
+    model = FunnelBaseModel(config) if base_model else FunnelModel(config)
 
     # Load weights from tf checkpoint
     load_tf_weights_in_funnel(model, config, tf_checkpoint_path)
@@ -57,5 +57,10 @@ def convert_tf_checkpoint_to_pytorch(tf_checkpoint_path, config_file, pytorch_du
     parser.add_argument(
         "--pytorch_dump_path", default=None, type=str, required=True, help="Path to the output PyTorch model."
     )
+    parser.add_argument(
+        "--base_model", action="store_true", help="Whether you want just the base model (no decoder) or not."
+    )
     args = parser.parse_args()
-    convert_tf_checkpoint_to_pytorch(args.tf_checkpoint_path, args.config_file, args.pytorch_dump_path)
+    convert_tf_checkpoint_to_pytorch(
+        args.tf_checkpoint_path, args.config_file, args.pytorch_dump_path, args.base_model
+    )


Test patch: