tf.saved_model.save is not worked on TFElectra* series.
Author: seopbo
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

Model I am using TFElectraModel:

Language I am using the model on English:

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
import tensorflow as tf
from pathlib import Path
from transformers import TFAutoModel

dump_dir = Path("test_electra")

if not dump_dir.exists():
    dump_dir.mkdir(parents=True)

model = TFAutoModel.from_pretrained("google/electra-base-discriminator")
tf.saved_model.save(model, export_dir=str(dump_dir))
```
```bash
Traceback (most recent call last):
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-a8a019d74543>", line 11, in <module>
    tf.saved_model.save(model, export_dir=str(dump_dir))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 951, in save
    obj, export_dir, signatures, options, meta_graph_def)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 1008, in _build_meta_graph
    checkpoint_graph_view)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/saved_model/signature_serialization.py", line 75, in find_function_to_export
    functions = saveable_view.list_functions(saveable_view.root)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 143, in list_functions
    self._serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 1656, in _list_functions_for_serialization
    Model, self)._list_functions_for_serialization(serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 2750, in _list_functions_for_serialization
    .list_functions_for_serialization(serialization_cache))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/base_serialization.py", line 87, in list_functions_for_serialization
    fns = self.functions_to_serialize(serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 77, in functions_to_serialize
    serialization_cache).functions_to_serialize)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 92, in _get_serialized_attributes
    serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/model_serialization.py", line 53, in _get_serialized_attributes_internal
    serialization_cache))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 101, in _get_serialized_attributes_internal
    functions = save_impl.wrap_layer_functions(self.obj, serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 153, in wrap_layer_functions
    original_fns = _replace_child_layer_functions(layer, serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 272, in _replace_child_layer_functions
    serialization_cache).functions)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 92, in _get_serialized_attributes
    serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/model_serialization.py", line 53, in _get_serialized_attributes_internal
    serialization_cache))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 101, in _get_serialized_attributes_internal
    functions = save_impl.wrap_layer_functions(self.obj, serialization_cache)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 163, in wrap_layer_functions
    '{}_layer_call_and_return_conditional_losses'.format(layer.name))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 503, in add_function
    self.add_trace(*self._input_signature)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 418, in add_trace
    trace_with_training(True)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 416, in trace_with_training
    fn.get_concrete_function(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 547, in get_concrete_function
    return super(LayerCall, self).get_concrete_function(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 959, in get_concrete_function
    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 865, in _get_concrete_function_garbage_collected
    self._initialize(args, kwargs, add_initializers_to=initializers)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 506, in _initialize
    *args, **kwds))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2446, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2777, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2667, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 981, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 441, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 524, in wrapper
    ret = method(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 170, in wrap_with_training_arg
    lambda: replace_training_and_call(False))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 65, in smart_cond
    pred, true_fn=true_fn, false_fn=false_fn, name=name)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/framework/smart_cond.py", line 54, in smart_cond
    return true_fn()
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 169, in <lambda>
    lambda: replace_training_and_call(True),
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 165, in replace_training_and_call
    return wrapped_call(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 566, in call_and_return_conditional_losses
    return layer_call(inputs, *args, **kwargs), layer.get_losses_for(inputs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/transformers/modeling_tf_electra.py", line 292, in call
    hidden_states = self.embeddings([input_ids, position_ids, token_type_ids, inputs_embeds], training=training)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 968, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 71, in return_outputs_and_add_losses
    outputs, losses = fn(inputs, *args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 170, in wrap_with_training_arg
    lambda: replace_training_and_call(False))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 65, in smart_cond
    pred, true_fn=true_fn, false_fn=false_fn, name=name)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/framework/smart_cond.py", line 54, in smart_cond
    return true_fn()
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 169, in <lambda>
    lambda: replace_training_and_call(True),
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 165, in replace_training_and_call
    return wrapped_call(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 541, in __call__
    self.call_collection.add_trace(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 418, in add_trace
    trace_with_training(True)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 416, in trace_with_training
    fn.get_concrete_function(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 547, in get_concrete_function
    return super(LayerCall, self).get_concrete_function(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 959, in get_concrete_function
    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 865, in _get_concrete_function_garbage_collected
    self._initialize(args, kwargs, add_initializers_to=initializers)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 506, in _initialize
    *args, **kwds))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2446, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2777, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2667, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 981, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 441, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 524, in wrapper
    ret = method(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 170, in wrap_with_training_arg
    lambda: replace_training_and_call(False))
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 65, in smart_cond
    pred, true_fn=true_fn, false_fn=false_fn, name=name)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/framework/smart_cond.py", line 54, in smart_cond
    return true_fn()
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 169, in <lambda>
    lambda: replace_training_and_call(True),
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 165, in replace_training_and_call
    return wrapped_call(*args, **kwargs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 566, in call_and_return_conditional_losses
    return layer_call(inputs, *args, **kwargs), layer.get_losses_for(inputs)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1627, in get_losses_for
    reachable = tf_utils.get_reachable_from_inputs(inputs, losses)
  File "/Users/nick/.pyenv/versions/conversion/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 140, in get_reachable_from_inputs
    raise TypeError('Expected Operation, Variable, or Tensor, got ' + str(x))
TypeError: Expected Operation, Variable, or Tensor, got None
```
## Expected behavior
```python
import tensorflow as tf
from pathlib import Path
from transformers import TFAutoModel

dump_dir = Path("test_distilbert")

if not dump_dir.exists():
    dump_dir.mkdir(parents=True)

model = TFAutoModel.from_pretrained("distilbert-base-uncased")
tf.saved_model.save(model, export_dir=str(dump_dir))
```
```bash
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x16538b0d0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x165347250>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x16531c2d0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x165e22f90>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x16528bd90>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x16526d850>, because it is not built.
INFO:tensorflow:Assets written to: test_distilbert/assets
```

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: Darwin-19.6.0-x86_64-i386-64bit
- Python version: 3.7.7
- PyTorch version (GPU?): 1.5.1 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.