When I use TFBertEncoder in my laptop, I get an error.I can not build a model. Here is a simple examples.
Author: shange1996
Pull request: no

# 🐛 Bug

## Information

Model I am using TFBertEncoder:

Language I am using the model on English:

The problem arises when using:
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. When I use, TFBertEncoder, I get an error.

Here is my code.
```py
import tensorflow as tf
import numpy as np
from transformers.modeling_tf_bert import BertConfig, TFBertEncoder

print(tf.__name__, tf.__version__)
input_a = tf.keras.layers.Input(shape=(91, 128))

config = BertConfig()
config.hidden_size = 128
config.num_attention_heads = 4

# config.output_attentions = False
# config.output_hidden_states = False

head_mask = [None for _ in range(config.num_hidden_layers)]
encoder_output = TFBertEncoder(config=config)([input_a, None, head_mask])[0]
print(encoder_output.shape)

test_out = tf.keras.layers.Dense(128)(encoder_output)
print(test_out.shape)
```
## Expected behavior
Here is the error:

```
(None, 91, 128)
2020-06-03 11:18:10.160647: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Failed precondition: Error while reading resource variable _AnonymousVar189 from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource localhost/_AnonymousVar189/class tensorflow::Var does not exist.
	 [[{{node output_23/dense/BiasAdd/ReadVariableOp}}]]
Traceback (most recent call last):
  File "D:/python/tx/TEST.py", line 16, in <module>
    a = tf.keras.layers.Dense(128)(encoder_output)
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 720, in __call__
    base_layer_utils.create_keras_history(inputs)
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer_utils.py", line 187, in create_keras_history
    _, created_layers = _create_keras_history_helper(tensors, set(), [])
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer_utils.py", line 249, in _create_keras_history_helper
    layer_inputs, processed_ops, created_layers)
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer_utils.py", line 249, in _create_keras_history_helper
    layer_inputs, processed_ops, created_layers)
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer_utils.py", line 249, in _create_keras_history_helper
    layer_inputs, processed_ops, created_layers)
  [Previous line repeated 5 more times]
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer_utils.py", line 247, in _create_keras_history_helper
    constants[i] = backend.function([], op_input)([])
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\keras\backend.py", line 3727, in __call__
    outputs = self._graph_fn(*converted_inputs)
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\eager\function.py", line 1551, in __call__
    return self._call_impl(args, kwargs)
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\eager\function.py", line 1591, in _call_impl
    return self._call_flat(args, self.captured_inputs, cancellation_manager)
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\eager\function.py", line 1692, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\eager\function.py", line 545, in call
    ctx=ctx)
  File "D:\Anaconda3\lib\site-packages\tensorflow_core\python\eager\execute.py", line 67, in quick_execute
    six.raise_from(core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.FailedPreconditionError:  Error while reading resource variable _AnonymousVar189 from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource localhost/_AnonymousVar189/class tensorflow::Var does not exist.
	 [[node output_23/dense/BiasAdd/ReadVariableOp (defined at /python/tx/TEST.py:16) ]] [Op:__inference_keras_scratch_graph_5205]

Function call stack:
keras_scratch_graph
```

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.3.0 (in conda list)
- Platform:
- Python version:3.7
- PyTorch version (GPU?):
- Tensorflow version (GPU?):TF2.1.0(GPU)
- Using GPU in script?:
- Using distributed or parallel set-up in script?:No


Comments:
I also meet this problem? Do you solve it? @shange1996 

> I also meet this problem? Do you solve it? @shange1996

No. The problem bothers me.

Me too. When I change use keras not tf.keras, it has another problem...

It bothers me, too.

> It bothers me, too.

Now I solve it. NOT import it, instead of copying the layer code in your main code.
Like this:
`class TFBertSelfAttention(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        if config.hidden_size % config.num_attention_heads != 0:
            raise ValueError(
                "The hidden size (%d) is not a multiple of the number of attention "
                "heads (%d)" % (config.hidden_size, config.num_attention_heads)
            )
        self.output_attentions = config.output_attentions

        self.num_attention_heads = config.num_attention_heads
        assert config.hidden_size % config.num_attention_heads == 0
        self.attention_head_size = int(config.hidden_size / config.num_attention_heads)
        self.all_head_size = self.num_attention_heads * self.attention_head_size

        self.query = tf.keras.layers.Dense(
            self.all_head_size, kernel_initializer=tf.keras.initializers.TruncatedNormal(config.initializer_range), name="query"
        )
        self.key = tf.keras.layers.Dense(
            self.all_head_size, kernel_initializer=tf.keras.initializers.TruncatedNormal(config.initializer_range), name="key"
        )
        self.value = tf.keras.layers.Dense(
            self.all_head_size, kernel_initializer=tf.keras.initializers.TruncatedNormal(config.initializer_range), name="value"
        )

        self.dropout = tf.keras.layers.Dropout(config.attention_probs_dropout_prob)

    def transpose_for_scores(self, x, batch_size):
        x = tf.reshape(x, (batch_size, -1, self.num_attention_heads, self.attention_head_size))
        return tf.transpose(x, perm=[0, 2, 1, 3])

    def call(self, inputs, training=False):
        hidden_states, attention_mask, head_mask = inputs

        batch_size = tf.shape(hidden_states)[0]
        mixed_query_layer = self.query(hidden_states)
        mixed_key_layer = self.key(hidden_states)
        mixed_value_layer = self.value(hidden_states)

        query_layer = self.transpose_for_scores(mixed_query_layer, batch_size)
        key_layer = self.transpose_for_scores(mixed_key_layer, batch_size)
        value_layer = self.transpose_for_scores(mixed_value_layer, batch_size)

        # Take the dot product between "query" and "key" to get the raw attention scores.
        attention_scores = tf.matmul(
            query_layer, key_layer, transpose_b=True
        )  # (batch size, num_heads, seq_len_q, seq_len_k)
        dk = tf.cast(tf.shape(key_layer)[-1], tf.float32)  # scale attention_scores
        attention_scores = attention_scores / tf.math.sqrt(dk)

        if attention_mask is not None:
            # Apply the attention mask is (precomputed for all layers in TFBertModel call() function)
            attention_scores = attention_scores + attention_mask

        # Normalize the attention scores to probabilities.
        attention_probs = tf.nn.softmax(attention_scores, axis=-1)

        # This is actually dropping out entire tokens to attend to, which might
        # seem a bit unusual, but is taken from the original Transformer paper.
        attention_probs = self.dropout(attention_probs, training=training)

        # Mask heads if we want to
        if head_mask is not None:
            attention_probs = attention_probs * head_mask

        context_layer = tf.matmul(attention_probs, value_layer)

        context_layer = tf.transpose(context_layer, perm=[0, 2, 1, 3])
        context_layer = tf.reshape(
            context_layer, (batch_size, -1, self.all_head_size)
        )  # (batch_size, seq_len_q, all_head_size)

        outputs = (context_layer, attention_probs) if self.output_attentions else (context_layer,)
        return outputs`

@shange1996  just copy `TFBertSelfAttention` class?

> @shange1996 just copy `TFBertSelfAttention` class?

Yes! Just copy， no import.

Hey guys, 

I looked into the issue and I think the best solution is to use a keras layer wrapper as follows:

```python
#!/usr/bin/env python3
import tensorflow as tf
import numpy as np
from transformers.modeling_tf_bert import BertConfig, TFBertEncoder

print(tf.__name__, tf.__version__)

config = BertConfig()
config.hidden_size = 128
config.num_attention_heads = 4


class NewTFBertEncoder(tf.keras.layers.Layer):

    def __init__(self, config):
        super(NewTFBertEncoder, self).__init__()
#        self.inputs = tf.keras.layers.Input(input_shape)  # not really needed here IMO.
        self.encoder = TFBertEncoder(config=config)
        self.dense = tf.keras.layers.Dense(config.hidden_size)

    def call(self, inputs):
        head_mask = [None for _ in range(config.num_hidden_layers)]
        output = self.encoder([inputs, None, head_mask])[0]
        dense_output = self.dense(output)

        return dense_output


new_bert_encoder = NewTFBertEncoder(config)
output = new_bert_encoder(np.ones((2, 91, 128)))  # batch size , sequence length, hidden size
```

Two things:

- If a customized layer is to be used with standard keras layers (as is the case here), as far as I know it is recommended to use a `keras.layers` wrapper class. Also see here: https://www.tensorflow.org/api_docs/python/tf/keras/layers/Lambda#variables_2
- I don't think that the tf.keras.inputs class is needed here (or do you have a speciifc use case in mind?). Keras usually creates such an instance under the hood anyways, see: https://www.tensorflow.org/api_docs/python/tf/keras/layers/InputLayer

Also pinging @jplu to check if my code proposal is the right choice here.

@shange1996 @etveritas - Let me know if the proposed solution works for you. If not feel free to re-open the issue :-) 
Also linking this issue to: https://github.com/huggingface/transformers/issues/5046. 

Both work for me , thanks!

Hey ! This error appears when some variable are initialized elsewhere than in the Layer itself. The solution that @patrickvonplaten proposes is a good one!! Good job people :)