Keras model created from individual Bert Layers has weights not shown in trainable_weights nor non_trainable_weights. model.summary() / utils.plot_model shows those weights as part of graph though
Author: Santosh-Gupta
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

I created a model which takes 2 layers from from a bert model, and creates a mini-bert model.

However, not all the weights/layers from the component layers are in this model. But These weights are shown was connected in `model.summary()` and in 

```
tf.keras.utils.plot_model(
    model, to_file='model.png', show_shapes=False, show_layer_names=True,
    rankdir='TB', expand_nested=False, dpi=96
)
```


Model I am using (Bert, XLNet ...): bert-base-uncased 

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ X] my own modified scripts: (give details below)


## To reproduce

Steps to reproduce the behavior:

Here is a Colabnotebook with a minimal example that reproduced the issue. 

https://colab.research.google.com/drive/1n3_XNhdgH6Qo7GT-M570lIKWAoU3TML5?usp=sharing

And here is the code

```
!pip install transformers --q
%tensorflow_version 2.x

from transformers import TFBertModel, AutoModel, TFRobertaModel, AutoTokenizer
import tensorflow as tf
import tensorflow_addons as tfa

tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)

from tensorflow import keras
from tensorflow.keras import layers
from copy import deepcopy

logger = tf.get_logger()
logger.info(tf.__version__)


def get_mini_models():
    tempModel = TFRobertaModel.from_pretrained('bert-base-uncased', from_pt=True)

    layer9 = deepcopy(tempModel.layers[0].encoder.layer[8])
    layer10 = deepcopy(tempModel.layers[0].encoder.layer[9])

    inputHiddenVals = tf.keras.Input(shape=[None, None], dtype=tf.float32, name='input_Q',
                                    batch_size=None) 

    hidden1 = layer9((inputHiddenVals, None, None))
    hidden2 = layer10((hidden1[0], None, None))
    modelNew = tf.keras.Model(inputs=inputHiddenVals, outputs=hidden2)

    del tempModel

    return modelNew

@tf.function
def loss_fn(_, probs):
    bs = tf.shape(probs)[0]
    labels = tf.eye(bs, bs)
    return tf.losses.categorical_crossentropy(labels,
                                              probs,
                                              from_logits=True)

model = get_mini_models()
model.compile(loss=loss_fn,
                optimizer=tfa.optimizers.AdamW(weight_decay=1e-4, learning_rate=1e-5, 
                                                epsilon=1e-06))

# Get model and layers directly to compare
tempModel = TFRobertaModel.from_pretrained('bert-base-uncased', from_pt=True)
layer9 = deepcopy(tempModel.layers[0].encoder.layer[8])
layer10 = deepcopy(tempModel.layers[0].encoder.layer[9])

# Only one layer, and that layer also has missing weights. 
for i, var in enumerate(model.weights):
    print(model.weights[i].name)

# Full weights for one layer 
for i, var in enumerate(layer9.weights):
    print(layer9.weights[i].name)

# Test what correct output should be 

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
inputt = tokenizer.encode('This is a sentence', return_tensors='tf')
outt = tempModel(inputt)[0]

# Test model output. Not the same. 

model(outt)

# Model summary somehow lists the weights 
model.summary()

# Model diagram shows the correct connections between all the layers. 

tf.keras.utils.plot_model(
    model, to_file='model.png', show_shapes=False, show_layer_names=True,
    rankdir='TB', expand_nested=False, dpi=96
)

```

Edit: I also tried making the layers from scratch, and setting the weights directly, same result. Here's a colab notebook that does htis. https://colab.research.google.com/drive/1EC_fObSp9lUsj_PFaYgFtRI93ErPYmU9?usp=sharing

## Expected behavior

The model should contain all the weights from the component layers, and the output of the model should be the same as executing the layers in the same way outside the model. 

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.11.0
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0+cu101 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?:no
- Using distributed or parallel set-up in script?: no



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.