the output type of TFBertModel is weird
Author: roccqqck
Pull request: no
Labels: name: wontfix

```
model = TFBertModel.from_pretrained('bert-base-chinese')
model.summary()

optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model.compile(optimizer=optimizer, loss=loss, metrics=[metric])

predictions = model.predict(validation_input_ids)
print(type(predictions))
print(predictions.shape)   
```
```
<class 'list'>
AttributeError: 'list' object has no attribute 'shape'
```
The type is weird.
It is a (N, 512, 768) shape numpy array inside a List.

I had to take it out from the List
```
predictions = predictions[0]
print(predictions.shape)
```
```
(8359, 512, 768)
```

Comments:
This is because in a bert pretraining progress, there are two tasks: masked token prediction and  next sentence predition . The first needs  hidden state of each tokens ( shape: [batch_size, sequence_length, hidden_size]) the second needs  the embedding of the whole sequence (shape : [batch_size, hidden_size] ) . 

And there is also position left for some one who want to get all the hidden state from each level inside the model  ( may represent different level of abstraction besides the last one ) or the attention matrix.  

> This is because in a bert pretraining progress, there are two tasks: masked token prediction and next sentence predition . The first needs hidden state of each tokens ( shape: [batch_size, sequence_length, hidden_size]) the second needs the embedding of the whole sequence (shape : [batch_size, hidden_size] ) .


Because of this
if I want use tf.keras to custom the layer below TFBertModel
I have to add this particular line
bert = bert[0]

```
input_layer = Input(shape = (512,), dtype='int64')  
bert = TFBertModel.from_pretrained('bert-base-chinese')(input_layer)

bert = bert[0]               # I have to add this particular line

dropout = Dropout(0.1)(bert)
flat = Flatten()(dropout)
classifier = Dense(units=5)(flat)                  
model = Model(inputs=input_layer, outputs=classifier)
model.summary()
```
```
Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 512)]             0         
_________________________________________________________________
tf_bert_model (TFBertModel)  ((None, 512, 768), (None, 102267648 
_________________________________________________________________
dropout_37 (Dropout)         (None, 512, 768)          0         
_________________________________________________________________
flatten (Flatten)            (None, 393216)            0         
_________________________________________________________________
dense (Dense)                (None, 5)                 1966085   
=================================================================
Total params: 104,233,733
Trainable params: 104,233,733
Non-trainable params: 0
```

> > This is because in a bert pretraining progress, there are two tasks: masked token prediction and next sentence predition . The first needs hidden state of each tokens ( shape: [batch_size, sequence_length, hidden_size]) the second needs the embedding of the whole sequence (shape : [batch_size, hidden_size] ) .
> 
> Because of this
> if I want use tf.keras to custom the layer below TFBertModel
> I have to add this particular line
> bert = bert[0]
> 
> ```
> input_layer = Input(shape = (512,), dtype='int64')  
> bert = TFBertModel.from_pretrained('bert-base-chinese')(input_layer)
> 
> bert = bert[0]               # I have to add this particular line
> 
> dropout = Dropout(0.1)(bert)
> flat = Flatten()(dropout)
> classifier = Dense(units=5)(flat)                  
> model = Model(inputs=input_layer, outputs=classifier)
> model.summary()
> ```
> 
> ```
> Model: "model"
> _________________________________________________________________
> Layer (type)                 Output Shape              Param #   
> =================================================================
> input_1 (InputLayer)         [(None, 512)]             0         
> _________________________________________________________________
> tf_bert_model (TFBertModel)  ((None, 512, 768), (None, 102267648 
> _________________________________________________________________
> dropout_37 (Dropout)         (None, 512, 768)          0         
> _________________________________________________________________
> flatten (Flatten)            (None, 393216)            0         
> _________________________________________________________________
> dense (Dense)                (None, 5)                 1966085   
> =================================================================
> Total params: 104,233,733
> Trainable params: 104,233,733
> Non-trainable params: 0
> ```

That's right.  But for sentence level classification , I recommend you to use the embedding of whole sequence .
```
bert = bert[1] # instead of  bert = bert[0]   
```

Just like what the official sequence classificiation does in **TFBertForSequenceClassification** class at
https://github.com/huggingface/transformers/blob/master/transformers/modeling_tf_bert.py

> ```
> bert = bert[1] # instead of  bert = bert[0]   
> ```

May I ask why?
It looks like it reduce the features of flatten layer.
It doesn't look like whole.
```
input_layer = Input(shape = (512,), dtype='int64')  
bert = TFBertModel.from_pretrained('bert-base-chinese')(input_layer)
bert = bert[1]    
dropout = Dropout(0.1)(bert)
flat = Flatten()(dropout)
classifier = Dense(units=5)(flat)                  
model = Model(inputs=input_layer, outputs=classifier)
model.summary()
```
```
Model: "model_5"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_6 (InputLayer)         [(None, 512)]             0         
_________________________________________________________________
tf_bert_model_5 (TFBertModel ((None, 512, 768), (None, 102267648 
_________________________________________________________________
dropout_225 (Dropout)        (None, 768)               0         
_________________________________________________________________
flatten_3 (Flatten)          (None, 768)               0         
_________________________________________________________________
dense_5 (Dense)              (None, 5)                 3845      
=================================================================
Total params: 102,271,493
Trainable params: 102,271,493
Non-trainable params: 0
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


It is still a problem

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.