`TFBertModel.predict()` and `TFBertModel()` return different results.
Author: a7744hsc
Pull request: no

## Environment info
- `transformers` version: 4.17.0
- Platform: Linux-5.4.0-99-generic-x86_64-with-glibc2.27
- Python version: 3.9.0
- PyTorch version (GPU?): 1.10.2+cu102 (True)
- Tensorflow version (GPU?): 2.8.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no



Models:
It affects multiple models, at least  BERT and Sentence-BERT.
- ALBERT, BERT, XLM, DeBERTa, DeBERTa-v2, ELECTRA, MobileBert, SqueezeBert: @LysandreJik 


## Information

Model I am using (Bert, XLNet ...): BERT-base

The problem arises when using:
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
Run the following script:
```
from transformers import AutoTokenizer,TFAutoModel,AutoConfig,TFAutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
model = TFAutoModel.from_pretrained('bert-base-uncased')

sentence='It can be any sentence here'
encoded_input = tokenizer(text, return_tensors='tf')
output0 = model(encoded_input)[0]
output1 = model(encoded_input,training=False)[0]
output2 = model(encoded_input,training=True)[0]
output3 = model.call(encoded_input)[0]
output4 = model.call(encoded_input,training=False)[0]
output5 = model.call(encoded_input,training=True)[0]
output6 = model.predict(encoded_input.values())[0]
print(output0[0,0,:10])
print(output1[0,0,:10])
print(output2[0,0,:10])
print(output3[0,0,:10])
print(output4[0,0,:10])
print(output5[0,0,:10])
print(output6[0,0,:10])
```
The output is :
```
tf.Tensor(
[-0.65482056  0.13908163 -0.7431607  -0.46033248 -0.834605    0.24891722
  1.042905    0.34481025 -0.19269213 -0.32815126], shape=(10,), dtype=float32)
tf.Tensor(
[-0.65482056  0.13908163 -0.7431607  -0.46033248 -0.834605    0.24891722
  1.042905    0.34481025 -0.19269213 -0.32815126], shape=(10,), dtype=float32)
tf.Tensor(
[-0.65266645 -0.03810687 -0.776067   -0.07178783 -0.9046151   0.18066
  0.82093364  0.53261256  0.06615249 -0.03404194], shape=(10,), dtype=float32)
tf.Tensor(
[-0.65482056  0.13908163 -0.7431607  -0.46033248 -0.834605    0.24891722
  1.042905    0.34481025 -0.19269213 -0.32815126], shape=(10,), dtype=float32)
tf.Tensor(
[-0.65482056  0.13908163 -0.7431607  -0.46033248 -0.834605    0.24891722
  1.042905    0.34481025 -0.19269213 -0.32815126], shape=(10,), dtype=float32)
tf.Tensor(
[-0.7354083   0.18443112 -1.049724   -0.532299   -0.8917109   0.42158493
  1.2287179   0.6927482   0.02612726 -0.5916267 ], shape=(10,), dtype=float32)
[-0.8285751   0.2074836  -0.36887544 -0.3653527  -1.0217763   0.608446
  0.9114009   0.4278982  -0.02307865 -0.09100434]
```

The output are different between `model(encoded_input,training=True)`  `model.call(encoded_input,training=True)`;
And the output are different between `model.predict(encoded_input.values())` and `model(encoded_input)`,  This one makes me very confuse and I believe this is a bug.

## Expected behavior
 `model.predict(encoded_input.values())` and `model(encoded_input)` should give the same output.



Comments:
Pinging @Rocketknight1 @gante 

I'll take it! Investigating now.

Hi @a7744hsc, I don't think there's a bug here. When the model is called with `training=True`, certain layers like Dropout are active. This means that `model(encoded_input,training=True)[0]` and `model.call(encoded_input,training=True)[0]` are giving different outputs, but not because they actually call different code - it's because layers like Dropout are randomly dropping out different neurons. You can see this by simply calling `model(encoded_input,training=True)[0]` twice, and noting that the results are different each time, whereas if you use `training=False` then the results are consistent, because Dropout is disabled.

The second issue you mentioned is the difference between `model.predict(encoded_input.values())` and `model(encoded_input)`. The issue here is that `encoded_input.values()` does not unpack the data the way the model expects - it loses the dict keys and so the model can't tell which array corresponds to which model argument. As a result, this line will give bad results. If you want to use `model.predict`, you could just try `model.predict(encoded_input["input_ids"])`. If you do this, you'll see the results are extremely close to the results you get from `model(encoded_input,training=False)`. The remaining small differences are on the order of `1e-4` or `1e-5` and are mostly just `float32` rounding errors, I think. (The technical reason is, I suspect, that `model.predict` compiles the code with `tf.function` internally, which means it takes a slightly different path than `model.call`, which is eager, and thus some small rounding errors can occur between the two implementations)

I'm going to close this issue for now, but if you think there are bugs in the code and this reply hasn't addressed them, feel free to re-open it with a new comment!

@Rocketknight1  Thanks a lot for your detailed explanation, it's quite clear for me.
Wish you have a good day.

by the way, the sequence for `predict` should be `model.predict([encoded_input["input_ids"],encoded_input["attention_mask"],encoded_input["token_type_ids"]])`