Index out of range when doing manual testing for TFBertModel
Author: lichenhao608
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.6.0
- Platform: Windows 10
- Python version: 3.8.5
- PyTorch version (GPU?):
- Tensorflow version (GPU?):2.4.1
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik @Rocketknight1 
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...):
TFBertModel, BertTokenizer

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

Want to check what input BertModel will take, so I tested with code
```
tokenizer = transformers.BertTokenizer.from_pretrained('bert-base-uncased')
model = transformers.TFBertModel.from_pretrained('bert-base-uncased')
model(**tokenizer(['i, ne']))
```
This gives an error
```
  File "C:\Users\liche\anaconda3\lib\site-packages\transformers\models\bert\modeling_tf_bert.py", line 887, in call
    outputs = self.bert(
  File "C:\Users\liche\anaconda3\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1012, in __call__
    outputs = call_fn(inputs, *args, **kwargs)
  File "C:\Users\liche\anaconda3\lib\site-packages\transformers\models\bert\modeling_tf_bert.py", line 645, in call
    embedding_output = self.embeddings(
  File "C:\Users\liche\anaconda3\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1012, in __call__
    outputs = call_fn(inputs, *args, **kwargs)
  File "C:\Users\liche\anaconda3\lib\site-packages\transformers\models\bert\modeling_tf_bert.py", line 199, in call
    position_embeds = tf.tile(input=position_embeds, multiples=(input_shape[0], 1, 1))
IndexError: list index out of range
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
I think my code should produce something, but not an error. So I check the code and find that `input_ids` for `TFBertEmbedding` is a () shape Tensor. Then tracing back to where produce it, and I finally end at function `input_processing` in modeling_tf_utils.py, and find that input_ids are a list of five Tensors, each is a shape of (). 

So here comes the problem. As shown in the documentation, `TFBertModel` takes input_ids as a type of `TFModelInputType`, which only accepts either Tensor or numpy array or a list of them. My tokenizer produces ` [[101, 1045, 1010, 11265, 102]]` as input_ids. If manually converting it whole to Tensor or numpy array, I will get a shape (1,5) variable and can be successfully fed to the model and get outputs. However, if directly fed the dict to the model (as the code above), since the model only accepts Tensor or numpy array type, so it will convert the list to a type it accepts. Then it doesn't correctly covert the whole list to tensor, instead, it converts each individual element, i.e. integers, as the accepted type. And after `input_processing`, not the list of Tensor is fed to `TFEmbedding` but each individual empty shape integer Tensor. And it raises the error.

It can be solved by converting it to Tensor as desired before calling the model but my code is logically correct, and expect it works.

Comments:
Hi, you're quite right with your diagnosis. The problem is that by default, the tokenizer creates a dict of Python lists, not Tensors. Our models don't really understand those list inputs, and so you get errors. 

You already found the solution of converting those lists to TF Tensors or Numpy arrays, but there is an easier way - just tell the Tokenizer that you want array output. Then you will get the dict you want, and the rest of your code will work correctly. Here's an updated code sample that returns a dict of Numpy arrays instead:

```
tokenizer = transformers.BertTokenizer.from_pretrained('bert-base-uncased')
model = transformers.TFBertModel.from_pretrained('bert-base-uncased')
model(**tokenizer(['i, ne'], return_tensors='np'))
```

Thanks, that really helps!