Tokenizers can not pad tensorized inputs
Author: Impelon
Pull request: no

### Who can help

@SaulLu or possibly @patrickvonplaten, who committed 538b3b46075dce22a61aeeafd2131979150359a9.

## To reproduce

Observe the following MWE:

```python3
>>> import transformers
>>> tokenizer = transformers.BartTokenizer.from_pretrained("facebook/bart-large")
>>> tokenizer.pad(tokenizer("hello world")) # works as expected
{'input_ids': [0, 42891, 232, 2], 'attention_mask': [1, 1, 1, 1]}
>>> tokenizer.pad(tokenizer("hello world", return_tensors="pt")) # does not work as expected
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../python3.8/site-packages/transformers/tokenization_utils_base.py", line 2742, in pad
    if not required_input:
RuntimeError: Boolean value of Tensor with more than one value is ambiguous
>>> tokenizer.pad(tokenizer("hello world", return_tensors="np")) # same problem here
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../python3.8/site-packages/transformers/tokenization_utils_base.py", line 2742, in pad
    if not required_input:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

The issue is not specific to the `BartTokenizer`, as it is present in `PreTrainedTokenizerBase`.
I'm quite sure this affects tensorflow's tensors as well.

## Expected behavior

The `pad` method of the tokenizer can handle any type of 'tensor'-type, not just regular lists.

## Explanation

As the exceptions suggest, the problem arises here:
https://github.com/huggingface/transformers/blob/0f69b924fbda6a442d721b10ece38ccfc6b67275/src/transformers/tokenization_utils_base.py#L2766-L2771

At that point, `required_input` points to the entry of `input_ids` (for BART in my case), which may be a tensor/array that cannot be evaluated as a boolean, as one can with a regular list.
I presume that the point of the problematic code snipped is to provide correct results for empty inputs.
In that case `if len(required_input) <= 0:` or similar would probably be a solution that works in all cases, but I may misunderstand what the code is designed to archive.

## Further Details

<details>
<summary>These are probably not that relevant to the problem, so I moved them down here:</summary>

## Environment info

- `transformers` version: 4.15.0
- Platform: Linux-5.13.0-27-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyTorch version (GPU?): 1.10.1+cu102 (False)

## Information

Model I am using: BART, PEGASUS

The problem arises when using:
* [x] my own modified scripts

I am implementing the pre-training procedures for BART and PEGASUS similar to the original papers, which require custom masking algorithms that need to run before converting tokens to ids and adding special tokens.
Because of this I'm only adding padding later on and stumbled across this problem.
</details>



Comments:
Hey @Impelon,

Coulcd you clarify why you would do:

```
tokenizer.pad(tokenizer("hello world")) # works as expected
```

instead of 

```python
tokenizer("hello world", padding=True, truncation=True)
```

?

The `__call__` method of `tokenzier` internally calls the `.pad(...)` method so it'd be a bit weird to wrap `tokenizer(...)` into `tokenizer.pad(...)`.

Hey @patrickvonplaten,
yes, you are right,`tokenizer.pad(tokenizer("hello world"))` is more or less for demonstration purposes, as that's not actually why I stumbled upon this problem.

As mentioned in _further details_, I'm in the process of implementing the pretraining procedures for BART and PEGASUS more or less as described in the papers.
For this I need to be able to work with the (tokenized) texts before they are converted to sequences of ids and special tokens are added (at the very least it makes things much easier). My workflow actually looks more like this:
```python3
tokens = tokenizer.tokenize(sentence)
# do custom masking for pre-training
ids = tokenizer.convert_tokens_to_ids(tokens)
batch = tokenizer.prepare_for_model(ids)
```
I then combine multiple sentences together. Because texts are of different lengths I may need to do some padding afterwards:
```python3
batch = tokenizer.pad(batch)
```
which will fail if I used tensors in `prepare_for_model` before (which I don't need to necessarily).

Truth be told, I'm not sure I necessarily need a fix for this. But this comment and the code following it suggests to me that the `pad(...)` method is expected to work with tensors, so I though I'd report this problem:
https://github.com/huggingface/transformers/blob/0f69b924fbda6a442d721b10ece38ccfc6b67275/src/transformers/tokenization_utils_base.py#L2773-L2775

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.