BartTokenizerFast cannot decode PyTorch tensors
Author: setu4993
Pull request: no
Labels: name: Fast Tokenizers

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: MacOS and Linux
- Python version: 3.6 and 3.7
- PyTorch version (GPU?): 1.6.0 (no and yes)
- Tensorflow version (GPU?): N/A
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

 examples/seq2seq: @sshleifer 
(Discovered in #6610.)

## Information

Model I am using (Bert, XLNet ...): Bart.

Any Bart model (reproduced with distilbart-cnn-12-6 and distilbart-xsum-1-1.

## To reproduce

Steps to reproduce the behavior:

```python
In [1]: from transformers import BartTokenizerFast, BartForConditionalGeneration

In [2]: model = BartForConditionalGeneration.from_pretrained("sshleifer/distilbart-xsum-1-1")

In [3]: tokenizer = BartTokenizerFast.from_pretrained("sshleifer/distilbart-xsum-1-1")

In [4]: input_ids = tokenizer("This is a test string.", return_tensors="pt")

In [5]: input_ids
Out[5]: {'input_ids': tensor([[   0,  713,   16,   10, 1296, 6755,    4,    2]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1]])}

In [6]: summary_ids = model.generate(input_ids['input_ids'], num_beams=4, max_length=5, early_stopping=True)

In [7]: print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-d476aca57720> in <module>
----> 1 print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids])

<ipython-input-7-d476aca57720> in <listcomp>(.0)
----> 1 print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids])

~/.pyenv/versions/finetuning-bart/lib/python3.7/site-packages/transformers/tokenization_utils_fast.py in decode(self, token_ids, skip_special_tokens, clean_up_tokenization_spaces)
    437         self, token_ids: List[int], skip_special_tokens: bool = False, clean_up_tokenization_spaces: bool = True
    438     ) -> str:
--> 439         text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
    440
    441         if clean_up_tokenization_spaces:

~/.pyenv/versions/finetuning-bart/lib/python3.7/site-packages/tokenizers/implementations/base_tokenizer.py in decode(self, ids, skip_special_tokens)
    265             raise ValueError("None input is not valid. Should be a list of integers.")
    266
--> 267         return self._tokenizer.decode(ids, skip_special_tokens=skip_special_tokens)
    268
    269     def decode_batch(

TypeError:

In [8]:
```

## Expected behavior

Fast tokenizer should be able to decode without producing an error.

Comments:
Faster snippet w same error
```python
import torch
tokenizer = BartTokenizerFast.from_pretrained("sshleifer/distilbart-xsum-1-1")
ids = torch.tensor([1,2,3], dtype=torch.long)
tokenizer.decode(ids)
```

Any update on this?

Hi, neither the slow or fast tokenizers can decode torch tensors. They can decode lists of Python integers, as it is stated in the [docs](https://huggingface.co/transformers/main_classes/tokenizer.html#transformers.PreTrainedTokenizer.decode).

@LysandreJik : That's not entirely accurate.

This works:

```python
from transformers import BartTokenizer
tokenizer = BartTokenizer.from_pretrained("facebook/bart-large")
import torch
ids = torch.tensor([1,2,3], dtype=torch.long)
tokenizer.decode(ids)
```

But this doesn't:

```python
from transformers import BartTokenizerFast
tokenizer = BartTokenizerFast.from_pretrained("facebook/bart-large")
import torch
ids = torch.tensor([1,2,3], dtype=torch.long)
tokenizer.decode(ids)
```

I understand the docs say it should only decode lists, but slow tokenizers do also decode tensors.

The above @setu4993 @LysandreJik  seems to give same code twice. You have to pass in a list of integers into the decode function. Official doc says decode function can process Torch.tensors, but it does not work well in all cases. Instead, give this a try

```python
from transformers import BartTokenizerFast
tokenizer = BartTokenizerFast.from_pretrained("facebook/bart-large")
import torch
ids = torch.tensor([1,2,3], dtype=torch.long)
tokenizer.decode(ids.tolist())
```

If tensor is [[...]], instead of [...], do

```python
from transformers import BartTokenizerFast
tokenizer = BartTokenizerFast.from_pretrained("facebook/bart-large")
import torch
ids = torch.tensor([1,2,3], dtype=torch.long)
tokenizer.decode(*ids.tolist())
```