BertTokenizerFast.convert_tokens_to_string converts ids to string, not tokens to string
Author: HHousen
Pull request: no
Labels: name: Core: Tokenization

# 🐛 Bug

The `BertTokenizerFast.convert_tokens_to_string` function expects a list of integers instead of a list of strings as the function implies. This does not happen for the normal `BertTokenizer`.

The [BertTokenizerFast](https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_bert.py#L550) does not override `convert_tokens_to_string` as it is defined in [tokenization_utils_fast.py](https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_utils_fast.py#L206), which causes this issue. Within `tokenization_utils_fast.py`, the `convert_tokens_to_string` function calls `self._tokenizer.decode` which expects ids (integers not strings).

This issue does not arise when using the normal BertTokenizer because that class overrides `convert_tokens_to_string` as can be seen [here](https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_bert.py#L230). However, the implementation in [tokenization_utils.py](https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_utils.py#L839) is incorrect according to the docstring. The function should return `" ".join(tokens)` by default and the call to `convert_ids_to_tokens` should be removed because that function accepts ids not tokens.


## Information

Model I am using (Bert, XLNet ...): Bert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

```
from transformers import BertTokenizerFast, BertTokenizer

# Error
tokenizer = BertTokenizerFast.from_pretrained("bert-base-uncased")

tokens = tokenizer.tokenize("This is a sentence.")
print(tokens)

output = tokenizer.convert_tokens_to_string(tokens)

# No Error because `convert_tokens_to_string` overridden
tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")

tokens = tokenizer.tokenize("This is a sentence.")
print(tokens)

output = tokenizer.convert_tokens_to_string(tokens)
```

Output:
```
['this', 'is', 'a', 'sentence', '.']
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    output = tokenizer.convert_tokens_to_string(tokens)
  File "/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 209, in convert_tokens_to_string
    return self._tokenizer.decode(tokens, skip_special_tokens=skip_special_tokens)
  File "/home/user/anaconda3/envs/testing/lib/python3.8/site-packages/tokenizers/implementations/base_tokenizer.py", line 267, in decode
    return self._tokenizer.decode(ids, skip_special_tokens=skip_special_tokens)
TypeError: 'str' object cannot be interpreted as an integer
```

## Expected behavior

The `BertTokenizerFast.convert_tokens_to_string` function converts a list of tokens (which are strings) to a single string. 

## Environment info
     
- `transformers` version: 2.11.0
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1+cu101 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Comments:
You're right, this method is actually not provided on the Fast tokenizers and wrongly linked to the `decode()` method.
We should remove it in the short-term.

Do you need it for a specific workflow?

I need to decode a sequence of input ids to a string. However, I cannot use `tokenizer.batch_decode` because I would like to remove all special tokens except for the [SEP] token, which I want to replace with a token that is not in the tokenizer's vocabulary (so I cannot change the input ids before decoding). To do this I modify the functionality of `tokenizer.convert_ids_to_tokens` to create my modified list of tokens, then I run `tokenizer.convert_tokens_to_string` and `tokenizer.clean_up_tokenization` to create my final sequence.

I see.

Can you add your special token at the end of the vocabulary without updating the model inputs and then just replace the SEP token by your new token id prior to decoding?

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
token.add_tokens('[MY_NEW_TOKEN]')
new_token_id = tokenizer.convert_tokens_to_ids('[MY_NEW_TOKEN]')

inputs = tokenizer.encode("hello how are you")
inputs = [new_token_id if tok == tokenizer.sep_token_id else tok for tok in inputs]
decoded_outputs = tokenizer.decode(inputs)
```

> I see.
> 
> Can you add your special token at the end of the vocabulary without updating the model inputs and then just replace the SEP token by your new token id prior to decoding?
> 
> ```python
> from transformers import AutoTokenizer
> 
> tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
> token.add_tokens('[MY_NEW_TOKEN]')
> new_token_id = tokenizer.convert_tokens_to_ids('[MY_NEW_TOKEN]')
> 
> inputs = tokenizer.encode("hello how are you")
> inputs = [new_token_id if tok == tokenizer.sep_token_id else tok for tok in inputs]
> decoded_outputs = tokenizer.decode(inputs)
> ```

Using this example works around this problem and simplifies my code. Thanks.