Whitespace at the start of generated sentence remains or get removed
Author: LalchandPandia
Pull request: no

### System Info

transformers version: 4.34.0
pytorch:1.12.1
python:3.8
I am running llama -2 (Llama-2-7b-hf).
If the generated content has whitepace at the beginning, the tokenizer skips it. For example: if tokens are [29871(id for space), id_for_Hey, id_for_there], the generated sentence is "Hey there". 
tokenizer is transformers.models.llama.tokenization_llama_fast.LlamaTokenizerFast
The above issue occurs on using tokenizer.decode but tokenizer.batch_decode works fine.
What is the reason for this and is it correct behavior?
My code:
```python 
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf", token=access_token)
encoded_input = tokenizer(" Hey there", return_tensors='pt')
#{'input_ids': tensor([[1, 1724, 526, 278, 3405, 29914, 12957, 10350, 310, 21589, 1056, 29875, 29915, 29879, 4593, 4918, 11367, 29973, 673, 29901, 29871, 29896, 29941, 30073, 29900, 29906, 29915, 29941, 29955, 29908, 29940, 29871, 29947, 29900, 30073, 29896, 29941, 29915, 29900, 29947, 29908, 29923, 13, 6747, 287, 491, 29901, 19556, 9716, 472, 3786, 29871, 29896, 29941, 29892]]), 'attention_mask': tensor([[1, 1, 1, 1]])}
print(tokenizer.decode(a[0][20:]))
print(tokenizer.decode(a[0][21:]))
#both the print outputs are same.
```


### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoTokenizer
import torch
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf", token=access_token)
a = torch.Tensor([[1, 1724, 526, 278, 3405, 29914, 12957, 10350, 310, 21589, 1056, 29875, 29915, 29879, 4593, 4918, 11367, 29973, 673, 29901, 29871, 29896, 29941, 30073, 29900, 29906, 29915, 29941, 29955, 29908, 29940, 29871, 29947, 29900, 30073, 29896, 29941, 29915, 29900, 29947, 29908, 29923, 13, 6747, 287, 491, 29901, 19556, 9716, 472, 3786, 29871, 29896, 29941, 29892]])
print(tokenizer.decode(a[0][20:]))
print(tokenizer.decode(a[0][21:]))
#both the print outputs are same.

### Expected behavior

The model should output whitespace at the beginning of the sentence

Comments:
Hey! The reason for this is that the tokenizer by default prepends a space to any input. Thus when decoding it is skipped. 

Hi, Thanks for the clarification. Is there any way to make sure, the space present in the output?. In my use case, I want to check how much probability it assigns to whitespace at each timestep.

You should be looking at the inputs ids not the string I think. This way you get rid of potential decoding issues. 
I'll try to add the `add_dummy_prefix_space` to our tokenizers to seamlessly support these kind of usecases

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.