[Bug]? how does the tokenizer encode the special tokens? 
Author: vpegasus
Pull request: no

### System Info

transformer version 4.28.1

### Who can help?

@ArthurZucker 
hi, maybe, the following issue should be asked here?
[[Bug]? how does the tokenizer encode the special tokens? #1263](https://github.com/huggingface/tokenizers/issues/1263)


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Hi, all, I used the tokenzier to process  data for llama model(already converted to hf formated) and set:
```python
tokenizer = AutoTokenizer.from_pretrained(llama_model_id, model_max_length=1024, padding_side='right',
                                              trust_remote_code=True)
tokenizer.add_special_tokens(  
            {
                "eos_token": "
",
                "bos_token": "
",
                "unk_token": "
",
            })
tokenizer.pad_token = tokenizer.eos_token
```
when tokenizing  a piece of text with an eos_token:

```python
tokenizer(['ASSISTANT: Hello!
']) # there is no space between ! and 
.
```

```
output:
{'input_ids': [[1, 319, 1799, 9047, 13566, 29901, 15043, 29991, 829, 29879, 29958]], 
 'token_type_ids': [[0, 0, 0, 0, 0, 0,0, 0, 0, 0, 0]], 
 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}
```
The `eos_token: 
` is encoded to ` 829, 29879, 29958` which means `
` is regarded as `</`,`s` and `>`.


```python
tokenizer(['ASSISTANT: Hello! 
'])  # there is a space between ! and 
.
```

```
output:
{'input_ids': [[1, 319, 1799, 9047, 13566, 29901, 15043, 29991, 2]],
  'token_type_ids': [[0, 0, 0, 0, 0, 0, 0, 0, 0]],
  'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1]]}
```
in this time, `
` is encoded correctly  (token id is 2).

As description above, does this mean we should add a space between text and `eos_token`? however, I find many popular projects like `Alpaca` concatenate text with `eos_token` without a space.

I previously thought tokenizer encode text in a greedy style, the `eos_token`  would be encoded correctly with or without a space. However, the experiments above seemed to not support my opinion.

could anyone help me, if there is something misunderstood by me? thx.


----

After some other experiments, I found some weird thing:



```python
tokenizer('我是谁')
output:
'input_ids': [1, 29871, 30672, 30392, 235, 179, 132] 
```
1 is bos_token_id, 29871 is the token id of ''

```python
tokenizer('我是谁
')
output:
'input_ids': [1, 29871, 30672, 30392, 235, 179, 132, 829, 29879, 29958]

tokenizer('who are you
')
output:
'input_ids': [1, 1058, 526, 366, 829, 29879, 29958] # there is no 29871.
```

when add a space ` ` between `谁` and `
`.
```python
tokenizer('我是谁 
') 
output:
'input_ids': [1, 29871, 30672, 30392, 235, 179, 132, 2] # the `
` is encoded correctly
```


when decode `[1, 29871, 30672, 30392, 235, 179, 132, 2] `
```
tokenizer.decode([1, 29871, 30672, 30392, 235, 179, 132, 2])
output:
'
 我是谁
' 
```
 the space ` ` is ignored!

When manually add token id 29871：
```
tokenizer.decode([1, 29871, 30672, 30392, 235, 179, 132, 29871, 2])
output:
'
 我是谁 
' 
```
this time, there is a space  ` ` between `谁` and `
`.

Does these experiments above means encode, decode methods are not completely Reciprocal reversible operation?




### Expected behavior

does above experiments show bugs? if not, how should I understand these? thanks

Comments:
#23818 

> #23818

@jiangwy99 
thanks very much, when set `use_fast=False`, this indeed encode 
 correctly, whether the space exists.

However,  
```python
tokenizer(['who are you', '你是谁'])
output:

outputs:
[
[1, 1058, 526, 366], 
[1, 29871, 30919, 30392, 235, 179, 132]
]
```
the space ` ` in front Chinese characters still exists.



> > #23818
> 
> @jiangwy99 thanks very much, when set `use_fast=False`, this indeed encode correctly, whether the space exists.
> 
> However,
> 
> ```python
> tokenizer(['who are you', '你是谁'])
> output:
> 
> outputs:
> [
> [1, 1058, 526, 366], 
> [1, 29871, 30919, 30392, 235, 179, 132]
> ]
> ```
> 
> the space ` ` in front Chinese characters still exists.

That's quite a problem. Your analysis of the problems on the tokenizer is more comprehensive than mine, and I look forward to these issues being resolved.

Hey, I basically answered in #23818, this is pretty much the same