Malfunctioning of PreTrainedTokenizer's tokenize method
Author: chless
Pull request: no

### System Info

* transformers.__version__
'4.25.1'

I found that at least a tokenizer's tokenized method shows wrong return.
Below is code for reproduce

```python
transformers.__version__
'4.25.1'
tokenizer = transformers.BartTokenizer.from_pretrained("facebook/bart-large")
tokenizer.tokenize("example input")
['example', 'Ġinput']
tokenizer.decode(tokenizer.encode("example input", add_special_tokens=False))
'example input'
```

As you see, if I use "tokenize" method, it prefixes each token with some strange character. However, the encode-decode method gives the correct answer.
If I'm right, should be fixed.



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```
transformers.__version__
'4.25.1'
tokenizer = transformers.BartTokenizer.from_pretrained("facebook/bart-large")
tokenizer.tokenize("example input")
['example', 'Ġinput']
tokenizer.decode(tokenizer.encode("example input", add_special_tokens=False))
'example input'
```

### Expected behavior

```
transformers.__version__
'4.25.1'
tokenizer = transformers.BartTokenizer.from_pretrained("facebook/bart-large")
tokenizer.tokenize("example input")
['example', 'input']
```

Comments:
Hi @chless, thanks for raising this issue. 

The `Ġ` symbol is a special character which represents a space. The reason it's seen here is that the sentence is tokenized into `"example"` and `" input"` i.e.  `Ġ` indicates there's a space in front of `input`.  We see the symmetric if the words are reversed:

```py
>>> tokenizer = BartTokenizer.from_pretrained("facebook/bart-large")
>>> tokenizer.tokenize("input example")
['input', 'Ġexample']
```

The reverse mapping, `tokenizer.decode(tokenizer.encode(...))`, is consistent as a space is added in front of `input` in the decoded sentence.

More discussion about the `Ġ` symbol and tokenization can be found [here in the forums](https://discuss.huggingface.co/t/bpe-tokenizers-and-spaces-before-words/475/2?u=amyeroberts). 

It looks like the behavior you're seeing is actually expected, and the Ġ symbol is a special character used by the tokenizer to represent spaces.

When you call tokenizer.tokenize("example input"), the tokenizer is splitting the input string into two tokens: "example" and "Ġinput". The Ġ symbol in front of "input" indicates that there is a space between "example" and "input".

Similarly, when you call tokenizer.decode(tokenizer.encode("example input", add_special_tokens=False)), the tokenizer is encoding the string into two tokens: "example" and "Ġinput", and then decoding those tokens back into the original string with a space between "example" and "input".

So the behavior you're seeing is actually expected, and there's no need to fix anything. If you want to remove the Ġ symbol, you can simply join the tokens returned by tokenizer.tokenize with spaces using the .join() method. 
For example:
tokens = tokenizer.tokenize("example input")
string = " ".join(tokens)

This should give you the output without  the Ġ symbol 


Thanks for your information.
Now I understand why the Ġ symbol comes, and it is proper usage.
I will close this issue.