BlenderbotSmall incorrect usage of start and end tokens
Author: xenova
Pull request: no

### System Info

- `transformers` version: 4.27.2
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.3
- Huggingface_hub version: 0.12.0
- PyTorch version (GPU?): 1.13.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

@ArthurZucker @younesbelkada @Narsil

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

As stated in the documentation: https://huggingface.co/docs/transformers/model_doc/blenderbot-small#transformers.BlenderbotSmallForConditionalGeneration.forward.example
the model should use `
` and `
` for separating the user input and response:
```python
from transformers import AutoTokenizer, BlenderbotSmallForConditionalGeneration

mname = "facebook/blenderbot_small-90M"
model = BlenderbotSmallForConditionalGeneration.from_pretrained(mname)
tokenizer = AutoTokenizer.from_pretrained(mname)
UTTERANCE = "My friends are cool but they eat too many carbs."
print("Human: ", UTTERANCE)

inputs = tokenizer([UTTERANCE], return_tensors="pt")
reply_ids = model.generate(**inputs)
print("Bot: ", tokenizer.batch_decode(reply_ids, skip_special_tokens=True)[0])

REPLY = "I'm not sure"
print("Human: ", REPLY)

NEXT_UTTERANCE = (
    "My friends are cool but they eat too many carbs.
 
what kind of carbs do they eat? "
    "i don't know much about carbs
 "
    "
 I'm not sure."
)
inputs = tokenizer([NEXT_UTTERANCE], return_tensors="pt")
next_reply_ids = model.generate(**inputs)
print("Bot: ", tokenizer.batch_decode(next_reply_ids, skip_special_tokens=True)[0])
```

However, these tokens are not present in the [vocabulary](https://huggingface.co/facebook/blenderbot_small-90M/blob/main/vocab.json) or [special tokens](https://huggingface.co/facebook/blenderbot_small-90M/blob/main/special_tokens_map.json)

I assume they should be replaced with `__start__` and `__end__`?

---

I have also tried to use the [ConversationPipeline](https://huggingface.co/docs/transformers/v4.27.2/en/main_classes/pipelines#transformers.ConversationalPipeline), and follow steps outlined [here](https://huggingface.co/tasks/conversational#inference), but I always get nonsensical results.

Even when trying the hosted inference API for the model (https://huggingface.co/facebook/blenderbot_small-90M), it either repeats itself, or doesn't follow in conversation.

### Expected behavior

The tokens should be correct, and the chatbot should engage in more realistic conversation

Comments:
Hey! Thanks for reporting, will investigate! 

Hey! When I use the Conversational pipeline I get the same outputs as you. 
Regarding the content of the special tokens, it does not really matter as long as the mapping is correct. If the model's bos_id is 1, then as long as `
` maps to `1` then the generation will make sense. 
And indeed we have:
```python 
In [35]: tokenizer.encode("
")
Out[35]: [3, 330, 1360]

In [36]: tokenizer.encode("__start__")
Out[36]: [1]
``` 
The doc example should be updated, or the tokenizer only should be updated. 
Nice catch (however, this does not seem to really change the output for this example).
Also I am not entirely sure of how these `eos` and `bos` should be used in the context of BlenderBot. They should mark the start and end of a conversation when training the model on different converstations, while `\n` is used to sperate different prompts (so from the user and the bot). 
I could not find anything online, gonna take a while to check with the messy original codebase


Just bumping this again (in response to being marked as stale)

When I checked the original PR that added BlenderBot (could not really find anyting on the original repo ... ) seems like the doc example should be updated to use `__end__` and `__start__`.  See #4803. 

Closed in #24092