How to separate multiturn dialog context in blenderbot?
Author: LiShaoyu5
Pull request: no

Hello. I want to perform multiturn dialog with blenderbot model. I followed the [documentation](https://huggingface.co/docs/transformers/master/en/model_doc/blenderbot#transformers.BlenderbotForConditionalGeneration) and here's the code:
```
chat_history = []
uttr_sep = '
 
'

while(True):
    UTTR_input = input('User: ')
    chat_history.append(UTTR_input)
    UTTR = uttr_sep.join(chat_history)
    print('Input: ' + UTTR)

    inputs = tokenizer([UTTR], return_tensors='pt')
    reply_ids = model.generate(**inputs)
    REPLY = tokenizer.batch_decode(reply_ids, skip_special_tokens=True)[0]
    print("Bot:\n" + REPLY)

    chat_history.append(REPLY)
```

But model treats context as user's input and sometimes `
 
` shows in model's responses.

In [this issue](https://github.com/huggingface/transformers/issues/9365), there are discussions about what to use to separate multiturns including `
 
`, `</sep>`, `\n`, `\t`, but no conclusions.




Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I'm facing the same. Maybe @patrickvonplaten or @sshleifer can give us a hint on which separator to use :) 

Gently pinging @Narsil and @patil-suraj here

Hi @LiShaoyu5 ,

A good first approximation is to use the ConversationalPipeline actually https://huggingface.co/docs/transformers/master/en/main_classes/pipelines#transformers.ConversationalPipeline

```python
pipe = pipeline(model="facebook/blenderbot-1B-distill")
conversation_1 = Conversation("Going to the movies tonight - any suggestions?")
pipe(conversation_1)
print(conversation_1)
conversation_1.add_user_input("And what about the next day ?")
pipe(conversation_1)
print(conversation_1)
```

`ConversationalPipeline` tries to use the very special (and private) method on the tokenizer `_build_conversation_input_ids`: https://github.com/huggingface/transformers/blob/master/src/transformers/models/blenderbot/tokenization_blenderbot_fast.py#L79

Which allows for model specific things like locution separator.

This code will work for `BlenderBot` not `BlenderBotSmall` (which has a very different architecture and separation).
The source for the information of `"  "` (double space) separation of locutors was taken directly from `fairseq`. I couldn't however make it into a proper link, since the code is pretty spread out. Overall good care was taken at the time to make sure the raw outputs of the model ended up being exactly similar.

Keep in mind, that `transformers` does not contain the full code of `blenderbot`. And while the model is the same, the actual `fairseq` implementation contains many more hardcoded things like the turn separator, some are even trickier like a banned word list, or even smaller models to assess the output of the model and discard it if it is deemed "injurious" for instance.
On the other end, using `transformers` to fine-tune should be easier.



Hello @Narsil and thanks for your help! 

i dont think this should be closed yet. first sorry for bad writing, i only have 1 hand now due to surgery so i type as i can. i tried the suggested and i think there is a bug somewhere, when using conversational pipeline the response to: "Hey, how are you doing?" is " I'm doing well, thank you. How are you? I am doing?" with blenderbot3b while using the model without the pipeline returns "I'm doing well. How are you? I am good..". I think this is strange and we should look into it, specially because the answer provided by the pipeline is deceptive. I'm also using 2 spaces when working without the pipeline...