Addition of a DialoguePipeline
Author: guillaume-be
Pull request: yes

- Addition of a Conversation object to keep track of multi-turn conversation
- Creation of a DialoguePipeline to process Conversations using the history context
- Integration tests for DialoguePipeline using `microsoft/DialoGPT-medium`

Comments:
This is a back-port of https://github.com/guillaume-be/rust-bert/pull/57. I did not implement the ConversationManager as I felt it did not quite fit the general API of this library. I however added the concept of `Conversations` keeping track of past user inputs, generated responses and the history token ids. Conversations include a print option that will display the entire dialogue.

```python
print(conversation)
```
```
Conversation id: 2716da3e-8cde-4071-97bc-218d88764b7b 
user >> What's the last book you have read? 
bot >> The Last Question 
user >> Why do you recommend it? 
bot >> It's a good book. 
```

(ps: note that this example is the response of `DialoGPT-medium` without sampling, which is an interesting coincidence for a computer-generated response)

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5516?src=pr&el=h1) Report
> Merging [#5516](https://codecov.io/gh/huggingface/transformers/pull/5516?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/91cb95461e438dc57555c4f57f8ce95a56328036&el=desc) will **increase** coverage by `1.49%`.
> The diff coverage is `84.34%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5516/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5516?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5516      +/-   ##
==========================================
+ Coverage   78.35%   79.85%   +1.49%     
==========================================
  Files         146      146              
  Lines       26454    26568     +114     
==========================================
+ Hits        20729    21215     +486     
+ Misses       5725     5353     -372     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5516?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `99.24% <ø> (ø)` | |
| [src/transformers/pipelines.py](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9waXBlbGluZXMucHk=) | `79.36% <84.34%> (+0.86%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_flaubert.py](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mbGF1YmVydC5weQ==) | `24.22% <0.00%> (-63.98%)` | :arrow_down: |
| [src/transformers/tokenization\_t5.py](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdDUucHk=) | `71.83% <0.00%> (-23.95%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9ncHQyLnB5) | `71.65% <0.00%> (-23.68%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.46% <0.00%> (+0.25%)` | :arrow_up: |
| [src/transformers/generation\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3V0aWxzLnB5) | `97.11% <0.00%> (+0.28%)` | :arrow_up: |
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `90.36% <0.00%> (+0.40%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9hdXRvLnB5) | `66.66% <0.00%> (+3.63%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9kaXN0aWxiZXJ0LnB5) | `98.79% <0.00%> (+34.61%)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/huggingface/transformers/pull/5516/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5516?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5516?src=pr&el=footer). Last update [91cb954...9734829](https://codecov.io/gh/huggingface/transformers/pull/5516?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@patrickvonplaten Thank you so much for the detailed review! I have pushed some changes addressing most of the suggested changes.

A few points still open for discussion:
- I am still unsure about using the default `max_length` from the model_config, especially since the default model used for the pipeline `DialoGPT` does not give this value and it would default to a very low value.
- I have updated the history truncation to take out entire spans (delimited by EOS tokens). I believe this would lead to a more natural truncation of a conversation (skip the first N turn) rather than possibly cutting an input/response in the middle
- You are right that the API differs a bit from the other pipeline (although the QA pipeline also takes special inputs). Here the challenge is that the conversations are by definition stateful. I actually started the Rust implementation storing the conversations in the model itself, allowing simpler inputs to be passed on. This requires the model to be mutable, which can be problematic for multithreading. Now in Python you may not have this issue, but you may want to deploy an conversational application that scales and deploys multiple conversation pipeline workers / containers (the Rust compiler actually points out to an interesting design decision). If you make these stateful, you'll have to ensure that you always send the conversation to the correct worker which can be sub-optimal for load balancing. I believe carrying the history in the conversation itself would be easier to handle for the full system that may leverage this pipeline. Happy to change if you'd like to allow the user to pass a simpler input, but a conversation `id` would probably be required anyway. In any case happy to expose `Conversation` to the library API if the design remains as is.

Thanks again for the feedback - looking forward to your thoughts on these few points.

Thanks a lot for the detailed answer @guillaume-be,

1. (answering to your comment above) - exactly, I would update the config on AWS 
    and it would then look like this:
```
{
  "activation_function": "gelu_new",
  "architectures": [
    "GPT2LMHeadModel"
  ],
  "attn_pdrop": 0.1,
  "bos_token_id": 50256,
  "embd_pdrop": 0.1,
  "eos_token_id": 50256,
  "initializer_range": 0.02,
  "layer_norm_epsilon": 1e-05,
  "model_type": "gpt2",
  "n_ctx": 1024,
  "n_embd": 1024,
  "n_head": 16,
  "n_layer": 24,
  "n_positions": 1024,
  "resid_pdrop": 0.1,
  "summary_activation": null,
  "summary_first_dropout": 0.1,
  "summary_proj_to_labels": true,
  "summary_type": "cls_index",
  "summary_use_proj": true,
  "vocab_size": 50257,
  "task_specific_params" : {
      "dialogue": { 
          "max_length": 1000
      }
  }
}
```
=> This way in the `__init__` of `Pipeline` the `task_specific_params` will overwrite the default params and the `max_length` will be 1000. I think this has following advantage: If the user wants to define a specific `max_length` via the model config, he could update or create a config to `model.config.task_specific_params.dialogue.max_length = 1000` and does not have to input any generate kwargs to the `__call__` function (otherwise the config would always be overwritten by 1000). As soon as we merge this PR, I can update all the DialoGPT configs with the `task_specific_params`.

2. Very nice! This indeed makes more sense and should lead to good (possible never-ending chats) :-) 

3. I agree - I think I'm also in favor of your design choice here. Sadly, I don't have the necessary Rust knowledge to have a better understanding of the consequences of stateful vs. stateless, but it makes a lot of sense what you say! Maybe looping in @mfuntowicz  here to see what he thinks :-) 

Thanks @guillaume-be for your PR!

I left a bunch of comments, but this is in good shape, can't wait to use it on https://huggingface.co/microsoft/DialoGPT-large 😄 

Re. statefulness, yes, we definitely want to keep a stateless system.

As an example, the way our https://convai.huggingface.co/ demo from last year works (and probably the way our conversational widget powered by this PR will work too 😉) is, we store an array of `Message` on the client side, grow it, and send it with each request. It's text so the bandwidth is not an issue.

Message looks like:

```
interface Message {
	incoming: boolean; // <- is it from bot or user
	content:  string;
}
```



LGTM

@julien-c @patrickvonplaten I believe all comments have been addressed - please let me know if I have missed anything. Just resolved the conflict with master. Getting an error with `code quality`, not quite sure what is wrong as I did not change the torch requirement with this PR.

Given that `patrickvonplaten` is off for one more week I believe, do you want to give this PR a last look-over @sgugger and merge if it's fine?

@LysandreJik Thank you very much for the review. Good catch on the behaviour of the `eos` token cut-off. I have updated based on  your suggestions, and added docstrings to the `Conversation` class. I have also added both `Conversation` and `ConversationalPipeline` to the top-level `__init__` for consistency with the other pipelines.

Thanks for the PR @guillaume-be 
Docstrings could be improved but I'll clean up the docs in the pipelines file soon, so will take of that. For future PRs, please remember that `thing` will render thing in italics in the docs, and not in code (you have to use ``thing`` or :obj:`thing`).

@sgugger Thank you for the review - was indeed a typo on my end. The tests got triggered again and unfortunately a hash verification on torch fails. Could you please restart the build if you have a chance?

This is awesome, congrats everyone on shipping this! 🔥

`test_torch_conversation` and `test_integration_torch_conversation` are broken on github actions CI. Could someone fix or delete? https://github.com/huggingface/transformers/runs/1289790225?check_suite_focus=true

Will take a look.

Should be fixed in https://github.com/huggingface/transformers/pull/7970