feat: allow prefix for any generative model
Author: borisdayma
Pull request: yes

Allow `padding_text` for any generative model.

### Motivations

* some models require some text at the beginning such as GPT-2 which should always have a white space. GPT-2 model could now have `config.padding_text` set to `" "` to ensure this is done
* this argument adds more customization to huggingface inference widget (related issue #5553). For huggingtweets models I can set `model.config.task_specific_params['text-generation']['padding_text'] = '
 '`

See [example](https://gist.github.com/borisdayma/89aaf1587390340add44c3c7081bffcd) of use of `padding_text` to create negative/positive sentences.

*Side note*: this is not completely obvious that a leading space should always be added for GPT-2 as `" \n This"` is not tokenized the same as `" \nThis"` which GPT-2 may have seen more in its training data (unless white spaces are added after new lines as well)

### Future improvements

In the future, we should also add `prefix` and `suffix`. This could let us do things similar to GPT-3 with the inference widget and specific tasks:
* `padding_text` would be used to initialize tasks (ex: 10 examples conversation with a bot). This is always cropped out and never displayed in the output
* `prefix` would add required text (ex: `" Human: "`). This is displayed in output.
* `suffix` would add required text after prompt (ex: `" \n AI: "`). This is displayed in output.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5885?src=pr&el=h1) Report
> Merging [#5885](https://codecov.io/gh/huggingface/transformers/pull/5885?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/42fddacd1cac3cc57c3326aa51a409f5090b1261?el=desc) will **decrease** coverage by `0.88%`.
> The diff coverage is `75.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5885/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5885?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5885      +/-   ##
==========================================
- Coverage   78.47%   77.58%   -0.89%     
==========================================
  Files         157      157              
  Lines       28569    28571       +2     
==========================================
- Hits        22420    22168     -252     
- Misses       6149     6403     +254     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5885?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/pipelines.py](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9waXBlbGluZXMucHk=) | `80.50% <75.00%> (+0.55%)` | :arrow_up: |
| [src/transformers/modeling\_reformer.py](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yZWZvcm1lci5weQ==) | `16.87% <0.00%> (-79.30%)` | :arrow_down: |
| [src/transformers/configuration\_reformer.py](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3JlZm9ybWVyLnB5) | `21.62% <0.00%> (-78.38%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9vcGVuYWkucHk=) | `22.58% <0.00%> (-72.26%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_flaubert.py](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mbGF1YmVydC5weQ==) | `24.53% <0.00%> (-63.81%)` | :arrow_down: |
| [src/transformers/modeling\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19tYXJpYW4ucHk=) | `60.00% <0.00%> (-30.00%)` | :arrow_down: |
| [src/transformers/activations.py](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9hY3RpdmF0aW9ucy5weQ==) | `85.00% <0.00%> (-5.00%)` | :arrow_down: |
| [src/transformers/tokenization\_dpr.py](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fZHByLnB5) | `53.15% <0.00%> (-4.51%)` | :arrow_down: |
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `95.05% <0.00%> (-0.35%)` | :arrow_down: |
| [src/transformers/tokenization\_utils\_base.py](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHNfYmFzZS5weQ==) | `93.49% <0.00%> (-0.28%)` | :arrow_down: |
| ... and [13 more](https://codecov.io/gh/huggingface/transformers/pull/5885/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5885?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5885?src=pr&el=footer). Last update [42fddac...00efd7d](https://codecov.io/gh/huggingface/transformers/pull/5885?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Another solution would be to just allow `prefix` element like in `TranslationPipeline`. The main difference will be that it becomes part of the display and it should not affect too much future plans such as suggested above with `padding_text`, `prefix` and `suffix`.

I still think that `padding_text` as proposed here will be useful in the future to easily provide "GPT-3 type" widgets.

I updated my [example](https://gist.github.com/borisdayma/89aaf1587390340add44c3c7081bffcd) of the feature with the case when it loads model config.

Let me know if I can provide any additional information or make any changes

Just checking if you need anything else from me @julien-c

I’m off for a couple of days so I’ll let others chime in

Enjoy your time off!

@julien-c @mfuntowicz sorry I'm just pinging.
I have a [demo](https://huggingface.co/huggingtweets/julien_c) I'm excited about and this feature just makes predictions so much better.
*Note: not on master branch - dev version [here](https://colab.research.google.com/github/borisdayma/huggingtweets/blob/feat-artifacts/huggingtweets-demo.ipynb)*

Thoughts @patrickvonplaten @sshleifer? (text generation pipeline option)

This PR adds the padding functionality from `run_generation` to pipelines (and allow for custom text).

However the main idea would be to eventually allow "GPT-3 style" API's.

![image](https://user-images.githubusercontent.com/715491/91374590-2489fb80-e7de-11ea-8872-6f5ea8aeb2c7.png)

It could be renamed "prefix" but I suggest to make a distinction between padding and prefix where you would potentially want to display prefix & suffix in an interface (for example huggingface widget) but not necessarily the "padding".

This kind of API could be used for T5 and can let people create more modular inference examples.

@sshleifer I gave more thoughts to it and I agree we should just name it `prefix` and maybe add a "deprecated warning" on `run_generation`.
It should be sufficient for now and we will always be able to add a more complex behavior in the future if necessary.

Note: I don't understand the CI errors, quality is good on my side and the other tests errors seem unrelated to this PR

style: you need to rebase and then `pip install black --upgrade`
Other CI failures are spurious, you can ignore them.

My last request would be to add test coverage for the new argument in test_pipelines.py.
see:
```
test_torch_text_generation
test_tf_text_generation
```



Changes:

* renamed `padding_text` to `prefix` in pipelines and `run_generation.py` (with warning of deprecated argument)
* since PR was made, we cannot tokenize empty text anymore so we now check for empty "prefix"
* since PR was made, we cannot pass extra args to `generate` so we explicitly add `prefix` to pipeline `__call__` method
* added tests for pipeline using prefix

Notes:

* we set default `prefix` to `None` (vs `""`) as we could pass `""` to overwrite model default value.
* sorry I should have done a merge instead of rebase as it changes the history. New commits start at "feat: rename padding_text to prefix"

Thanks @LysandreJik. Yes I noticed later that my results were better without the initial space as well.

Intuitively it seemed to me like a waste of encoding differentiating between having a space or not (for example even after new lines) as the start of a document can simply be identified by the special token.
I thought there were more examples without the space within the original training set which would help my fine-tuning, as long as I do the same both for training and inference.
In the end, I get my best results with just `
` between each sample and stripping every extra space.

@LysandreJik let me know if anything else is required on my side.
Since it's a relatively large PR it can be hard to remain in sync with master.

I think we can merge this: @LysandreJik I leave it to you to merge :-) 

+ @mfuntowicz can you update the transformers dependency in the inference API when this lands on master, so that @borisdayma can showcase his magic ;)

@mfuntowicz can you let me know when you update the inference API so I do some tests on the widget?

@borisdayma Everything has been deployed on the inference API, servers are up. 

Thanks for taking care of this 💪 🙏 

That was fast! Thanks it looks to be working great!