Pipeline for Text Generation: GenerationPipeline
Author: enzoampil
Pull request: yes

### This PR implements a text generation pipeline, `GenerationPipeline`, which works on any `ModelWithLMHead` head, and resolves issue #3728

This pipeline predicts the words that will follow a specified text prompt for autoregressive language models. I've registered it to the pipeline function using `gpt2` as the default `model_type`.

The implementation is based on the approach taken in [run_generation.py](https://github.com/huggingface/transformers/blob/master/examples/run_generation.py), which means the forward pass uses the `PreTrainedModel.generate()` method in [modeling_utils.py](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_utils.py#L116:1), as recommended to me by @julien-c and @patrickvonplaten .

Sample code:

```
# Pip install
# If you're using Google Colab, make sure to reset runtime after installing
!pip install -e git+git://github.com/enzoampil/transformers.git@generation_pipeline#egg=transformers

# Pipeline uses `gpt2` by default
from transformers import pipeline
gpt = pipeline('generation', num_return_sequences=1, length=40)
gpt("Natural language processing is amazing!")

# ["Natural language processing is amazing! Just take a look at these some of the features. Go off and read up on them all…\n\nSay hello to the world of BitLocker with ES2016. It's a game."]
``` 

**Google Colab tutorial [here](https://colab.research.google.com/drive/1PHmYRpgzdMeSR68i4w5tPfUjlv0npCQz) for running GenerationPipeline for the following LM models:**

1. OpenAI GPT
2. OpenAI GPT-2
3. Transformer-XL
4. XML
5. XLNet
6. T5
7. CTRL (colab RAM is too small to read this model)

For context, I also plan to use the above `GenerationPipeline` for my Humor Generation Bot ([issue](https://github.com/enzoampil/tito-joker/issues/29)).

I'm very keen to get feedback for the above, so please let me know if I should change anything, or perform additional steps to bring its quality to an acceptable level.

Comments:
Hi @enzoampil, 

Thanks again for the PR - I reviewed it. I think we can start by deleting a lot of the code and keeping it simple. It can be quite hard to get used to all the "under-the-hood" behavior that happens in pipelines. I think we should stick to the format that was used for the `summarization` pipeline e.g. and we shouldn't need a `__init__` fn in the beginning. 

We should also add tests for generation in `tests/test_pipelines.py` .

Let me know if the comments are clear! If the PR seems too much, just let me know - I can help then a bit as well :-) 

Hi @patrickvonplaten , thank you for the very clear comments and concrete changes requested. I will work on this by this weekend :)

That sounds great :-) 
Don't worry yet about adding the `task_specific_params` to each of the models configs - I will do this at a later stage! Regarding the tests, you can use the same test logic that was used for `summarization` :-) 

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3758?src=pr&el=h1) Report
> Merging [#3758](https://codecov.io/gh/huggingface/transformers/pull/3758?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/9180e3fa4a396fc5a066ab88b85445e26d69bc4c&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3758/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3758?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3758   +/-   ##
=======================================
  Coverage   78.58%   78.58%           
=======================================
  Files         106      106           
  Lines       18003    18003           
=======================================
  Hits        14148    14148           
  Misses       3855     3855           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3758?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3758?src=pr&el=footer). Last update [9180e3f...9180e3f](https://codecov.io/gh/huggingface/transformers/pull/3758?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@patrickvonplaten I've applied the requested changes (all tests passing), including the pipeline tests, and changing the class name to `TextGenerationPipeline`, as recommended by @julien-c and @thomwolf .

Keen to get feedback on this! Thanks :smile:

Great work @enzoampil! We will add the generation pipeline in the next release and I think it's gonna be a very useful and widely used feature! 

@enzoampil, @thomwolf, @LysandreJik, @julien-c  - After the changes requested above, I'd be happy to merge from my side. 
A couple of things that were added in this PR and would need discussion are:
1. `XLM` is not supported in this `TextGeneration` pipeline. I played around with multiple models of `XLM` and never had any reasonable results for generation, so I think it's best to remove it here. The other models `GPT1, GPT2, CTRL, XLNet, Transfo-XL` work very well. 
2. `XLNet` and `Transfo-XL` need a padding text to work well. This is added and removed afterward, so the user doesn't notice it at all. In a follow-up PR we could maybe add a warning about it.
3. We can now also introduce `text_generation` task_specific_params to the dicts (Transfo-XL and XLNet need a new min and max length) - I can do this after the PR is merged. 
4. I think we can remove the `run_generation` script then completely no? 
5. Tensorflow tests should also be added in a future PR (or feel free to add them here @enzoampil - forgot to mention that in the review actually)

Thank you so much @patrickvonplaten ! Will apply the rest of the changes within the next day or two :) 

Awesome, thanks @enzoampil! LGTM.

Wanted to thank you guys again for guiding me through my first (relatively) big PR for `transformers` @patrickvonplaten @julien-c @thomwolf 😄 

The work of HuggingFace with both the implementation and democratisation of state of the art NLP is something I deeply resonate with. I've been an industry practitioner of NLP for the passed few years and `transformers` has really helped me a lot.

With this, I've recently decided to dedicate a large chunk of my open source time contributing to this package! Looking forward to helping out more and more.

I will keep an eye out for specific issues that I can help out with, and am very open to advice on how I can help in a way that's most useful 🙂 

@LysandreJik 

1. I think we can fix the XLNet generation issue by setting `max_length` as the max length of the *generated* text, rather than the full text. This can be implemented by ensuring that we add the number of tokens in `prompt_text` to the `max_length` argument. Something like below:
```
max_length = max_length + len(input_ids.squeeze())
```

However, this may require that we set `max_length` as an explicit argument for `__call__`, rather than as part of `generate_kwargs`. @patrickvonplaten Do you think this makes sense to do?

2. Sure thing, will work on adding `TextGenerationPipeline` to `./docs/source/main_classes/pipelines.rst`

Sorry, I forgot to add the `max_length` as generation task specific params to the XLNet and TransfoXL configs. I will do this now.

@enzoampil - Sorry for fiddling in your code so much :D 
It's actually not as easy as I thought to have the final output correct for XLNet and Transfo-XL. My commits suggestions now should work. You should run `make style` once they are integrated :-)  

Maybe we should also add an optional `padding` argument to the `__call__` function that overwrites `self.PADDING` for XLNet and Transfo-XL @LysandreJik. But we can do this in a separate PR @enzoampil - let's try to merge this one first.

> Sorry, I forgot to add the `max_length` as generation task specific params to the XLNet and TransfoXL configs. I will do this now.

Ok added it to the config of Transfo-XL and XLNet 

@LysandreJik @thomwolf, we also might want to discuss the default generation params for each model. I think it might e.g. be better to set `do_sample=True` for all models that can generate.

I don't have any strong opinions on whether we should sample or not; However, I think whatever the choice we should make sure that it is explicit in the pipeline documentation that we may control it from the pipeline directly. 

Maybe a link linking to the `generate` method would do the trick, alongside a small explanation that all kwargs will be passed to this underlying method.

@patrickvonplaten  Ran `make_style` and just fixed a minor bug from the `generation` line I think being accidentally taken out from one of your prior [commits](https://github.com/huggingface/transformers/pull/3758/commits/29ce6d82e835e1225c26b5cc4c4ce9f6fe1451ff). The pipeline seems to work fine now :smile:

Also, not sure if this is specific to this PR, but there are tests that are suddenly returning an error for the lines that contain `self._create_and_check_torchscript(config, inputs_dict)`.

Sample error:
```
_____________ AlbertModelTest.test_torchscript_output_hidden_state _____________
[gw7] linux -- Python 3.7.7 /usr/local/bin/python

self = <tests.test_modeling_albert.AlbertModelTest testMethod=test_torchscript_output_hidden_state>

    def test_torchscript_output_hidden_state(self):
        config, inputs_dict = self.model_tester.prepare_config_and_inputs_for_common()
    
        config.output_hidden_states = True
>       self._create_and_check_torchscript(config, inputs_dict)

tests/test_modeling_common.py:197: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_modeling_common.py:206: in _create_and_check_torchscript
    model = model_class(config=configs_no_init)
/usr/local/lib/python3.7/site-packages/transformers/modeling_albert.py:455: in __init__
    self.init_weights()
/usr/local/lib/python3.7/site-packages/transformers/modeling_utils.py:392: in init_weights
    self.apply(self._init_weights)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py:289: in apply
    module.apply(fn)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py:289: in apply
    module.apply(fn)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py:290: in apply
    fn(self)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = AlbertModel(
  (embeddings): AlbertEmbeddings(
    (word_embeddings): Embedding(99, 128, padding_idx=0)
    (position_...     )
      )
    )
  )
  (pooler): Linear(in_features=36, out_features=36, bias=True)
  (pooler_activation): Tanh()
)
module = Embedding(99, 128, padding_idx=0)

    def _init_weights(self, module):
        """ Initialize the weights.
        """
        if isinstance(module, (nn.Linear, nn.Embedding)):
            # Slightly different from the TF version which uses truncated_normal for initialization
            # cf https://github.com/pytorch/pytorch/pull/5617
>           module.weight.data.normal_(mean=0.0, std=self.config.initializer_range)
E           RuntimeError: normal_ expects std > 0.0, but found std=0

/usr/local/lib/python3.7/site-packages/transformers/modeling_albert.py:377: RuntimeError
________________________ BertModelTest.test_headmasking ________________________
[gw1] linux -- Python 3.7.7 /usr/local/bin/python

self = <tests.test_modeling_bert.BertModelTest testMethod=test_headmasking>

    def test_headmasking(self):
        if not self.test_head_masking:
            return
    
        global_rng.seed(42)
        config, inputs_dict = self.model_tester.prepare_config_and_inputs_for_common()
        global_rng.seed()
    
        config.output_attentions = True
        config.output_hidden_states = True
        configs_no_init = _config_zero_init(config)  # To be sure we have no Nan
        for model_class in self.all_model_classes:
>           model = model_class(config=configs_no_init)

tests/test_modeling_common.py:260: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.7/site-packages/transformers/modeling_bert.py:619: in __init__
    self.init_weights()
/usr/local/lib/python3.7/site-packages/transformers/modeling_utils.py:392: in init_weights
    self.apply(self._init_weights)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py:289: in apply
    module.apply(fn)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py:289: in apply
    module.apply(fn)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py:290: in apply
    fn(self)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = BertModel(
  (embeddings): BertEmbeddings(
    (word_embeddings): Embedding(99, 32, padding_idx=0)
    (position_embed...
  (pooler): BertPooler(
    (dense): Linear(in_features=32, out_features=32, bias=True)
    (activation): Tanh()
  )
)
module = Embedding(99, 32, padding_idx=0)

    def _init_weights(self, module):
        """ Initialize the weights """
        if isinstance(module, (nn.Linear, nn.Embedding)):
            # Slightly different from the TF version which uses truncated_normal for initialization
            # cf https://github.com/pytorch/pytorch/pull/5617
>           module.weight.data.normal_(mean=0.0, std=self.config.initializer_range)
E           RuntimeError: normal_ expects std > 0.0, but found std=0

/usr/local/lib/python3.7/site-packages/transformers/modeling_bert.py:525: RuntimeError
```

Those test are probably falling because the new Pytorch version was released. Can you just tense your branch in master?: 
```
$ git fetch upstream
$ git rebase upstream/master
```
(Assuming that you added the master branch as a remote branch  "upstream").

The test should then pass :-)

@patrickvonplaten Apologies, I'm having issues with the rebase suggested above.

I initially tried it but ended up showing up as a co-committer with the rebased commits, which explains why I performed a `force-push` above to revert the rebase. It *might* be related to an issue I'm having where I'm forced to do a `rebase --skip` with each of the conflicts (same situation as [here](https://stackoverflow.com/questions/14410421/git-rebase-merge-conflict-cannot-continue)).

May I please ask for some assistance / advice with this?

Once again, thanks so much! Looking forward to contributing more in the future 😄@patrickvonplaten @julien-c