correct greedy generation when doing beam search
Author: patrickvonplaten
Pull request: yes

This PR changes the behavior of greedy beam search generation as discussed and wished in #2415 . 
Also two assertion statements are added: 

1. It is not allowed to generate multiple sequences from the same input_ids when greedy generation (`num_return_sequences > 1`, `do_sample=False`, `num_beams` == 1 => `AssertionError`) because it would always lead to the same output sequence for all `num_return_sequences`.

2. It is not allowed to generate more sequences when doing greedy beam serach generation than the number of beams that are used (`num_return_sequences` <= `num_beams`, `do_sample=False` => `AssertionError`) because this is not possible or would also lead to the same output sequences. 

Discussion:

- [x] the generation function becomes bigger and bigger handling more and more exceptions - might need a big refactoring at some point which modularizes it for more flexibility and more readability. Also when thinking about including the encoder-decoder models in the model.generate() function.
Also maybe the `no_beam_search_generation` fn could simply be handled by `beam_search_generation(num_beams=1)` ? 

- [x] beam search when do_sample=True still does not work really (see PR #2317  ). Should discuss how exactly it should be implemented. 

@thomwolf, @LysandreJik, @sshleifer 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3078?src=pr&el=h1) Report
> Merging [#3078](https://codecov.io/gh/huggingface/transformers/pull/3078?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/c0135194ebc5de4b1bbef98b31f9c457a0bf746a?src=pr&el=desc) will **decrease** coverage by `0.99%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3078/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3078?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #3078    +/-   ##
=======================================
- Coverage    77.6%   76.6%    -1%     
=======================================
  Files          98      98            
  Lines       16221   16230     +9     
=======================================
- Hits        12588   12433   -155     
- Misses       3633    3797   +164
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3078?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3078/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `93.02% <100%> (+0.26%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_pytorch\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3078/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9weXRvcmNoX3V0aWxzLnB5) | `8.72% <0%> (-81.21%)` | :arrow_down: |
| [src/transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/3078/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yb2JlcnRhLnB5) | `85.71% <0%> (-10%)` | :arrow_down: |
| [src/transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/3078/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG5ldC5weQ==) | `73.48% <0%> (-2.3%)` | :arrow_down: |
| [src/transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/3078/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19jdHJsLnB5) | `96.03% <0%> (-2.21%)` | :arrow_down: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/3078/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `80.2% <0%> (-1.35%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3078?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3078?src=pr&el=footer). Last update [c013519...5b9164a](https://codecov.io/gh/huggingface/transformers/pull/3078?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


The self-hosted runner tests that fail are:

FAILED tests/test_pipelines.py::MonoColumnInputTestCase::test_tf_feature_extraction
FAILED tests/test_pipelines.py::MonoColumnInputTestCase::test_tf_fill_mask
FAILED tests/test_pipelines.py::MonoColumnInputTestCase::test_tf_ner
FAILED tests/test_pipelines.py::MonoColumnInputTestCase::test_tf_sentiment_analysis

and are all related to memory exhaustion (`Resource exhausted: OOM when allocating tensor ... `). 
The tests are not related to the PR. Not sure what to do @julien-c 


LGTM for merging

Discussed with @thomwolf as well and also agreed that generate() function is not too complex and good as it is now. I will take a closer look at the issue with beam search decoding when `do_sample=True` (PR #2317 ) in a separate PR. Good to merge for me!