Add soft length regulation for sequence generation
Author: kevinpl07
Pull request: yes

# What does this PR do?

This PR enables users to softly regulate the length when using sampling method in model.generate(). We had the use case where we wanted to keep a generated sequence adequately short with no "hard" cutoff - we wanted the model to come to an end without just setting max_length, so that the end_token is also predicted at a meaningful position.

The SoftLengthLogitsProcessor exponentially increases the score of the eos_token_id until it is generated. It can be configured by these parameters inside model.generate():
- length_regulation_start
- length_regulation_factor

I am aware that there is already a length_penalty in place, but as far as I understood it is used in beam search and did not fulfil our need.

Usage:
```python
gen_tokens = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=600, length_regulation_start=300, length_regulation_factor=1.01)
```

Example:
Here are the lengths of 5 samples generated for a length-regulated and default sequence-generation. The goal was to end the sequence between 300 and 400 tokens without a hard cutoff.
Code
```python
print("Regular output with max_length=600")
for index in range(5):
    gen_tokens = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=600, pad_token_id=50256)
    print(gen_tokens.shape[1])

print("--------------")
print("Soft regulation (length_regulation_start=300, length_regulation_factor=1.01):")
for index in range(5):
    gen_tokens = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=600, length_regulation_start=300, length_regulation_factor=1.01, pad_token_id=50256)
    print(gen_tokens.shape[1])
```
Results
```
Regular output with max_length=600
600
600
600
600
600
--------------
Soft regulation (length_regulation_start=300, length_regulation_factor=1.01)
361
365
381
370
396
```

Here we see one length-regulated example output. The <endoftext> token is being reasonably placed:
```
In recent events the conflict between Elon Musk and Donald Trump heated up. Trump criticised Musk after he was quoted by The New York Times saying that a ‘solution’ to the US’s problems in the technology industry might be “a bullet to the head”.

In return Musk sent the president a series of tweets where he called the president “a traitor”, and “an idiot” as well as said he wouldn’t take the challenge of building the president a $10 million Tesla Roadster.

“It’s not my style to engage in such juvenile behaviour — it’s never been my style to engage in such juvenile behaviour — but the stakes for positive change have become that important,” said Musk in an interview with Forbes magazine, which is to be published on Wednesday.

The interview was published a few days before Musk went public about buying the Boring Company, which he described in the interview as a “vertical tunnel boring machine.”

After a few hours after this, Musk issued a statement about his tweet regarding the Roadster and the $10 million challenge to the president.

“The idea that the government could take ownership of all privately owned infrastructure in the U.S., just because I want it, is absurd on its face.”

He said he was being “clearly sarcastic.”

This was enough for Trump and his advisors to get angry, with Trump telling CNN that Musk is “a very, very, different kind of person than me.”

“I just think that he’s very, very smart,” Trump added.

```

This works as well with other EOS tokens that may be used in few shot learning scenarios such as "\n".


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
@patrickvonplaten


Comments:
Thank you for your PR! The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15245). All of your documentation changes will be reflected on that endpoint.

@patrickvonplaten, @LysandreJik

Hey @kevinpl07,

Thanks a lot for your PR! Do you know whether this method is used in any paper? Could you give me more on its scientific background?

In general, I think it's ok to add such a method here though

@patrickvonplaten 
Regarding scientific background of this method. We honestly just used intuition to solve our problem. It could very well be that this kind of LengthDecay is used in some paper.


Hey @kevinpl07,

Cool think we can move forward then with this logits processor. It would also be nice to add a test in the end :-)

@kevinpl07 - sorry I think the github commit history is broken. Could you resubmit a PR here? :-)

> @kevinpl07 - sorry I think the github commit history is broken. Could you resubmit a PR here? :-)

@patrickvonplaten I updated my PR, with the agreed changes and tests. I don't know if that fixed the commit history problem. Will squashing the commit upon merging fix this? Otherwise let me know how I can clean the history for you.

Hey @kevinpl07 - could you maybe rebase your branch and I think then we can merge :-)

@kevinpl07 ,

Be careful I think you added many unrelated files (687 were modified.).

If you're unsure how to fix it, tell us, we can probably help.

@patrickvonplaten changed the docstrings accordingly and rebased :)

Don't worry about the `run_tests_hub` test - not related to this PR ;-)

So everything good from my side? @patrickvonplaten 

@sgugger anything from my side, that I can do support the merge?

Merging - thanks for your work here!