`min_new_tokens` argument in generate() implementation
Author: kotikkonstantin
Pull request: no

### Feature request

As many said: 
- [link1](https://github.com/huggingface/transformers/issues/20614) cc @silverriver 
- [link2](https://github.com/huggingface/transformers/issues/20756) cc @gonced8

A new parameter `min_new_tokens` to the `.generate()` method to limit the length of newly generated tokens. The current parameter `min_length` limits the length of `prompt + newly generated tokens`, not the length of newly generated tokens.

I've come up with a solution by creating a new logits processor:
```python
import torch

from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers.generation_utils import MinLengthLogitsProcessor


class MinNewTokensLengthLogitsProcessor(MinLengthLogitsProcessor):
    r"""
        [`MinLengthLogitsProcessor`] enforcing a min-length of new tokens by setting EOS probability to 0.
        Args:
            min_length (`int`):
                The minimum length below which the score of `eos_token_id` is set to `-float("Inf")`.
            eos_token_id (`int`):
                The id of the *end-of-sequence* token.
        """
    def __init__(self, min_length: int, eos_token_id: int):
        super().__init__(min_length, eos_token_id)

        self.prompt_length_to_skip = None

    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        if self.prompt_length_to_skip is None:
            self.prompt_length_to_skip = input_ids.shape[-1]

        current_length = input_ids.shape[-1] - self.prompt_length_to_skip
        if current_length < self.min_length:
            scores[:, self.eos_token_id] = -float("inf")

        return scores


if __name__ == '__main__':

    device = "cuda" if torch.cuda.is_available() else "cpu"
    model_name = "inkoziev/rugpt_chitchat"
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    tokenizer.add_special_tokens({'bos_token': '
', 'eos_token': '
', 'pad_token': '<pad>'})
    model = AutoModelForCausalLM.from_pretrained(model_name)
    model.to(device)
    model.eval()

    input_text = """
- Привет! Что делаешь?
    - Привет :) В такси еду
    -"""

    encoded_prompt = tokenizer.encode(input_text, add_special_tokens=False, return_tensors="pt").to(device)

    output_sequences = model.generate(input_ids=encoded_prompt,
                                      logits_processor=[MinNewTokensLengthLogitsProcessor(16, tokenizer.eos_token_id)],
                                      max_length=100, num_return_sequences=1, pad_token_id=tokenizer.pad_token_id
                                      )

    text = tokenizer.decode(output_sequences[0].tolist(), clean_up_tokenization_spaces=True)[len(input_text)+1:]
    text = text[: text.find('
')]

    print(f"Length of generated text W/ MinNewTokensLengthLogitsProcessor: {len(text)}")
    print(text)

    output_sequences = model.generate(input_ids=encoded_prompt, min_length=16,
                                      max_length=100, num_return_sequences=1, pad_token_id=tokenizer.pad_token_id
                                      )

    text = tokenizer.decode(output_sequences[0].tolist(), clean_up_tokenization_spaces=True)[len(input_text) + 1:]
    text = text[: text.find('
')]
    print(f"Length of generated text W/O MinNewTokensLengthLogitsProcessor: {len(text)}")
    print(text)
```
**outcome of script executing**:
<img width="1575" alt="image" src="https://user-images.githubusercontent.com/22777646/208291420-9874c4ad-e63e-4ef8-bf8a-ec249748c50f.png">

Used transformers package version: 4.24.0

**But I'd recommend to do it simpler, by requested `min_new_tokens` argument in `.generate()`**



### Motivation

**The motivation is to control min length of the newly generated replica**

### Your contribution

by submitting a PR 

Comments:
I have made a PR #20819 to add this augment to the `generate()` implementation. r

Hey @kotikkonstantin 👋 Can you open a PR with your proposed `MinNewTokensLengthLogitsProcessor`? It looks good to me, except for one detail -- it shouldn't inherit from `MinLengthLogitsProcessor`, as our long run goal is to deprecate it :)

After we merge `MinNewTokensLengthLogitsProcessor`, we can integrate it with `generate` with @silverriver's PR!

Hi @gante ! Sure) I'm making it for a couple of days. Thank you very much for the feedback!

Hey @gante 👋 A PR is ready 

Actually, this is not done yet, as #20819 needs to be merged to be usable with `generate` :)

Hi, I have closed my original PR #20819 and made a new one #21044 to avoid messing with a bunch of other commits when I tried to rebase my commit.

#21044 is implemented based on the new `MinNewTokensLengthLogitsProcessor`. @gante Please have a look.

(should be done now :) )