Repository: huggingface/transformers
Issue category: Feature Request

min_new_tokens option in generate() implementation
### Feature request

Similarly to the `max_new_tokens`, a `min_new_tokens` option would count only the newly generated tokens, ignoring the tokens of the input sequence (prompt) in decoder only models.

### Motivation

The option `min_length` of the `generate()` method might be ambiguous for decoder only models. It is not clear if decoder only models consider the length of the input (prompt) for the `min_length` condition or only the newly generated tokens.
In Encoder Decoder (seq2seq) it is clear though.

### Your contribution

Not that I remember. But I could test it.


Functions to edit:
src/transformers/generation/configuration_utils.py:GenerationConfig.__init__
src/transformers/generation/utils.py:GenerationMixin._get_logits_processor

Patch:
diff --git a/src/transformers/generation/configuration_utils.py b/src/transformers/generation/configuration_utils.py
index a01222c8b41e..ada03f9c7bd7 100644
--- a/src/transformers/generation/configuration_utils.py
+++ b/src/transformers/generation/configuration_utils.py
@@ -75,7 +75,11 @@ class GenerationConfig(PushToHubMixin):
         max_new_tokens (`int`, *optional*):
             The maximum numbers of tokens to generate, ignoring the number of tokens in the prompt.
         min_length (`int`, *optional*, defaults to 0):
-            The minimum length of the sequence to be generated.
+            The minimum length of the sequence to be generated. Corresponds to the length of the input prompt +
+            `min_new_tokens`. In general, prefer the use of `min_new_tokens`, which ignores the number of tokens in the
+            prompt.
+        min_new_tokens (`int`, *optional*):
+            The minimum numbers of tokens to generate, ignoring the number of tokens in the prompt.
         early_stopping (`bool`, *optional*, defaults to `False`):
             Whether to stop the beam search when at least `num_beams` sentences are finished per batch or not.
         max_time(`float`, *optional*):
@@ -207,6 +211,7 @@ def __init__(self, **kwargs):
         self.max_length = kwargs.pop("max_length", 20)
         self.max_new_tokens = kwargs.pop("max_new_tokens", None)
         self.min_length = kwargs.pop("min_length", 0)
+        self.min_new_tokens = kwargs.pop("min_new_tokens", None)
         self.early_stopping = kwargs.pop("early_stopping", False)
         self.max_time = kwargs.pop("max_time", None)
 
diff --git a/src/transformers/generation/utils.py b/src/transformers/generation/utils.py
index 65c968a3d55a..4a48d43b21a9 100644
--- a/src/transformers/generation/utils.py
+++ b/src/transformers/generation/utils.py
@@ -48,6 +48,7 @@
     LogitNormalization,
     LogitsProcessorList,
     MinLengthLogitsProcessor,
+    MinNewTokensLengthLogitsProcessor,
     NoBadWordsLogitsProcessor,
     NoRepeatNGramLogitsProcessor,
     PrefixConstrainedLogitsProcessor,
@@ -822,6 +823,16 @@ def _get_logits_processor(
             and generation_config.min_length > 0
         ):
             processors.append(MinLengthLogitsProcessor(generation_config.min_length, generation_config.eos_token_id))
+        if (
+            generation_config.min_new_tokens is not None
+            and generation_config.eos_token_id is not None
+            and generation_config.min_new_tokens > 0
+        ):
+            processors.append(
+                MinNewTokensLengthLogitsProcessor(
+                    input_ids_seq_length, generation_config.min_new_tokens, generation_config.eos_token_id
+                )
+            )
         if prefix_allowed_tokens_fn is not None:
             processors.append(
                 PrefixConstrainedLogitsProcessor(