query() of generator `max_length` being succeeded
Author: danielbellhv
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: Visual Studio Code, Windows 10, WSL 2 Bash
- Python version: 3.8.8
- PyTorch CUDA: 10.2

### Who can help

Models:
- GPT-2, GPT: @patrickvonplaten, @LysandreJik

Library:
- Pipelines: @Narsil

 -->

## Information

Model I am using (GPT-2):

The problem arises when using:
* [X] the official example scripts: (give details below)
* [] my own modified scripts: (give details below)

The tasks I am working on is:
* [] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## Expected behavior

All outputs' lengths should be within `min_length` and `max_length`.

---

Based on [SO post](https://stackoverflow.com/q/71338307/17840900).

Goal: set `min_length` and `max_length` in Hugging Face 🤗 Transformers generator query.

I've passed `50, 200` as these parameters. Yet, the length of my outputs are much higher...

There's no runtime failure.

```python
from transformers import pipeline, set_seed
generator = pipeline('text-generation', model='gpt2')
set_seed(42)

def query(payload, multiple, min_char_len, max_char_len):
    print(min_char_len, max_char_len)
    list_dict = generator(payload, min_length=min_char_len, max_length=max_char_len, num_return_sequences=multiple)
    test = [d['generated_text'].split(payload)[1].strip() for d in list_dict]
    for t in test: print(len(t))
    return test

query('example', 1, 50, 200)
```

Output:
```
50 200
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
1015
```

Comments:
Hi @danielbellhv ,

This is actually working as intended.
`max_length` is expressed in number of tokens (which is not necessarily obvious I admit).

First of all I would recommend using `max_new_tokens=200` which are the number of "additional" tokens which is easier for you to control. `max_length` still exists but it causes confusion regularly. (We don't have `min_new_tokens` yet, but I'll note the suggestion actually).

Then what's a token ? Models, don't ingest the text one character at a time, but one `token` at a time. There are different algorithms to achieve this but basically "My name is Nicolas" gets transformers into ["my", " name", " is", " nic", "olas"] for instance, and each of those `tokens` have a number.

So when you are generating tokens, they can contain themselves 1 or more characters (usually several and almost any common word for instance). That's why you are seeing 1015 instead of your expected 200 (the tokens here have an average of 5 chars)

Does this explain a bit better what is happening ?

Also this parameter is usually used to control the amount of time your model takes to run (the more tokens the more time, it scales linearly roughly)

Also thanks for reporting this, pipelines are aimed at people not knowing machine learning, and we certainly don't expect you to know what a token is. We're starting some thoughts on how to make all pipelines smoother.

Cheers, have a great day ! 

Hi @Narsil. Ah yes; tokens explains the length of the outputs.

However, when swapping out `max_length=max_char_len` for `max_new_tokens=max_char_len` - I got similar lengths.

Is there such a parameter that controls the length as characters? I doubt it, since as you explained the model processes tokens and not characters.

**Solution:**
Rename `min_char_len, max_char_len` to `min_tokens, max_tokens` and simply reduce their values by a ~1/4 or 1/5.

Cheers

Yes @danielbellhv ,

As you said, currently there's not option to get `char` control over the generated text.

We can definitely think about it, it's a rather large change which is not obvious, but definitely aligned with our vision of pipelines

@Narsil Yeah, it might be a huge undertaking just for that slight benefit.

Users could just generate with `n tokens` and string split the first `n chars` instead