Forced to hit `UserWarning` when generating with `temperature=0`
Author: jamesbraza
Pull request: no
Labels: name: bug

### System Info

With `transformers==4.50.0` and constructing `GenerationConfig(do_sample=False, temperature=0.0)` you get a `UserWarning`:

```none
/path/to/.venv/lib/python3.12/site-packages/transformers/generation/configuration_utils.py:628: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.0` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
```

So then I use `GenerationConfig(do_sample=True, temperature=0.0)`. Passing that into `generate` that ultimately takes us [here](https://github.com/huggingface/transformers/blob/v4.50.0/src/transformers/generation/utils.py#L1107-L1108), which blows up with [this crash](https://github.com/huggingface/transformers/blob/v4.50.0/src/transformers/generation/logits_process.py#L276-L279):

```none
ValueError: `temperature` (=0.0) has to be a strictly positive float, otherwise your next token scores will be invalid. If you're looking for greedy decoding strategies, set `do_sample=False`.
```

Thus it seems we are forced to eat a `UserWarning` with `temperature=0.0`

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```python
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    GenerationConfig,
    LlamaForCausalLM,
)

# Hits warning
GenerationConfig(do_sample=False, temperature=0.0)

tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
model: LlamaForCausalLM = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
)
# Hits error
model.generate(
    tokenizer("Hello", return_tensors="pt").input_ids,
    generation_config=GenerationConfig(do_sample=True, temperature=0.0),
)
```

### Expected behavior

Not being forced to eat a `UserWarning`

Comments:
cc @gante 

The warning is correct: you should not set a non-default temperature when greedy decoding since all valid **positive** temperatures will lead to the same result. The default "do-nothing" temperature is **1.0**, not 0 😊

On top of that, 0 is an invalid value for temperature. Lets see why it needs to be positive:

$$
p_i = \frac{\exp(\log p_i / T)}{\sum_j \exp(\log p_j / T)}
$$
- if $T = 1$, logits stay unchanged (default behavior).
- if $T < 1$, output is sharper (more confident).
- if $T > 1$, output is softer (more random).
- if $T = 0$, the formula breaks (division by zero, invalid).

Thus, if you want greedy decoding (`do_sample=False`), do not specify temperature or set it to the default 1.0 to avoid the warning and the T=0 crash. Setting it to 0 will make the LogitProcessor divide by 0.

Thanks for sharing the rationale @manueldeprada , appreciated! However, I think then can reword the `UserWarning` to:
- Give a bit more rationale
- State possible resolutions

We go from:

> UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.0` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.

To something like:

> UserWarning: `do_sample` is set to `False`, which means greedily decode. However, `temperature` is set to `0.0` -- but this flag is only used in sample-based generation modes. If you want to greedily decode, don't specify a `temperature`. If you want to sample, set `do_sample=True`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.

I will make a PR for this

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Not stale, made https://github.com/huggingface/transformers/pull/38387 for this

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.