Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs

Authors: Nguyen Nhat Minh, Andrew Baker, Clement Neo, Allen G Roush, Andreas Kirsch, Ravid Shwartz-Ziv
Primary area: foundation or frontier models, including LLMs
Keywords: natural language processing, large language models, text generation, sampling methods, truncation sampling, stochastic sampling, min-p sampling, top-p sampling, nucleus sampling, temperature sampling, decoding methods, deep learning, artificial intelligence

Abstract:
Large Language Models (LLMs) generate text by sampling the next token from a probability distribution over the vocabulary at each decoding step. Popular sampling methods like top-p (nucleus sampling) often struggle to balance quality and diversity, especially at higher temperatures which lead to incoherent or repetitive outputs. We propose min-p sampling, a dynamic truncation method that adjusts the sampling threshold based on the model's confidence by using the top token's probability as a scaling factor. Our experiments on benchmarks including GPQA, GSM8K, and AlpacaEval Creative Writing show that min-p sampling improves both the quality and diversity of generated text across different model families (Mistral and Llama 3) and model sizes (1B to 123B parameters), especially at higher temperatures. Human evaluations further show a clear preference for min-p sampling, in both text quality and creativity. Min-p sampling has been adopted by popular open-source LLM frameworks, including Hugging Face Transformers, VLLM, and many others, highlighting its significant impact on improving text generation quality.

Decision: accept (oral)
Reviewer scores: 8, 10, 10, 6

Reviews and discussion:
<thread>
creation date:
1730116380094

thread_type:
official_review

<thread_object>
creator: reviewer_nzfq

<content>
summary:
The paper introduces a novel dynamic truncation method called min-p sampling, which adeptly adjusts the sampling threshold based on the model’s confidence by scaling according to the top token’s probability. This approach presents a significant advancement over traditional sampling methods like top-p and top-k, demonstrating improved balance between the quality and diversity of generated text.
The authors conducted experiments across three datasets, yielding compelling results that underscore the effectiveness of min-p sampling. The findings indicate that this method not only enhances the quality of text generation but also fosters greater diversity, which is a critical aspect in natural language processing tasks.
The writing in this paper is clear and accessible, making the concepts relatively easy to understand. The methodology is straightforward and provides a meaningful contribution to the field. Overall, this paper presents insights and a potential solution to the challenges of text generation, which may be of interest to researchers and practitioners.

strengths:
1. The paper is well-written and easy to follow.
2. The proposed min-p sampling makes an effective balance between coherence and diversity in text generation.

weaknesses:
Please refer to Questions.

questions:
Q1: In Table 2, the experimental results on the GPQA Main and GSM8K datasets demonstrate that Min-p sampling achieves better accuracy compared to other sampling methods when the temperature is set to 1 or higher. Additionally, it appears that all sampling methods perform better at lower temperature values. 

We are particularly interested in the ceiling performance of these methods on these two datasets. However, when the temperature is set to 0.7, min-p sampling does not show a significant advantage over top-p sampling. If the temperature is further decreased (e.g., to 0.5 or 0.3), will the performance of top-p sampling continue to improve? Furthermore, does min-p sampling still maintain a significant advantage over top-p sampling at these lower temperature settings?

Q2: Figure 1 shows that top-p sampling can ensure diversity in generation but may result in incoherent content. On the other hand, top-k sampling can ensure generation of high probability text but may lose diversity. Can a combination of top-p and top-k sampling compensate for their respective shortcomings and better balance coherence and diversity? Would min-p sampling be more effective than the combined method of top-p and top-k sampling?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730661386118

thread_type:
official_review

<thread_object>
creator: reviewer_d38h

<content>
summary:
Simple but effective and highly influential contribution to LLM research

strengths:
This paper presents compelling evidence that its single contribution, min-p sampling, is highly effective.  The usage of it in 54,000 Github repositories alone is very impressive.  In addition to that, they produced theoretical reasoning why their method works, LLM-generated statistics with explanations about how to interpret these statistics, additional statistics which involved human participants, examples of seeing how the logits are transformed under different distributions which give additional insight into why this method is better than existing methods, and code to try out the method.  It is a very simple paper, but it clearly makes the case for its own importance.

weaknesses:
The one contribution of this paper, min-p sampling, is extremely simple and not mathematically "deep" at all - no theorems were presented, and the code implementation literally (was provided and) took less than one page.  However, I think that having such a paper in a conference proceeding is not a bad thing.

questions:
It seems clear that the advantage of this approach is that it lets you "turn up the heat" - use temperature values that otherwise would provide gibberish.  Can you be more specific about what this particular change (going to higher temperature) - as opposed to min-p as a technique - is inherently something you'd want to do (are there benefits beyond diversity, and can you cite evidence for these benefits)?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730668961333

thread_type:
official_review

<thread_object>
creator: reviewer_fwnb

<content>
summary:
[UPDATE] Based on the rebuttal I have increased my score (8->10), but kept the rest of the review unchanged

The authors propose a new sampling mechanism, which is a minor but important twist to the popular nucleus-sampling (`p`). Instead of having a fixed threshold `p`, this proposal has `p` be dependant of the probability of the most probable token. The intuition is that in cases that the model is confident only few tokens are kept as support set, while that set is extended when the confidence is low

strengths:
* Sampling is one of those areas were the model per se needs to be complemented with an outside algorithm, allowing for creativity on how to set this up. This work proposes an original twist to a popular choice

* The proposal is simple, appealing and 

* has good empirical results, both as measured on benchmarks and (more important) by adoption of the community

weaknesses:
The new 10p limit has not been handled wisely in my opinion, and the paper could do more with less text. In particular, Sect 4 could be removed without much loss to the overall apper

Having experiments on a 123B has to be commended. The paper would be stronger however if the authors could show that the results hold on different model families (eg, llama and mistral), as otherwise it is not clear if this method provides gains on one family only

questions:
* You claim a widespread open-source usage. Could you review the usage of those and classify them by model family?

* Fig 1: different from what the caption reads, (b) seems to refer to top-k and (c) to top-p
</content>
</thread_object>

</thread>
<thread>
creation date:
1730681981834

thread_type:
official_review

<thread_object>
creator: reviewer_w4rz

<content>
summary:
This paper proposes the Min-p Sampling method, which dynamically adjusts the probability threshold based on the model's confidence level. This method aims to enhance creativity without sacrificing coherence. The method is validates through experiments on benchmark and human evaluations, showing better coherence and diversity compared to other sampling methods. The method has been widely adopted in the open-source community.

strengths:
- New sampling method: This paper proposes the Min-p Sampling method for better control over the diversity of generated outputs compared to fixed threshold methods like top-p.

- Conducted experiments: The authors conducted experiments across tasks, ablation studies, and human evaluation.

- High reproducibility: The author released the implementation, code, and repo with implementation guidelines, which enhances the reproducibility.

- Wide Applicability: The proposed method can be easily integrated with existing open-source LLMs, and the authors show the broad potential applications that can be applied.

- The ablation study shows that min-p sampling is barely impacted by the output length, which is interesting.

weaknesses:
- The experiment is limited to Mistral models and fails to demonstrate applicability with other models. It would be more comprehensive and interesting to see results from additional models, such as LLaMA3.

- The effectiveness of min-p sampling highly depends on the base probability thresholds. As shown in Table 6 (ablation study results), the choice of thresholds significantly impacts LLM performance. This indicates that optimal performance requires careful tuning, which could limit the method’s potential effectiveness and ease of use in applications.

- The paper claims that the experiment is intended to demonstrate that min-p sampling balances creativity and coherence (line 290); however, metrics relevant to creativity are missing. Diversity is not enough for creativity assessment. LLMs-as-judge approach is widely used for creativity assessment. Please consider adding such an experiment.

questions:
- The paper exceeds the 10-page limit. Please be careful with submission guidelines, as the paper could otherwise face desk rejection.

- Does min-p sampling make it more difficult to control LLMs, such as for lexically constrained generation?

- Details on the human evaluation are missing. What is the inter-annotator agreement rate? How many participants were recruited? The paper mentions receiving 70 initial responses; does each response contain one participant's evaluation for all data points? The paper claims that participants were recruited from Prolific. I would like to see the survey template, as it is important for reviewers to evaluate the effectiveness of the human evaluation.

- The paper states that min-p sampling has "Extensive human evaluations further confirmed a strong preference for min-p sampling over top-p, highlighting its practical advantages in real-world applications" (line 510). Could you provide some examples of real-world applications? In what scenarios would min-p sampling be preferable to other sampling methods? The paper has not included a relevant discussion on this.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732288889260

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response [1/2]

comment:
**General Response**

We sincerely thank all the reviewers for their thoughtful and constructive feedback. Your insights have been invaluable in improving our work. In response to your comments, we have conducted new experiments, provided additional analyses, and clarified key points to strengthen our paper. Below, we address your main concerns and highlight the enhancements made.

---

## 1. Expanded Experiments on Llama 3 Models

To address concerns about the generalizability of our method beyond Mistral models, we have conducted extensive new experiments on the **Llama 3** model family. Specifically, we tested the following models on the GPQA and GSM8K datasets across six different temperatures, comparing min-p sampling with both top-p and pure temperature sampling:

- **Llama 3.2 1B-Instruct**
- **Llama 3.2 3B-Instruct**
- **Llama 3.1 8B-Instruct**
- **Llama 3.1 70B-Instruct**


**Findings** (For full results, please see our updated Appendix D.1):

- **Consistent Performance**: Our results demonstrate that **min-p sampling** consistently outperforms the other sampling methods across different model sizes and temperature settings.
- **High-Temperature Robustness**: The advantages of min-p sampling are more pronounced at higher temperatures (30–90% better), where it maintains coherence and accuracy better than other methods. This advantage becomes particularly significant in longer contexts, where per-token degradation can compound into semantic incoherence, as noted by Holtzman et al. (2020). [\[1\]](https://arxiv.org/abs/1904.09751)

- **Low-Temperature Regime**: At lower temperatures, min-p performs slightly better than other methods, but all sampling methods perform similarly, as expected due to the peaked token distribution.

**Conclusion**: These additional experiments confirm that min-p sampling's benefits extend beyond a specific model family and validate our core thesis that min-p provides more robust performance across temperature settings while maintaining coherence at higher temperatures.

---

## 2. Enhanced Creativity Assessment with New Experiments

In addition to our existing evaluations—including the automated AlpacaEval Creative Writing benchmark and our human evaluation, both of which focused on creative qualities and showed a consistent preference for min-p-generated content—we have conducted several new experiments to strengthen our assessment of creativity. Full details and results are provided in Appendix D.4.

- **New constrained/structured LLM-as-Judge Evaluation**: We conducted an additional evaluation using a large language model as a judge, focusing on constrained generation creative writing tasks assessed across five different metrics (e.g., originality, narrative flow, emotional impact) at various temperatures and hyperparameters. We tested two models:

  - **Llama-3.2-1B-Instruct** (1B parameters)
  - **Mistral-7B-v0.1** (7B parameters)

- **Results**: Min-p sampling consistently outperformed top-p sampling across all creative metrics for both models, especially at higher temperatures, with improvements ranging from 0.43 to 0.70 points. Moreover, min-p demonstrated greater robustness to temperature changes, showing remarkable stability across different temperature settings.

**Conclusion**: These additional assessments further supports our original claim that min-p sampling enhances diversity for creative outputs without compromising coherence. The technique not only outperforms top-p in absolute terms but also demonstrates superior stability across different temperature settings and model scales.


---

## 3. Experiments with Very Low Temperatures

To evaluate min-p's performance at low temperatures, we compared min-p and top-p sampling on Mistral 7B at temperatures ranging from 0.0 to 0.5 on the GPQA and GSM8K datasets. Full results are presented in Appendix D.2.

**Key Findings**:

- At low temperatures, all sampling methods converge to nearly identical performance (less than 1% difference), as expected.
- These results confirm that at very low temperatures, the choice of sampling method has minimal impact due to the extremely peaked token distribution.

---

## 4. Combining Top-p and Top-k Sampling

In response to queries about combining sampling methods:

- **Experiments Conducted**: We tested various combinations of top-p and top-k sampling with Mistral 7B on GPQA and GSM8K COT, with the same setup as our previous experiments in our main paper. Full results are presented in in Appendix D.3.
- **Findings**: Combining these methods did not yield performance improvements over using min-p sampling alone.
 - **Complexity vs. Benefit**: The added complexity of tuning multiple hyperparameters does not justify the marginal gains, if any.

**Conclusion**: Min-p sampling offers a simpler and more effective approach to balancing creativity and coherence.
</content>
</thread_object>

</thread>
<thread>
creation date:
1742125640022

thread_type:
public_comment

<thread_object>
creator: ~nguyen_nhat_minh1

<content>
title:
Final Camera-Ready Updates: Supplementary Human Evaluation, External Validation, and Methodology Clarifications

comment:
Dear Area Chairs and Program Chairs,

As we prepare the camera-ready version of our paper "Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs" (Submission number: 11935), we would like to transparently disclose several substantive updates we have made since the review process concluded. These changes were undertaken to improve accuracy and transparency following post-review discussions with researchers clarifying our results.

The core claims, methodology, and findings of our paper remain unchanged. However, we have made the following updates to ensure greater scientific rigor:

1. **Human Evaluation Methodology**: We conducted an additional human evaluation using the VLLM inference engine instead of Hugging Face Transformers after discovering that Hugging Face applies temperature after truncation sampling (rather than before), which reduces the effect of truncation. Other methodology improvements include using Prolific's newly introduced AI Testers feature, full-length story outputs instead of short samples to assess longform textual output coherence, more rigorous attention checks, and clearer evaluation criteria. This new evaluation, detailed in *Appendix C.2: Additional Human Evaluation with VLLM Inference Engine*, shows dramatically more pronounced advantages for min-p at high temperatures - 5/10 (min-p) vs 1/10 (top-p and baseline) for Quality and Creativity at temperature 3.0, strengthening our findings.

2. **Additional External Validation**: We added brief reference to independent EQ-Bench evaluations that further validate min-p's advantages for creative writing (scores of 62 vs baseline 51.5). Responding to prior reviewer queries regarding real-world applications, we also detail examples of inference providers that have adopted min-p or papers that have replicated our results, such as "Training a Generally Curious Agent" (Tajwar et al., 2025) [1], which specifically cites min-p's benefits for generating high-quality and diverse training data.

3. **Community Adoption Metrics Clarification**: We revised our statement about GitHub adoption to be more conservative and easily verifiable. Our original claim of "54,000 repositories and 1.1 million stars" was based on preliminary GitHub searches that included false positives. Since it is hard to exhaustively search through and verify thousands of integrations, we now report only verified integrations in the top dozen or so major frameworks (575k stars, 290k+ downstream repositories), with detailed methodology in *Appendix A.5: Detailed Community Adoption Statistics.*

These changes provide more accurate metrics, additional supporting evidence, and greater methodological transparency. We are grateful for the thorough review process that helped strengthen our work.

Sincerely,

The Authors

[1] Tajwar, F., Jiang, Y., Thankaraj, A., Rahman, S. S., Kolter, J. Z., Schneider, J., & Salakhutdinov, R. (2025). Training a Generally Curious Agent. *International Conference on Learning Representations*. Retrieved from https://openreview.net/forum?id=aC6Dc9hiu1
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper provides a new sampling technique for text generation. The new technique is simple and is already widely adopted by the community (as mentioned by D38H, “The usage of it in 54,000 Github repositories alone is very impressive”). The authors provide a comprehensive analysis of their sampling technique comparing it to the previous methods, and demonstrating that in the low temperature regime, this new technique provides a significant advantage. 

The reviews raised issues that are not really concerns about the paper’s quality but suggestions to improve it: Test more LLMs, evaluate creativity in additional ways, etc. In the rebuttal, the authors provided additional experiments and clarified some issues raised in the reviews, addressing almost all of the points raised in the reviews, even those that I found to be “nice-to-have".  

The resulting review scores reflect the high quality of the paper: It presents convincing experiments, thorough analysis, and the provided method has an extremely high impact. The paper should be a great addition to ICLR.