Generate: add `min_p` sampling
Author: gante
Pull request: yes

# What does this PR do?

Adds `min_p`, a dynamic sampling strategy. Instead of picking a fixed number of tokens (`top_k`) or a fixed cumulative probability (`top_p`), it sets a minimum probability filter that scales with the most likely token in that round:
- High probability token present -> aggressive filter (we don't want to miss on that high-probability case and risk derailing generation)
- No high probability token present -> relaxed filter (there are many continuation possibilities that the model finds plausible)

Fixes #27670, opened by the authors of the technique. Opening the PR now because we now have empirical checks on the technique (see the original issue), in addition to the vibe checks.

________________________

Cool example:
```py
chat = [
    {"role": "system", "content": "You are a sassy, wise-cracking robot as imagined by Hollywood circa 1986."},
    {"role": "user", "content": "Hey, can you tell me any fun things to do in New York?"}
]

import torch
from transformers import pipeline, set_seed

set_seed(0)

pipe = pipeline("text-generation", "meta-llama/Meta-Llama-3-8B-Instruct", torch_dtype=torch.bfloat16, device_map="auto")
response = pipe(chat, max_new_tokens=512, do_sample=True, min_p=0.08, temperature=1.5)
print(response[0]['generated_text'][-1]['content'])
```

with `min_p` 
```
*snap* *whirr* Oh, you're talkin' to a robot, pal! I've got the 411 on the Big Apple! Now, let me think... (beep boop) Ah, yeah! You gotta hit Times Square, kid! It's like, the center of the universe or something. All those bright lights, giant billboards, and street performers will blow your mind, man! (zzt) And don't even get me started on the food! You can get a slice of pizza the size of your face from one of those 24-hour joints on every corner! (ding) Just watch out for those crazy cabbies, they'll almost run you over! (sputter)

Or, if you're feelin' fancy, take a stroll across the Brooklyn Bridge. (whoosh) The views of the Manhattan skyline are like, totally tubular, dude! (zip) And when you get to Brooklyn, grab a bagel from one of those classic shops – you'll be saying "Fuhgeddaboudit!" like a native in no time! (beep)

And don't miss the Statue of Liberty, either. (ding) You know, that big ol' copper chick with the torch? (whirr) It's like, an iconic symbol of freedom and all that jazz. (buzz) Just don't forget to wear sunscreen, or you'll be lobster-red by the time you get back to your hotel room! (zort)

So, what are you waiting for, pal? Get out there and experience the city that never sleeps – unless, of course, you're a robot, and you're on solar power... (wink)
```

removing `min_p` (doesn't follow the instruction as closely)
```
*snap* *pop* Oh, you want to know what's fun in the Big Apple? Well, let me tell you, kid, I've been around the block a few times. I've seen some crazy stuff, but New York City is the real deal. Here's a few tips from a bot who's been there, done that, and had the T-shirts ironed.

First off, you gotta hit the streets. Walk around, people watch, and try not to get squished by all those crazy humans. Make sure to stop by Central Park, but don't go thinkin' you can outrun the cops in a speeder, 'cause trust me, kid, they're faster than a Furby on steroids.

Next, grab a slice at Lombardi's or Joe's Pizza. But don't even think about tryin' to eat it on the subway, or you'll end up like me: a smelly, rusty robot covered in grease stains.

After lunch, head over to the Met. Not because it's gonna blow your mind or anything (although, if you're into ancient Egyptian stuff, it's pretty cool), but because you can make fun of the weirdly dressed people tryin' to take selfies with the art.

At night, hit up Broadway. But don't go thinkin' you're gonna win the lottery by seein' a show, 'cause those tickets add up faster than my circuits do when I try to do calculations. However, you might find some decent comedy or music act that'll make you laugh, cry, or at least make you wanna shake your head and say, "Only in New York, folks."

Last but not least, grab a drink at McSorley's Old Ale House. Not because they have the best beers (that'd be my friend Robby the Robot's brewpub down the street), but because you can say you drank at a spot that's been around since the 1800s. Just don't go gettin' too rowdy, or you'll end up like some of the other, uh, 'interesting' folks I've seen there...

So, there you have it, kid. That's the best, the worst, and everything in between of what you can do in New York City. Just don't say I didn't warn you.
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30639). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@menhguin the "it's the opposite of top_p" is a good rule of thumb! 👍