Generate: have an example on each `LogitsProcessor` class docstring
Author: gante
Pull request: no
Labels: name: Good First Issue

name: Good First Documentation Issue

# Context

`.generate()` can be extensively manipulated through `LogitsProcessor` (and `LogitsWarper`) classes. Those classes are the code implementation behind flags like `temperature` or `top_k`. 

Most of our `LogitsProcessor` classes have a docstring that briefly describes their effect. However, unless you are an expert in text generation, it's hard to fully grasp the impact of using each class. In some cases, it is also non-trivial to prepare the arguments to initialize the `LogitsProcessor` class. As such, each class should have a clear usage example with `.generate()` in their docstring! 💪 

Here is an example: [SequenceBiasLogitsProcessor docstring](https://github.com/huggingface/transformers/blob/f1732e1374a082bf8e43bd0e4aa8a2da21a32a21/src/transformers/generation/logits_process.py#L559). Contrarily to the other classes (at the time of writing), we can quickly learn how to use it just by reading its docstring. We are also immediately aware of a few caveats 🤓 

Bonus points: our docstring examples are part of our CI, so we would be beefing up our tests to ensure we don't add regressions 🤗 

This issue is part of the [text generation docs rework](https://github.com/huggingface/transformers/issues/24575).

# How to participate?
1. Ensure you've read our contributing [guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) 📜 
2. Claim your `LogitProcessor` class in this thread (confirm no one is working on it). You can check the full list of classes below, and you can find their implementation in [this file](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/logits_process.py) 🎯 
    - You may need to do some detective work to fully understand the purpose of the class. For instance, some classes were created as part of a paper to be applied to any model, others are model-specific, and some exist to avoid weird bugs 🕵️ 
    - Looking at the git history is a great way to understand how a `LogitsProcessor` came to be.
3. Implement your changes, taking the [SequenceBiasLogitsProcessor docstring](https://github.com/huggingface/transformers/blob/f1732e1374a082bf8e43bd0e4aa8a2da21a32a21/src/transformers/generation/logits_process.py#L559) as reference  💪 
    - Add a clear example that calls the processor through `.generate()`. Make sure the example's outputs are correct and that the model used in the test is a small model (anything larger than GPT2 needs explicit approval);
    - If you feel like the original docstring could be better, feel free to enhance it as well!
    - Don't forget to run `make fixup` before your final commit.
4. Open the PR and tag me in it  🎊 

# Tracker
- [x] MinNewTokensLengthLogitsProcessor
- [x] TemperatureLogitsWarper
- [x] RepetitionPenaltyLogitsProcessor
- [ ] EncoderRepetitionPenaltyLogitsProcessor
- [x] TopPLogitsWarper
- [ ] TopKLogitsWarper
- [ ] TypicalLogitsWarper
- [x] EpsilonLogitsWarper
- [x] EtaLogitsWarper
- [x] NoRepeatNGramLogitsProcessor
- [ ] EncoderNoRepeatNGramLogitsProcessor
- [x] SequenceBiasLogitsProcessor
- [x] NoBadWordsLogitsProcessor
- [ ] PrefixConstrainedLogitsProcessor
- [x] HammingDiversityLogitsProcessor
- [ ] ForcedBOSTokenLogitsProcessor
- [ ] ForcedEOSTokenLogitsProcessor
- [ ] InfNanRemoveLogitsProcessor
- [ ] ExponentialDecayLengthPenalty
- [ ] LogitNormalization
- [ ] SuppressTokensAtBeginLogitsProcessor
- [ ] SuppressTokensLogitsProcessor
- [ ] ForceTokensLogitsProcessor
- [ ] WhisperTimeStampLogitsProcessor
- [ ] ClassifierFreeGuidanceLogitsProcessor 


Comments:
Hi @gante  I'm happy to give one of these a go as it seems a nice learning experience... `TemperatureLogitsWarper` feels as good as anyone else, can you assign it to me if it's not taken yet?

@nablabits thank you for your interest. It's all yours! 

(The assignment is informal -- the first one to mention a certain class gets automatically assigned to it 🤗 )

Hello @gante 👋
Thanks for opening up this contrib!
Would like to give it a try to [NoBadWordsLogitsProcessor ](https://github.com/huggingface/transformers/blob/f1732e1374a082bf8e43bd0e4aa8a2da21a32a21/src/transformers/generation/logits_process.py#L725).  Alredy tried `bad_words_ids` argument and looking forward to digging and learning more into the impact of `eos_token_id` for this class. 

LKM if that works for you !

Hey @gante 👋 

I would like to try [RepetitionPenaltyLogitsProcessor ](https://github.com/huggingface/transformers/blob/5bb4430edc7df9f9950d412d98bbe505cc4d328b/src/transformers/generation/logits_process.py#L194)to start with.

I hope that works!

Hey Shauray (@shauray8), Nice one! I can see you have opened a PR for the RepetitionPenaltyLogitsProcessor. I was working on it.  Just a note for next time, please go through the "How to Participate" and confirm no one is working on it.

@gante I'll look at other Processor Classes soon and take up something else. 👍 

Hey @Rishab26, I didn't go through the comments and I appreciate your understanding.


Hey @Rishab26 I would like to highlight your level of empathy towards this OSS Governance issue.
IMO, example of level 4  in [trust-level system](https://blog.discourse.org/2018/06/understanding-discourse-trust-levels/) HF relies upon . 

🤗 Thanks for setting a positive example for the Open Source Community. 🤗



Working on `TopKLogitsWarper`

Hey Folks!
Still working on this, having fun though.
Im opening the WIP in this [repo](https://github.com/SoyGema/contrib_schema/)  in case someone wants to have a look before PR.
Have already some things , but giving it a careful thought to the example and digging into some things.
Found this gem #22168 !


Hey @gante I would like to work on `SuppressTokensLogitsProcessor` 🙂

Hey @gante I am working on `NoRepeatNGramLogitsProcessor` 🙂

hey @gante I am working on  **TypicalLogitsWarper**

hey @gante I would like to work on **EtaLogitsWarper** 😊

Hi! May I claim `TopPLogitsWarper`?

Hey @gante, yes, me again :upside_down_face: , are you happy for me to pick `MinNewTokensLengthLogitsProcessor`?

Hi @gante I want to claim `ForcedEOSTokenLogitsProcessor` 😄 

Hi @gante can I try to work on `LogitNormalization`?

Hi @gante , can I try working on `ForceTokensLogitsProcessor`?

hi @gante i am working on `EncoderRepetitionPenaltyLogitsProcessor`, is there a way I can create a new issue named `EncoderRepetitionPenaltyLogitsProcessor` l then link that here and it shows active in the list?
like this, you can put a checklist there

![image](https://github.com/huggingface/transformers/assets/64583161/6efd3f34-c7ec-4360-95c8-671f808c852f)


Hello @gante, I would like to work on `InfNanRemoveLogitsProcessor`. I see no one has referred to it in the comments section. But just wanted to make sure no one is working on it. Could you please confirm?

> hi @gante i am working on EncoderRepetitionPenaltyLogitsProcessor, is there a way I can create a new issue named EncoderRepetitionPenaltyLogitsProcessor l then link that here and it shows active in the list?
like this, you can put a checklist there

@rajveer43 We can, but I don't see the benefit of it (each issue would be very small and straightforward), so I'm not convinced about having this extra step :D 

Hi @gante , I'd love to try writing examples for `HammingDiversityLogitsProcessor` !

Hi @gante can I work on ExponentialDecayLengthPenalty? Thanks!

Hi @gante , 
Working on [EpsilonLogitsWarper](https://github.com/huggingface/transformers/blob/a6e6b1c622d8d08e2510a82cb6266d7b654f1cbf/src/transformers/generation/logits_process.py#L442)

Hi @gante ,
Can I work on SuppressTokensAtBeginLogitsProcessor ~TopPLogitsWarper~ ? Thanks! :) 

> Hi @gante , Can I work on TopPLogitsWarper? Thanks! :)

Hi! I'm already working on it.

> > Hi @gante , Can I work on TopPLogitsWarper? Thanks! :)
> 
> Hi! I'm already working on it.

Ah I see, It looks like you spelled it as `TopPLogitsWrapper` rather than `TopPLogitsWarper` so I couldn't find it with a CTRL + F my bad! I'll take up SuppressTokensAtBeginLogitsProcessor, don't want to steal that from you haha 👍 

> > > Hi @gante , Can I work on TopPLogitsWarper? Thanks! :)
> > 
> > 
> > Hi! I'm already working on it.
> 
> Ah I see, It looks like you spelled it as `TopPLogitsWrapper` rather than `TopPLogitsWarper` so I couldn't find it with a CTRL + F my bad! I'll take up SuppressTokensAtBeginLogitsProcessor, don't want to steal that from you haha 👍

My bad, I'll correct it. Thanks!

Hello, I'll work on the WhisperTimeStampLogitsProcessor. 

@gante While working on the docs, I noticed there might be an issue with the current implementation of ExponentialDecayLengthPenalty.

The processor is intended to exponentially increase the score of the eos_token_id after start_index has been reached, allowing generating shorter sequences without having a hard cutoff.

When working with shorter sequences (Up to ~200) it doesn't necessarily cut the sequence, no matter how large the decay factor is.
In the following line, the processor attempts to increase the score of EOS. However when EOS score is negative, this actually decreases the score, as the exponent will be positive. As I understand, giving a negative decay factor won't work as well due to the power. Due to this it will only succeed if EOS becomes positive.
https://github.com/huggingface/transformers/blob/f1732e1374a082bf8e43bd0e4aa8a2da21a32a21/src/transformers/generation/logits_process.py#L982

Two questions:

1. Is this actually an issue?
2. If it is, I believe I can fix it. Should I open an issue, or just fix it with the docs PR?

@pokjay from your description, it seems to be an issue indeed! Can I kindly ask you to open a new issue with your findings (and tag me there)? 🙏 

Hey @gante I am working on **ClassifierFreeGuidanceLogitsProcessor**

Hey everyone! `.generate()` documentation is a focus of the next release, so I'm going to fast-track this issue internally. Thank you for your participation -- your contributions help our ecosystem thrive 💛 

👉 What does this mean?
1. I will NOT be accepting new assignments in this thread (and, as such, I'm closing it)
2. I will be accepting existing and new PRs for assigned logits processors over the next week. After that, I will take the remaining classes 🤗