add CFG for .generate()
Author: Vermeille
Pull request: yes

This commit [implements CFG](https://github.com/huggingface/transformers/issues/24536)

Fixes #24536 (I did not touch MusicGen)

Hope you enjoy it!

@sanchit-gandhi 
@gante 

Comments:
@Vermeille -- @sanchit-gandhi raises good points about the attention mask and taking the first item of the batch in the unconditional logits. As it stands, it will only work with batch size = 1, and our logits processors should be flexible wrt batch size :) 

good catch with the batch size! As for the attention mask, could you guide me to a working solution with that? I'm quite unfamiliar with huggingface tbh.

Tests are on the way.

All right. we only need to address use_cache / attention_mask.

* use_cache: currently, the forward passes take care of automatically appending to the negative prompt. I don't think such a thing happens with use_cache=False so I gotta do the concat myself. probably meaning I have to make two branches based on the value of use_cache?
* attention_mask: Does it even make sense then to read out.logits[:, -1]? is -1 a valid index if that position has an attention_mask of 0 due to padding? If so, then I will concat a valid id to padding and the attention_mask will be something like [1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1], won't that screw up positional encoding with the "empty" slots?

Basically, I think .generate() had to answer the same questions so you guys will be able to answer them quite easily. Also I will need your guidance as for the API design to integrate this seamlessly.

@gante I think we're good. The failure looks totally unrelated.

Indeed. no more failed test.

@Vermeille 

> * use_cache: currently, the forward passes take care of automatically appending to the negative prompt. I don't think such a thing happens with use_cache=False so I gotta do the concat myself. probably meaning I have to make two branches based on the value of use_cache?

As I've replied in the dedicated thread, don't worry about the uncached case :) Make sure an exception is thrown, though!
EDIT: I see that you've handled the uncached case. In that case, since you've already written the code, you can leave it be :)

> * attention_mask: Does it even make sense then to read out.logits[:, -1]? is -1 a valid index if that position has an attention_mask of 0 due to padding? If so, then I will concat a valid id to padding and the attention_mask will be something like [1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1], won't that screw up positional encoding with the "empty" slots?

That's a non-issue: `.generate()` must always be used with left-padding, so you won't run into the case of picking a padded token with `-1` indexing 🙌 


_The documentation is not available anymore as the PR was closed or merged._

Following @MPKonst remarks:
- the CFG Rescale "technique" has been removed as it is just a different parameterization of the guidance scale
- The final log_softmax has been removed, which corresponds to how the calculations were performed for the benchmarks anyway. It was introduced as later stage for the GPT4All experiments, as part of the CFG Rescale integration, and it seems it was not a good idea. 

That's weird, the tests have been written for a while but did not show up in the PR. They do now.
I also addressed your latest comments.

I answered some comments but will finish the PR next week. I'm unavailable until then. Sorry for the delay.

@gante  I need you to answer about the `model_kwargs` validation before I can submit a new version of the PR

That would be an amazing feature. Thanks for working on this @Vermeille 
Fingers crossed it will get reviewed and accepted soon

+1 for this PR. I hope that it can be merged soon.

@Vermeille answered in the thread! 

LMK if there is any other decision I can help with -- and tag me when you think the PR is in a finalized state, for a quick check and approval ✅ 

@gante looks like we're good now :)

(@sgugger this one possibly did not get through your notifications, gently pinging :) )

@Vermeille would you be able to retouch the tests? We can merge right after that change :)

I'm currently in vacations. What's the problem with the tests?

@Vermeille there are a few patterns in the tests that we usually avoid in our codebase (like thin wrappers). However, it's a minor issue, and this feature is being requested by the community, so I'm favoring merging it now.

I understand the review process is long and somewhat tedious on your end. We err on the strict side, as we bear the cost of future maintenance. Thank you for collaborating with us, and looking forward to future contributions 🤗

Next steps: we will be communicating the feature on our end. Amplification and/or communication on your end will help bring awareness to the feature! 🔥  

Thanks for your contribution @Vermeille and congrats on the PR!