Customizable Logit Warping Strategies for Generation
Author: PamelaBha
Pull request: no
Labels: name: Feature request

### Feature request

Improve the generate() API by supporting custom, declarative logit warping strategies. Make it easier for users to plug in standard and custom LogitProcessors via configuration or arguments without needing to subclass or dive into internals.



### Motivation


The generation module already supports rich logit manipulation through LogitProcessorList, but:

- It is undocumented and hard to use for casual users
- Requires advanced subclassing to customize behaviors (e.g., word bans, domain constraints)
- Doesn’t support JSON- or dict-style configuration like many other parts of Transformers

Making logit warping more accessible enables:

- Prompt engineers and power users to fine-tune generation behavior
- Safer generation via blacklists or probability shifting
- Dynamic controls like repetition penalties or temperature annealing



### Your contribution

 
- Introduce a registry of LogitProcessors exposed to users via a high-level interface
- Support a new logit_processors argument in pipeline() and generate()
- Accepts a list of dicts specifying processor type and parameters
- Extend LogitProcessorList to be constructed from configs

Comments:
cc @gante 

Hi @PamelaBha @zucchini-nlp , I’m interested in taking this issue. Please let me know if I can proceed.

Hi @Uvi-12 - I was planning to work on this myself as my first contribution to Transformers.

@zucchini-nlp - do you think this a good feature I can take up?

I will let @(gante) to decide as the maintainer of generation code :)

> Hi [@Uvi-12](https://github.com/Uvi-12) - I was planning to work on this myself as my first contribution to Transformers.

Thanks for letting me know. Please go ahead.



@gante - thoughts on this?

Hi @Uvi-12 @PamelaBha 👋 

In general, I'm on board with the idea -- the order of application of the logits processors enforced by default on `generate` is indeed suboptimal in some cases, and passing a list of `logits_processors` is unknown to most users. Adding (de)serialization support to that custom `logits_processors` is a nice-to-have.

Before committing code, and to save us both time, let's briefly chat about the features to be added!

______________________________________________________

From your description, what I have in mind would be a json file like

```
{
    class_name_of_first_to_be_applied: {
        arg_1: foo,
        arg_2: bar,
    },
    class_name_of_second_to_be_applied: {
        arg_1: baz
    },
}
```

Key notes:
- ~accept both a file path and the actual json (as a string).~ EDIT: the json data can only be passed through `generation_config`, and not directly through the `logits_processor` argument in `generate`
- Uses the class name as key. That way, external logits processors can easily be supported (all a user has to do is import the external logit processor class, I believe);
- From the `generate` side, `getattr` + class name + args solves it
- In the `GenerationConfig`, we can (de)serialize it. When we serialize it, we should serialize the json string (and not the path)
- Open question: do we allow "normal" arguments? E.g. `model.generate(..., generation_config=generation_config_instance_with_json_data, temperature=0.5)` would apply temperature after? (If yes, we should try to catch and warn about duplicate processors when they exist)
- docs and tests will be super important

WDYT?

Thank you @gante for sharing your thoughts.

This is my first contribution to Transformers and I am super excited.

I agree with your suggestions, Do you want me to send a draft PR highlighting where I plan to make changes? We can trash it once we agree on the changes. 

@PamelaBha yeah, that sounds perfect! Thank you 🤗 

(Since we have agreed on the general plan, I suspect we won't need to trash the draft PR 😉 )

@PamelaBha after chatting with @manueldeprada , we'd like to propose one more constraint: 

- the json data can only be passed through `generation_config`, and not directly through the `logits_processor` argument in `generate`

This is because we want to nudge our users towards using `generation_config` (for things that can be serialized) OR instantiated objects (for things that can't be serialized). It will also allow us to keep the interface to `generate` simpler.

Hi @gante - I don't think I have permission to commit my branch


@PamelaBha , You need to create a fork of the repo and then raise the PR. Refer the below docs for starters.

docs: https://huggingface.co/docs/transformers/en/contributing#create-a-pull-request

Thank you @yaswanth19 !

I need help reviewing my PR.. I pinged multiple times.. please help @gante  @Uvi-12 @zucchini-nlp 

Ping on this to help review PR and close this @gante 

@zucchini-nlp  @ArthurZucker @Cyrilvallez - can any of you kindly review my PR and help unblock me?