Add Prompt Enhancer and Processor

#13
by dg845 - opened
No description provided.

This PR adds prompt_enhancer and processor components to the Lightricks/LTX-2.5-Diffusers repo based on the google/gemma-4-E2B-it repo, following the diffusers docs. The changes can be tested by loading with the revision corresponding to this PR:

import torch
from diffusers import LTX2Pipeline

pipe = LTX2Pipeline.from_pretrained("Lightricks/LTX-2.5-Diffusers", revision="refs/pr/13", torch_dtype=torch.bfloat16)
...

or for the modular pipeline:

import torch
from diffusers import ComponentsManager, ModularPipeline

cm = ComponentsManager()
pipe = ModularPipeline.from_pretrained("Lightricks/LTX-2.5-Diffusers", components_manager=cm)
pipe.load_components(revision="refs/pr/13", dtype=torch.bfloat16)
...

Saving the modular pipeline with new prompt_enhancer and processor components using save_pretrained changes the modular_model_config.json config file with respect to main by adding library and class info for all components (not just the new ones). I don't think these changes should break anything; as far as I can tell, loading and performing inference on the revised checkpoint should work as expected.

dg845 changed pull request status to open

Saving the modular pipeline with new prompt_enhancer and processor components using save_pretrained changes the modular_model_config.json config file with respect to main by adding library and class info for all components (not just the new ones). I don't think these changes should break anything; as far as I can tell, loading and performing inference on the revised checkpoint should work as expected.

Thx for the PR @dg845
For the prompt_enhancer do we need both the shards and the model.safetensors? It is a duplicate weights and extra 10GB of unnecessary download for the users?

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment