Instructions to use 100percentrobot/LTX-2.3-Audio-Reactive-LORA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use 100percentrobot/LTX-2.3-Audio-Reactive-LORA with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Lightricks/LTX-2.3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("100percentrobot/LTX-2.3-Audio-Reactive-LORA") prompt = "continuous audio-reactive video, audio-reactive" input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png") image = pipe(image=input_image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Workflow?
Nice concept, is it possible to upload a test workflow for this Lora please?
Nice concept, is it possible to upload a test workflow for this Lora please?
Just tried the lora myself. Assuming you mean ComfyUI since you said workflow..
Can try the workflow here if you want https://huggingface.co/RuneXX/LTX-2.3-Workflows/tree/main/Custom-Audio (the custom audio one)
(its based on Kijai's spilt models, but that can be beneficial since each file is smaller, instead of one huge all-in-one model)
(a couple of quick test runs in the videos above, used an LLM to generate prompts based on the prompt template at the front page model card of this repo. And/or you can use the text-generate node native in comfy to get a basic LLM response as well. The workflow above has built in prompt enhancer with just that)
A combo one... Audio Reactive with the IC-Transiton lora ;-)
a bit trippy.. but nice effect ;-)
Very nice! Thanks for posting the workflow. I mostly use WANGP for LTX. The IC-Transition lora is a great combo, especially for multiple injected frames
Working on V2 training data the last few weeks in my spare time. I'm hoping it will introduce some more realistic, less trippy vibes, but I definitely want to keep the gooey/warpy AI aesthetic as well, all while introducing more motion