Instructions to use diffusers/ideogram4-prompt-enhancer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use diffusers/ideogram4-prompt-enhancer with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("diffusers/ideogram4-prompt-enhancer", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Ideogram4PromptEnhancerBlock rewrites a prompt into Ideogram4's structured JSON caption format.
The block and Ideogram4PromptEnhancerHead implementation live in this repository; neither imports
the implementation from diffusers.pipelines.ideogram4.
Install Diffusers from source and the generation dependencies:
pip install git+https://github.com/huggingface/diffusers.git transformers accelerate outlines
import torch
from diffusers import ModularPipeline
enhancer = ModularPipeline.from_pretrained(
"diffusers/ideogram4-prompt-enhancer", trust_remote_code=True
)
enhancer.load_components(dtype=torch.bfloat16, trust_remote_code=True)
enhancer.to("cuda")
captions = enhancer(
prompt="A red bicycle leaning against a yellow wall.",
height=768,
width=1024,
generator=torch.Generator().manual_seed(42),
output="prompt",
)
print(captions[0])
The output is a list of JSON strings, one per input prompt. Pass a string or a list of strings to prompt.
height and width default to 2048 and determine the aspect ratio in the caption instruction.
max_new_tokens defaults to 2048. prompt_upsampling_temperature defaults to 1.0; set it to 0 for greedy decoding.
Outlines constrains the JSON schema when installed. A token limit that is too short can still truncate the caption.
To reuse an existing Ideogram4 pipeline's encoder and tokenizer, replace the loading section with:
enhancer.update_components(text_encoder=pipe.text_encoder, tokenizer=pipe.tokenizer)
enhancer.load_components(
names="prompt_enhancer_head", dtype=torch.bfloat16, trust_remote_code=True
)
enhancer.to("cuda")
The encoder is shared by reference. Keep the shared encoder and the head on the same device and use compatible dtypes.
Pass the resulting captions to the image pipeline with prompt_upsampling=False.
modular_model_index.json records all three components. The encoder and tokenizer come from
Qwen/Qwen3-VL-8B-Instruct. The head is loaded through diffusers.AutoModel from this repository's
prompt_enhancer_head/ directory with trust_remote_code=True.
Its weights are unchanged from
diffusers/qwen3-vl-8b-instruct-lm-head.
The caption instructions, generation helpers, and head implementation are adapted from Diffusers at d5df05ed2.
- Downloads last month
- -
Model tree for diffusers/ideogram4-prompt-enhancer
Base model
Qwen/Qwen3-VL-8B-Instruct