Ideogram4 DreamBooth LoRA - linoyts/ideogram4-yarn-art-fp8

Model description

These are linoyts/ideogram4-yarn-art-fp8 DreamBooth LoRA weights for Disty0/Ideogram-4-SDNQ-FP8.

The weights were trained using DreamBooth with the Ideogram4 diffusers trainer.

This LoRA was trained with FP8 SDNQ quantization of the base model.

Trigger words

You should use yarn art style to trigger the image generation.

Ideogram 4 is trained on structured JSON captions, so for best results pass a caption in the JSON schema (see the prompt format) or a short prompt with prompt_upsampling=True.

Download model

Download the *.safetensors LoRA in the Files & versions tab.

Use it with the 🧨 diffusers library

Note: the base is an SDNQ fp8 checkpoint. For inference, install sdnq and dequantize the transformer to bf16 (from sdnq.quantizer import dequantize_sdnq_model), or load a non-quantized base.

from diffusers import Ideogram4Pipeline
import torch

pipeline = Ideogram4Pipeline.from_pretrained("Disty0/Ideogram-4-SDNQ-FP8", torch_dtype=torch.bfloat16)
pipeline.to('cuda')
pipeline.load_lora_weights('linoyts/ideogram4-yarn-art-fp8', weight_name='pytorch_lora_weights.safetensors')
image = pipeline('{"high_level_description":"A whimsical yarn-craft illustration of Yoda rendered entirely in knitted and crocheted fiber, mounted inside a round wooden embroidery hoop on a linen backdrop, a soft diffused daylight photograph with 1:1 aspect ratio.","compositional_deconstruction":{"background":"Neutral pale linen backdrop filling the frame, fine grain texture visible, soft diffused daylight from upper-left casting gentle shadows across the surface, palette dominated by mossy greens, cream, oatmeal browns, and warm wood tones against the pale linen ground.","elements":[{"type":"obj","desc":"Round wooden embroidery hoop framing the entire piece, light birch finish with a small brass tension screw at the bottom edge. The hoop sits flat on the linen backdrop, centered in the frame."},{"type":"obj","desc":"Yoda's head and shoulders sculpted from soft moss-green yarn, filling roughly 80% of the hoop interior, centered. Large pointed ears with visible stitch texture and tiny pink felt inner-ear patches. Wrinkled forehead formed by gathered cream-colored yarn tufts. Deep-set eyes made from two small black beads set into white felt circles. Long hooked nose crafted from a single rolled cord of darker green yarn. Mouth stitched as a small closed curve in dark thread. Wispy white beard and sideburns made from looped mohair fibers cascading down the chin. Three-dimensional stitch texture reads clearly under the soft directional light."},{"type":"obj","desc":"Small brown robe collar at the base of Yoda's neck, crocheted in chunky oatmeal-brown yarn with a loose cable-knit pattern, sitting just below the chin within the lower portion of the hoop."},{"type":"obj","desc":"Scattered loose yarn scraps in moss-green, cream, and oatmeal-brown resting on the linen surface beneath the lower-left edge of the hoop, falling into soft focus."},{"type":"obj","desc":"A pair of small wooden crochet hooks lying crossed on the linen surface beneath the lower-right edge of the hoop, light wood handles with tapered metal tips, softly out of focus."}]}}').images[0]

For more details, including weighting, merging and fusing LoRAs, check the documentation on loading LoRAs in diffusers

License

Please adhere to the licensing terms of the base model Disty0/Ideogram-4-SDNQ-FP8.

Intended uses & limitations

How to use

# TODO: add an example code snippet for running this diffusion pipeline

Limitations and bias

[TODO: provide examples of latent issues and potential remediations]

Training details

[TODO: describe the data used to train the model]