OzzyGT/moss_image_custom_blocks

MOSS-Image Modular Diffusers

Modular Diffusers blocks for OpenMOSS-Team/MOSS-Image-SFT, a 5B text-to-image model with a Qwen3.5 text encoder and the FLUX.2 VAE that renders English and Chinese text and supports layout prompts with bounding boxes. The model code and weights are in OzzyGT/MOSS_Image_SFT_diffusers, and there are SDNQ quantized versions in 8-bit and 4-bit.

Note: This model needs diffusers installed from source, pip install git+https://github.com/huggingface/diffusers. With every component on the GPU, the example below needs more than 32 GB of VRAM at 2048 x 1152. For smaller GPUs, use one of the SDNQ versions as the components_repo and add import sdnq before loading.

Note: both flags run Python code downloaded from the Hub: trust_remote_code=True runs the code of this blocks repository, and trust_components_code=True runs the code shipped in the components_repo. Only use them with repositories you trust, and review their code before passing different ones.

Sample image

This image was generated with the following code, seed 7:

import json

import torch
from diffusers import ModularPipelineBlocks


blocks = ModularPipelineBlocks.from_pretrained(
    "OzzyGT/moss_image_custom_blocks",
    trust_remote_code=True,
    components_repo="OzzyGT/MOSS_Image_SFT_diffusers",
    trust_components_code=True,
)
pipe = blocks.init_pipeline()
pipe.load_components(dtype={"default": torch.bfloat16, "vae": torch.float32})
pipe.to("cuda")

layout = {
    "summary": {
        "meta": ["photo"],
        "canvas": "single",
        "caption": (
            "A photograph of the inside of a small bicycle repair workshop, shot on a Sony A7 IV with a 35mm f/1.8 "
            "lens at f/2.8, 1/125 s, ISO 1600, handheld at eye level, wide 16:9 frame. Low late-afternoon sunlight "
            "comes through a large front window on the right in hard shafts with dust in the air, the rest of the "
            "room is lit by bare fluorescent tubes on the ceiling. Shallow depth of field with the back wall slightly "
            "out of focus, visible sensor grain. A red road bicycle is clamped in a repair stand in the center while "
            "an adult mechanic in a grease-stained navy work apron adjusts its rear derailleur. A painted wooden sign "
            'on the back wall reads "SPOKE & CHAIN". A pegboard of hand tools hangs on the left wall above an A-frame '
            "chalkboard with a price list, and an adult black cat sleeps curled up on a metal stool by the window. "
            "Oil stains on a grey concrete floor, spare tires hanging from hooks, scuffed white walls."
        ),
    },
    "items": [
        {
            "label": "text",
            "caption": "A long painted wooden sign on the back wall, dark green with cream serif capital letters.",
            "text": "SPOKE & CHAIN",
            "bbox": [-496, -464, -346, 236],
        },
        {
            "label": "object",
            "caption": "A perforated pegboard with wrenches, screwdrivers, tire levers and a chain whip on hooks.",
            "bbox": [-396, -984, 124, -544],
        },
        {
            "label": "object",
            "caption": "A red steel road bicycle with drop handlebars, clamped by its seat post in a grey repair stand.",
            "bbox": [-196, -504, 329, 371],
        },
        {
            "label": "object",
            "caption": (
                "An adult man in his forties with short grey hair, a grey t-shirt and a grease-stained navy work "
                "apron, standing and turning a small screwdriver on the rear derailleur."
            ),
            "bbox": [-371, 396, 504, 736],
        },
        {
            "label": "text",
            "caption": "A black A-frame chalkboard on the floor with a handwritten price list in white chalk.",
            "text": "FLAT REPAIR $12\nFULL TUNE-UP $45\nOPEN 8-6",
            "bbox": [184, -964, 524, -604],
        },
        {
            "label": "object",
            "caption": "An adult black cat sleeping curled up on the seat of a metal shop stool in a patch of sunlight.",
            "bbox": [104, 766, 209, 981],
        },
    ],
    "bbox_coord_space": "abs_centralized",
    "bbox_format": "yxyx",
}
prompt = json.dumps(layout, ensure_ascii=False)

image = pipe(
    prompt=prompt,
    num_inference_steps=28,
    height=1152,
    width=2048,
    generator=torch.Generator("cuda").manual_seed(7),
    output="images",
)[0]
image.save("bike_workshop.png")

Tips

The model accepts natural language prompts, tag lists and layout prompts. A layout prompt is a JSON object, passed as a string with json.dumps, that describes the whole scene in summary.caption and lists each element in items with a label (object or text), a caption, the exact text to render for text items, and a bbox. Boxes are [y_min, x_min, y_max, x_max] in pixels with the origin at the center of the canvas, so for a 2048 x 1152 image x goes from -1024 to 1024 and y from -576 to 576. See the user guide for the full format and more prompting advice.

To control line breaks in rendered text, put \n in the text.

The blocks default to 28 steps and a guidance scale of 3.0, which give good results in about half the time of the official recommendation of 50 steps or more. Raise num_inference_steps for the best quality, and the guidance scale for short prompts. The recommended sizes are 1 to 2 megapixels, but the model works up to 4 megapixels (the sample is 2048 x 1152, about 2.4). Width and height must be multiples of 16.