AiArtLab/mossling

MOSS-Image pruned to fit a 16 GB card: a 30-block DiT instead of 42, a 0.8B text encoder with a 158M adapter instead of the 15.6 GB 9B one. Text-to-image at 1024 px, 12.4 GB peak, one consumer GPU.

This is an alpha. It was assembled in two days on a single RTX 4080 (16 GB) and it is undertrained: see Status. Treat it as a working proof that the base model survives a 26.7% cut and a 16× smaller encoder, not as a finished model.

transformer — 30 blocks, 3.723 B parameters, bf16 (upstream: 42 blocks, 5.082 B). Twelve blocks were fused into their neighbours by learned rank-32 merges and then baked into the weights.

text encoder — Qwen3.5-0.8B, 1.7 GB, frozen, plus a 158M adapter head (upstream: Qwen3.5-VL-9B, 15.63 GB). Cosine 0.924 against the native encoder's condition on held-out captions.

VAE — untouched AutoencoderKLFlux2, fp32.

scheduler — MOSSImageScheduler, flow-matching Euler, static shift 5.0.

resolution — 1024 by default, any positive multiple of 16; 3:4 and 4:3 work (the samples below are 864x1152 and 1152x1152).

peak VRAM — 12.4 GB at 1024 px / 30 steps, one forward pair per step, no offload. 9.7 GB at 768x1152 when the encoder is freed before the DiT runs; ~1.15 it/s on a 4080.

What changed

Block fusion. Middle blocks of a DiT move the stream little and partly cancel each other, so three adjacent blocks become one: block s stays and gets a LoRA, blocks s+1, s+2 are deleted, and the LoRA is trained to make block s reproduce the original output of block s+2. Seven windows were trained at 1024 px, six went into the artifact — 42 − 12 = 30 blocks. The merge is baked into the weights (W' = W + B·A·scale), so inference is not slowed down and nothing extra has to be loaded.

Encoder replacement. The 15.63 GiB 9B encoder is replaced by Qwen3.5-0.8B plus an adapter that imitates what the 9B encoder emitted, wired in as the text_encoder component (MOSSAdapterTextEncoder). It returns the same (B, L, 4096) states, so encode_prompt, the CFG negative branch and every __call__ argument keep working.

Block skipping is off by default in this folder. The stock 42-block recipe skips layer 22 in the unconditional branch; in a 30-block DiT that index points at a different block and visibly wrecks the image (acid colours, red skin) — measured, not guessed. Pass uncond_skip_layers only with indices you measure yourself.

Both earlier models (the 42-block DiT and the 9B encoder) are not in this folder. They are unmodified files from OpenMOSS-Team/MOSS-Image-Base: re-download that repo into a scratch directory to rebuild the compression with src/run_hi.sh.

Why "Mossling"

A mossling is a small moss. The base model is MOSS-Image, this is the same plant, 26.7% smaller, growing on a 16 GB card. It is an unofficial derivative, not a MOSS-Image release and not affiliated with OpenMOSS / MOSI.AI.

Examples

Every image below comes from this folder: 50 steps, CFG 3, seed 7, each at the resolution of its source row. All four prompts are rows of the fine-tune set, copied verbatim — tag lists included, since that is the text the model was trained on.

	prompt
	black sweater, brown skirt, closed mouth, ear piercing, earrings, hair behind ear, hair intakes, hairclip, jacket, jewelry, large breasts, long hair, long sleeves, off shoulder, pencil skirt, piercing, skirt, sweater, sweater tucked in, undressing, white jacket
	This highly detailed CGI rendering captures the iconic character Wolverine from Marvel Comics in his futuristic armor suit. The costume features intricate blue and gold plates, with sharp claws protruding from both hands, emphasizing his signature weapon. The background is dimly lit, highlighting the armored figure's muscular physique and determined expression under a glowing helmet.
	animal hands, animal print, brown eyes, cat hair ornament, cat paws, cat print, cowboy shot, dragon tail, fish skeleton print, hair ornament, japanese clothes, kimono, limited palette, long hair, obi, obijime, parted lips, sash, standing, white hair, yukata
	A cartoon of a grey-furred female mouse with golden eyes, her body retaining all the natural, unmistakable features of a mouse—small, round ears, a long, twitching tail, and paws with delicate fingers. She wears simple beige clothing suited for travel, with small, practical bronze armor pieces tailored perfectly for her size. A long, curved sword rests on her shoulder, the hilt grasped firmly in her paw as her stance remains low to the ground, reflecting her nimbleness. The background is an arid desert, with the ruins of an ancient temple in the distance, weathered by time and partially overtaken by shifting sands.

The mouse row's caption continues with twelve random words ("Florgnax qwermbly …"); they are dropped here and were kept out of the prompt.

Usage

import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(".", custom_pipeline="pipeline_moss_image",
                                         trust_remote_code=True,
                                         dtype={"default": torch.bfloat16, "vae": torch.float32})
pipe.to("cuda")

image = pipe("a red fox walks through a snowy forest at dusk, cinematic, 85mm",
             height=1152, width=768, num_inference_steps=50, guidance_scale=3.0,
             generator=torch.Generator("cuda").manual_seed(7)).images[0]

model_index.json declares our text encoder as the text_encoder component, so prompt= works; the adapter head is taken from the sibling adapter/adapter.safetensors. Verified on diffusers 0.41.0.dev0 — the dtype= dict form is recent (model_index.json records _diffusers_version: 0.38.0); older releases expect torch_dtype=.

Low-VRAM CLI from the adapter repo — it encodes the condition up front and frees the 0.8B encoder before the DiT runs:

python3 adapter/src/example.py \
    --prompt "a red fox walks through a snowy forest at dusk, cinematic, 85mm" --out fox.png

Training and compression scripts are in src/ (src/README.md): window.py + run_hi.sh rebuild the block fusion from the original 42-block weights, make_dataset.py turns captioned images into cached latents/conditions, train.py is a fine-tuning loop that fits the 3.72 B DiT into 16 GB.

Status

Honest list, this is what "alpha" means here:

• The fine-tune is one epoch and stops there. The DiT in this folder is the fused 30-block build after a single pass over 39 576 captioned rows: 4 947 steps at batch 8 (lr 1e-5 cosine to 1e-6, 500-step warmup, no EMA). The loss flattens early and stays noisy — 0.32 to 0.50 with no trend after the first thousand steps. The previous build of this card stopped at 25 000 of 39 025 steps (batch 1) of an earlier run; the final epoch is not a clear win over it on photo prompts: on the four prompts that were on this card before, the final weights keep fine texture but print structure errors (a faint second profile behind the sitter's hair, fused feet, extra fingers), and the images read less photographic. On dataset-style tag prompts — the four above — output stays clean, which is what the training saw. Expect roughness on hands, complex geometry and anything outside the training distribution.
• Not a full re-train. The base model's own training is inherited; only the block merges and the short fine-tune above are new.
• Text inside the image degrades — signs and JSON layouts come out garbled, though one large word can land. Two of the fused blocks carried that ability, and one block plus a low-rank patch does not reproduce it.
• JSON layout prompts are the weakest case for the adapter as well (cosine 0.756): it was trained on captions, not on layout instructions — the two limitations stack on sign/layout prompts.
• One seed per prompt in the samples. Quality varies with seed, and the four images above are one draw each, seed 7, nothing re-rolled. Four random rows were generated, two were kept; the armoured-figure and mouse rows were on the card before and were re-shot on the final weights.
• English prompts, as in the dataset behind the adapter.

Quality against the base model, measured on the pruned-but-not-fine-tuned build (not re-measured for the final epoch):

	stock 42-block + 9B	this folder
peak VRAM, 1024 px / 30 steps	~25.4 GiB	12.4 GiB
CLIP, 1024 px / 30 steps	29.59	29.12
CLIP, 512 px / 20 steps	29.49	29.18

What was tried and rejected (refit on self-generated trajectories, FFN narrowing) is written down in src/README.md so nobody repeats it.

Files

pipeline_moss_image.py   MOSSImagePipeline, MOSSImageTransformer, MOSSAdapterTextEncoder, scheduler
model_index.json         diffusers wiring: 30-block transformer, adapter encoder, VAE, scheduler
transformer/             DiT config + bf16 weights (3.723 B)
text_encoder/            Qwen3.5-0.8B, frozen
adapter/                 adapter v1: adapter.safetensors (158M head) + its own src/ and samples/
vae/                     untouched AutoencoderKLFlux2, fp32
scheduler/, tokenizer/   untouched
samples/                 the four images above + hero.jpg
src/                     compression and training scripts, plus the method write-up

License and NOTICE

The weights and the bundled inference code are released under the Apache License, Version 2.0.

This is a derivative of OpenMOSS-Team/MOSS-Image-Base (MOSS-Image, MOSI.AI). The 30-block DiT, the 0.8B encoder and the adapter are modifications of / replacements for components of that model. It is not an official MOSS-Image release and is not endorsed by the upstream authors.

We thank the Diffusers, Qwen, FLUX and OpenMOSS teams.

Contacts

Please contact us if you can provide GPUs or money for training.

• telegram recoilme preferred
• mail at aiartlab.org (slow response)

Citation

@misc{mossling,
  title={Mossling: MOSS-Image pruned to a 16 GB card},
  author={recoilme and AiArtLab Team},
  url={https://huggingface.co/AiArtLab/mossling},
  year={2026}
}