diffu-moss-hand-pahlman

A LoRA adapter for MOSS-Image-Base that writes in the hand of Carl Fredrik Påhlman's travel journals (1820s Swedish cursive). It is part of diffu, which draws synthetic archive pages with exact ALTO and PAGE labels, for training handwriting and page-reading models.

A page window drawn by MOSS-Image-Base with this adapter; not a scan.

Files

File	What
adapter_config.json	a PEFT LoraConfig: rank 32, alpha 32, on 265 Linear layers of the DiT
adapter_model.safetensors	the LoRA weights (bfloat16), PEFT's key layout

The adapter covers the DiT's 42 joint blocks (q, k, v, o, ffn_up_gate, ffn_down), its text refiner, its two embedders and its final layer. The module names are MOSS-Image's own, so it applies to any implementation that keeps them; diffu's MossTransformer does.

Use

With diffu, loaded onto MOSS-Image-Base's own DiT at draw time (no merged model needed; diffu's docs/MOSS.md, "Hand LoRAs", and docs/INFERENCE.md):

hf download OpenMOSS-Team/MOSS-Image-Base --revision 85975793d8b3d432f1b6e27f352abaa4c2a4c0d3 --local-dir models/MOSS-Image-Base
hf download Gabriel/diffu-moss-hand-pahlman --local-dir models/adapters/pahlman
uv run diffu-moss-init --base models/MOSS-Image-Base --out models/moss-base
uv run diffu-run --jobs run.jsonl --draw-settings draw.json --model models/moss-base \
    --lora hand=models/adapters/pahlman --out out/pahlman --gpus 1

"fuse_loras": true in draw.json merges the adapter into the DiT's weights at load (needed with MXFP8 or compiled blocks).

Or with PEFT alone, merged into the DiT, which diffu then draws with like any MOSS-Image model:

from peft import PeftModel

from diffu_moss.transformer import MossTransformer

dit = MossTransformer.from_pretrained("<MOSS-Image-Base snapshot>/transformer")
dit = PeftModel.from_pretrained(dit, "Gabriel/diffu-moss-hand-pahlman").merge_and_unload()
dit.save_pretrained("dit-pahlman")

diffu-moss-init --base <MOSS-Image-Base snapshot> --transformer dit-pahlman --out moss-pahlman

Base snapshot: OpenMOSS-Team/MOSS-Image-Base at revision 85975793d8b3d432f1b6e27f352abaa4c2a4c0d3. The adapter takes no sample page: the hand comes from the adapter and the prompt.

Prompting

Name the collection in the prompt's caption as in training: "…a handwritten page from Carl Fredrik Påhlman's travel journals, dated 1825–1826…". diffu builds that caption from a page job's style (its provenance title and years).

Draw as MOSS-Image is drawn: about 1 MP, 50 steps, guidance 3, shift 5, block 22 skipped in the unconditional pass.

Training

• Data: 292 windows of about 1 MP cut from pages of Swedish archival handwriting ground truth, each captioned as a MOSS Layout prompt (the collection named, every line's exact text and box).
• Trainer: KohakuMossTrainer, a LoRA (LyCORIS, algo lora), rank 32, alpha 32; AdamW at 1e-4, 700 steps; MOSS-Image's own noise-level sampling; text encoder and VAE frozen.
• Format: converted from the trainer's LyCORIS export to PEFT's layout. The conversion was checked layer by layer: every key lands on a layer PEFT builds from this config, none is missing or left over, and scaling x B @ A equals the trained (alpha / r) x up @ down exactly.

Limitations

• One hand, from one collection; other hands and kinds of page are untested.
• The drawn text must be checked: diffu reads every drawn line back and tags a line it cannot confirm as unclear in the labels.

Licence

Apache-2.0, as MOSS-Image-Base.