LITHE adapters for FLUX.1-dev

This repository holds the 110 rank-16 FLUX.1-dev adapters evaluated in the NeurIPS 2026 paper "LITHE: Lattice-Indexed Twin Hadamard Encoding for Diffusion Personalization" (to appear), stored in the LITHE index format. Code: https://github.com/JianJiang1995/LITHE

What the files are

Each .lithe file is one adapter. It contains no float weights: every 64-value block of every LoRA factor is stored as three 9-bit codeword IDs into a fixed Hadamard codebook that is rebuilt from a closed-form rule at load time (nothing is learned or shipped besides the IDs). The ID stream (uint16) is compressed with LZMA (preset 9, extreme) and preceded by a small compressed JSON header with the module names, factor shapes, codebook parameters, checksums and the training metadata listed in manifest.csv.

• Configuration: Kohya sd-scripts FLUX layout, 304 fused linears per adapter (attention, MLP and modulation linears of all 19 double-stream and 38 single-stream blocks; header field module_set = kohya-304), rank 16, alpha 16, K = 512 codewords, 3 residual levels, block size 64. This is the configuration used for the quality results of the paper.
• Size: 3,589,632 indices per adapter; 3.23 MB per file on average (355.7 MB for all 110). The decoded bf16 LoRA is about 153 MB.
• The adapters were trained natively in the discrete space (straight-through estimator), not quantised after training.

Usage

git clone https://github.com/JianJiang1995/LITHE.git && cd LITHE
pip install -e ".[flux]"
huggingface-cli download Fanjue/LITHE --local-dir ./lithe_adapters
python scripts/decode_to_lora.py --lithe ./lithe_adapters/indices/dreambooth30/db_cat.lithe --out db_cat_lora.safetensors

The decoded file uses Kohya LoRA keys, which the stock diffusers loader converts on load:

import torch
from diffusers import FluxPipeline

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to("cuda")
pipe.load_lora_weights("db_cat_lora.safetensors")
image = pipe("a photo of a <v_sub_001> cat on a beach", num_inference_steps=28, guidance_scale=3.5).images[0]

Or decode in memory without writing a file:

import lithe
sd = lithe.to_lora_state_dict(lithe.load_lithe("lithe_adapters/indices/dreambooth30/db_cat.lithe"))
pipe.load_lora_weights(sd)

scripts/infer.py --mode peft and scripts/serve_demo.py in the code repository also accept these files. Each file's trigger token and an example prompt are in its header metadata and in manifest.csv. The paper evaluated at 1024 x 1024, 28 steps, guidance 3.5.

Contents

Pool	Folder	Adapters	Trigger tokens	Training	Mean size	Notes
DreamBooth-30	indices/dreambooth30/	30	<v_sub_001> ... <v_sub_030>	500 steps, 512x512	3.31 MB	Subjects of the public DreamBooth dataset (objects and pets), 4 to 6 images each.
CC101-20	indices/cc101_20/	20	<v_sub_C001> ... <v_sub_C020>	500 steps, 512x512	3.32 MB	20 subjects sampled from the public CustomConcept101 dataset, one per category, 3 to 10 images each.
SCIdentity	indices/scidentity/	20	<v_id_001> ... <v_id_020>	500 steps, 512x512	3.27 MB	20 synthetic animal identities (12 generated images each; images not released).
SC-40	indices/sc40/	40	<v_outfit_001> ... <v_outfit_008>; <v_light_001> ... <v_light_008>; <v_pose_001> ... <v_pose_008>; <v_pstyle_001> ... <v_pstyle_008>; <v_anime_001> ... <v_anime_008>	750 steps, 512x512	3.11 MB	40 synthetic concept-transfer targets: 8 outfits, 8 lighting setups, 8 poses, 8 painter styles, 8 anime styles (20 generated images each; images not released).

Prompt pattern: identity adapters were trained with a single caption such as a photo of a <v_sub_001> cat. SC-40 adapters were trained on per-image captions that contain the trigger and the concept label; manifest.csv gives the evaluation prompt used for each (SC-40 groups: anime style 8, lighting 8, outfit 8, painter style 8, pose 8).

manifest.csv columns: pool, file, concept_id, trigger, description, example_prompt, module_set, n_modules, rank, n_indices, bytes, sha256, train_steps, train_resolution. SHA256SUMS lists checksums of all files.

Data and license

• The adapters are derived from FLUX.1-dev and inherit the FLUX.1 [dev] Non-Commercial License. Use them only under its terms.
• DreamBooth-30 adapters were trained on the DreamBooth dataset (Ruiz et al., CVPR 2023, https://github.com/google/dreambooth). CC101-20 adapters were trained on CustomConcept101 (Kumari et al., CVPR 2023, https://github.com/adobe-research/custom-diffusion). The images are not redistributed here; see the original datasets for their terms.
• SCIdentity and SC-40 adapters were trained on synthetic images generated for the paper. Their training data is not released; the generation protocol is described in the paper appendix.

Citation

@inproceedings{jiang2026lithe,
  title     = {{LITHE}: Lattice-Indexed Twin Hadamard Encoding for Diffusion Personalization},
  author    = {Jiang, Jian and Celiktutan, Oya and Wang, Yaohui and Ban, Yutong},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026},
  note      = {To appear. Code: https://github.com/JianJiang1995/LITHE}
}