gatilin/llada2_0_uni_image_tokenizer

LLaDA2.0-Uni image tokenizer

Vision encoder, VQ codebook, quant_conv and post_quant_conv; original sigvq_embedding.pt retained as an auxiliary decoder-conditioning artifact. PIL image -> discrete token IDs. The separate diffusion decoder and image VAE are excluded; inverse VQ features are not reconstructed pixels.

图像 tokenizer：保留视觉编码器、量化码本与投影，以及原始解码条件 embedding；不含独立扩散渲染器和图像 VAE。

Provenance

Extracted from inclusionAI/LLaDA2.0-Uni at dc2cde43e008aafd53ab6e4049187ae7e1cbc22e. Native code revision and attribution are recorded in code_provenance.json and NOTICE. No language model is instantiated by the loader. This repository is an unofficial component export, not an official release from the original authors.

• Tensors: 488; stored parameters/elements: 1,199,457,280.
• Tensor payload: 2,398,914,560 bytes; dtype: ['BF16'].
• Main checkpoint SHA256: e0a11a82ad221ac1f3b917abfce31ffaaec3571200ae7ee5318a223ff2eedc49.
• Full source configuration is preserved separately as source_config.json; the custom root config documents this package only. This is an explicit Python loader, not a registered AutoModel or timm architecture.

Use

Install requirements.txt in a dedicated environment. Source files are vendored at a pinned upstream revision. The loader strictly checks every state key.

import sys, torch
from huggingface_hub import snapshot_download
path = snapshot_download("gatilin/llada2_0_uni_image_tokenizer")
sys.path.insert(0, path)
from component import load_model
model = load_model(path)  # CPU; BF16 for UniAR, FP32 for other components
from PIL import Image
from component import tokenizer_view
tokenizer = tokenizer_view(model, path)
# Caller handles resizing/cropping. Dimensions must be divisible by 16.
info = tokenizer.encode_with_info(Image.new("RGB", (512, 512), "white"))
ids, grid = info["token_ids"], info["grid_thw"]
# IDs are local VQ indices. Apply the LLaDA language model's image_token_offset
# only when passing them into that model; this package does not add the offset.

Validation

validation.json records all checks: original source tensor bytes match, strict loading passes, 3 native equivalence cases pass, plus the useful-resolution smoke case 256_rgb_tokenization. Tests ran with FP32 on CPU. Where supported, native eager reference attention was compared to SDPA in the exported loader. Discrete token/bit outputs are checked exactly; floating outputs use tolerance documented in the validation script.

The reference loads original source ranges directly; the exported loader independently reads the assembled Safetensors checkpoint. Tests cover only the component APIs. They do not establish downstream understanding accuracy, text-to-image quality, or GPU/FlashAttention equivalence. Synthetic input images/tensors are used; reconstruction is lossy and no pixel-fidelity benchmark is claimed.

Reproduce extraction with extract_unified_components.py llada_image_tokenizer (default outputs alongside the script); then run python validate_unified_components.py PATH. The byte comparison requires .download source chunks, which extraction creates but this repository intentionally does not upload.

License and attribution

The original weight and code licenses remain applicable. Preserve upstream author notices. See LICENSE, NOTICE, SOURCE_README.md, and source-file headers.

sigvq_embedding.pt is retained byte-for-byte and verified against its upstream LFS SHA256 (auxiliary_provenance.json). It is decoder conditioning, separate from the VQ codebook. Use torch.load(..., weights_only=True) if inspecting it. The encoder source provides no pixel-decoding API.

Decoder conditioning bridge

from component import load_sigvq exposes the additional, strictly loaded upstream SigVQ bridge. load_sigvq(path)(torch.tensor([[0, 1, 16383]])) returns decoder-conditioning features (4096 channels), not pixels. sigvq_validation.json records exact reference equality on boundary and interior VQ token IDs, in addition to verification of the original .pt file LFS SHA256. The standard diffusion renderer, turbo renderer and pixel VAE remain separate upstream components.