gatilin/uniddt_512_visual_components

UniDDT 512 — Noisy ViT and diffusion decoder

Lossless extraction of the official EMA inference weights from MCG-NJU/UniDDT, revision 68e02101a2815baebf0b760914d3ea1666b79c90, file vlm_uniddt_512.ckpt. Native code/config revision: d04e037c0e1011a64703ad97d7bc4993bb69eade.

Included

• Noisy ViT: 24 layers, width 1024, the final merger and all three trained DeepStack mergers; 566,295,552 parameters.
• FlattenDiT: 16 denoising layers and all four condition-refiner layers, input/output/timestep/position modules; 1,064,238,144 parameters.
• Original float32 EMA payloads, exact tensor-level provenance and ZIP CRC/SHA256 checks, native architecture, portable CPU implementation and reproducible validation.

The language backbone, its tokenizer, FLUX VAE and training projection/optimizer states are excluded. Duplicate denoiser.* aliases are not copied: the official release export ties these aliases to the ema_denoiser.* storage. Component prefixes are removed only at the documented file boundary; internal native names, shapes, dtypes and payload bytes are preserved.

Load offline

import torch
from component import load_model
encoder = load_model('/path/to/package', component='noisy_encoder')
decoder = load_model('/path/to/package', component='diffusion_decoder')
# Original pipeline uses BF16 mixed precision with FP32 EMA master weights.
with torch.inference_mode(), torch.autocast('cpu', dtype=torch.bfloat16):
    # Call the native forward interfaces with original packed patches,
    # positions, timestep, sequence lengths and language hidden states.
    pass

The Noisy ViT accepts packed 16-channel FLUX VAE latent patches, not RGB pixels. Each 2x2 patch is flattened in original channel/patch-row/patch-column order to 64 values. Spatial merging groups four patches. Position and normalized-position arrays follow the original packing order. The Noisy ViT outputs 2560-wide language inputs plus three 2560-wide DeepStack features.

The decoder accepts packed patches and the original Qwen3-VL-4B visual-position hidden states. It gathers the visual positions before condition refinement; those positions already carry language context from the complete language backbone. The original conditioning path is recomputed at denoising timesteps. A generic static text encoder is not a replacement. Both complete visual modules are reusable research components, not a self-contained prompt-to-image or image-understanding pipeline.

config.json defines both architectures. original_config.yaml retains the full upstream sampling/FLUX dependency contract. upstream/ contains the exact original architecture sources. native_adaptation.json records portable changes: private imports, eager execution, and dense SDPA evaluation of the same native packed mask predicates. Learned computations, original fixed packed modulation, and original condition-refiner normalization are retained. No silent architectural correction is applied.

Validation

Run python verify_release.py /path/to/package. The report records checks actually completed: every exported tensor SHA256, strict complete loading, all 24 Noisy ViT layers/three DeepStack outputs, all 16+4 decoder layers, independent explicit-softmax attention, timestep/conditioning effects and a short packed-latent ODE interface fixture. Tests use CPU BF16 autocast with original FP32 weights and small synthetic fixtures. The short Euler fixture does not reproduce the original 25-step AdamLM sampler. There is no full-language/FLUX decode, image-quality or paper-score claim.

Provenance and license

extraction.json, source_index.json, tensor_mapping.json and tensor_sha256.json record selection and integrity. The containing 22.6 GB checkpoint was indexed and selected storage blocks were fetched; this is not a whole-checkpoint LFS verification claim. Every selected original ZIP storage CRC32 and every persisted output payload SHA256 are checked.

The pinned repository has no top-level license file or weight-license model card. Included Qwen-derived files explicitly retain Apache-2.0 headers; other applicable upstream/base-model terms remain in force. This extraction does not invent a permissive weight license; see LICENSE.