UniSpace

Unified Visual Representation and Scalable Multimodal Modeling

HF Paper · arXiv · Code · Project Page

UniSpace is a unified visual representation framework spanning image understanding, reconstruction, generation, and instruction-based editing. This repository provides the selected inference checkpoints and normalization statistics associated with the paper.

Overview

UniSpace contains two related components:

1. Patch-reparameterized vision encoders. PR-SigLIP2, PR-DINOv2, and PR-Qwen-ViT retain the semantic representation of pretrained vision encoders while adding the image detail needed for faithful reconstruction and high-quality generation.
2. The UniSpace multimodal model. A Qwen-based patch-reparameterized visual tokenizer is coupled with a Qwen3-8B Mixture-of-Transformers model for visual understanding, text-to-image generation, and image editing.

The public code release is inference and evaluation only. UniSpace training code, optimizer states, private data pipelines, and cluster-specific files are not included.

Checkpoints

Artifact	Role	Used for
encoders/pr-siglip2-tokenizer.pt	Unified PR-SigLIP2 tokenizer	Reconstruction and ImageNet generation
encoders/pr-dinov2-tokenizer.pt	PR-DINOv2 tokenizer	Reconstruction and generation
encoders/pr-qwen-vit-tokenizer.pt	PR-Qwen-ViT tokenizer	Reconstruction and UniSpace initialization
encoders/pr-siglip2-dit.pt	Class-conditional DiT	PR-SigLIP2 ImageNet generation
encoders/pr-dinov2-dit.pt	Class-conditional DiT	PR-DINOv2 ImageNet generation
stats/*.pt	Latent normalization statistics	Matching encoder evaluation configs
unispace-sft-0012000/model.safetensors	Final selected UniSpace SFT checkpoint	Understanding, generation, and editing

The selected UniSpace model is SFT step 0012000, initialized from stage-3 step 0060000. Earlier training checkpoints are not required for inference. File digests are provided in SHA256SUMS.

Download

Download the complete release snapshot:

hf download yjb6/UniSpace --local-dir checkpoints/UniSpace

Or download one artifact:

hf download yjb6/UniSpace \
  encoders/pr-dinov2-tokenizer.pt \
  --local-dir checkpoints/UniSpace

Expected layout:

checkpoints/UniSpace/
├── encoders/
│   ├── pr-siglip2-tokenizer.pt
│   ├── pr-dinov2-tokenizer.pt
│   ├── pr-qwen-vit-tokenizer.pt
│   ├── pr-siglip2-dit.pt
│   └── pr-dinov2-dit.pt
├── stats/
│   ├── pr-siglip2-normalization-stats.pt
│   ├── pr-dinov2-normalization-stats.pt
│   └── pr-qwen-vit-normalization-stats.pt
└── unispace-sft-0012000/
    └── model.safetensors

Installation

git clone https://github.com/yjb6/UniSpace.git
cd UniSpace
conda env create -f environment.yml
conda activate rae
hf download yjb6/UniSpace --local-dir checkpoints/UniSpace

The verified environment uses Python 3.10, PyTorch 2.8.0, torchvision 0.23.0, Transformers 4.57.3, and Accelerate 1.12.0. See the GitHub reproduction guide for dataset preparation, model dependencies, distributed launch commands, and canonical evaluator setup.

Reproduce encoder reconstruction

Prepare ImageNet-1K in its standard class-directory layout and set the paths described in the code repository. For example:

cd patch-reparameterization

bash run_eval_only.sh configs/release/pr-siglip2-imagenet256.yaml \
  ../checkpoints/UniSpace/encoders/pr-siglip2-tokenizer.pt \
  --output-dir ../outputs/pr-siglip2-recon \
  --num-samples 50000 --batch-size 32 --no-zeroshot

bash run_eval_only.sh configs/release/pr-dinov2-imagenet256.yaml \
  ../checkpoints/UniSpace/encoders/pr-dinov2-tokenizer.pt \
  --output-dir ../outputs/pr-dinov2-recon \
  --num-samples 50000 --batch-size 64 --no-zeroshot

bash run_eval_only.sh configs/release/pr-qwen-vit-imagenet256.yaml \
  ../checkpoints/UniSpace/encoders/pr-qwen-vit-tokenizer.pt \
  --output-dir ../outputs/pr-qwen-vit-recon \
  --num-samples 50000 --batch-size 16 --no-zeroshot

Each run evaluates EMA weights and reports PSNR, SSIM, rFID, and sample count. The complete generation and multimodal benchmark commands are documented in the main README.

Results

Patch-reparameterized encoders: reconstruction

ImageNet-1K validation, 256 × 256, 50,000 images:

Encoder	PSNR ↑	SSIM ↑	rFID ↓
PR-SigLIP2	29.64	0.87	0.18
PR-DINOv2	30.84	0.90	0.14
PR-Qwen-ViT	30.16	0.88	0.17

Patch-reparameterized encoders: ImageNet generation

ImageNet-1K class-conditional generation, 256 × 256, 50,000 images:

Encoder	CFG	gFID ↓	sFID ↓	IS ↑	Precision ↑	Recall ↑
PR-SigLIP2	1.0	4.409	6.675	190.66	0.722	0.646
PR-SigLIP2	1.2	2.799	6.055	248.05	0.777	0.611
PR-DINOv2	1.0	2.100	5.377	216.97	0.779	0.637
PR-DINOv2	1.2	1.877	4.888	274.16	0.822	0.605

UniSpace generation

Benchmark	Breakdown	Scores	Overall ↑
GenEval	Single / Two / Count / Color / Position / Attribute	0.98 / 0.92 / 0.69 / 0.88 / 0.83 / 0.73	0.84
DPG-Bench	Global / Entity / Attribute / Relation / Other	84.80 / 92.26 / 90.00 / 94.97 / 88.80	86.49
OneIG EN	Align / Text / Reason / Style / Diversity	0.860 / 0.937 / 0.311 / 0.467 / 0.233	0.561
OneIG ZH	Align / Text / Reason / Style / Diversity	0.807 / 0.881 / 0.276 / 0.455 / 0.244	0.533

UniSpace editing

Benchmark	Breakdown	Scores	Overall ↑
ImgEdit	Add / Adjust / Extract / Replace / Remove / Background / Style / Hybrid / Action	4.53 / 4.38 / 3.61 / 4.67 / 4.42 / 4.23 / 4.55 / 2.70 / 4.47	4.28
GEdit EN	Semantic consistency / Perceptual quality	8.287 / 7.055	7.407
GEdit ZH	Semantic consistency / Perceptual quality	8.270 / 6.998	7.382

Judge-based editing scores may vary with the evaluator endpoint and model version. The clean ImgEdit rerun generated all 737 expected outputs and scored 4.25 overall; category differences from the paper result were within 0.30.

Intended use and limitations

The release is intended for research on visual representation learning, multimodal inference, image generation, image editing, and reproducible benchmarking. Generated or edited images may contain factual, compositional, text-rendering, or perceptual errors. Users should independently assess outputs before using them in consequential settings. The release does not add application-specific safety guarantees beyond those of its base models.

Citation

@article{yan2026unispace,
  title   = {UniSpace: Unified Visual Representation and Scalable Multimodal Modeling},
  author  = {Yan, Jinbo and Qiao, Limeng and Qin, Jie and He, Jun-Yan and Wu, Feize and Wan, Guanglu},
  journal = {arXiv preprint arXiv:2608.08676},
  year    = {2026}
}

License and third-party components

Repository code is distributed under the licenses included in the GitHub release. Base-model and benchmark components retain their respective licenses and terms. See THIRD_PARTY.md before redistribution or commercial use.