ViewWeaver

Code: https://github.com/liyaowei-stu/ViewWeaver

ViewWeaver: Geometry-Grounded Generative Rendering for 3D-Aware Image Customization

ViewWeaver is a geometry-grounded generative rendering framework for 3D-aware image customization. Given multi-view reference images, a target camera, and a text instruction, it generates customized subject images while preserving subject identity and 3D structure across viewpoints and scenes.

This repository contains the ViewWeaver adapter and view-conditioning weights. Use them with the ViewWeaver inference code, FLUX.1-Kontext-dev, and VGGT-1B.

Checkpoint contents

File	Size (bytes)	Component
lora_dit.safetensors	1,388,016,360	Diffusion transformer LoRA weights
view_moe_double.safetensors	89,694,626	View-conditioning module for double-stream blocks
view_moe_single.safetensors	89,694,626	View-conditioning module for single-stream blocks

All three files are required. The FLUX base model and VGGT weights must be downloaded separately. These files require the ViewWeaver custom loader and do not form a standalone Diffusers pipeline.

Download

Run from the ViewWeaver code directory. First obtain access to FLUX.1-Kontext-dev and accept its terms on its model page.

hf auth login
hf download Yw22/ViewWeaver --local-dir checkpoints/viewweaver
hf download facebook/VGGT-1B --local-dir checkpoints/vggt
hf download black-forest-labs/FLUX.1-Kontext-dev --local-dir checkpoints/flux-kontext

Expected layout:

checkpoints/
├── viewweaver/
│   ├── lora_dit.safetensors
│   ├── view_moe_double.safetensors
│   └── view_moe_single.safetensors
├── vggt/
│   └── model.safetensors
└── flux-kontext/
    ├── model_index.json
    ├── transformer/
    ├── vae/
    ├── text_encoder/
    ├── text_encoder_2/
    ├── tokenizer/
    ├── tokenizer_2/
    └── scheduler/

Inference

Use a CUDA GPU with bfloat16 support. The following installation commands match the ViewWeaver project instructions for CUDA 12.8; choose an appropriate PyTorch build for other CUDA versions.

conda create -n viewweaver python=3.12 -y
conda activate viewweaver
python -m pip install torch==2.8.0 torchvision==0.23.0 \
  --index-url https://download.pytorch.org/whl/cu128

cd /path/to/ViewWeaver
python -m pip install -e '.[prepare]' \
  'numpy==1.26.4' 'opencv-python==4.11.0.86'
export PYTHON="$(command -v python)"

After downloading the checkpoints, run the included Mario example from the code directory:

CUDA_VISIBLE_DEVICES=0 bash scripts/run_case.sh --case examples/mario \
  --reference-view 0 --azimuth -15 --elevation 15 \
  --camera-distance 4.5 --offset-y 0.18 --save-comparison

The default workflow uses eight reference images with foreground masks, performs VGGT geometry preparation, and generates one target view. Generation defaults are 1024 × 1024 pixels, 32 steps, and guidance scale 3.5. Outputs are written to outputs/<case>/<timestamp>/, including the generated image, reference–target comparison, and result.json.

For your own inputs, provide a case directory with case.json, eight reference images, and matching foreground masks (white foreground, black background). RGBA images can supply foreground masks through their alpha channel. The ViewWeaver code documentation describes the full case format, reusable preparation caches, and camera controls.

License and dependencies

This repository retains its MIT license metadata. FLUX.1-Kontext-dev, VGGT, and any example assets remain subject to their respective licenses and terms; the repository metadata does not replace those terms.

Citation

@inproceedings{li2026viewweaver,
  title={ViewWeaver: Geometry-Grounded Generative Rendering for 3D-Aware Image Customization},
  author={Li, Yaowei and Li, Xiaoyu and Zhang, Zhaoyang and Li, Hongxiang and Chen, Long and Shan, Ying and Zou, Yuexian},
  booktitle={Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers},
  pages={1--12},
  year={2026}
}