GPOcc: Generalizing Visual Geometry Priors to Sparse Gaussian Occupancy Prediction

CVPR 2026 · Paper · Project Page

GPOcc takes a single RGB image, lifts it into sparse 3D semantic Gaussians using visual geometry priors (VGGT / Depth-Anything-V2), and splats them into a 60×60×36 semantic occupancy grid (8 cm voxels). The models were trained on OccScanNet.

Models

This repo contains two models, one per subfolder:

Subfolder	Backbone	Params	Size
vggt/ (default)	VGGT-1B aggregator	942M	3.8 GB
dpt/	Depth-Anything-V2 ViT-B	98M	0.4 GB

Usage

Install the self-contained inference package from the demo/ folder of the GitHub repo (pip install -r requirements.txt). It needs PyTorch with CUDA and nvcc. The CUDA op is compiled automatically the first time you load a model.

from gpocc_hub import GPOcc, CLASS_NAMES

model = GPOcc.from_pretrained("JuIvyy/GPOcc").cuda().eval()                    # VGGT
# model = GPOcc.from_pretrained("JuIvyy/GPOcc", subfolder="dpt").cuda().eval()  # DPT

out = model.predict("room.jpg", intrinsics=K)   # K: 3x3, optional
out["occupancy"]      # (60, 60, 36) class index per voxel, 12 = empty
out["voxel_centers"]  # (60, 60, 36, 3) world coordinates

Command-line demo (saves the occupancy as a colored .ply point cloud):

python demo.py --model JuIvyy/GPOcc --image examples/scene0000_00_00061.jpg

Classes: ceiling, floor, wall, window, chair, bed, sofa, table, tvs, furniture, objects (plus empty).

License

For non-commercial research use only. The weights are derived from VGGT-1B and Depth-Anything-V2-Base (CC-BY-NC-4.0).

Citation

@inproceedings{zhou2026generalizing,
  title={Generalizing Visual Geometry Priors to Sparse Gaussian Occupancy Prediction},
  author={Zhou, Changqing and Luo, Yueru and Chen, Changhao},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={28578--28587},
  year={2026}
}