PINE: Point, Instance and Normal Estimation in One-Step Diffusion Model

Zeyu Ren, Jie Ni, Jiahao Wei, Hao Tang, Yongtai Liu, Zeyu Zhang, Rui Qian

Code | Website

PINE is a one-step Diffusion Transformer that predicts the point map, the surface normals and the instance map of an image in one forward pass. Each map is a token stream with its own input and output layers; the streams share the transformer blocks, and a stream mask decides which streams exchange tokens. A camera-aware ray encoding positions every token by its camera ray, and grounding attention adds the affinity of frozen DINOv3 features to the attention logits.

Models

All models run at 512×512 with one sampling step. Rel^p is the point map error on seven zero-shot datasets (DIODE, KITTI, NYUv2, ETH3D, HAMMER, iBims-1, Booster), with and without camera intrinsics.

Model	File	DINOv3 encoder	Parameters	Relp with K	Relp without K
PINE-B	pine-b-512.pth (0.6 GB)	ViT-B/16	0.24 B	4.69	5.62
PINE-L	pine-l-512.pth (2.0 GB)	ViT-L/16	0.80 B	3.76	4.70
PINE-H	pine-h-512.pth (4.0 GB)	ViT-H+/16	1.85 B	3.33	4.16

A file holds the EMA weights of the last training epoch. The frozen DINOv3 encoder is not part of the files: its weights are gated. Request access on the DINOv3 repository and place them in pretrained/dinov3/.

Usage

git clone https://github.com/AIGeeksGroup/PINE.git && cd PINE
hf download zeyuren2002/PINE pine-l-512.pth --local-dir pretrained
IMAGE_DIR=assets/demo bash scripts/demo_l_512.sh

import torch
from util.checkpoint import load_model

model = load_model('pretrained/pine-l-512.pth')            # one sampling step
image = ...                                                 # (1, 3, H, W) in [-1, 1], H and W multiples of 16
with torch.amp.autocast('cuda', dtype=torch.bfloat16):
    maps = model.generate(image.cuda(), intrinsics=None)   # (1, 9, H, W): point map, normals, instance map
points, normals, instances = maps.float().chunk(3, dim=1)
normals = torch.nn.functional.normalize(normals, dim=1)

See the code repository for installation, evaluation and training.

License

The weights are for non-commercial use only, under the CC BY-NC 4.0 license. The code is released under the Apache License 2.0.

Citation

@article{ren2026pine,
  title={PINE: Point, Instance and Normal Estimation in One-Step Diffusion Model},
  author={Ren, Zeyu and Ni, Jie and Wei, Jiahao and Tang, Hao and Liu, Yongtai and Zhang, Zeyu and Qian, Rui},
  year={2026}
}