π⁴: A Foundation Model for 4D Scene Geometry and Motion

Physical Intelligence Team · Shanghai Artificial Intelligence Laboratory

Official Pi4 model weights. See the citation.

From pixels to a scene in motion. Pi4 brings geometric reconstruction and motion estimation into a single model. It accepts single images, unordered image collections, and videos without supplied camera calibration, building a shared 3D scene representation in one forward pass.

  • One compact model. Approximately 0.5B parameters for joint geometry and motion.
  • Strong across tasks. The best overall average rank among the compared methods across the paper's 16 geometry evaluation settings, with competitive results in single- and multiview 2D/3D tracking and moving-object segmentation.
  • Thousands of images, one forward pass. Joint reconstruction and tracking from thousands of input images, despite training with at most 64 views.

Usage

Install the Pi4 repository first. Inference requires a CUDA-capable NVIDIA GPU.

import torch
from pi4 import Pi4

model = Pi4.from_pretrained("yyfz233/Pi4").cuda().eval()
images = torch.rand(1, 2, 3, 256, 384, device="cuda")
prediction = model(images, query_frame=0)

Replace the example tensor with RGB images in [0, 1], shaped [B, T, 3, H, W]. Height and width must be divisible by 16. Tracking requires at least two images. Omit query_frame for geometry only, including single-image inference.

From the repository root:

python example.py examples/figure_skating.mp4 \
  --output outputs/quickstart --query-frame 0 --stride 1 --max-frames 150 --viewer

Weights are downloaded and cached automatically. You can also download model.safetensors manually and pass --weights /path/to/model.safetensors, or load it with Pi4.from_pretrained("/path/to/model.safetensors").

See the inference guide for input preparation, output formats, memory requirements, and the viewer.

License

Pi4 model weights are released under CC BY-NC 4.0 for noncommercial use. Third-party materials retain their upstream licenses; see NOTICE and the bundled license texts.

Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
0.5B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Space using yyfz233/Pi4 1