Model card for act.dvae.runpei-dong

An ACT self-supervised pretraining model (masked point transformer distilled from a 2D image transformer). Pretrained on ShapeNet-55.

Model Details

• Model Type: Self-supervised pretraining
• Model Stats:
  • Params (M): 116.2
• Dataset: ShapeNet-55
• Paper: Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?
• Converted from: RunpeiDong/ACT (MIT)
• Library: torch-pointcloud

Install

pip install torch-pointcloud

Usage

import torch
import torch_pointcloud as tp
from torch_pointcloud.utils.data import collate

model, info = tp.create_model(
    "act.dvae.runpei-dong",
    pretrained=True,
    return_info=True,
)
model = model.eval()

# synthetic sample with the keys a dataset provides
num_points = 8192
sample = {
    "pos": torch.randn(num_points, 3),
}
data = collate([sample])

with torch.no_grad():
    out = model(data.get("x"), data["pos"], data["batch"])

Citation

@inproceedings{dong2023act,
  title     = {Autoencoders as Cross-Modal Teachers: Can Pretrained {2D} Image Transformers Help {3D} Representation Learning?},
  author    = {Dong, Runpei and Qi, Zekun and Zhang, Linfeng and Zhang, Junbo and Sun, Jianjian and Ge, Zheng and Yi, Li and Ma, Kaisheng},
  booktitle = {ICLR},
  year      = {2023}
}

@article{chang2015shapenet,
  author  = {Chang, Angel X. and Funkhouser, Thomas and Guibas, Leonidas and Hanrahan, Pat and Huang, Qixing and Li, Zimo and Savarese, Silvio and Savva, Manolis and Song, Shuran and Su, Hao and Xiao, Jianxiong and Yi, Li and Yu, Fisher},
  title   = {{ShapeNet}: An Information-Rich {3D} Model Repository},
  journal = {arXiv preprint arXiv:1512.03012},
  year    = {2015},
}

@software{dujardin2026pytorchpointcloud,
  author  = {Arthur Dujardin},
  title   = {PyTorch PointCloud},
  year    = {2026},
  doi     = {10.5281/zenodo.22159632},
  url     = {https://github.com/arthurdjn/pytorch-pointcloud},
}