timm/vit_large_patch16_sapiens2.fb

NOTE: This is a native timm (EVA) remap of facebook/sapiens2-pretrain-0.4b. Checkpoint keys have been converted to timm naming; the weights have not been fine-tuned. The original Sapiens2 License applies. The upstream model card is reproduced below with timm usage instructions.

Sapiens2-0.4B

Sapiens2 is a family of high-resolution vision transformers pretrained on 1 billion human images — designed for human-centric tasks such as pose estimation, body-part segmentation, surface normals, and pointmaps.

This repository contains the 0.4B parameter pretrained backbone. It produces dense per-patch features suitable for fine-tuning downstream task heads.

• 📄 Paper: arXiv:2604.21681
• 🌐 Project Page: rawalkhirodkar.github.io/sapiens2
• 💻 Code: github.com/facebookresearch/sapiens2

Model Details

• Developed by: Meta
• Model type: Vision Transformer
• License: Sapiens2 License
• Task: pretrain
• Format: safetensors
• File: model.safetensors

Quick Start

Use a timm version that includes Sapiens2 support.

import torch
import timm
from PIL import Image

device = "cuda" if torch.cuda.is_available() else "cpu"
model = timm.create_model(
    "hf-hub:timm/vit_large_patch16_sapiens2.fb", pretrained=True, use_naflex=False,
).eval().to(device)

data_config = timm.data.resolve_model_data_config(model)
transform = timm.data.create_transform(**data_config, is_training=False)
image = Image.open("image.jpg").convert("RGB")
x = transform(image).unsqueeze(0).to(device)

with torch.inference_mode():
    tokens = model.forward_features(x)
    cls_features = tokens[:, 0]
    patch_features = tokens[:, model.num_prefix_tokens:]  # exclude CLS and register tokens

model(x) uses CLS-token pooling by default, matching the original Sapiens2 convention. Pass global_pool="avg" to create_model for average pooling over patch tokens.

Model Card

Field	Value
Architecture	Sapiens2 ViT (RoPE, GQA, SwiGLU, RMSNorm, QK-norm)
Parameters	0.398 B
FLOPs	1.260 T
Embedding dim	1024
Layers	24
Attention heads	16
Pretraining resolution	1024 × 768 (H × W)
Patch size	16
Pretraining data	1B human images

Sapiens2 Family

Model	Params	FLOPs	Embed dim	Layers	Heads
Sapiens2-0.1B	0.114 B	0.342 T	768	12	12
Sapiens2-0.4B (this)	0.398 B	1.260 T	1024	24	16
Sapiens2-0.8B	0.818 B	2.592 T	1280	32	16
Sapiens2-1B	1.462 B	4.715 T	1536	40	24
Sapiens2-1B-4K	1.607 B	—	1536	40	24
Sapiens2-5B	5.071 B	15.722 T	2432	56	32

See the Sapiens2 Collection for all variants and downstream task checkpoints (pose, segmentation, normals, pointmaps).

Intended Use

• Feature extraction for human-centric downstream tasks
• Initialization for fine-tuning task heads (pose, segmentation, normals, pointmap)
• Research on human-centric vision

License

Released under the Sapiens2 License.

Citation

@article{khirodkarsapiens2,
  title={Sapiens2},
  author={Khirodkar, Rawal and Wen, He and Martinez, Julieta and Dong, Yuan and Su, Zhaoen and Saito, Shunsuke},
  journal={arXiv preprint arXiv:2604.21681},
  year={2026}
}