fastforks/dinov2-linear

DINOv2 linear (fastforks)

DINOv2 (Oquab et al. 2023) linear probe: the frozen self-supervised ViT-S/B/L/14, a batch norm and a 1x1 classifier on the last block's patch tokens; the authors' ADE20K linear heads, sliding-window inference.

This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/facebookresearch/dinov2 at 7764ea0f912e.

Task: segmentation. Tier: stable. Outputs: logits. Backends: portable, kernels.

Input: one RGB image. Output: per-pixel class logits.

Variants

Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.

variant	weights (author link)	SHA-256	weights licence	commercial use
dinov2-b-linear-ade20k@1	dinov2-b-linear-ade20k.backbone.pth dinov2-b-linear-ade20k.head.pth	0b8b82f85de9 7c3545c3a6f7	Apache-2.0	yes
dinov2-l-linear-ade20k@1	dinov2-l-linear-ade20k.backbone.pth dinov2-l-linear-ade20k.head.pth	d5383ea8f487 59137f319d7e	Apache-2.0	yes
dinov2-s-linear-ade20k@1	dinov2-s-linear-ade20k.backbone.pth dinov2-s-linear-ade20k.head.pth	b938bf1bc15c 67e10225e0bf	Apache-2.0	yes

Licence

Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Files: .

Training data: LVD-142M (DINOv2's curated web-image set, not released), ADE20K (SceneParsing, 150 classes).

Licence evidence: https://github.com/facebookresearch/dinov2/blob/7764ea0f912e53c92e82eb78a2a1631e92725fc8/README.md#L660

Input size

any H x W >= 1, resized or padded inside the model Tested up to (portable, fp16, RTX 5060 Ti 16 GB): dinov2-b-linear-ade20k 3000x4000 (7.3 GiB peak allocated, 8.1 reserved); dinov2-l-linear-ade20k 3000x4000 (7.7 GiB peak allocated, 8.5 reserved); dinov2-s-linear-ade20k 3000x4000 (7.2 GiB peak allocated, 7.9 reserved)

Speed and memory

Segmentation, synthetic, 1024x2048

model	time (ms): original fp32	original	portable	kernels	kernels speed-up: vs original	vs portable	peak memory (MiB, allocated / reserved): original	portable	kernels
dinov2-b-linear-ade20k	115.19	40.15	33.285	22.63	1.8x	1.5x	2063 / 2426	765 / 884	771 / 904
dinov2-l-linear-ade20k	332.04	95.67	83.939	56.78	1.7x	1.5x	3308 / 3726	1181 / 1244	1189 / 1244
dinov2-s-linear-ade20k	55.83	24.66	18.336	12.44	2.0x	1.5x	1694 / 2046	642 / 696	645 / 676

Startup and first call

model	load (s): original	portable	kernels	first call (s): original	portable	kernels
dinov2-b-linear-ade20k	12.01 (512x512)	0.27	0.40†	0.02 (512x512)	0.10	0.07†
dinov2-l-linear-ade20k	7.86 (512x512)	0.81	0.92†	0.38 (512x512)	0.16	0.10†
dinov2-s-linear-ade20k	4.02 (512x512)	0.12	0.23†	0.30 (512x512)	0.09	0.06†

torch.compile and TensorRT

model	input	portable ms	compile ms	compile s (cold)	TensorRT fp16 ms	ONNX + build s	kernels ms	peak MiB (alloc / reserved): portable	compile	TensorRT (device)
dinov2-s-linear-ade20k	1024x2048	18.336	18.3	0.2	40.3	36	12.44	642 / 696	639 / 676	1794

Fixed-size speed and peak memory, all models

model	mode	640x480	1280x720	1920x1080	12 MP 4000x3000
dinov2-b-linear-ade20k	portable	22.5 ms / 922 / 1040	35.5 ms / 1217 / 1398	40.0 ms / 1230 / 1416	151.7 ms / 1289 / 1960
dinov2-b-linear-ade20k	kernels	18.3 ms / 929 / 1080	28.4 ms / 1224 / 1438	34.1 ms / 1237 / 1456	147.9 ms / 1296 / 2000
dinov2-l-linear-ade20k	portable	56.5 ms / 1330 / 1524	86.3 ms / 1624 / 1876	91.1 ms / 1638 / 1890	184.9 ms / 1697 / 2266
dinov2-l-linear-ade20k	kernels	39.6 ms / 1338 / 1526	62.8 ms / 1632 / 1884	67.7 ms / 1645 / 1894	168.5 ms / 1704 / 2264
dinov2-s-linear-ade20k	portable	12.6 ms / 792 / 834	20.1 ms / 1088 / 1154	25.1 ms / 1101 / 1168	141.7 ms / 1160 / 1522
dinov2-s-linear-ade20k	kernels	11.5 ms / 796 / 854	18.5 ms / 1092 / 1168	23.5 ms / 1105 / 1174	140.4 ms / 1163 / 1774

Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).

Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the authors' code at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.

Accuracy

Parity with the original code

Every variant is checked against golden outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.05 and p99 <= 0.25 (absolute error of the output logits).

Usage

from fastforks import segmentation

model = segmentation.load("dinov2-b-linear-ade20k")  # mode="auto": the preferred backend from recorded measurements
result = model("photo.jpg")
result.labels  # [H, W] per-pixel class index
result.backend  # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')

Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: dinov2-b-linear-ade20k, dinov2-l-linear-ade20k, dinov2-s-linear-ade20k.

Kernels backend

segmentation.load("dinov2-b-linear-ade20k", mode="kernels") runs our CUDA kernels (fastbb, seg_up). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.