aibyou0830/supcon-multiview-for-fur

SupCon Multi-View — Fursuit Recognition Projection Head

Supervised Contrastive Learning (SupCon) projection head for fursuit identity embedding. This checkpoint is the recognition stage of the Immich Machine Learning — Furry Recognition pipeline: detected fursuit head crops are encoded with a frozen DINOv3 ViT-S/16 backbone, then mapped to a 512-dimensional embedding space by this projection head for cosine-distance clustering in Immich.

Built with DINOv3 — This checkpoint is a derivative work trained on features extracted from DINOv3. Redistribution requires compliance with the DINOv3 License.

Model Description

Property	Value
Component	SupCon projection head (MLP) only
Backbone (runtime, not included)	facebook/dinov3-vits16-pretrain-lvd1689m
Input features	DINOv3 pooler output (384-d)
Output embedding	512-d, L2-normalized
Head architecture	384 → 1024 → Tanh → Dropout(0.3) → 1024 → Tanh → Dropout(0.3) → 512
Crop preprocessing	Resize 512×512, ImageNet normalization
Minimum crop size	64 px (shorter side)
Checkpoint file	supcon_projection_head_best.pth
Checkpoint size	~7.5 MB

This repository contains only the fine-tuned projection head. The DINOv3 backbone must be downloaded separately from Hugging Face (gated model — accept the DINOv3 License and provide a valid HF_TOKEN).

The head was trained with a multi-view SupCon objective on fursuit head crops from a private, author-collected dataset. It is not a human face recognizer.

Intended Use

• Primary use: Produce identity embeddings for detected fursuit heads so photos can be clustered by character / wearer in Immich.
• Pair with: An object detector for fursuit heads (e.g. the companion RF-DETR fursuit detector).
• Out of scope: Human facial recognition, biometric identification, law enforcement, or any safety-critical identity verification.

Recommended Inference Settings

These defaults match the Immich ML Furry Recognition service:

Setting	Value	Notes
Embedding normalization	L2 normalize	Applied after projection
Immich max recognition distance	0.15	Cosine distance; lower = stricter clustering
Immich min detection score	0.5	From the upstream RF-DETR detector
Immich min recognized faces	3	Minimum detections before Immich creates a person

Tune recognition distance based on your photo quality and desired cluster granularity.

Training Details

Aspect	Detail
Objective	Supervised Contrastive Learning (SupCon)
Training views	Multi-view augmentations of fursuit head crops
Feature extractor	Frozen DINOv3 ViT-S/16 (dinov3-vits16-pretrain-lvd1689m)
Trainable component	Projection head only
Dropout	0.3 (between linear layers)
Activation	Tanh

Training Data

The projection head was trained exclusively on images captured and preprocessed by the project author. No public third-party datasets were used. The dataset is not publicly released.

Detailed training hyperparameters (learning rate, epochs, batch size, etc.) are not bundled with this checkpoint.

Usage

Requirements

Accept the DINOv3 license on Hugging Face, then install dependencies:

pip install torch torchvision transformers pillow huggingface-hub
export HF_TOKEN=hf_...   # token with DINOv3 access

End-to-end embedding (backbone + projection head)

import os
from pathlib import Path

import torch
import torch.nn as nn
import torch.nn.functional as F
from huggingface_hub import login
from PIL import Image
from torchvision import transforms
from transformers import DINOv3ViTModel

login(os.environ["HF_TOKEN"])

BACKBONE = "facebook/dinov3-vits16-pretrain-lvd1689m"
HEAD_CKPT = "supcon_projection_head_best.pth"

eval_transform = transforms.Compose([
    transforms.Resize((512, 512)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

device = "cuda" if torch.cuda.is_available() else "cpu"

backbone = DINOv3ViTModel.from_pretrained(BACKBONE).eval().to(device)

state_dict = torch.load(HEAD_CKPT, map_location=device, weights_only=True)
projector = nn.Sequential(
    nn.Linear(384, 1024), nn.Tanh(), nn.Dropout(0.3),
    nn.Linear(1024, 1024), nn.Tanh(), nn.Dropout(0.3),
    nn.Linear(1024, 512),
)
remapped = {k.replace("head.", ""): v for k, v in state_dict.items()}
projector.load_state_dict(remapped)
projector.eval().to(device)

def embed(crop: Image.Image) -> torch.Tensor:
    x = eval_transform(crop.convert("RGB")).unsqueeze(0).to(device)
    with torch.inference_mode():
        outputs = backbone(pixel_values=x)
        features = outputs.pooler_output
        if features is None:
            features = outputs.last_hidden_state[:, 0, :]
        vector = projector(features)
        return F.normalize(vector, p=2, dim=-1).squeeze(0).cpu()

embedding = embed(Image.open("fursuit_crop.jpg"))
print(embedding.shape)  # torch.Size([512])

Download from Hugging Face Hub

hf download hf://aibyou0830/supcon-multiview-for-fur/supcon_projection_head_best.pth

Immich ML integration

Place the checkpoint at:

immich_ml/supcon_multiview/supcon_projection_head_best.pth

Set HF_TOKEN in .env so the service can download DINOv3 at runtime. In Immich, select facial recognition model buffalo_l (API-compatible routing name). See the project README for full setup.

Limitations

• Requires the gated DINOv3 backbone at inference time; without a valid Hugging Face token and accepted license, the pipeline cannot run.
• Trained on a private fursuit dataset; embedding quality may degrade on out-of-distribution subjects (partial suits, mascots, plushies, artwork, extreme poses, heavy occlusion).
• Not interchangeable with InsightFace or other human face embedding models.
• Clustering quality depends on both detection accuracy and the Immich recognition distance threshold — this head alone does not perform clustering.
• Dropout layers are present in the saved architecture; at eval() mode they are disabled during inference.

License

This checkpoint is a derivative work of DINOv3, governed by the DINOv3 License.

When redistributing this checkpoint, you must:

1. Provide a copy of the DINOv3 License.
2. Prominently display "Built with DINOv3" in related product documentation.

The surrounding Immich ML Furry Recognition service code is licensed under AGPL-3.0 when distributed as part of that project.

Citation

If you use SupCon, please cite:

@inproceedings{khosla2020supcon,
  title={Supervised Contrastive Learning},
  author={Khosla, Prannay and Teterwak, Piotr and Wang, Chen and others},
  booktitle={NeurIPS},
  year={2020}
}

If you use DINOv3, follow the citation guidance on the model card.

Acknowledgments

• Meta DINOv3 — frozen feature backbone
• Immich — ML service framework
• FurPhotos by 久代さん — inspiration for the fursuit recognition pipeline