---
tags:
- image-classification
- timm
library_name: timm
license: other
license_name: lunit-non-commercial
license_link: https://github.com/lunit-io/benchmark-ssl-pathology/blob/main/LICENSE
datasets:
- 1aurent/BACH
- 1aurent/NCT-CRC-HE
- 1aurent/PatchCamelyon
pipeline_tag: image-classification
---

# Model card for vit_small_patch16_224.lunit_dino

A Vision Transformer (ViT) image classification model. \
Trained on 33M histology patches from various pathology datasets.

![](https://github.com/lunit-io/benchmark-ssl-pathology/raw/main/assets/ssl_teaser.png)

## Model Details

- **Model Type:** Feature backbone
- **SSL Method:** DINO
- **Model Stats:**
  - Params (M): 21.1
  - Image sizes: 224 × 224 x 3
- **Papers:**
  - Benchmarking Self-Supervised Learning on Diverse Pathology Datasets: https://arxiv.org/abs/2212.04690
- **Datasets:**
  -  BACH
  -  CRC
  -  MHIST
  -  PatchCamelyon
  -  CoNSeP
- **Original:** https://github.com/lunit-io/benchmark-ssl-pathology
- **License:** [lunit-non-commercial](https://github.com/lunit-io/benchmark-ssl-pathology/blob/main/LICENSE)

## Model Usage

### Image Embeddings
```python
from urllib.request import urlopen
from PIL import Image
import timm

# get example histology image
img = Image.open(
  urlopen(
    "https://github.com/owkin/HistoSSLscaling/raw/main/assets/example.tif"
  )
)

# load model from the hub
model = timm.create_model(
  model_name="hf-hub:1aurent/vit_small_patch16_224.lunit_dino",
  pretrained=True,
).eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor
```

## Citation
```bibtex
@inproceedings{kang2022benchmarking,
  author    = {Kang, Mingu and Song, Heon and Park, Seonwook and Yoo, Donggeun and Pereira, Sérgio},
  title     = {Benchmarking Self-Supervised Learning on Diverse Pathology Datasets},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2023},
  pages     = {3344-3354}
}
```