Edit model card

Model card for vit_base_patch16_224.owkin_pancancer_ft_kather2016

A Vision Transformer (ViT) image classification model.
Trained by Owkin on 40M pan-cancer histology tiles from TCGA.
Fine-tuned on Kather Texture 2016 dataset.

Model Details

Model Usage

Image Classification

from urllib.request import urlopen
from PIL import Image
import timm

# get example histology image
img = Image.open(
  urlopen(
    "https://datasets-server.huggingface.co/assets/1aurent/Kather-texture-2016/--/default/train/0/image/image.jpg"
  )
)

# load model from the hub
model = timm.create_model(
  model_name="hf-hub:1aurent/vit_base_patch16_224.owkin_pancancer_ft_kather2016",
  pretrained=True,
).eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # unsqueeze single image into batch of 1

Image Embeddings

from urllib.request import urlopen
from PIL import Image
import timm

# get example histology image
img = Image.open(
  urlopen(
    "https://datasets-server.huggingface.co/assets/1aurent/Kather-texture-2016/--/default/train/0/image/image.jpg"
  )
)

# load model from the hub
model = timm.create_model(
  model_name="hf-hub:1aurent/vit_base_patch16_224.owkin_pancancer_ft_kather2016",
  pretrained=True,
  num_classes=0,
).eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor

Citation

@article {Filiot2023.07.21.23292757,
  author = {Alexandre Filiot and Ridouane Ghermi and Antoine Olivier and Paul Jacob and Lucas Fidon and Alice Mac Kain and Charlie Saillard and Jean-Baptiste Schiratti},
  title = {Scaling Self-Supervised Learning for Histopathology with Masked Image Modeling},
  elocation-id = {2023.07.21.23292757},
  year = {2023},
  doi = {10.1101/2023.07.21.23292757},
  publisher = {Cold Spring Harbor Laboratory Press},
  URL = {https://www.medrxiv.org/content/early/2023/09/14/2023.07.21.23292757},
  eprint = {https://www.medrxiv.org/content/early/2023/09/14/2023.07.21.23292757.full.pdf},
  journal = {medRxiv}
}
Downloads last month
1
Safetensors
Model size
85.8M params
Tensor type
F32
·
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Finetuned from

Dataset used to train 1aurent/vit_base_patch16_224.owkin_pancancer_ft_kather2016

Evaluation results