File size: 3,968 Bytes
d86df45 210b2c4 d86df45 210b2c4 d86df45 210b2c4 fb802e3 210b2c4 fb802e3 210b2c4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 |
---
tags:
- timm
- feature-extraction
- image-classification
library_name: timm
license: other
license_name: kaiko-non-commercial
license_link: https://github.com/kaiko-ai/towards_large_pathology_fms/blob/a62a0c54719d858371aefa0fcab6ec4b34c86c4c/LICENSE
metrics:
- accuracy
model-index:
- name: kaiko
results:
- task:
type: image-classification
name: Image Classification
dataset:
name: BACH
type: image-classification
metrics:
- type: accuracy
value: 0.870
name: Accuracy
verified: false
- task:
type: image-classification
name: Image Classification
dataset:
name: CRC-NCT-HE
type: image-classification
metrics:
- type: accuracy
value: 0.930
name: Accuracy
verified: false
- task:
type: image-classification
name: Image Classification
dataset:
name: MHIST
type: image-classification
metrics:
- type: accuracy
value: 0.809
name: Accuracy
verified: false
- task:
type: image-classification
name: Image Classification
dataset:
name: PCam
type: image-classification
metrics:
- type: accuracy
value: 0.898
name: Accuracy
verified: false
- task:
type: image-classification
name: Image Classification
dataset:
name: TP53
type: image-classification
metrics:
- type: accuracy
value: 0.656
name: Accuracy
verified: false
- task:
type: image-classification
name: Image Classification
dataset:
name: CoNSeP
type: image-classification
metrics:
- type: accuracy
value: 0.679
name: Accuracy
verified: false
---
# Model card for vit_large_patch14_reg4_224.kaiko_ai_towards_large_pathology_fms
## Model Details
- **Model Type:** Feature backbone
- **Model Stats:**
- Params: 304M (large)
- Image size: 224 x 224 x 3
- Patch size: 14 x 14 x 3
- **Repository:** [github.com:kaiko-ai/towards_large_pathology_fms](https://github.com/kaiko-ai/towards_large_pathology_fms)
- **Original Weights:** [github.com:kaiko-ai/towards_large_pathology_fms/0.0.1](https://github.com/kaiko-ai/towards_large_pathology_fms/releases/tag/0.0.1)
- **Papers:**
- [Towards Large-Scale Training of Pathology Foundation Models](https://arxiv.org/abs/2404.15217)
## Model Usage
### Image Embeddings
```python
from torchvision.transforms import v2
from PIL import Image
import requests
import torch
import timm
import io
# get example histology image
url = "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQc7_xZpGOfQT7sxKwf2w5lL4GAq6IX_CbTzP1NGeenzA&s"
image = Image.open(io.BytesIO(requests.get(url).content))
# load model from the hub
model = timm.create_model(
model_name="hf-hub:1aurent/vit_large_patch14_reg4_224.kaiko_ai_towards_large_pathology_fms",
dynamic_img_size=True,
pretrained=True,
).eval()
# get image transform
preprocessing = v2.Compose(
[
v2.ToImage(),
v2.Resize(size=224),
v2.CenterCrop(size=224),
v2.ToDtype(torch.float32, scale=True),
v2.Normalize(
mean=(0.5, 0.5, 0.5),
std=(0.5, 0.5, 0.5),
),
]
)
data = preprocessing(image).unsqueeze(0) # input is a (batch_size, num_channels, img_size, img_size) shaped tensor
output = model(data) # output is a (batch_size, num_features) shaped tensor
```
## Citation
```bibtex
@misc{ai2024largescale,
title = {Towards Large-Scale Training of Pathology Foundation Models},
author = {kaiko.ai and Nanne Aben and Edwin D. de Jong and Ioannis Gatopoulos and Nicolas Känzig and Mikhail Karasikov and Axel Lagré and Roman Moser and Joost van Doorn and Fei Tang},
year = {2024},
eprint = {2404.15217},
archivePrefix = {arXiv},
primaryClass = {cs.CV}
}
```
|