Update model config and README
Browse files- README.md +126 -1
- config.json +2 -1
- model.safetensors +3 -0
README.md
CHANGED
@@ -3,5 +3,130 @@ tags:
|
|
3 |
- image-classification
|
4 |
- timm
|
5 |
library_tag: timm
|
|
|
|
|
|
|
|
|
|
|
6 |
---
|
7 |
-
# Model card for eva_large_patch14_196.in22k_ft_in22k_in1k
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
- image-classification
|
4 |
- timm
|
5 |
library_tag: timm
|
6 |
+
license: mit
|
7 |
+
datasets:
|
8 |
+
- imagenet-1k
|
9 |
+
- imagenet-22k
|
10 |
+
- imagenet-22k
|
11 |
---
|
12 |
+
# Model card for eva_large_patch14_196.in22k_ft_in22k_in1k
|
13 |
+
|
14 |
+
An EVA image classification model. Pretrained on ImageNet-22k with masked image modeling (using EVA-CLIP as a MIM teacher) and fine-tuned on ImageNet-22k then on ImageNet-1k by paper authors.
|
15 |
+
|
16 |
+
NOTE: `timm` checkpoints are float32 for consistency with other models. Original checkpoints are float16 or bfloat16 in some cases, see originals if that's preferred.
|
17 |
+
|
18 |
+
|
19 |
+
## Model Details
|
20 |
+
- **Model Type:** Image classification / feature backbone
|
21 |
+
- **Model Stats:**
|
22 |
+
- Params (M): 304.1
|
23 |
+
- GMACs: 61.6
|
24 |
+
- Activations (M): 63.5
|
25 |
+
- Image size: 196 x 196
|
26 |
+
- **Papers:**
|
27 |
+
- EVA: Exploring the Limits of Masked Visual Representation Learning at Scale: https://arxiv.org/abs/2211.07636
|
28 |
+
- **Pretrain Dataset:**
|
29 |
+
- ImageNet-22k
|
30 |
+
- ImageNet-22k
|
31 |
+
- **Dataset:** ImageNet-1k
|
32 |
+
- **Original:**
|
33 |
+
- https://github.com/baaivision/EVA
|
34 |
+
- https://huggingface.co/BAAI/EVA
|
35 |
+
|
36 |
+
## Model Usage
|
37 |
+
### Image Classification
|
38 |
+
```python
|
39 |
+
from urllib.request import urlopen
|
40 |
+
from PIL import Image
|
41 |
+
import timm
|
42 |
+
|
43 |
+
img = Image.open(urlopen(
|
44 |
+
'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
|
45 |
+
))
|
46 |
+
|
47 |
+
model = timm.create_model('eva_large_patch14_196.in22k_ft_in22k_in1k', pretrained=True)
|
48 |
+
model = model.eval()
|
49 |
+
|
50 |
+
# get model specific transforms (normalization, resize)
|
51 |
+
data_config = timm.data.resolve_model_data_config(model)
|
52 |
+
transforms = timm.data.create_transform(**data_config, is_training=False)
|
53 |
+
|
54 |
+
output = model(transforms(img).unsqueeze(0)) # unsqueeze single image into batch of 1
|
55 |
+
|
56 |
+
top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)
|
57 |
+
```
|
58 |
+
|
59 |
+
### Image Embeddings
|
60 |
+
```python
|
61 |
+
from urllib.request import urlopen
|
62 |
+
from PIL import Image
|
63 |
+
import timm
|
64 |
+
|
65 |
+
img = Image.open(urlopen(
|
66 |
+
'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
|
67 |
+
))
|
68 |
+
|
69 |
+
model = timm.create_model(
|
70 |
+
'eva_large_patch14_196.in22k_ft_in22k_in1k',
|
71 |
+
pretrained=True,
|
72 |
+
num_classes=0, # remove classifier nn.Linear
|
73 |
+
)
|
74 |
+
model = model.eval()
|
75 |
+
|
76 |
+
# get model specific transforms (normalization, resize)
|
77 |
+
data_config = timm.data.resolve_model_data_config(model)
|
78 |
+
transforms = timm.data.create_transform(**data_config, is_training=False)
|
79 |
+
|
80 |
+
output = model(transforms(img).unsqueeze(0)) # output is (batch_size, num_features) shaped tensor
|
81 |
+
|
82 |
+
# or equivalently (without needing to set num_classes=0)
|
83 |
+
|
84 |
+
output = model.forward_features(transforms(img).unsqueeze(0))
|
85 |
+
# output is unpooled, a (1, 197, 1024) shaped tensor
|
86 |
+
|
87 |
+
output = model.forward_head(output, pre_logits=True)
|
88 |
+
# output is a (1, num_features) shaped tensor
|
89 |
+
```
|
90 |
+
|
91 |
+
## Model Comparison
|
92 |
+
Explore the dataset and runtime metrics of this model in timm [model results](https://github.com/huggingface/pytorch-image-models/tree/main/results).
|
93 |
+
|
94 |
+
|model |top1 |top5 |param_count|img_size|
|
95 |
+
|-----------------------------------------------|------|------|-----------|--------|
|
96 |
+
|eva02_large_patch14_448.mim_m38m_ft_in22k_in1k |90.054|99.042|305.08 |448 |
|
97 |
+
|eva02_large_patch14_448.mim_in22k_ft_in22k_in1k|89.946|99.01 |305.08 |448 |
|
98 |
+
|eva_giant_patch14_560.m30m_ft_in22k_in1k |89.792|98.992|1014.45 |560 |
|
99 |
+
|eva02_large_patch14_448.mim_in22k_ft_in1k |89.626|98.954|305.08 |448 |
|
100 |
+
|eva02_large_patch14_448.mim_m38m_ft_in1k |89.57 |98.918|305.08 |448 |
|
101 |
+
|eva_giant_patch14_336.m30m_ft_in22k_in1k |89.56 |98.956|1013.01 |336 |
|
102 |
+
|eva_giant_patch14_336.clip_ft_in1k |89.466|98.82 |1013.01 |336 |
|
103 |
+
|eva_large_patch14_336.in22k_ft_in22k_in1k |89.214|98.854|304.53 |336 |
|
104 |
+
|eva_giant_patch14_224.clip_ft_in1k |88.882|98.678|1012.56 |224 |
|
105 |
+
|eva02_base_patch14_448.mim_in22k_ft_in22k_in1k |88.692|98.722|87.12 |448 |
|
106 |
+
|eva_large_patch14_336.in22k_ft_in1k |88.652|98.722|304.53 |336 |
|
107 |
+
|eva_large_patch14_196.in22k_ft_in22k_in1k |88.592|98.656|304.14 |196 |
|
108 |
+
|eva02_base_patch14_448.mim_in22k_ft_in1k |88.23 |98.564|87.12 |448 |
|
109 |
+
|eva_large_patch14_196.in22k_ft_in1k |87.934|98.504|304.14 |196 |
|
110 |
+
|eva02_small_patch14_336.mim_in22k_ft_in1k |85.74 |97.614|22.13 |336 |
|
111 |
+
|eva02_tiny_patch14_336.mim_in22k_ft_in1k |80.658|95.524|5.76 |336 |
|
112 |
+
|
113 |
+
## Citation
|
114 |
+
```bibtex
|
115 |
+
@article{EVA,
|
116 |
+
title={EVA: Exploring the Limits of Masked Visual Representation Learning at Scale},
|
117 |
+
author={Fang, Yuxin and Wang, Wen and Xie, Binhui and Sun, Quan and Wu, Ledell and Wang, Xinggang and Huang, Tiejun and Wang, Xinlong and Cao, Yue},
|
118 |
+
journal={arXiv preprint arXiv:2211.07636},
|
119 |
+
year={2022}
|
120 |
+
}
|
121 |
+
```
|
122 |
+
```bibtex
|
123 |
+
@misc{rw2019timm,
|
124 |
+
author = {Ross Wightman},
|
125 |
+
title = {PyTorch Image Models},
|
126 |
+
year = {2019},
|
127 |
+
publisher = {GitHub},
|
128 |
+
journal = {GitHub repository},
|
129 |
+
doi = {10.5281/zenodo.4414861},
|
130 |
+
howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
|
131 |
+
}
|
132 |
+
```
|
config.json
CHANGED
@@ -28,6 +28,7 @@
|
|
28 |
"num_classes": 1000,
|
29 |
"pool_size": null,
|
30 |
"first_conv": "patch_embed.proj",
|
31 |
-
"classifier": "head"
|
|
|
32 |
}
|
33 |
}
|
|
|
28 |
"num_classes": 1000,
|
29 |
"pool_size": null,
|
30 |
"first_conv": "patch_embed.proj",
|
31 |
+
"classifier": "head",
|
32 |
+
"license": "mit"
|
33 |
}
|
34 |
}
|
model.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:344971ca59cf0c12194061924f848b66c2e932a13c5dd6fc07d1c9779b781fc3
|
3 |
+
size 1216597406
|