timm
/

Image Classification
timm
PyTorch
Safetensors
rwightman HF staff commited on
Commit
6f23652
1 Parent(s): 000f118
Files changed (4) hide show
  1. README.md +166 -0
  2. config.json +33 -0
  3. model.safetensors +3 -0
  4. pytorch_model.bin +3 -0
README.md ADDED
@@ -0,0 +1,166 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - image-classification
4
+ - timm
5
+ library_name: timm
6
+ license: apache-2.0
7
+ datasets:
8
+ - imagenet-1k
9
+ ---
10
+ # Model card for vit_pwee_patch16_reg1_gap_256.sbb_in1k
11
+
12
+ A Vision Transformer (ViT) image classification model. This is a `timm` specific variation of the architecture with registers, global average pooling.
13
+
14
+ There are a number of models in the lower end of model scales that originate in `timm`:
15
+
16
+ | variant | width | mlp width (mult) | heads | depth | timm orig |
17
+ | ------- | ----- | ---------------- | ----- | ----- | ---- |
18
+ | tiny | 192 | 768 (4) | 3 | 12 | n |
19
+ | wee | 256 | 1280 (5) | 4 | 14 | y |
20
+ | pwee | 256 | 1280 (5) | 4 | 16 (parallel) | y |
21
+ | small | 384 | 1536 (4) | 6 | 12 | n |
22
+ | little | 320 | 1792 (5.6) | 5 | 14 | y |
23
+ | medium | 512 | 2048 (4) | 8 | 12 | y |
24
+ | mediumd | 512 | 2048 (4) | 8 | 20 | y |
25
+ | betwixt | 640 | 2560 (4) | 10 | 12 | y |
26
+ | base | 768 | 3072 (4) | 12 | 12 | n |
27
+
28
+ Trained on ImageNet-1k in `timm` using recipe template described below.
29
+
30
+ Recipe details:
31
+ * Searching for better baselines. Influced by Swin/DeiT/DeiT-III but w/ increased weight decay, moderate (in12k) to high (in1k) augmentation. Layer-decay used for fine-tune. Some runs used BCE and/or NAdamW instead of AdamW.
32
+ * See [train_hparams.yaml](./train_hparams.yaml) for specifics of each model.
33
+
34
+
35
+ ## Model Details
36
+ - **Model Type:** Image classification / feature backbone
37
+ - **Model Stats:**
38
+ - Params (M): 15.3
39
+ - GMACs: 3.8
40
+ - Activations (M): 10.6
41
+ - Image size: 256 x 256
42
+ - **Papers:**
43
+ - Vision Transformers Need Registers: https://arxiv.org/abs/2309.16588
44
+ - An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale: https://arxiv.org/abs/2010.11929v2
45
+ - **Dataset:** ImageNet-1k
46
+ - **Original:** https://github.com/huggingface/pytorch-image-models
47
+
48
+ ## Model Usage
49
+ ### Image Classification
50
+ ```python
51
+ from urllib.request import urlopen
52
+ from PIL import Image
53
+ import timm
54
+
55
+ img = Image.open(urlopen(
56
+ 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
57
+ ))
58
+
59
+ model = timm.create_model('vit_pwee_patch16_reg1_gap_256.sbb_in1k', pretrained=True)
60
+ model = model.eval()
61
+
62
+ # get model specific transforms (normalization, resize)
63
+ data_config = timm.data.resolve_model_data_config(model)
64
+ transforms = timm.data.create_transform(**data_config, is_training=False)
65
+
66
+ output = model(transforms(img).unsqueeze(0)) # unsqueeze single image into batch of 1
67
+
68
+ top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)
69
+ ```
70
+
71
+ ### Feature Map Extraction
72
+ ```python
73
+ from urllib.request import urlopen
74
+ from PIL import Image
75
+ import timm
76
+
77
+ img = Image.open(urlopen(
78
+ 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
79
+ ))
80
+
81
+ model = timm.create_model(
82
+ 'vit_pwee_patch16_reg1_gap_256.sbb_in1k',
83
+ pretrained=True,
84
+ features_only=True,
85
+ )
86
+ model = model.eval()
87
+
88
+ # get model specific transforms (normalization, resize)
89
+ data_config = timm.data.resolve_model_data_config(model)
90
+ transforms = timm.data.create_transform(**data_config, is_training=False)
91
+
92
+ output = model(transforms(img).unsqueeze(0)) # unsqueeze single image into batch of 1
93
+
94
+ for o in output:
95
+ # print shape of each feature map in output
96
+ # e.g.:
97
+ # torch.Size([1, 256, 16, 16])
98
+ # torch.Size([1, 256, 16, 16])
99
+ # torch.Size([1, 256, 16, 16])
100
+
101
+ print(o.shape)
102
+ ```
103
+
104
+ ### Image Embeddings
105
+ ```python
106
+ from urllib.request import urlopen
107
+ from PIL import Image
108
+ import timm
109
+
110
+ img = Image.open(urlopen(
111
+ 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
112
+ ))
113
+
114
+ model = timm.create_model(
115
+ 'vit_pwee_patch16_reg1_gap_256.sbb_in1k',
116
+ pretrained=True,
117
+ num_classes=0, # remove classifier nn.Linear
118
+ )
119
+ model = model.eval()
120
+
121
+ # get model specific transforms (normalization, resize)
122
+ data_config = timm.data.resolve_model_data_config(model)
123
+ transforms = timm.data.create_transform(**data_config, is_training=False)
124
+
125
+ output = model(transforms(img).unsqueeze(0)) # output is (batch_size, num_features) shaped tensor
126
+
127
+ # or equivalently (without needing to set num_classes=0)
128
+
129
+ output = model.forward_features(transforms(img).unsqueeze(0))
130
+ # output is unpooled, a (1, 257, 256) shaped tensor
131
+
132
+ output = model.forward_head(output, pre_logits=True)
133
+ # output is a (1, num_features) shaped tensor
134
+ ```
135
+
136
+ ## Model Comparison
137
+ Explore the dataset and runtime metrics of this model in timm [model results](https://github.com/huggingface/pytorch-image-models/tree/main/results).
138
+
139
+ ## Citation
140
+ ```bibtex
141
+ @misc{rw2019timm,
142
+ author = {Ross Wightman},
143
+ title = {PyTorch Image Models},
144
+ year = {2019},
145
+ publisher = {GitHub},
146
+ journal = {GitHub repository},
147
+ doi = {10.5281/zenodo.4414861},
148
+ howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
149
+ }
150
+ ```
151
+ ```bibtex
152
+ @article{darcet2023vision,
153
+ title={Vision Transformers Need Registers},
154
+ author={Darcet, Timoth{'e}e and Oquab, Maxime and Mairal, Julien and Bojanowski, Piotr},
155
+ journal={arXiv preprint arXiv:2309.16588},
156
+ year={2023}
157
+ }
158
+ ```
159
+ ```bibtex
160
+ @article{dosovitskiy2020vit,
161
+ title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
162
+ author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
163
+ journal={ICLR},
164
+ year={2021}
165
+ }
166
+ ```
config.json ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architecture": "vit_pwee_patch16_reg1_gap_256",
3
+ "num_classes": 1000,
4
+ "num_features": 256,
5
+ "global_pool": "avg",
6
+ "pretrained_cfg": {
7
+ "tag": "sbb_in1k",
8
+ "custom_load": false,
9
+ "input_size": [
10
+ 3,
11
+ 256,
12
+ 256
13
+ ],
14
+ "fixed_input_size": true,
15
+ "interpolation": "bicubic",
16
+ "crop_pct": 0.95,
17
+ "crop_mode": "center",
18
+ "mean": [
19
+ 0.5,
20
+ 0.5,
21
+ 0.5
22
+ ],
23
+ "std": [
24
+ 0.5,
25
+ 0.5,
26
+ 0.5
27
+ ],
28
+ "num_classes": 1000,
29
+ "pool_size": null,
30
+ "first_conv": "patch_embed.proj",
31
+ "classifier": "head"
32
+ }
33
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff8005cd38fe9933d3f046838a319b3835b0efa8e9ae047a1798bf4a53259291
3
+ size 61028064
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:038f5d4c6b906e17c4dd3858c79766f6112006dec4dacfd83a0fc090e9ca37e9
3
+ size 61068714