timm
/

Image Classification
timm
PyTorch
rwightman HF staff commited on
Commit
2161963
1 Parent(s): 7826d9a
Files changed (3) hide show
  1. README.md +131 -0
  2. config.json +35 -0
  3. pytorch_model.bin +3 -0
README.md ADDED
@@ -0,0 +1,131 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - image-classification
4
+ - timm
5
+ library_tag: timm
6
+ license: apache-2.0
7
+ datasets:
8
+ - imagenet-1k
9
+ ---
10
+ # Model card for davit_base.msft_in1k
11
+
12
+ A DaViT image classification model. Trained on ImageNet-1k by paper authors.
13
+
14
+ Thanks to [Fredo Guan](https://github.com/fffffgggg54) for bringing the classification backbone to `timm`.
15
+
16
+
17
+ ## Model Details
18
+ - **Model Type:** Image classification / feature backbone
19
+ - **Model Stats:**
20
+ - Params (M): 88.0
21
+ - GMACs: 15.5
22
+ - Activations (M): 40.7
23
+ - Image size: 224 x 224
24
+ - **Papers:**
25
+ - DaViT: Dual Attention Vision Transformers: https://arxiv.org/abs/2204.03645
26
+ - **Original:** https://github.com/dingmyu/davit
27
+ - **Dataset:** ImageNet-1k
28
+
29
+ ## Model Usage
30
+ ### Image Classification
31
+ ```python
32
+ from urllib.request import urlopen
33
+ from PIL import Image
34
+ import timm
35
+
36
+ img = Image.open(
37
+ urlopen('https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'))
38
+
39
+ model = timm.create_model('davit_base.msft_in1k', pretrained=True)
40
+ model = model.eval()
41
+
42
+ # get model specific transforms (normalization, resize)
43
+ data_config = timm.data.resolve_model_data_config(model)
44
+ transforms = timm.data.create_transform(**data_config, is_training=False)
45
+
46
+ output = model(transforms(img).unsqueeze(0)) # unsqueeze single image into batch of 1
47
+
48
+ top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)
49
+ ```
50
+
51
+ ### Feature Map Extraction
52
+ ```python
53
+ from urllib.request import urlopen
54
+ from PIL import Image
55
+ import timm
56
+
57
+ img = Image.open(
58
+ urlopen('https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'))
59
+
60
+ model = timm.create_model(
61
+ 'davit_base.msft_in1k',
62
+ pretrained=True,
63
+ features_only=True,
64
+ )
65
+ model = model.eval()
66
+
67
+ # get model specific transforms (normalization, resize)
68
+ data_config = timm.data.resolve_model_data_config(model)
69
+ transforms = timm.data.create_transform(**data_config, is_training=False)
70
+
71
+ output = model(transforms(img).unsqueeze(0)) # unsqueeze single image into batch of 1
72
+
73
+ for o in output:
74
+ # print shape of each feature map in output
75
+ # e.g.:
76
+ # torch.Size([1, 96, 56, 56])
77
+ # torch.Size([1, 192, 28, 28])
78
+ # torch.Size([1, 384, 14, 14])
79
+ # torch.Size([1, 768, 7, 7]
80
+ print(o.shape)
81
+ ```
82
+
83
+ ### Image Embeddings
84
+ ```python
85
+ from urllib.request import urlopen
86
+ from PIL import Image
87
+ import timm
88
+
89
+ img = Image.open(
90
+ urlopen('https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'))
91
+
92
+ model = timm.create_model(
93
+ 'davit_base.msft_in1k',
94
+ pretrained=True,
95
+ num_classes=0, # remove classifier nn.Linear
96
+ )
97
+ model = model.eval()
98
+
99
+ # get model specific transforms (normalization, resize)
100
+ data_config = timm.data.resolve_model_data_config(model)
101
+ transforms = timm.data.create_transform(**data_config, is_training=False)
102
+
103
+ output = model(transforms(img).unsqueeze(0)) # output is (batch_size, num_features) shaped tensor
104
+
105
+ # or equivalently (without needing to set num_classes=0)
106
+
107
+ output = model.forward_features(transforms(img).unsqueeze(0))
108
+ # output is unpooled (ie.e a (batch_size, num_features, H, W) tensor
109
+
110
+ output = model.forward_head(output, pre_logits=True)
111
+ # output is (batch_size, num_features) tensor
112
+ ```
113
+
114
+ ## Model Comparison
115
+ ### By Top-1
116
+
117
+ |model |top1 |top1_err|top5 |top5_err|param_count|img_size|crop_pct|interpolation|
118
+ |---------------------|------|--------|------|--------|-----------|--------|--------|-------------|
119
+ |davit_base.msft_in1k |84.634|15.366 |97.014|2.986 |87.95 |224 |0.95 |bicubic |
120
+ |davit_small.msft_in1k|84.25 |15.75 |96.94 |3.06 |49.75 |224 |0.95 |bicubic |
121
+ |davit_tiny.msft_in1k |82.676|17.324 |96.276|3.724 |28.36 |224 |0.95 |bicubic |
122
+
123
+ ## Citation
124
+ ```bibtex
125
+ @inproceedings{ding2022davit,
126
+ title={DaViT: Dual Attention Vision Transformer},
127
+ author={Ding, Mingyu and Xiao, Bin and Codella, Noel and Luo, Ping and Wang, Jingdong and Yuan, Lu},
128
+ booktitle={ECCV},
129
+ year={2022},
130
+ }
131
+ ```
config.json ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architecture": "davit_base",
3
+ "num_classes": 1000,
4
+ "num_features": 1024,
5
+ "pretrained_cfg": {
6
+ "tag": "msft_in1k",
7
+ "custom_load": false,
8
+ "input_size": [
9
+ 3,
10
+ 224,
11
+ 224
12
+ ],
13
+ "fixed_input_size": false,
14
+ "interpolation": "bicubic",
15
+ "crop_pct": 0.9,
16
+ "crop_mode": "center",
17
+ "mean": [
18
+ 0.485,
19
+ 0.456,
20
+ 0.406
21
+ ],
22
+ "std": [
23
+ 0.229,
24
+ 0.224,
25
+ 0.225
26
+ ],
27
+ "num_classes": 1000,
28
+ "pool_size": [
29
+ 7,
30
+ 7
31
+ ],
32
+ "first_conv": "stem.conv",
33
+ "classifier": "head.fc"
34
+ }
35
+ }
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:064f0449df34109ec5f8ff31dddf9d9c31a3fe4b03129e0a5b229aea2b0659fe
3
+ size 351971805