timm
/

Image Classification
timm
PyTorch
Safetensors
rwightman HF staff commited on
Commit
4f83cf4
1 Parent(s): 67a43af
Files changed (4) hide show
  1. README.md +145 -0
  2. config.json +35 -0
  3. model.safetensors +3 -0
  4. pytorch_model.bin +3 -0
README.md ADDED
@@ -0,0 +1,145 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - image-classification
4
+ - timm
5
+ library_name: timm
6
+ license: apache-2.0
7
+ datasets:
8
+ - imagenet-1k
9
+ - unknown-6m
10
+ ---
11
+ # Model card for nextvit_base.bd_ssld_6m_in1k
12
+
13
+ A Next-ViT image classification model. Trained by paper authors on an unknown 6M sample dataset and ImageNet-1k using SSLD distillation.
14
+
15
+
16
+
17
+ ## Model Details
18
+ - **Model Type:** Image classification / feature backbone
19
+ - **Model Stats:**
20
+ - Params (M): 44.8
21
+ - GMACs: 8.2
22
+ - Activations (M): 22.5
23
+ - Image size: 224 x 224
24
+ - **Pretrain Dataset:** Unknown-6M
25
+ - **Dataset:** ImageNet-1k
26
+ - **Papers:**
27
+ - Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios: https://arxiv.org/abs/2207.05501
28
+ - **Original:** https://github.com/bytedance/Next-ViT
29
+
30
+ ## Model Usage
31
+ ### Image Classification
32
+ ```python
33
+ from urllib.request import urlopen
34
+ from PIL import Image
35
+ import timm
36
+
37
+ img = Image.open(urlopen(
38
+ 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
39
+ ))
40
+
41
+ model = timm.create_model('nextvit_base.bd_ssld_6m_in1k', pretrained=True)
42
+ model = model.eval()
43
+
44
+ # get model specific transforms (normalization, resize)
45
+ data_config = timm.data.resolve_model_data_config(model)
46
+ transforms = timm.data.create_transform(**data_config, is_training=False)
47
+
48
+ output = model(transforms(img).unsqueeze(0)) # unsqueeze single image into batch of 1
49
+
50
+ top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)
51
+ ```
52
+
53
+ ### Feature Map Extraction
54
+ ```python
55
+ from urllib.request import urlopen
56
+ from PIL import Image
57
+ import timm
58
+
59
+ img = Image.open(urlopen(
60
+ 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
61
+ ))
62
+
63
+ model = timm.create_model(
64
+ 'nextvit_base.bd_ssld_6m_in1k',
65
+ pretrained=True,
66
+ features_only=True,
67
+ )
68
+ model = model.eval()
69
+
70
+ # get model specific transforms (normalization, resize)
71
+ data_config = timm.data.resolve_model_data_config(model)
72
+ transforms = timm.data.create_transform(**data_config, is_training=False)
73
+
74
+ output = model(transforms(img).unsqueeze(0)) # unsqueeze single image into batch of 1
75
+
76
+ for o in output:
77
+ # print shape of each feature map in output
78
+ # e.g.:
79
+ # torch.Size([1, 96, 56, 56])
80
+ # torch.Size([1, 256, 28, 28])
81
+ # torch.Size([1, 512, 14, 14])
82
+ # torch.Size([1, 1024, 7, 7])
83
+
84
+ print(o.shape)
85
+ ```
86
+
87
+ ### Image Embeddings
88
+ ```python
89
+ from urllib.request import urlopen
90
+ from PIL import Image
91
+ import timm
92
+
93
+ img = Image.open(urlopen(
94
+ 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
95
+ ))
96
+
97
+ model = timm.create_model(
98
+ 'nextvit_base.bd_ssld_6m_in1k',
99
+ pretrained=True,
100
+ num_classes=0, # remove classifier nn.Linear
101
+ )
102
+ model = model.eval()
103
+
104
+ # get model specific transforms (normalization, resize)
105
+ data_config = timm.data.resolve_model_data_config(model)
106
+ transforms = timm.data.create_transform(**data_config, is_training=False)
107
+
108
+ output = model(transforms(img).unsqueeze(0)) # output is (batch_size, num_features) shaped tensor
109
+
110
+ # or equivalently (without needing to set num_classes=0)
111
+
112
+ output = model.forward_features(transforms(img).unsqueeze(0))
113
+ # output is unpooled, a (1, 1024, 7, 7) shaped tensor
114
+
115
+ output = model.forward_head(output, pre_logits=True)
116
+ # output is a (1, num_features) shaped tensor
117
+ ```
118
+
119
+ ## Model Comparison
120
+ ### By Top-1
121
+
122
+ |model |top1 |top1_err|top5 |top5_err|param_count|
123
+ |---------------------------------|------|--------|------|--------|-----------|
124
+ |nextvit_large.bd_ssld_6m_in1k_384|86.542|13.458 |98.142|1.858 |57.87 |
125
+ |nextvit_base.bd_ssld_6m_in1k_384 |86.352|13.648 |98.04 |1.96 |44.82 |
126
+ |nextvit_small.bd_ssld_6m_in1k_384|85.964|14.036 |97.908|2.092 |31.76 |
127
+ |nextvit_large.bd_ssld_6m_in1k |85.48 |14.52 |97.696|2.304 |57.87 |
128
+ |nextvit_base.bd_ssld_6m_in1k |85.186|14.814 |97.59 |2.41 |44.82 |
129
+ |nextvit_large.bd_in1k_384 |84.924|15.076 |97.294|2.706 |57.87 |
130
+ |nextvit_small.bd_ssld_6m_in1k |84.862|15.138 |97.382|2.618 |31.76 |
131
+ |nextvit_base.bd_in1k_384 |84.706|15.294 |97.224|2.776 |44.82 |
132
+ |nextvit_small.bd_in1k_384 |84.022|15.978 |96.99 |3.01 |31.76 |
133
+ |nextvit_large.bd_in1k |83.626|16.374 |96.694|3.306 |57.87 |
134
+ |nextvit_base.bd_in1k |83.472|16.528 |96.656|3.344 |44.82 |
135
+ |nextvit_small.bd_in1k |82.61 |17.39 |96.226|3.774 |31.76 |
136
+
137
+ ## Citation
138
+ ```bibtex
139
+ @article{li2022next,
140
+ title={Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios},
141
+ author={Li, Jiashi and Xia, Xin and Li, Wei and Li, Huixia and Wang, Xing and Xiao, Xuefeng and Wang, Rui and Zheng, Min and Pan, Xin},
142
+ journal={arXiv preprint arXiv:2207.05501},
143
+ year={2022}
144
+ }
145
+ ```
config.json ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architecture": "nextvit_base",
3
+ "num_classes": 1000,
4
+ "num_features": 1024,
5
+ "pretrained_cfg": {
6
+ "tag": "bd_ssld_6m_in1k",
7
+ "custom_load": false,
8
+ "input_size": [
9
+ 3,
10
+ 224,
11
+ 224
12
+ ],
13
+ "fixed_input_size": false,
14
+ "interpolation": "bicubic",
15
+ "crop_pct": 0.95,
16
+ "crop_mode": "center",
17
+ "mean": [
18
+ 0.485,
19
+ 0.456,
20
+ 0.406
21
+ ],
22
+ "std": [
23
+ 0.229,
24
+ 0.224,
25
+ 0.225
26
+ ],
27
+ "num_classes": 1000,
28
+ "pool_size": [
29
+ 7,
30
+ 7
31
+ ],
32
+ "first_conv": "stem.0.conv",
33
+ "classifier": "head.fc"
34
+ }
35
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e7bde38b7332fb61316bd8c339aeb8b2caef84216b4cc374c2679483f99ac6b
3
+ size 179589280
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:182fc53730a78ba21453440158ade39f45555617ce454642059439a8d808c4cd
3
+ size 179771610