Hugging Face's logo

fxmarty
/
20220712-h08m05s32_

Image Classification
TensorBoard
vit
Model card Files Metrics Community
1
fxmarty HF staff commited on
Commit
23d84ab
•
1 Parent(s): 2d63843

add experience

Browse files
Files changed (9) hide show
  1. README.md +156 -0
  2. runs.json +1728 -0
  3. tensorboard/1657613137.713749/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.1 +3 -0
  4. tensorboard/1657613137.7153692/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.2 +3 -0
  5. tensorboard/1657613137.7166162/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.3 +3 -0
  6. tensorboard/1657613137.7177505/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.4 +3 -0
  7. tensorboard/1657613137.7188103/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.5 +3 -0
  8. tensorboard/1657613137.7203975/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.6 +3 -0
  9. tensorboard/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.0 +3 -0
README.md ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: image-classification
3
+ datasets:
4
+ - beans
5
+ metrics:
6
+ - accuracy
7
+ tags:
8
+ - vit
9
+ ---
10
+
11
+ **task**: `image-classification`
12
+ **Backend:** `sagemaker-training`
13
+ **Backend args:** `{'instance_type': 'ml.g4dn.2xlarge', 'supported_instructions': None}`
14
+ **Number of evaluation samples:** `All dataset`
15
+
16
+ Fixed parameters:
17
+ * **model_name_or_path**: `nateraw/vit-base-beans`
18
+ * **dataset**:
19
+ * **path**: `beans`
20
+ * **eval_split**: `validation`
21
+ * **data_keys**: `{'primary': 'image'}`
22
+ * **ref_keys**: `['labels']`
23
+ * **quantization_approach**: `dynamic`
24
+ * **node_exclusion**: `[]`
25
+ * **framework**: `onnxruntime`
26
+ * **framework_args**:
27
+ * **opset**: `11`
28
+ * **optimization_level**: `1`
29
+ * **aware_training**: `False`
30
+
31
+ Benchmarked parameters:
32
+ * **operators_to_quantize**: `['Add', 'MatMul']`, `['Add']`, `[]`
33
+ * **per_channel**: `False`, `True`
34
+
35
+ # Evaluation
36
+ ## Non-time metrics
37
+ | operators_to_quantize | per_channel | | accuracy (original) | accuracy (optimized) |
38
+ | :-------------------: | :---------: | :-: | :-----------------: | :------------------: |
39
+ | `['Add', 'MatMul']` | `False` | \| | 0.980 | 0.980 |
40
+ | `['Add', 'MatMul']` | `True` | \| | 0.980 | 0.980 |
41
+ | `['Add']` | `False` | \| | 0.980 | 0.980 |
42
+ | `['Add']` | `True` | \| | 0.980 | 0.980 |
43
+ | `[]` | `False` | \| | 0.980 | 0.980 |
44
+ | `[]` | `True` | \| | 0.980 | 0.980 |
45
+
46
+ ## Time metrics
47
+ Time benchmarks were run for 15 seconds per config.
48
+
49
+
50
+ Below, time metrics for batch size = 1, input length = 32.
51
+
52
+ | operators_to_quantize | per_channel | | latency_mean (original, ms) | latency_mean (optimized, ms) | | throughput (original, /s) | throughput (optimized, /s) |
53
+ | :-------------------: | :---------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: |
54
+ | `['Add', 'MatMul']` | `False` | \| | 201.25 | 70.30 | \| | 5.00 | 14.27 |
55
+ | `['Add', 'MatMul']` | `True` | \| | 203.52 | 72.48 | \| | 4.93 | 13.80 |
56
+ | `['Add']` | `False` | \| | 166.03 | 150.93 | \| | 6.07 | 6.67 |
57
+ | `['Add']` | `True` | \| | 200.82 | 163.17 | \| | 5.00 | 6.13 |
58
+ | `[]` | `False` | \| | 190.99 | 162.06 | \| | 5.27 | 6.20 |
59
+ | `[]` | `True` | \| | 155.15 | 162.52 | \| | 6.47 | 6.20 |
60
+
61
+
62
+ Below, time metrics for batch size = 1, input length = 64.
63
+
64
+ | operators_to_quantize | per_channel | | latency_mean (original, ms) | latency_mean (optimized, ms) | | throughput (original, /s) | throughput (optimized, /s) |
65
+ | :-------------------: | :---------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: |
66
+ | `['Add', 'MatMul']` | `False` | \| | 165.85 | 70.60 | \| | 6.07 | 14.20 |
67
+ | `['Add', 'MatMul']` | `True` | \| | 161.41 | 72.71 | \| | 6.20 | 13.80 |
68
+ | `['Add']` | `False` | \| | 200.45 | 129.40 | \| | 5.00 | 7.73 |
69
+ | `['Add']` | `True` | \| | 154.68 | 136.42 | \| | 6.47 | 7.40 |
70
+ | `[]` | `False` | \| | 166.97 | 162.15 | \| | 6.00 | 6.20 |
71
+ | `[]` | `True` | \| | 166.32 | 162.81 | \| | 6.07 | 6.20 |
72
+
73
+
74
+ Below, time metrics for batch size = 1, input length = 128.
75
+
76
+ | operators_to_quantize | per_channel | | latency_mean (original, ms) | latency_mean (optimized, ms) | | throughput (original, /s) | throughput (optimized, /s) |
77
+ | :-------------------: | :---------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: |
78
+ | `['Add', 'MatMul']` | `False` | \| | 199.48 | 70.98 | \| | 5.07 | 14.13 |
79
+ | `['Add', 'MatMul']` | `True` | \| | 199.65 | 71.78 | \| | 5.07 | 13.93 |
80
+ | `['Add']` | `False` | \| | 199.08 | 137.97 | \| | 5.07 | 7.27 |
81
+ | `['Add']` | `True` | \| | 189.93 | 162.45 | \| | 5.33 | 6.20 |
82
+ | `[]` | `False` | \| | 191.63 | 162.54 | \| | 5.27 | 6.20 |
83
+ | `[]` | `True` | \| | 200.38 | 162.55 | \| | 5.00 | 6.20 |
84
+
85
+
86
+ Below, time metrics for batch size = 4, input length = 32.
87
+
88
+ | operators_to_quantize | per_channel | | latency_mean (original, ms) | latency_mean (optimized, ms) | | throughput (original, /s) | throughput (optimized, /s) |
89
+ | :-------------------: | :---------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: |
90
+ | `['Add', 'MatMul']` | `False` | \| | 655.84 | 243.33 | \| | 1.53 | 4.13 |
91
+ | `['Add', 'MatMul']` | `True` | \| | 661.27 | 221.16 | \| | 1.53 | 4.53 |
92
+ | `['Add']` | `False` | \| | 662.84 | 529.28 | \| | 1.53 | 1.93 |
93
+ | `['Add']` | `True` | \| | 512.47 | 470.66 | \| | 2.00 | 2.13 |
94
+ | `[]` | `False` | \| | 562.81 | 501.77 | \| | 1.80 | 2.00 |
95
+ | `[]` | `True` | \| | 505.81 | 521.20 | \| | 2.00 | 1.93 |
96
+
97
+
98
+ Below, time metrics for batch size = 4, input length = 64.
99
+
100
+ | operators_to_quantize | per_channel | | latency_mean (original, ms) | latency_mean (optimized, ms) | | throughput (original, /s) | throughput (optimized, /s) |
101
+ | :-------------------: | :---------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: |
102
+ | `['Add', 'MatMul']` | `False` | \| | 654.58 | 258.54 | \| | 1.53 | 3.93 |
103
+ | `['Add', 'MatMul']` | `True` | \| | 617.44 | 234.05 | \| | 1.67 | 4.33 |
104
+ | `['Add']` | `False` | \| | 661.51 | 478.81 | \| | 1.53 | 2.13 |
105
+ | `['Add']` | `True` | \| | 657.01 | 660.23 | \| | 1.53 | 1.53 |
106
+ | `[]` | `False` | \| | 661.64 | 474.28 | \| | 1.53 | 2.13 |
107
+ | `[]` | `True` | \| | 661.29 | 471.09 | \| | 1.53 | 2.13 |
108
+
109
+
110
+ Below, time metrics for batch size = 4, input length = 128.
111
+
112
+ | operators_to_quantize | per_channel | | latency_mean (original, ms) | latency_mean (optimized, ms) | | throughput (original, /s) | throughput (optimized, /s) |
113
+ | :-------------------: | :---------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: |
114
+ | `['Add', 'MatMul']` | `False` | \| | 654.80 | 219.38 | \| | 1.53 | 4.60 |
115
+ | `['Add', 'MatMul']` | `True` | \| | 663.50 | 222.37 | \| | 1.53 | 4.53 |
116
+ | `['Add']` | `False` | \| | 625.56 | 529.02 | \| | 1.60 | 1.93 |
117
+ | `['Add']` | `True` | \| | 655.08 | 499.41 | \| | 1.53 | 2.07 |
118
+ | `[]` | `False` | \| | 655.92 | 473.01 | \| | 1.53 | 2.13 |
119
+ | `[]` | `True` | \| | 505.54 | 659.92 | \| | 2.00 | 1.53 |
120
+
121
+
122
+ Below, time metrics for batch size = 8, input length = 32.
123
+
124
+ | operators_to_quantize | per_channel | | latency_mean (original, ms) | latency_mean (optimized, ms) | | throughput (original, /s) | throughput (optimized, /s) |
125
+ | :-------------------: | :---------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: |
126
+ | `['Add', 'MatMul']` | `False` | \| | 968.83 | 443.80 | \| | 1.07 | 2.27 |
127
+ | `['Add', 'MatMul']` | `True` | \| | 1255.70 | 489.55 | \| | 0.80 | 2.07 |
128
+ | `['Add']` | `False` | \| | 1301.35 | 938.14 | \| | 0.80 | 1.07 |
129
+ | `['Add']` | `True` | \| | 1279.54 | 931.91 | \| | 0.80 | 1.13 |
130
+ | `[]` | `False` | \| | 1292.66 | 1318.07 | \| | 0.80 | 0.80 |
131
+ | `[]` | `True` | \| | 1290.35 | 1314.74 | \| | 0.80 | 0.80 |
132
+
133
+
134
+ Below, time metrics for batch size = 8, input length = 64.
135
+
136
+ | operators_to_quantize | per_channel | | latency_mean (original, ms) | latency_mean (optimized, ms) | | throughput (original, /s) | throughput (optimized, /s) |
137
+ | :-------------------: | :---------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: |
138
+ | `['Add', 'MatMul']` | `False` | \| | 1305.45 | 438.06 | \| | 0.80 | 2.33 |
139
+ | `['Add', 'MatMul']` | `True` | \| | 1296.68 | 450.40 | \| | 0.80 | 2.27 |
140
+ | `['Add']` | `False` | \| | 968.21 | 949.81 | \| | 1.07 | 1.07 |
141
+ | `['Add']` | `True` | \| | 1012.35 | 1317.46 | \| | 1.00 | 0.80 |
142
+ | `[]` | `False` | \| | 1213.91 | 961.79 | \| | 0.87 | 1.07 |
143
+ | `[]` | `True` | \| | 956.39 | 945.41 | \| | 1.07 | 1.07 |
144
+
145
+
146
+ Below, time metrics for batch size = 8, input length = 128.
147
+
148
+ | operators_to_quantize | per_channel | | latency_mean (original, ms) | latency_mean (optimized, ms) | | throughput (original, /s) | throughput (optimized, /s) |
149
+ | :-------------------: | :---------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: |
150
+ | `['Add', 'MatMul']` | `False` | \| | 1120.12 | 497.17 | \| | 0.93 | 2.07 |
151
+ | `['Add', 'MatMul']` | `True` | \| | 1289.50 | 443.46 | \| | 0.80 | 2.27 |
152
+ | `['Add']` | `False` | \| | 1294.65 | 930.97 | \| | 0.80 | 1.13 |
153
+ | `['Add']` | `True` | \| | 1181.21 | 933.82 | \| | 0.87 | 1.13 |
154
+ | `[]` | `False` | \| | 1245.61 | 1318.07 | \| | 0.87 | 0.80 |
155
+ | `[]` | `True` | \| | 1285.81 | 1318.82 | \| | 0.80 | 0.80 |
156
+
runs.json ADDED
@@ -0,0 +1,1728 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "model_name_or_path": "nateraw/vit-base-beans",
4
+ "task": "image-classification",
5
+ "dataset": {
6
+ "path": "beans",
7
+ "eval_split": "validation",
8
+ "data_keys": {
9
+ "primary": "image",
10
+ "secondary": null
11
+ },
12
+ "ref_keys": [
13
+ "labels"
14
+ ],
15
+ "name": null,
16
+ "calibration_split": null
17
+ },
18
+ "quantization_approach": "dynamic",
19
+ "operators_to_quantize": [],
20
+ "node_exclusion": [],
21
+ "aware_training": false,
22
+ "per_channel": false,
23
+ "calibration": null,
24
+ "framework": "onnxruntime",
25
+ "framework_args": {
26
+ "opset": 11,
27
+ "optimization_level": 1
28
+ },
29
+ "hardware": "Architecture: x86_64\nCPU op-mode(s): 32-bit, 64-bit\nByte Order: Little Endian\nAddress sizes: 46 bits physical, 48 bits virtual\nCPU(s): 8\nOn-line CPU(s) list: 0-7\nThread(s) per core: 2\nCore(s) per socket: 4\nSocket(s): 1\nNUMA node(s): 1\nVendor ID: GenuineIntel\nCPU family: 6\nModel: 85\nModel name: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz\nStepping: 7\nCPU MHz: 3100.973\nBogoMIPS: 4999.99\nHypervisor vendor: KVM\nVirtualization type: full\nL1d cache: 128 KiB\nL1i cache: 128 KiB\nL2 cache: 4 MiB\nL3 cache: 35.8 MiB\nNUMA node0 CPU(s): 0-7\nVulnerability Itlb multihit: KVM: Vulnerable\nVulnerability L1tf: Mitigation; PTE Inversion\nVulnerability Mds: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown\nVulnerability Meltdown: Mitigation; PTI\nVulnerability Spec store bypass: Vulnerable\nVulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization\nVulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling\nVulnerability Srbds: Not affected\nVulnerability Tsx async abort: Not affected\nFlags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke avx512_vnni\n",
30
+ "versions": {
31
+ "transformers": "4.20.1",
32
+ "optimum": "1.2.3.dev0",
33
+ "optimum_hash": "5ac9c0d9fd7e7cca55b2f9935b961ed5b6c50112"
34
+ },
35
+ "evaluation": {
36
+ "time": [
37
+ {
38
+ "batch_size": 8,
39
+ "input_length": 32,
40
+ "baseline": {
41
+ "nb_forwards": 12,
42
+ "throughput": 0.8,
43
+ "latency_mean": 1292.6622040833333,
44
+ "latency_std": 14.378985581543033,
45
+ "latency_50": 1292.895404,
46
+ "latency_90": 1309.2992319,
47
+ "latency_95": 1312.26437405,
48
+ "latency_99": 1315.14182841,
49
+ "latency_999": 1315.789255641
50
+ },
51
+ "optimized": {
52
+ "nb_forwards": 12,
53
+ "throughput": 0.8,
54
+ "latency_mean": 1318.07484525,
55
+ "latency_std": 2.1649871576445947,
56
+ "latency_50": 1317.9739275,
57
+ "latency_90": 1321.6588582,
58
+ "latency_95": 1322.10577285,
59
+ "latency_99": 1322.27261777,
60
+ "latency_999": 1322.3101578770002
61
+ }
62
+ },
63
+ {
64
+ "batch_size": 8,
65
+ "input_length": 64,
66
+ "baseline": {
67
+ "nb_forwards": 13,
68
+ "throughput": 0.87,
69
+ "latency_mean": 1213.9143344615386,
70
+ "latency_std": 110.1812092312708,
71
+ "latency_50": 1259.642669,
72
+ "latency_90": 1280.0082366,
73
+ "latency_95": 1282.0939434000002,
74
+ "latency_99": 1283.78822388,
75
+ "latency_999": 1284.169436988
76
+ },
77
+ "optimized": {
78
+ "nb_forwards": 16,
79
+ "throughput": 1.07,
80
+ "latency_mean": 961.791823,
81
+ "latency_std": 104.26168200680283,
82
+ "latency_50": 928.5080535,
83
+ "latency_90": 1027.710494,
84
+ "latency_95": 1167.36817775,
85
+ "latency_99": 1291.45591475,
86
+ "latency_999": 1319.3756555749999
87
+ }
88
+ },
89
+ {
90
+ "batch_size": 1,
91
+ "input_length": 64,
92
+ "baseline": {
93
+ "nb_forwards": 90,
94
+ "throughput": 6.0,
95
+ "latency_mean": 166.9657096111111,
96
+ "latency_std": 19.234928655802502,
97
+ "latency_50": 155.146231,
98
+ "latency_90": 197.2404852,
99
+ "latency_95": 197.6184337,
100
+ "latency_99": 198.50318102,
101
+ "latency_999": 199.948491002
102
+ },
103
+ "optimized": {
104
+ "nb_forwards": 93,
105
+ "throughput": 6.2,
106
+ "latency_mean": 162.14788376344086,
107
+ "latency_std": 0.6774441784666013,
108
+ "latency_50": 162.056419,
109
+ "latency_90": 162.494086,
110
+ "latency_95": 162.57127219999998,
111
+ "latency_99": 163.26029604,
112
+ "latency_999": 167.65193600400002
113
+ }
114
+ },
115
+ {
116
+ "batch_size": 8,
117
+ "input_length": 128,
118
+ "baseline": {
119
+ "nb_forwards": 13,
120
+ "throughput": 0.87,
121
+ "latency_mean": 1245.6068604615386,
122
+ "latency_std": 77.66408642086917,
123
+ "latency_50": 1272.574299,
124
+ "latency_90": 1274.933101,
125
+ "latency_95": 1275.4852415999999,
126
+ "latency_99": 1275.65235792,
127
+ "latency_999": 1275.689959092
128
+ },
129
+ "optimized": {
130
+ "nb_forwards": 12,
131
+ "throughput": 0.8,
132
+ "latency_mean": 1318.074927,
133
+ "latency_std": 2.5544177471077765,
134
+ "latency_50": 1317.6566105,
135
+ "latency_90": 1321.9495818,
136
+ "latency_95": 1322.64612275,
137
+ "latency_99": 1323.13381655,
138
+ "latency_999": 1323.243547655
139
+ }
140
+ },
141
+ {
142
+ "batch_size": 4,
143
+ "input_length": 32,
144
+ "baseline": {
145
+ "nb_forwards": 27,
146
+ "throughput": 1.8,
147
+ "latency_mean": 562.8057038518518,
148
+ "latency_std": 71.38555755484205,
149
+ "latency_50": 509.265269,
150
+ "latency_90": 654.6393252,
151
+ "latency_95": 654.9679175,
152
+ "latency_99": 656.29719274,
153
+ "latency_999": 656.685855274
154
+ },
155
+ "optimized": {
156
+ "nb_forwards": 30,
157
+ "throughput": 2.0,
158
+ "latency_mean": 501.7736329333333,
159
+ "latency_std": 55.80762294417158,
160
+ "latency_50": 480.2483315,
161
+ "latency_90": 542.9033260000002,
162
+ "latency_95": 658.5849407000001,
163
+ "latency_99": 663.27316083,
164
+ "latency_999": 664.471430883
165
+ }
166
+ },
167
+ {
168
+ "batch_size": 1,
169
+ "input_length": 128,
170
+ "baseline": {
171
+ "nb_forwards": 79,
172
+ "throughput": 5.27,
173
+ "latency_mean": 191.62850325316455,
174
+ "latency_std": 17.746003780554855,
175
+ "latency_50": 199.661154,
176
+ "latency_90": 203.1430258,
177
+ "latency_95": 204.9078804,
178
+ "latency_99": 208.3248753,
179
+ "latency_999": 208.39238312999998
180
+ },
181
+ "optimized": {
182
+ "nb_forwards": 93,
183
+ "throughput": 6.2,
184
+ "latency_mean": 162.54266559139785,
185
+ "latency_std": 0.5169025249532633,
186
+ "latency_50": 162.432963,
187
+ "latency_90": 163.1380686,
188
+ "latency_95": 163.295702,
189
+ "latency_99": 164.58895547999998,
190
+ "latency_999": 164.993086548
191
+ }
192
+ },
193
+ {
194
+ "batch_size": 4,
195
+ "input_length": 64,
196
+ "baseline": {
197
+ "nb_forwards": 23,
198
+ "throughput": 1.53,
199
+ "latency_mean": 661.6382227391304,
200
+ "latency_std": 0.8913331138378804,
201
+ "latency_50": 661.329055,
202
+ "latency_90": 662.309612,
203
+ "latency_95": 663.6607806000001,
204
+ "latency_99": 664.23226672,
205
+ "latency_999": 664.339438972
206
+ },
207
+ "optimized": {
208
+ "nb_forwards": 32,
209
+ "throughput": 2.13,
210
+ "latency_mean": 474.27704703125,
211
+ "latency_std": 8.662769183818762,
212
+ "latency_50": 474.1899635,
213
+ "latency_90": 485.4294334,
214
+ "latency_95": 485.85774355,
215
+ "latency_99": 487.66797085,
216
+ "latency_999": 488.291721385
217
+ }
218
+ },
219
+ {
220
+ "batch_size": 4,
221
+ "input_length": 128,
222
+ "baseline": {
223
+ "nb_forwards": 23,
224
+ "throughput": 1.53,
225
+ "latency_mean": 655.924965826087,
226
+ "latency_std": 3.296725754580373,
227
+ "latency_50": 654.734062,
228
+ "latency_90": 660.6470922000001,
229
+ "latency_95": 662.5686226,
230
+ "latency_99": 664.9523018,
231
+ "latency_999": 665.51169338
232
+ },
233
+ "optimized": {
234
+ "nb_forwards": 32,
235
+ "throughput": 2.13,
236
+ "latency_mean": 473.009812875,
237
+ "latency_std": 8.845356653932509,
238
+ "latency_50": 471.9876055,
239
+ "latency_90": 482.61548319999997,
240
+ "latency_95": 485.2878379,
241
+ "latency_99": 488.35063118,
242
+ "latency_999": 489.241902818
243
+ }
244
+ },
245
+ {
246
+ "batch_size": 1,
247
+ "input_length": 32,
248
+ "baseline": {
249
+ "nb_forwards": 79,
250
+ "throughput": 5.27,
251
+ "latency_mean": 190.98778439240508,
252
+ "latency_std": 17.30226702851961,
253
+ "latency_50": 199.368468,
254
+ "latency_90": 200.088627,
255
+ "latency_95": 200.4188227,
256
+ "latency_99": 200.62923102000002,
257
+ "latency_999": 200.790088002
258
+ },
259
+ "optimized": {
260
+ "nb_forwards": 93,
261
+ "throughput": 6.2,
262
+ "latency_mean": 162.05550177419354,
263
+ "latency_std": 0.2734454225531516,
264
+ "latency_50": 162.042985,
265
+ "latency_90": 162.41303159999998,
266
+ "latency_95": 162.54073480000002,
267
+ "latency_99": 162.64262688,
268
+ "latency_999": 162.750607188
269
+ }
270
+ }
271
+ ],
272
+ "others": {
273
+ "baseline": {
274
+ "accuracy": 0.98
275
+ },
276
+ "optimized": {
277
+ "accuracy": 0.98
278
+ }
279
+ }
280
+ },
281
+ "max_eval_samples": null,
282
+ "time_benchmark_args": {
283
+ "duration": 15,
284
+ "warmup_runs": 5
285
+ },
286
+ "model_type": "vit"
287
+ },
288
+ {
289
+ "model_name_or_path": "nateraw/vit-base-beans",
290
+ "task": "image-classification",
291
+ "dataset": {
292
+ "path": "beans",
293
+ "eval_split": "validation",
294
+ "data_keys": {
295
+ "primary": "image",
296
+ "secondary": null
297
+ },
298
+ "ref_keys": [
299
+ "labels"
300
+ ],
301
+ "name": null,
302
+ "calibration_split": null
303
+ },
304
+ "quantization_approach": "dynamic",
305
+ "operators_to_quantize": [
306
+ "Add",
307
+ "MatMul"
308
+ ],
309
+ "node_exclusion": [],
310
+ "aware_training": false,
311
+ "per_channel": false,
312
+ "calibration": null,
313
+ "framework": "onnxruntime",
314
+ "framework_args": {
315
+ "opset": 11,
316
+ "optimization_level": 1
317
+ },
318
+ "hardware": "Architecture: x86_64\nCPU op-mode(s): 32-bit, 64-bit\nByte Order: Little Endian\nAddress sizes: 46 bits physical, 48 bits virtual\nCPU(s): 8\nOn-line CPU(s) list: 0-7\nThread(s) per core: 2\nCore(s) per socket: 4\nSocket(s): 1\nNUMA node(s): 1\nVendor ID: GenuineIntel\nCPU family: 6\nModel: 85\nModel name: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz\nStepping: 7\nCPU MHz: 3102.494\nBogoMIPS: 4999.99\nHypervisor vendor: KVM\nVirtualization type: full\nL1d cache: 128 KiB\nL1i cache: 128 KiB\nL2 cache: 4 MiB\nL3 cache: 35.8 MiB\nNUMA node0 CPU(s): 0-7\nVulnerability Itlb multihit: KVM: Vulnerable\nVulnerability L1tf: Mitigation; PTE Inversion\nVulnerability Mds: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown\nVulnerability Meltdown: Mitigation; PTI\nVulnerability Spec store bypass: Vulnerable\nVulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization\nVulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling\nVulnerability Srbds: Not affected\nVulnerability Tsx async abort: Not affected\nFlags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke avx512_vnni\n",
319
+ "versions": {
320
+ "transformers": "4.20.1",
321
+ "optimum": "1.2.3.dev0",
322
+ "optimum_hash": "5ac9c0d9fd7e7cca55b2f9935b961ed5b6c50112"
323
+ },
324
+ "evaluation": {
325
+ "time": [
326
+ {
327
+ "batch_size": 8,
328
+ "input_length": 64,
329
+ "baseline": {
330
+ "nb_forwards": 12,
331
+ "throughput": 0.8,
332
+ "latency_mean": 1305.4477120833333,
333
+ "latency_std": 35.538171285940294,
334
+ "latency_50": 1292.307282,
335
+ "latency_90": 1365.7192274000001,
336
+ "latency_95": 1365.90585495,
337
+ "latency_99": 1366.05358539,
338
+ "latency_999": 1366.0868247390001
339
+ },
340
+ "optimized": {
341
+ "nb_forwards": 35,
342
+ "throughput": 2.33,
343
+ "latency_mean": 438.0590386857143,
344
+ "latency_std": 9.397001314248586,
345
+ "latency_50": 437.299136,
346
+ "latency_90": 450.9606182,
347
+ "latency_95": 453.12420089999995,
348
+ "latency_99": 464.37316491999997,
349
+ "latency_999": 468.224622292
350
+ }
351
+ },
352
+ {
353
+ "batch_size": 1,
354
+ "input_length": 32,
355
+ "baseline": {
356
+ "nb_forwards": 75,
357
+ "throughput": 5.0,
358
+ "latency_mean": 201.245758,
359
+ "latency_std": 1.6999127687727664,
360
+ "latency_50": 201.408908,
361
+ "latency_90": 203.4231772,
362
+ "latency_95": 203.8507052,
363
+ "latency_99": 205.37529206,
364
+ "latency_999": 205.54267850600002
365
+ },
366
+ "optimized": {
367
+ "nb_forwards": 214,
368
+ "throughput": 14.27,
369
+ "latency_mean": 70.30087592990655,
370
+ "latency_std": 0.35279272958603625,
371
+ "latency_50": 70.2417255,
372
+ "latency_90": 70.7161781,
373
+ "latency_95": 70.98808025,
374
+ "latency_99": 71.56634268,
375
+ "latency_999": 71.98565450699999
376
+ }
377
+ },
378
+ {
379
+ "batch_size": 8,
380
+ "input_length": 32,
381
+ "baseline": {
382
+ "nb_forwards": 16,
383
+ "throughput": 1.07,
384
+ "latency_mean": 968.8260539375,
385
+ "latency_std": 15.812966073710593,
386
+ "latency_50": 971.2266795,
387
+ "latency_90": 989.217858,
388
+ "latency_95": 991.6490175,
389
+ "latency_99": 993.7897995,
390
+ "latency_999": 994.27147545
391
+ },
392
+ "optimized": {
393
+ "nb_forwards": 34,
394
+ "throughput": 2.27,
395
+ "latency_mean": 443.7999996176471,
396
+ "latency_std": 14.010678308843712,
397
+ "latency_50": 439.840854,
398
+ "latency_90": 470.9668241,
399
+ "latency_95": 472.79326175,
400
+ "latency_99": 481.8465039,
401
+ "latency_999": 485.13520569
402
+ }
403
+ },
404
+ {
405
+ "batch_size": 4,
406
+ "input_length": 32,
407
+ "baseline": {
408
+ "nb_forwards": 23,
409
+ "throughput": 1.53,
410
+ "latency_mean": 655.8404614782609,
411
+ "latency_std": 1.9853736674833404,
412
+ "latency_50": 655.276323,
413
+ "latency_90": 657.7868594,
414
+ "latency_95": 659.0648136,
415
+ "latency_99": 662.15786138,
416
+ "latency_999": 662.908107338
417
+ },
418
+ "optimized": {
419
+ "nb_forwards": 62,
420
+ "throughput": 4.13,
421
+ "latency_mean": 243.33499380645162,
422
+ "latency_std": 19.880611383661382,
423
+ "latency_50": 258.4691415,
424
+ "latency_90": 260.5706684,
425
+ "latency_95": 260.7413557,
426
+ "latency_99": 262.49824262,
427
+ "latency_999": 263.22405246200003
428
+ }
429
+ },
430
+ {
431
+ "batch_size": 8,
432
+ "input_length": 128,
433
+ "baseline": {
434
+ "nb_forwards": 14,
435
+ "throughput": 0.93,
436
+ "latency_mean": 1120.115416357143,
437
+ "latency_std": 162.01666423442674,
438
+ "latency_50": 1093.1451635,
439
+ "latency_90": 1307.5234237,
440
+ "latency_95": 1317.8936085999999,
441
+ "latency_99": 1320.5370161199999,
442
+ "latency_999": 1321.131782812
443
+ },
444
+ "optimized": {
445
+ "nb_forwards": 31,
446
+ "throughput": 2.07,
447
+ "latency_mean": 497.1726317419355,
448
+ "latency_std": 45.71850065729019,
449
+ "latency_50": 528.362607,
450
+ "latency_90": 532.55914,
451
+ "latency_95": 533.1437515,
452
+ "latency_99": 536.8183057,
453
+ "latency_999": 538.16474467
454
+ }
455
+ },
456
+ {
457
+ "batch_size": 4,
458
+ "input_length": 128,
459
+ "baseline": {
460
+ "nb_forwards": 23,
461
+ "throughput": 1.53,
462
+ "latency_mean": 654.7982457826088,
463
+ "latency_std": 0.9735164168304667,
464
+ "latency_50": 654.840949,
465
+ "latency_90": 656.1865757999999,
466
+ "latency_95": 656.3044589,
467
+ "latency_99": 656.7216594600001,
468
+ "latency_999": 656.827115646
469
+ },
470
+ "optimized": {
471
+ "nb_forwards": 69,
472
+ "throughput": 4.6,
473
+ "latency_mean": 219.3826653768116,
474
+ "latency_std": 4.794156646156331,
475
+ "latency_50": 217.991558,
476
+ "latency_90": 225.22350740000002,
477
+ "latency_95": 230.0291534,
478
+ "latency_99": 233.93463011999998,
479
+ "latency_999": 236.29746481200002
480
+ }
481
+ },
482
+ {
483
+ "batch_size": 4,
484
+ "input_length": 64,
485
+ "baseline": {
486
+ "nb_forwards": 23,
487
+ "throughput": 1.53,
488
+ "latency_mean": 654.579026173913,
489
+ "latency_std": 0.9392631004917424,
490
+ "latency_50": 654.222943,
491
+ "latency_90": 655.8028544,
492
+ "latency_95": 656.1702655,
493
+ "latency_99": 657.1910353,
494
+ "latency_999": 657.44114593
495
+ },
496
+ "optimized": {
497
+ "nb_forwards": 59,
498
+ "throughput": 3.93,
499
+ "latency_mean": 258.5395601016949,
500
+ "latency_std": 0.9767678625811486,
501
+ "latency_50": 258.393859,
502
+ "latency_90": 259.7185146,
503
+ "latency_95": 260.0651381,
504
+ "latency_99": 261.55840048,
505
+ "latency_999": 261.836545048
506
+ }
507
+ },
508
+ {
509
+ "batch_size": 1,
510
+ "input_length": 128,
511
+ "baseline": {
512
+ "nb_forwards": 76,
513
+ "throughput": 5.07,
514
+ "latency_mean": 199.48443313157895,
515
+ "latency_std": 0.597793597350561,
516
+ "latency_50": 199.475957,
517
+ "latency_90": 200.222022,
518
+ "latency_95": 200.34930725,
519
+ "latency_99": 200.67836,
520
+ "latency_999": 201.2770931
521
+ },
522
+ "optimized": {
523
+ "nb_forwards": 212,
524
+ "throughput": 14.13,
525
+ "latency_mean": 70.97961195754718,
526
+ "latency_std": 0.7368611290841657,
527
+ "latency_50": 70.863433,
528
+ "latency_90": 71.6552091,
529
+ "latency_95": 71.9515168,
530
+ "latency_99": 73.01431738,
531
+ "latency_999": 73.983875939
532
+ }
533
+ },
534
+ {
535
+ "batch_size": 1,
536
+ "input_length": 64,
537
+ "baseline": {
538
+ "nb_forwards": 91,
539
+ "throughput": 6.07,
540
+ "latency_mean": 165.84583646153845,
541
+ "latency_std": 19.342885270197435,
542
+ "latency_50": 156.269802,
543
+ "latency_90": 201.686354,
544
+ "latency_95": 202.9267375,
545
+ "latency_99": 204.1038235,
546
+ "latency_999": 204.40679995
547
+ },
548
+ "optimized": {
549
+ "nb_forwards": 213,
550
+ "throughput": 14.2,
551
+ "latency_mean": 70.59720956338029,
552
+ "latency_std": 0.2543356304868392,
553
+ "latency_50": 70.562235,
554
+ "latency_90": 70.9770522,
555
+ "latency_95": 71.05967240000001,
556
+ "latency_99": 71.12613707999999,
557
+ "latency_999": 71.243581772
558
+ }
559
+ }
560
+ ],
561
+ "others": {
562
+ "baseline": {
563
+ "accuracy": 0.98
564
+ },
565
+ "optimized": {
566
+ "accuracy": 0.98
567
+ }
568
+ }
569
+ },
570
+ "max_eval_samples": null,
571
+ "time_benchmark_args": {
572
+ "duration": 15,
573
+ "warmup_runs": 5
574
+ },
575
+ "model_type": "vit"
576
+ },
577
+ {
578
+ "model_name_or_path": "nateraw/vit-base-beans",
579
+ "task": "image-classification",
580
+ "dataset": {
581
+ "path": "beans",
582
+ "eval_split": "validation",
583
+ "data_keys": {
584
+ "primary": "image",
585
+ "secondary": null
586
+ },
587
+ "ref_keys": [
588
+ "labels"
589
+ ],
590
+ "name": null,
591
+ "calibration_split": null
592
+ },
593
+ "quantization_approach": "dynamic",
594
+ "operators_to_quantize": [],
595
+ "node_exclusion": [],
596
+ "aware_training": false,
597
+ "per_channel": true,
598
+ "calibration": null,
599
+ "framework": "onnxruntime",
600
+ "framework_args": {
601
+ "opset": 11,
602
+ "optimization_level": 1
603
+ },
604
+ "hardware": "Architecture: x86_64\nCPU op-mode(s): 32-bit, 64-bit\nByte Order: Little Endian\nAddress sizes: 46 bits physical, 48 bits virtual\nCPU(s): 8\nOn-line CPU(s) list: 0-7\nThread(s) per core: 2\nCore(s) per socket: 4\nSocket(s): 1\nNUMA node(s): 1\nVendor ID: GenuineIntel\nCPU family: 6\nModel: 85\nModel name: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz\nStepping: 7\nCPU MHz: 3094.719\nBogoMIPS: 4999.99\nHypervisor vendor: KVM\nVirtualization type: full\nL1d cache: 128 KiB\nL1i cache: 128 KiB\nL2 cache: 4 MiB\nL3 cache: 35.8 MiB\nNUMA node0 CPU(s): 0-7\nVulnerability Itlb multihit: KVM: Vulnerable\nVulnerability L1tf: Mitigation; PTE Inversion\nVulnerability Mds: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown\nVulnerability Meltdown: Mitigation; PTI\nVulnerability Spec store bypass: Vulnerable\nVulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization\nVulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling\nVulnerability Srbds: Not affected\nVulnerability Tsx async abort: Not affected\nFlags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke avx512_vnni\n",
605
+ "versions": {
606
+ "transformers": "4.20.1",
607
+ "optimum": "1.2.3.dev0",
608
+ "optimum_hash": "5ac9c0d9fd7e7cca55b2f9935b961ed5b6c50112"
609
+ },
610
+ "evaluation": {
611
+ "time": [
612
+ {
613
+ "batch_size": 1,
614
+ "input_length": 64,
615
+ "baseline": {
616
+ "nb_forwards": 91,
617
+ "throughput": 6.07,
618
+ "latency_mean": 166.31874561538461,
619
+ "latency_std": 19.72451378066001,
620
+ "latency_50": 155.056089,
621
+ "latency_90": 199.330499,
622
+ "latency_95": 200.1285845,
623
+ "latency_99": 201.3523765,
624
+ "latency_999": 202.36380325
625
+ },
626
+ "optimized": {
627
+ "nb_forwards": 93,
628
+ "throughput": 6.2,
629
+ "latency_mean": 162.8075684516129,
630
+ "latency_std": 0.4899426115628021,
631
+ "latency_50": 162.752657,
632
+ "latency_90": 163.428205,
633
+ "latency_95": 163.61467280000002,
634
+ "latency_99": 164.02712072,
635
+ "latency_999": 164.65947177200002
636
+ }
637
+ },
638
+ {
639
+ "batch_size": 4,
640
+ "input_length": 64,
641
+ "baseline": {
642
+ "nb_forwards": 23,
643
+ "throughput": 1.53,
644
+ "latency_mean": 661.2918193478262,
645
+ "latency_std": 1.2312122042374622,
646
+ "latency_50": 660.888877,
647
+ "latency_90": 662.8252232000001,
648
+ "latency_95": 664.1392218999999,
649
+ "latency_99": 664.48232834,
650
+ "latency_999": 664.5344920340001
651
+ },
652
+ "optimized": {
653
+ "nb_forwards": 32,
654
+ "throughput": 2.13,
655
+ "latency_mean": 471.08557159375,
656
+ "latency_std": 6.881870741980042,
657
+ "latency_50": 471.846767,
658
+ "latency_90": 477.006957,
659
+ "latency_95": 479.45303175,
660
+ "latency_99": 483.97764233,
661
+ "latency_999": 484.659785333
662
+ }
663
+ },
664
+ {
665
+ "batch_size": 8,
666
+ "input_length": 64,
667
+ "baseline": {
668
+ "nb_forwards": 16,
669
+ "throughput": 1.07,
670
+ "latency_mean": 956.3890503125,
671
+ "latency_std": 17.516545911122616,
672
+ "latency_50": 955.3814065,
673
+ "latency_90": 971.4774225,
674
+ "latency_95": 986.509896,
675
+ "latency_99": 1003.5322224,
676
+ "latency_999": 1007.36224584
677
+ },
678
+ "optimized": {
679
+ "nb_forwards": 16,
680
+ "throughput": 1.07,
681
+ "latency_mean": 945.406017875,
682
+ "latency_std": 34.15676228112691,
683
+ "latency_50": 935.284914,
684
+ "latency_90": 995.0056065,
685
+ "latency_95": 1010.02209725,
686
+ "latency_99": 1033.38679385,
687
+ "latency_999": 1038.6438505849999
688
+ }
689
+ },
690
+ {
691
+ "batch_size": 4,
692
+ "input_length": 32,
693
+ "baseline": {
694
+ "nb_forwards": 30,
695
+ "throughput": 2.0,
696
+ "latency_mean": 505.8109903,
697
+ "latency_std": 8.216224152297352,
698
+ "latency_50": 503.793211,
699
+ "latency_90": 511.9732179,
700
+ "latency_95": 520.8387349,
701
+ "latency_99": 533.81575209,
702
+ "latency_999": 537.108679209
703
+ },
704
+ "optimized": {
705
+ "nb_forwards": 29,
706
+ "throughput": 1.93,
707
+ "latency_mean": 521.2037808275862,
708
+ "latency_std": 80.14289996554862,
709
+ "latency_50": 476.865559,
710
+ "latency_90": 659.4999777999999,
711
+ "latency_95": 660.1967527999999,
712
+ "latency_99": 662.42166376,
713
+ "latency_999": 663.0680583760001
714
+ }
715
+ },
716
+ {
717
+ "batch_size": 8,
718
+ "input_length": 128,
719
+ "baseline": {
720
+ "nb_forwards": 12,
721
+ "throughput": 0.8,
722
+ "latency_mean": 1285.8075280833332,
723
+ "latency_std": 14.878247016495994,
724
+ "latency_50": 1276.9767825,
725
+ "latency_90": 1305.3958274000001,
726
+ "latency_95": 1313.57056855,
727
+ "latency_99": 1320.24036971,
728
+ "latency_999": 1321.741074971
729
+ },
730
+ "optimized": {
731
+ "nb_forwards": 12,
732
+ "throughput": 0.8,
733
+ "latency_mean": 1318.8150754166668,
734
+ "latency_std": 3.555651675868726,
735
+ "latency_50": 1318.741799,
736
+ "latency_90": 1322.4943963,
737
+ "latency_95": 1324.30691675,
738
+ "latency_99": 1325.98598095,
739
+ "latency_999": 1326.363770395
740
+ }
741
+ },
742
+ {
743
+ "batch_size": 4,
744
+ "input_length": 128,
745
+ "baseline": {
746
+ "nb_forwards": 30,
747
+ "throughput": 2.0,
748
+ "latency_mean": 505.5449488333333,
749
+ "latency_std": 2.9916510998013135,
750
+ "latency_50": 505.719641,
751
+ "latency_90": 509.8432626,
752
+ "latency_95": 510.09288375,
753
+ "latency_99": 510.2705112,
754
+ "latency_999": 510.32699682
755
+ },
756
+ "optimized": {
757
+ "nb_forwards": 23,
758
+ "throughput": 1.53,
759
+ "latency_mean": 659.9233325217391,
760
+ "latency_std": 1.3320484773330812,
761
+ "latency_50": 659.844342,
762
+ "latency_90": 661.1736396,
763
+ "latency_95": 662.8099841000001,
764
+ "latency_99": 663.25936154,
765
+ "latency_999": 663.329135354
766
+ }
767
+ },
768
+ {
769
+ "batch_size": 1,
770
+ "input_length": 32,
771
+ "baseline": {
772
+ "nb_forwards": 97,
773
+ "throughput": 6.47,
774
+ "latency_mean": 155.1479355773196,
775
+ "latency_std": 2.333512915391771,
776
+ "latency_50": 155.187476,
777
+ "latency_90": 157.83224819999998,
778
+ "latency_95": 158.7266684,
779
+ "latency_99": 159.5641865199999,
780
+ "latency_999": 166.83555695199996
781
+ },
782
+ "optimized": {
783
+ "nb_forwards": 93,
784
+ "throughput": 6.2,
785
+ "latency_mean": 162.52027249462367,
786
+ "latency_std": 0.3889875917639462,
787
+ "latency_50": 162.494248,
788
+ "latency_90": 163.1346838,
789
+ "latency_95": 163.3073516,
790
+ "latency_99": 163.47998024,
791
+ "latency_999": 163.503808424
792
+ }
793
+ },
794
+ {
795
+ "batch_size": 1,
796
+ "input_length": 128,
797
+ "baseline": {
798
+ "nb_forwards": 75,
799
+ "throughput": 5.0,
800
+ "latency_mean": 200.38047430666668,
801
+ "latency_std": 1.8423713060693128,
802
+ "latency_50": 200.733902,
803
+ "latency_90": 202.2674594,
804
+ "latency_95": 203.2803858,
805
+ "latency_99": 204.16154648,
806
+ "latency_999": 204.63530424799998
807
+ },
808
+ "optimized": {
809
+ "nb_forwards": 93,
810
+ "throughput": 6.2,
811
+ "latency_mean": 162.5525952580645,
812
+ "latency_std": 0.5907606366490742,
813
+ "latency_50": 162.381627,
814
+ "latency_90": 163.299415,
815
+ "latency_95": 163.5639292,
816
+ "latency_99": 164.28743187999999,
817
+ "latency_999": 165.355540288
818
+ }
819
+ },
820
+ {
821
+ "batch_size": 8,
822
+ "input_length": 32,
823
+ "baseline": {
824
+ "nb_forwards": 12,
825
+ "throughput": 0.8,
826
+ "latency_mean": 1290.3547091666667,
827
+ "latency_std": 25.877029400989002,
828
+ "latency_50": 1284.203467,
829
+ "latency_90": 1326.1323579,
830
+ "latency_95": 1341.7394276,
831
+ "latency_99": 1353.55684552,
832
+ "latency_999": 1356.215764552
833
+ },
834
+ "optimized": {
835
+ "nb_forwards": 12,
836
+ "throughput": 0.8,
837
+ "latency_mean": 1314.744368,
838
+ "latency_std": 2.6878210594945364,
839
+ "latency_50": 1314.675457,
840
+ "latency_90": 1317.7293187999999,
841
+ "latency_95": 1318.6035473,
842
+ "latency_99": 1319.4279286600001,
843
+ "latency_999": 1319.613414466
844
+ }
845
+ }
846
+ ],
847
+ "others": {
848
+ "baseline": {
849
+ "accuracy": 0.98
850
+ },
851
+ "optimized": {
852
+ "accuracy": 0.98
853
+ }
854
+ }
855
+ },
856
+ "max_eval_samples": null,
857
+ "time_benchmark_args": {
858
+ "duration": 15,
859
+ "warmup_runs": 5
860
+ },
861
+ "model_type": "vit"
862
+ },
863
+ {
864
+ "model_name_or_path": "nateraw/vit-base-beans",
865
+ "task": "image-classification",
866
+ "dataset": {
867
+ "path": "beans",
868
+ "eval_split": "validation",
869
+ "data_keys": {
870
+ "primary": "image",
871
+ "secondary": null
872
+ },
873
+ "ref_keys": [
874
+ "labels"
875
+ ],
876
+ "name": null,
877
+ "calibration_split": null
878
+ },
879
+ "quantization_approach": "dynamic",
880
+ "operators_to_quantize": [
881
+ "Add"
882
+ ],
883
+ "node_exclusion": [],
884
+ "aware_training": false,
885
+ "per_channel": true,
886
+ "calibration": null,
887
+ "framework": "onnxruntime",
888
+ "framework_args": {
889
+ "opset": 11,
890
+ "optimization_level": 1
891
+ },
892
+ "hardware": "Architecture: x86_64\nCPU op-mode(s): 32-bit, 64-bit\nByte Order: Little Endian\nAddress sizes: 46 bits physical, 48 bits virtual\nCPU(s): 8\nOn-line CPU(s) list: 0-7\nThread(s) per core: 2\nCore(s) per socket: 4\nSocket(s): 1\nNUMA node(s): 1\nVendor ID: GenuineIntel\nCPU family: 6\nModel: 85\nModel name: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz\nStepping: 7\nCPU MHz: 3098.350\nBogoMIPS: 4999.99\nHypervisor vendor: KVM\nVirtualization type: full\nL1d cache: 128 KiB\nL1i cache: 128 KiB\nL2 cache: 4 MiB\nL3 cache: 35.8 MiB\nNUMA node0 CPU(s): 0-7\nVulnerability Itlb multihit: KVM: Vulnerable\nVulnerability L1tf: Mitigation; PTE Inversion\nVulnerability Mds: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown\nVulnerability Meltdown: Mitigation; PTI\nVulnerability Spec store bypass: Vulnerable\nVulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization\nVulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling\nVulnerability Srbds: Not affected\nVulnerability Tsx async abort: Not affected\nFlags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke avx512_vnni\n",
893
+ "versions": {
894
+ "transformers": "4.20.1",
895
+ "optimum": "1.2.3.dev0",
896
+ "optimum_hash": "5ac9c0d9fd7e7cca55b2f9935b961ed5b6c50112"
897
+ },
898
+ "evaluation": {
899
+ "time": [
900
+ {
901
+ "batch_size": 1,
902
+ "input_length": 32,
903
+ "baseline": {
904
+ "nb_forwards": 75,
905
+ "throughput": 5.0,
906
+ "latency_mean": 200.81783336,
907
+ "latency_std": 0.6847088030325814,
908
+ "latency_50": 200.719484,
909
+ "latency_90": 201.5889308,
910
+ "latency_95": 202.1003811,
911
+ "latency_99": 202.8800941,
912
+ "latency_999": 203.30975400999998
913
+ },
914
+ "optimized": {
915
+ "nb_forwards": 92,
916
+ "throughput": 6.13,
917
+ "latency_mean": 163.16872673913045,
918
+ "latency_std": 1.5526071244167148,
919
+ "latency_50": 162.727792,
920
+ "latency_90": 164.082629,
921
+ "latency_95": 164.58716825,
922
+ "latency_99": 171.05059297999998,
923
+ "latency_999": 171.827268698
924
+ }
925
+ },
926
+ {
927
+ "batch_size": 8,
928
+ "input_length": 32,
929
+ "baseline": {
930
+ "nb_forwards": 12,
931
+ "throughput": 0.8,
932
+ "latency_mean": 1279.5420475833332,
933
+ "latency_std": 24.090200248013595,
934
+ "latency_50": 1270.306791,
935
+ "latency_90": 1306.8130119,
936
+ "latency_95": 1325.1522126,
937
+ "latency_99": 1341.54179372,
938
+ "latency_999": 1345.2294494720002
939
+ },
940
+ "optimized": {
941
+ "nb_forwards": 17,
942
+ "throughput": 1.13,
943
+ "latency_mean": 931.9069634117648,
944
+ "latency_std": 11.481934784441602,
945
+ "latency_50": 929.600892,
946
+ "latency_90": 952.1709192000001,
947
+ "latency_95": 952.655295,
948
+ "latency_99": 954.1036662,
949
+ "latency_999": 954.4295497200001
950
+ }
951
+ },
952
+ {
953
+ "batch_size": 4,
954
+ "input_length": 32,
955
+ "baseline": {
956
+ "nb_forwards": 30,
957
+ "throughput": 2.0,
958
+ "latency_mean": 512.4660677333334,
959
+ "latency_std": 27.823912989062475,
960
+ "latency_50": 506.9528315,
961
+ "latency_90": 512.7785411,
962
+ "latency_95": 515.4465188,
963
+ "latency_99": 619.0371795800002,
964
+ "latency_999": 656.605258058
965
+ },
966
+ "optimized": {
967
+ "nb_forwards": 32,
968
+ "throughput": 2.13,
969
+ "latency_mean": 470.661379375,
970
+ "latency_std": 6.879698154147933,
971
+ "latency_50": 469.55909,
972
+ "latency_90": 481.2147577,
973
+ "latency_95": 482.3090949,
974
+ "latency_99": 486.69427243,
975
+ "latency_999": 488.18883274300003
976
+ }
977
+ },
978
+ {
979
+ "batch_size": 1,
980
+ "input_length": 128,
981
+ "baseline": {
982
+ "nb_forwards": 80,
983
+ "throughput": 5.33,
984
+ "latency_mean": 189.92974869999998,
985
+ "latency_std": 20.0085609938314,
986
+ "latency_50": 200.9213655,
987
+ "latency_90": 204.7879244,
988
+ "latency_95": 206.2371924,
989
+ "latency_99": 210.71894763999998,
990
+ "latency_999": 211.282830364
991
+ },
992
+ "optimized": {
993
+ "nb_forwards": 93,
994
+ "throughput": 6.2,
995
+ "latency_mean": 162.44556237634407,
996
+ "latency_std": 0.3515497302340489,
997
+ "latency_50": 162.410174,
998
+ "latency_90": 162.9352528,
999
+ "latency_95": 163.0813806,
1000
+ "latency_99": 163.21715432,
1001
+ "latency_999": 163.446762032
1002
+ }
1003
+ },
1004
+ {
1005
+ "batch_size": 4,
1006
+ "input_length": 64,
1007
+ "baseline": {
1008
+ "nb_forwards": 23,
1009
+ "throughput": 1.53,
1010
+ "latency_mean": 657.0060832608697,
1011
+ "latency_std": 1.6045997297419479,
1012
+ "latency_50": 656.987305,
1013
+ "latency_90": 658.750338,
1014
+ "latency_95": 659.6307931,
1015
+ "latency_99": 660.7225840599999,
1016
+ "latency_999": 660.978761806
1017
+ },
1018
+ "optimized": {
1019
+ "nb_forwards": 23,
1020
+ "throughput": 1.53,
1021
+ "latency_mean": 660.2285839130435,
1022
+ "latency_std": 1.684238379976331,
1023
+ "latency_50": 659.931222,
1024
+ "latency_90": 662.106911,
1025
+ "latency_95": 662.5187496,
1026
+ "latency_99": 664.16473886,
1027
+ "latency_999": 664.572299486
1028
+ }
1029
+ },
1030
+ {
1031
+ "batch_size": 1,
1032
+ "input_length": 64,
1033
+ "baseline": {
1034
+ "nb_forwards": 97,
1035
+ "throughput": 6.47,
1036
+ "latency_mean": 154.67651497938144,
1037
+ "latency_std": 2.265059008007973,
1038
+ "latency_50": 154.530164,
1039
+ "latency_90": 158.0063524,
1040
+ "latency_95": 158.761275,
1041
+ "latency_99": 159.63890179999999,
1042
+ "latency_999": 161.83127588
1043
+ },
1044
+ "optimized": {
1045
+ "nb_forwards": 111,
1046
+ "throughput": 7.4,
1047
+ "latency_mean": 136.42054264864865,
1048
+ "latency_std": 13.669195772141308,
1049
+ "latency_50": 129.657713,
1050
+ "latency_90": 162.972988,
1051
+ "latency_95": 163.5093255,
1052
+ "latency_99": 164.2452398,
1053
+ "latency_999": 164.73538388999998
1054
+ }
1055
+ },
1056
+ {
1057
+ "batch_size": 8,
1058
+ "input_length": 64,
1059
+ "baseline": {
1060
+ "nb_forwards": 15,
1061
+ "throughput": 1.0,
1062
+ "latency_mean": 1012.3499144666667,
1063
+ "latency_std": 101.51911082567241,
1064
+ "latency_50": 971.620166,
1065
+ "latency_90": 1118.203026,
1066
+ "latency_95": 1211.1398262999996,
1067
+ "latency_99": 1307.9172236599998,
1068
+ "latency_999": 1329.6921380660003
1069
+ },
1070
+ "optimized": {
1071
+ "nb_forwards": 12,
1072
+ "throughput": 0.8,
1073
+ "latency_mean": 1317.46020725,
1074
+ "latency_std": 3.756500822804119,
1075
+ "latency_50": 1317.000587,
1076
+ "latency_90": 1322.3999772,
1077
+ "latency_95": 1323.5495317,
1078
+ "latency_99": 1324.5479823399999,
1079
+ "latency_999": 1324.772633734
1080
+ }
1081
+ },
1082
+ {
1083
+ "batch_size": 4,
1084
+ "input_length": 128,
1085
+ "baseline": {
1086
+ "nb_forwards": 23,
1087
+ "throughput": 1.53,
1088
+ "latency_mean": 655.0756931304347,
1089
+ "latency_std": 1.4599780685925838,
1090
+ "latency_50": 654.878779,
1091
+ "latency_90": 656.67802,
1092
+ "latency_95": 658.3078226,
1093
+ "latency_99": 658.6252101,
1094
+ "latency_999": 658.6617698099999
1095
+ },
1096
+ "optimized": {
1097
+ "nb_forwards": 31,
1098
+ "throughput": 2.07,
1099
+ "latency_mean": 499.40584877419354,
1100
+ "latency_std": 69.82345023327741,
1101
+ "latency_50": 469.970446,
1102
+ "latency_90": 659.315274,
1103
+ "latency_95": 661.5421805,
1104
+ "latency_99": 662.4330344,
1105
+ "latency_999": 662.63913944
1106
+ }
1107
+ },
1108
+ {
1109
+ "batch_size": 8,
1110
+ "input_length": 128,
1111
+ "baseline": {
1112
+ "nb_forwards": 13,
1113
+ "throughput": 0.87,
1114
+ "latency_mean": 1181.2107936923078,
1115
+ "latency_std": 146.80801899246256,
1116
+ "latency_50": 1260.661433,
1117
+ "latency_90": 1303.1271828,
1118
+ "latency_95": 1325.1133756,
1119
+ "latency_99": 1343.53688152,
1120
+ "latency_999": 1347.682170352
1121
+ },
1122
+ "optimized": {
1123
+ "nb_forwards": 17,
1124
+ "throughput": 1.13,
1125
+ "latency_mean": 933.8200267058824,
1126
+ "latency_std": 25.86898009147869,
1127
+ "latency_50": 927.867181,
1128
+ "latency_90": 954.8453836,
1129
+ "latency_95": 967.3439077999999,
1130
+ "latency_99": 1006.87954876,
1131
+ "latency_999": 1015.775067976
1132
+ }
1133
+ }
1134
+ ],
1135
+ "others": {
1136
+ "baseline": {
1137
+ "accuracy": 0.98
1138
+ },
1139
+ "optimized": {
1140
+ "accuracy": 0.98
1141
+ }
1142
+ }
1143
+ },
1144
+ "max_eval_samples": null,
1145
+ "time_benchmark_args": {
1146
+ "duration": 15,
1147
+ "warmup_runs": 5
1148
+ },
1149
+ "model_type": "vit"
1150
+ },
1151
+ {
1152
+ "model_name_or_path": "nateraw/vit-base-beans",
1153
+ "task": "image-classification",
1154
+ "dataset": {
1155
+ "path": "beans",
1156
+ "eval_split": "validation",
1157
+ "data_keys": {
1158
+ "primary": "image",
1159
+ "secondary": null
1160
+ },
1161
+ "ref_keys": [
1162
+ "labels"
1163
+ ],
1164
+ "name": null,
1165
+ "calibration_split": null
1166
+ },
1167
+ "quantization_approach": "dynamic",
1168
+ "operators_to_quantize": [
1169
+ "Add",
1170
+ "MatMul"
1171
+ ],
1172
+ "node_exclusion": [],
1173
+ "aware_training": false,
1174
+ "per_channel": true,
1175
+ "calibration": null,
1176
+ "framework": "onnxruntime",
1177
+ "framework_args": {
1178
+ "opset": 11,
1179
+ "optimization_level": 1
1180
+ },
1181
+ "hardware": "Architecture: x86_64\nCPU op-mode(s): 32-bit, 64-bit\nByte Order: Little Endian\nAddress sizes: 46 bits physical, 48 bits virtual\nCPU(s): 8\nOn-line CPU(s) list: 0-7\nThread(s) per core: 2\nCore(s) per socket: 4\nSocket(s): 1\nNUMA node(s): 1\nVendor ID: GenuineIntel\nCPU family: 6\nModel: 85\nModel name: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz\nStepping: 7\nCPU MHz: 3101.736\nBogoMIPS: 4999.99\nHypervisor vendor: KVM\nVirtualization type: full\nL1d cache: 128 KiB\nL1i cache: 128 KiB\nL2 cache: 4 MiB\nL3 cache: 35.8 MiB\nNUMA node0 CPU(s): 0-7\nVulnerability Itlb multihit: KVM: Vulnerable\nVulnerability L1tf: Mitigation; PTE Inversion\nVulnerability Mds: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown\nVulnerability Meltdown: Mitigation; PTI\nVulnerability Spec store bypass: Vulnerable\nVulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization\nVulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling\nVulnerability Srbds: Not affected\nVulnerability Tsx async abort: Not affected\nFlags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke avx512_vnni\n",
1182
+ "versions": {
1183
+ "transformers": "4.20.1",
1184
+ "optimum": "1.2.3.dev0",
1185
+ "optimum_hash": "5ac9c0d9fd7e7cca55b2f9935b961ed5b6c50112"
1186
+ },
1187
+ "evaluation": {
1188
+ "time": [
1189
+ {
1190
+ "batch_size": 1,
1191
+ "input_length": 64,
1192
+ "baseline": {
1193
+ "nb_forwards": 93,
1194
+ "throughput": 6.2,
1195
+ "latency_mean": 161.41171223655914,
1196
+ "latency_std": 14.870354336577428,
1197
+ "latency_50": 155.939994,
1198
+ "latency_90": 197.53404419999998,
1199
+ "latency_95": 197.912589,
1200
+ "latency_99": 198.83022556,
1201
+ "latency_999": 199.676923456
1202
+ },
1203
+ "optimized": {
1204
+ "nb_forwards": 207,
1205
+ "throughput": 13.8,
1206
+ "latency_mean": 72.70885894202898,
1207
+ "latency_std": 0.27964445519343206,
1208
+ "latency_50": 72.716042,
1209
+ "latency_90": 73.0758788,
1210
+ "latency_95": 73.17664590000001,
1211
+ "latency_99": 73.3802671,
1212
+ "latency_999": 73.51221445200001
1213
+ }
1214
+ },
1215
+ {
1216
+ "batch_size": 1,
1217
+ "input_length": 128,
1218
+ "baseline": {
1219
+ "nb_forwards": 76,
1220
+ "throughput": 5.07,
1221
+ "latency_mean": 199.65074372368423,
1222
+ "latency_std": 1.5160773445469795,
1223
+ "latency_50": 199.944024,
1224
+ "latency_90": 201.29589,
1225
+ "latency_95": 201.80196475,
1226
+ "latency_99": 202.12260975,
1227
+ "latency_999": 202.166467875
1228
+ },
1229
+ "optimized": {
1230
+ "nb_forwards": 209,
1231
+ "throughput": 13.93,
1232
+ "latency_mean": 71.78032969856459,
1233
+ "latency_std": 0.3993047349686967,
1234
+ "latency_50": 71.737218,
1235
+ "latency_90": 72.21902340000001,
1236
+ "latency_95": 72.5808502,
1237
+ "latency_99": 73.12714656,
1238
+ "latency_999": 73.66801959200001
1239
+ }
1240
+ },
1241
+ {
1242
+ "batch_size": 4,
1243
+ "input_length": 64,
1244
+ "baseline": {
1245
+ "nb_forwards": 25,
1246
+ "throughput": 1.67,
1247
+ "latency_mean": 617.44332692,
1248
+ "latency_std": 67.01575583161043,
1249
+ "latency_50": 660.252227,
1250
+ "latency_90": 663.977544,
1251
+ "latency_95": 666.9361547999999,
1252
+ "latency_99": 668.6067565599999,
1253
+ "latency_999": 668.8766228559999
1254
+ },
1255
+ "optimized": {
1256
+ "nb_forwards": 65,
1257
+ "throughput": 4.33,
1258
+ "latency_mean": 234.05122506153845,
1259
+ "latency_std": 19.589519412086375,
1260
+ "latency_50": 223.845147,
1261
+ "latency_90": 265.7009672,
1262
+ "latency_95": 266.8488142,
1263
+ "latency_99": 268.70655439999996,
1264
+ "latency_999": 269.60366863999997
1265
+ }
1266
+ },
1267
+ {
1268
+ "batch_size": 4,
1269
+ "input_length": 32,
1270
+ "baseline": {
1271
+ "nb_forwards": 23,
1272
+ "throughput": 1.53,
1273
+ "latency_mean": 661.2726233478261,
1274
+ "latency_std": 1.608488914998379,
1275
+ "latency_50": 661.364855,
1276
+ "latency_90": 663.035304,
1277
+ "latency_95": 663.3135758,
1278
+ "latency_99": 665.07419858,
1279
+ "latency_999": 665.515344758
1280
+ },
1281
+ "optimized": {
1282
+ "nb_forwards": 68,
1283
+ "throughput": 4.53,
1284
+ "latency_mean": 221.16239911764706,
1285
+ "latency_std": 5.801385463356238,
1286
+ "latency_50": 220.699395,
1287
+ "latency_90": 224.3565545,
1288
+ "latency_95": 225.22985880000002,
1289
+ "latency_99": 238.96943696999995,
1290
+ "latency_999": 260.98804769700024
1291
+ }
1292
+ },
1293
+ {
1294
+ "batch_size": 8,
1295
+ "input_length": 32,
1296
+ "baseline": {
1297
+ "nb_forwards": 12,
1298
+ "throughput": 0.8,
1299
+ "latency_mean": 1255.69552375,
1300
+ "latency_std": 102.9288237877332,
1301
+ "latency_50": 1286.0972765,
1302
+ "latency_90": 1316.4811832,
1303
+ "latency_95": 1321.2916282,
1304
+ "latency_99": 1324.5891536400002,
1305
+ "latency_999": 1325.331096864
1306
+ },
1307
+ "optimized": {
1308
+ "nb_forwards": 31,
1309
+ "throughput": 2.07,
1310
+ "latency_mean": 489.54871912903224,
1311
+ "latency_std": 47.715982761163175,
1312
+ "latency_50": 473.097218,
1313
+ "latency_90": 543.395976,
1314
+ "latency_95": 545.5166235,
1315
+ "latency_99": 551.4498183,
1316
+ "latency_999": 553.21678713
1317
+ }
1318
+ },
1319
+ {
1320
+ "batch_size": 4,
1321
+ "input_length": 128,
1322
+ "baseline": {
1323
+ "nb_forwards": 23,
1324
+ "throughput": 1.53,
1325
+ "latency_mean": 663.4977195217391,
1326
+ "latency_std": 1.2918393132646473,
1327
+ "latency_50": 663.486377,
1328
+ "latency_90": 665.3720026,
1329
+ "latency_95": 665.7250922000001,
1330
+ "latency_99": 666.08418616,
1331
+ "latency_999": 666.167132716
1332
+ },
1333
+ "optimized": {
1334
+ "nb_forwards": 68,
1335
+ "throughput": 4.53,
1336
+ "latency_mean": 222.37462554411766,
1337
+ "latency_std": 2.8550137685990054,
1338
+ "latency_50": 222.140647,
1339
+ "latency_90": 226.3398684,
1340
+ "latency_95": 227.585471,
1341
+ "latency_99": 228.76303193,
1342
+ "latency_999": 229.374908693
1343
+ }
1344
+ },
1345
+ {
1346
+ "batch_size": 8,
1347
+ "input_length": 64,
1348
+ "baseline": {
1349
+ "nb_forwards": 12,
1350
+ "throughput": 0.8,
1351
+ "latency_mean": 1296.6752090833334,
1352
+ "latency_std": 18.972208965807603,
1353
+ "latency_50": 1289.740581,
1354
+ "latency_90": 1318.5068230999998,
1355
+ "latency_95": 1326.9780795,
1356
+ "latency_99": 1334.9525383,
1357
+ "latency_999": 1336.7467915299999
1358
+ },
1359
+ "optimized": {
1360
+ "nb_forwards": 34,
1361
+ "throughput": 2.27,
1362
+ "latency_mean": 450.402188617647,
1363
+ "latency_std": 15.121871215842573,
1364
+ "latency_50": 446.2523015,
1365
+ "latency_90": 476.4106683,
1366
+ "latency_95": 481.48467215,
1367
+ "latency_99": 487.75100149,
1368
+ "latency_999": 489.587782249
1369
+ }
1370
+ },
1371
+ {
1372
+ "batch_size": 8,
1373
+ "input_length": 128,
1374
+ "baseline": {
1375
+ "nb_forwards": 12,
1376
+ "throughput": 0.8,
1377
+ "latency_mean": 1289.5001165833332,
1378
+ "latency_std": 17.973529354626326,
1379
+ "latency_50": 1282.8982245,
1380
+ "latency_90": 1316.4851546,
1381
+ "latency_95": 1319.94446375,
1382
+ "latency_99": 1321.75708555,
1383
+ "latency_999": 1322.164925455
1384
+ },
1385
+ "optimized": {
1386
+ "nb_forwards": 34,
1387
+ "throughput": 2.27,
1388
+ "latency_mean": 443.4559223235294,
1389
+ "latency_std": 5.080053264660657,
1390
+ "latency_50": 443.1060375,
1391
+ "latency_90": 450.62357180000004,
1392
+ "latency_95": 451.23264075,
1393
+ "latency_99": 455.59029821,
1394
+ "latency_999": 457.427172821
1395
+ }
1396
+ },
1397
+ {
1398
+ "batch_size": 1,
1399
+ "input_length": 32,
1400
+ "baseline": {
1401
+ "nb_forwards": 74,
1402
+ "throughput": 4.93,
1403
+ "latency_mean": 203.5212293918919,
1404
+ "latency_std": 2.0551978472652848,
1405
+ "latency_50": 203.3370845,
1406
+ "latency_90": 205.9722848,
1407
+ "latency_95": 206.7128496,
1408
+ "latency_99": 208.65631527,
1409
+ "latency_999": 210.207952827
1410
+ },
1411
+ "optimized": {
1412
+ "nb_forwards": 207,
1413
+ "throughput": 13.8,
1414
+ "latency_mean": 72.47834015458938,
1415
+ "latency_std": 0.31101237142899824,
1416
+ "latency_50": 72.463726,
1417
+ "latency_90": 72.8683692,
1418
+ "latency_95": 72.9479522,
1419
+ "latency_99": 73.2749897,
1420
+ "latency_999": 73.51287976
1421
+ }
1422
+ }
1423
+ ],
1424
+ "others": {
1425
+ "baseline": {
1426
+ "accuracy": 0.98
1427
+ },
1428
+ "optimized": {
1429
+ "accuracy": 0.98
1430
+ }
1431
+ }
1432
+ },
1433
+ "max_eval_samples": null,
1434
+ "time_benchmark_args": {
1435
+ "duration": 15,
1436
+ "warmup_runs": 5
1437
+ },
1438
+ "model_type": "vit"
1439
+ },
1440
+ {
1441
+ "model_name_or_path": "nateraw/vit-base-beans",
1442
+ "task": "image-classification",
1443
+ "dataset": {
1444
+ "path": "beans",
1445
+ "eval_split": "validation",
1446
+ "data_keys": {
1447
+ "primary": "image",
1448
+ "secondary": null
1449
+ },
1450
+ "ref_keys": [
1451
+ "labels"
1452
+ ],
1453
+ "name": null,
1454
+ "calibration_split": null
1455
+ },
1456
+ "quantization_approach": "dynamic",
1457
+ "operators_to_quantize": [
1458
+ "Add"
1459
+ ],
1460
+ "node_exclusion": [],
1461
+ "aware_training": false,
1462
+ "per_channel": false,
1463
+ "calibration": null,
1464
+ "framework": "onnxruntime",
1465
+ "framework_args": {
1466
+ "opset": 11,
1467
+ "optimization_level": 1
1468
+ },
1469
+ "hardware": "Architecture: x86_64\nCPU op-mode(s): 32-bit, 64-bit\nByte Order: Little Endian\nAddress sizes: 46 bits physical, 48 bits virtual\nCPU(s): 8\nOn-line CPU(s) list: 0-7\nThread(s) per core: 2\nCore(s) per socket: 4\nSocket(s): 1\nNUMA node(s): 1\nVendor ID: GenuineIntel\nCPU family: 6\nModel: 85\nModel name: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz\nStepping: 7\nCPU MHz: 3101.587\nBogoMIPS: 4999.99\nHypervisor vendor: KVM\nVirtualization type: full\nL1d cache: 128 KiB\nL1i cache: 128 KiB\nL2 cache: 4 MiB\nL3 cache: 35.8 MiB\nNUMA node0 CPU(s): 0-7\nVulnerability Itlb multihit: KVM: Vulnerable\nVulnerability L1tf: Mitigation; PTE Inversion\nVulnerability Mds: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown\nVulnerability Meltdown: Mitigation; PTI\nVulnerability Spec store bypass: Vulnerable\nVulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization\nVulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling\nVulnerability Srbds: Not affected\nVulnerability Tsx async abort: Not affected\nFlags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke avx512_vnni\n",
1470
+ "versions": {
1471
+ "transformers": "4.20.1",
1472
+ "optimum": "1.2.3.dev0",
1473
+ "optimum_hash": "5ac9c0d9fd7e7cca55b2f9935b961ed5b6c50112"
1474
+ },
1475
+ "evaluation": {
1476
+ "time": [
1477
+ {
1478
+ "batch_size": 4,
1479
+ "input_length": 64,
1480
+ "baseline": {
1481
+ "nb_forwards": 23,
1482
+ "throughput": 1.53,
1483
+ "latency_mean": 661.5120061304348,
1484
+ "latency_std": 2.106123083282921,
1485
+ "latency_50": 661.334389,
1486
+ "latency_90": 663.6329132000001,
1487
+ "latency_95": 664.633404,
1488
+ "latency_99": 666.27847948,
1489
+ "latency_999": 666.671275048
1490
+ },
1491
+ "optimized": {
1492
+ "nb_forwards": 32,
1493
+ "throughput": 2.13,
1494
+ "latency_mean": 478.80772409375,
1495
+ "latency_std": 9.072497172419535,
1496
+ "latency_50": 478.339513,
1497
+ "latency_90": 490.9494227,
1498
+ "latency_95": 491.7442059,
1499
+ "latency_99": 499.86555171000003,
1500
+ "latency_999": 502.846627371
1501
+ }
1502
+ },
1503
+ {
1504
+ "batch_size": 4,
1505
+ "input_length": 32,
1506
+ "baseline": {
1507
+ "nb_forwards": 23,
1508
+ "throughput": 1.53,
1509
+ "latency_mean": 662.8350984782609,
1510
+ "latency_std": 2.1823429520784754,
1511
+ "latency_50": 662.268507,
1512
+ "latency_90": 666.2232022000001,
1513
+ "latency_95": 666.476752,
1514
+ "latency_99": 666.62039362,
1515
+ "latency_999": 666.6544751619999
1516
+ },
1517
+ "optimized": {
1518
+ "nb_forwards": 29,
1519
+ "throughput": 1.93,
1520
+ "latency_mean": 529.2792873448276,
1521
+ "latency_std": 87.518562392806,
1522
+ "latency_50": 475.50113,
1523
+ "latency_90": 660.0331162,
1524
+ "latency_95": 661.2807788,
1525
+ "latency_99": 661.9307412,
1526
+ "latency_999": 662.12196762
1527
+ }
1528
+ },
1529
+ {
1530
+ "batch_size": 1,
1531
+ "input_length": 64,
1532
+ "baseline": {
1533
+ "nb_forwards": 75,
1534
+ "throughput": 5.0,
1535
+ "latency_mean": 200.44544238666666,
1536
+ "latency_std": 0.9950016888231685,
1537
+ "latency_50": 200.190004,
1538
+ "latency_90": 201.82102559999998,
1539
+ "latency_95": 202.5074946,
1540
+ "latency_99": 203.1678269,
1541
+ "latency_999": 203.50243529
1542
+ },
1543
+ "optimized": {
1544
+ "nb_forwards": 116,
1545
+ "throughput": 7.73,
1546
+ "latency_mean": 129.39618714655174,
1547
+ "latency_std": 2.067964919431859,
1548
+ "latency_50": 129.4016065,
1549
+ "latency_90": 131.648931,
1550
+ "latency_95": 132.2263295,
1551
+ "latency_99": 136.81370259999997,
1552
+ "latency_999": 138.54515077000002
1553
+ }
1554
+ },
1555
+ {
1556
+ "batch_size": 8,
1557
+ "input_length": 128,
1558
+ "baseline": {
1559
+ "nb_forwards": 12,
1560
+ "throughput": 0.8,
1561
+ "latency_mean": 1294.6512800833332,
1562
+ "latency_std": 16.84406616166231,
1563
+ "latency_50": 1290.4992525,
1564
+ "latency_90": 1317.8022775999998,
1565
+ "latency_95": 1319.4061700999998,
1566
+ "latency_99": 1320.8464036199998,
1567
+ "latency_999": 1321.170456162
1568
+ },
1569
+ "optimized": {
1570
+ "nb_forwards": 17,
1571
+ "throughput": 1.13,
1572
+ "latency_mean": 930.9679744117648,
1573
+ "latency_std": 16.6232043529702,
1574
+ "latency_50": 932.665064,
1575
+ "latency_90": 951.8628876,
1576
+ "latency_95": 955.4653676,
1577
+ "latency_99": 962.11386552,
1578
+ "latency_999": 963.6097775520001
1579
+ }
1580
+ },
1581
+ {
1582
+ "batch_size": 8,
1583
+ "input_length": 64,
1584
+ "baseline": {
1585
+ "nb_forwards": 16,
1586
+ "throughput": 1.07,
1587
+ "latency_mean": 968.208967375,
1588
+ "latency_std": 21.2604505171286,
1589
+ "latency_50": 965.257975,
1590
+ "latency_90": 992.8459695,
1591
+ "latency_95": 999.56778275,
1592
+ "latency_99": 1010.44061495,
1593
+ "latency_999": 1012.8870021949999
1594
+ },
1595
+ "optimized": {
1596
+ "nb_forwards": 16,
1597
+ "throughput": 1.07,
1598
+ "latency_mean": 949.805699,
1599
+ "latency_std": 35.16724690173305,
1600
+ "latency_50": 939.915342,
1601
+ "latency_90": 1000.7328375,
1602
+ "latency_95": 1014.5590895,
1603
+ "latency_99": 1031.9948771,
1604
+ "latency_999": 1035.9179293099999
1605
+ }
1606
+ },
1607
+ {
1608
+ "batch_size": 8,
1609
+ "input_length": 32,
1610
+ "baseline": {
1611
+ "nb_forwards": 12,
1612
+ "throughput": 0.8,
1613
+ "latency_mean": 1301.3549284166668,
1614
+ "latency_std": 32.02669612806352,
1615
+ "latency_50": 1287.5151945,
1616
+ "latency_90": 1354.4028801,
1617
+ "latency_95": 1358.3970296,
1618
+ "latency_99": 1361.5798787200001,
1619
+ "latency_999": 1362.2960197720001
1620
+ },
1621
+ "optimized": {
1622
+ "nb_forwards": 16,
1623
+ "throughput": 1.07,
1624
+ "latency_mean": 938.1431063125,
1625
+ "latency_std": 10.912116938304163,
1626
+ "latency_50": 935.909206,
1627
+ "latency_90": 950.346758,
1628
+ "latency_95": 953.7962335,
1629
+ "latency_99": 959.9162083,
1630
+ "latency_999": 961.29320263
1631
+ }
1632
+ },
1633
+ {
1634
+ "batch_size": 4,
1635
+ "input_length": 128,
1636
+ "baseline": {
1637
+ "nb_forwards": 24,
1638
+ "throughput": 1.6,
1639
+ "latency_mean": 625.5643148333334,
1640
+ "latency_std": 64.81901502308723,
1641
+ "latency_50": 664.440001,
1642
+ "latency_90": 666.4835666,
1643
+ "latency_95": 666.6322609,
1644
+ "latency_99": 667.7429191699999,
1645
+ "latency_999": 668.034834917
1646
+ },
1647
+ "optimized": {
1648
+ "nb_forwards": 29,
1649
+ "throughput": 1.93,
1650
+ "latency_mean": 529.020664137931,
1651
+ "latency_std": 84.62326588659242,
1652
+ "latency_50": 480.707711,
1653
+ "latency_90": 665.4676168,
1654
+ "latency_95": 665.8970696,
1655
+ "latency_99": 666.84264784,
1656
+ "latency_999": 667.137783184
1657
+ }
1658
+ },
1659
+ {
1660
+ "batch_size": 1,
1661
+ "input_length": 32,
1662
+ "baseline": {
1663
+ "nb_forwards": 91,
1664
+ "throughput": 6.07,
1665
+ "latency_mean": 166.03000134065934,
1666
+ "latency_std": 19.132517193102455,
1667
+ "latency_50": 157.036032,
1668
+ "latency_90": 202.654865,
1669
+ "latency_95": 203.5852275,
1670
+ "latency_99": 204.973207,
1671
+ "latency_999": 205.58968180000002
1672
+ },
1673
+ "optimized": {
1674
+ "nb_forwards": 100,
1675
+ "throughput": 6.67,
1676
+ "latency_mean": 150.93316497,
1677
+ "latency_std": 15.730253522641698,
1678
+ "latency_50": 162.552646,
1679
+ "latency_90": 163.3473,
1680
+ "latency_95": 163.7605152,
1681
+ "latency_99": 165.11680815,
1682
+ "latency_999": 165.673785615
1683
+ }
1684
+ },
1685
+ {
1686
+ "batch_size": 1,
1687
+ "input_length": 128,
1688
+ "baseline": {
1689
+ "nb_forwards": 76,
1690
+ "throughput": 5.07,
1691
+ "latency_mean": 199.0767317368421,
1692
+ "latency_std": 0.6088183471893983,
1693
+ "latency_50": 198.975675,
1694
+ "latency_90": 199.7272365,
1695
+ "latency_95": 200.26232875,
1696
+ "latency_99": 200.9589925,
1697
+ "latency_999": 201.30218545
1698
+ },
1699
+ "optimized": {
1700
+ "nb_forwards": 109,
1701
+ "throughput": 7.27,
1702
+ "latency_mean": 137.9706248715596,
1703
+ "latency_std": 15.205285051787508,
1704
+ "latency_50": 129.352799,
1705
+ "latency_90": 162.6187408,
1706
+ "latency_95": 162.7682896,
1707
+ "latency_99": 162.88476124000002,
1708
+ "latency_999": 162.94030542800002
1709
+ }
1710
+ }
1711
+ ],
1712
+ "others": {
1713
+ "baseline": {
1714
+ "accuracy": 0.98
1715
+ },
1716
+ "optimized": {
1717
+ "accuracy": 0.98
1718
+ }
1719
+ }
1720
+ },
1721
+ "max_eval_samples": null,
1722
+ "time_benchmark_args": {
1723
+ "duration": 15,
1724
+ "warmup_runs": 5
1725
+ },
1726
+ "model_type": "vit"
1727
+ }
1728
+ ]
tensorboard/1657613137.713749/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c60208dfb0f65261910cfe99be8285d77deaa8204ce20104f03a2684478a848a
3
+ size 717
tensorboard/1657613137.7153692/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:125e35f10746ffd2aa47f7ea87f02826a7182b78a9f56a3199f2d5d6ca0c7a26
3
+ size 716
tensorboard/1657613137.7166162/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:69d188a9fbf1f849b01f032e672cf5f6eb719e02edf75b321084719c85aa7f3e
3
+ size 707
tensorboard/1657613137.7177505/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c88aad85e58fda751a35106a501f7dd86be9968188d39cc616c59296d42f90d
3
+ size 706
tensorboard/1657613137.7188103/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1a4cc9bd0baddc21971282ad821bca10fca4fe351cc3b7551344dc961147df5
3
+ size 703
tensorboard/1657613137.7203975/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.6 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a89c88942d70064790a326a7ac45441d2b42e0533f71dae88b442f9f62a2b147
3
+ size 702
tensorboard/events.out.tfevents.1657613137.ip-10-0-107-255.ec2.internal.1.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d5001adbabe10e29288d08640d258997a3ebe34f2509e74831d27039bd7a08d3
3
+ size 40