Hugging Face's logo

vuiseng9
/
ov-mpt-7b-gsm8k-sparse70

Text Generation
OpenVINO
OpenVINO
mpt
sparse
quantization
Model card Files Community
vuiseng9 commited on
Commit
c04ef54
•
1 Parent(s): c83e8ea

Add content

Browse files
Files changed (29) hide show
  1. .gitattributes +1 -0
  2. README.md +44 -0
  3. benchmarkapp_w8a8.bash +1 -0
  4. benchmarkapp_w8a8_sparse70.bash +1 -0
  5. models/mpt-7b-gsm8k-pt/w8a8-sparse70/args.json +7 -0
  6. models/mpt-7b-gsm8k-pt/w8a8-sparse70/compressed_graph.dot +0 -0
  7. models/mpt-7b-gsm8k-pt/w8a8-sparse70/config.json +48 -0
  8. models/mpt-7b-gsm8k-pt/w8a8-sparse70/log.log +0 -0
  9. models/mpt-7b-gsm8k-pt/w8a8-sparse70/openvino_config.json +50 -0
  10. models/mpt-7b-gsm8k-pt/w8a8-sparse70/openvino_model.bin +3 -0
  11. models/mpt-7b-gsm8k-pt/w8a8-sparse70/openvino_model.xml +0 -0
  12. models/mpt-7b-gsm8k-pt/w8a8-sparse70/original_graph.dot +0 -0
  13. models/mpt-7b-gsm8k-pt/w8a8-sparse70/ov_weights_type.md +328 -0
  14. models/mpt-7b-gsm8k-pt/w8a8-sparse70/special_tokens_map.json +6 -0
  15. models/mpt-7b-gsm8k-pt/w8a8-sparse70/tokenizer.json +0 -0
  16. models/mpt-7b-gsm8k-pt/w8a8-sparse70/tokenizer_config.json +212 -0
  17. models/mpt-7b-gsm8k-pt/w8a8/args.json +6 -0
  18. models/mpt-7b-gsm8k-pt/w8a8/compressed_graph.dot +0 -0
  19. models/mpt-7b-gsm8k-pt/w8a8/config.json +48 -0
  20. models/mpt-7b-gsm8k-pt/w8a8/log.log +903 -0
  21. models/mpt-7b-gsm8k-pt/w8a8/openvino_config.json +50 -0
  22. models/mpt-7b-gsm8k-pt/w8a8/openvino_model.bin +3 -0
  23. models/mpt-7b-gsm8k-pt/w8a8/openvino_model.xml +0 -0
  24. models/mpt-7b-gsm8k-pt/w8a8/original_graph.dot +0 -0
  25. models/mpt-7b-gsm8k-pt/w8a8/ov_weights_type.md +328 -0
  26. models/mpt-7b-gsm8k-pt/w8a8/special_tokens_map.json +6 -0
  27. models/mpt-7b-gsm8k-pt/w8a8/tokenizer.json +0 -0
  28. models/mpt-7b-gsm8k-pt/w8a8/tokenizer_config.json +212 -0
  29. tld0.6.json +5 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ *.bin.part* filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ tags:
4
+ - openvino
5
+ - mpt
6
+ - sparse
7
+ - quantization
8
+ library_name: "OpenVINO"
9
+ ---
10
+
11
+ The intent of this repo is to compare the performance delta between dense quantized MPT-7B and 70% sparse-quantized MPT-7B on OpenVINO. Quantization here is 8-bit on both weight and activation. Benchmarking here is decoding (next token) latency with context length 512.
12
+
13
+ Target HW: Intel 4th gen Xeon (Sapphire Rapids)
14
+
15
+ SW
16
+ ```
17
+ git clone https://huggingface.co/vuiseng9/ov-mpt-7b-gsm8k-sparse70
18
+ pip install openvino==2024.2.0
19
+ ```
20
+
21
+ ## Benchmarking with OpenVINO
22
+
23
+ 1. ./benchmarkapp_w8a8.bash
24
+ 2. ./benchmarkapp_w8a8_sparse70.bash
25
+
26
+ Note: do remove the numactl if your node does not support it.
27
+
28
+
29
+ ## Implementation of Sparse Weight Decompression in OpenVINO
30
+ * This is the first commit of Sparse Weight Decompression on OpenVINO’s fork of oneDNN.
31
+ https://github.com/openvinotoolkit/oneDNN/pull/158/files
32
+
33
+ * you can browse this via the left pane.
34
+
35
+ * initialization: src/cpu/reorder/simple_sparse_reorder.hpp ([line 113](https://github.com/openvinotoolkit/oneDNN/pull/158/files#diff-f1445f832cd9979d9756873e3d8c30716976f51b6ce4640eae12762a417284e3R113))
36
+
37
+ * decompression: src/cpu/x64/jit_brgemm_decompress_kernel.cpp ([line 41](https://github.com/openvinotoolkit/oneDNN/pull/158/files#diff-98844e424b6687de78d47737e62f206dc9befcec6887dac8b2c52d0303dd3576R41))
38
+
39
+
40
+ * If you'd like to build OpenVINO runtime from source for debug, [see wiki page](https://github.com/openvinotoolkit/openvino/blob/master/docs/dev/build.md). Benchmark_app is compiled as well.
41
+
42
+ ## Related materials:
43
+ [OpenVINO blog on Sparse-Quantized BERT](https://blog.openvino.ai/blog-posts/accelerate-inference-of-sparse-transformer-models-with-openvino-tm-and-4th-gen-intel-r-xe[]on-r-scalable-processors) ([corresponding notebook](https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/116-sparsity-optimization/116-sparsity-optimization.ipynb))
44
+
benchmarkapp_w8a8.bash ADDED
@@ -0,0 +1 @@
 
 
1
+ numactl --cpunodebind 0 --membind 0 benchmark_app -m ./models/mpt-7b-gsm8k-pt/w8a8/openvino_model.xml -data_shape input_ids[1,1],attention_mask[1,512],past_key_values.0.value[1,32,511,128],past_key_values.0.key[1,32,511,128],past_key_values.1.value[1,32,511,128],past_key_values.1.key[1,32,511,128],past_key_values.2.value[1,32,511,128],past_key_values.2.key[1,32,511,128],past_key_values.3.value[1,32,511,128],past_key_values.3.key[1,32,511,128],past_key_values.4.value[1,32,511,128],past_key_values.4.key[1,32,511,128],past_key_values.5.value[1,32,511,128],past_key_values.5.key[1,32,511,128],past_key_values.6.value[1,32,511,128],past_key_values.6.key[1,32,511,128],past_key_values.7.value[1,32,511,128],past_key_values.7.key[1,32,511,128],past_key_values.8.value[1,32,511,128],past_key_values.8.key[1,32,511,128],past_key_values.9.value[1,32,511,128],past_key_values.9.key[1,32,511,128],past_key_values.10.value[1,32,511,128],past_key_values.10.key[1,32,511,128],past_key_values.11.value[1,32,511,128],past_key_values.11.key[1,32,511,128],past_key_values.12.value[1,32,511,128],past_key_values.12.key[1,32,511,128],past_key_values.13.value[1,32,511,128],past_key_values.13.key[1,32,511,128],past_key_values.14.value[1,32,511,128],past_key_values.14.key[1,32,511,128],past_key_values.15.value[1,32,511,128],past_key_values.15.key[1,32,511,128],past_key_values.16.value[1,32,511,128],past_key_values.16.key[1,32,511,128],past_key_values.17.value[1,32,511,128],past_key_values.17.key[1,32,511,128],past_key_values.18.value[1,32,511,128],past_key_values.18.key[1,32,511,128],past_key_values.19.value[1,32,511,128],past_key_values.19.key[1,32,511,128],past_key_values.20.value[1,32,511,128],past_key_values.20.key[1,32,511,128],past_key_values.21.value[1,32,511,128],past_key_values.21.key[1,32,511,128],past_key_values.22.value[1,32,511,128],past_key_values.22.key[1,32,511,128],past_key_values.23.value[1,32,511,128],past_key_values.23.key[1,32,511,128],past_key_values.24.value[1,32,511,128],past_key_values.24.key[1,32,511,128],past_key_values.25.value[1,32,511,128],past_key_values.25.key[1,32,511,128],past_key_values.26.value[1,32,511,128],past_key_values.26.key[1,32,511,128],past_key_values.27.value[1,32,511,128],past_key_values.27.key[1,32,511,128],past_key_values.28.value[1,32,511,128],past_key_values.28.key[1,32,511,128],past_key_values.29.value[1,32,511,128],past_key_values.29.key[1,32,511,128],past_key_values.30.value[1,32,511,128],past_key_values.30.key[1,32,511,128],past_key_values.31.value[1,32,511,128],past_key_values.31.key[1,32,511,128] -t 120 -hint latency -infer_precision f32
benchmarkapp_w8a8_sparse70.bash ADDED
@@ -0,0 +1 @@
 
 
1
+ numactl --cpunodebind 0 --membind 0 benchmark_app -m ./models/mpt-7b-gsm8k-pt/w8a8-sparse70/openvino_model.xml -data_shape input_ids[1,1],attention_mask[1,512],past_key_values.0.value[1,32,511,128],past_key_values.0.key[1,32,511,128],past_key_values.1.value[1,32,511,128],past_key_values.1.key[1,32,511,128],past_key_values.2.value[1,32,511,128],past_key_values.2.key[1,32,511,128],past_key_values.3.value[1,32,511,128],past_key_values.3.key[1,32,511,128],past_key_values.4.value[1,32,511,128],past_key_values.4.key[1,32,511,128],past_key_values.5.value[1,32,511,128],past_key_values.5.key[1,32,511,128],past_key_values.6.value[1,32,511,128],past_key_values.6.key[1,32,511,128],past_key_values.7.value[1,32,511,128],past_key_values.7.key[1,32,511,128],past_key_values.8.value[1,32,511,128],past_key_values.8.key[1,32,511,128],past_key_values.9.value[1,32,511,128],past_key_values.9.key[1,32,511,128],past_key_values.10.value[1,32,511,128],past_key_values.10.key[1,32,511,128],past_key_values.11.value[1,32,511,128],past_key_values.11.key[1,32,511,128],past_key_values.12.value[1,32,511,128],past_key_values.12.key[1,32,511,128],past_key_values.13.value[1,32,511,128],past_key_values.13.key[1,32,511,128],past_key_values.14.value[1,32,511,128],past_key_values.14.key[1,32,511,128],past_key_values.15.value[1,32,511,128],past_key_values.15.key[1,32,511,128],past_key_values.16.value[1,32,511,128],past_key_values.16.key[1,32,511,128],past_key_values.17.value[1,32,511,128],past_key_values.17.key[1,32,511,128],past_key_values.18.value[1,32,511,128],past_key_values.18.key[1,32,511,128],past_key_values.19.value[1,32,511,128],past_key_values.19.key[1,32,511,128],past_key_values.20.value[1,32,511,128],past_key_values.20.key[1,32,511,128],past_key_values.21.value[1,32,511,128],past_key_values.21.key[1,32,511,128],past_key_values.22.value[1,32,511,128],past_key_values.22.key[1,32,511,128],past_key_values.23.value[1,32,511,128],past_key_values.23.key[1,32,511,128],past_key_values.24.value[1,32,511,128],past_key_values.24.key[1,32,511,128],past_key_values.25.value[1,32,511,128],past_key_values.25.key[1,32,511,128],past_key_values.26.value[1,32,511,128],past_key_values.26.key[1,32,511,128],past_key_values.27.value[1,32,511,128],past_key_values.27.key[1,32,511,128],past_key_values.28.value[1,32,511,128],past_key_values.28.key[1,32,511,128],past_key_values.29.value[1,32,511,128],past_key_values.29.key[1,32,511,128],past_key_values.30.value[1,32,511,128],past_key_values.30.key[1,32,511,128],past_key_values.31.value[1,32,511,128],past_key_values.31.key[1,32,511,128] -t 120 -hint latency -load_config ./tld0.6.json -infer_precision f32
models/mpt-7b-gsm8k-pt/w8a8-sparse70/args.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_id": "neuralmagic/mpt-7b-gsm8k-pt",
3
+ "run_name": "w8a8-sparse70",
4
+ "quant_mode": "W8A8",
5
+ "ref_sparse_onnx": "neuralmagic/mpt-7b-gsm8k-pruned70-quant-ds",
6
+ "force_run": true
7
+ }
models/mpt-7b-gsm8k-pt/w8a8-sparse70/compressed_graph.dot ADDED
The diff for this file is too large to render. See raw diff
 
models/mpt-7b-gsm8k-pt/w8a8-sparse70/config.json ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "neuralmagic/mpt-7b-gsm8k-pt",
3
+ "architectures": [
4
+ "MPTForCausalLM"
5
+ ],
6
+ "attn_config": {
7
+ "model_type": ""
8
+ },
9
+ "auto_map": {
10
+ "AutoConfig": "neuralmagic/mpt-7b-gsm8k-pt--configuration_mpt.MPTConfig",
11
+ "AutoModelForCausalLM": "neuralmagic/mpt-7b-gsm8k-pt--modeling_mpt.MPTForCausalLM"
12
+ },
13
+ "d_model": 4096,
14
+ "emb_pdrop": 0,
15
+ "embedding_fraction": 1.0,
16
+ "expansion_ratio": 4,
17
+ "init_config": {
18
+ "emb_init_std": null,
19
+ "emb_init_uniform_lim": null,
20
+ "fan_mode": "fan_in",
21
+ "init_div_is_residual": true,
22
+ "init_gain": 0,
23
+ "init_nonlinearity": "relu",
24
+ "init_std": 0.02,
25
+ "name": "kaiming_normal_",
26
+ "verbose": 0
27
+ },
28
+ "init_device": "cpu",
29
+ "initializer_range": 0.02,
30
+ "layer_norm_epsilon": 1e-05,
31
+ "learned_pos_emb": true,
32
+ "logit_scale": null,
33
+ "max_seq_len": 2048,
34
+ "model_type": "mpt",
35
+ "n_heads": 32,
36
+ "n_layers": 32,
37
+ "no_bias": true,
38
+ "norm_type": "low_precision_layernorm",
39
+ "resid_pdrop": 0,
40
+ "tie_weights": false,
41
+ "tie_word_embeddings": false,
42
+ "tokenizer_name": "EleutherAI/gpt-neox-20b",
43
+ "torch_dtype": "bfloat16",
44
+ "transformers_version": "4.34.1",
45
+ "use_cache": true,
46
+ "verbose": 0,
47
+ "vocab_size": 50432
48
+ }
models/mpt-7b-gsm8k-pt/w8a8-sparse70/log.log ADDED
The diff for this file is too large to render. See raw diff
 
models/mpt-7b-gsm8k-pt/w8a8-sparse70/openvino_config.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "compression": {
3
+ "algorithm": "quantization",
4
+ "export_to_onnx_standard_ops": false,
5
+ "ignored_scopes": [
6
+ "{re}.*layer_norm_.*"
7
+ ],
8
+ "initializer": {
9
+ "batchnorm_adaptation": {
10
+ "num_bn_adaptation_samples": 0
11
+ },
12
+ "range": {
13
+ "num_init_samples": 4,
14
+ "type": "mean_min_max"
15
+ }
16
+ },
17
+ "overflow_fix": "disable",
18
+ "preset": "performance",
19
+ "scope_overrides": {
20
+ "activations": {
21
+ "{re}.*matmul_0": {
22
+ "mode": "symmetric"
23
+ }
24
+ }
25
+ }
26
+ },
27
+ "input_info": [
28
+ {
29
+ "keyword": "input_ids",
30
+ "sample_size": [
31
+ 1,
32
+ 8
33
+ ],
34
+ "type": "long"
35
+ },
36
+ {
37
+ "keyword": "attention_mask",
38
+ "sample_size": [
39
+ 1,
40
+ 8
41
+ ],
42
+ "type": "long"
43
+ }
44
+ ],
45
+ "log_dir": "models/neuralmagic/mpt-7b-gsm8k-pt/w8a8-sparse70",
46
+ "optimum_version": "1.14.1",
47
+ "save_onnx_model": false,
48
+ "target_device": "CPU",
49
+ "transformers_version": "4.34.1"
50
+ }
models/mpt-7b-gsm8k-pt/w8a8-sparse70/openvino_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:386c12c5bce4f4fa6f2a6de4ba2dce71472e076cdc9eb9ea6936772777486d3c
3
+ size 6655271181
models/mpt-7b-gsm8k-pt/w8a8-sparse70/openvino_model.xml ADDED
The diff for this file is too large to render. See raw diff
 
models/mpt-7b-gsm8k-pt/w8a8-sparse70/original_graph.dot ADDED
The diff for this file is too large to render. See raw diff
 
models/mpt-7b-gsm8k-pt/w8a8-sparse70/ov_weights_type.md ADDED
@@ -0,0 +1,328 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ | | name | shape | type | sparsity |
2
+ |----:|:----------------------------------------------------------------|:-------------|:------------------|-----------:|
3
+ | 0 | Constant_172536 | [50432,4096] | <Type: 'int8_t'> | 0.0132285 |
4
+ | 1 | Constant_172538 | [50432,1] | <Type: 'float32'> | 0 |
5
+ | 2 | Constant_165621 | [1,1,4096] | <Type: 'float32'> | 0 |
6
+ | 3 | Constant_172540 | [12288,4096] | <Type: 'int8_t'> | 0.741039 |
7
+ | 4 | Constant_172542 | [12288,1] | <Type: 'float32'> | 0 |
8
+ | 5 | __module.model.transformer.blocks.0.attn/aten::slice/Slice_2965 | [32,1,2048] | <Type: 'float32'> | 0 |
9
+ | 6 | Constant_172544 | [4096,4096] | <Type: 'int8_t'> | 0.700155 |
10
+ | 7 | Constant_172546 | [4096,1] | <Type: 'float32'> | 0 |
11
+ | 8 | Constant_165659 | [1,1,4096] | <Type: 'float32'> | 0 |
12
+ | 9 | Constant_172548 | [16384,4096] | <Type: 'int8_t'> | 0.700153 |
13
+ | 10 | Constant_172550 | [16384,1] | <Type: 'float32'> | 0 |
14
+ | 11 | Constant_172552 | [4096,16384] | <Type: 'int8_t'> | 0.700551 |
15
+ | 12 | Constant_172554 | [4096,1] | <Type: 'float32'> | 0 |
16
+ | 13 | Constant_165676 | [1,1,4096] | <Type: 'float32'> | 0 |
17
+ | 14 | Constant_172556 | [12288,4096] | <Type: 'int8_t'> | 0.700357 |
18
+ | 15 | Constant_172558 | [12288,1] | <Type: 'float32'> | 0 |
19
+ | 16 | Constant_172560 | [4096,4096] | <Type: 'int8_t'> | 0.701611 |
20
+ | 17 | Constant_172562 | [4096,1] | <Type: 'float32'> | 0 |
21
+ | 18 | Constant_165714 | [1,1,4096] | <Type: 'float32'> | 0 |
22
+ | 19 | Constant_172564 | [16384,4096] | <Type: 'int8_t'> | 0.700289 |
23
+ | 20 | Constant_172566 | [16384,1] | <Type: 'float32'> | 0 |
24
+ | 21 | Constant_172568 | [4096,16384] | <Type: 'int8_t'> | 0.700118 |
25
+ | 22 | Constant_172570 | [4096,1] | <Type: 'float32'> | 0 |
26
+ | 23 | Constant_165731 | [1,1,4096] | <Type: 'float32'> | 0 |
27
+ | 24 | Constant_172572 | [12288,4096] | <Type: 'int8_t'> | 0.700334 |
28
+ | 25 | Constant_172574 | [12288,1] | <Type: 'float32'> | 0 |
29
+ | 26 | Constant_172576 | [4096,4096] | <Type: 'int8_t'> | 0.701027 |
30
+ | 27 | Constant_172578 | [4096,1] | <Type: 'float32'> | 0 |
31
+ | 28 | Constant_165769 | [1,1,4096] | <Type: 'float32'> | 0 |
32
+ | 29 | Constant_172580 | [16384,4096] | <Type: 'int8_t'> | 0.700194 |
33
+ | 30 | Constant_172582 | [16384,1] | <Type: 'float32'> | 0 |
34
+ | 31 | Constant_172584 | [4096,16384] | <Type: 'int8_t'> | 0.700151 |
35
+ | 32 | Constant_172586 | [4096,1] | <Type: 'float32'> | 0 |
36
+ | 33 | Constant_165786 | [1,1,4096] | <Type: 'float32'> | 0 |
37
+ | 34 | Constant_172588 | [12288,4096] | <Type: 'int8_t'> | 0.700264 |
38
+ | 35 | Constant_172590 | [12288,1] | <Type: 'float32'> | 0 |
39
+ | 36 | Constant_172592 | [4096,4096] | <Type: 'int8_t'> | 0.701121 |
40
+ | 37 | Constant_172594 | [4096,1] | <Type: 'float32'> | 0 |
41
+ | 38 | Constant_165824 | [1,1,4096] | <Type: 'float32'> | 0 |
42
+ | 39 | Constant_172596 | [16384,4096] | <Type: 'int8_t'> | 0.700187 |
43
+ | 40 | Constant_172598 | [16384,1] | <Type: 'float32'> | 0 |
44
+ | 41 | Constant_172600 | [4096,16384] | <Type: 'int8_t'> | 0.700264 |
45
+ | 42 | Constant_172602 | [4096,1] | <Type: 'float32'> | 0 |
46
+ | 43 | Constant_165841 | [1,1,4096] | <Type: 'float32'> | 0 |
47
+ | 44 | Constant_172604 | [12288,4096] | <Type: 'int8_t'> | 0.700241 |
48
+ | 45 | Constant_172606 | [12288,1] | <Type: 'float32'> | 0 |
49
+ | 46 | Constant_172608 | [4096,4096] | <Type: 'int8_t'> | 0.701052 |
50
+ | 47 | Constant_172610 | [4096,1] | <Type: 'float32'> | 0 |
51
+ | 48 | Constant_165879 | [1,1,4096] | <Type: 'float32'> | 0 |
52
+ | 49 | Constant_172612 | [16384,4096] | <Type: 'int8_t'> | 0.700179 |
53
+ | 50 | Constant_172614 | [16384,1] | <Type: 'float32'> | 0 |
54
+ | 51 | Constant_172616 | [4096,16384] | <Type: 'int8_t'> | 0.700188 |
55
+ | 52 | Constant_172618 | [4096,1] | <Type: 'float32'> | 0 |
56
+ | 53 | Constant_165896 | [1,1,4096] | <Type: 'float32'> | 0 |
57
+ | 54 | Constant_172620 | [12288,4096] | <Type: 'int8_t'> | 0.70021 |
58
+ | 55 | Constant_172622 | [12288,1] | <Type: 'float32'> | 0 |
59
+ | 56 | Constant_172624 | [4096,4096] | <Type: 'int8_t'> | 0.700989 |
60
+ | 57 | Constant_172626 | [4096,1] | <Type: 'float32'> | 0 |
61
+ | 58 | Constant_165934 | [1,1,4096] | <Type: 'float32'> | 0 |
62
+ | 59 | Constant_172628 | [16384,4096] | <Type: 'int8_t'> | 0.70017 |
63
+ | 60 | Constant_172630 | [16384,1] | <Type: 'float32'> | 0 |
64
+ | 61 | Constant_172632 | [4096,16384] | <Type: 'int8_t'> | 0.700191 |
65
+ | 62 | Constant_172634 | [4096,1] | <Type: 'float32'> | 0 |
66
+ | 63 | Constant_165951 | [1,1,4096] | <Type: 'float32'> | 0 |
67
+ | 64 | Constant_172636 | [12288,4096] | <Type: 'int8_t'> | 0.700195 |
68
+ | 65 | Constant_172638 | [12288,1] | <Type: 'float32'> | 0 |
69
+ | 66 | Constant_172640 | [4096,4096] | <Type: 'int8_t'> | 0.705793 |
70
+ | 67 | Constant_172642 | [4096,1] | <Type: 'float32'> | 0 |
71
+ | 68 | Constant_165989 | [1,1,4096] | <Type: 'float32'> | 0 |
72
+ | 69 | Constant_172644 | [16384,4096] | <Type: 'int8_t'> | 0.700168 |
73
+ | 70 | Constant_172646 | [16384,1] | <Type: 'float32'> | 0 |
74
+ | 71 | Constant_172648 | [4096,16384] | <Type: 'int8_t'> | 0.700159 |
75
+ | 72 | Constant_172650 | [4096,1] | <Type: 'float32'> | 0 |
76
+ | 73 | Constant_166006 | [1,1,4096] | <Type: 'float32'> | 0 |
77
+ | 74 | Constant_172652 | [12288,4096] | <Type: 'int8_t'> | 0.700183 |
78
+ | 75 | Constant_172654 | [12288,1] | <Type: 'float32'> | 0 |
79
+ | 76 | Constant_172656 | [4096,4096] | <Type: 'int8_t'> | 0.700801 |
80
+ | 77 | Constant_172658 | [4096,1] | <Type: 'float32'> | 0 |
81
+ | 78 | Constant_166044 | [1,1,4096] | <Type: 'float32'> | 0 |
82
+ | 79 | Constant_172660 | [16384,4096] | <Type: 'int8_t'> | 0.700179 |
83
+ | 80 | Constant_172662 | [16384,1] | <Type: 'float32'> | 0 |
84
+ | 81 | Constant_172664 | [4096,16384] | <Type: 'int8_t'> | 0.70027 |
85
+ | 82 | Constant_172666 | [4096,1] | <Type: 'float32'> | 0 |
86
+ | 83 | Constant_166061 | [1,1,4096] | <Type: 'float32'> | 0 |
87
+ | 84 | Constant_172668 | [12288,4096] | <Type: 'int8_t'> | 0.700177 |
88
+ | 85 | Constant_172670 | [12288,1] | <Type: 'float32'> | 0 |
89
+ | 86 | Constant_172672 | [4096,4096] | <Type: 'int8_t'> | 0.700593 |
90
+ | 87 | Constant_172674 | [4096,1] | <Type: 'float32'> | 0 |
91
+ | 88 | Constant_166099 | [1,1,4096] | <Type: 'float32'> | 0 |
92
+ | 89 | Constant_172676 | [16384,4096] | <Type: 'int8_t'> | 0.700156 |
93
+ | 90 | Constant_172678 | [16384,1] | <Type: 'float32'> | 0 |
94
+ | 91 | Constant_172680 | [4096,16384] | <Type: 'int8_t'> | 0.700139 |
95
+ | 92 | Constant_172682 | [4096,1] | <Type: 'float32'> | 0 |
96
+ | 93 | Constant_166116 | [1,1,4096] | <Type: 'float32'> | 0 |
97
+ | 94 | Constant_172684 | [12288,4096] | <Type: 'int8_t'> | 0.700173 |
98
+ | 95 | Constant_172686 | [12288,1] | <Type: 'float32'> | 0 |
99
+ | 96 | Constant_172688 | [4096,4096] | <Type: 'int8_t'> | 0.700544 |
100
+ | 97 | Constant_172690 | [4096,1] | <Type: 'float32'> | 0 |
101
+ | 98 | Constant_166154 | [1,1,4096] | <Type: 'float32'> | 0 |
102
+ | 99 | Constant_172692 | [16384,4096] | <Type: 'int8_t'> | 0.700152 |
103
+ | 100 | Constant_172694 | [16384,1] | <Type: 'float32'> | 0 |
104
+ | 101 | Constant_172696 | [4096,16384] | <Type: 'int8_t'> | 0.700401 |
105
+ | 102 | Constant_172698 | [4096,1] | <Type: 'float32'> | 0 |
106
+ | 103 | Constant_166171 | [1,1,4096] | <Type: 'float32'> | 0 |
107
+ | 104 | Constant_172700 | [12288,4096] | <Type: 'int8_t'> | 0.700158 |
108
+ | 105 | Constant_172702 | [12288,1] | <Type: 'float32'> | 0 |
109
+ | 106 | Constant_172704 | [4096,4096] | <Type: 'int8_t'> | 0.700559 |
110
+ | 107 | Constant_172706 | [4096,1] | <Type: 'float32'> | 0 |
111
+ | 108 | Constant_166209 | [1,1,4096] | <Type: 'float32'> | 0 |
112
+ | 109 | Constant_172708 | [16384,4096] | <Type: 'int8_t'> | 0.700153 |
113
+ | 110 | Constant_172710 | [16384,1] | <Type: 'float32'> | 0 |
114
+ | 111 | Constant_172712 | [4096,16384] | <Type: 'int8_t'> | 0.700149 |
115
+ | 112 | Constant_172714 | [4096,1] | <Type: 'float32'> | 0 |
116
+ | 113 | Constant_166226 | [1,1,4096] | <Type: 'float32'> | 0 |
117
+ | 114 | Constant_172716 | [12288,4096] | <Type: 'int8_t'> | 0.70015 |
118
+ | 115 | Constant_172718 | [12288,1] | <Type: 'float32'> | 0 |
119
+ | 116 | Constant_172720 | [4096,4096] | <Type: 'int8_t'> | 0.700426 |
120
+ | 117 | Constant_172722 | [4096,1] | <Type: 'float32'> | 0 |
121
+ | 118 | Constant_166264 | [1,1,4096] | <Type: 'float32'> | 0 |
122
+ | 119 | Constant_172724 | [16384,4096] | <Type: 'int8_t'> | 0.700154 |
123
+ | 120 | Constant_172726 | [16384,1] | <Type: 'float32'> | 0 |
124
+ | 121 | Constant_172728 | [4096,16384] | <Type: 'int8_t'> | 0.700191 |
125
+ | 122 | Constant_172730 | [4096,1] | <Type: 'float32'> | 0 |
126
+ | 123 | Constant_166281 | [1,1,4096] | <Type: 'float32'> | 0 |
127
+ | 124 | Constant_172732 | [12288,4096] | <Type: 'int8_t'> | 0.700154 |
128
+ | 125 | Constant_172734 | [12288,1] | <Type: 'float32'> | 0 |
129
+ | 126 | Constant_172736 | [4096,4096] | <Type: 'int8_t'> | 0.700569 |
130
+ | 127 | Constant_172738 | [4096,1] | <Type: 'float32'> | 0 |
131
+ | 128 | Constant_166319 | [1,1,4096] | <Type: 'float32'> | 0 |
132
+ | 129 | Constant_172740 | [16384,4096] | <Type: 'int8_t'> | 0.700148 |
133
+ | 130 | Constant_172742 | [16384,1] | <Type: 'float32'> | 0 |
134
+ | 131 | Constant_172744 | [4096,16384] | <Type: 'int8_t'> | 0.700142 |
135
+ | 132 | Constant_172746 | [4096,1] | <Type: 'float32'> | 0 |
136
+ | 133 | Constant_166336 | [1,1,4096] | <Type: 'float32'> | 0 |
137
+ | 134 | Constant_172748 | [12288,4096] | <Type: 'int8_t'> | 0.700148 |
138
+ | 135 | Constant_172750 | [12288,1] | <Type: 'float32'> | 0 |
139
+ | 136 | Constant_172752 | [4096,4096] | <Type: 'int8_t'> | 0.700566 |
140
+ | 137 | Constant_172754 | [4096,1] | <Type: 'float32'> | 0 |
141
+ | 138 | Constant_166374 | [1,1,4096] | <Type: 'float32'> | 0 |
142
+ | 139 | Constant_172756 | [16384,4096] | <Type: 'int8_t'> | 0.700148 |
143
+ | 140 | Constant_172758 | [16384,1] | <Type: 'float32'> | 0 |
144
+ | 141 | Constant_172760 | [4096,16384] | <Type: 'int8_t'> | 0.700171 |
145
+ | 142 | Constant_172762 | [4096,1] | <Type: 'float32'> | 0 |
146
+ | 143 | Constant_166391 | [1,1,4096] | <Type: 'float32'> | 0 |
147
+ | 144 | Constant_172764 | [12288,4096] | <Type: 'int8_t'> | 0.700143 |
148
+ | 145 | Constant_172766 | [12288,1] | <Type: 'float32'> | 0 |
149
+ | 146 | Constant_172768 | [4096,4096] | <Type: 'int8_t'> | 0.700591 |
150
+ | 147 | Constant_172770 | [4096,1] | <Type: 'float32'> | 0 |
151
+ | 148 | Constant_166429 | [1,1,4096] | <Type: 'float32'> | 0 |
152
+ | 149 | Constant_172772 | [16384,4096] | <Type: 'int8_t'> | 0.700145 |
153
+ | 150 | Constant_172774 | [16384,1] | <Type: 'float32'> | 0 |
154
+ | 151 | Constant_172776 | [4096,16384] | <Type: 'int8_t'> | 0.700161 |
155
+ | 152 | Constant_172778 | [4096,1] | <Type: 'float32'> | 0 |
156
+ | 153 | Constant_166446 | [1,1,4096] | <Type: 'float32'> | 0 |
157
+ | 154 | Constant_172780 | [12288,4096] | <Type: 'int8_t'> | 0.700138 |
158
+ | 155 | Constant_172782 | [12288,1] | <Type: 'float32'> | 0 |
159
+ | 156 | Constant_172784 | [4096,4096] | <Type: 'int8_t'> | 0.700433 |
160
+ | 157 | Constant_172786 | [4096,1] | <Type: 'float32'> | 0 |
161
+ | 158 | Constant_166484 | [1,1,4096] | <Type: 'float32'> | 0 |
162
+ | 159 | Constant_172788 | [16384,4096] | <Type: 'int8_t'> | 0.700144 |
163
+ | 160 | Constant_172790 | [16384,1] | <Type: 'float32'> | 0 |
164
+ | 161 | Constant_172792 | [4096,16384] | <Type: 'int8_t'> | 0.70022 |
165
+ | 162 | Constant_172794 | [4096,1] | <Type: 'float32'> | 0 |
166
+ | 163 | Constant_166501 | [1,1,4096] | <Type: 'float32'> | 0 |
167
+ | 164 | Constant_172796 | [12288,4096] | <Type: 'int8_t'> | 0.700138 |
168
+ | 165 | Constant_172798 | [12288,1] | <Type: 'float32'> | 0 |
169
+ | 166 | Constant_172800 | [4096,4096] | <Type: 'int8_t'> | 0.700493 |
170
+ | 167 | Constant_172802 | [4096,1] | <Type: 'float32'> | 0 |
171
+ | 168 | Constant_166539 | [1,1,4096] | <Type: 'float32'> | 0 |
172
+ | 169 | Constant_172804 | [16384,4096] | <Type: 'int8_t'> | 0.700137 |
173
+ | 170 | Constant_172806 | [16384,1] | <Type: 'float32'> | 0 |
174
+ | 171 | Constant_172808 | [4096,16384] | <Type: 'int8_t'> | 0.700188 |
175
+ | 172 | Constant_172810 | [4096,1] | <Type: 'float32'> | 0 |
176
+ | 173 | Constant_166556 | [1,1,4096] | <Type: 'float32'> | 0 |
177
+ | 174 | Constant_172812 | [12288,4096] | <Type: 'int8_t'> | 0.70014 |
178
+ | 175 | Constant_172814 | [12288,1] | <Type: 'float32'> | 0 |
179
+ | 176 | Constant_172816 | [4096,4096] | <Type: 'int8_t'> | 0.700479 |
180
+ | 177 | Constant_172818 | [4096,1] | <Type: 'float32'> | 0 |
181
+ | 178 | Constant_166594 | [1,1,4096] | <Type: 'float32'> | 0 |
182
+ | 179 | Constant_172820 | [16384,4096] | <Type: 'int8_t'> | 0.700134 |
183
+ | 180 | Constant_172822 | [16384,1] | <Type: 'float32'> | 0 |
184
+ | 181 | Constant_172824 | [4096,16384] | <Type: 'int8_t'> | 0.700158 |
185
+ | 182 | Constant_172826 | [4096,1] | <Type: 'float32'> | 0 |
186
+ | 183 | Constant_166611 | [1,1,4096] | <Type: 'float32'> | 0 |
187
+ | 184 | Constant_172828 | [12288,4096] | <Type: 'int8_t'> | 0.700129 |
188
+ | 185 | Constant_172830 | [12288,1] | <Type: 'float32'> | 0 |
189
+ | 186 | Constant_172832 | [4096,4096] | <Type: 'int8_t'> | 0.700606 |
190
+ | 187 | Constant_172834 | [4096,1] | <Type: 'float32'> | 0 |
191
+ | 188 | Constant_166649 | [1,1,4096] | <Type: 'float32'> | 0 |
192
+ | 189 | Constant_172836 | [16384,4096] | <Type: 'int8_t'> | 0.700133 |
193
+ | 190 | Constant_172838 | [16384,1] | <Type: 'float32'> | 0 |
194
+ | 191 | Constant_172840 | [4096,16384] | <Type: 'int8_t'> | 0.700149 |
195
+ | 192 | Constant_172842 | [4096,1] | <Type: 'float32'> | 0 |
196
+ | 193 | Constant_166666 | [1,1,4096] | <Type: 'float32'> | 0 |
197
+ | 194 | Constant_172844 | [12288,4096] | <Type: 'int8_t'> | 0.700137 |
198
+ | 195 | Constant_172846 | [12288,1] | <Type: 'float32'> | 0 |
199
+ | 196 | Constant_172848 | [4096,4096] | <Type: 'int8_t'> | 0.700495 |
200
+ | 197 | Constant_172850 | [4096,1] | <Type: 'float32'> | 0 |
201
+ | 198 | Constant_166704 | [1,1,4096] | <Type: 'float32'> | 0 |
202
+ | 199 | Constant_172852 | [16384,4096] | <Type: 'int8_t'> | 0.700151 |
203
+ | 200 | Constant_172854 | [16384,1] | <Type: 'float32'> | 0 |
204
+ | 201 | Constant_172856 | [4096,16384] | <Type: 'int8_t'> | 0.700191 |
205
+ | 202 | Constant_172858 | [4096,1] | <Type: 'float32'> | 0 |
206
+ | 203 | Constant_166721 | [1,1,4096] | <Type: 'float32'> | 0 |
207
+ | 204 | Constant_172860 | [12288,4096] | <Type: 'int8_t'> | 0.700137 |
208
+ | 205 | Constant_172862 | [12288,1] | <Type: 'float32'> | 0 |
209
+ | 206 | Constant_172864 | [4096,4096] | <Type: 'int8_t'> | 0.700711 |
210
+ | 207 | Constant_172866 | [4096,1] | <Type: 'float32'> | 0 |
211
+ | 208 | Constant_166759 | [1,1,4096] | <Type: 'float32'> | 0 |
212
+ | 209 | Constant_172868 | [16384,4096] | <Type: 'int8_t'> | 0.700142 |
213
+ | 210 | Constant_172870 | [16384,1] | <Type: 'float32'> | 0 |
214
+ | 211 | Constant_172872 | [4096,16384] | <Type: 'int8_t'> | 0.700121 |
215
+ | 212 | Constant_172874 | [4096,1] | <Type: 'float32'> | 0 |
216
+ | 213 | Constant_166776 | [1,1,4096] | <Type: 'float32'> | 0 |
217
+ | 214 | Constant_172876 | [12288,4096] | <Type: 'int8_t'> | 0.700136 |
218
+ | 215 | Constant_172878 | [12288,1] | <Type: 'float32'> | 0 |
219
+ | 216 | Constant_172880 | [4096,4096] | <Type: 'int8_t'> | 0.7006 |
220
+ | 217 | Constant_172882 | [4096,1] | <Type: 'float32'> | 0 |
221
+ | 218 | Constant_166814 | [1,1,4096] | <Type: 'float32'> | 0 |
222
+ | 219 | Constant_172884 | [16384,4096] | <Type: 'int8_t'> | 0.70013 |
223
+ | 220 | Constant_172886 | [16384,1] | <Type: 'float32'> | 0 |
224
+ | 221 | Constant_172888 | [4096,16384] | <Type: 'int8_t'> | 0.700133 |
225
+ | 222 | Constant_172890 | [4096,1] | <Type: 'float32'> | 0 |
226
+ | 223 | Constant_166831 | [1,1,4096] | <Type: 'float32'> | 0 |
227
+ | 224 | Constant_172892 | [12288,4096] | <Type: 'int8_t'> | 0.700133 |
228
+ | 225 | Constant_172894 | [12288,1] | <Type: 'float32'> | 0 |
229
+ | 226 | Constant_172896 | [4096,4096] | <Type: 'int8_t'> | 0.700553 |
230
+ | 227 | Constant_172898 | [4096,1] | <Type: 'float32'> | 0 |
231
+ | 228 | Constant_166869 | [1,1,4096] | <Type: 'float32'> | 0 |
232
+ | 229 | Constant_172900 | [16384,4096] | <Type: 'int8_t'> | 0.700133 |
233
+ | 230 | Constant_172902 | [16384,1] | <Type: 'float32'> | 0 |
234
+ | 231 | Constant_172904 | [4096,16384] | <Type: 'int8_t'> | 0.700228 |
235
+ | 232 | Constant_172906 | [4096,1] | <Type: 'float32'> | 0 |
236
+ | 233 | Constant_166886 | [1,1,4096] | <Type: 'float32'> | 0 |
237
+ | 234 | Constant_172908 | [12288,4096] | <Type: 'int8_t'> | 0.700142 |
238
+ | 235 | Constant_172910 | [12288,1] | <Type: 'float32'> | 0 |
239
+ | 236 | Constant_172912 | [4096,4096] | <Type: 'int8_t'> | 0.700653 |
240
+ | 237 | Constant_172914 | [4096,1] | <Type: 'float32'> | 0 |
241
+ | 238 | Constant_166924 | [1,1,4096] | <Type: 'float32'> | 0 |
242
+ | 239 | Constant_172916 | [16384,4096] | <Type: 'int8_t'> | 0.700127 |
243
+ | 240 | Constant_172918 | [16384,1] | <Type: 'float32'> | 0 |
244
+ | 241 | Constant_172920 | [4096,16384] | <Type: 'int8_t'> | 0.700153 |
245
+ | 242 | Constant_172922 | [4096,1] | <Type: 'float32'> | 0 |
246
+ | 243 | Constant_166941 | [1,1,4096] | <Type: 'float32'> | 0 |
247
+ | 244 | Constant_172924 | [12288,4096] | <Type: 'int8_t'> | 0.700134 |
248
+ | 245 | Constant_172926 | [12288,1] | <Type: 'float32'> | 0 |
249
+ | 246 | Constant_172928 | [4096,4096] | <Type: 'int8_t'> | 0.700658 |
250
+ | 247 | Constant_172930 | [4096,1] | <Type: 'float32'> | 0 |
251
+ | 248 | Constant_166979 | [1,1,4096] | <Type: 'float32'> | 0 |
252
+ | 249 | Constant_172932 | [16384,4096] | <Type: 'int8_t'> | 0.700127 |
253
+ | 250 | Constant_172934 | [16384,1] | <Type: 'float32'> | 0 |
254
+ | 251 | Constant_172936 | [4096,16384] | <Type: 'int8_t'> | 0.700131 |
255
+ | 252 | Constant_172938 | [4096,1] | <Type: 'float32'> | 0 |
256
+ | 253 | Constant_166996 | [1,1,4096] | <Type: 'float32'> | 0 |
257
+ | 254 | Constant_172940 | [12288,4096] | <Type: 'int8_t'> | 0.700133 |
258
+ | 255 | Constant_172942 | [12288,1] | <Type: 'float32'> | 0 |
259
+ | 256 | Constant_172944 | [4096,4096] | <Type: 'int8_t'> | 0.701035 |
260
+ | 257 | Constant_172946 | [4096,1] | <Type: 'float32'> | 0 |
261
+ | 258 | Constant_167034 | [1,1,4096] | <Type: 'float32'> | 0 |
262
+ | 259 | Constant_172948 | [16384,4096] | <Type: 'int8_t'> | 0.70012 |
263
+ | 260 | Constant_172950 | [16384,1] | <Type: 'float32'> | 0 |
264
+ | 261 | Constant_172952 | [4096,16384] | <Type: 'int8_t'> | 0.700101 |
265
+ | 262 | Constant_172954 | [4096,1] | <Type: 'float32'> | 0 |
266
+ | 263 | Constant_167051 | [1,1,4096] | <Type: 'float32'> | 0 |
267
+ | 264 | Constant_172956 | [12288,4096] | <Type: 'int8_t'> | 0.700144 |
268
+ | 265 | Constant_172958 | [12288,1] | <Type: 'float32'> | 0 |
269
+ | 266 | Constant_172960 | [4096,4096] | <Type: 'int8_t'> | 0.700789 |
270
+ | 267 | Constant_172962 | [4096,1] | <Type: 'float32'> | 0 |
271
+ | 268 | Constant_167089 | [1,1,4096] | <Type: 'float32'> | 0 |
272
+ | 269 | Constant_172964 | [16384,4096] | <Type: 'int8_t'> | 0.700119 |
273
+ | 270 | Constant_172966 | [16384,1] | <Type: 'float32'> | 0 |
274
+ | 271 | Constant_172968 | [4096,16384] | <Type: 'int8_t'> | 0.700095 |
275
+ | 272 | Constant_172970 | [4096,1] | <Type: 'float32'> | 0 |
276
+ | 273 | Constant_167106 | [1,1,4096] | <Type: 'float32'> | 0 |
277
+ | 274 | Constant_172972 | [12288,4096] | <Type: 'int8_t'> | 0.700135 |
278
+ | 275 | Constant_172974 | [12288,1] | <Type: 'float32'> | 0 |
279
+ | 276 | Constant_172976 | [4096,4096] | <Type: 'int8_t'> | 0.700611 |
280
+ | 277 | Constant_172978 | [4096,1] | <Type: 'float32'> | 0 |
281
+ | 278 | Constant_167144 | [1,1,4096] | <Type: 'float32'> | 0 |
282
+ | 279 | Constant_172980 | [16384,4096] | <Type: 'int8_t'> | 0.700116 |
283
+ | 280 | Constant_172982 | [16384,1] | <Type: 'float32'> | 0 |
284
+ | 281 | Constant_172984 | [4096,16384] | <Type: 'int8_t'> | 0.700122 |
285
+ | 282 | Constant_172986 | [4096,1] | <Type: 'float32'> | 0 |
286
+ | 283 | Constant_167161 | [1,1,4096] | <Type: 'float32'> | 0 |
287
+ | 284 | Constant_172988 | [12288,4096] | <Type: 'int8_t'> | 0.700128 |
288
+ | 285 | Constant_172990 | [12288,1] | <Type: 'float32'> | 0 |
289
+ | 286 | Constant_172992 | [4096,4096] | <Type: 'int8_t'> | 0.700809 |
290
+ | 287 | Constant_172994 | [4096,1] | <Type: 'float32'> | 0 |
291
+ | 288 | Constant_167199 | [1,1,4096] | <Type: 'float32'> | 0 |
292
+ | 289 | Constant_172996 | [16384,4096] | <Type: 'int8_t'> | 0.700118 |
293
+ | 290 | Constant_172998 | [16384,1] | <Type: 'float32'> | 0 |
294
+ | 291 | Constant_173000 | [4096,16384] | <Type: 'int8_t'> | 0.700141 |
295
+ | 292 | Constant_173002 | [4096,1] | <Type: 'float32'> | 0 |
296
+ | 293 | Constant_167216 | [1,1,4096] | <Type: 'float32'> | 0 |
297
+ | 294 | Constant_173004 | [12288,4096] | <Type: 'int8_t'> | 0.700134 |
298
+ | 295 | Constant_173006 | [12288,1] | <Type: 'float32'> | 0 |
299
+ | 296 | Constant_173008 | [4096,4096] | <Type: 'int8_t'> | 0.700483 |
300
+ | 297 | Constant_173010 | [4096,1] | <Type: 'float32'> | 0 |
301
+ | 298 | Constant_167254 | [1,1,4096] | <Type: 'float32'> | 0 |
302
+ | 299 | Constant_173012 | [16384,4096] | <Type: 'int8_t'> | 0.70012 |
303
+ | 300 | Constant_173014 | [16384,1] | <Type: 'float32'> | 0 |
304
+ | 301 | Constant_173016 | [4096,16384] | <Type: 'int8_t'> | 0.700152 |
305
+ | 302 | Constant_173018 | [4096,1] | <Type: 'float32'> | 0 |
306
+ | 303 | Constant_167271 | [1,1,4096] | <Type: 'float32'> | 0 |
307
+ | 304 | Constant_173020 | [12288,4096] | <Type: 'int8_t'> | 0.700134 |
308
+ | 305 | Constant_173022 | [12288,1] | <Type: 'float32'> | 0 |
309
+ | 306 | Constant_173024 | [4096,4096] | <Type: 'int8_t'> | 0.700556 |
310
+ | 307 | Constant_173026 | [4096,1] | <Type: 'float32'> | 0 |
311
+ | 308 | Constant_167309 | [1,1,4096] | <Type: 'float32'> | 0 |
312
+ | 309 | Constant_173028 | [16384,4096] | <Type: 'int8_t'> | 0.700115 |
313
+ | 310 | Constant_173030 | [16384,1] | <Type: 'float32'> | 0 |
314
+ | 311 | Constant_173032 | [4096,16384] | <Type: 'int8_t'> | 0.700185 |
315
+ | 312 | Constant_173034 | [4096,1] | <Type: 'float32'> | 0 |
316
+ | 313 | Constant_167326 | [1,1,4096] | <Type: 'float32'> | 0 |
317
+ | 314 | Constant_173036 | [12288,4096] | <Type: 'int8_t'> | 0.700115 |
318
+ | 315 | Constant_173038 | [12288,1] | <Type: 'float32'> | 0 |
319
+ | 316 | Constant_173040 | [4096,4096] | <Type: 'int8_t'> | 0.700204 |
320
+ | 317 | Constant_173042 | [4096,1] | <Type: 'float32'> | 0 |
321
+ | 318 | Constant_167364 | [1,1,4096] | <Type: 'float32'> | 0 |
322
+ | 319 | Constant_173044 | [16384,4096] | <Type: 'int8_t'> | 0.70015 |
323
+ | 320 | Constant_173046 | [16384,1] | <Type: 'float32'> | 0 |
324
+ | 321 | Constant_173048 | [4096,16384] | <Type: 'int8_t'> | 0.700184 |
325
+ | 322 | Constant_173050 | [4096,1] | <Type: 'float32'> | 0 |
326
+ | 323 | Constant_167381 | [1,1,4096] | <Type: 'float32'> | 0 |
327
+ | 324 | Constant_173052 | [50432,4096] | <Type: 'int8_t'> | 0.0132285 |
328
+ | 325 | Constant_173054 | [50432,1] | <Type: 'float32'> | 0 |
models/mpt-7b-gsm8k-pt/w8a8-sparse70/special_tokens_map.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": "<|endoftext|>",
3
+ "eos_token": "<|endoftext|>",
4
+ "pad_token": "<|endoftext|>",
5
+ "unk_token": "<|endoftext|>"
6
+ }
models/mpt-7b-gsm8k-pt/w8a8-sparse70/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
models/mpt-7b-gsm8k-pt/w8a8-sparse70/tokenizer_config.json ADDED
@@ -0,0 +1,212 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<|padding|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "50254": {
21
+ "content": " ",
22
+ "lstrip": false,
23
+ "normalized": true,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": false
27
+ },
28
+ "50255": {
29
+ "content": " ",
30
+ "lstrip": false,
31
+ "normalized": true,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": false
35
+ },
36
+ "50256": {
37
+ "content": " ",
38
+ "lstrip": false,
39
+ "normalized": true,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": false
43
+ },
44
+ "50257": {
45
+ "content": " ",
46
+ "lstrip": false,
47
+ "normalized": true,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": false
51
+ },
52
+ "50258": {
53
+ "content": " ",
54
+ "lstrip": false,
55
+ "normalized": true,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": false
59
+ },
60
+ "50259": {
61
+ "content": " ",
62
+ "lstrip": false,
63
+ "normalized": true,
64
+ "rstrip": false,
65
+ "single_word": false,
66
+ "special": false
67
+ },
68
+ "50260": {
69
+ "content": " ",
70
+ "lstrip": false,
71
+ "normalized": true,
72
+ "rstrip": false,
73
+ "single_word": false,
74
+ "special": false
75
+ },
76
+ "50261": {
77
+ "content": " ",
78
+ "lstrip": false,
79
+ "normalized": true,
80
+ "rstrip": false,
81
+ "single_word": false,
82
+ "special": false
83
+ },
84
+ "50262": {
85
+ "content": " ",
86
+ "lstrip": false,
87
+ "normalized": true,
88
+ "rstrip": false,
89
+ "single_word": false,
90
+ "special": false
91
+ },
92
+ "50263": {
93
+ "content": " ",
94
+ "lstrip": false,
95
+ "normalized": true,
96
+ "rstrip": false,
97
+ "single_word": false,
98
+ "special": false
99
+ },
100
+ "50264": {
101
+ "content": " ",
102
+ "lstrip": false,
103
+ "normalized": true,
104
+ "rstrip": false,
105
+ "single_word": false,
106
+ "special": false
107
+ },
108
+ "50265": {
109
+ "content": " ",
110
+ "lstrip": false,
111
+ "normalized": true,
112
+ "rstrip": false,
113
+ "single_word": false,
114
+ "special": false
115
+ },
116
+ "50266": {
117
+ "content": " ",
118
+ "lstrip": false,
119
+ "normalized": true,
120
+ "rstrip": false,
121
+ "single_word": false,
122
+ "special": false
123
+ },
124
+ "50267": {
125
+ "content": " ",
126
+ "lstrip": false,
127
+ "normalized": true,
128
+ "rstrip": false,
129
+ "single_word": false,
130
+ "special": false
131
+ },
132
+ "50268": {
133
+ "content": " ",
134
+ "lstrip": false,
135
+ "normalized": true,
136
+ "rstrip": false,
137
+ "single_word": false,
138
+ "special": false
139
+ },
140
+ "50269": {
141
+ "content": " ",
142
+ "lstrip": false,
143
+ "normalized": true,
144
+ "rstrip": false,
145
+ "single_word": false,
146
+ "special": false
147
+ },
148
+ "50270": {
149
+ "content": " ",
150
+ "lstrip": false,
151
+ "normalized": true,
152
+ "rstrip": false,
153
+ "single_word": false,
154
+ "special": false
155
+ },
156
+ "50271": {
157
+ "content": " ",
158
+ "lstrip": false,
159
+ "normalized": true,
160
+ "rstrip": false,
161
+ "single_word": false,
162
+ "special": false
163
+ },
164
+ "50272": {
165
+ "content": " ",
166
+ "lstrip": false,
167
+ "normalized": true,
168
+ "rstrip": false,
169
+ "single_word": false,
170
+ "special": false
171
+ },
172
+ "50273": {
173
+ "content": " ",
174
+ "lstrip": false,
175
+ "normalized": true,
176
+ "rstrip": false,
177
+ "single_word": false,
178
+ "special": false
179
+ },
180
+ "50274": {
181
+ "content": " ",
182
+ "lstrip": false,
183
+ "normalized": true,
184
+ "rstrip": false,
185
+ "single_word": false,
186
+ "special": false
187
+ },
188
+ "50275": {
189
+ "content": " ",
190
+ "lstrip": false,
191
+ "normalized": true,
192
+ "rstrip": false,
193
+ "single_word": false,
194
+ "special": false
195
+ },
196
+ "50276": {
197
+ "content": " ",
198
+ "lstrip": false,
199
+ "normalized": true,
200
+ "rstrip": false,
201
+ "single_word": false,
202
+ "special": false
203
+ }
204
+ },
205
+ "bos_token": "<|endoftext|>",
206
+ "clean_up_tokenization_spaces": true,
207
+ "eos_token": "<|endoftext|>",
208
+ "model_max_length": 512,
209
+ "pad_token": "<|endoftext|>",
210
+ "tokenizer_class": "GPTNeoXTokenizer",
211
+ "unk_token": "<|endoftext|>"
212
+ }
models/mpt-7b-gsm8k-pt/w8a8/args.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "model_id": "neuralmagic/mpt-7b-gsm8k-pt",
3
+ "run_name": "w8a8",
4
+ "quant_mode": "W8A8",
5
+ "force_run": true
6
+ }
models/mpt-7b-gsm8k-pt/w8a8/compressed_graph.dot ADDED
The diff for this file is too large to render. See raw diff
 
models/mpt-7b-gsm8k-pt/w8a8/config.json ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "neuralmagic/mpt-7b-gsm8k-pt",
3
+ "architectures": [
4
+ "MPTForCausalLM"
5
+ ],
6
+ "attn_config": {
7
+ "model_type": ""
8
+ },
9
+ "auto_map": {
10
+ "AutoConfig": "neuralmagic/mpt-7b-gsm8k-pt--configuration_mpt.MPTConfig",
11
+ "AutoModelForCausalLM": "neuralmagic/mpt-7b-gsm8k-pt--modeling_mpt.MPTForCausalLM"
12
+ },
13
+ "d_model": 4096,
14
+ "emb_pdrop": 0,
15
+ "embedding_fraction": 1.0,
16
+ "expansion_ratio": 4,
17
+ "init_config": {
18
+ "emb_init_std": null,
19
+ "emb_init_uniform_lim": null,
20
+ "fan_mode": "fan_in",
21
+ "init_div_is_residual": true,
22
+ "init_gain": 0,
23
+ "init_nonlinearity": "relu",
24
+ "init_std": 0.02,
25
+ "name": "kaiming_normal_",
26
+ "verbose": 0
27
+ },
28
+ "init_device": "cpu",
29
+ "initializer_range": 0.02,
30
+ "layer_norm_epsilon": 1e-05,
31
+ "learned_pos_emb": true,
32
+ "logit_scale": null,
33
+ "max_seq_len": 2048,
34
+ "model_type": "mpt",
35
+ "n_heads": 32,
36
+ "n_layers": 32,
37
+ "no_bias": true,
38
+ "norm_type": "low_precision_layernorm",
39
+ "resid_pdrop": 0,
40
+ "tie_weights": false,
41
+ "tie_word_embeddings": false,
42
+ "tokenizer_name": "EleutherAI/gpt-neox-20b",
43
+ "torch_dtype": "bfloat16",
44
+ "transformers_version": "4.34.1",
45
+ "use_cache": true,
46
+ "verbose": 0,
47
+ "vocab_size": 50432
48
+ }
models/mpt-7b-gsm8k-pt/w8a8/log.log ADDED
@@ -0,0 +1,903 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, onnx, openvino
2
+ Args(model_id='neuralmagic/mpt-7b-gsm8k-pt', run_name='w8a8', quant_mode='W8A8', force_run=True)
3
+
4
+ transformer.wte.weight torch.Size([50432, 4096])
5
+ transformer.blocks.0.norm_1.weight torch.Size([4096])
6
+ transformer.blocks.0.attn.Wqkv.weight torch.Size([12288, 4096])
7
+ transformer.blocks.0.attn.out_proj.weight torch.Size([4096, 4096])
8
+ transformer.blocks.0.norm_2.weight torch.Size([4096])
9
+ transformer.blocks.0.ffn.up_proj.weight torch.Size([16384, 4096])
10
+ transformer.blocks.0.ffn.down_proj.weight torch.Size([4096, 16384])
11
+ transformer.blocks.1.norm_1.weight torch.Size([4096])
12
+ transformer.blocks.1.attn.Wqkv.weight torch.Size([12288, 4096])
13
+ transformer.blocks.1.attn.out_proj.weight torch.Size([4096, 4096])
14
+ transformer.blocks.1.norm_2.weight torch.Size([4096])
15
+ transformer.blocks.1.ffn.up_proj.weight torch.Size([16384, 4096])
16
+ transformer.blocks.1.ffn.down_proj.weight torch.Size([4096, 16384])
17
+ transformer.blocks.2.norm_1.weight torch.Size([4096])
18
+ transformer.blocks.2.attn.Wqkv.weight torch.Size([12288, 4096])
19
+ transformer.blocks.2.attn.out_proj.weight torch.Size([4096, 4096])
20
+ transformer.blocks.2.norm_2.weight torch.Size([4096])
21
+ transformer.blocks.2.ffn.up_proj.weight torch.Size([16384, 4096])
22
+ transformer.blocks.2.ffn.down_proj.weight torch.Size([4096, 16384])
23
+ transformer.blocks.3.norm_1.weight torch.Size([4096])
24
+ transformer.blocks.3.attn.Wqkv.weight torch.Size([12288, 4096])
25
+ transformer.blocks.3.attn.out_proj.weight torch.Size([4096, 4096])
26
+ transformer.blocks.3.norm_2.weight torch.Size([4096])
27
+ transformer.blocks.3.ffn.up_proj.weight torch.Size([16384, 4096])
28
+ transformer.blocks.3.ffn.down_proj.weight torch.Size([4096, 16384])
29
+ transformer.blocks.4.norm_1.weight torch.Size([4096])
30
+ transformer.blocks.4.attn.Wqkv.weight torch.Size([12288, 4096])
31
+ transformer.blocks.4.attn.out_proj.weight torch.Size([4096, 4096])
32
+ transformer.blocks.4.norm_2.weight torch.Size([4096])
33
+ transformer.blocks.4.ffn.up_proj.weight torch.Size([16384, 4096])
34
+ transformer.blocks.4.ffn.down_proj.weight torch.Size([4096, 16384])
35
+ transformer.blocks.5.norm_1.weight torch.Size([4096])
36
+ transformer.blocks.5.attn.Wqkv.weight torch.Size([12288, 4096])
37
+ transformer.blocks.5.attn.out_proj.weight torch.Size([4096, 4096])
38
+ transformer.blocks.5.norm_2.weight torch.Size([4096])
39
+ transformer.blocks.5.ffn.up_proj.weight torch.Size([16384, 4096])
40
+ transformer.blocks.5.ffn.down_proj.weight torch.Size([4096, 16384])
41
+ transformer.blocks.6.norm_1.weight torch.Size([4096])
42
+ transformer.blocks.6.attn.Wqkv.weight torch.Size([12288, 4096])
43
+ transformer.blocks.6.attn.out_proj.weight torch.Size([4096, 4096])
44
+ transformer.blocks.6.norm_2.weight torch.Size([4096])
45
+ transformer.blocks.6.ffn.up_proj.weight torch.Size([16384, 4096])
46
+ transformer.blocks.6.ffn.down_proj.weight torch.Size([4096, 16384])
47
+ transformer.blocks.7.norm_1.weight torch.Size([4096])
48
+ transformer.blocks.7.attn.Wqkv.weight torch.Size([12288, 4096])
49
+ transformer.blocks.7.attn.out_proj.weight torch.Size([4096, 4096])
50
+ transformer.blocks.7.norm_2.weight torch.Size([4096])
51
+ transformer.blocks.7.ffn.up_proj.weight torch.Size([16384, 4096])
52
+ transformer.blocks.7.ffn.down_proj.weight torch.Size([4096, 16384])
53
+ transformer.blocks.8.norm_1.weight torch.Size([4096])
54
+ transformer.blocks.8.attn.Wqkv.weight torch.Size([12288, 4096])
55
+ transformer.blocks.8.attn.out_proj.weight torch.Size([4096, 4096])
56
+ transformer.blocks.8.norm_2.weight torch.Size([4096])
57
+ transformer.blocks.8.ffn.up_proj.weight torch.Size([16384, 4096])
58
+ transformer.blocks.8.ffn.down_proj.weight torch.Size([4096, 16384])
59
+ transformer.blocks.9.norm_1.weight torch.Size([4096])
60
+ transformer.blocks.9.attn.Wqkv.weight torch.Size([12288, 4096])
61
+ transformer.blocks.9.attn.out_proj.weight torch.Size([4096, 4096])
62
+ transformer.blocks.9.norm_2.weight torch.Size([4096])
63
+ transformer.blocks.9.ffn.up_proj.weight torch.Size([16384, 4096])
64
+ transformer.blocks.9.ffn.down_proj.weight torch.Size([4096, 16384])
65
+ transformer.blocks.10.norm_1.weight torch.Size([4096])
66
+ transformer.blocks.10.attn.Wqkv.weight torch.Size([12288, 4096])
67
+ transformer.blocks.10.attn.out_proj.weight torch.Size([4096, 4096])
68
+ transformer.blocks.10.norm_2.weight torch.Size([4096])
69
+ transformer.blocks.10.ffn.up_proj.weight torch.Size([16384, 4096])
70
+ transformer.blocks.10.ffn.down_proj.weight torch.Size([4096, 16384])
71
+ transformer.blocks.11.norm_1.weight torch.Size([4096])
72
+ transformer.blocks.11.attn.Wqkv.weight torch.Size([12288, 4096])
73
+ transformer.blocks.11.attn.out_proj.weight torch.Size([4096, 4096])
74
+ transformer.blocks.11.norm_2.weight torch.Size([4096])
75
+ transformer.blocks.11.ffn.up_proj.weight torch.Size([16384, 4096])
76
+ transformer.blocks.11.ffn.down_proj.weight torch.Size([4096, 16384])
77
+ transformer.blocks.12.norm_1.weight torch.Size([4096])
78
+ transformer.blocks.12.attn.Wqkv.weight torch.Size([12288, 4096])
79
+ transformer.blocks.12.attn.out_proj.weight torch.Size([4096, 4096])
80
+ transformer.blocks.12.norm_2.weight torch.Size([4096])
81
+ transformer.blocks.12.ffn.up_proj.weight torch.Size([16384, 4096])
82
+ transformer.blocks.12.ffn.down_proj.weight torch.Size([4096, 16384])
83
+ transformer.blocks.13.norm_1.weight torch.Size([4096])
84
+ transformer.blocks.13.attn.Wqkv.weight torch.Size([12288, 4096])
85
+ transformer.blocks.13.attn.out_proj.weight torch.Size([4096, 4096])
86
+ transformer.blocks.13.norm_2.weight torch.Size([4096])
87
+ transformer.blocks.13.ffn.up_proj.weight torch.Size([16384, 4096])
88
+ transformer.blocks.13.ffn.down_proj.weight torch.Size([4096, 16384])
89
+ transformer.blocks.14.norm_1.weight torch.Size([4096])
90
+ transformer.blocks.14.attn.Wqkv.weight torch.Size([12288, 4096])
91
+ transformer.blocks.14.attn.out_proj.weight torch.Size([4096, 4096])
92
+ transformer.blocks.14.norm_2.weight torch.Size([4096])
93
+ transformer.blocks.14.ffn.up_proj.weight torch.Size([16384, 4096])
94
+ transformer.blocks.14.ffn.down_proj.weight torch.Size([4096, 16384])
95
+ transformer.blocks.15.norm_1.weight torch.Size([4096])
96
+ transformer.blocks.15.attn.Wqkv.weight torch.Size([12288, 4096])
97
+ transformer.blocks.15.attn.out_proj.weight torch.Size([4096, 4096])
98
+ transformer.blocks.15.norm_2.weight torch.Size([4096])
99
+ transformer.blocks.15.ffn.up_proj.weight torch.Size([16384, 4096])
100
+ transformer.blocks.15.ffn.down_proj.weight torch.Size([4096, 16384])
101
+ transformer.blocks.16.norm_1.weight torch.Size([4096])
102
+ transformer.blocks.16.attn.Wqkv.weight torch.Size([12288, 4096])
103
+ transformer.blocks.16.attn.out_proj.weight torch.Size([4096, 4096])
104
+ transformer.blocks.16.norm_2.weight torch.Size([4096])
105
+ transformer.blocks.16.ffn.up_proj.weight torch.Size([16384, 4096])
106
+ transformer.blocks.16.ffn.down_proj.weight torch.Size([4096, 16384])
107
+ transformer.blocks.17.norm_1.weight torch.Size([4096])
108
+ transformer.blocks.17.attn.Wqkv.weight torch.Size([12288, 4096])
109
+ transformer.blocks.17.attn.out_proj.weight torch.Size([4096, 4096])
110
+ transformer.blocks.17.norm_2.weight torch.Size([4096])
111
+ transformer.blocks.17.ffn.up_proj.weight torch.Size([16384, 4096])
112
+ transformer.blocks.17.ffn.down_proj.weight torch.Size([4096, 16384])
113
+ transformer.blocks.18.norm_1.weight torch.Size([4096])
114
+ transformer.blocks.18.attn.Wqkv.weight torch.Size([12288, 4096])
115
+ transformer.blocks.18.attn.out_proj.weight torch.Size([4096, 4096])
116
+ transformer.blocks.18.norm_2.weight torch.Size([4096])
117
+ transformer.blocks.18.ffn.up_proj.weight torch.Size([16384, 4096])
118
+ transformer.blocks.18.ffn.down_proj.weight torch.Size([4096, 16384])
119
+ transformer.blocks.19.norm_1.weight torch.Size([4096])
120
+ transformer.blocks.19.attn.Wqkv.weight torch.Size([12288, 4096])
121
+ transformer.blocks.19.attn.out_proj.weight torch.Size([4096, 4096])
122
+ transformer.blocks.19.norm_2.weight torch.Size([4096])
123
+ transformer.blocks.19.ffn.up_proj.weight torch.Size([16384, 4096])
124
+ transformer.blocks.19.ffn.down_proj.weight torch.Size([4096, 16384])
125
+ transformer.blocks.20.norm_1.weight torch.Size([4096])
126
+ transformer.blocks.20.attn.Wqkv.weight torch.Size([12288, 4096])
127
+ transformer.blocks.20.attn.out_proj.weight torch.Size([4096, 4096])
128
+ transformer.blocks.20.norm_2.weight torch.Size([4096])
129
+ transformer.blocks.20.ffn.up_proj.weight torch.Size([16384, 4096])
130
+ transformer.blocks.20.ffn.down_proj.weight torch.Size([4096, 16384])
131
+ transformer.blocks.21.norm_1.weight torch.Size([4096])
132
+ transformer.blocks.21.attn.Wqkv.weight torch.Size([12288, 4096])
133
+ transformer.blocks.21.attn.out_proj.weight torch.Size([4096, 4096])
134
+ transformer.blocks.21.norm_2.weight torch.Size([4096])
135
+ transformer.blocks.21.ffn.up_proj.weight torch.Size([16384, 4096])
136
+ transformer.blocks.21.ffn.down_proj.weight torch.Size([4096, 16384])
137
+ transformer.blocks.22.norm_1.weight torch.Size([4096])
138
+ transformer.blocks.22.attn.Wqkv.weight torch.Size([12288, 4096])
139
+ transformer.blocks.22.attn.out_proj.weight torch.Size([4096, 4096])
140
+ transformer.blocks.22.norm_2.weight torch.Size([4096])
141
+ transformer.blocks.22.ffn.up_proj.weight torch.Size([16384, 4096])
142
+ transformer.blocks.22.ffn.down_proj.weight torch.Size([4096, 16384])
143
+ transformer.blocks.23.norm_1.weight torch.Size([4096])
144
+ transformer.blocks.23.attn.Wqkv.weight torch.Size([12288, 4096])
145
+ transformer.blocks.23.attn.out_proj.weight torch.Size([4096, 4096])
146
+ transformer.blocks.23.norm_2.weight torch.Size([4096])
147
+ transformer.blocks.23.ffn.up_proj.weight torch.Size([16384, 4096])
148
+ transformer.blocks.23.ffn.down_proj.weight torch.Size([4096, 16384])
149
+ transformer.blocks.24.norm_1.weight torch.Size([4096])
150
+ transformer.blocks.24.attn.Wqkv.weight torch.Size([12288, 4096])
151
+ transformer.blocks.24.attn.out_proj.weight torch.Size([4096, 4096])
152
+ transformer.blocks.24.norm_2.weight torch.Size([4096])
153
+ transformer.blocks.24.ffn.up_proj.weight torch.Size([16384, 4096])
154
+ transformer.blocks.24.ffn.down_proj.weight torch.Size([4096, 16384])
155
+ transformer.blocks.25.norm_1.weight torch.Size([4096])
156
+ transformer.blocks.25.attn.Wqkv.weight torch.Size([12288, 4096])
157
+ transformer.blocks.25.attn.out_proj.weight torch.Size([4096, 4096])
158
+ transformer.blocks.25.norm_2.weight torch.Size([4096])
159
+ transformer.blocks.25.ffn.up_proj.weight torch.Size([16384, 4096])
160
+ transformer.blocks.25.ffn.down_proj.weight torch.Size([4096, 16384])
161
+ transformer.blocks.26.norm_1.weight torch.Size([4096])
162
+ transformer.blocks.26.attn.Wqkv.weight torch.Size([12288, 4096])
163
+ transformer.blocks.26.attn.out_proj.weight torch.Size([4096, 4096])
164
+ transformer.blocks.26.norm_2.weight torch.Size([4096])
165
+ transformer.blocks.26.ffn.up_proj.weight torch.Size([16384, 4096])
166
+ transformer.blocks.26.ffn.down_proj.weight torch.Size([4096, 16384])
167
+ transformer.blocks.27.norm_1.weight torch.Size([4096])
168
+ transformer.blocks.27.attn.Wqkv.weight torch.Size([12288, 4096])
169
+ transformer.blocks.27.attn.out_proj.weight torch.Size([4096, 4096])
170
+ transformer.blocks.27.norm_2.weight torch.Size([4096])
171
+ transformer.blocks.27.ffn.up_proj.weight torch.Size([16384, 4096])
172
+ transformer.blocks.27.ffn.down_proj.weight torch.Size([4096, 16384])
173
+ transformer.blocks.28.norm_1.weight torch.Size([4096])
174
+ transformer.blocks.28.attn.Wqkv.weight torch.Size([12288, 4096])
175
+ transformer.blocks.28.attn.out_proj.weight torch.Size([4096, 4096])
176
+ transformer.blocks.28.norm_2.weight torch.Size([4096])
177
+ transformer.blocks.28.ffn.up_proj.weight torch.Size([16384, 4096])
178
+ transformer.blocks.28.ffn.down_proj.weight torch.Size([4096, 16384])
179
+ transformer.blocks.29.norm_1.weight torch.Size([4096])
180
+ transformer.blocks.29.attn.Wqkv.weight torch.Size([12288, 4096])
181
+ transformer.blocks.29.attn.out_proj.weight torch.Size([4096, 4096])
182
+ transformer.blocks.29.norm_2.weight torch.Size([4096])
183
+ transformer.blocks.29.ffn.up_proj.weight torch.Size([16384, 4096])
184
+ transformer.blocks.29.ffn.down_proj.weight torch.Size([4096, 16384])
185
+ transformer.blocks.30.norm_1.weight torch.Size([4096])
186
+ transformer.blocks.30.attn.Wqkv.weight torch.Size([12288, 4096])
187
+ transformer.blocks.30.attn.out_proj.weight torch.Size([4096, 4096])
188
+ transformer.blocks.30.norm_2.weight torch.Size([4096])
189
+ transformer.blocks.30.ffn.up_proj.weight torch.Size([16384, 4096])
190
+ transformer.blocks.30.ffn.down_proj.weight torch.Size([4096, 16384])
191
+ transformer.blocks.31.norm_1.weight torch.Size([4096])
192
+ transformer.blocks.31.attn.Wqkv.weight torch.Size([12288, 4096])
193
+ transformer.blocks.31.attn.out_proj.weight torch.Size([4096, 4096])
194
+ transformer.blocks.31.norm_2.weight torch.Size([4096])
195
+ transformer.blocks.31.ffn.up_proj.weight torch.Size([16384, 4096])
196
+ transformer.blocks.31.ffn.down_proj.weight torch.Size([4096, 16384])
197
+ transformer.norm_f.weight torch.Size([4096])
198
+ lm_head.weight torch.Size([50432, 4096])
199
+
200
+ transformer
201
+ transformer.wte
202
+ transformer.blocks
203
+ transformer.blocks.0
204
+ transformer.blocks.0.norm_1
205
+ transformer.blocks.0.attn
206
+ transformer.blocks.0.attn.Wqkv
207
+ transformer.blocks.0.attn.out_proj
208
+ transformer.blocks.0.norm_2
209
+ transformer.blocks.0.ffn
210
+ transformer.blocks.0.ffn.up_proj
211
+ transformer.blocks.0.ffn.act
212
+ transformer.blocks.0.ffn.down_proj
213
+ transformer.blocks.0.resid_attn_dropout
214
+ transformer.blocks.1
215
+ transformer.blocks.1.norm_1
216
+ transformer.blocks.1.attn
217
+ transformer.blocks.1.attn.Wqkv
218
+ transformer.blocks.1.attn.out_proj
219
+ transformer.blocks.1.norm_2
220
+ transformer.blocks.1.ffn
221
+ transformer.blocks.1.ffn.up_proj
222
+ transformer.blocks.1.ffn.act
223
+ transformer.blocks.1.ffn.down_proj
224
+ transformer.blocks.1.resid_attn_dropout
225
+ transformer.blocks.2
226
+ transformer.blocks.2.norm_1
227
+ transformer.blocks.2.attn
228
+ transformer.blocks.2.attn.Wqkv
229
+ transformer.blocks.2.attn.out_proj
230
+ transformer.blocks.2.norm_2
231
+ transformer.blocks.2.ffn
232
+ transformer.blocks.2.ffn.up_proj
233
+ transformer.blocks.2.ffn.act
234
+ transformer.blocks.2.ffn.down_proj
235
+ transformer.blocks.2.resid_attn_dropout
236
+ transformer.blocks.3
237
+ transformer.blocks.3.norm_1
238
+ transformer.blocks.3.attn
239
+ transformer.blocks.3.attn.Wqkv
240
+ transformer.blocks.3.attn.out_proj
241
+ transformer.blocks.3.norm_2
242
+ transformer.blocks.3.ffn
243
+ transformer.blocks.3.ffn.up_proj
244
+ transformer.blocks.3.ffn.act
245
+ transformer.blocks.3.ffn.down_proj
246
+ transformer.blocks.3.resid_attn_dropout
247
+ transformer.blocks.4
248
+ transformer.blocks.4.norm_1
249
+ transformer.blocks.4.attn
250
+ transformer.blocks.4.attn.Wqkv
251
+ transformer.blocks.4.attn.out_proj
252
+ transformer.blocks.4.norm_2
253
+ transformer.blocks.4.ffn
254
+ transformer.blocks.4.ffn.up_proj
255
+ transformer.blocks.4.ffn.act
256
+ transformer.blocks.4.ffn.down_proj
257
+ transformer.blocks.4.resid_attn_dropout
258
+ transformer.blocks.5
259
+ transformer.blocks.5.norm_1
260
+ transformer.blocks.5.attn
261
+ transformer.blocks.5.attn.Wqkv
262
+ transformer.blocks.5.attn.out_proj
263
+ transformer.blocks.5.norm_2
264
+ transformer.blocks.5.ffn
265
+ transformer.blocks.5.ffn.up_proj
266
+ transformer.blocks.5.ffn.act
267
+ transformer.blocks.5.ffn.down_proj
268
+ transformer.blocks.5.resid_attn_dropout
269
+ transformer.blocks.6
270
+ transformer.blocks.6.norm_1
271
+ transformer.blocks.6.attn
272
+ transformer.blocks.6.attn.Wqkv
273
+ transformer.blocks.6.attn.out_proj
274
+ transformer.blocks.6.norm_2
275
+ transformer.blocks.6.ffn
276
+ transformer.blocks.6.ffn.up_proj
277
+ transformer.blocks.6.ffn.act
278
+ transformer.blocks.6.ffn.down_proj
279
+ transformer.blocks.6.resid_attn_dropout
280
+ transformer.blocks.7
281
+ transformer.blocks.7.norm_1
282
+ transformer.blocks.7.attn
283
+ transformer.blocks.7.attn.Wqkv
284
+ transformer.blocks.7.attn.out_proj
285
+ transformer.blocks.7.norm_2
286
+ transformer.blocks.7.ffn
287
+ transformer.blocks.7.ffn.up_proj
288
+ transformer.blocks.7.ffn.act
289
+ transformer.blocks.7.ffn.down_proj
290
+ transformer.blocks.7.resid_attn_dropout
291
+ transformer.blocks.8
292
+ transformer.blocks.8.norm_1
293
+ transformer.blocks.8.attn
294
+ transformer.blocks.8.attn.Wqkv
295
+ transformer.blocks.8.attn.out_proj
296
+ transformer.blocks.8.norm_2
297
+ transformer.blocks.8.ffn
298
+ transformer.blocks.8.ffn.up_proj
299
+ transformer.blocks.8.ffn.act
300
+ transformer.blocks.8.ffn.down_proj
301
+ transformer.blocks.8.resid_attn_dropout
302
+ transformer.blocks.9
303
+ transformer.blocks.9.norm_1
304
+ transformer.blocks.9.attn
305
+ transformer.blocks.9.attn.Wqkv
306
+ transformer.blocks.9.attn.out_proj
307
+ transformer.blocks.9.norm_2
308
+ transformer.blocks.9.ffn
309
+ transformer.blocks.9.ffn.up_proj
310
+ transformer.blocks.9.ffn.act
311
+ transformer.blocks.9.ffn.down_proj
312
+ transformer.blocks.9.resid_attn_dropout
313
+ transformer.blocks.10
314
+ transformer.blocks.10.norm_1
315
+ transformer.blocks.10.attn
316
+ transformer.blocks.10.attn.Wqkv
317
+ transformer.blocks.10.attn.out_proj
318
+ transformer.blocks.10.norm_2
319
+ transformer.blocks.10.ffn
320
+ transformer.blocks.10.ffn.up_proj
321
+ transformer.blocks.10.ffn.act
322
+ transformer.blocks.10.ffn.down_proj
323
+ transformer.blocks.10.resid_attn_dropout
324
+ transformer.blocks.11
325
+ transformer.blocks.11.norm_1
326
+ transformer.blocks.11.attn
327
+ transformer.blocks.11.attn.Wqkv
328
+ transformer.blocks.11.attn.out_proj
329
+ transformer.blocks.11.norm_2
330
+ transformer.blocks.11.ffn
331
+ transformer.blocks.11.ffn.up_proj
332
+ transformer.blocks.11.ffn.act
333
+ transformer.blocks.11.ffn.down_proj
334
+ transformer.blocks.11.resid_attn_dropout
335
+ transformer.blocks.12
336
+ transformer.blocks.12.norm_1
337
+ transformer.blocks.12.attn
338
+ transformer.blocks.12.attn.Wqkv
339
+ /home/ubuntu/miniconda3/envs/deepsparse/lib/python3.10/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
340
+ You can remove this warning by passing 'token=False' instead.
341
+ warnings.warn(
342
+ Repo card metadata block was not found. Setting CardData to empty.
343
+ transformer.blocks.12.attn.out_proj
344
+ transformer.blocks.12.norm_2
345
+ transformer.blocks.12.ffn
346
+ transformer.blocks.12.ffn.up_proj
347
+ transformer.blocks.12.ffn.act
348
+ transformer.blocks.12.ffn.down_proj
349
+ transformer.blocks.12.resid_attn_dropout
350
+ transformer.blocks.13
351
+ transformer.blocks.13.norm_1
352
+ transformer.blocks.13.attn
353
+ transformer.blocks.13.attn.Wqkv
354
+ transformer.blocks.13.attn.out_proj
355
+ transformer.blocks.13.norm_2
356
+ transformer.blocks.13.ffn
357
+ transformer.blocks.13.ffn.up_proj
358
+ transformer.blocks.13.ffn.act
359
+ transformer.blocks.13.ffn.down_proj
360
+ transformer.blocks.13.resid_attn_dropout
361
+ transformer.blocks.14
362
+ transformer.blocks.14.norm_1
363
+ transformer.blocks.14.attn
364
+ transformer.blocks.14.attn.Wqkv
365
+ transformer.blocks.14.attn.out_proj
366
+ transformer.blocks.14.norm_2
367
+ transformer.blocks.14.ffn
368
+ transformer.blocks.14.ffn.up_proj
369
+ transformer.blocks.14.ffn.act
370
+ transformer.blocks.14.ffn.down_proj
371
+ transformer.blocks.14.resid_attn_dropout
372
+ transformer.blocks.15
373
+ transformer.blocks.15.norm_1
374
+ transformer.blocks.15.attn
375
+ transformer.blocks.15.attn.Wqkv
376
+ transformer.blocks.15.attn.out_proj
377
+ transformer.blocks.15.norm_2
378
+ transformer.blocks.15.ffn
379
+ transformer.blocks.15.ffn.up_proj
380
+ transformer.blocks.15.ffn.act
381
+ transformer.blocks.15.ffn.down_proj
382
+ transformer.blocks.15.resid_attn_dropout
383
+ transformer.blocks.16
384
+ transformer.blocks.16.norm_1
385
+ transformer.blocks.16.attn
386
+ transformer.blocks.16.attn.Wqkv
387
+ transformer.blocks.16.attn.out_proj
388
+ transformer.blocks.16.norm_2
389
+ transformer.blocks.16.ffn
390
+ transformer.blocks.16.ffn.up_proj
391
+ transformer.blocks.16.ffn.act
392
+ transformer.blocks.16.ffn.down_proj
393
+ transformer.blocks.16.resid_attn_dropout
394
+ transformer.blocks.17
395
+ transformer.blocks.17.norm_1
396
+ transformer.blocks.17.attn
397
+ transformer.blocks.17.attn.Wqkv
398
+ transformer.blocks.17.attn.out_proj
399
+ transformer.blocks.17.norm_2
400
+ transformer.blocks.17.ffn
401
+ transformer.blocks.17.ffn.up_proj
402
+ transformer.blocks.17.ffn.act
403
+ transformer.blocks.17.ffn.down_proj
404
+ transformer.blocks.17.resid_attn_dropout
405
+ transformer.blocks.18
406
+ transformer.blocks.18.norm_1
407
+ transformer.blocks.18.attn
408
+ transformer.blocks.18.attn.Wqkv
409
+ transformer.blocks.18.attn.out_proj
410
+ transformer.blocks.18.norm_2
411
+ transformer.blocks.18.ffn
412
+ transformer.blocks.18.ffn.up_proj
413
+ transformer.blocks.18.ffn.act
414
+ transformer.blocks.18.ffn.down_proj
415
+ transformer.blocks.18.resid_attn_dropout
416
+ transformer.blocks.19
417
+ transformer.blocks.19.norm_1
418
+ transformer.blocks.19.attn
419
+ transformer.blocks.19.attn.Wqkv
420
+ transformer.blocks.19.attn.out_proj
421
+ transformer.blocks.19.norm_2
422
+ transformer.blocks.19.ffn
423
+ transformer.blocks.19.ffn.up_proj
424
+ transformer.blocks.19.ffn.act
425
+ transformer.blocks.19.ffn.down_proj
426
+ transformer.blocks.19.resid_attn_dropout
427
+ transformer.blocks.20
428
+ transformer.blocks.20.norm_1
429
+ transformer.blocks.20.attn
430
+ transformer.blocks.20.attn.Wqkv
431
+ transformer.blocks.20.attn.out_proj
432
+ transformer.blocks.20.norm_2
433
+ transformer.blocks.20.ffn
434
+ transformer.blocks.20.ffn.up_proj
435
+ transformer.blocks.20.ffn.act
436
+ transformer.blocks.20.ffn.down_proj
437
+ transformer.blocks.20.resid_attn_dropout
438
+ transformer.blocks.21
439
+ transformer.blocks.21.norm_1
440
+ transformer.blocks.21.attn
441
+ transformer.blocks.21.attn.Wqkv
442
+ transformer.blocks.21.attn.out_proj
443
+ transformer.blocks.21.norm_2
444
+ transformer.blocks.21.ffn
445
+ transformer.blocks.21.ffn.up_proj
446
+ transformer.blocks.21.ffn.act
447
+ transformer.blocks.21.ffn.down_proj
448
+ transformer.blocks.21.resid_attn_dropout
449
+ transformer.blocks.22
450
+ transformer.blocks.22.norm_1
451
+ transformer.blocks.22.attn
452
+ transformer.blocks.22.attn.Wqkv
453
+ transformer.blocks.22.attn.out_proj
454
+ transformer.blocks.22.norm_2
455
+ transformer.blocks.22.ffn
456
+ transformer.blocks.22.ffn.up_proj
457
+ transformer.blocks.22.ffn.act
458
+ transformer.blocks.22.ffn.down_proj
459
+ transformer.blocks.22.resid_attn_dropout
460
+ transformer.blocks.23
461
+ transformer.blocks.23.norm_1
462
+ transformer.blocks.23.attn
463
+ transformer.blocks.23.attn.Wqkv
464
+ transformer.blocks.23.attn.out_proj
465
+ transformer.blocks.23.norm_2
466
+ transformer.blocks.23.ffn
467
+ transformer.blocks.23.ffn.up_proj
468
+ transformer.blocks.23.ffn.act
469
+ transformer.blocks.23.ffn.down_proj
470
+ transformer.blocks.23.resid_attn_dropout
471
+ transformer.blocks.24
472
+ transformer.blocks.24.norm_1
473
+ transformer.blocks.24.attn
474
+ transformer.blocks.24.attn.Wqkv
475
+ transformer.blocks.24.attn.out_proj
476
+ transformer.blocks.24.norm_2
477
+ transformer.blocks.24.ffn
478
+ transformer.blocks.24.ffn.up_proj
479
+ transformer.blocks.24.ffn.act
480
+ transformer.blocks.24.ffn.down_proj
481
+ transformer.blocks.24.resid_attn_dropout
482
+ transformer.blocks.25
483
+ transformer.blocks.25.norm_1
484
+ transformer.blocks.25.attn
485
+ transformer.blocks.25.attn.Wqkv
486
+ transformer.blocks.25.attn.out_proj
487
+ transformer.blocks.25.norm_2
488
+ transformer.blocks.25.ffn
489
+ transformer.blocks.25.ffn.up_proj
490
+ transformer.blocks.25.ffn.act
491
+ transformer.blocks.25.ffn.down_proj
492
+ transformer.blocks.25.resid_attn_dropout
493
+ transformer.blocks.26
494
+ transformer.blocks.26.norm_1
495
+ transformer.blocks.26.attn
496
+ transformer.blocks.26.attn.Wqkv
497
+ transformer.blocks.26.attn.out_proj
498
+ transformer.blocks.26.norm_2
499
+ transformer.blocks.26.ffn
500
+ transformer.blocks.26.ffn.up_proj
501
+ transformer.blocks.26.ffn.act
502
+ transformer.blocks.26.ffn.down_proj
503
+ transformer.blocks.26.resid_attn_dropout
504
+ transformer.blocks.27
505
+ transformer.blocks.27.norm_1
506
+ transformer.blocks.27.attn
507
+ transformer.blocks.27.attn.Wqkv
508
+ transformer.blocks.27.attn.out_proj
509
+ transformer.blocks.27.norm_2
510
+ transformer.blocks.27.ffn
511
+ transformer.blocks.27.ffn.up_proj
512
+ transformer.blocks.27.ffn.act
513
+ transformer.blocks.27.ffn.down_proj
514
+ transformer.blocks.27.resid_attn_dropout
515
+ transformer.blocks.28
516
+ transformer.blocks.28.norm_1
517
+ transformer.blocks.28.attn
518
+ transformer.blocks.28.attn.Wqkv
519
+ transformer.blocks.28.attn.out_proj
520
+ transformer.blocks.28.norm_2
521
+ transformer.blocks.28.ffn
522
+ transformer.blocks.28.ffn.up_proj
523
+ transformer.blocks.28.ffn.act
524
+ transformer.blocks.28.ffn.down_proj
525
+ transformer.blocks.28.resid_attn_dropout
526
+ transformer.blocks.29
527
+ transformer.blocks.29.norm_1
528
+ transformer.blocks.29.attn
529
+ transformer.blocks.29.attn.Wqkv
530
+ transformer.blocks.29.attn.out_proj
531
+ transformer.blocks.29.norm_2
532
+ transformer.blocks.29.ffn
533
+ transformer.blocks.29.ffn.up_proj
534
+ transformer.blocks.29.ffn.act
535
+ transformer.blocks.29.ffn.down_proj
536
+ transformer.blocks.29.resid_attn_dropout
537
+ transformer.blocks.30
538
+ transformer.blocks.30.norm_1
539
+ transformer.blocks.30.attn
540
+ transformer.blocks.30.attn.Wqkv
541
+ transformer.blocks.30.attn.out_proj
542
+ transformer.blocks.30.norm_2
543
+ transformer.blocks.30.ffn
544
+ transformer.blocks.30.ffn.up_proj
545
+ transformer.blocks.30.ffn.act
546
+ transformer.blocks.30.ffn.down_proj
547
+ transformer.blocks.30.resid_attn_dropout
548
+ transformer.blocks.31
549
+ transformer.blocks.31.norm_1
550
+ transformer.blocks.31.attn
551
+ transformer.blocks.31.attn.Wqkv
552
+ transformer.blocks.31.attn.out_proj
553
+ transformer.blocks.31.norm_2
554
+ transformer.blocks.31.ffn
555
+ transformer.blocks.31.ffn.up_proj
556
+ transformer.blocks.31.ffn.act
557
+ transformer.blocks.31.ffn.down_proj
558
+ transformer.blocks.31.resid_attn_dropout
559
+ transformer.norm_f
560
+ lm_head
561
+ >>>> WARNING!!! Patching NNCF config to the following:
562
+ Using framework PyTorch: 2.1.1+cpu
563
+ Overriding 1 configuration item(s)
564
+ - use_cache -> True
565
+ /home/ubuntu/miniconda3/envs/deepsparse/lib/python3.10/site-packages/transformers/models/mpt/modeling_mpt.py:424: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
566
+ if input_shape[1] + past_key_values_length != attention_mask.shape[1]:
567
+ /home/ubuntu/miniconda3/envs/deepsparse/lib/python3.10/site-packages/transformers/models/mpt/modeling_mpt.py:434: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
568
+ if src_length > 1:
569
+ /home/ubuntu/miniconda3/envs/deepsparse/lib/python3.10/site-packages/transformers/models/mpt/modeling_mpt.py:71: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
570
+ if past_key_values_length > 0:
571
+ /home/ubuntu/miniconda3/envs/deepsparse/lib/python3.10/site-packages/transformers/models/mpt/modeling_mpt.py:164: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
572
+ position_bias_query_index = max(0, position_bias.size(1) - query_length)
573
+ /home/ubuntu/miniconda3/envs/deepsparse/lib/python3.10/site-packages/transformers/models/mpt/modeling_mpt.py:165: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
574
+ position_bias_key_index = max(0, position_bias.size(2) - key_length)
575
+ Configuration saved in models/neuralmagic/mpt-7b-gsm8k-pt/w8a8/openvino_config.json
576
+ {'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': None, 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': True, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'chunk_size_feed_forward': 0, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': None, 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'bos_token_id': None, 'pad_token_id': None, 'eos_token_id': None, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': '', '_commit_hash': None, 'transformers_version': None, 'compression': {'algorithm': 'quantization', 'preset': 'performance', 'overflow_fix': 'disable', 'initializer': {'range': {'num_init_samples': 4, 'type': 'mean_min_max'}, 'batchnorm_adaptation': {'num_bn_adaptation_samples': 0}}, 'scope_overrides': {'activations': {'{re}.*matmul_0': {'mode': 'symmetric'}}}, 'ignored_scopes': ['{re}.*layer_norm_.*'], 'export_to_onnx_standard_ops': False}, 'input_info': [{'sample_size': [1, 8], 'type': 'long', 'keyword': 'input_ids'}, {'sample_size': [1, 8], 'type': 'long', 'keyword': 'attention_mask'}], 'save_onnx_model': False, 'optimum_version': None, 'log_dir': 'models/neuralmagic/mpt-7b-gsm8k-pt/w8a8', 'target_device': 'CPU'}
577
+ name shape type sparsity
578
+ 0 Constant_172536 [50432, 4096] <Type: 'int8_t'> 0.013229
579
+ 1 Constant_172538 [50432, 1] <Type: 'float32'> 0.000000
580
+ 2 Constant_165621 [1, 1, 4096] <Type: 'float32'> 0.000000
581
+ 3 Constant_172540 [12288, 4096] <Type: 'int8_t'> 0.034184
582
+ 4 Constant_172542 [12288, 1] <Type: 'float32'> 0.000000
583
+ 5 __module.model.transformer.blocks.0.attn/aten::slice/Slice_2965 [32, 1, 2048] <Type: 'float32'> 0.000000
584
+ 6 Constant_172544 [4096, 4096] <Type: 'int8_t'> 0.034051
585
+ 7 Constant_172546 [4096, 1] <Type: 'float32'> 0.000000
586
+ 8 Constant_165659 [1, 1, 4096] <Type: 'float32'> 0.000000
587
+ 9 Constant_172548 [16384, 4096] <Type: 'int8_t'> 0.015438
588
+ 10 Constant_172550 [16384, 1] <Type: 'float32'> 0.000000
589
+ 11 Constant_172552 [4096, 16384] <Type: 'int8_t'> 0.179220
590
+ 12 Constant_172554 [4096, 1] <Type: 'float32'> 0.000000
591
+ 13 Constant_165676 [1, 1, 4096] <Type: 'float32'> 0.000000
592
+ 14 Constant_172556 [12288, 4096] <Type: 'int8_t'> 0.023685
593
+ 15 Constant_172558 [12288, 1] <Type: 'float32'> 0.000000
594
+ 16 Constant_172560 [4096, 4096] <Type: 'int8_t'> 0.035988
595
+ 17 Constant_172562 [4096, 1] <Type: 'float32'> 0.000000
596
+ 18 Constant_165714 [1, 1, 4096] <Type: 'float32'> 0.000000
597
+ 19 Constant_172564 [16384, 4096] <Type: 'int8_t'> 0.016436
598
+ 20 Constant_172566 [16384, 1] <Type: 'float32'> 0.000000
599
+ 21 Constant_172568 [4096, 16384] <Type: 'int8_t'> 0.024194
600
+ 22 Constant_172570 [4096, 1] <Type: 'float32'> 0.000000
601
+ 23 Constant_165731 [1, 1, 4096] <Type: 'float32'> 0.000000
602
+ 24 Constant_172572 [12288, 4096] <Type: 'int8_t'> 0.022408
603
+ 25 Constant_172574 [12288, 1] <Type: 'float32'> 0.000000
604
+ 26 Constant_172576 [4096, 4096] <Type: 'int8_t'> 0.036617
605
+ 27 Constant_172578 [4096, 1] <Type: 'float32'> 0.000000
606
+ 28 Constant_165769 [1, 1, 4096] <Type: 'float32'> 0.000000
607
+ 29 Constant_172580 [16384, 4096] <Type: 'int8_t'> 0.017184
608
+ 30 Constant_172582 [16384, 1] <Type: 'float32'> 0.000000
609
+ 31 Constant_172584 [4096, 16384] <Type: 'int8_t'> 0.028299
610
+ 32 Constant_172586 [4096, 1] <Type: 'float32'> 0.000000
611
+ 33 Constant_165786 [1, 1, 4096] <Type: 'float32'> 0.000000
612
+ 34 Constant_172588 [12288, 4096] <Type: 'int8_t'> 0.021993
613
+ 35 Constant_172590 [12288, 1] <Type: 'float32'> 0.000000
614
+ 36 Constant_172592 [4096, 4096] <Type: 'int8_t'> 0.031169
615
+ 37 Constant_172594 [4096, 1] <Type: 'float32'> 0.000000
616
+ 38 Constant_165824 [1, 1, 4096] <Type: 'float32'> 0.000000
617
+ 39 Constant_172596 [16384, 4096] <Type: 'int8_t'> 0.017389
618
+ 40 Constant_172598 [16384, 1] <Type: 'float32'> 0.000000
619
+ 41 Constant_172600 [4096, 16384] <Type: 'int8_t'> 0.024672
620
+ 42 Constant_172602 [4096, 1] <Type: 'float32'> 0.000000
621
+ 43 Constant_165841 [1, 1, 4096] <Type: 'float32'> 0.000000
622
+ 44 Constant_172604 [12288, 4096] <Type: 'int8_t'> 0.021743
623
+ 45 Constant_172606 [12288, 1] <Type: 'float32'> 0.000000
624
+ 46 Constant_172608 [4096, 4096] <Type: 'int8_t'> 0.030760
625
+ 47 Constant_172610 [4096, 1] <Type: 'float32'> 0.000000
626
+ 48 Constant_165879 [1, 1, 4096] <Type: 'float32'> 0.000000
627
+ 49 Constant_172612 [16384, 4096] <Type: 'int8_t'> 0.017657
628
+ 50 Constant_172614 [16384, 1] <Type: 'float32'> 0.000000
629
+ 51 Constant_172616 [4096, 16384] <Type: 'int8_t'> 0.024935
630
+ 52 Constant_172618 [4096, 1] <Type: 'float32'> 0.000000
631
+ 53 Constant_165896 [1, 1, 4096] <Type: 'float32'> 0.000000
632
+ 54 Constant_172620 [12288, 4096] <Type: 'int8_t'> 0.020559
633
+ 55 Constant_172622 [12288, 1] <Type: 'float32'> 0.000000
634
+ 56 Constant_172624 [4096, 4096] <Type: 'int8_t'> 0.028402
635
+ 57 Constant_172626 [4096, 1] <Type: 'float32'> 0.000000
636
+ 58 Constant_165934 [1, 1, 4096] <Type: 'float32'> 0.000000
637
+ 59 Constant_172628 [16384, 4096] <Type: 'int8_t'> 0.018163
638
+ 60 Constant_172630 [16384, 1] <Type: 'float32'> 0.000000
639
+ 61 Constant_172632 [4096, 16384] <Type: 'int8_t'> 0.025113
640
+ 62 Constant_172634 [4096, 1] <Type: 'float32'> 0.000000
641
+ 63 Constant_165951 [1, 1, 4096] <Type: 'float32'> 0.000000
642
+ 64 Constant_172636 [12288, 4096] <Type: 'int8_t'> 0.020458
643
+ 65 Constant_172638 [12288, 1] <Type: 'float32'> 0.000000
644
+ 66 Constant_172640 [4096, 4096] <Type: 'int8_t'> 0.027694
645
+ 67 Constant_172642 [4096, 1] <Type: 'float32'> 0.000000
646
+ 68 Constant_165989 [1, 1, 4096] <Type: 'float32'> 0.000000
647
+ 69 Constant_172644 [16384, 4096] <Type: 'int8_t'> 0.018449
648
+ 70 Constant_172646 [16384, 1] <Type: 'float32'> 0.000000
649
+ 71 Constant_172648 [4096, 16384] <Type: 'int8_t'> 0.025266
650
+ 72 Constant_172650 [4096, 1] <Type: 'float32'> 0.000000
651
+ 73 Constant_166006 [1, 1, 4096] <Type: 'float32'> 0.000000
652
+ 74 Constant_172652 [12288, 4096] <Type: 'int8_t'> 0.020351
653
+ 75 Constant_172654 [12288, 1] <Type: 'float32'> 0.000000
654
+ 76 Constant_172656 [4096, 4096] <Type: 'int8_t'> 0.027611
655
+ 77 Constant_172658 [4096, 1] <Type: 'float32'> 0.000000
656
+ 78 Constant_166044 [1, 1, 4096] <Type: 'float32'> 0.000000
657
+ 79 Constant_172660 [16384, 4096] <Type: 'int8_t'> 0.018578
658
+ 80 Constant_172662 [16384, 1] <Type: 'float32'> 0.000000
659
+ 81 Constant_172664 [4096, 16384] <Type: 'int8_t'> 0.034481
660
+ 82 Constant_172666 [4096, 1] <Type: 'float32'> 0.000000
661
+ 83 Constant_166061 [1, 1, 4096] <Type: 'float32'> 0.000000
662
+ 84 Constant_172668 [12288, 4096] <Type: 'int8_t'> 0.020421
663
+ 85 Constant_172670 [12288, 1] <Type: 'float32'> 0.000000
664
+ 86 Constant_172672 [4096, 4096] <Type: 'int8_t'> 0.026674
665
+ 87 Constant_172674 [4096, 1] <Type: 'float32'> 0.000000
666
+ 88 Constant_166099 [1, 1, 4096] <Type: 'float32'> 0.000000
667
+ 89 Constant_172676 [16384, 4096] <Type: 'int8_t'> 0.018748
668
+ 90 Constant_172678 [16384, 1] <Type: 'float32'> 0.000000
669
+ 91 Constant_172680 [4096, 16384] <Type: 'int8_t'> 0.024820
670
+ 92 Constant_172682 [4096, 1] <Type: 'float32'> 0.000000
671
+ 93 Constant_166116 [1, 1, 4096] <Type: 'float32'> 0.000000
672
+ 94 Constant_172684 [12288, 4096] <Type: 'int8_t'> 0.020432
673
+ 95 Constant_172686 [12288, 1] <Type: 'float32'> 0.000000
674
+ 96 Constant_172688 [4096, 4096] <Type: 'int8_t'> 0.027259
675
+ 97 Constant_172690 [4096, 1] <Type: 'float32'> 0.000000
676
+ 98 Constant_166154 [1, 1, 4096] <Type: 'float32'> 0.000000
677
+ 99 Constant_172692 [16384, 4096] <Type: 'int8_t'> 0.018932
678
+ 100 Constant_172694 [16384, 1] <Type: 'float32'> 0.000000
679
+ 101 Constant_172696 [4096, 16384] <Type: 'int8_t'> 0.097153
680
+ 102 Constant_172698 [4096, 1] <Type: 'float32'> 0.000000
681
+ 103 Constant_166171 [1, 1, 4096] <Type: 'float32'> 0.000000
682
+ 104 Constant_172700 [12288, 4096] <Type: 'int8_t'> 0.020314
683
+ 105 Constant_172702 [12288, 1] <Type: 'float32'> 0.000000
684
+ 106 Constant_172704 [4096, 4096] <Type: 'int8_t'> 0.026722
685
+ 107 Constant_172706 [4096, 1] <Type: 'float32'> 0.000000
686
+ 108 Constant_166209 [1, 1, 4096] <Type: 'float32'> 0.000000
687
+ 109 Constant_172708 [16384, 4096] <Type: 'int8_t'> 0.019104
688
+ 110 Constant_172710 [16384, 1] <Type: 'float32'> 0.000000
689
+ 111 Constant_172712 [4096, 16384] <Type: 'int8_t'> 0.025015
690
+ 112 Constant_172714 [4096, 1] <Type: 'float32'> 0.000000
691
+ 113 Constant_166226 [1, 1, 4096] <Type: 'float32'> 0.000000
692
+ 114 Constant_172716 [12288, 4096] <Type: 'int8_t'> 0.020237
693
+ 115 Constant_172718 [12288, 1] <Type: 'float32'> 0.000000
694
+ 116 Constant_172720 [4096, 4096] <Type: 'int8_t'> 0.026993
695
+ 117 Constant_172722 [4096, 1] <Type: 'float32'> 0.000000
696
+ 118 Constant_166264 [1, 1, 4096] <Type: 'float32'> 0.000000
697
+ 119 Constant_172724 [16384, 4096] <Type: 'int8_t'> 0.019208
698
+ 120 Constant_172726 [16384, 1] <Type: 'float32'> 0.000000
699
+ 121 Constant_172728 [4096, 16384] <Type: 'int8_t'> 0.025477
700
+ 122 Constant_172730 [4096, 1] <Type: 'float32'> 0.000000
701
+ 123 Constant_166281 [1, 1, 4096] <Type: 'float32'> 0.000000
702
+ 124 Constant_172732 [12288, 4096] <Type: 'int8_t'> 0.020467
703
+ 125 Constant_172734 [12288, 1] <Type: 'float32'> 0.000000
704
+ 126 Constant_172736 [4096, 4096] <Type: 'int8_t'> 0.026980
705
+ 127 Constant_172738 [4096, 1] <Type: 'float32'> 0.000000
706
+ 128 Constant_166319 [1, 1, 4096] <Type: 'float32'> 0.000000
707
+ 129 Constant_172740 [16384, 4096] <Type: 'int8_t'> 0.019221
708
+ 130 Constant_172742 [16384, 1] <Type: 'float32'> 0.000000
709
+ 131 Constant_172744 [4096, 16384] <Type: 'int8_t'> 0.025053
710
+ 132 Constant_172746 [4096, 1] <Type: 'float32'> 0.000000
711
+ 133 Constant_166336 [1, 1, 4096] <Type: 'float32'> 0.000000
712
+ 134 Constant_172748 [12288, 4096] <Type: 'int8_t'> 0.020501
713
+ 135 Constant_172750 [12288, 1] <Type: 'float32'> 0.000000
714
+ 136 Constant_172752 [4096, 4096] <Type: 'int8_t'> 0.027188
715
+ 137 Constant_172754 [4096, 1] <Type: 'float32'> 0.000000
716
+ 138 Constant_166374 [1, 1, 4096] <Type: 'float32'> 0.000000
717
+ 139 Constant_172756 [16384, 4096] <Type: 'int8_t'> 0.019351
718
+ 140 Constant_172758 [16384, 1] <Type: 'float32'> 0.000000
719
+ 141 Constant_172760 [4096, 16384] <Type: 'int8_t'> 0.025049
720
+ 142 Constant_172762 [4096, 1] <Type: 'float32'> 0.000000
721
+ 143 Constant_166391 [1, 1, 4096] <Type: 'float32'> 0.000000
722
+ 144 Constant_172764 [12288, 4096] <Type: 'int8_t'> 0.020253
723
+ 145 Constant_172766 [12288, 1] <Type: 'float32'> 0.000000
724
+ 146 Constant_172768 [4096, 4096] <Type: 'int8_t'> 0.025943
725
+ 147 Constant_172770 [4096, 1] <Type: 'float32'> 0.000000
726
+ 148 Constant_166429 [1, 1, 4096] <Type: 'float32'> 0.000000
727
+ 149 Constant_172772 [16384, 4096] <Type: 'int8_t'> 0.019464
728
+ 150 Constant_172774 [16384, 1] <Type: 'float32'> 0.000000
729
+ 151 Constant_172776 [4096, 16384] <Type: 'int8_t'> 0.025207
730
+ 152 Constant_172778 [4096, 1] <Type: 'float32'> 0.000000
731
+ 153 Constant_166446 [1, 1, 4096] <Type: 'float32'> 0.000000
732
+ 154 Constant_172780 [12288, 4096] <Type: 'int8_t'> 0.020326
733
+ 155 Constant_172782 [12288, 1] <Type: 'float32'> 0.000000
734
+ 156 Constant_172784 [4096, 4096] <Type: 'int8_t'> 0.026164
735
+ 157 Constant_172786 [4096, 1] <Type: 'float32'> 0.000000
736
+ 158 Constant_166484 [1, 1, 4096] <Type: 'float32'> 0.000000
737
+ 159 Constant_172788 [16384, 4096] <Type: 'int8_t'> 0.019670
738
+ 160 Constant_172790 [16384, 1] <Type: 'float32'> 0.000000
739
+ 161 Constant_172792 [4096, 16384] <Type: 'int8_t'> 0.025884
740
+ 162 Constant_172794 [4096, 1] <Type: 'float32'> 0.000000
741
+ 163 Constant_166501 [1, 1, 4096] <Type: 'float32'> 0.000000
742
+ 164 Constant_172796 [12288, 4096] <Type: 'int8_t'> 0.020532
743
+ 165 Constant_172798 [12288, 1] <Type: 'float32'> 0.000000
744
+ 166 Constant_172800 [4096, 4096] <Type: 'int8_t'> 0.026195
745
+ 167 Constant_172802 [4096, 1] <Type: 'float32'> 0.000000
746
+ 168 Constant_166539 [1, 1, 4096] <Type: 'float32'> 0.000000
747
+ 169 Constant_172804 [16384, 4096] <Type: 'int8_t'> 0.019645
748
+ 170 Constant_172806 [16384, 1] <Type: 'float32'> 0.000000
749
+ 171 Constant_172808 [4096, 16384] <Type: 'int8_t'> 0.025279
750
+ 172 Constant_172810 [4096, 1] <Type: 'float32'> 0.000000
751
+ 173 Constant_166556 [1, 1, 4096] <Type: 'float32'> 0.000000
752
+ 174 Constant_172812 [12288, 4096] <Type: 'int8_t'> 0.020482
753
+ 175 Constant_172814 [12288, 1] <Type: 'float32'> 0.000000
754
+ 176 Constant_172816 [4096, 4096] <Type: 'int8_t'> 0.026548
755
+ 177 Constant_172818 [4096, 1] <Type: 'float32'> 0.000000
756
+ 178 Constant_166594 [1, 1, 4096] <Type: 'float32'> 0.000000
757
+ 179 Constant_172820 [16384, 4096] <Type: 'int8_t'> 0.019645
758
+ 180 Constant_172822 [16384, 1] <Type: 'float32'> 0.000000
759
+ 181 Constant_172824 [4096, 16384] <Type: 'int8_t'> 0.025222
760
+ 182 Constant_172826 [4096, 1] <Type: 'float32'> 0.000000
761
+ 183 Constant_166611 [1, 1, 4096] <Type: 'float32'> 0.000000
762
+ 184 Constant_172828 [12288, 4096] <Type: 'int8_t'> 0.020014
763
+ 185 Constant_172830 [12288, 1] <Type: 'float32'> 0.000000
764
+ 186 Constant_172832 [4096, 4096] <Type: 'int8_t'> 0.026339
765
+ 187 Constant_172834 [4096, 1] <Type: 'float32'> 0.000000
766
+ 188 Constant_166649 [1, 1, 4096] <Type: 'float32'> 0.000000
767
+ 189 Constant_172836 [16384, 4096] <Type: 'int8_t'> 0.019627
768
+ 190 Constant_172838 [16384, 1] <Type: 'float32'> 0.000000
769
+ 191 Constant_172840 [4096, 16384] <Type: 'int8_t'> 0.025042
770
+ 192 Constant_172842 [4096, 1] <Type: 'float32'> 0.000000
771
+ 193 Constant_166666 [1, 1, 4096] <Type: 'float32'> 0.000000
772
+ 194 Constant_172844 [12288, 4096] <Type: 'int8_t'> 0.020141
773
+ 195 Constant_172846 [12288, 1] <Type: 'float32'> 0.000000
774
+ 196 Constant_172848 [4096, 4096] <Type: 'int8_t'> 0.025817
775
+ 197 Constant_172850 [4096, 1] <Type: 'float32'> 0.000000
776
+ 198 Constant_166704 [1, 1, 4096] <Type: 'float32'> 0.000000
777
+ 199 Constant_172852 [16384, 4096] <Type: 'int8_t'> 0.019474
778
+ 200 Constant_172854 [16384, 1] <Type: 'float32'> 0.000000
779
+ 201 Constant_172856 [4096, 16384] <Type: 'int8_t'> 0.024555
780
+ 202 Constant_172858 [4096, 1] <Type: 'float32'> 0.000000
781
+ 203 Constant_166721 [1, 1, 4096] <Type: 'float32'> 0.000000
782
+ 204 Constant_172860 [12288, 4096] <Type: 'int8_t'> 0.020283
783
+ 205 Constant_172862 [12288, 1] <Type: 'float32'> 0.000000
784
+ 206 Constant_172864 [4096, 4096] <Type: 'int8_t'> 0.026397
785
+ 207 Constant_172866 [4096, 1] <Type: 'float32'> 0.000000
786
+ 208 Constant_166759 [1, 1, 4096] <Type: 'float32'> 0.000000
787
+ 209 Constant_172868 [16384, 4096] <Type: 'int8_t'> 0.019443
788
+ 210 Constant_172870 [16384, 1] <Type: 'float32'> 0.000000
789
+ 211 Constant_172872 [4096, 16384] <Type: 'int8_t'> 0.024146
790
+ 212 Constant_172874 [4096, 1] <Type: 'float32'> 0.000000
791
+ 213 Constant_166776 [1, 1, 4096] <Type: 'float32'> 0.000000
792
+ 214 Constant_172876 [12288, 4096] <Type: 'int8_t'> 0.020223
793
+ 215 Constant_172878 [12288, 1] <Type: 'float32'> 0.000000
794
+ 216 Constant_172880 [4096, 4096] <Type: 'int8_t'> 0.026004
795
+ 217 Constant_172882 [4096, 1] <Type: 'float32'> 0.000000
796
+ 218 Constant_166814 [1, 1, 4096] <Type: 'float32'> 0.000000
797
+ 219 Constant_172884 [16384, 4096] <Type: 'int8_t'> 0.019259
798
+ 220 Constant_172886 [16384, 1] <Type: 'float32'> 0.000000
799
+ 221 Constant_172888 [4096, 16384] <Type: 'int8_t'> 0.023658
800
+ 222 Constant_172890 [4096, 1] <Type: 'float32'> 0.000000
801
+ 223 Constant_166831 [1, 1, 4096] <Type: 'float32'> 0.000000
802
+ 224 Constant_172892 [12288, 4096] <Type: 'int8_t'> 0.020529
803
+ 225 Constant_172894 [12288, 1] <Type: 'float32'> 0.000000
804
+ 226 Constant_172896 [4096, 4096] <Type: 'int8_t'> 0.027820
805
+ 227 Constant_172898 [4096, 1] <Type: 'float32'> 0.000000
806
+ 228 Constant_166869 [1, 1, 4096] <Type: 'float32'> 0.000000
807
+ 229 Constant_172900 [16384, 4096] <Type: 'int8_t'> 0.019257
808
+ 230 Constant_172902 [16384, 1] <Type: 'float32'> 0.000000
809
+ 231 Constant_172904 [4096, 16384] <Type: 'int8_t'> 0.023891
810
+ 232 Constant_172906 [4096, 1] <Type: 'float32'> 0.000000
811
+ 233 Constant_166886 [1, 1, 4096] <Type: 'float32'> 0.000000
812
+ 234 Constant_172908 [12288, 4096] <Type: 'int8_t'> 0.020522
813
+ 235 Constant_172910 [12288, 1] <Type: 'float32'> 0.000000
814
+ 236 Constant_172912 [4096, 4096] <Type: 'int8_t'> 0.025421
815
+ 237 Constant_172914 [4096, 1] <Type: 'float32'> 0.000000
816
+ 238 Constant_166924 [1, 1, 4096] <Type: 'float32'> 0.000000
817
+ 239 Constant_172916 [16384, 4096] <Type: 'int8_t'> 0.019101
818
+ 240 Constant_172918 [16384, 1] <Type: 'float32'> 0.000000
819
+ 241 Constant_172920 [4096, 16384] <Type: 'int8_t'> 0.023139
820
+ 242 Constant_172922 [4096, 1] <Type: 'float32'> 0.000000
821
+ 243 Constant_166941 [1, 1, 4096] <Type: 'float32'> 0.000000
822
+ 244 Constant_172924 [12288, 4096] <Type: 'int8_t'> 0.020493
823
+ 245 Constant_172926 [12288, 1] <Type: 'float32'> 0.000000
824
+ 246 Constant_172928 [4096, 4096] <Type: 'int8_t'> 0.025315
825
+ 247 Constant_172930 [4096, 1] <Type: 'float32'> 0.000000
826
+ 248 Constant_166979 [1, 1, 4096] <Type: 'float32'> 0.000000
827
+ 249 Constant_172932 [16384, 4096] <Type: 'int8_t'> 0.019065
828
+ 250 Constant_172934 [16384, 1] <Type: 'float32'> 0.000000
829
+ 251 Constant_172936 [4096, 16384] <Type: 'int8_t'> 0.022510
830
+ 252 Constant_172938 [4096, 1] <Type: 'float32'> 0.000000
831
+ 253 Constant_166996 [1, 1, 4096] <Type: 'float32'> 0.000000
832
+ 254 Constant_172940 [12288, 4096] <Type: 'int8_t'> 0.020720
833
+ 255 Constant_172942 [12288, 1] <Type: 'float32'> 0.000000
834
+ 256 Constant_172944 [4096, 4096] <Type: 'int8_t'> 0.026040
835
+ 257 Constant_172946 [4096, 1] <Type: 'float32'> 0.000000
836
+ 258 Constant_167034 [1, 1, 4096] <Type: 'float32'> 0.000000
837
+ 259 Constant_172948 [16384, 4096] <Type: 'int8_t'> 0.018880
838
+ 260 Constant_172950 [16384, 1] <Type: 'float32'> 0.000000
839
+ 261 Constant_172952 [4096, 16384] <Type: 'int8_t'> 0.021828
840
+ 262 Constant_172954 [4096, 1] <Type: 'float32'> 0.000000
841
+ 263 Constant_167051 [1, 1, 4096] <Type: 'float32'> 0.000000
842
+ 264 Constant_172956 [12288, 4096] <Type: 'int8_t'> 0.021428
843
+ 265 Constant_172958 [12288, 1] <Type: 'float32'> 0.000000
844
+ 266 Constant_172960 [4096, 4096] <Type: 'int8_t'> 0.025231
845
+ 267 Constant_172962 [4096, 1] <Type: 'float32'> 0.000000
846
+ 268 Constant_167089 [1, 1, 4096] <Type: 'float32'> 0.000000
847
+ 269 Constant_172964 [16384, 4096] <Type: 'int8_t'> 0.018872
848
+ 270 Constant_172966 [16384, 1] <Type: 'float32'> 0.000000
849
+ 271 Constant_172968 [4096, 16384] <Type: 'int8_t'> 0.021447
850
+ 272 Constant_172970 [4096, 1] <Type: 'float32'> 0.000000
851
+ 273 Constant_167106 [1, 1, 4096] <Type: 'float32'> 0.000000
852
+ 274 Constant_172972 [12288, 4096] <Type: 'int8_t'> 0.020942
853
+ 275 Constant_172974 [12288, 1] <Type: 'float32'> 0.000000
854
+ 276 Constant_172976 [4096, 4096] <Type: 'int8_t'> 0.025028
855
+ 277 Constant_172978 [4096, 1] <Type: 'float32'> 0.000000
856
+ 278 Constant_167144 [1, 1, 4096] <Type: 'float32'> 0.000000
857
+ 279 Constant_172980 [16384, 4096] <Type: 'int8_t'> 0.018863
858
+ 280 Constant_172982 [16384, 1] <Type: 'float32'> 0.000000
859
+ 281 Constant_172984 [4096, 16384] <Type: 'int8_t'> 0.021032
860
+ 282 Constant_172986 [4096, 1] <Type: 'float32'> 0.000000
861
+ 283 Constant_167161 [1, 1, 4096] <Type: 'float32'> 0.000000
862
+ 284 Constant_172988 [12288, 4096] <Type: 'int8_t'> 0.020559
863
+ 285 Constant_172990 [12288, 1] <Type: 'float32'> 0.000000
864
+ 286 Constant_172992 [4096, 4096] <Type: 'int8_t'> 0.023134
865
+ 287 Constant_172994 [4096, 1] <Type: 'float32'> 0.000000
866
+ 288 Constant_167199 [1, 1, 4096] <Type: 'float32'> 0.000000
867
+ 289 Constant_172996 [16384, 4096] <Type: 'int8_t'> 0.018861
868
+ 290 Constant_172998 [16384, 1] <Type: 'float32'> 0.000000
869
+ 291 Constant_173000 [4096, 16384] <Type: 'int8_t'> 0.020515
870
+ 292 Constant_173002 [4096, 1] <Type: 'float32'> 0.000000
871
+ 293 Constant_167216 [1, 1, 4096] <Type: 'float32'> 0.000000
872
+ 294 Constant_173004 [12288, 4096] <Type: 'int8_t'> 0.020131
873
+ 295 Constant_173006 [12288, 1] <Type: 'float32'> 0.000000
874
+ 296 Constant_173008 [4096, 4096] <Type: 'int8_t'> 0.023759
875
+ 297 Constant_173010 [4096, 1] <Type: 'float32'> 0.000000
876
+ 298 Constant_167254 [1, 1, 4096] <Type: 'float32'> 0.000000
877
+ 299 Constant_173012 [16384, 4096] <Type: 'int8_t'> 0.018787
878
+ 300 Constant_173014 [16384, 1] <Type: 'float32'> 0.000000
879
+ 301 Constant_173016 [4096, 16384] <Type: 'int8_t'> 0.020463
880
+ 302 Constant_173018 [4096, 1] <Type: 'float32'> 0.000000
881
+ 303 Constant_167271 [1, 1, 4096] <Type: 'float32'> 0.000000
882
+ 304 Constant_173020 [12288, 4096] <Type: 'int8_t'> 0.019650
883
+ 305 Constant_173022 [12288, 1] <Type: 'float32'> 0.000000
884
+ 306 Constant_173024 [4096, 4096] <Type: 'int8_t'> 0.023612
885
+ 307 Constant_173026 [4096, 1] <Type: 'float32'> 0.000000
886
+ 308 Constant_167309 [1, 1, 4096] <Type: 'float32'> 0.000000
887
+ 309 Constant_173028 [16384, 4096] <Type: 'int8_t'> 0.017992
888
+ 310 Constant_173030 [16384, 1] <Type: 'float32'> 0.000000
889
+ 311 Constant_173032 [4096, 16384] <Type: 'int8_t'> 0.021538
890
+ 312 Constant_173034 [4096, 1] <Type: 'float32'> 0.000000
891
+ 313 Constant_167326 [1, 1, 4096] <Type: 'float32'> 0.000000
892
+ 314 Constant_173036 [12288, 4096] <Type: 'int8_t'> 0.019520
893
+ 315 Constant_173038 [12288, 1] <Type: 'float32'> 0.000000
894
+ 316 Constant_173040 [4096, 4096] <Type: 'int8_t'> 0.024048
895
+ 317 Constant_173042 [4096, 1] <Type: 'float32'> 0.000000
896
+ 318 Constant_167364 [1, 1, 4096] <Type: 'float32'> 0.000000
897
+ 319 Constant_173044 [16384, 4096] <Type: 'int8_t'> 0.021858
898
+ 320 Constant_173046 [16384, 1] <Type: 'float32'> 0.000000
899
+ 321 Constant_173048 [4096, 16384] <Type: 'int8_t'> 0.041241
900
+ 322 Constant_173050 [4096, 1] <Type: 'float32'> 0.000000
901
+ 323 Constant_167381 [1, 1, 4096] <Type: 'float32'> 0.000000
902
+ 324 Constant_173052 [50432, 4096] <Type: 'int8_t'> 0.013229
903
+ 325 Constant_173054 [50432, 1] <Type: 'float32'> 0.000000
models/mpt-7b-gsm8k-pt/w8a8/openvino_config.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "compression": {
3
+ "algorithm": "quantization",
4
+ "export_to_onnx_standard_ops": false,
5
+ "ignored_scopes": [
6
+ "{re}.*layer_norm_.*"
7
+ ],
8
+ "initializer": {
9
+ "batchnorm_adaptation": {
10
+ "num_bn_adaptation_samples": 0
11
+ },
12
+ "range": {
13
+ "num_init_samples": 4,
14
+ "type": "mean_min_max"
15
+ }
16
+ },
17
+ "overflow_fix": "disable",
18
+ "preset": "performance",
19
+ "scope_overrides": {
20
+ "activations": {
21
+ "{re}.*matmul_0": {
22
+ "mode": "symmetric"
23
+ }
24
+ }
25
+ }
26
+ },
27
+ "input_info": [
28
+ {
29
+ "keyword": "input_ids",
30
+ "sample_size": [
31
+ 1,
32
+ 8
33
+ ],
34
+ "type": "long"
35
+ },
36
+ {
37
+ "keyword": "attention_mask",
38
+ "sample_size": [
39
+ 1,
40
+ 8
41
+ ],
42
+ "type": "long"
43
+ }
44
+ ],
45
+ "log_dir": "models/neuralmagic/mpt-7b-gsm8k-pt/w8a8",
46
+ "optimum_version": "1.14.1",
47
+ "save_onnx_model": false,
48
+ "target_device": "CPU",
49
+ "transformers_version": "4.34.1"
50
+ }
models/mpt-7b-gsm8k-pt/w8a8/openvino_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72d92fbbc40bfbf456c55fb6817ae3a26bfe454b82a954b3c462d5f8ed6c1553
3
+ size 6655271181
models/mpt-7b-gsm8k-pt/w8a8/openvino_model.xml ADDED
The diff for this file is too large to render. See raw diff
 
models/mpt-7b-gsm8k-pt/w8a8/original_graph.dot ADDED
The diff for this file is too large to render. See raw diff
 
models/mpt-7b-gsm8k-pt/w8a8/ov_weights_type.md ADDED
@@ -0,0 +1,328 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ | | name | shape | type | sparsity |
2
+ |----:|:----------------------------------------------------------------|:-------------|:------------------|-----------:|
3
+ | 0 | Constant_172536 | [50432,4096] | <Type: 'int8_t'> | 0.0132285 |
4
+ | 1 | Constant_172538 | [50432,1] | <Type: 'float32'> | 0 |
5
+ | 2 | Constant_165621 | [1,1,4096] | <Type: 'float32'> | 0 |
6
+ | 3 | Constant_172540 | [12288,4096] | <Type: 'int8_t'> | 0.0341836 |
7
+ | 4 | Constant_172542 | [12288,1] | <Type: 'float32'> | 0 |
8
+ | 5 | __module.model.transformer.blocks.0.attn/aten::slice/Slice_2965 | [32,1,2048] | <Type: 'float32'> | 0 |
9
+ | 6 | Constant_172544 | [4096,4096] | <Type: 'int8_t'> | 0.0340514 |
10
+ | 7 | Constant_172546 | [4096,1] | <Type: 'float32'> | 0 |
11
+ | 8 | Constant_165659 | [1,1,4096] | <Type: 'float32'> | 0 |
12
+ | 9 | Constant_172548 | [16384,4096] | <Type: 'int8_t'> | 0.0154377 |
13
+ | 10 | Constant_172550 | [16384,1] | <Type: 'float32'> | 0 |
14
+ | 11 | Constant_172552 | [4096,16384] | <Type: 'int8_t'> | 0.17922 |
15
+ | 12 | Constant_172554 | [4096,1] | <Type: 'float32'> | 0 |
16
+ | 13 | Constant_165676 | [1,1,4096] | <Type: 'float32'> | 0 |
17
+ | 14 | Constant_172556 | [12288,4096] | <Type: 'int8_t'> | 0.0236854 |
18
+ | 15 | Constant_172558 | [12288,1] | <Type: 'float32'> | 0 |
19
+ | 16 | Constant_172560 | [4096,4096] | <Type: 'int8_t'> | 0.0359877 |
20
+ | 17 | Constant_172562 | [4096,1] | <Type: 'float32'> | 0 |
21
+ | 18 | Constant_165714 | [1,1,4096] | <Type: 'float32'> | 0 |
22
+ | 19 | Constant_172564 | [16384,4096] | <Type: 'int8_t'> | 0.0164362 |
23
+ | 20 | Constant_172566 | [16384,1] | <Type: 'float32'> | 0 |
24
+ | 21 | Constant_172568 | [4096,16384] | <Type: 'int8_t'> | 0.0241944 |
25
+ | 22 | Constant_172570 | [4096,1] | <Type: 'float32'> | 0 |
26
+ | 23 | Constant_165731 | [1,1,4096] | <Type: 'float32'> | 0 |
27
+ | 24 | Constant_172572 | [12288,4096] | <Type: 'int8_t'> | 0.0224082 |
28
+ | 25 | Constant_172574 | [12288,1] | <Type: 'float32'> | 0 |
29
+ | 26 | Constant_172576 | [4096,4096] | <Type: 'int8_t'> | 0.0366175 |
30
+ | 27 | Constant_172578 | [4096,1] | <Type: 'float32'> | 0 |
31
+ | 28 | Constant_165769 | [1,1,4096] | <Type: 'float32'> | 0 |
32
+ | 29 | Constant_172580 | [16384,4096] | <Type: 'int8_t'> | 0.0171842 |
33
+ | 30 | Constant_172582 | [16384,1] | <Type: 'float32'> | 0 |
34
+ | 31 | Constant_172584 | [4096,16384] | <Type: 'int8_t'> | 0.0282992 |
35
+ | 32 | Constant_172586 | [4096,1] | <Type: 'float32'> | 0 |
36
+ | 33 | Constant_165786 | [1,1,4096] | <Type: 'float32'> | 0 |
37
+ | 34 | Constant_172588 | [12288,4096] | <Type: 'int8_t'> | 0.0219931 |
38
+ | 35 | Constant_172590 | [12288,1] | <Type: 'float32'> | 0 |
39
+ | 36 | Constant_172592 | [4096,4096] | <Type: 'int8_t'> | 0.031169 |
40
+ | 37 | Constant_172594 | [4096,1] | <Type: 'float32'> | 0 |
41
+ | 38 | Constant_165824 | [1,1,4096] | <Type: 'float32'> | 0 |
42
+ | 39 | Constant_172596 | [16384,4096] | <Type: 'int8_t'> | 0.0173885 |
43
+ | 40 | Constant_172598 | [16384,1] | <Type: 'float32'> | 0 |
44
+ | 41 | Constant_172600 | [4096,16384] | <Type: 'int8_t'> | 0.0246724 |
45
+ | 42 | Constant_172602 | [4096,1] | <Type: 'float32'> | 0 |
46
+ | 43 | Constant_165841 | [1,1,4096] | <Type: 'float32'> | 0 |
47
+ | 44 | Constant_172604 | [12288,4096] | <Type: 'int8_t'> | 0.0217427 |
48
+ | 45 | Constant_172606 | [12288,1] | <Type: 'float32'> | 0 |
49
+ | 46 | Constant_172608 | [4096,4096] | <Type: 'int8_t'> | 0.0307601 |
50
+ | 47 | Constant_172610 | [4096,1] | <Type: 'float32'> | 0 |
51
+ | 48 | Constant_165879 | [1,1,4096] | <Type: 'float32'> | 0 |
52
+ | 49 | Constant_172612 | [16384,4096] | <Type: 'int8_t'> | 0.0176567 |
53
+ | 50 | Constant_172614 | [16384,1] | <Type: 'float32'> | 0 |
54
+ | 51 | Constant_172616 | [4096,16384] | <Type: 'int8_t'> | 0.0249349 |
55
+ | 52 | Constant_172618 | [4096,1] | <Type: 'float32'> | 0 |
56
+ | 53 | Constant_165896 | [1,1,4096] | <Type: 'float32'> | 0 |
57
+ | 54 | Constant_172620 | [12288,4096] | <Type: 'int8_t'> | 0.0205595 |
58
+ | 55 | Constant_172622 | [12288,1] | <Type: 'float32'> | 0 |
59
+ | 56 | Constant_172624 | [4096,4096] | <Type: 'int8_t'> | 0.0284023 |
60
+ | 57 | Constant_172626 | [4096,1] | <Type: 'float32'> | 0 |
61
+ | 58 | Constant_165934 | [1,1,4096] | <Type: 'float32'> | 0 |
62
+ | 59 | Constant_172628 | [16384,4096] | <Type: 'int8_t'> | 0.0181628 |
63
+ | 60 | Constant_172630 | [16384,1] | <Type: 'float32'> | 0 |
64
+ | 61 | Constant_172632 | [4096,16384] | <Type: 'int8_t'> | 0.0251128 |
65
+ | 62 | Constant_172634 | [4096,1] | <Type: 'float32'> | 0 |
66
+ | 63 | Constant_165951 | [1,1,4096] | <Type: 'float32'> | 0 |
67
+ | 64 | Constant_172636 | [12288,4096] | <Type: 'int8_t'> | 0.0204584 |
68
+ | 65 | Constant_172638 | [12288,1] | <Type: 'float32'> | 0 |
69
+ | 66 | Constant_172640 | [4096,4096] | <Type: 'int8_t'> | 0.0276942 |
70
+ | 67 | Constant_172642 | [4096,1] | <Type: 'float32'> | 0 |
71
+ | 68 | Constant_165989 | [1,1,4096] | <Type: 'float32'> | 0 |
72
+ | 69 | Constant_172644 | [16384,4096] | <Type: 'int8_t'> | 0.0184493 |
73
+ | 70 | Constant_172646 | [16384,1] | <Type: 'float32'> | 0 |
74
+ | 71 | Constant_172648 | [4096,16384] | <Type: 'int8_t'> | 0.0252663 |
75
+ | 72 | Constant_172650 | [4096,1] | <Type: 'float32'> | 0 |
76
+ | 73 | Constant_166006 | [1,1,4096] | <Type: 'float32'> | 0 |
77
+ | 74 | Constant_172652 | [12288,4096] | <Type: 'int8_t'> | 0.0203506 |
78
+ | 75 | Constant_172654 | [12288,1] | <Type: 'float32'> | 0 |
79
+ | 76 | Constant_172656 | [4096,4096] | <Type: 'int8_t'> | 0.0276108 |
80
+ | 77 | Constant_172658 | [4096,1] | <Type: 'float32'> | 0 |
81
+ | 78 | Constant_166044 | [1,1,4096] | <Type: 'float32'> | 0 |
82
+ | 79 | Constant_172660 | [16384,4096] | <Type: 'int8_t'> | 0.0185779 |
83
+ | 80 | Constant_172662 | [16384,1] | <Type: 'float32'> | 0 |
84
+ | 81 | Constant_172664 | [4096,16384] | <Type: 'int8_t'> | 0.0344812 |
85
+ | 82 | Constant_172666 | [4096,1] | <Type: 'float32'> | 0 |
86
+ | 83 | Constant_166061 | [1,1,4096] | <Type: 'float32'> | 0 |
87
+ | 84 | Constant_172668 | [12288,4096] | <Type: 'int8_t'> | 0.0204209 |
88
+ | 85 | Constant_172670 | [12288,1] | <Type: 'float32'> | 0 |
89
+ | 86 | Constant_172672 | [4096,4096] | <Type: 'int8_t'> | 0.0266736 |
90
+ | 87 | Constant_172674 | [4096,1] | <Type: 'float32'> | 0 |
91
+ | 88 | Constant_166099 | [1,1,4096] | <Type: 'float32'> | 0 |
92
+ | 89 | Constant_172676 | [16384,4096] | <Type: 'int8_t'> | 0.0187477 |
93
+ | 90 | Constant_172678 | [16384,1] | <Type: 'float32'> | 0 |
94
+ | 91 | Constant_172680 | [4096,16384] | <Type: 'int8_t'> | 0.0248203 |
95
+ | 92 | Constant_172682 | [4096,1] | <Type: 'float32'> | 0 |
96
+ | 93 | Constant_166116 | [1,1,4096] | <Type: 'float32'> | 0 |
97
+ | 94 | Constant_172684 | [12288,4096] | <Type: 'int8_t'> | 0.0204322 |
98
+ | 95 | Constant_172686 | [12288,1] | <Type: 'float32'> | 0 |
99
+ | 96 | Constant_172688 | [4096,4096] | <Type: 'int8_t'> | 0.0272591 |
100
+ | 97 | Constant_172690 | [4096,1] | <Type: 'float32'> | 0 |
101
+ | 98 | Constant_166154 | [1,1,4096] | <Type: 'float32'> | 0 |
102
+ | 99 | Constant_172692 | [16384,4096] | <Type: 'int8_t'> | 0.0189323 |
103
+ | 100 | Constant_172694 | [16384,1] | <Type: 'float32'> | 0 |
104
+ | 101 | Constant_172696 | [4096,16384] | <Type: 'int8_t'> | 0.0971526 |
105
+ | 102 | Constant_172698 | [4096,1] | <Type: 'float32'> | 0 |
106
+ | 103 | Constant_166171 | [1,1,4096] | <Type: 'float32'> | 0 |
107
+ | 104 | Constant_172700 | [12288,4096] | <Type: 'int8_t'> | 0.0203143 |
108
+ | 105 | Constant_172702 | [12288,1] | <Type: 'float32'> | 0 |
109
+ | 106 | Constant_172704 | [4096,4096] | <Type: 'int8_t'> | 0.0267224 |
110
+ | 107 | Constant_172706 | [4096,1] | <Type: 'float32'> | 0 |
111
+ | 108 | Constant_166209 | [1,1,4096] | <Type: 'float32'> | 0 |
112
+ | 109 | Constant_172708 | [16384,4096] | <Type: 'int8_t'> | 0.0191041 |
113
+ | 110 | Constant_172710 | [16384,1] | <Type: 'float32'> | 0 |
114
+ | 111 | Constant_172712 | [4096,16384] | <Type: 'int8_t'> | 0.0250151 |
115
+ | 112 | Constant_172714 | [4096,1] | <Type: 'float32'> | 0 |
116
+ | 113 | Constant_166226 | [1,1,4096] | <Type: 'float32'> | 0 |
117
+ | 114 | Constant_172716 | [12288,4096] | <Type: 'int8_t'> | 0.0202371 |
118
+ | 115 | Constant_172718 | [12288,1] | <Type: 'float32'> | 0 |
119
+ | 116 | Constant_172720 | [4096,4096] | <Type: 'int8_t'> | 0.0269926 |
120
+ | 117 | Constant_172722 | [4096,1] | <Type: 'float32'> | 0 |
121
+ | 118 | Constant_166264 | [1,1,4096] | <Type: 'float32'> | 0 |
122
+ | 119 | Constant_172724 | [16384,4096] | <Type: 'int8_t'> | 0.0192083 |
123
+ | 120 | Constant_172726 | [16384,1] | <Type: 'float32'> | 0 |
124
+ | 121 | Constant_172728 | [4096,16384] | <Type: 'int8_t'> | 0.0254771 |
125
+ | 122 | Constant_172730 | [4096,1] | <Type: 'float32'> | 0 |
126
+ | 123 | Constant_166281 | [1,1,4096] | <Type: 'float32'> | 0 |
127
+ | 124 | Constant_172732 | [12288,4096] | <Type: 'int8_t'> | 0.0204674 |
128
+ | 125 | Constant_172734 | [12288,1] | <Type: 'float32'> | 0 |
129
+ | 126 | Constant_172736 | [4096,4096] | <Type: 'int8_t'> | 0.0269803 |
130
+ | 127 | Constant_172738 | [4096,1] | <Type: 'float32'> | 0 |
131
+ | 128 | Constant_166319 | [1,1,4096] | <Type: 'float32'> | 0 |
132
+ | 129 | Constant_172740 | [16384,4096] | <Type: 'int8_t'> | 0.0192206 |
133
+ | 130 | Constant_172742 | [16384,1] | <Type: 'float32'> | 0 |
134
+ | 131 | Constant_172744 | [4096,16384] | <Type: 'int8_t'> | 0.0250526 |
135
+ | 132 | Constant_172746 | [4096,1] | <Type: 'float32'> | 0 |
136
+ | 133 | Constant_166336 | [1,1,4096] | <Type: 'float32'> | 0 |
137
+ | 134 | Constant_172748 | [12288,4096] | <Type: 'int8_t'> | 0.0205014 |
138
+ | 135 | Constant_172750 | [12288,1] | <Type: 'float32'> | 0 |
139
+ | 136 | Constant_172752 | [4096,4096] | <Type: 'int8_t'> | 0.0271882 |
140
+ | 137 | Constant_172754 | [4096,1] | <Type: 'float32'> | 0 |
141
+ | 138 | Constant_166374 | [1,1,4096] | <Type: 'float32'> | 0 |
142
+ | 139 | Constant_172756 | [16384,4096] | <Type: 'int8_t'> | 0.0193506 |
143
+ | 140 | Constant_172758 | [16384,1] | <Type: 'float32'> | 0 |
144
+ | 141 | Constant_172760 | [4096,16384] | <Type: 'int8_t'> | 0.0250494 |
145
+ | 142 | Constant_172762 | [4096,1] | <Type: 'float32'> | 0 |
146
+ | 143 | Constant_166391 | [1,1,4096] | <Type: 'float32'> | 0 |
147
+ | 144 | Constant_172764 | [12288,4096] | <Type: 'int8_t'> | 0.0202526 |
148
+ | 145 | Constant_172766 | [12288,1] | <Type: 'float32'> | 0 |
149
+ | 146 | Constant_172768 | [4096,4096] | <Type: 'int8_t'> | 0.0259429 |
150
+ | 147 | Constant_172770 | [4096,1] | <Type: 'float32'> | 0 |
151
+ | 148 | Constant_166429 | [1,1,4096] | <Type: 'float32'> | 0 |
152
+ | 149 | Constant_172772 | [16384,4096] | <Type: 'int8_t'> | 0.0194639 |
153
+ | 150 | Constant_172774 | [16384,1] | <Type: 'float32'> | 0 |
154
+ | 151 | Constant_172776 | [4096,16384] | <Type: 'int8_t'> | 0.0252066 |
155
+ | 152 | Constant_172778 | [4096,1] | <Type: 'float32'> | 0 |
156
+ | 153 | Constant_166446 | [1,1,4096] | <Type: 'float32'> | 0 |
157
+ | 154 | Constant_172780 | [12288,4096] | <Type: 'int8_t'> | 0.0203257 |
158
+ | 155 | Constant_172782 | [12288,1] | <Type: 'float32'> | 0 |
159
+ | 156 | Constant_172784 | [4096,4096] | <Type: 'int8_t'> | 0.0261641 |
160
+ | 157 | Constant_172786 | [4096,1] | <Type: 'float32'> | 0 |
161
+ | 158 | Constant_166484 | [1,1,4096] | <Type: 'float32'> | 0 |
162
+ | 159 | Constant_172788 | [16384,4096] | <Type: 'int8_t'> | 0.0196701 |
163
+ | 160 | Constant_172790 | [16384,1] | <Type: 'float32'> | 0 |
164
+ | 161 | Constant_172792 | [4096,16384] | <Type: 'int8_t'> | 0.0258841 |
165
+ | 162 | Constant_172794 | [4096,1] | <Type: 'float32'> | 0 |
166
+ | 163 | Constant_166501 | [1,1,4096] | <Type: 'float32'> | 0 |
167
+ | 164 | Constant_172796 | [12288,4096] | <Type: 'int8_t'> | 0.0205318 |
168
+ | 165 | Constant_172798 | [12288,1] | <Type: 'float32'> | 0 |
169
+ | 166 | Constant_172800 | [4096,4096] | <Type: 'int8_t'> | 0.0261948 |
170
+ | 167 | Constant_172802 | [4096,1] | <Type: 'float32'> | 0 |
171
+ | 168 | Constant_166539 | [1,1,4096] | <Type: 'float32'> | 0 |
172
+ | 169 | Constant_172804 | [16384,4096] | <Type: 'int8_t'> | 0.0196445 |
173
+ | 170 | Constant_172806 | [16384,1] | <Type: 'float32'> | 0 |
174
+ | 171 | Constant_172808 | [4096,16384] | <Type: 'int8_t'> | 0.0252786 |
175
+ | 172 | Constant_172810 | [4096,1] | <Type: 'float32'> | 0 |
176
+ | 173 | Constant_166556 | [1,1,4096] | <Type: 'float32'> | 0 |
177
+ | 174 | Constant_172812 | [12288,4096] | <Type: 'int8_t'> | 0.0204819 |
178
+ | 175 | Constant_172814 | [12288,1] | <Type: 'float32'> | 0 |
179
+ | 176 | Constant_172816 | [4096,4096] | <Type: 'int8_t'> | 0.0265484 |
180
+ | 177 | Constant_172818 | [4096,1] | <Type: 'float32'> | 0 |
181
+ | 178 | Constant_166594 | [1,1,4096] | <Type: 'float32'> | 0 |
182
+ | 179 | Constant_172820 | [16384,4096] | <Type: 'int8_t'> | 0.0196449 |
183
+ | 180 | Constant_172822 | [16384,1] | <Type: 'float32'> | 0 |
184
+ | 181 | Constant_172824 | [4096,16384] | <Type: 'int8_t'> | 0.0252218 |
185
+ | 182 | Constant_172826 | [4096,1] | <Type: 'float32'> | 0 |
186
+ | 183 | Constant_166611 | [1,1,4096] | <Type: 'float32'> | 0 |
187
+ | 184 | Constant_172828 | [12288,4096] | <Type: 'int8_t'> | 0.0200144 |
188
+ | 185 | Constant_172830 | [12288,1] | <Type: 'float32'> | 0 |
189
+ | 186 | Constant_172832 | [4096,4096] | <Type: 'int8_t'> | 0.0263395 |
190
+ | 187 | Constant_172834 | [4096,1] | <Type: 'float32'> | 0 |
191
+ | 188 | Constant_166649 | [1,1,4096] | <Type: 'float32'> | 0 |
192
+ | 189 | Constant_172836 | [16384,4096] | <Type: 'int8_t'> | 0.0196273 |
193
+ | 190 | Constant_172838 | [16384,1] | <Type: 'float32'> | 0 |
194
+ | 191 | Constant_172840 | [4096,16384] | <Type: 'int8_t'> | 0.025042 |
195
+ | 192 | Constant_172842 | [4096,1] | <Type: 'float32'> | 0 |
196
+ | 193 | Constant_166666 | [1,1,4096] | <Type: 'float32'> | 0 |
197
+ | 194 | Constant_172844 | [12288,4096] | <Type: 'int8_t'> | 0.020141 |
198
+ | 195 | Constant_172846 | [12288,1] | <Type: 'float32'> | 0 |
199
+ | 196 | Constant_172848 | [4096,4096] | <Type: 'int8_t'> | 0.0258166 |
200
+ | 197 | Constant_172850 | [4096,1] | <Type: 'float32'> | 0 |
201
+ | 198 | Constant_166704 | [1,1,4096] | <Type: 'float32'> | 0 |
202
+ | 199 | Constant_172852 | [16384,4096] | <Type: 'int8_t'> | 0.0194735 |
203
+ | 200 | Constant_172854 | [16384,1] | <Type: 'float32'> | 0 |
204
+ | 201 | Constant_172856 | [4096,16384] | <Type: 'int8_t'> | 0.0245554 |
205
+ | 202 | Constant_172858 | [4096,1] | <Type: 'float32'> | 0 |
206
+ | 203 | Constant_166721 | [1,1,4096] | <Type: 'float32'> | 0 |
207
+ | 204 | Constant_172860 | [12288,4096] | <Type: 'int8_t'> | 0.020283 |
208
+ | 205 | Constant_172862 | [12288,1] | <Type: 'float32'> | 0 |
209
+ | 206 | Constant_172864 | [4096,4096] | <Type: 'int8_t'> | 0.0263975 |
210
+ | 207 | Constant_172866 | [4096,1] | <Type: 'float32'> | 0 |
211
+ | 208 | Constant_166759 | [1,1,4096] | <Type: 'float32'> | 0 |
212
+ | 209 | Constant_172868 | [16384,4096] | <Type: 'int8_t'> | 0.0194434 |
213
+ | 210 | Constant_172870 | [16384,1] | <Type: 'float32'> | 0 |
214
+ | 211 | Constant_172872 | [4096,16384] | <Type: 'int8_t'> | 0.0241456 |
215
+ | 212 | Constant_172874 | [4096,1] | <Type: 'float32'> | 0 |
216
+ | 213 | Constant_166776 | [1,1,4096] | <Type: 'float32'> | 0 |
217
+ | 214 | Constant_172876 | [12288,4096] | <Type: 'int8_t'> | 0.0202228 |
218
+ | 215 | Constant_172878 | [12288,1] | <Type: 'float32'> | 0 |
219
+ | 216 | Constant_172880 | [4096,4096] | <Type: 'int8_t'> | 0.0260037 |
220
+ | 217 | Constant_172882 | [4096,1] | <Type: 'float32'> | 0 |
221
+ | 218 | Constant_166814 | [1,1,4096] | <Type: 'float32'> | 0 |
222
+ | 219 | Constant_172884 | [16384,4096] | <Type: 'int8_t'> | 0.0192591 |
223
+ | 220 | Constant_172886 | [16384,1] | <Type: 'float32'> | 0 |
224
+ | 221 | Constant_172888 | [4096,16384] | <Type: 'int8_t'> | 0.0236581 |
225
+ | 222 | Constant_172890 | [4096,1] | <Type: 'float32'> | 0 |
226
+ | 223 | Constant_166831 | [1,1,4096] | <Type: 'float32'> | 0 |
227
+ | 224 | Constant_172892 | [12288,4096] | <Type: 'int8_t'> | 0.0205292 |
228
+ | 225 | Constant_172894 | [12288,1] | <Type: 'float32'> | 0 |
229
+ | 226 | Constant_172896 | [4096,4096] | <Type: 'int8_t'> | 0.0278201 |
230
+ | 227 | Constant_172898 | [4096,1] | <Type: 'float32'> | 0 |
231
+ | 228 | Constant_166869 | [1,1,4096] | <Type: 'float32'> | 0 |
232
+ | 229 | Constant_172900 | [16384,4096] | <Type: 'int8_t'> | 0.0192568 |
233
+ | 230 | Constant_172902 | [16384,1] | <Type: 'float32'> | 0 |
234
+ | 231 | Constant_172904 | [4096,16384] | <Type: 'int8_t'> | 0.0238914 |
235
+ | 232 | Constant_172906 | [4096,1] | <Type: 'float32'> | 0 |
236
+ | 233 | Constant_166886 | [1,1,4096] | <Type: 'float32'> | 0 |
237
+ | 234 | Constant_172908 | [12288,4096] | <Type: 'int8_t'> | 0.0205223 |
238
+ | 235 | Constant_172910 | [12288,1] | <Type: 'float32'> | 0 |
239
+ | 236 | Constant_172912 | [4096,4096] | <Type: 'int8_t'> | 0.0254208 |
240
+ | 237 | Constant_172914 | [4096,1] | <Type: 'float32'> | 0 |
241
+ | 238 | Constant_166924 | [1,1,4096] | <Type: 'float32'> | 0 |
242
+ | 239 | Constant_172916 | [16384,4096] | <Type: 'int8_t'> | 0.0191006 |
243
+ | 240 | Constant_172918 | [16384,1] | <Type: 'float32'> | 0 |
244
+ | 241 | Constant_172920 | [4096,16384] | <Type: 'int8_t'> | 0.0231392 |
245
+ | 242 | Constant_172922 | [4096,1] | <Type: 'float32'> | 0 |
246
+ | 243 | Constant_166941 | [1,1,4096] | <Type: 'float32'> | 0 |
247
+ | 244 | Constant_172924 | [12288,4096] | <Type: 'int8_t'> | 0.0204926 |
248
+ | 245 | Constant_172926 | [12288,1] | <Type: 'float32'> | 0 |
249
+ | 246 | Constant_172928 | [4096,4096] | <Type: 'int8_t'> | 0.0253145 |
250
+ | 247 | Constant_172930 | [4096,1] | <Type: 'float32'> | 0 |
251
+ | 248 | Constant_166979 | [1,1,4096] | <Type: 'float32'> | 0 |
252
+ | 249 | Constant_172932 | [16384,4096] | <Type: 'int8_t'> | 0.019065 |
253
+ | 250 | Constant_172934 | [16384,1] | <Type: 'float32'> | 0 |
254
+ | 251 | Constant_172936 | [4096,16384] | <Type: 'int8_t'> | 0.0225103 |
255
+ | 252 | Constant_172938 | [4096,1] | <Type: 'float32'> | 0 |
256
+ | 253 | Constant_166996 | [1,1,4096] | <Type: 'float32'> | 0 |
257
+ | 254 | Constant_172940 | [12288,4096] | <Type: 'int8_t'> | 0.0207202 |
258
+ | 255 | Constant_172942 | [12288,1] | <Type: 'float32'> | 0 |
259
+ | 256 | Constant_172944 | [4096,4096] | <Type: 'int8_t'> | 0.0260396 |
260
+ | 257 | Constant_172946 | [4096,1] | <Type: 'float32'> | 0 |
261
+ | 258 | Constant_167034 | [1,1,4096] | <Type: 'float32'> | 0 |
262
+ | 259 | Constant_172948 | [16384,4096] | <Type: 'int8_t'> | 0.0188797 |
263
+ | 260 | Constant_172950 | [16384,1] | <Type: 'float32'> | 0 |
264
+ | 261 | Constant_172952 | [4096,16384] | <Type: 'int8_t'> | 0.0218278 |
265
+ | 262 | Constant_172954 | [4096,1] | <Type: 'float32'> | 0 |
266
+ | 263 | Constant_167051 | [1,1,4096] | <Type: 'float32'> | 0 |
267
+ | 264 | Constant_172956 | [12288,4096] | <Type: 'int8_t'> | 0.0214279 |
268
+ | 265 | Constant_172958 | [12288,1] | <Type: 'float32'> | 0 |
269
+ | 266 | Constant_172960 | [4096,4096] | <Type: 'int8_t'> | 0.0252308 |
270
+ | 267 | Constant_172962 | [4096,1] | <Type: 'float32'> | 0 |
271
+ | 268 | Constant_167089 | [1,1,4096] | <Type: 'float32'> | 0 |
272
+ | 269 | Constant_172964 | [16384,4096] | <Type: 'int8_t'> | 0.018872 |
273
+ | 270 | Constant_172966 | [16384,1] | <Type: 'float32'> | 0 |
274
+ | 271 | Constant_172968 | [4096,16384] | <Type: 'int8_t'> | 0.0214473 |
275
+ | 272 | Constant_172970 | [4096,1] | <Type: 'float32'> | 0 |
276
+ | 273 | Constant_167106 | [1,1,4096] | <Type: 'float32'> | 0 |
277
+ | 274 | Constant_172972 | [12288,4096] | <Type: 'int8_t'> | 0.0209417 |
278
+ | 275 | Constant_172974 | [12288,1] | <Type: 'float32'> | 0 |
279
+ | 276 | Constant_172976 | [4096,4096] | <Type: 'int8_t'> | 0.0250281 |
280
+ | 277 | Constant_172978 | [4096,1] | <Type: 'float32'> | 0 |
281
+ | 278 | Constant_167144 | [1,1,4096] | <Type: 'float32'> | 0 |
282
+ | 279 | Constant_172980 | [16384,4096] | <Type: 'int8_t'> | 0.0188633 |
283
+ | 280 | Constant_172982 | [16384,1] | <Type: 'float32'> | 0 |
284
+ | 281 | Constant_172984 | [4096,16384] | <Type: 'int8_t'> | 0.0210319 |
285
+ | 282 | Constant_172986 | [4096,1] | <Type: 'float32'> | 0 |
286
+ | 283 | Constant_167161 | [1,1,4096] | <Type: 'float32'> | 0 |
287
+ | 284 | Constant_172988 | [12288,4096] | <Type: 'int8_t'> | 0.0205585 |
288
+ | 285 | Constant_172990 | [12288,1] | <Type: 'float32'> | 0 |
289
+ | 286 | Constant_172992 | [4096,4096] | <Type: 'int8_t'> | 0.0231342 |
290
+ | 287 | Constant_172994 | [4096,1] | <Type: 'float32'> | 0 |
291
+ | 288 | Constant_167199 | [1,1,4096] | <Type: 'float32'> | 0 |
292
+ | 289 | Constant_172996 | [16384,4096] | <Type: 'int8_t'> | 0.0188606 |
293
+ | 290 | Constant_172998 | [16384,1] | <Type: 'float32'> | 0 |
294
+ | 291 | Constant_173000 | [4096,16384] | <Type: 'int8_t'> | 0.0205154 |
295
+ | 292 | Constant_173002 | [4096,1] | <Type: 'float32'> | 0 |
296
+ | 293 | Constant_167216 | [1,1,4096] | <Type: 'float32'> | 0 |
297
+ | 294 | Constant_173004 | [12288,4096] | <Type: 'int8_t'> | 0.0201311 |
298
+ | 295 | Constant_173006 | [12288,1] | <Type: 'float32'> | 0 |
299
+ | 296 | Constant_173008 | [4096,4096] | <Type: 'int8_t'> | 0.0237586 |
300
+ | 297 | Constant_173010 | [4096,1] | <Type: 'float32'> | 0 |
301
+ | 298 | Constant_167254 | [1,1,4096] | <Type: 'float32'> | 0 |
302
+ | 299 | Constant_173012 | [16384,4096] | <Type: 'int8_t'> | 0.0187869 |
303
+ | 300 | Constant_173014 | [16384,1] | <Type: 'float32'> | 0 |
304
+ | 301 | Constant_173016 | [4096,16384] | <Type: 'int8_t'> | 0.0204634 |
305
+ | 302 | Constant_173018 | [4096,1] | <Type: 'float32'> | 0 |
306
+ | 303 | Constant_167271 | [1,1,4096] | <Type: 'float32'> | 0 |
307
+ | 304 | Constant_173020 | [12288,4096] | <Type: 'int8_t'> | 0.0196505 |
308
+ | 305 | Constant_173022 | [12288,1] | <Type: 'float32'> | 0 |
309
+ | 306 | Constant_173024 | [4096,4096] | <Type: 'int8_t'> | 0.0236116 |
310
+ | 307 | Constant_173026 | [4096,1] | <Type: 'float32'> | 0 |
311
+ | 308 | Constant_167309 | [1,1,4096] | <Type: 'float32'> | 0 |
312
+ | 309 | Constant_173028 | [16384,4096] | <Type: 'int8_t'> | 0.0179917 |
313
+ | 310 | Constant_173030 | [16384,1] | <Type: 'float32'> | 0 |
314
+ | 311 | Constant_173032 | [4096,16384] | <Type: 'int8_t'> | 0.0215376 |
315
+ | 312 | Constant_173034 | [4096,1] | <Type: 'float32'> | 0 |
316
+ | 313 | Constant_167326 | [1,1,4096] | <Type: 'float32'> | 0 |
317
+ | 314 | Constant_173036 | [12288,4096] | <Type: 'int8_t'> | 0.0195202 |
318
+ | 315 | Constant_173038 | [12288,1] | <Type: 'float32'> | 0 |
319
+ | 316 | Constant_173040 | [4096,4096] | <Type: 'int8_t'> | 0.0240483 |
320
+ | 317 | Constant_173042 | [4096,1] | <Type: 'float32'> | 0 |
321
+ | 318 | Constant_167364 | [1,1,4096] | <Type: 'float32'> | 0 |
322
+ | 319 | Constant_173044 | [16384,4096] | <Type: 'int8_t'> | 0.0218584 |
323
+ | 320 | Constant_173046 | [16384,1] | <Type: 'float32'> | 0 |
324
+ | 321 | Constant_173048 | [4096,16384] | <Type: 'int8_t'> | 0.0412405 |
325
+ | 322 | Constant_173050 | [4096,1] | <Type: 'float32'> | 0 |
326
+ | 323 | Constant_167381 | [1,1,4096] | <Type: 'float32'> | 0 |
327
+ | 324 | Constant_173052 | [50432,4096] | <Type: 'int8_t'> | 0.0132285 |
328
+ | 325 | Constant_173054 | [50432,1] | <Type: 'float32'> | 0 |
models/mpt-7b-gsm8k-pt/w8a8/special_tokens_map.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": "<|endoftext|>",
3
+ "eos_token": "<|endoftext|>",
4
+ "pad_token": "<|endoftext|>",
5
+ "unk_token": "<|endoftext|>"
6
+ }
models/mpt-7b-gsm8k-pt/w8a8/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
models/mpt-7b-gsm8k-pt/w8a8/tokenizer_config.json ADDED
@@ -0,0 +1,212 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<|padding|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "50254": {
21
+ "content": " ",
22
+ "lstrip": false,
23
+ "normalized": true,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": false
27
+ },
28
+ "50255": {
29
+ "content": " ",
30
+ "lstrip": false,
31
+ "normalized": true,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": false
35
+ },
36
+ "50256": {
37
+ "content": " ",
38
+ "lstrip": false,
39
+ "normalized": true,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": false
43
+ },
44
+ "50257": {
45
+ "content": " ",
46
+ "lstrip": false,
47
+ "normalized": true,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": false
51
+ },
52
+ "50258": {
53
+ "content": " ",
54
+ "lstrip": false,
55
+ "normalized": true,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": false
59
+ },
60
+ "50259": {
61
+ "content": " ",
62
+ "lstrip": false,
63
+ "normalized": true,
64
+ "rstrip": false,
65
+ "single_word": false,
66
+ "special": false
67
+ },
68
+ "50260": {
69
+ "content": " ",
70
+ "lstrip": false,
71
+ "normalized": true,
72
+ "rstrip": false,
73
+ "single_word": false,
74
+ "special": false
75
+ },
76
+ "50261": {
77
+ "content": " ",
78
+ "lstrip": false,
79
+ "normalized": true,
80
+ "rstrip": false,
81
+ "single_word": false,
82
+ "special": false
83
+ },
84
+ "50262": {
85
+ "content": " ",
86
+ "lstrip": false,
87
+ "normalized": true,
88
+ "rstrip": false,
89
+ "single_word": false,
90
+ "special": false
91
+ },
92
+ "50263": {
93
+ "content": " ",
94
+ "lstrip": false,
95
+ "normalized": true,
96
+ "rstrip": false,
97
+ "single_word": false,
98
+ "special": false
99
+ },
100
+ "50264": {
101
+ "content": " ",
102
+ "lstrip": false,
103
+ "normalized": true,
104
+ "rstrip": false,
105
+ "single_word": false,
106
+ "special": false
107
+ },
108
+ "50265": {
109
+ "content": " ",
110
+ "lstrip": false,
111
+ "normalized": true,
112
+ "rstrip": false,
113
+ "single_word": false,
114
+ "special": false
115
+ },
116
+ "50266": {
117
+ "content": " ",
118
+ "lstrip": false,
119
+ "normalized": true,
120
+ "rstrip": false,
121
+ "single_word": false,
122
+ "special": false
123
+ },
124
+ "50267": {
125
+ "content": " ",
126
+ "lstrip": false,
127
+ "normalized": true,
128
+ "rstrip": false,
129
+ "single_word": false,
130
+ "special": false
131
+ },
132
+ "50268": {
133
+ "content": " ",
134
+ "lstrip": false,
135
+ "normalized": true,
136
+ "rstrip": false,
137
+ "single_word": false,
138
+ "special": false
139
+ },
140
+ "50269": {
141
+ "content": " ",
142
+ "lstrip": false,
143
+ "normalized": true,
144
+ "rstrip": false,
145
+ "single_word": false,
146
+ "special": false
147
+ },
148
+ "50270": {
149
+ "content": " ",
150
+ "lstrip": false,
151
+ "normalized": true,
152
+ "rstrip": false,
153
+ "single_word": false,
154
+ "special": false
155
+ },
156
+ "50271": {
157
+ "content": " ",
158
+ "lstrip": false,
159
+ "normalized": true,
160
+ "rstrip": false,
161
+ "single_word": false,
162
+ "special": false
163
+ },
164
+ "50272": {
165
+ "content": " ",
166
+ "lstrip": false,
167
+ "normalized": true,
168
+ "rstrip": false,
169
+ "single_word": false,
170
+ "special": false
171
+ },
172
+ "50273": {
173
+ "content": " ",
174
+ "lstrip": false,
175
+ "normalized": true,
176
+ "rstrip": false,
177
+ "single_word": false,
178
+ "special": false
179
+ },
180
+ "50274": {
181
+ "content": " ",
182
+ "lstrip": false,
183
+ "normalized": true,
184
+ "rstrip": false,
185
+ "single_word": false,
186
+ "special": false
187
+ },
188
+ "50275": {
189
+ "content": " ",
190
+ "lstrip": false,
191
+ "normalized": true,
192
+ "rstrip": false,
193
+ "single_word": false,
194
+ "special": false
195
+ },
196
+ "50276": {
197
+ "content": " ",
198
+ "lstrip": false,
199
+ "normalized": true,
200
+ "rstrip": false,
201
+ "single_word": false,
202
+ "special": false
203
+ }
204
+ },
205
+ "bos_token": "<|endoftext|>",
206
+ "clean_up_tokenization_spaces": true,
207
+ "eos_token": "<|endoftext|>",
208
+ "model_max_length": 512,
209
+ "pad_token": "<|endoftext|>",
210
+ "tokenizer_class": "GPTNeoXTokenizer",
211
+ "unk_token": "<|endoftext|>"
212
+ }
tld0.6.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "CPU": {
3
+ "CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE": "0.6"
4
+ }
5
+ }