ssyok commited on
Commit
4c0b500
1 Parent(s): 705be82

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +41 -3
README.md CHANGED
@@ -1,3 +1,41 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ pipeline_tag: text-generation
4
+ tags:
5
+ - ONNX
6
+ - DML
7
+ - ONNXRuntime
8
+ - phi3
9
+ - nlp
10
+ - conversational
11
+ - custom_code
12
+ inference: false
13
+ language:
14
+ - en
15
+ ---
16
+ # EmbeddedLLM/Phi-3-mini-4k-instruct-onnx-cpu-int4-rtn-block-32-acc-level-4
17
+
18
+ ## Performance Metrics
19
+
20
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
21
+ ### CPU-INT4-RTN-BLOCK-32-ACC-LEVEL-4
22
+ We measured the performance of CPU-INT4-RTN-BLOCK-32-ACC-LEVEL-4 on AMD Ryzen 9 7940HS /w Radeon 78
23
+
24
+ | Prompt Length | Generation Length | Average Throughput (tps) |
25
+ |---------------------------|-------------------|-----------------------------|
26
+ | 128 | 128 | - |
27
+ | 128 | 256 | - |
28
+ | 128 | 512 | - |
29
+ | 128 | 1024 | - |
30
+ | 256 | 128 | - |
31
+ | 256 | 256 | - |
32
+ | 256 | 512 | - |
33
+ | 256 | 1024 | - |
34
+ | 512 | 128 | - |
35
+ | 512 | 256 | - |
36
+ | 512 | 512 | - |
37
+ | 512 | 1024 | - |
38
+ | 1024 | 128 | - |
39
+ | 1024 | 256 | - |
40
+ | 1024 | 512 | - |
41
+ | 1024 | 1024 | - |