hfl
/

llama-3-chinese-8b-instruct-gguf

Inference Endpoints

Model card Files Files and versions

hfl-rc commited on Apr 30

Commit

eb73cf7

•

1 Parent(s): 499d7d3

Update README.md

Files changed (1) hide show

README.md +11 -11

README.md CHANGED Viewed

@@ -21,17 +21,17 @@ Further details (performance, usage, etc.) should refer to GitHub project page:
 Metric: PPL, lower is better
-| Quant |     Size |  PPL |
-| :---: | -------: | ---: |
-| Q2_K  |  2.96 GB |      |
-| Q3_K  |  3.74 GB |      |
-| Q4_0  |  4.34 GB |      |
-| Q4_K  |  4.58 GB |      |
-| Q5_0  |  5.21 GB |      |
-| Q5_K  |  5.34 GB |      |
-| Q6_K  |  6.14 GB |      |
-| Q8_0  |  7.95 GB |      |
-|  F16  | 14.97 GB |      |
 ## Others

 Metric: PPL, lower is better
+| Quant |     Size |                 PPL |        PPL (`-im`) |
+| :---: | -------: | ------------------: | -----------------: |
+| Q2_K  |  2.96 GB | 10.3918 +/- 0.13288 | **9.1722 +/- 0.11502** |
+| Q3_K  |  3.74 GB |  6.3018 +/- 0.07849 | **6.1901 +/- 0.07734** |
+| Q4_0  |  4.34 GB |  6.0628 +/- 0.07501 | **5.9623 +/- 0.07444** |
+| Q4_K  |  4.58 GB |  5.9066 +/- 0.07419 | **5.8847 +/- 0.07406** |
+| Q5_0  |  5.21 GB |  5.8562 +/- 0.07355 | **5.8032 +/- 0.07287** |
+| Q5_K  |  5.34 GB |  5.8062 +/- 0.07331 | **5.8058 +/- 0.07329** |
+| Q6_K  |  6.14 GB |  5.7757 +/- 0.07298 | **5.7745 +/- 0.07287** |
+| Q8_0  |  7.95 GB |  5.7626 +/- 0.07272 | 5.7626 +/- 0.07272 |
+|  F16  | 14.97 GB |  5.7628 +/- 0.07275 |                N/A |
 ## Others