neuralmagic
/

Qwen2-0.5B-Instruct-quantized.w8a8

Text Generation

text-generation-inference

Inference Endpoints

8-bit precision

compressed-tensors

Model card Files Files and versions Community

alexmarques commited on Jul 18, 2024

Commit

3b3a605

·

verified ·

1 Parent(s): a2590bd

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -6,7 +6,7 @@ license: apache-2.0
 license_link: https://www.apache.org/licenses/LICENSE-2.0
 ---
-# Qwen2-0.5B-Instruct-quantized.w8a16
 ## Model Overview
 - **Model Architecture:** Qwen2
@@ -185,7 +185,7 @@ lm_eval \
    </td>
    <td><strong>Qwen2-0.5B-Instruct</strong>
    </td>
-   <td><strong>Qwen2-0.5B-Instruct-quantized.w8a16(this model)</strong>
    </td>
    <td><strong>Recovery</strong>
    </td>

 license_link: https://www.apache.org/licenses/LICENSE-2.0
 ---
+# Qwen2-0.5B-Instruct-quantized.w8a8
 ## Model Overview
 - **Model Architecture:** Qwen2
    </td>
    <td><strong>Qwen2-0.5B-Instruct</strong>
    </td>
+   <td><strong>Qwen2-0.5B-Instruct-quantized.w8a8 (this model)</strong>
    </td>
    <td><strong>Recovery</strong>
    </td>