Update README.md
Browse files
README.md
CHANGED
@@ -12,10 +12,10 @@ tags:
|
|
12 |
* <h3 style="display: inline;">Release Date:</h3> June 14, 2024
|
13 |
* <h3 style="display: inline;">Model Developers:</h3> Neural Magic
|
14 |
|
15 |
-
Qwen2-1.5B-Instruct quantized to FP8 weights and activations using per-tensor quantization through the AutoFP8 repository, ready for inference with vLLM >= 0.5.0.
|
16 |
Calibrated with 512 UltraChat samples to achieve 99% performance recovery on the Open LLM Benchmark evaluations.
|
17 |
Reduces space on disk by ~40%.
|
18 |
-
Part of the FP8 LLMs for vLLM collection.
|
19 |
|
20 |
|
21 |
## Usage and Creation
|
|
|
12 |
* <h3 style="display: inline;">Release Date:</h3> June 14, 2024
|
13 |
* <h3 style="display: inline;">Model Developers:</h3> Neural Magic
|
14 |
|
15 |
+
Qwen2-1.5B-Instruct quantized to FP8 weights and activations using per-tensor quantization through the [AutoFP8 repository](https://github.com/neuralmagic/AutoFP8), ready for inference with vLLM >= 0.5.0.
|
16 |
Calibrated with 512 UltraChat samples to achieve 99% performance recovery on the Open LLM Benchmark evaluations.
|
17 |
Reduces space on disk by ~40%.
|
18 |
+
Part of the [FP8 LLMs for vLLM collection](https://huggingface.co/collections/neuralmagic/fp8-llms-for-vllm-666742ed2b78b7ac8df13127).
|
19 |
|
20 |
|
21 |
## Usage and Creation
|