Update README.md
Browse files
README.md
CHANGED
|
@@ -14,6 +14,13 @@ tags:
|
|
| 14 |
- [Anastasiia Zhiboedova](https://www.linkedin.com/in/azhiboedova/)
|
| 15 |
- [Mike Arbuzov](https://www.linkedin.com/in/mike-arbuzov/)
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
**Model Architecture**
|
| 18 |
The Llama 3.1 8B model is a state-of-the-art language model designed for a wide range of conversational and text generation tasks. By applying the Adaptive Quantization Learning Mechanism (AQLM) developed by Yandex Research, the model's size has been significantly reduced without sacrificing its powerful capabilities. This approach dynamically adjusts the precision of model parameters during training, optimizing for both performance and efficiency.
|
| 19 |
|
|
|
|
| 14 |
- [Anastasiia Zhiboedova](https://www.linkedin.com/in/azhiboedova/)
|
| 15 |
- [Mike Arbuzov](https://www.linkedin.com/in/mike-arbuzov/)
|
| 16 |
|
| 17 |
+
**Model Comparison: Quantized vs Basic Model**
|
| 18 |
+
|
| 19 |
+
| Model Type | Meta-Llama-3.1-8B-Instruct | Meta-Llama-3.1-8B-Instruct-AQLM-2Bit-1x16 - quantized|
|
| 20 |
+
|-----------------------------|----------------------------|------------------------------------------------------|
|
| 21 |
+
| Parameters | 8.03B | 2.04B |
|
| 22 |
+
| Peak Memory Usage | 20.15 GB | 4.22 GB |
|
| 23 |
+
|
| 24 |
**Model Architecture**
|
| 25 |
The Llama 3.1 8B model is a state-of-the-art language model designed for a wide range of conversational and text generation tasks. By applying the Adaptive Quantization Learning Mechanism (AQLM) developed by Yandex Research, the model's size has been significantly reduced without sacrificing its powerful capabilities. This approach dynamically adjusts the precision of model parameters during training, optimizing for both performance and efficiency.
|
| 26 |
|