Update README.md

README.md

@@ -20,6 +20,7 @@ tags:
 |-----------------------------|----------------------------|------------------------------------------------------|
 | Parameters                  |     8.03B                  |          2.04B                                       |
 | Peak Memory Usage           |    20.15 GB                |          4.22 GB                                     |
+| MMLU Accuracy               |    60.9%                   |          45.5%                                       |
 
 **Model Architecture**
 The Llama 3.1 8B model is a state-of-the-art language model designed for a wide range of conversational and text generation tasks. By applying the Adaptive Quantization Learning Mechanism (AQLM) developed by Yandex Research, the model's size has been significantly reduced without sacrificing its powerful capabilities. This approach dynamically adjusts the precision of model parameters during training, optimizing for both performance and efficiency.
@@ -34,15 +35,15 @@ Incorporating the innovative AQLM (Adaptive Quantization Learning Mechanism), th
 The model was compressed using Vast AI with 8x A100 GPUs, taking approximately 5-6 hours to complete the process.
 
 **Evaluations**
-The quantized Llama 3.1 8B model was rigorously evaluated using the Massive Multitask Language Understanding (MMLU) benchmark with a 5-shot setting. MMLU is a comprehensive benchmark designed to evaluate large language models through multiple-choice questions across 57 subjects, including math, history, law, and ethics. For more information, visit the [MMLU GitHub page](https://github.com/hendrycks/test).
+The quantized Llama 3.1 8B model was rigorously evaluated using the [The MMLU (Massive Multitask Language Understanding)](https://huggingface.co/datasets/tasksource/mmlu) dataset, available on Hugging Face, is designed to evaluate the performance of language models across a wide range of subjects. It includes multiple-choice questions covering diverse topics such as math, history, law, and ethics. Each question is accompanied by four possible answers, making it an ideal benchmark for assessing the accuracy and generalization capabilities of language models.
 
-For evaluation, the library `deepeval.benchmarks.mmlu` was utilized: 
+For the evaluation of our model, we utilized the transformers library from Hugging Face. Below is an example of how to set up and run the evaluation:
 ```python
-from deepeval.benchmarks.mmlu import MMLU
+dataset = load_dataset("tasksource/mmlu", subset, split="validation")
 ```
 [Colab Notebook with Model Evaluation](https://colab.research.google.com/drive/16hXI7pd9KSTeUMNfGCB0wGMAzcBqzdZM?usp=sharing)
 
-The results demonstrate that the quantized model maintains competitive performance compared to the original, non-quantized model, with reduced computational and storage requirements.
+<img src="https://huggingface.co/azhiboedova/Meta-Llama-3.1-8B-Instruct-AQLM-2Bit-1x16/resolve/main/Model%20Performance%20by%20Subject.png" alt="Model Perfomance" width="500">
 
 **How to use**
 To import this model with Python and run it, you can use the following code: