Update README.md

README.md

@@ -25,7 +25,9 @@ tags:
   <i>or what ChatGPT suggests, <b>"Crafting a Rapid prototype of an Intelligent llm App using open source resources"</b>.</i>
 </p>
 
-The initial objective of the CRIA project is to develop a comprehensive end-to-end chatbot system, starting from the instruction-tuning of a large language model and extending to its deployment on the web using frameworks such as Next.js. Specifically, we have fine-tuned the `llama-2-7b-chat-hf` model with QLoRA (4-bit precision) using the [mlabonne/CodeLlama-2-20k](https://huggingface.co/datasets/mlabonne/CodeLlama-2-20k) dataset. This fine-tuned model serves as the backbone for the [CRIA chat](https://chat.walterteng.com) platform.
+The initial objective of the CRIA project is to develop a comprehensive end-to-end chatbot system, starting from the instruction-tuning of a large language model and extending to its deployment on the web using frameworks such as Next.js. 
+
+Specifically, we have fine-tuned the `llama-2-7b-chat-hf` model with QLoRA (4-bit precision) using the [mlabonne/CodeLlama-2-20k](https://huggingface.co/datasets/mlabonne/CodeLlama-2-20k) dataset. This fine-tuned model serves as the backbone for the [CRIA chat](https://chat.walterteng.com) platform.
 
 ## 📦 Model Release
 
@@ -39,6 +41,26 @@ CRIA v1.3 comes with several variants.
 
 It was trained on a Google Colab notebook with a T4 GPU and high RAM.
 
+### Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: False
+- bnb_4bit_compute_dtype: float16
+  
+### Framework versions
+
+
+- PEFT 0.4.0
+
+
 ## 💻 Usage
 
 ```python
@@ -77,3 +99,4 @@ We'd like to thank:
 
 - [mlabonne](https://huggingface.co/mlabonne) for his article and resources on implementation of instruction tuning
 - [TheBloke](https://huggingface.co/TheBloke) for his script for LLM quantization.
+