Update README.md

README.md

@@ -20,18 +20,19 @@ Welcome to the official HuggingFace repository for BiMediX, the bilingual medica
 - **Evaluation Benchmark for Arabic Medical LLMs**: Comprehensive benchmark for evaluating Arabic medical language models, setting a new standard in the field.
 - **State-of-the-Art Performance**: Outperforms existing models in medical benchmarks, while 8-times faster than comparable existing models.
 
+For full details of this model please read our [paper (pre-print)](#).
 
 ## Getting Started
 
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-model_id = "TODO"
-tokenizer = AutoTokenizer.from_pretrained(model_id)
+model_id = "BiMediX/BiMediX-Bi"
 
+tokenizer = AutoTokenizer.from_pretrained(model_id)
 model = AutoModelForCausalLM.from_pretrained(model_id)
 
-text = "TODO"
+text = "Hello BiMediX! I've been experiencing increased tiredness in the past week."
 inputs = tokenizer(text, return_tensors="pt")
 
 outputs = model.generate(**inputs, max_new_tokens=500)
@@ -41,7 +42,8 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 
 ## Model Details
 
-(Describe the model's architecture, focusing on its mixture of experts design.)
+
+The BiMediX model, built on a Mixture of Experts (MoE) architecture, leverages the Mixtral-8x7B base network. This approach enables the model to scale significantly by utilizing a sparse operation method, where only a subset of its 47 billion parameters are active during inference, enhancing efficiency. It features a sophisticated router network to allocate tasks to the most relevant experts, each being a specialized feedforward block within the model. The training utilized the BiMed1.3M dataset, focusing on bilingual medical interactions in both English and Arabic, with a substantial corpus of over 632 million healthcare-specialized tokens. The model's fine-tuning process includes a low-rank adaptation technique (QLoRA) to efficiently adapt the model to specific tasks while keeping computational demands manageable. 
 
 ## Dataset