Update README.md

README.md

@@ -1,3 +1,150 @@
----
-license: llama3.2
----
+---
+license: llama3.2
+datasets:
+- bigbio/med_qa
+language:
+- en
+base_model:
+- meta-llama/Llama-3.2-1B-Instruct
+pipeline_tag: text-generation
+tags:
+- medical
+- SandLogic
+- Meta
+- Conversational
+---
+# SandLogic Technology - Quantized Llama-3.2-1B-Instruct-Medical-GGUF
+
+## Model Description
+
+We have quantized the Llama-3.2-1B-Instruct-Medical-GGUF model into two variants:
+
+1. Q5_KM
+2. Q4_KM
+
+These quantized models offer improved efficiency while maintaining performance in medical-related tasks.
+
+Discover our full range of quantized language models by visiting our [SandLogic Lexicon](https://github.com/sandlogic/SandLogic-Lexicon) GitHub. To learn more about our company and services, check out our website at [SandLogic](https://www.sandlogic.com).
+
+## Original Model Information
+
+- **Base Model**: [Meta Llama 3.2 1B Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)
+- **Developer**: Meta (base model)
+- **Model Type**: Multilingual large language model (LLM)
+- **Architecture**: Auto-regressive language model with optimized transformer architecture
+- **Parameters**: 1 billion
+- **Training Approach**: Supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF)
+
+## Fine-tuning Details
+
+- **Dataset**: [bigbio/med_qa](https://huggingface.co/datasets/bigbio/med_qa)
+- **Languages**: English, simplified Chinese, and traditional Chinese
+- **Dataset Size**: 
+  - English: 12,723 questions
+  - Simplified Chinese: 34,251 questions
+  - Traditional Chinese: 14,123 questions
+- **Data Type**: Free-form multiple-choice OpenQA for medical problems, collected from professional medical board exams
+
+## Model Capabilities
+
+This model is optimized for medical-related dialogue and tasks, including:
+
+- Answering medical questions
+- Summarizing medical information
+- Assisting with medical problem-solving
+
+## Intended Use in Medical Domain
+
+1. **Medical Education**: Assisting medical students in exam preparation and learning
+2. **Clinical Decision Support**: Providing quick references for healthcare professionals
+3. **Patient Education**: Explaining medical concepts in simple terms for patients
+4. **Medical Literature Review**: Summarizing and extracting key information from medical texts
+5. **Differential Diagnosis**: Assisting in generating potential diagnoses based on symptoms
+6. **Medical Coding**: Aiding in the accurate coding of medical procedures and diagnoses
+7. **Drug Information**: Providing information on medications, their uses, and potential interactions
+8. **Medical Translation**: Assisting with medical translations across supported languages
+
+## Quantized Variants
+
+1. **Q5_KM**: 5-bit quantization using the KM method
+2. **Q4_KM**: 4-bit quantization using the KM method
+
+These quantized models aim to reduce model size and improve inference speed while maintaining performance as close to the original model as possible.
+
+
+
+## Usage
+
+```bash
+pip install llama-cpp-python 
+```
+Please refer to the llama-cpp-python [documentation](https://llama-cpp-python.readthedocs.io/en/latest/) to install with GPU support.
+
+### Basic Text Completion
+Here's an example demonstrating how to use the high-level API for basic text completion:
+
+```bash
+from llama_cpp import Llama
+
+llm = Llama(
+    model_path="./models/Llama-3.2-1B-Medical_Q4_KM.gguf",
+    verbose=False,
+    # n_gpu_layers=-1, # Uncomment to use GPU acceleration
+    # n_ctx=2048, # Uncomment to increase the context window
+)
+
+output = llm.create_chat_completion(
+    messages =[
+    {
+        "role": "system",
+        "content": """ You are a helpful, respectful and honest medical assistant. Yu are developed by SandLogic Technologies 
+            Always answer as helpfully as possible, while being safe. 
+            Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. 
+            Please ensure that your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. 
+            If you don’t know the answer to a question, please don’t share false information."""
+            
+        ,
+    },
+    {"role": "user", "content": "I have been experiencing a persistent cough for the last two weeks, along with a mild fever and fatigue. What could be the possible causes of these symptoms?"},
+]
+)
+
+print(output["choices"][0]['message']['content'])
+```
+
+## Download
+You can download `Llama` models in `gguf` format directly from Hugging Face using the `from_pretrained` method. This feature requires the `huggingface-hub` package.
+
+To install it, run: `pip install huggingface-hub`
+
+```bash
+from llama_cpp import Llama
+
+llm = Llama.from_pretrained(
+    repo_id="SandLogicTechnologies/Llama-3.2-1B-Instruct-Medical-GGUF",
+    filename="*Llama-3.2-1B-Medical_Q5_KM.gguf",
+    verbose=False
+)
+```
+By default, from_pretrained will download the model to the Hugging Face cache directory. You can manage installed model files using the huggingface-cli tool.
+
+## Ethical Considerations and Limitations
+
+- This model is not a substitute for professional medical advice, diagnosis, or treatment
+- Users should be aware of potential biases in the training data
+- The model's knowledge cutoff date may limit its awareness of recent medical developments
+
+
+
+
+## Acknowledgements
+
+We thank Meta for developing the original Llama-3.2-1B-Instruct model and the creators of the bigbio/med_qa dataset.
+Special thanks to Georgi Gerganov and the entire llama.cpp development team for their outstanding contributions.
+## Contact
+
+For any inquiries or support, please contact us at support@sandlogic.com or visit our [support page](https://www.sandlogic.com/LingoForge/support).
+
+## Explore More
+
+For any inquiries or support, please contact us at support@sandlogic.com or visit our [support page](https://www.sandlogic.com/LingoForge/support).