|
--- |
|
library_name: peft |
|
base_model: mistralai/Mistral-7B-Instruct-v0.2 |
|
license: apache-2.0 |
|
tags: |
|
- text-generation-inference |
|
- transformers |
|
- ruslanmv |
|
- mistral |
|
- trl |
|
datasets: |
|
- ruslanmv/ai-medical-chatbot |
|
language: |
|
- en |
|
--- |
|
|
|
# Medical-Mixtral-7B-v2k |
|
[![](future.jpg)](https://ruslanmv.com/) |
|
## Description |
|
Fine-tuned Mixtral model for answering medical assistance questions. This model is a novel version of mistralai/Mistral-7B-Instruct-v0.2, adapted to a subset of 2.0k records from the AI Medical Chatbot dataset, which contains 250k records (https://huggingface.co/datasets/ruslanmv/ai-medical-chatbot). The purpose of this model is to provide a ready chatbot to answer questions related to medical assistance. |
|
|
|
## Intended Use |
|
This model is intended for providing assistance and answering questions related to medical inquiries. It is suitable for use in chatbot applications where users seek medical advice, information, or assistance. |
|
|
|
## Installation |
|
``` |
|
pip install -qU transformers==4.36.2 datasets python-dotenv peft bitsandbytes accelerate |
|
``` |
|
## Example Usage |
|
```python |
|
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, logging, BitsAndBytesConfig |
|
import os, torch |
|
|
|
# Define the name of your fine-tuned model |
|
finetuned_model = 'ruslanmv/Medical-Mixtral-7B-v2k' |
|
|
|
# Load fine-tuned model |
|
bnb_config = BitsAndBytesConfig( |
|
load_in_4bit=True, |
|
bnb_4bit_quant_type="nf4", |
|
bnb_4bit_compute_dtype=torch.bfloat16, |
|
bnb_4bit_use_double_quant=False, |
|
) |
|
model_pretrained = AutoModelForCausalLM.from_pretrained( |
|
finetuned_model, |
|
load_in_4bit=True, |
|
quantization_config=bnb_config, |
|
torch_dtype=torch.bfloat16, |
|
device_map="auto", |
|
trust_remote_code=True |
|
) |
|
|
|
# Load tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained(finetuned_model, trust_remote_code=True) |
|
|
|
# Set pad_token_id to eos_token_id |
|
model_pretrained.config.pad_token_id = tokenizer.eos_token_id |
|
|
|
pipe = pipeline(task="text-generation", model=model_pretrained, tokenizer=tokenizer, max_length=100) |
|
|
|
def build_prompt(question): |
|
prompt=f"[INST]@Enlighten. {question} [/INST]" |
|
return prompt |
|
|
|
question = "What does abutment of the nerve root mean?" |
|
prompt = build_prompt(question) |
|
|
|
# Generate text based on the prompt |
|
result = pipe(prompt)[0] |
|
generated_text = result['generated_text'] |
|
|
|
# Remove the prompt from the generated text |
|
generated_text = generated_text.replace(prompt, "", 1).strip() |
|
|
|
print(generated_text) |
|
|
|
|
|
``` |
|
you will get somethinng like |
|
``` |
|
Please help. For more information consult an internal medicine physician online ➜ http://iclinic.com/e/gastroenterologist-online-consultation.php. |
|
``` |
|
|
|
|
|
|
|
also you can |
|
|
|
```python |
|
def ask(question): |
|
promptEnding = "[/INST]" |
|
# Guide for answering questions |
|
testGuide = 'Answer the following question, at the end of your response say thank you for your query.\n' |
|
# Build the question prompt |
|
question = testGuide + question + "\n" |
|
print(question) |
|
# Build the prompt |
|
prompt = build_prompt(question) |
|
# Generate answer |
|
result = pipe(prompt) |
|
llmAnswer = result[0]['generated_text'] |
|
# Remove the prompt from the generated answer |
|
index = llmAnswer.find(promptEnding) |
|
llmAnswer = llmAnswer[len(promptEnding) + index:] |
|
print("LLM Answer:") |
|
print(llmAnswer) |
|
|
|
question = "For how long should I take Kalachikai powder to overcome PCOD problem?" |
|
ask(question) |
|
``` |
|
|
|
|
|
|
|
## Training Data |
|
- **Dataset Name:** AI Medical Chatbot |
|
- **Dataset URL:** https://huggingface.co/datasets/ruslanmv/ai-medical-chatbot |
|
- **Dataset Size:** 250k records |
|
- **Subset Used:** 2.0k records |
|
|
|
## Limitations |
|
The model's performance may vary depending on the complexity and specificity of the medical questions. |
|
The model may not provide accurate answers for every medical query, and users should consult medical professionals for critical healthcare concerns. |
|
|
|
## Ethical Considerations |
|
Users should be informed that the model's responses are generated based on patterns in the training data and may not always be accurate or suitable for medical decision-making. |
|
The model should not be used as a replacement for professional medical advice or diagnosis. |
|
Sensitive patient data should not be shared with the model, and user privacy should be protected. |