metadata

library_name: peft
base_model: mistralai/Mistral-7B-Instruct-v0.2

Medical-Mixtral-7B-v2k

Description

Fine-tuned Mixtral model for answering medical assistance questions. This model is a novel version of mistralai/Mixtral-8x7B-Instruct-v0.1, adapted to a subset of 2.0k records from the AI Medical Chatbot dataset, which contains 250k records (https://huggingface.co/datasets/ruslanmv/ai-medical-chatbot). The purpose of this model is to provide a ready chatbot to answer questions related to medical assistance.

Intended Use

This model is intended for providing assistance and answering questions related to medical inquiries. It is suitable for use in chatbot applications where users seek medical advice, information, or assistance.

Installation

pip install -qU transformers==4.36.2 datasets python-dotenv peft bitsandbytes accelerate

Example Usage


from transformers import AutoModelForCausalLM, AutoTokenizer

# Define the name of your fine-tuned model
finetuned_model = 'ruslanmv/Medical-Mixtral-7B-v2k'

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(finetuned_model, trust_remote_code=True)

# Load the model with the provided adapter configuration and weights
model_pretrained = AutoModelForCausalLM.from_pretrained(finetuned_model, trust_remote_code=True, torch_dtype=torch.float16)

messages = [
    {'role': 'user', 'content': 'What should I do to reduce my weight gained due to genetic hypothyroidism?'},
    {'role': 'assistant', 'content': ''},
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors='pt').to('cuda')
outputs = model_pretrained.generate(input_ids, max_new_tokens=500)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

For Gpus


# Define the name of your fine-tuned model
finetuned_model = 'ruslanmv/{new_model}'

# Load fine-tuned model
bnb_config = BitsAndBytesConfig(
    load_in_4bit= True,
    bnb_4bit_quant_type= "nf4",
    bnb_4bit_compute_dtype= torch.bfloat16,
    bnb_4bit_use_double_quant= False,
)
model_pretrained = AutoModelForCausalLM.from_pretrained(
        finetuned_model,
        load_in_4bit=True,
        quantization_config=bnb_config,
        torch_dtype=torch.bfloat16,
        device_map="auto",
        trust_remote_code=True,
        cache_dir=cache_dir
)
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(finetuned_model,
                                          trust_remote_code=True,
                                          cache_dir=cache_dir)
pipe = pipeline(task="text-generation",
                model=model_pretrained, 
                tokenizer=tokenizer, max_length=200)
def build_prompt(question):
  prompt=f"[INST]@Enlighten. {question} [/INST]"
  return prompt

question = "What does abutment of the nerve root mean?"
prompt = build_prompt(question)
result = pipe(prompt)

Training Data

Dataset Name: AI Medical Chatbot
Dataset URL: https://huggingface.co/datasets/ruslanmv/ai-medical-chatbot
Dataset Size: 250k records
Subset Used: 2.0k records

Limitations

The model's performance may vary depending on the complexity and specificity of the medical questions. The model may not provide accurate answers for every medical query, and users should consult medical professionals for critical healthcare concerns.

Ethical Considerations

Users should be informed that the model's responses are generated based on patterns in the training data and may not always be accurate or suitable for medical decision-making. The model should not be used as a replacement for professional medical advice or diagnosis. Sensitive patient data should not be shared with the model, and user privacy should be protected.