---
library_name: peft
base_model: mistralai/Mistral-7B-Instruct-v0.2
---

# Medical-Mixtral-7B-v2k

## Description
Fine-tuned Mixtral model for answering medical assistance questions. This model is a novel version of mistralai/Mixtral-8x7B-Instruct-v0.1, adapted to a subset of 2.0k records from the AI Medical Chatbot dataset, which contains 250k records (https://huggingface.co/datasets/ruslanmv/ai-medical-chatbot). The purpose of this model is to provide a ready chatbot to answer questions related to medical assistance.

## Intended Use
This model is intended for providing assistance and answering questions related to medical inquiries. It is suitable for use in chatbot applications where users seek medical advice, information, or assistance.

## Installation 

pip install -qU  transformers==4.36.2  datasets python-dotenv peft bitsandbytes accelerate

## Example Usage
```python

from transformers import AutoModelForCausalLM, AutoTokenizer

# Define the name of your fine-tuned model
finetuned_model = 'ruslanmv/Medical-Mixtral-7B-v2k'

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(finetuned_model, trust_remote_code=True)

# Load the model with the provided adapter configuration and weights
model_pretrained = AutoModelForCausalLM.from_pretrained(finetuned_model, trust_remote_code=True, torch_dtype=torch.float16)

messages = [
    {'role': 'user', 'content': 'What should I do to reduce my weight gained due to genetic hypothyroidism?'},
    {'role': 'assistant', 'content': ''},
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors='pt').to('cuda')
outputs = model_pretrained.generate(input_ids, max_new_tokens=500)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))


```

For Gpus
```python

# Define the name of your fine-tuned model
finetuned_model = 'ruslanmv/{new_model}'

# Load fine-tuned model
bnb_config = BitsAndBytesConfig(
    load_in_4bit= True,
    bnb_4bit_quant_type= "nf4",
    bnb_4bit_compute_dtype= torch.bfloat16,
    bnb_4bit_use_double_quant= False,
)
model_pretrained = AutoModelForCausalLM.from_pretrained(
        finetuned_model,
        load_in_4bit=True,
        quantization_config=bnb_config,
        torch_dtype=torch.bfloat16,
        device_map="auto",
        trust_remote_code=True,
        cache_dir=cache_dir
)
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(finetuned_model,
                                          trust_remote_code=True,
                                          cache_dir=cache_dir)
pipe = pipeline(task="text-generation",
                model=model_pretrained, 
                tokenizer=tokenizer, max_length=200)
def build_prompt(question):
  prompt=f"[INST]@Enlighten. {question} [/INST]"
  return prompt

question = "What does abutment of the nerve root mean?"
prompt = build_prompt(question)
result = pipe(prompt)


```

## Training Data
- **Dataset Name:** AI Medical Chatbot
- **Dataset URL:** https://huggingface.co/datasets/ruslanmv/ai-medical-chatbot
- **Dataset Size:** 250k records
- **Subset Used:** 2.0k records

## Limitations
The model's performance may vary depending on the complexity and specificity of the medical questions.
The model may not provide accurate answers for every medical query, and users should consult medical professionals for critical healthcare concerns.

## Ethical Considerations
Users should be informed that the model's responses are generated based on patterns in the training data and may not always be accurate or suitable for medical decision-making.
The model should not be used as a replacement for professional medical advice or diagnosis.
Sensitive patient data should not be shared with the model, and user privacy should be protected.