BotMed/Llama-3-8B-Instruct-BotMed-v1

BotMed-v1 is a fine-tuned version of Llama 3 8B Instruct, adapted for medical chatbot use cases using LoRA.

Model Description

BotMed-v1 is a fine-tuned version of Meta's Llama 3 8B Instruct, specialized for medical chatbot and question-answering use cases. It was trained using LoRA (Low-Rank Adaptation) on the ruslanmv/ai-medical-chatbot dataset, with the base model loaded in 4-bit NF4 quantization to enable efficient fine-tuning on Kaggle GPUs.

The LoRA adapters (rank 8, alpha 16) target the attention and MLP projection layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj), allowing the model to adapt to medical dialogue patterns while preserving the general instruction-following capabilities of the base Llama 3 model. Training was run for 2 epochs, using a cosine learning rate schedule and the paged AdamW 8-bit optimizer.

This is the v1 release of BotMed and the first fine-tuning run in the project. It is not a certified diagnostic tool and should not be used as a substitute for professional medical advice.

• Developed by: Bindupautra Jyotibrat (BJyotibrat)
• Shared by: BJyotibrat
• Model type: Causal language model, fine-tuned with LoRA
• Language(s) (NLP): English
• License: Llama 3 Community License
• Finetuned from model: meta-llama/Meta-Llama-3-8B-Instruct

Model Sources

• Repository: GitHub Repository
• Demo: BotMed Official Website

Uses

Direct Use

Intended for medical chatbot / medical question-answering conversational use cases.

Downstream Use

As the first fine-tuning run in the BotMed project, Llama-3-8B-Instruct-BotMed-v1 is best suited to exploratory and internal downstream uses rather than production deployment. Potential applications include:

(1) baseline comparison, serving as a reference point against later BotMed versions (e.g. v2) to evaluate the impact of changes in LoRA configuration, epochs, or dataset handling,

(2) early-stage prototyping of medical chatbot interfaces and RAG pipelines before swapping in a more thoroughly evaluated model,

(3) internal experimentation with prompt design and fine-tuning workflows for medical dialogue generation, and

(4) further fine-tuning or continued training as a starting checkpoint for downstream medical NLP experiments.

(Note: This model is built with Llama and distributed under the Llama 3 Community License. Any downstream applications, derivative models, or software integrating this model must comply with Meta's Acceptable Use Policy and include the required "Built with Llama" attribution.)

Given the lack of formal evaluation for this version (see Bias, Risks, and Limitations), it is not recommended for downstream use cases that reach real end-users without further validation — prefer v2 for those.

Out-of-Scope Use

Not a certified medical diagnostic tool. Unsafe and inappropriate for end-users seeking actual medical advice, diagnosis, or treatment — a licensed human medical professional should be the only resource used for that. Any use must comply with the Llama 3 Acceptable Use Policy.

Bias, Risks, and Limitations

• Fabricated attribution: Because the training data (ruslanmv/ai-medical-chatbot) consists of real doctor-patient Q&A transcripts, the model has learned to sign off responses with fabricated doctor names, specialties, and closing remarks (e.g. "Regards, Dr. Shinas Hussain, General & Family Physician"). These are stylistic artifacts inherited from training data, not real credentialed input, and should never be presented to end-users as if they came from an actual licensed physician.
• No formal evaluation: Unlike v2, this v1 release has not been scored against held-out reference answers using metrics such as F1, ROUGE, Cosine Similarity, or BERTScore. Its quality has only been assessed qualitatively via example prompts.
• No clinical validation: The model has not been evaluated by medical professionals or validated against clinical guidelines.
• Dataset-driven bias: The model's knowledge and tone are shaped entirely by the ruslanmv/ai-medical-chatbot dataset, which may not reflect current medical guidelines and may skew toward specific conditions/specialties represented in that dataset.
• Hallucination risk: As with any LLM, the model can generate plausible-sounding but incorrect or incomplete medical information.

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. In particular:

• Do not present model-generated doctor names, credentials, or signatures as genuine.
• Always pair model output with human review, especially in any workflow that reaches an end-user seeking health guidance.
• Treat outputs as a drafting/assistive aid, not a source of medical truth.
• Given the lack of formal evaluation for this version, prefer v2 for anything beyond experimentation, where possible.

How to Get Started with the Model

Use the code below to get started with the model.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

model_id = "BJyotibrat/Llama-3-8B-Instruct-BotMed-v1"  # full merged model (base + LoRA already merged)

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto",
    dtype=torch.float16,
)

model.eval()

prompts = [
    "I have frequent fever and pain in chest"
]

for idx, t in enumerate(prompts, start=1):
    print(f"\nPrompt {idx}: {t}\n")

    messages = [{"role": "user", "content": t}]
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

    inputs = tokenizer(
        prompt,
        return_tensors="pt",
        truncation=True,
        max_length=2048
    ).to("cuda")

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=512,
            do_sample=True,
            temperature=0.7,
            top_p=0.9,
            repetition_penalty=1.2,
            pad_token_id=tokenizer.eos_token_id
        )

    input_length = inputs["input_ids"].shape[1]
    answer = tokenizer.decode(
        outputs[0][input_length:],
        skip_special_tokens=True,
        clean_up_tokenization_spaces=False
    ).strip()

    print(f"Answer {idx}: {answer}\n")

Training Details

Training Data

Fine-tuned on the ruslanmv/ai-medical-chatbot dataset.

Training Procedure

Fine-tuned using LoRA (Low-Rank Adaptation) on Kaggle GPUs, with the base model loaded in 4-bit quantization via BitsAndBytes.

Preprocessing

Quantization (BitsAndBytes):

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=bnb_config,
    device_map="auto",
    attn_implementation=attn_implementation,
    dtype=torch.float16,
)

model = prepare_model_for_kbit_training(model)

LoRA Configuration:

peft_config = LoraConfig(
    r=8,
    lora_alpha=16,
    lora_dropout=0.1,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
)

model = get_peft_model(model, peft_config)
model.print_trainable_parameters()

Training Hyperparameters

• Training regime: fp16=False, bf16=False (mixed-precision compute via bnb_4bit_compute_dtype=torch.float16)
• Batch size: 2 per device (train and eval), with gradient accumulation steps of 4
• Optimizer: paged_adamw_8bit
• Epochs: 2 configured, with early stopping (training was manually stopped once eval loss stopped improving, ending at ~1.52 epochs)
• Evaluation/Save strategy: every 200 steps
• Logging: every 50 steps
• Warmup steps: 36
• Learning rate: 1e-4, cosine scheduler
• Gradient checkpointing: enabled
• Checkpoints kept: 2 (save_total_limit=2)
• Experiment tracking: Weights & Biases (wandb)

training_arguments = SFTConfig(
    output_dir=new_model,

    per_device_train_batch_size=2,
    per_device_eval_batch_size=2,
    gradient_accumulation_steps=4,

    optim="paged_adamw_8bit",
    num_train_epochs=2,

    eval_strategy="steps",
    eval_steps=200,
    save_strategy="steps",
    save_steps=200,

    logging_steps=50,

    warmup_steps=36,
    learning_rate=1e-4,
    lr_scheduler_type="cosine",

    fp16=False,
    bf16=False,

    report_to="wandb",
    gradient_checkpointing=True,

    load_best_model_at_end=True,
    metric_for_best_model="eval_loss",
    greater_is_better=False,

    save_total_limit=2,
)

Speeds, Sizes, Times

Final logged Weights & Biases metrics (last recorded checkpoint, global step 1800):

Metric	Value
eval/loss	2.271549
eval/entropy	2.144395
eval/mean_token_accuracy	0.509577
eval/num_tokens	376,339
eval/runtime (s)	267.4897
eval/samples_per_second	1.869
eval/steps_per_second	0.935
train/epoch	1.515368

Full run history is available as a CSV export: wandb_run_history - BotMed v1.csv.

Evaluation

Testing Data, Factors & Metrics

Testing Data

No formal held-out evaluation set has been scored for this version.

Results

No formal evaluation metrics (F1, ROUGE, Cosine Similarity, BERTScore) have been computed for this version.

Example Outputs

Prompt: I have frequent fever and pain in chest

Answer: Hello, I have gone through your query. From what you have mentioned, it seems that you are having a dry cough. I would suggest you to use some cough syrups like Tab Robitussin or Tab Mucodyne for 3-5 days. Also, you can use some expectorant like Tab Mucopain for 3-5 days. If you have a fever, then you can use some antipyretic like Tab Paracetamol for 3-5 days.

(Note: raw model outputs include fabricated sign-offs such as doctor names and specialties)

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: Kaggle GPUs
• Hours used: 6 hours
• Cloud Provider: Kaggle

Technical Specifications

Model Architecture and Objective

Llama 3 8B Instruct architecture, fine-tuned via LoRA (rank 8, alpha 16) targeting the q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj modules, for medical chatbot dialogue generation. Base model loaded in 4-bit NF4 quantization for training.

Compute Infrastructure

Kaggle GPUs

Hardware

Kaggle GPU instance(s).

Software

🤗 Transformers, PEFT (LoRA), TRL (SFTConfig/SFTTrainer), BitsAndBytes, Weights & Biases and Kaggle.

Authors

Bindupautra Jyotibrat (BJyotibrat)

Contact

Email: bjyotibrat@gmail.com