ulvxa/DeepSeek-R1-Distill-Qwen-7B-Lora-Thesis

DeepSeek-R1-7B Wellness Assistant — QLoRA Adapter

A QLoRA fine-tuned LoRA adapter for deepseek-ai/DeepSeek-R1-Distill-Qwen-7B, trained as part of a Bachelor's thesis on generative AI-based wellness assistance.

This adapter specialises the base model across three wellness domains:

• Pharmaceutical information — drug composition, indications, side effects, dosage, manufacturer
• Emotional support — empathetic responses, emotional validation, therapeutic dialogue
• Nutritional information — calorie and macronutrient queries based on the USDA nutrient database

---

Model Details

Field	Value
Developed by	Ulvi Aliyev
Model type	Causal LM — LoRA adapter (PEFT)
Base model	deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
Language	English
License	MIT
Thesis	Evaluation and Implementation of a Generative AI-Based Wellness Assistant

---

Training Details

Parameter	Value
Method	QLoRA (4-bit NF4 quantization)
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training format	Alpaca instruction-tuning
Epochs	3
Effective batch size	32
Learning rate	2e-4
Max sequence length	16,384 tokens
Hardware	NVIDIA A100-SXM4-40GB (Google Colab)
Training time	~14 hours
Total training samples	195,229 instruction pairs

---

Training Dataset

Domain	Source	Samples
Pharmaceutical	Indian drug database	59,125
Therapy	GPT-based therapeutic conversations	60,839
Emotion	Labeled emotion classification dataset	60,000
Nutrition	USDA National Nutrient Database	36,958
Total		216,922 (after deduplication: 195,229)

All samples were converted to the Alpaca instruction-tuning format (instruction, input, output) and shuffled before training.

---

Usage

Install dependencies:

pip install transformers peft bitsandbytes accelerate

Load and run inference:

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

BASE_MODEL = "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"
ADAPTER    = "ulvxa/deepseek-r1-7b-wellness-assistant"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(ADAPTER)
base      = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL, quantization_config=bnb_config, device_map="auto"
)
model     = PeftModel.from_pretrained(base, ADAPTER)
model.eval()


def ask(instruction: str, input_text: str = "") -> str:
    if input_text.strip():
        prompt = (
            "Below is an instruction that describes a task, paired with an input "
            "that provides further context. Write a response that appropriately "
            f"completes the request.\n\n### Instruction:\n{instruction}\n\n"
            f"### Input:\n{input_text}\n\n### Response:\n"
        )
    else:
        prompt = (
            "Below is an instruction that describes a task. Write a response that "
            f"appropriately completes the request.\n\n### Instruction:\n{instruction}"
            "\n\n### Response:\n"
        )
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model.generate(
            **inputs,
            max_new_tokens=256,
            do_sample=False,
            repetition_penalty=1.1,
            pad_token_id=tokenizer.eos_token_id,
        )
    return tokenizer.decode(
        out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True
    ).strip()


# Pharmaceutical
print(ask("What is Prulastin-M Tablet and what is it used for?"))

# Emotional support
print(ask("Respond empathetically to this message.", "i feel so alone lately"))

# Nutrition
print(ask("How many calories are in 100g of cooked brown rice?"))

Tip — concise therapy responses

Therapy outputs can be verbose. Limit to 128 tokens for tighter replies:

out = model.generate(**inputs, max_new_tokens=128, ...)

---

Limitations

• Not a substitute for professional advice. Medical and pharmaceutical information comes from training data and must not replace a licensed physician or pharmacist.
• Therapy verbosity. Emotional support responses tend to be long; cap max_new_tokens if brevity matters.
• Multilingual noise. Occasional Spanish or Chinese fragments may appear in emotional support responses due to multilingual noise in the therapy training split.
• Nutritional approximation. Primary calorie figures are accurate; full macronutrient profiles may drift slightly from USDA ground truth for edge cases.
• English only. The model was trained on English instruction pairs and performs best in English.
• Hallucination risk. Like all instruction-tuned LLMs, the model can generate plausible-sounding but incorrect information. Always verify medical and pharmaceutical output against authoritative sources.

---

Citation

@misc{aliyev2026wellness,
  author = {Ulvi Aliyev},
  title  = {Evaluation and Implementation of a Generative AI-Based Wellness Assistant},
  year   = {2026},
  note   = {Bachelor's Thesis}
}