SoulTalk LoRA — Marathi Mental Wellness Model
Fine-tuned LoRA adapter on Gemma 2 2B for mental wellness conversations in Marathi-English mix. Built for SoulTalk — AI mental wellness platform for Indian users.
Model Details
- Developed by: Aishhh369
- Base model: unsloth/gemma-2-2b-bnb-4bit
- Model type: LoRA adapter (PEFT)
- Language: Marathi + English (Hinglish style)
- License: CC BY 4.0
- Fine-tuning framework: Unsloth + TRL (SFTTrainer)
- Hardware: Kaggle T4 GPU
Training Data
- Dataset: SoulTalk Marathi Mental Wellness Dataset
- 250 multi-turn conversations, 1242 user-assistant pairs
- 10 categories: Career Stress, Loneliness, Grief/Loss, Health Anxiety, Social Anxiety, Academic Pressure, Family Conflict, Burnout, Relationship Issues, Self-Esteem
Training Hyperparameters
- Epochs: 3
- Learning rate: 2e-4
- Batch size: 1 (gradient accumulation: 4)
- Max sequence length: 1024
- Optimizer: adamw_8bit
- Quantization: 4-bit (bnb)
How to Use
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = "unsloth/gemma-2-2b-bnb-4bit"
tokenizer = AutoTokenizer.from_pretrained("Aishhh369/soultalk-lora")
model = AutoModelForCausalLM.from_pretrained(base_model)
model = PeftModel.from_pretrained(model, "Aishhh369/soultalk-lora")
prompt = "Career chi khup chinta vattey"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0]))
Intended Use
- Mental wellness chatbot for Indian users
- Marathi/Hindi/English multilingual support
- Empathetic response generation
Limitations
- Fine-tuned on synthetic data, real-world performance may vary
- Not a substitute for professional mental health care
- Crisis situations should be redirected to helplines
- Downloads last month
- 27