jpworld/gemma-mental-health

Model Card for Fine-Tuned Gemma-2B (Healthcare Chatbot for Mental Health)

Model Details

Model Description

This is a fine-tuned version of Gemma-2B, optimized for mental health conversations in a healthcare chatbot. The model has been fine-tuned on carefully curated mental health support dialogues to provide empathetic and informed responses.

Developed by: Jaiprakash (MCAOL Final Year Project - IGNOU)
Funded by (optional): Self-funded
Shared by (optional): Jai Prakash
Model type: Causal Language Model (LLM)
Language(s) (NLP): English
License: Apache 2.0 (or specify based on dataset usage)
Fine-tuned from model: google/gemma-2b

Model Sources

Repository: [Hugging Face Model Repo Link]
Paper (optional): Not applicable
Demo (optional): Not available yet

Uses

Direct Use

This model is intended for mental health support chatbots and can be used for:
- Conversational AI for mental health guidance
- Initial screening and support for mental health concerns
- Providing empathy-driven responses to user queries

Downstream Use (optional)

Researchers and developers can fine-tune this model for further improvements or expand it to other healthcare areas.

Out-of-Scope Use

Not a replacement for professional mental health services
Should not be used for diagnosing medical conditions
Not suitable for real-time emergency interventions

Bias, Risks, and Limitations

The model is not a licensed medical professional and should not provide medical advice.
Bias in training data: Since the dataset is curated, biases in responses are possible.
Ethical concerns: Users should be aware of limitations and avoid using the model for critical healthcare decisions.

Recommendations

Users should integrate this model alongside professional healthcare support, ensuring human oversight in medical applications.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "your-hf-username/finetuned-gemma-2b"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

text = "I'm feeling anxious today. What should I do?"
inputs = tokenizer(text, return_tensors="pt")
output = model.generate(**inputs)
print(tokenizer.decode(output[0]))

Training Details

Training Data

Fine-tuned using a dataset of mental health conversations
Sources include publicly available therapy conversations, mental health forums, and synthetic dialogues

Training Procedure

Preprocessing (optional)

Tokenized using Gemma tokenizer
Cleaned text data for neutral and non-triggering responses

Training Hyperparameters

Training Regime: LoRA (Low-Rank Adaptation) applied
Batch size: 2
Epochs: 3
Learning Rate: 5e-5
Precision: FP16 for efficiency

Speeds, Sizes, Times (optional)

Training on Google Colab T4 GPU (or specify your hardware)
Took ~4 hours for fine-tuning

Evaluation

Testing Data, Factors & Metrics

Testing Data

Used a held-out validation set of mental health queries

Factors

Empathy & Relevance of responses
Accuracy in mental health guidance

Metrics

BLEU Score
ROUGE Score
Human Evaluation (optional)

Results

Model shows improved empathy and coherence in responses compared to base Gemma-2B
Can handle basic mental health questions well

Environmental Impact

Hardware Type: Google Colab (T4 GPU)
Hours used: ~4 hours
Cloud Provider: Google
Compute Region: N/A
Carbon Emitted: Minimal (since using shared cloud resources)

Technical Specifications (optional)

Model Architecture and Objective

Based on Gemma-2B, which is a decoder-only transformer model
Optimized for conversational AI tasks

Compute Infrastructure

Hardware: Google Colab (T4 GPU)
Software: PyTorch, Hugging Face Transformers, PEFT (LoRA)

Citation (optional)

If you use this model, please cite:

BibTeX:

@misc{jaiprakash2025gemma2b,
  title={Fine-tuned Gemma-2B for Mental Health Chatbot},
  author={Jaiprakash},
  year={2025},
  howpublished={\url{https://huggingface.co/your-hf-username/finetuned-gemma-2b}}
}

APA:

Jaiprakash. (2025). Fine-tuned Gemma-2B for Mental Health Chatbot. Retrieved from [Hugging Face Link]

Glossary (optional)

LoRA (Low-Rank Adaptation): A technique to efficiently fine-tune large models
Causal LM: A language model trained to predict the next token in a sequence
Empathy Score: A metric to measure the model's empathy in responses

More Information (optional)

Model Repository: [Hugging Face Repo Link]
Contact: Jaiprakash (your email or HF profile)

Inference Providers (optional)

This model can be deployed on:

Hugging Face Inference API (if configured)
Google Colab
Local Machine (with Ollama)