|
--- |
|
model_name: gemma-2b-mental-health-chatbot-finetuned |
|
tags: |
|
- causal-lm |
|
- fine-tuned |
|
dataset: heliosbrahma/mental_health_chatbot_dataset |
|
description: This model is fine-tuned using the Mental Health Chatbot dataset for |
|
generating responses to mental health-related queries. |
|
--- |
|
## Model Overview |
|
The Mental Health Chatbot model is a fine-tuned version of the Gemma-2B model, adapted to provide empathetic and informative responses to mental health-related queries. The model was fine-tuned using the heliosbrahma/Mental_Health_Chatbot_Dataset, which contains real-world conversations about mental health between patients and healthcare providers. |
|
|
|
|
|
## Model Description |
|
**Developed by**: BM Son, SH Park, SK Hwang |
|
**Activity with**: MLB 2024, Gemma Sprint |
|
**Model type**: Causal Language Model (GemmaCausalLM) |
|
**Finetuned from model**: google/gemma-2b |
|
**API used**: PyTorch and Hugging Face Transformers |
|
**Dataset**: Hugging Face heliosbrahma/mental_health_chatbot_dataset |
|
**Code**: Custom Python Code (shared on Hugging Face or Colab) |
|
**Language(s) (NLP)**: English |
|
**Training**: LoRA (Low-Rank Adaptation) applied with a rank of 32; trained with 8-bit quantization using NF4 type for resource efficiency. |
|
|
|
## Dataset Description |
|
The dataset used for fine-tuning contains conversational pairs where patients ask about various mental health topics and healthcare providers offer advice. All personally identifiable information (PII) has been removed to ensure privacy. |
|
|
|
**Data Fields**: |
|
**Text**: Contains a series of human questions and assistant responses in the format `<HUMAN>`: for questions and `<ASSISTANT>`: for answers. |
|
|
|
## Training Procedure |
|
The model was trained using the LoRA technique for parameter-efficient fine-tuning. It was optimized using 8-bit quantization (NF4 type) to make the training process more memory efficient. The model was trained on a mix of conversational mental health data, focusing on improving its ability to generate contextually relevant and empathetic responses. |
|
|
|
Optimizer: PagedAdamW (8-bit) |
|
Batch Size: 1 per device |
|
Learning Rate: 2e-4 |
|
Max Steps: 100 |
|
Gradient Accumulation: 4 steps |
|
Loss function: Cross-entropy |
|
|
|
## Example Usage |
|
To generate a response to a mental health-related query, you can input a question using the following format: |
|
|
|
```python |
|
query = "How can I manage anxiety?" |
|
response = model.generate_response(query) |
|
print(response) |
|
``` |
|
|
|
**Example Output**: |
|
``` |
|
<HUMAN>: How can I manage anxiety? |
|
<ASSISTANT>: To manage anxiety, consider practicing mindfulness, staying physically active, and seeking support from friends, family, or a professional. Techniques like deep breathing and grounding exercises can also help in calming your mind during anxious moments. |
|
Conclusion |
|
This Mental Health Chatbot aims to serve as a supplemental tool for individuals seeking mental health advice. While it cannot replace professional care, it provides accessible and empathetic responses to common questions, available anytime. |
|
``` |
|
|
|
|
|
## Conclusion |
|
This Mental Health Chatbot aims to serve as a supplemental tool for individuals seeking mental health advice. While it cannot replace professional care, it provides accessible and empathetic responses to common questions, available anytime. |
|
|
