---
model_name: gemma-2b-mental-health-chatbot-finetuned
tags:
- causal-lm
- fine-tuned
dataset: heliosbrahma/mental_health_chatbot_dataset
description: This model is fine-tuned using the Mental Health Chatbot dataset for
  generating responses to mental health-related queries.
---
## Model Overview
The Mental Health Chatbot model is a fine-tuned version of the Gemma-2B model, adapted to provide empathetic and informative responses to mental health-related queries. The model was fine-tuned using the heliosbrahma/Mental_Health_Chatbot_Dataset, which contains real-world conversations about mental health between patients and healthcare providers.


## Model Description
**Developed by**: BM Son, SH Park, SK Hwang    
**Activity with**: MLB 2024, Gemma Sprint    
**Model type**: Causal Language Model (GemmaCausalLM)    
**Finetuned from model**: google/gemma-2b    
**API used**: PyTorch and Hugging Face Transformers    
**Dataset**: Hugging Face heliosbrahma/mental_health_chatbot_dataset   
**Code**: Custom Python Code (shared on Hugging Face or Colab)   
**Language(s) (NLP)**: English     
**Training**: LoRA (Low-Rank Adaptation) applied with a rank of 32; trained with 8-bit quantization using NF4 type for resource efficiency.  

## Dataset Description
The dataset used for fine-tuning contains conversational pairs where patients ask about various mental health topics and healthcare providers offer advice. All personally identifiable information (PII) has been removed to ensure privacy.

**Data Fields**:    
**Text**: Contains a series of human questions and assistant responses in the format `<HUMAN>`: for questions and `<ASSISTANT>`: for answers.

## Training Procedure
The model was trained using the LoRA technique for parameter-efficient fine-tuning. It was optimized using 8-bit quantization (NF4 type) to make the training process more memory efficient. The model was trained on a mix of conversational mental health data, focusing on improving its ability to generate contextually relevant and empathetic responses.

Optimizer: PagedAdamW (8-bit)
Batch Size: 1 per device
Learning Rate: 2e-4
Max Steps: 100
Gradient Accumulation: 4 steps
Loss function: Cross-entropy

## Example Usage
To generate a response to a mental health-related query, you can input a question using the following format:

```python
query = "How can I manage anxiety?"
response = model.generate_response(query)
print(response)
```

**Example Output**:
```
<HUMAN>: How can I manage anxiety?
<ASSISTANT>: To manage anxiety, consider practicing mindfulness, staying physically active, and seeking support from friends, family, or a professional. Techniques like deep breathing and grounding exercises can also help in calming your mind during anxious moments.
Conclusion
This Mental Health Chatbot aims to serve as a supplemental tool for individuals seeking mental health advice. While it cannot replace professional care, it provides accessible and empathetic responses to common questions, available anytime.
```


## Conclusion
This Mental Health Chatbot aims to serve as a supplemental tool for individuals seeking mental health advice. While it cannot replace professional care, it provides accessible and empathetic responses to common questions, available anytime.