README.md

---
library_name: transformers
tags:
- unsloth
- trl
- sft
---

# Model Card for CBTLlama: Fine Tuning LLaMA for CBT Thought Distortions

## Model Details

### Model Description

Developed by David Schiff, this Hugging Face transformers model, dubbed CBTLlama, is fine-tuned on the LLaMA-3 8B architecture. 
It is specifically tailored to enhance Cognitive Behavioral Therapy (CBT) by detecting thought distortions and raising possible challenges for them.
The model is trained on synthetic data generated by claude that includes a variety of different demographic and emotional states to produce CBT scenarios,
aiming to make CBT more accessible and effective.

This model is not inteded to use without any professional assistance!

## Disclaimer

### Limitation of Liability

The developer of CBTLlama ("the model") provides this model on an "AS IS" basis and makes no warranties regarding its performance, accuracy, reliability, or suitability for any particular task or to achieve any specific results. The developer expressly disclaims any warranties of fitness for a particular purpose or non-infringement. In no event shall the developer be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this model, even if advised of the possibility of such damage.

This model is not intended to be a substitute for professional advice, diagnosis, or treatment. Users should always seek the advice of qualified health providers with any questions regarding their mental health or medical conditions. The developer assumes no responsibility for errors or omissions in the contents of the model or the consequences of its use.


- **Developed by:** David Schiff
- **Model type:** Fine-tuned LLaMA-3 8B
- **Language(s) (NLP):** English
- **License:** MIT
- **Finetuned from model:** LLaMA-3 8B

### Model Sources

- **Repository:** (URL to GitHub or similar)
- **Paper [optional]:** (Link to any published research or documentation)
- **Demo [optional]:** (Link to a model demonstration or interactive API)

## Uses

### Direct Use

CBTLlama is intended to be used directly by mental health practitioners to train their patients in identifying cognitive distortions
and challenging them.

### Downstream Use

While primarily designed for CBT, this model could be extended to other forms of therapy that require scenario generation or tailored mental health interventions.

### Out-of-Scope Use

This model is not intended to replace therapists or make clinical decisions. It should not be used as the sole method for diagnosing or treating mental health conditions.

## Bias, Risks, and Limitations

The model might exhibit biases based on the demographic data it was trained on. Users should critically assess the scenarios it generates, 
especially when using the model with diverse populations.

### Recommendations

It is recommended that all outputs be reviewed by qualified professionals to ensure they are appropriate and sensitive to individual circumstances.

## How to Get Started with the Model

To start using CBTLlama, you can access the model via the Hugging Face API or download it directly from the repository.

## Training Details

### Training Data

The training data comprised simulated CBT scenarios generated by Claude, based on diverse demographic profiles and emotional states, ensuring broad coverage of potential therapy situations.

dataset:
https://huggingface.co/datasets/epsilon3/cbt-triggering-events
https://huggingface.co/datasets/epsilon3/cbt-cognitive-distortions-analysis

### Training Procedure

#### Training Hyperparameters

- **Training regime:** Mixed precision training for efficiency

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data


#### Metrics


### Results

Results indicated that CBTLlama produces highly accurate detections and challenges of thought distortions.

#### Example
It's completely natural and understandable to feel terrible, remorseful, and regretful in this situation. Letting your team down and causing them unnecessary stress due to procrastination is a difficult thing to cope with, and your emotions are valid. It's important to acknowledge and process these feelings as part of learning and growth.

Here are some potential cognitive distortions and challenges based on the situation you described:

* Overgeneralization: "I always let my team down and cause them stress."
Challenge: Recognize that this specific incident does not define a permanent pattern. Consider times when you have been responsible and met deadlines on time, which discredits the overgeneralization.

* Labeling: "I'm a terrible team member for letting this happen."
Challenge: Avoid defining yourself based on a single event. You are a complex individual with many qualities, and one mistake does not negate your overall value as a team member.

* Magnification (catastrophizing): "This one mistake ruins everything and makes me a failure."
Challenge: Put the situation into perspective. While it was an important deadline, it does not negate all your other contributions and successes. Consider how much this specific incident will matter in the long run.

* Should statements: "I should have managed my time better" or "I shouldn't have let this happen."
Challenge: Replace these "should" statements with more realistic and compassionate language, such as "I wish I had managed my time better" or "I'm sorry this situation occurred." Recognize that everyone makes mistakes and that being hard on yourself is not productive.

Remember, everyone faces challenges and makes mistakes from time to time. The most important thing is to learn from this experience, take responsibility for your actions, and find ways to prevent similar situations in the future. Be kind to yourself and focus on moving forward productively.

## Technical Specifications

### Model Architecture and Objective

The model utilizes the LLaMA-3 architecture with modifications to specifically suit CBT cognitive distortions analysis

## Citation
CBTLlama: Fine Tuning Large Language Models
For Identifying Thought Distortions
David Schiff
davidschiff100@gmail.com