--- license: mit datasets: - BrunoGR/HEAR-Hispanic_Emotional_Accompaniment_Responses - >- BrunoGR/HRECPW-Hispanic_Responses_for_Emotional_Classification_based_on_Plutchik_Wheel language: - es tags: - Emotional - Emotional Support - Emotional Accompaniment - chatbot library_name: transformers pipeline_tag: text-generation --- # S贸lo Esc煤chame: Spanish Emotional Accompaniment Chatbot 馃挰馃 S贸lo Esc煤chame is an open-source Spanish emotional assistance chatbot designed to provide psychological support. It is built upon the LLaMA-2-7b-Chat model and fine-tuned using the HEAR (Hispanic Emotional Accompaniment Responses) dataset. ## Overview Mental health issues have been rapidly increasing, with suicide being the fourth leading cause of death among individuals aged 15 to 29 in 2019, according to the World Health Organization (WHO). S贸lo Esc煤chame aims to address this urgent need by offering a supplementary tool for psychological support, especially for Spanish speakers who may not have immediate access to professional help. ## Features - **Emotional Assistance**: Provides empathetic and supportive responses to users' emotional situations. - **HEAR Dataset**: Trained on a specialized dataset for emotional accompaniment, compiled from multiple English sources and translated into Spanish. - **Open-Source**: Available for public use and contribution, facilitating reproducibility and further research. - **CPU Efficient**: Runs efficiently on CPUs, making it accessible to a wider audience. ## Model S贸lo Esc煤chame is a fine-tuned version of the LLaMA-2-7b-Chat model. It utilizes the Rotary Positional Embedding (RoPE) and Grouped-Query Attention (GQA) techniques to enhance context length and model performance. The model has been quantized to 2, 4, and 8 bits to ensure accessibility. ## Training The model was trained using LoRA (Low Rank Adaptation) on the HEAR dataset. The training parameters were optimized for performance and efficiency. ## Dataset ### Hispanic Emotion Recognition Based on Plutchik鈥檚 Wheel (HRECPW) Dataset - **Source**: Translated from diverse English sources including TweetEval, DailyDialog, HappyDB, and survey responses. - **Classes**: 11 emotion classes - affection, happiness, admiration, anger, sadness, optimism, hate, surprise, fear, calm, and disgust. - **Size**: 121,000 training examples, 2,200 validation examples, and 1,320 test examples. ### Hispanic Emotional Accompaniment Responses (HEAR) Dataset - **Purpose**: Used to train the S贸lo Esc煤chame model for generating empathetic and suitable responses. - **Size**: 41,481 training examples, 2,200 validation examples, and 1,320 test examples. ## Evaluation The model's performance was evaluated using two main criteria: ### Active Listening Technique | Evaluation trait | GPT-3.5 | LLaMA-2-7b-Chat | Mixtral8x7b | GPT-2-124M | Solo Esc煤chame | |:----------------------------:|:-------:|:---------------:|:-----------:|:----------:|:--------------:| | **Contextual Attention** | 1256 | 1260 | 1277 | 462 | **1240** | | **Clarifying Questions** | 776 | 718 | 531 | 199 | **913** | | **Deeper Conversation** | 1215 | 1240 | 1185 | 470 | **1254** | | **Absence of Judgment** | 1292 | 1278 | 1299 | 517 | **1300** | | **Demonstration of Empathy** | 1246 | 1274 | 1287 | 502 | **1278** | ### Socratic Method | Evaluation trait | GPT-3.5 | LLaMA-2-7b-Chat | Mixtral8x7b | GPT-2-124M | Solo Esc煤chame | |:-------------------------------------------:|:-------:|:---------------:|:-----------:|:----------:|:--------------:| | **Use of Inductive Questions** | 1077 | 1033 | 872 | 502 | **1224** | | **Non-Imposition of Ideas** | 1236 | 1170 | 1200 | 536 | **1299** | | **Expansion and Construction of Knowledge** | 1031 | 1071 | 972 | 473 | **1245** | | **Generation of Cognitive Dissonance** | 45 | 36 | 34 | 16 | **69** | | **Guided Discovery** | 1089 | 1076 | 988 | 498 | **1253** | ### Final Scores for Psychological Accompaniment Evaluation | **Model** | **Active Listening** | **Socratic Method** | |:-------------------------:|:--------------------:|:-------------------:| | GPT2-124M | 32.57 | 30.68 | | Mixtral 8x7b | 84.52 | 61.60 | | LLaMA-2-7b-Chat | 87.42 | 66.45 | | GPT-3.5 | 87.62 | 67.84 | | **S贸lo Esc煤chame (ours)** | **90.67** | **77.12** | *Table: Final Scores for Psychological Accompaniment Evaluation in Language Models (LMs)* ## Usage The S贸lo Esc煤chame model and datasets are publicly available on Hugging Face: - **Model**: [S贸lo Esc煤chame](https://huggingface.co/BrunoGR/Just_HEAR_Me) - **Datasets**: - [HRECPW Dataset](https://huggingface.co/datasets/BrunoGR/HRECPW-Hispanic_Responses_for_Emotional_Classification_based_on_Plutchik_Wheel) - [HEAR Dataset](https://huggingface.co/datasets/BrunoGR/HEAR-Hispanic_Emotional_Accompaniment_Responses) ## Installation and Setup To use the S贸lo Esc煤chame model, follow these steps: 1. Clone the repository: `git clone https://github.com/BrunoGilRamirez/Just_HEAR_ME` 2. Install the required dependencies: `pip install -r requirements.txt` 3. Load the model and dataset from Hugging Face: `from transformers import AutoModelForCausalLM, AutoTokenizer` ## License S贸lo Esc煤chame is released under the MIT License. ## Citation If you use S贸lo Esc煤chame (Just_HEAR_Me) in your research, please cite the following paper: ```bibtex @article{Gil2024, title={S贸lo Esc煤chame: Spanish Emotional Accompaniment Chatbot}, author={Gil Ram铆rez, Bruno and L贸pez Espejel, Jessica and Santiago D铆az, Mar铆a del Carmen and Rub铆n Linares, Gustavo Trinidad}, journal={arxiv}, year={2024} } ``` ## Contact For any questions or inquiries, please contact: - Bruno Gil Ram铆rez: bruno.gilram@gmail.com