DOI: 10.5281/zenodo.16418411
title: AffectGraph

description: # AffectGraph: Emotionally Intelligent Recommendation System for Mental Health

## 🧠 Overview

This repository contains the implementation and research code for **AffectGraph**, a neural-symbolic framework that integrates emotional intelligence into recommendation systems, designed specifically for **personalized mental health interventions**. It combines multimodal emotion recognition, symbolic reasoning, and cognitive-affective modeling to deliver emotionally resonant and context-aware suggestions.

The system introduces:- **AffectGraph Network (AGN)** for modeling emotional trajectories using dynamic graph structures.- **Cognitive-Affective Fusion Strategy (CAFS)** to integrate cognitive priors and contextual semantics for therapeutic alignment.

## 🚀 Key Features

- 🔄 **Multimodal Emotion Understanding**: Incorporates visual, auditory, and contextual data.- 📊 **Temporal and Relational Graph Modeling**: Uses dynamic graph construction to capture emotional dependencies.- 🧩 **Symbolic Integration**: Fuses affective reasoning with structured symbolic priors.- 🧠 **Cognitive-Affective Fusion**: Enhances interpretability and psychological alignment using cognitive priors and contextual modulation.- 📈 **Superior Accuracy**: Outperforms baseline recommender systems across multiple datasets like COCO, MPII, Human3.6M, and LSP.

## 📚 Methodology

### 1. AffectGraph Network (AGN)- Constructs a temporal-affective graph.- Utilizes **Graph Neural Networks (GNNs)** for relational reasoning.- Encodes emotional transitions and context via multimodal fusion and symbolic node embedding.

### 2. Cognitive-Affective Fusion Strategy (CAFS)- Incorporates task-specific context and emotional expectations.- Modulates relational weights dynamically using contextual priors.- Enables **zero-shot generalization** to new emotional contexts.

### 3. Structural Regularization- Enforces **temporal smoothness** and **symbolic consistency** in emotional predictions.- Supports interpretability and alignment with psychological goals.

## 🧪 Datasets & Evaluation

Used benchmark datasets for evaluation:

| Dataset      | Type             | Purpose                                 ||--------------|------------------|-----------------------------------------|| COCO         | 2D Pose Estimation | Multimodal emotion context              || MPII         | 2D Pose Estimation | Real-world emotional diversity          || Human3.6M    | 3D Pose Estimation | Temporal and 3D emotion trajectory      || LSP          | Sports Poses      | Challenging non-standard emotional cues |

Evaluation metrics include:- Accuracy- F1 Score- AUC (Area Under Curve)- Mean Per Joint Position Error (MPJPE)

## 🧰 Tech Stack

- Python 3.x- PyTorch- Graph Neural Networks (GNNs)- Transformer-based Vision Models- Cognitive-Affective Reasoning Blocks

## 📂 Repository Structure

```bash├── models/                 # AGN and CAFS implementation├── datasets/              # Preprocessing scripts for COCO, MPII, etc.├── experiments/           # Training and evaluation code├── utils/                 # Utility functions and helper classes├── configs/               # YAML config files for experiments├── README.md              # This file