spaceverseai 1.0.0
Advanced Real-Time Voice Chat System with Emotion Detection and AI Integration
License: MIT
Requires Python: >=3.8
Keywords: ai, voice-chat, emotion-detection, openai, websocket, real-time, speech, chatbot, machine-learning

# SpaceVerse AI - Advanced Real-Time Voice Chat System

[![Python](https://img.shields.io/badge/Python-3.8+-blue.svg)](https://python.org)
[![FastAPI](https://img.shields.io/badge/FastAPI-0.100+-green.svg)](https://fastapi.tiangolo.com)
[![OpenAI](https://img.shields.io/badge/OpenAI-GPT--4o-orange.svg)](https://openai.com)
[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

## 🌟 Overview

SpaceVerse AI is a cutting-edge real-time voice chat system that combines OpenAI's GPT-4o with advanced emotion detection, user preference analysis, and dynamic personalization. The system provides an intelligent, emotionally-aware conversational AI that adapts to users' communication styles and emotional states in real-time.

### 🚀 Key Features

- **Real-Time Voice Communication**: Seamless voice chat with OpenAI's GPT-4o
- **Facial Emotion Detection**: Real-time emotion analysis using deep learning models
- **User Preference Learning**: Dynamic adaptation to individual communication styles
- **Threat Detection**: Advanced safety monitoring for crisis intervention
- **Multi-Language Support**: Support for 12+ languages
- **Session Persistence**: Intelligent session management with metadata tracking
- **WebSocket Architecture**: Real-time bidirectional communication
- **Emotional Intelligence**: AI responses that match detected emotions
- **Onboarding System**: Dynamic question generation for personalization

## 🛠️ Installation

### Quick Installation (Recommended)

Install SpaceVerse AI directly from PyPI:

```bash
pip install spaceverseai
```

That's it! All dependencies will be installed automatically.

## 🚀 Quick Start

### Async Usage

```python
import asyncio
from spaceverseai import SpaceverseAIClient, VoiceType, LanguageCode

async def main():
    async with SpaceverseAIClient() as client:
        # Check server health
        health = await client.health_check()
        print(f"Server status: {health.status}")
        
        # Connect to voice chat
        await client.connect_voice_chat()
        
        # Set preferences
        await client.set_user_settings(VoiceType.ALLOY, LanguageCode.ENGLISH)

if __name__ == "__main__":
    asyncio.run(main())
```

### Synchronous Usage

```python
from spaceverseai import SpaceverseAIClientSync, EmotionType

with SpaceverseAIClientSync() as client:
    health = client.health_check()
    print(f"Server health: {health.status}")
    
    client.connect_voice_chat()
    client.set_client_emotion(EmotionType.HAPPY)
```

### Run Examples

```bash
# Run interactive examples
spaceverseai-demo --interactive

# Run all examples
spaceverseai-demo
```

## 📋 Prerequisites

### System Requirements

- **Python**: 3.8 or higher
- **RAM**: Minimum 4GB (8GB recommended)
- **Storage**: 2GB free space
- **Network**: Stable internet connection
- **GPU**: Optional (CUDA-compatible for faster emotion detection)

### API Keys Required

- **OpenAI API Key**: Required for GPT-4o access
- **Environment Variable**: `OPENAI_API_KEY`

## 🏗️ Architecture

### System Components

```
SpaceVerse AI
├── 🎤 Voice Processing
│   ├── Real-time audio streaming
│   ├── Audio format conversion (PCM16)
│   └── Sample rate resampling (24kHz)
├── 🎭 Emotion Detection
│   ├── Facial emotion analysis
│   ├── Real-time image processing
│   └── Emotion-to-response mapping
├── 🧠 AI Intelligence
│   ├── OpenAI GPT-4o integration
│   ├── User preference analysis
│   └── Threat detection system
├── 💾 Session Management
│   ├── Persistent session storage
│   ├── Metadata tracking
│   └── Event logging
└── 🌐 Web Interface
    ├── WebSocket connections
    ├── REST API endpoints
    └── Real-time updates
```

### Technology Stack

- **Backend**: FastAPI (Python)
- **AI Model**: OpenAI GPT-4o Realtime
- **Emotion Detection**: Hugging Face Transformers
- **Real-time Communication**: WebSockets
- **Data Processing**: NumPy, PyTorch
- **Image Processing**: PIL (Pillow)
- **Concurrency**: asyncio, ThreadPoolExecutor

## 🎭 Emotion Detection System

### Facial Emotion Processing

The system uses a pre-trained deep learning model to detect facial emotions in real-time:

```python
# Emotion detection pipeline
processor = AutoImageProcessor.from_pretrained("dima806/facial_emotions_image_detection")
model = AutoModelForImageClassification.from_pretrained("dima806/facial_emotions_image_detection")
```

### Emotion-to-Response Mapping

The AI adapts its communication style based on detected emotions:

| Emotion | Tone | Speech Pattern | Approach |
|---------|------|----------------|----------|
| Happy/Joy | Joyful | Quick | High energy |
| Sad/Sadness | Empathetic | Soft | Gentle |
| Angry/Anger | Calm | Measured | Controlled |
| Fear/Fearful | Reassuring | Slow | Protective |
| Surprise | Engaging | Normal | Curious |
| Disgust | Understanding | Normal | Accepting |
| Neutral | Balanced | Normal | Moderate |

## 🛡️ Safety and Threat Detection

### Crisis Intervention System

The system includes advanced threat detection for user safety:

```python
# Threat detection function for OpenAI
threat_detection_tool = {
    "type": "function",
    "name": "detect_threat_or_suicide",
    "description": "CRITICAL SAFETY FUNCTION: Detect suicidal intent, self-harm, or violence threats",
    "parameters": {
        "type": "object",
        "properties": {
            "contains_threat": {
                "type": "boolean",
                "description": "True if concerning content detected"
            }
        }
    }
}
```

### Safety Features

- **Real-time Monitoring**: Continuous analysis of user messages
- **Context Awareness**: Distinguishes between academic discussion and genuine risk
- **Emotional Context**: Considers facial and voice emotions in threat assessment
- **Crisis Resources**: Provides immediate help when threats are detected
- **False Positive Prevention**: Careful analysis to avoid unnecessary alerts

## 🔧 Configuration

### Environment Variables

| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `OPENAI_API_KEY` | OpenAI API key for GPT-4o access | None | ✅ Yes |
| `LOG_LEVEL` | Logging level (DEBUG, INFO, WARNING, ERROR) | INFO | ❌ No |
| `OPENAI_WS_URL` | OpenAI WebSocket URL | GPT-4o Realtime | ❌ No |

### Model Configuration

The application uses pre-trained models for emotion detection:

```python
# Facial Emotion Detection Model
model = AutoModelForImageClassification.from_pretrained("dima806/facial_emotions_image_detection")
processor = AutoImageProcessor.from_pretrained("dima806/facial_emotions_image_detection")
```

**Supported Emotions**: happy, sad, angry, fearful, surprise, disgust, neutral

## 📊 API Endpoints

### Health and Status

#### GET `/health`
Get application health status and connection information.

**Response:**
```json
{
  "status": "healthy",
  "message": "OpenAI Realtime Voice Chat API is running",
  "connections": {
    "voice_websockets": 2,
    "image_websockets": 1,
    "openai_connections": 2,
    "total_sessions": 2
  }
}
```

### Session Management

#### GET `/api/sessions`
List all active sessions with metadata.

#### GET `/api/session/{session_id}/status`
Get comprehensive session status and configuration.

#### GET `/api/session/{session_id}/history`
Get session information (OpenAI manages conversation history).

#### GET `/api/session/{session_id}/events`
Get OpenAI event IDs tracked for the session.

### Client Configuration

#### GET `/api/client/{client_id}/emotion`
Get current detected emotion for a client.

#### POST `/api/client/{client_id}/emotion`
Manually set detected emotion and trigger session update.

#### GET `/api/client/{client_id}/onboarding-data`
Get current onboarding data for a client.

#### POST `/api/client/{client_id}/onboarding-data`
Set onboarding data and trigger session update.

## 🌐 Multi-Language Support

### Supported Languages

| Language | Code | Voice Support | Transcription |
|----------|------|---------------|---------------|
| English | `en` | ✅ | ✅ |
| Spanish | `es` | ✅ | ✅ |
| French | `fr` | ✅ | ✅ |
| German | `de` | ✅ | ✅ |
| Italian | `it` | ✅ | ✅ |
| Portuguese | `pt` | ✅ | ✅ |
| Russian | `ru` | ✅ | ✅ |
| Japanese | `ja` | ✅ | ✅ |
| Korean | `ko` | ✅ | ✅ |
| Chinese | `zh` | ✅ | ✅ |
| Arabic | `ar` | ✅ | ✅ |
| Hindi | `hi` | ✅ | ✅ |

## 🤝 Contributing

### Development Setup

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new features
5. Submit a pull request

### Code Style

- Follow PEP 8 Python style guide
- Add comprehensive docstrings
- Include type hints
- Write unit tests for new features

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## 🙏 Acknowledgments

- **OpenAI**: For providing the GPT-4o Realtime API
- **Hugging Face**: For the emotion detection models
- **FastAPI**: For the excellent web framework
- **Community**: For feedback and contributions

## 📞 Support

### Getting Help

- **Documentation**: This README and inline code comments
- **Issues**: Create GitHub issues for bugs or feature requests
- **Discussions**: Use GitHub Discussions for questions
- **Email**: Contact the development team

### Common Issues

#### Connection Problems
- Check OpenAI API key is valid
- Verify internet connection
- Check firewall settings

#### Audio Issues
- Ensure microphone permissions
- Check audio format compatibility
- Verify sample rate settings

#### Emotion Detection Issues
- Ensure camera permissions
- Check image quality
- Verify model loading

---

**SpaceVerse AI** - Bringing intelligence and emotion to real-time conversations. 🚀