checking changes

README.md

@@ -1,79 +1,95 @@
----
-title: OpenTriage AI Engine
-emoji: 🧠
-colorFrom: purple
-colorTo: pink
-sdk: docker
-app_port: 7860
-pinned: false
----
+# OpenTriage AI Engine - Full Backend
 
-# 🧠 OpenTriage AI Engine (Microservice)
+Full-featured AI backend for OpenTriage, lifted directly from the original Python codebase.
 
-This is the **AI Brain** of the OpenTriage platform. It is a specialized Python microservice acting as a "Sidecar" to the main TypeScript backend.
+## Features
 
-It hosts the heavy-lifting AI logic, including RAG (Retrieval-Augmented Generation), Issue Triage, and Mentor Matching, keeping the main application fast and lightweight.
+- **Issue Triage** - AI-powered classification and labeling
+- **RAG Chatbot** - Repository-aware Q&A using retrieval-augmented generation  
+- **Mentor Matching** - Tech stack-based mentor recommendations
+- **Hype Generator** - Celebration messages for PRs
+- **AI Chat** - General assistance for contributors
 
-## 🚀 Features
+## API Endpoints
 
-* **🔍 AI Triage:** Automatically classifies GitHub issues by complexity and type.
-* **📚 RAG Chatbot:** "Chat with Repo" functionality using vector search.
-* **🤝 Mentor Match:** Connects contributors to mentors based on tech stack analysis.
-* **🎉 Hype Generator:** Generates celebratory messages for PR merges.
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/health` | GET | Health check |
+| `/triage` | POST | Issue classification |
+| `/chat` | POST | AI chat assistant |
+| `/rag/chat` | POST | RAG-based Q&A |
+| `/rag/index` | POST | Index repository for RAG |
+| `/rag/suggestions` | GET | Suggested questions |
+| `/mentor-match` | POST | Find mentor matches |
+| `/hype` | POST | Generate PR celebration |
+| `/rag/prepare` | POST | Prepare RAG documents |
+| `/rag/chunks` | GET | Get chunks for embedding |
 
----
+## Quick Start
 
-## 🔗 How to Connect
-
-This service exposes a REST API via **FastAPI**. It is designed to be called by the Main Backend, not directly by users.
-
-### Base URL
-Your API is live at:
-`https://[YOUR_USERNAME]-opentriage-ai-engine.hf.space`
-
-*(Check the "Embed this space" menu in the top right to get your exact direct URL)*
-
-### API Endpoints
-
-| Method | Endpoint | Description |
-| :--- | :--- | :--- |
-| `GET` | `/health` | Server health check (returns 200 OK) |
-| `POST` | `/triage` | Classifies an issue description |
-| `POST` | `/chat` | General AI assistant response |
-| `POST` | `/rag/chat` | Context-aware repository Q&A |
-| `POST` | `/mentor-match` | Finds best matching mentors |
-
----
-
-## 🛠 Deployment Configuration
-
-### 1. Environment Variables (Secrets)
-You must set these in the **Settings** tab of this Space under "Variables and secrets":
-
-| Secret Name | Required? | Description |
-| :--- | :--- | :--- |
-| `OPENROUTER_API_KEY` | **Yes** | Required for all AI generation |
-| `MONGO_URL` | Optional | If your RAG needs persistent vector storage |
-
-### 2. Docker Configuration
-This Space uses a custom Dockerfile to ensure all Python scientific libraries (NumPy, Scikit-learn, LangChain) are installed in a compatible environment.
-
-* **Port:** 7860 (Standard HF Space port)
-* **User:** Runs as non-root user `user` (ID 1000) for security.
-
----
-
-## 💻 Local Development
-
-If you want to run this brain locally:
+### Local Development
 
 ```bash
-# 1. Create venv
+# Create virtual environment
 python -m venv venv
 source venv/bin/activate
 
-# 2. Install
+# Install dependencies
 pip install -r requirements.txt
 
-# 3. Run
-uvicorn main:app --reload --port 8000
+# Set environment variables
+export OPENROUTER_API_KEY=your_key_here
+
+# Run the server
+python main.py
+```
+
+### Docker
+
+```bash
+docker build -t opentriage-ai-engine .
+docker run -p 7860:7860 -e OPENROUTER_API_KEY=your_key opentriage-ai-engine
+```
+
+## Hugging Face Spaces Deployment
+
+1. Create a new Space (Docker SDK)
+2. Add secrets in Space settings:
+   - `OPENROUTER_API_KEY` (required)
+   - `MONGO_URL` (optional, for DB features)
+3. Push this folder to the Space
+
+The Dockerfile is pre-configured for HF Spaces (non-root user, port 7860).
+
+## Environment Variables
+
+| Variable | Required | Description |
+|----------|----------|-------------|
+| `OPENROUTER_API_KEY` | Yes | OpenRouter API key |
+| `MONGO_URL` | No | MongoDB connection string |
+| `FRONTEND_URL` | No | Frontend URL for CORS |
+| `CORS_ORIGINS` | No | Comma-separated CORS origins |
+| `ENVIRONMENT` | No | `development` or `production` |
+
+## Architecture
+
+```
+ai-engine/
+├── main.py              # FastAPI wrapper (new)
+├── Dockerfile           # HF-compliant Docker config
+├── requirements.txt     # Python dependencies
+├── config/
+│   └── settings.py      # Environment configuration
+├── models/              # Pydantic models (original)
+├── services/            # AI services (original, unchanged)
+│   ├── ai_service.py
+│   ├── rag_chatbot_service.py
+│   ├── mentor_matching_service.py
+│   ├── hype_generator_service.py
+│   └── rag_data_prep.py
+└── utils/               # Utilities (original)
+```
+
+## License
+
+MIT - See parent repository LICENSE file.

config/redis.py

@@ -1,225 +0,0 @@
-"""
-Redis Connection Utility
-
-Provides Redis client with connection pooling and helper methods for caching.
-Falls back gracefully when Redis is unavailable.
-"""
-
-import os
-import logging
-import hashlib
-import json
-from typing import Optional, Any
-from redis import Redis, ConnectionPool, RedisError
-
-logger = logging.getLogger(__name__)
-
-# Redis configuration from environment
-REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379")
-DEFAULT_TTL = 86400  # 24 hours in seconds
-
-# Global connection pool
-_redis_pool: Optional[ConnectionPool] = None
-_redis_client: Optional[Redis] = None
-
-
-def get_redis_client() -> Optional[Redis]:
-    """
-    Get Redis client with connection pooling.
-    Returns None if Redis is unavailable.
-    """
-    global _redis_pool, _redis_client
-    
-    if _redis_client is not None:
-        return _redis_client
-    
-    try:
-        if _redis_pool is None:
-            _redis_pool = ConnectionPool.from_url(
-                REDIS_URL,
-                max_connections=10,
-                decode_responses=True,
-                socket_timeout=5,
-                socket_connect_timeout=5
-            )
-        
-        _redis_client = Redis(connection_pool=_redis_pool)
-        
-        # Test connection
-        _redis_client.ping()
-        logger.info(f"Redis connected successfully at {REDIS_URL}")
-        return _redis_client
-        
-    except RedisError as e:
-        logger.warning(f"Redis connection failed: {e}. Caching will be disabled.")
-        return None
-    except Exception as e:
-        logger.error(f"Unexpected error connecting to Redis: {e}")
-        return None
-
-
-def generate_cache_key(prefix: str, data: dict) -> str:
-    """
-    Generate a consistent cache key from a data dictionary.
-    
-    Args:
-        prefix: Key prefix (e.g., "triage", "prediction")
-        data: Dictionary containing the data to hash
-        
-    Returns:
-        Cache key string
-    """
-    # Create a stable JSON representation
-    json_str = json.dumps(data, sort_keys=True, separators=(',', ':'))
-    
-    # Generate SHA256 hash
-    hash_obj = hashlib.sha256(json_str.encode('utf-8'))
-    hash_hex = hash_obj.hexdigest()[:16]  # Use first 16 chars
-    
-    return f"{prefix}:{hash_hex}"
-
-
-def cache_get(key: str) -> Optional[Any]:
-    """
-    Get value from Redis cache.
-    
-    Args:
-        key: Cache key
-        
-    Returns:
-        Cached value (parsed from JSON) or None if not found or error
-    """
-    client = get_redis_client()
-    if client is None:
-        return None
-    
-    try:
-        value = client.get(key)
-        if value is None:
-            return None
-        
-        # Parse JSON
-        return json.loads(value)
-        
-    except RedisError as e:
-        logger.warning(f"Redis get error for key '{key}': {e}")
-        return None
-    except json.JSONDecodeError as e:
-        logger.error(f"Failed to decode cached value for key '{key}': {e}")
-        return None
-    except Exception as e:
-        logger.error(f"Unexpected error getting cache key '{key}': {e}")
-        return None
-
-
-def cache_set(key: str, value: Any, ttl: int = DEFAULT_TTL) -> bool:
-    """
-    Set value in Redis cache with TTL.
-    
-    Args:
-        key: Cache key
-        value: Value to cache (will be JSON serialized)
-        ttl: Time to live in seconds (default: 24 hours)
-        
-    Returns:
-        True if successful, False otherwise
-    """
-    client = get_redis_client()
-    if client is None:
-        return False
-    
-    try:
-        # Serialize to JSON
-        json_value = json.dumps(value)
-        
-        # Set with expiration
-        client.setex(key, ttl, json_value)
-        logger.debug(f"Cached key '{key}' with TTL {ttl}s")
-        return True
-        
-    except RedisError as e:
-        logger.warning(f"Redis set error for key '{key}': {e}")
-        return False
-    except (TypeError, ValueError) as e:
-        logger.error(f"Failed to serialize value for key '{key}': {e}")
-        return False
-    except Exception as e:
-        logger.error(f"Unexpected error setting cache key '{key}': {e}")
-        return False
-
-
-def cache_delete(key: str) -> bool:
-    """
-    Delete value from Redis cache.
-    
-    Args:
-        key: Cache key
-        
-    Returns:
-        True if successful, False otherwise
-    """
-    client = get_redis_client()
-    if client is None:
-        return False
-    
-    try:
-        client.delete(key)
-        logger.debug(f"Deleted cache key '{key}'")
-        return True
-        
-    except RedisError as e:
-        logger.warning(f"Redis delete error for key '{key}': {e}")
-        return False
-    except Exception as e:
-        logger.error(f"Unexpected error deleting cache key '{key}': {e}")
-        return False
-
-
-def cache_exists(key: str) -> bool:
-    """
-    Check if key exists in Redis cache.
-    
-    Args:
-        key: Cache key
-        
-    Returns:
-        True if key exists, False otherwise
-    """
-    client = get_redis_client()
-    if client is None:
-        return False
-    
-    try:
-        return client.exists(key) > 0
-        
-    except RedisError as e:
-        logger.warning(f"Redis exists error for key '{key}': {e}")
-        return False
-    except Exception as e:
-        logger.error(f"Unexpected error checking cache key '{key}': {e}")
-        return False
-
-
-def get_cache_stats() -> dict:
-    """
-    Get Redis cache statistics.
-    
-    Returns:
-        Dictionary with cache stats or empty dict if unavailable
-    """
-    client = get_redis_client()
-    if client is None:
-        return {"status": "unavailable"}
-    
-    try:
-        info = client.info("stats")
-        return {
-            "status": "connected",
-            "total_keys": client.dbsize(),
-            "hits": info.get("keyspace_hits", 0),
-            "misses": info.get("keyspace_misses", 0),
-            "evicted_keys": info.get("evicted_keys", 0),
-        }
-    except RedisError as e:
-        logger.warning(f"Failed to get cache stats: {e}")
-        return {"status": "error", "error": str(e)}

config/turso.py

@@ -99,18 +99,10 @@ class TursoDatabase:
                     receiver_id TEXT NOT NULL,
                     content TEXT NOT NULL,
                     read INTEGER DEFAULT 0,
-                    timestamp TEXT NOT NULL,
-                    edited_at TEXT
+                    timestamp TEXT NOT NULL
                 )
             """)
             
-            # Add edited_at column if it doesn't exist (migration for existing tables)
-            try:
-                conn.execute("ALTER TABLE messages ADD COLUMN edited_at TEXT")
-                conn.commit()
-            except Exception:
-                pass  # Column already exists
-            
             # Mentorships table
             conn.execute("""
                 CREATE TABLE IF NOT EXISTS mentorships (
@@ -158,14 +150,10 @@ class TursoDatabase:
             if timestamp and not isinstance(timestamp, str):
                 timestamp = timestamp.isoformat()
             
-            edited_at = message.get('edited_at')
-            if edited_at and not isinstance(edited_at, str):
-                edited_at = edited_at.isoformat()
-            
             conn.execute(
                 """
-                INSERT OR IGNORE INTO messages (id, sender_id, receiver_id, content, read, timestamp, edited_at)
-                VALUES (?, ?, ?, ?, ?, ?, ?)
+                INSERT OR IGNORE INTO messages (id, sender_id, receiver_id, content, read, timestamp)
+                VALUES (?, ?, ?, ?, ?, ?)
                 """,
                 (
                     message.get('id'),
@@ -173,8 +161,7 @@ class TursoDatabase:
                     message.get('receiver_id'),
                     message.get('content'),
                     1 if message.get('read') else 0,
-                    timestamp,
-                    edited_at
+                    timestamp
                 )
             )
             conn.commit()

models/mentor_leaderboard.py

@@ -2,7 +2,7 @@
 Mentor Leaderboard Models - AI-powered ranking with sentiment analysis.
 """
 from pydantic import BaseModel, Field
-from typing import Optional, List, Dict
+from typing import Optional, List, Dict, Any
 from datetime import datetime, timezone
 from enum import Enum
 import uuid
@@ -58,8 +58,8 @@ class LeaderboardEdit(BaseModel):
     
     # What was changed
     field: str  # Which field was edited
-    old_value: any
-    new_value: any
+    old_value: Any
+    new_value: Any
     reason: Optional[str] = None
     
     timestamp: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))

pytest.ini

@@ -0,0 +1,7 @@
+[pytest]
+testpaths = tests
+python_files = test_*.py
+python_classes = Test*
+python_functions = test_*
+addopts = -v --tb=short
+asyncio_mode = auto

tests/conftest.py

@@ -0,0 +1,61 @@
+"""
+Configuration for pytest fixtures and shared test utilities.
+"""
+
+import pytest
+import os
+import sys
+
+# Add the project root to the path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+# Set test environment variables
+os.environ['TESTING'] = 'true'
+os.environ['JWT_SECRET'] = 'test-jwt-secret-for-testing'
+os.environ['ENVIRONMENT'] = 'test'
+
+
+@pytest.fixture
+def mock_user():
+    """Fixture providing a mock user object."""
+    return {
+        "id": "test-user-123",
+        "username": "testuser",
+        "role": "CONTRIBUTOR",
+        "avatarUrl": "https://github.com/test.png"
+    }
+
+
+@pytest.fixture
+def mock_maintainer():
+    """Fixture providing a mock maintainer user."""
+    return {
+        "id": "maintainer-123",
+        "username": "maintainer",
+        "role": "MAINTAINER",
+        "avatarUrl": "https://github.com/maintainer.png"
+    }
+
+
+@pytest.fixture
+def auth_headers(mock_user):
+    """Fixture providing authorization headers with a valid token."""
+    import jwt
+    import time
+    
+    token = jwt.encode(
+        {
+            "user_id": mock_user["id"],
+            "role": mock_user["role"],
+            "exp": int(time.time()) + 3600
+        },
+        os.environ['JWT_SECRET'],
+        algorithm="HS256"
+    )
+    return {"Authorization": f"Bearer {token}"}
+
+
+@pytest.fixture
+def api_key_headers():
+    """Fixture providing API key headers for service-to-service calls."""
+    return {"X-API-Key": os.environ.get("AI_ENGINE_API_KEY", "")}

tests/test_auth.py

@@ -0,0 +1,118 @@
+"""
+Tests for authentication middleware and helpers.
+"""
+
+import pytest
+import jwt
+import time
+import os
+
+
+class TestMiddlewareAuth:
+    """Test cases for middleware authentication functions."""
+    
+    def test_verify_api_key_valid(self):
+        """Test that valid API key is accepted."""
+        from middleware import verify_api_key
+        
+        # When no API key is configured, all requests should pass
+        os.environ.pop("AI_ENGINE_API_KEY", None)
+        assert verify_api_key(None) is True
+    
+    def test_verify_api_key_with_key_set(self):
+        """Test API key verification when key is set."""
+        import importlib
+        import middleware
+        
+        # Set the API key and reload the module
+        os.environ["AI_ENGINE_API_KEY"] = "test-api-key"
+        importlib.reload(middleware)
+        
+        # Valid key
+        assert middleware.verify_api_key("test-api-key") is True
+        
+        # Invalid key
+        assert middleware.verify_api_key("wrong-key") is False
+        
+        # No key provided
+        assert middleware.verify_api_key(None) is False
+        
+        # Clean up and reload
+        os.environ.pop("AI_ENGINE_API_KEY", None)
+        importlib.reload(middleware)
+    
+    def test_verify_jwt_token_valid(self):
+        """Test JWT token verification with valid token."""
+        from middleware import verify_jwt_token
+        
+        token = jwt.encode(
+            {
+                "user_id": "user-123",
+                "role": "CONTRIBUTOR",
+                "exp": int(time.time()) + 3600
+            },
+            os.environ.get("JWT_SECRET", "test-secret"),
+            algorithm="HS256"
+        )
+        
+        payload = verify_jwt_token(token)
+        assert payload["user_id"] == "user-123"
+        assert payload["role"] == "CONTRIBUTOR"
+    
+    def test_verify_jwt_token_expired(self):
+        """Test JWT token verification with expired token."""
+        from middleware import verify_jwt_token
+        from fastapi import HTTPException
+        
+        token = jwt.encode(
+            {
+                "user_id": "user-123",
+                "role": "CONTRIBUTOR",
+                "exp": int(time.time()) - 3600  # Expired 1 hour ago
+            },
+            os.environ.get("JWT_SECRET", "test-secret"),
+            algorithm="HS256"
+        )
+        
+        with pytest.raises(HTTPException) as exc_info:
+            verify_jwt_token(token)
+        assert exc_info.value.status_code == 401
+        assert "expired" in str(exc_info.value.detail).lower()
+    
+    def test_verify_jwt_token_invalid(self):
+        """Test JWT token verification with invalid token."""
+        from middleware import verify_jwt_token
+        from fastapi import HTTPException
+        
+        with pytest.raises(HTTPException) as exc_info:
+            verify_jwt_token("invalid-token")
+        assert exc_info.value.status_code == 401
+
+
+class TestOriginValidation:
+    """Test cases for origin validation."""
+    
+    def test_validate_origin_allowed(self):
+        """Test that allowed origins pass validation."""
+        from middleware import ALLOWED_ORIGINS
+        
+        allowed = [
+            "http://localhost:3000",
+            "http://localhost:5173",
+            "https://open-triage.vercel.app",
+        ]
+        
+        for origin in allowed:
+            assert origin in ALLOWED_ORIGINS
+    
+    def test_validate_origin_blocked(self):
+        """Test that non-allowed origins are blocked."""
+        from middleware import ALLOWED_ORIGINS
+        
+        blocked = [
+            "http://malicious-site.com",
+            "https://fake-opentriage.com",
+        ]
+        
+        for origin in blocked:
+            assert origin not in ALLOWED_ORIGINS

tests/test_endpoints.py

@@ -0,0 +1,186 @@
+"""
+Tests for API endpoints.
+"""
+
+import pytest
+from fastapi.testclient import TestClient
+import os
+import sys
+
+# Add parent directory to path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+
+class TestHealthEndpoints:
+    """Test cases for health check endpoints."""
+    
+    @pytest.fixture
+    def client(self):
+        """Create a test client."""
+        from main import app
+        return TestClient(app)
+    
+    def test_health_check(self, client):
+        """Test the health check endpoint."""
+        response = client.get("/health")
+        assert response.status_code == 200
+        data = response.json()
+        assert data["status"] == "healthy"
+        assert "timestamp" in data
+    
+    def test_root_endpoint(self, client):
+        """Test the root endpoint returns service info."""
+        response = client.get("/")
+        assert response.status_code == 200
+        data = response.json()
+        assert "service" in data
+        assert "version" in data
+        assert "endpoints" in data
+
+
+class TestTriageEndpoint:
+    """Test cases for the triage endpoint."""
+    
+    @pytest.fixture
+    def client(self):
+        """Create a test client."""
+        from main import app
+        return TestClient(app)
+    
+    def test_triage_requires_auth(self, client):
+        """Test that triage endpoint requires authentication."""
+        response = client.post(
+            "/triage",
+            json={
+                "title": "Test Issue",
+                "body": "This is a test issue body",
+            }
+        )
+        # Should require auth
+        assert response.status_code in [401, 422]  # 422 if validation runs first
+    
+    def test_triage_with_auth(self, client, auth_headers):
+        """Test triage endpoint with valid authentication."""
+        response = client.post(
+            "/triage",
+            json={
+                "title": "Bug: Application crashes on startup",
+                "body": "The application crashes when I try to start it.",
+                "authorName": "testuser",
+                "isPR": False,
+            },
+            headers=auth_headers
+        )
+        # May fail due to AI service, but should not be 401
+        assert response.status_code != 401
+
+
+class TestChatEndpoint:
+    """Test cases for the chat endpoint."""
+    
+    @pytest.fixture
+    def client(self):
+        """Create a test client."""
+        from main import app
+        return TestClient(app)
+    
+    def test_chat_requires_auth(self, client):
+        """Test that chat endpoint requires authentication."""
+        response = client.post(
+            "/chat",
+            json={
+                "message": "Hello, how can you help me?",
+            }
+        )
+        assert response.status_code in [401, 422]
+    
+    def test_chat_with_auth(self, client, auth_headers):
+        """Test chat endpoint with valid authentication."""
+        response = client.post(
+            "/chat",
+            json={
+                "message": "Hello, how can you help me?",
+            },
+            headers=auth_headers
+        )
+        # May fail due to AI service, but should not be 401
+        assert response.status_code != 401
+
+
+class TestRAGEndpoints:
+    """Test cases for RAG chatbot endpoints."""
+    
+    @pytest.fixture
+    def client(self):
+        """Create a test client."""
+        from main import app
+        return TestClient(app)
+    
+    def test_rag_chat_requires_auth(self, client):
+        """Test that RAG chat endpoint requires authentication."""
+        response = client.post(
+            "/rag/chat",
+            json={
+                "question": "How does the authentication work?",
+            }
+        )
+        assert response.status_code in [401, 422]
+    
+    def test_rag_index_requires_auth(self, client):
+        """Test that RAG index endpoint requires authentication."""
+        response = client.post(
+            "/rag/index",
+            json={
+                "repo_name": "facebook/react",
+            }
+        )
+        assert response.status_code in [401, 422]
+    
+    def test_rag_suggestions_public(self, client):
+        """Test that RAG suggestions endpoint is publicly accessible."""
+        response = client.get("/rag/suggestions")
+        # Should not require auth
+        assert response.status_code != 401
+
+
+class TestMentorMatchEndpoint:
+    """Test cases for mentor matching endpoint."""
+    
+    @pytest.fixture
+    def client(self):
+        """Create a test client."""
+        from main import app
+        return TestClient(app)
+    
+    def test_mentor_match_requires_auth(self, client):
+        """Test that mentor match endpoint requires authentication."""
+        response = client.post(
+            "/mentor-match",
+            json={
+                "user_id": "user-123",
+                "username": "testuser",
+            }
+        )
+        assert response.status_code in [401, 422]
+
+
+class TestHypeEndpoint:
+    """Test cases for hype generator endpoint."""
+    
+    @pytest.fixture
+    def client(self):
+        """Create a test client."""
+        from main import app
+        return TestClient(app)
+    
+    def test_hype_requires_auth(self, client):
+        """Test that hype endpoint requires authentication."""
+        response = client.post(
+            "/hype",
+            json={
+                "pr_title": "Add new feature",
+                "additions": 100,
+                "deletions": 10,
+            }
+        )
+        assert response.status_code in [401, 422]