nflx-aims-copilot 0.1.0
A Model Context Protocol (MCP) server that provides access to AIMS Copilot tools for AB testing and productivity features.
Requires Python: >=3.10

# nflx-aims-copilot

AIMS Productivity Tools MCP Server

This MCP server provides access to AIMS Copilot tools for AB testing and other productivity features.

## Features

### AB Test Metadata Lookup
- **Tool**: `get_ab_test_metadata`
- **Description**: Retrieves current status, state, and metadata for AB tests
- **Usage**: Ideal for obtaining insights into ongoing AB tests and their configurations
- **Example**: `Can you tell me about AB test 65959?`

### Stale Model Detection
- **Tool**: `detect_stale_model`
- **Description**: Analyzes Runway models, AB tests, and Dagobah pipelines to determine if a model is truly stale
- **Usage**: Determine if a model is really stale or merely deallocated
- **Example**: `I got an email about stale models. Can you check if ae-ranking.model.hydrarank.ab68330-cell_3 is really stale or just deallocated?`

### Dagobah Schedules by Group
- **Tool**: `get_dagobah_schedules_by_group`
- **Description**: Retrieves currently active schedules associated with a Dagobah group
- **Usage**: Find all active schedules in a specific Dagobah group
- **Example**: `What are the active schedules in the ae-page-aro Dagobah group?`

### Dagobah Requests by Schedule
- **Tool**: `get_dagobah_requests_by_schedule`
- **Description**: Retrieves requests associated with a specific Dagobah schedule
- **Usage**: Find recent requests for a particular schedule to monitor pipeline activity
- **Example**: `Show me the recent requests for the data-processing-pipeline schedule`

### Dagobah Health Analysis
- **Tool**: `get_dagobah_health`
- **Description**: Analyzes the health of Dagobah schedules or requests
- **Usage**: Monitor pipeline health, identify failures, and track performance metrics
- **Example**: `Look up the schedules for the ae-page-aro Dagobah group and give me a health check on any unhealthy schedules. For any backed up requests, give me a detailed health analysis with links.`

## Prerequisites

- Netflix VPN connection (required for accessing internal services)
- Valid metatron certificates for mTLS authentication

## Tools

### get_ab_test_metadata

Retrieves AB test metadata from Netflix's internal systems.

**Parameters:**
- `test_id` (required): Integer AB test ID to lookup
- `include_info` (optional): Include detailed test information (default: true)
- `include_annotations` (optional): Include test annotations (default: true) 
- `include_plans` (optional): Include allocation plans (default: true)

**Returns:**
JSON object containing AB test metadata including test status, allocation plans, and configuration details.

### detect_stale_model

Analyzes whether a Runway model is truly stale by examining associated AB tests and Dagobah pipelines.

**Parameters:**
- `model_id` (required): String runway model ID to check (e.g., 'ae-ranking.model.hydrarank.ab68330-cell_3')
- `ab_test_id` (optional): AB test ID override (extracted from model_id if not provided)
- `include_runway_data` (optional): Include Runway model data in analysis (default: true)
- `include_dagobah_data` (optional): Include Dagobah pipeline data in analysis (default: true)
- `include_ab_test_data` (optional): Include AB test data in analysis (default: true)

**Returns:**
Formatted report analyzing model staleness with recommendations.

### get_dagobah_schedules_by_group

Retrieves currently active schedules associated with a Dagobah group.

**Parameters:**
- `dagobah_group_name` (required): String name of the Dagobah group to search for

**Returns:**
Formatted text containing list of active schedules with names, IDs, and URLs to Dagobah interface.

### get_dagobah_requests_by_schedule

Retrieves recent requests associated with a specific Dagobah schedule.

**Parameters:**
- `dagobah_schedule_name` (required): String name of the Dagobah schedule to search for

**Returns:**
Formatted text containing list of requests with names, IDs, states, creation times, and URLs to Dagobah interface.

### get_dagobah_health

Analyzes the health of Dagobah schedules or requests with detailed failure analysis and performance metrics.

**Parameters:**
- `dagobah_schedule_names` (optional): Array of schedule names to analyze for health
- `dagobah_request_names` (optional): Array of request names to analyze for health  
- `lookback_days` (optional): Number of days to look back for request history (default: 5)
- `production_only` (optional): When true, only consider requests with branch=master (production) (default: true)

**Note:** Must provide either `dagobah_schedule_names` or `dagobah_request_names`.

**Returns:**
Detailed health report including failure rates, performance statistics, error analysis, and links to failed executions.

## Installing and Running the MCP Client

There are 3 main ways to use the server: running locally, running through VSCode locally to use through Cline/Roo, and through Dev Workspaces.

### Option 1: Run the MCP Server Locally

Start the MCP server using the simplified `uvx` command:
```bash
uvx nflx-aims-copilot@latest
```

### Option 2: Integrate with VSCode locally (for Cline/Roo)

1. Open the VSCode MCP settings file:
    1. Open the Cline/Roo extension
    2. Click the MCP Server tab at the top (Symbol: 3 horizontal servers)
    3. For Cline: Click on "Installed" -> "Configure MCP Servers". For Roo: "Edit Global MCP"
    
    For Cline:

    - On macOS: `~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json`
    - On Linux: `~/.config/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json`
    - On Windows: `%APPDATA%\Code\User\globalStorage\saoudrizwan.claude-dev\settings\cline_mcp_settings.json`
    
    For Roo:
    
    - On macOS: `~/Library/Application Support/Code/User/globalStorage/rooveterinaryinc.roo-cline/settings/mcp_settings.json`
    - On Linux: `~/.config/Code/User/globalStorage/rooveterinaryinc.roo-cline/settings/mcp_settings.json`
    - On Windows: `%APPDATA%\Code\User\globalStorage\rooveterinaryinc.roo-cline/settings/mcp_settings.json`

2. Add the MCP server configuration:
```json
{
  "mcpServers": {
    "nflx-aims-copilot": {
      "command": "uvx",
      "args": [
        "nflx-aims-copilot@latest"
      ],
      "disabled": false
    }
  }
}
```

### Option 3: Dev Workspaces (go/dev) - non-Claude Code

If you're running a workspace through go/dev, the AIMS Copilot MCP Server is automatically tightly integrated.

1. Install the server (if not already installed automatically):
```bash
sudo apt update && sudo apt install -y nflx-aims-copilot
```

2. Open the VSCode Cline/Roo MPC config file:
    1. Open the Cline/Roo extension
    2. Click the MCP Server tab at the top (Symbol: 3 horizontal servers)
    3. For Cline: Click on "Installed" -> "Configure MCP Servers". For Roo: "Edit Global MCP"

3. Add the MCP server configuration:
```json
{
  "mcpServers": {
    "nflx-aims-copilot": {
      "command": "/apps/nflx-aims-copilot/bin/nflx-aims-copilot",
      "disabled": false
    }
  }
}
```

### Option 3: Dev Workspaces (go/dev) - Claude Code

If you're running a workspace through go/dev, the AIMS Copilot MCP Server is automatically tightly integrated.

1. Install the server (if not already installed automatically):
```bash
sudo apt update && sudo apt install -y nflx-aims-copilot
```

2. Add the MCP server config via `claude mcp add-json "nflx-aims-copilot" '{"command": "/apps/nflx-aims-copilot/bin/nflx-aims-copilot", "disabled": false}'`

In order to have this happen automatically every time you create a new workspace, you can utilize your [dotfiles repo](https://manuals.netflix.net/view/workspaces/mkdocs/main/howto/dotfiles/) to automatically add the tools:

1. Within your dotfiles repo, create an `locations/workspace/installers/zzz_aims_copilot_installer.sh` (note: the name is important as it needs to run after the claude installer) file with the following contents:
```bash
sudo apt update && sudo apt install -y nflx-aims-copilot
claude mcp remove "nflx-aims-copilot" 2>/dev/null || true; claude mcp add-json -s user "nflx-aims-copilot" '{"command": "/apps/nflx-aims-copilot/bin/nflx-aims-copilot", "disabled": false, "permissions":{ "tools": "auto-approve"}}'
```
2. Change the perms to make the script executable: `chmod +x locations/workspace/installers/zzz_aims_copilot_installer.sh`
3. Commit and push this commit to your dotfiles repo.

Now, new workspaces (and restarted workspaces) will install the AIMS Copilot and you will have access to the tools.

## Local Development

### Setup
```bash
# Install dependencies
uv sync

# Install with test dependencies
uv sync --extra test
```

### Configuration
To use this MCP server locally with Claude Desktop, add the following entry to your `~/.claude.json` file under the `mcpServers` section:

```json
{
  "mcpServers": {
    "nflx-aims-copilot": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/your/genai-mcp-servers/src/nflx-aims-copilot",
        "run",
        "nflx-aims-copilot"
      ]
    }
  }
}
```

**Note**: Replace `/path/to/your/genai-mcp-servers/src/nflx-aims-copilot` with the actual path to your local clone of this repository.

### Testing
```bash
# Run all tests
uv run pytest

# Run with verbose output
uv run pytest -v

# Run only unit tests (excludes integration tests that require VPN)
uv run pytest -m "not integration"

# Run integration tests (requires Netflix VPN connection)
uv run pytest -m integration

# Run specific test files
uv run pytest tests/test_aballocmeta.py                           # Core AB metadata tests
uv run pytest tests/test_stale_model_detector.py                  # Stale model detector tests
uv run pytest tests/test_dagobah_schedules_by_group.py             # Dagobah schedules by group tests
uv run pytest tests/test_dagobah_requests_by_schedule.py           # Dagobah requests by schedule tests
uv run pytest tests/test_dagobah_health.py                        # Dagobah health analysis tests
uv run pytest tests/test_integration.py                           # MCP server integration tests

# Run specific integration test files
uv run pytest tests/test_aballocmeta_integration.py               # AB metadata integration tests
uv run pytest tests/test_stale_model_detector_integration.py      # Stale model detector integration tests  
uv run pytest tests/test_dagobah_schedules_by_group_integration.py # Dagobah schedules integration tests
uv run pytest tests/test_dagobah_requests_by_schedule_integration.py # Dagobah requests integration tests
uv run pytest tests/test_dagobah_health_integration.py            # Dagobah health integration tests
```

**Test Requirements:**
- Unit tests: No special requirements, use mocked data
- Integration tests: Require Netflix VPN connection and valid metatron certificates
- Integration tests are automatically skipped when VPN is not available

### Building and Publishing

To prepare the package for distribution:

1. Sync dependencies and update lockfile:
```bash
uv sync
```

2. Build package distributions:
```bash
uv build
```

This will create source and wheel distributions in the `dist/` directory.

3. Publish to PyPI:
```bash
uv publish
```

Note: You'll need to set PyPI credentials via environment variables or command flags:
- Token: `--token` or `UV_PUBLISH_TOKEN`
- Or username/password: `--username`/`UV_PUBLISH_USERNAME` and `--password`/`UV_PUBLISH_PASSWORD`

## Claude Code Aliases

For quick access to AIMS Copilot functionality from the command line, you can add these aliases to your `~/.bashrc`, `~/.zshrc`, or equivalent shell config file:

```bash
# AIMS Copilot CLI Aliases using Claude Code

# Stale Model Analyzer - paste email content and get analysis
stale_model_analyzer() {
    echo "Paste your stale model email content (press Ctrl+D on empty line when done):"
    local email_content=""
    local line

    while IFS= read -r line; do
        email_content+="$line"$'\n'
    done

    if [ -z "$email_content" ]; then
        echo "No content provided. Exiting."
        return 1
    fi

    echo "Analyzing stale models from email..."
    local prompt="I have a stale model email. Extract each model ID from this email and analyze whether each one is truly stale or just deallocated from AB tests.

For each model, use the detect_stale_model tool to get a proper analysis. Format the results as a clear table with:
- Model ID
- AB Test Status (if applicable)
- Recommendation
- Action Required

Email content:
$email_content"

    echo "$prompt" | claude code -p
}

# AB Test Lookup - quick AB test info
ab_lookup() {
    if [ -z "$1" ]; then
        echo "Usage: ab_lookup <test_id>"
        echo "Example: ab_lookup 70198"
        return 1
    fi

    echo "What can you tell me about AB$1? Include status, allocation, and key details." | claude code -p
}

# Dagobah Health Check - check health of schedules in a group
dagobah_health() {
    if [ -z "$1" ]; then
        echo "Usage: dagobah_health <group_name>"
        echo "Example: dagobah_health ae-page-aro"
        return 1
    fi

    echo "Look up the schedules for the dagobah group $1 and for each prod and AB test schedule, give me a health check on any unhealthy schedules. Format as a table with links." | claude code -p
}

# Model Staleness Quick Check - single model analysis
check_model() {
    if [ -z "$1" ]; then
        echo "Usage: check_model <model_id>"
        echo "Example: check_model ae-ranking.model.hydrarank.ab69422-cell_3"
        return 1
    fi

    echo "Can you check if model '$1' is really stale or just deallocated? Give me a quick summary with recommendation." | claude code -p
}
```

### Usage Examples

```bash
# Analyze stale model email (interactive - paste content)
$ stale_model_analyzer
Paste your stale model email content (press Ctrl+D on empty line when done):
[paste email content]
[Ctrl+D]

# Quick AB test lookup
$ ab_lookup 70198

# Check Dagobah group health
$ dagobah_health ae-page-aro

# Single model staleness check
$ check_model ae-ranking.model.hydrarank.ab69422-cell_3
```

### Setup Instructions

1. **Add aliases to your [dotfiles repo](https://manuals.netflix.net/view/workspaces/mkdocs/main/howto/dotfiles/):**
   
   * Add .aims_copilot.aliases to your repo [e.g.](https://github.netflix.net/cstella/dotfiles/blob/main/.aims_copilot.aliases)
   * `source ~/.aims_copilot.aliases` from your `.zshrc` or `.bashrc` [e.g.](https://github.netflix.net/cstella/dotfiles/blob/main/.zshrc#L102)
   * commit changes to your dotfiles repo 

2. **Reload your shell config:**
   
   * Restart any relevant workspaces or, alternatively, call `work dotfiles` and re-login