npm package: @relayplane/openclaw
Description: OpenClaw skill adapter for RelayPlane agent optimization
Author: RelayPlane
Latest version: 0.1.0

# @relayplane/proxy

**Intelligent LLM Proxy — Route smarter, spend less.**

RelayPlane is a local proxy that sits between your AI tools (like Claude Code, Cursor, or custom apps) and LLM providers (Anthropic, OpenAI, etc.). It automatically routes your requests to the most cost-effective model for each task, potentially saving 50-80% on API costs.

---

> ⚠️ **Important: Cost Monitoring Required**
>
> RelayPlane routes requests directly to LLM provider APIs using your API keys. **This incurs real costs on your account.**
>
> Before using RelayPlane:
> - Set up billing alerts with your LLM providers (Anthropic, OpenAI, etc.)
> - Monitor your usage through your provider's dashboard
> - Start with test requests to understand routing behavior
> - Review the `/control/stats` endpoint to track request volume
>
> RelayPlane provides cost *optimization*, not cost *elimination*. You are responsible for monitoring your actual API spending.

---

---

## Table of Contents

- [What Does This Do?](#what-does-this-do)
- [Quick Start](#quick-start)
- [Security](#security)
- [How It Works](#how-it-works)
- [Configuration](#configuration)
- [API Reference](#api-reference)
- [Troubleshooting](#troubleshooting)

---

## What Does This Do?

### The Problem

You're using Claude Code, Cursor, or another AI tool. Every request goes to expensive models like Claude Opus or GPT-4, even for simple tasks like "what's 2+2?" or "fix this typo."

### The Solution

RelayPlane intercepts your API calls and:
1. **Analyzes the task** — Is this a simple question or complex analysis?
2. **Routes intelligently** — Simple tasks → cheap models (Haiku), complex tasks → powerful models (Opus)
3. **Learns from outcomes** — Tracks what works and improves over time

**Result:** Same quality results, 50-80% lower costs.

### Key Features

| Feature | What It Does |
|---------|--------------|
| **Cascade Routing** | Starts with cheap model, escalates if response seems uncertain |
| **Model Suffixes** | Add `:cost`, `:fast`, or `:quality` to any model name |
| **Provider Cooldowns** | Automatically skips failing providers |
| **100% Local** | All data stays on your machine — no cloud dependency |
| **Hot Reload** | Edit config file, changes apply immediately |

---

## Quick Start

### Step 1: Install

```bash
npm install -g @relayplane/proxy
```

### Step 2: Start the Proxy

```bash
relayplane-proxy --port 3001 --verbose
```

You should see:
```
RelayPlane proxy listening on http://127.0.0.1:3001
  Endpoints:
    POST /v1/messages          - Native Anthropic API
    POST /v1/chat/completions  - OpenAI-compatible API
  Models: relayplane:auto, relayplane:cost, relayplane:fast, relayplane:quality
```

### Step 3: Point Your Tools at the Proxy

Add this to your shell profile (`~/.bashrc`, `~/.zshrc`, etc.):

```bash
export ANTHROPIC_BASE_URL=http://localhost:3001
```

Then restart your terminal or run `source ~/.bashrc`.

### Step 4: Use Your Tools Normally

Claude Code, Cursor, or any OpenAI-compatible tool will now route through RelayPlane automatically. You don't need to change anything else.

---

## Security

⚠️ **Important: Read this section before deploying.**

### Local-Only by Default

RelayPlane binds to `127.0.0.1` (localhost) by default. This means:

- ✅ Only programs on your computer can access it
- ✅ It's not exposed to your network or the internet
- ✅ No authentication is required (you're the only user)

### What RelayPlane Has Access To

When you route through RelayPlane, it can see:

| Data | Access Level |
|------|--------------|
| Your prompts | Full access (needed for routing decisions) |
| API responses | Full access (needed for format conversion) |
| API keys | Passed through to providers, not stored permanently |
| Usage data | Stored locally in `~/.relayplane/data.db` |

### What RelayPlane Does NOT Do

- ❌ Send data to any external service (except the LLM providers you configure)
- ❌ Store API keys on disk (they're only in memory during requests)
- ❌ Log prompts or responses to files
- ❌ Phone home or check for updates

### If You Expose It to the Network

**Don't expose RelayPlane to the public internet or untrusted networks.** If you do:

- Control endpoints (`/control/*`) have no authentication
- Anyone who can reach the proxy can enable/disable routing
- Anyone can read your config and usage stats
- Consider adding a reverse proxy with authentication (nginx, Caddy, etc.)

### Security Hardening Checklist

| Setting | Recommendation |
|---------|----------------|
| Bind address | Keep as `127.0.0.1` (default) |
| API keys | Use environment variables, not config files |
| Config file | Stored at `~/.relayplane/config.json` with 600 permissions |
| Data file | Stored at `~/.relayplane/data.db` — delete to reset |
| Request size | Limited to 10MB to prevent abuse |

---

## How It Works

### The Request Flow

```
Your Tool (Claude Code, Cursor, etc.)
         │
         ▼
    RelayPlane Proxy (localhost:3001)
         │
         ├─── 1. Extract task type from prompt
         ├─── 2. Check routing rules
         ├─── 3. Select optimal model
         ├─── 4. Forward to provider
         ├─── 5. Convert response format if needed
         └─── 6. Return to your tool
         │
         ▼
    LLM Provider (Anthropic, OpenAI, etc.)
```

### Routing Modes

| Mode | Behavior | When to Use |
|------|----------|-------------|
| `relayplane:auto` | Cascade routing — starts cheap, escalates if needed | Default, best for most cases |
| `relayplane:cost` | Always use cheapest model | When cost is critical |
| `relayplane:fast` | Use fastest model/provider | Real-time applications |
| `relayplane:quality` | Always use best model | When quality is critical |

### Model Suffixes

Append a suffix to any model name to override routing for that request:

```bash
# Force cheap routing for this request
model: "claude-3-5-sonnet:cost"

# Force quality routing
model: "gpt-4o:quality"

# Force fast routing  
model: "relayplane:auto:fast"
```

### Cascade Routing (Default)

When you use `relayplane:auto`, here's what happens:

1. **Send to cheap model first** (e.g., Claude Haiku)
2. **Check response for uncertainty** — phrases like "I'm not sure", "I don't know"
3. **If uncertain, escalate** to the next model (e.g., Claude Sonnet)
4. **Return first confident response**

This means simple questions get fast, cheap answers, while complex questions automatically get smarter models.

### Provider Cooldowns

If a provider fails repeatedly (3 times in 60 seconds by default), RelayPlane temporarily stops sending requests to it. This prevents cascading failures and wasted API calls.

After the cooldown period (120 seconds by default), RelayPlane tries the provider again.

---

## Configuration

### Config File Location

```
~/.relayplane/config.json
```

This file is created automatically on first run. Changes are applied immediately (hot reload).

### Full Config Example

```json
{
  "enabled": true,
  
  "modelOverrides": {
    "claude-opus-4-5": "claude-sonnet-4"
  },
  
  "routing": {
    "mode": "cascade",
    "cascade": {
      "enabled": true,
      "models": [
        "claude-3-haiku-20240307",
        "claude-3-5-sonnet-20241022",
        "claude-3-opus-20240229"
      ],
      "escalateOn": "uncertainty",
      "maxEscalations": 1
    },
    "complexity": {
      "enabled": true,
      "simple": "claude-3-haiku-20240307",
      "moderate": "claude-3-5-sonnet-20241022",
      "complex": "claude-3-opus-20240229"
    }
  },
  
  "reliability": {
    "cooldowns": {
      "enabled": true,
      "allowedFails": 3,
      "windowSeconds": 60,
      "cooldownSeconds": 120
    }
  }
}
```

### Config Options Explained

#### `enabled` (boolean)
- `true` — Normal routing behavior
- `false` — Pure passthrough mode (no routing, no telemetry)

#### `modelOverrides` (object)
Automatically rewrite model names. Useful for forcing cheaper models:

```json
{
  "modelOverrides": {
    "claude-opus-4-5": "claude-sonnet-4",
    "gpt-4o": "gpt-4o-mini"
  }
}
```

#### `routing.mode` ("cascade" | "standard")
- `"cascade"` — Start cheap, escalate on uncertainty (recommended)
- `"standard"` — Use complexity classifier only

#### `routing.cascade.models` (array)
Order matters! First model is tried first (should be cheapest), last model is the fallback (should be best).

#### `routing.cascade.escalateOn` ("uncertainty" | "refusal" | "error")
- `"uncertainty"` — Escalate when model says "I'm not sure", etc.
- `"refusal"` — Escalate when model refuses ("I can't help with that")
- `"error"` — Escalate on API errors only

#### `reliability.cooldowns`
| Setting | Default | Description |
|---------|---------|-------------|
| `enabled` | `true` | Enable provider cooldowns |
| `allowedFails` | `3` | Failures before cooldown |
| `windowSeconds` | `60` | Time window for counting failures |
| `cooldownSeconds` | `120` | How long to cool down |

### Environment Variables

| Variable | Description | Required? |
|----------|-------------|-----------|
| `ANTHROPIC_API_KEY` | Your Anthropic API key | Only if not using Claude Code auth |
| `OPENAI_API_KEY` | Your OpenAI API key | Only for GPT models |
| `GEMINI_API_KEY` | Your Google API key | Only for Gemini models |
| `XAI_API_KEY` | Your xAI API key | Only for Grok models |
| `RELAYPLANE_CONFIG_PATH` | Custom config location | No (default: `~/.relayplane/config.json`) |

---

## API Reference

### Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/v1/messages` | POST | Anthropic native format |
| `/v1/chat/completions` | POST | OpenAI format |
| `/v1/messages/count_tokens` | POST | Count tokens |
| `/v1/models` | GET | List available models |
| `/control/status` | GET | Current config and status |
| `/control/enable` | POST | Enable routing |
| `/control/disable` | POST | Disable routing |
| `/control/config` | POST | Update config |
| `/health` | GET | Health check |

### Request Headers

| Header | Description |
|--------|-------------|
| `Authorization` | Bearer token (passed through to Anthropic) |
| `x-api-key` | API key (alternative to Authorization) |
| `X-RelayPlane-Bypass: true` | Skip routing for this request |
| `X-RelayPlane-Model: <model>` | Force specific model for this request |

### Response Metadata

Successful responses include `_relayplane` metadata:

```json
{
  "id": "msg_xxx",
  "choices": [...],
  "_relayplane": {
    "runId": "abc123",
    "routedTo": "anthropic/claude-3-haiku-20240307",
    "taskType": "question_answering",
    "confidence": 0.42,
    "durationMs": 653,
    "mode": "auto"
  }
}
```

---

## Troubleshooting

### "Connection refused" errors

**Problem:** Your tool can't connect to the proxy.

**Solutions:**
1. Make sure the proxy is running: `relayplane-proxy --port 3001`
2. Check you're using the right port
3. Verify `ANTHROPIC_BASE_URL` is set correctly

### "Missing authentication" errors

**Problem:** RelayPlane doesn't have API credentials.

**Solutions:**
1. For Claude Code: Make sure you're logged in (`claude login`)
2. For API keys: Set `ANTHROPIC_API_KEY` environment variable
3. Check the proxy logs for more details

### Requests are slow

**Problem:** Responses take longer than expected.

**Causes:**
1. Cascade routing is escalating (check logs for "Escalating from...")
2. Provider cooldowns (check logs for "cooled down")
3. Network latency to providers

**Solution:** Use `:cost` suffix to skip cascade, or set `routing.mode: "standard"`

### Config changes aren't applying

**Problem:** You edited the config but nothing changed.

**Solutions:**
1. Check for JSON syntax errors in your config file
2. Look at proxy logs — it should say "Reloaded config from..."
3. Make sure you're editing the right file (`~/.relayplane/config.json`)

### "Unknown model" errors

**Problem:** RelayPlane doesn't recognize a model name.

**Solutions:**
1. Check spelling — model names are exact (e.g., `claude-3-5-sonnet-20241022`)
2. Use `relayplane:auto` instead of specific models
3. Check provider API key is set for that model's provider

---

## Getting Help

- **GitHub Issues:** [github.com/RelayPlane/proxy/issues](https://github.com/RelayPlane/proxy/issues)
- **Documentation:** [relayplane.com/docs](https://relayplane.com/docs)

---

## License

MIT