npm package: @khanglvm/ai-router
Description: Generic AI Router Proxy (local + Cloudflare Worker)
Latest version: 1.0.8
Deprecated: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.

# ai-router-proxy

> [!WARNING]
> This package is deprecated and replaced by `@khanglvm/llm-router`.
> Please migrate to `@khanglvm/llm-router` for all new installs and updates.

Generic API proxy for AI providers with:
- Local runtime (`~/.ai-router.json`)
- Cloudflare Worker runtime (`AI_ROUTER_CONFIG_JSON` secret)
- Snap-based CLI + TUI setup/deploy flows
- Unified + dual response endpoints:
  - root/unified: `/`, `/v1`, `/v1/messages`, `/v1/chat/completions`
  - `/anthropic` (Anthropic-compatible responses)
  - `/openai` (OpenAI-compatible responses)
- OS startup install/uninstall/status (macOS/Linux)
- Auto-restart local server when config file changes
- Auto-handle ai-router upgrades for running instances
- Lifecycle commands: `stop`, `reload`, `update`

## Install / Run

```bash
npm install
```

Global CLI (after `npm i -g`):
- `ai-router` or `ai-router start` -> start local proxy
- `ai-router setup` -> provider/model/master-key/startup manager (TUI by default)
- `ai-router deploy` -> Cloudflare deploy helper (TUI by default)
- `ai-router stop` -> stop running instance (startup-managed or manual)
- `ai-router reload` -> restart running instance
- `ai-router update` -> `npm install -g @khanglvm/ai-router@latest` + auto-reload running instance

### Interactive setup (TUI)

```bash
npx ai-router-proxy setup
# or globally:
ai-router setup
```

This command:
- asks for provider name + API key, then endpoint candidates + model list
- requires a provider id (slug/camelCase, e.g. `openrouter` or `myProvider`)
- probes endpoint(s) x model(s) with live requests
- auto-detects supported format(s) per endpoint and model support per format
- saves config to `~/.ai-router.json`

### Non-interactive setup

```bash
npx ai-router-proxy setup \
  --operation=upsert-provider \
  --provider-id=openrouter \
  --name=OpenRouter \
  --api-key=sk-... \
  --endpoints=https://openrouter.ai/api/v1 \
  --models=gpt-4o,claude-3-5-sonnet-latest \
  --headers='{"User-Agent":"Mozilla/5.0"}'
```

If `--headers` is omitted, setup saves a default `User-Agent` header to reduce provider compatibility issues. To remove it explicitly, set `--headers='{\"User-Agent\":null}'`.

### Start local proxy (default command)

```bash
ai-router
# same as:
ai-router start --port=8787
```

`ai-router start`:
- starts local proxy on localhost
- prints proxy URLs and provider/model summary
- exits with guidance if config/providers are missing
- watches `~/.ai-router.json` and auto-restarts on changes (default enabled)
- watches ai-router binary updates and relaunches latest version automatically (default enabled)
- if the requested port is already occupied, it auto-kills the listening process and starts normally

When a startup-managed instance is already running, invoking bare `ai-router` restarts that OS-managed service and exits (instead of launching a second manual instance).

```bash
ai-router start --port=8787 --watch-config=true
```

Secure local mode (require master key on every endpoint):

```bash
ai-router setup --operation=set-master-key --master-key=local_secret_key
ai-router start --require-auth=true
```

Then call local endpoints with:
- `Authorization: Bearer local_secret_key`
- or `x-api-key: local_secret_key`

Local mode reads `~/.ai-router.json` on each request. Auth is optional by default, and enforced when `--require-auth=true`.

Endpoint bases:
- Anthropic-compatible: `http://127.0.0.1:8787/anthropic`
- OpenAI-compatible: `http://127.0.0.1:8787/openai`

### Setup manager operations (CLI mode)

```bash
ai-router setup --operation=list
ai-router setup --operation=remove-provider --provider-id=openrouter
ai-router setup --operation=remove-model --provider-id=openrouter --model=gpt-4o
ai-router setup --operation=set-model-fallbacks --provider-id=openrouter --model=gpt-4o --fallback-models=anthropic/claude-3-7-sonnet,openrouter/gpt-4.1-mini
ai-router setup --operation=set-model-fallbacks --provider-id=openrouter --model=gpt-4o --clear-fallbacks=true
ai-router setup --operation=set-master-key --master-key=your_master_key
```

`set-model-fallbacks` configures silent fallback targets for one source model. In TUI mode it prompts:
- provider -> source model -> multi-select fallback models (`${provider_id}/${model}`)
- available fallback choices include every other configured model across all providers
- stale fallback references are auto-cleaned when providers/models are updated or removed

Runtime routing uses a hybrid fallback strategy:
- Per-request fallback chain: primary model first, then configured fallback models.
- Circuit-assisted reroute: retryable failures (`429`, `408`, `5xx`, network errors) are tracked per `provider/model@format`.
- After repeated retryable failures, that target is temporarily deprioritized so requests start with healthier fallback targets.
- A successful call or a non-retryable failure clears that target's circuit state.

Optional environment variables:
- `AI_ROUTER_FALLBACK_CIRCUIT_FAILURES` (default `2`): consecutive retryable failures required to open circuit.
- `AI_ROUTER_FALLBACK_CIRCUIT_COOLDOWN_MS` (default `30000`): temporary deprioritization window in milliseconds. Set `0` to disable circuit-assisted reroute.

### Lifecycle commands

```bash
ai-router stop
ai-router reload
ai-router update
```

- `stop`: stops the running instance (startup-managed or manual terminal process).
- `reload`: restarts startup-managed service if running; otherwise restarts running manual instance in the current terminal.
- `update`: runs `npm install -g @khanglvm/ai-router@latest` and auto-reloads any running instance.

### OS startup (start with OS)

```bash
ai-router setup --operation=startup-install
ai-router setup --operation=startup-install --require-auth=true
ai-router setup --operation=startup-status
ai-router setup --operation=startup-uninstall
```

On macOS this installs a LaunchAgent, on Linux a `systemd --user` service. The startup service runs `ai-router start` with config-watch enabled.
With `--watch-binary=true` (default), startup-managed instances also self-exit on binary updates so launchd/systemd relaunches the new version.

## Provider Smoke Test Suite (Real-Usage Simulation)

This repo includes a CLI smoke test that simulates real local usage:
- backs up `~/.ai-router.json` (if it exists)
- removes the active file to start clean
- runs `ai-router setup --operation=upsert-provider` for the provided providers/models
- starts `ai-router`
- sends live API requests for every configured provider/model
- prints a terminal report
- removes test config and restores the original `~/.ai-router.json` backup

### Local secrets file (git-ignored)

Use a local env file (already git-ignored by `.gitignore`) to store provider credentials and model lists:

```bash
cp .env.test-suite.example .env.test-suite
```

Fill in your provider details in `.env.test-suite`.

Optional header fields per provider key:
- `AI_ROUTER_TEST_<KEY>_HEADERS_JSON`
- `AI_ROUTER_TEST_<KEY>_OPENAI_HEADERS_JSON`
- `AI_ROUTER_TEST_<KEY>_CLAUDE_HEADERS_JSON`

These must be JSON objects and are passed into `ai-router setup --headers=...`.

### Run the smoke suite

```bash
npm run test:provider-smoke -- --env-file=.env.test-suite --provider-keys=RAMCLOUDS,ZAI
```

You can also pass provider specs directly via CLI JSON:

```bash
node scripts/provider-smoke-suite.mjs --providers-json='[{"id":"myprovider","name":"My Provider","apiKey":"sk-...","openaiBaseUrl":"https://api.example.com/v1","models":["model-a"]}]'
```

Notes:
- If a provider exposes separate OpenAI and Anthropic base URLs, the suite creates endpoint-specific setup entries (e.g. `provider-openai`, `provider-claude`) so both paths are tested.
- `--skip-probe=true` is the default for repeatable test runs. Set `--skip-probe=false` to include live setup probing.

## Config File

Default location: `~/.ai-router.json`

```json
{
  "version": 1,
  "masterKey": "your-worker-master-key",
  "defaultModel": "openrouter/gpt-4o",
  "providers": [
    {
      "id": "openrouter",
      "name": "OpenRouter",
      "baseUrl": "https://openrouter.ai/api/v1",
      "apiKey": "sk-...",
      "format": "openai",
      "formats": ["openai"],
      "auth": { "type": "bearer" },
      "models": [
        { "id": "gpt-4o", "fallbackModels": ["openrouter/claude-3-5-sonnet-latest"] },
        { "id": "claude-3-5-sonnet-latest" }
      ]
    }
  ]
}
```

Model names used by clients must follow:

```txt
${provider}/${model-name}
```

Examples:
- `openrouter/gpt-4o`
- `myProvider/claude-3-5-sonnet-latest`

## Worker Deploy

1) Export worker payload (requires `masterKey` and at least one provider):

```bash
ai-router deploy --export-only=true --out=.ai-router.worker.json
```

2) Upload payload as Worker secret:

```bash
wrangler secret put AI_ROUTER_CONFIG_JSON < .ai-router.worker.json
```

3) Deploy:

```bash
npm run deploy
```

Or one command (requires Wrangler installed/login):

```bash
ai-router deploy
ai-router deploy --env=production
```

Quick master-key rotation on the Worker (runtime override secret, no full config re-export required):

```bash
ai-router worker-key --master-key=new_master_key
ai-router worker-key --env=production --master-key=new_master_key
# or reuse the local config's masterKey:
ai-router worker-key --use-config-key=true
```

The Worker reads config from:
- `AI_ROUTER_CONFIG_JSON` (primary)
- `PROXY_CONFIG_JSON` or `AI_ROUTER_JSON` (aliases)
- `AI_ROUTER_MASTER_KEY` (optional runtime override)

## Endpoints

- `GET /health`
- `POST /` (unified, auto-detect request format)
- `POST /v1` (unified, auto-detect request format)
- `GET /anthropic/v1/models`
- `GET /openai/v1/models`
- `POST /anthropic/v1/messages` (Anthropic-compatible)
- `POST /openai/v1/chat/completions` (OpenAI-compatible)

Backward-compatible routes also work:
- `GET /v1/models`
- `POST /v1/messages`
- `POST /v1/chat/completions`

Worker auth note:
- In Cloudflare Worker mode, all endpoints require the master key (including `GET /health`, `GET /`, and `GET /.../models`).
- Send it as `Authorization: Bearer <masterKey>` (or `x-api-key: <masterKey>`).

## Claude Code Example

Point Claude Code at the Anthropic endpoint and use a qualified model id:

```json
{
  "env": {
    "ANTHROPIC_AUTH_TOKEN": "their_master_key",
    "ANTHROPIC_BASE_URL": "https://your-worker.example.workers.dev/anthropic"
  },
  "ANTHROPIC_DEFAULT_OPUS_MODEL": "provider-name/model-name"
}
```

## Notes

- Request translation currently supports:
  - Claude request -> OpenAI provider
  - OpenAI request -> Claude provider
  - OpenAI stream/non-stream response -> Claude response
- Claude stream/non-stream response -> OpenAI response
- If a provider/model supports the endpoint format directly, the proxy bypasses the translator.