Skill: pathemata-mathemata/clawhub-skill-2 (version 0.1.4)

# Xrouter Xrouter is an open-source inference router that sits between OpenClaw and your LLM providers. It uses a fast, hardware-aware classifier to route each request to the most cost-effective model that can handle the task. This project is MIT licensed. See the [MIT License](LICENSE). **Core Features** - OpenAI-compatible reverse proxy at `POST /v1/chat/completions`. - 3-tier classifier (0 = cheap, 1 = medium, 2 = frontier) with early stream cutoff. - Hardware detection helper to recommend local engine. - Provider selection wizard to choose local and cloud endpoints. - Cache layer with Redis or in-memory LRU fallback. - Full cloud mode when local inference is not viable. - Token tracking dashboard at `/dashboard`. **Workflow** ```mermaid flowchart TD A["Client / OpenClaw request"] --> B["Router (OpenAI-compatible)"] B --> C{"Classifier enabled?"} C -->|No| F["Route to Frontier provider"] C -->|Yes| D["Classifier (0 / 1 / 2)"] D --> E{"Decision"} E -->|0| G["Route to Cheap provider"] E -->|1| M["Route to Medium provider"] E -->|2| F G --> H["Provider adapter (auto or explicit)"] M --> H F --> H H --> I["Upstream API call"] I --> J["Stream/Response back to client"] ``` **Repository Layout** - `src/server.js`: router and streaming proxy. - `src/classifier.js`: classifier call and retry logic. - `src/config.js`: configuration and env parsing. - `src/cache.js`: Redis + LRU cache. - `src/token_tracker.js`: token tracking. - `scripts/check_hw.js`: hardware detection. - `scripts/configure_providers.js`: interactive provider setup. **Requirements** - Node.js 20+. - Local classifier engine (optional). - A frontier provider endpoint (required). **Quickstart** 1. Install dependencies. 2. (Optional) Start a local model server. 3. Run the configuration wizard. 4. Start the router. ```bash npm install npm run configure npm run dev ``` **How To Use** 1. Start your local model server (optional but recommended). 2. Run the wizard to configure providers and models. 3. Start the router. 4. Send OpenAI-compatible requests to the router. 5. Inspect routing decisions in response headers or the dashboard. Example local setup (Ollama): ```bash ollama pull llama3.1 ollama run llama3.1 ``` Run the wizard: ```bash npm run configure ``` Start the router: ```bash npm run dev ``` Test a request: ```bash curl -i http://localhost:3000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"model":"any","messages":[{"role":"user","content":"Fix this sentence: I has a apple."}]}' ``` Look for these headers: - `X-Xrouter-decision`: `0`, `1`, or `2`. - `X-Xrouter-upstream`: `cheap`, `medium`, or `frontier`. Open the dashboard: - `http://localhost:3000/dashboard` Raw usage JSON: - `http://localhost:3000/usage` **Provider Selection (Terminal Wizard)** Run: ```bash npm run configure ``` The wizard: - Scans hardware and recommends a local engine. - Suggests a local classifier model. - Lets you choose provider base URLs, API keys, and model overrides for cheap/medium/frontier routes. - Writes `upstreams.json` and optionally updates `.env`. **Quick Start Mode** - If your machine can run a local model, you can choose Quick Start. - Quick Start auto-configures the local classifier. - Cheap route always uses the same local model as the classifier to avoid Ollama model swapping. - You only need to choose medium and frontier providers/models. - On Apple Silicon (Ollama), the wizard lists installed Ollama models and can auto-download a recommended model. **Routing Behavior** - The classifier is called for each uncached request. - The first `0`, `1`, or `2` token returned decides the route. - If classification fails, the router defaults to the frontier route. - When the classifier is enabled, cheap, medium, and frontier routes must be configured. **Compatibility** - The router accepts OpenAI-style requests and translates when needed. - Provider type can be explicit (`xrouter`, `openai_compatible`, `openai`, `anthropic`, `gemini`, `cohere`, `azure_openai`, `mistral`, `groq`, `together`, `perplexity`) or `auto`. - `auto` infers the provider adapter from the base URL or API key. - Providers that expose OpenAI-compatible endpoints use the `openai_compatible` adapter. - Anthropic/Gemini/Cohere streaming is translated into OpenAI-style SSE chunks. - Non-OpenAI adapters currently support text-only messages and basic sampling params (temperature/top_p/stop). **Token Tracking Dashboard** - `GET /usage`: returns cumulative token usage for `cheap`, `medium`, and `frontier`. - `GET /dashboard`: UI that displays token split and totals. - Local usage is counted inside `cheap` when cheap uses the local model. **Environment Summary** - `HOST`: bind host, default `0.0.0.0`. - `PORT`: bind port, default `3000`. - `ROUTER_API_KEY`: require `Authorization: Bearer <key>`. - `LOG_LEVEL`: log level (debug/info/warn/error). - `LOG_TO_FILE`: set `true` to write logs to files. - `LOG_DIR`: directory for log files (default `./logs`). - `CLASSIFIER_ENABLED`: set `false` to disable local classification. - `CLASSIFIER_BASE_URL`: OpenAI-compatible classifier endpoint. - `CLASSIFIER_MODEL`: classifier model name. - `CLASSIFIER_SYSTEM_PROMPT`: classifier prompt (single line). - `CLASSIFIER_TIMEOUT_MS`: classifier timeout. - `CLASSIFIER_FORCE_STREAM`: force streaming classifier request. - `CLASSIFIER_WARMUP`: warm the classifier on server start. - `CLASSIFIER_WARMUP_DELAY_MS`: delay before warmup request (ms). - `CLASSIFIER_KEEP_ALIVE_MS`: keep-alive interval for classifier warmup (ms). - `CLASSIFIER_LOADING_RETRY_MS`: delay between retries when the model is loading. - `CLASSIFIER_LOADING_MAX_RETRIES`: max retries when the model is loading. - `CHEAP_BASE_URL`: optional, defaults to classifier base URL. - `CHEAP_API_KEY`: cheap provider API key. - `CHEAP_MODEL`: optional model override for cheap route. - `CHEAP_PROVIDER`: provider type for cheap route (`auto` if empty). - `CHEAP_HEADERS`: optional JSON headers for cheap provider (stringified object). - `CHEAP_DEPLOYMENT`: Azure deployment override for cheap route. - `CHEAP_API_VERSION`: Azure API version override for cheap route. - `MEDIUM_BASE_URL`: required when classifier is enabled. - `MEDIUM_API_KEY`: medium provider API key. - `MEDIUM_MODEL`: optional model override for medium route. - `MEDIUM_PROVIDER`: provider type for medium route (`auto` if empty). - `MEDIUM_HEADERS`: optional JSON headers for medium provider (stringified object). - `MEDIUM_DEPLOYMENT`: Azure deployment override for medium route. - `MEDIUM_API_VERSION`: Azure API version override for medium route. - `FRONTIER_BASE_URL`: OpenAI-compatible frontier endpoint. - `FRONTIER_API_KEY`: frontier API key. - `FRONTIER_MODEL`: optional model override for frontier route. - `FRONTIER_PROVIDER`: provider type for frontier route (`auto` if empty). - `FRONTIER_HEADERS`: optional JSON headers for frontier provider (stringified object). - `FRONTIER_DEPLOYMENT`: Azure deployment override for frontier route. - `FRONTIER_API_VERSION`: Azure API version override for frontier route. - `REDIS_URL`: if set, enables Redis cache. **Local Model Installation & Run Guides** Ollama (best for Mac, easiest cross-platform) - Install: [Ollama Quickstart](https://ollama.readthedocs.io/en/quickstart/) - Pull a model: `ollama pull llama3.1` - Run: `ollama run llama3.1` - Base URL: `http://localhost:11434` - Router config: `CLASSIFIER_BASE_URL=http://localhost:11434` `CLASSIFIER_MODEL=llama3.1` vLLM (NVIDIA GPU) - OpenAI server: [vLLM OpenAI Server](https://docs.vllm.ai/en/stable/serving/openai_compatible_server/) - Example: `vllm serve NousResearch/Meta-Llama-3-8B-Instruct --dtype auto --api-key token-abc123` - Base URL: `http://localhost:8000` - Router config: `CLASSIFIER_BASE_URL=http://localhost:8000` [REDACTED_SECRET] TensorRT-LLM (NVIDIA, max speed) - Repo: [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) - Server: [trtllm-serve](https://nvidia.github.io/TensorRT-LLM/1.0.0rc2/commands/trtllm-serve.html) - Base URL: `http://<host>:<port>` - Router config: `CLASSIFIER_BASE_URL=http://<host>:<port>` `CLASSIFIER_MODEL=<your model>` llama.cpp (CPU/AMD fallback) - Repo: [llama.cpp](https://github.com/ggml-org/llama.cpp) - Example: `llama-server -m model.gguf --port 8080` - Base URL: `http://localhost:8080` - Router config: `CLASSIFIER_BASE_URL=http://localhost:8080` `CLASSIFIER_MODEL=<gguf model name>` **Docker** Build and run the router with Redis: ```bash docker compose -f deploy/docker-compose.yml up --build ``` **Hardware Detection** Run: ```bash npm run check-hw ``` This prints the recommended engine: - `tensorrt-llm` for large NVIDIA GPUs. - `vllm` for standard NVIDIA GPUs. - `mlx` for Apple Silicon. - `llama.cpp` for CPU/AMD fallback. **Model List Fetching** - The wizard queries provider model list endpoints when possible. - OpenAI-compatible: `/v1/models` - Anthropic: `/v1/models` - Gemini: `/v1beta/models` - Cohere: `/v1/models` - If listing fails, the wizard falls back to `scripts/cloud_model_catalog.json`. **Star History** [![Star History Chart](https://api.star-history.com/svg?repos=pathemata-mathemata/xrouter&type=Date)](https://star-history.com/#pathemata-mathemata/xrouter&Date)


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: suspicious | SkillSpector: clean (LOW)