npm package: @subtextdev/subtext-cli
Description: CLI and SDK for Subtext — FullStory's agent browser testing tool
Latest version: 0.3.5

# @subtextdev/subtext-cli

CLI and SDK for Subtext — drive a hosted browser, capture screenshots, run assertions, record evidence bundles, and validate your app from the command line.

## Installation

```bash
npm install @subtextdev/subtext-cli
```

Or install globally:

```bash
npm install -g @subtextdev/subtext-cli
```

---

## For Agents — Quickstart

This section is first because it's the most important. Everything else expands on it.

**Subtext is agentic engineering validation.** Connect it to your staging environment, your localhost dev server, or your PR preview during code review. It is NOT a general-purpose web scraper — it is built for testing YOUR app, with YOUR sightmap, in YOUR CI.

### 5-Command JSON Sequence

Copy-paste this verbatim. Replace the URL with your app.

```bash
export SUBTEXT_API_KEY="your-api-key"

subtext doctor --json
subtext connect https://staging.myapp.com --json
subtext snapshot $CONN --interactive --json
subtext assert text $CONN "Dashboard" --json
subtext report $CONN --case smoke --out ./evidence --json
```

Replace `$CONN` with the `data.connection_id` from the `connect` output.

**Every command emits a `JsonEnvelope` on stdout:**

```jsonc
// success
{ "command": "connect", "success": true, "data": { ... }, "connectionId": "conn_abc", "durationMs": 312 }

// failure — parse this, don't regex stderr
{ "command": "click", "success": false, "error": { "category": "target_not_found", "message": "...", "suggestion": "..." }, "durationMs": 45 }
```

### Canonical Agentic Workflow

```
doctor --json          # verify env before spending tokens on session setup
  ↓ allPassed: true
connect <url> --json   # opens hosted browser, auto-tunnels localhost, auto-uploads .sightmap/
  ↓ data.connection_id  →  CONN
snapshot $CONN --interactive --json   # component tree with UIDs, sightmap-enriched labels
  ↓ data.components[].uid  →  pick targets
assert text $CONN "expected text" --json   # validate state
click $CONN <uid> --json                    # interact
assert visible $CONN <uid> --json           # verify result
  ↓ 10 assertion types — see below
report $CONN --case smoke --out ./evidence --json   # bundle + score
  ↓ manifest.json with quality score 0–100
disconnect $CONN --json   # cleanup
```

### Error Taxonomy — All 9 Categories

When `success: false`, `error.category` is always one of these. Take action based on the category, not the message string.

| Category | Exit | What Happened | Do This |
|----------|------|---------------|----------|
| `auth_required` | 5 | API key missing or invalid | Check `SECRET_SUBTEXT_API_KEY` / `SUBTEXT_API_KEY`. Run `doctor --json` first. |
| `session_disconnected` | 3 | Session expired or closed | Re-run `connect <url>` to get a new `connection_id`. |
| `timeout_waiting` | 4 | Polling timed out before condition was true | Increase `--timeout` or verify the condition becomes true (check with `snapshot`). |
| `navigation_blocked` | 3 | URL blocked by CORS, CSP, or navigation guard | Check if the app blocks the route. Use `logs --json` to see console errors. |
| `target_offscreen` | 2 | Element exists but is outside the viewport | Scroll into view or `resize` viewport before interacting. |
| `target_obscured` | 2 | Element covered by modal, tooltip, or overlay | Dismiss the overlay first (click or `dialog dismiss`). |
| `target_not_interactive` | 2 | Element is disabled or read-only | Wait for it to become enabled — use `assert visible <uid> --timeout 5000`. |
| `target_stale` | 2 | Component was removed from DOM since last snapshot | Take a fresh `snapshot` and use the new UID. |
| `target_not_found` | 2 | Component UID or selector not found | Take a fresh `snapshot`. Check the UID matches exactly. |

**Exit code pattern:** 0 = success, 1 = assertion fail, 2 = target error, 3 = session/nav error, 4 = timeout, 5 = auth. Use these in shell conditionals.

### Assert Commands Quick Reference

```bash
# 10 assertion types
subtext assert text $CONN "Welcome back" --json
subtext assert url $CONN "/dashboard" --json
subtext assert title $CONN "My App" --json
subtext assert visible $CONN <uid> --json
subtext assert absent $CONN <uid> --json
subtext assert value $CONN <uid> "expected" --json
subtext assert count $CONN <selector> 3 --json
subtext assert modal-closed $CONN "Confirm" --json
subtext assert network $CONN --url-contains "/api/users" --status 200 --json
subtext assert console $CONN --severity error --json   # fails if any console errors

# Polling: retry until pass or timeout
subtext assert text $CONN "Loading done" --timeout 10000 --json

# Screenshot on failure (requires SUBTEXT_SCREENSHOT_DIR)
subtext assert text $CONN "Dashboard" --screenshot-on-fail --json
```

### Evidence Bundle Workflow

```bash
export SUBTEXT_EVIDENCE_DIR=./evidence
export SUBTEXT_SCREENSHOT_DIR=./evidence/screenshots

subtext connect https://staging.myapp.com --json
# ... run your test flow ...
subtext report $CONN --case login-flow --out ./reports/login --json
```

**Output bundle:**

```
reports/login/
  manifest.json       ← quality score + sightmap section
  summary.md          ← human-readable report
  viewer-url.txt      ← FullStory session replay link
  screenshots/        ← failure screenshots
  snapshots/          ← component tree captures
  actions.jsonl       ← every interaction logged
  assertions.jsonl    ← assertion results
  app-filtered.jsonl  ← sightmap-matched API requests
```

**manifest.json schema:**

```jsonc
{
  "version": "1",
  "caseName": "login-flow",
  "connectionId": "conn_abc123",
  "quality": {
    "score": 75,       // (trueCount / 8) * 100
    "trueCount": 6,
    "classes": {
      "visual": true,       // screenshots captured
      "semantic": true,     // component trees captured
      "behavioral": true,   // interaction actions recorded
      "network": true,      // network requests captured
      "console": false,     // console log data captured
      "replay": true,       // FullStory session replay URL
      "persistence": true,  // fill/upload/eval actions
      "environment": false  // session metadata present
    }
  },
  "sightmap": { "loaded": true, "componentCount": 42, "viewCount": 8, "requestCount": 15 }
}
```

### Doctor Command

```bash
subtext doctor --json
```

Runs 6 checks (api-key, node-version, api-reachable, sightmap, screenshot-dir, evidence-dir). Exit 0 if all pass, exit 1 if any fail. Warnings don't affect exit code.

---

## Sightmap — The Competitive Moat

Sightmap is what makes Subtext the right tool for agentic engineering validation, not general automation.

**Without sightmap,** `snapshot` returns raw accessibility tree nodes:

```
btn#83a2f1 (button, 42,892,120,36)
btn#7c4d2e (button, 180,892,80,36)
```

An agent must guess from position (breaks on any layout change) or parse aria-labels (inconsistent). Every interaction is a potential failure.

**With sightmap,** the same snapshot returns:

```
SubmitButton (button, uid=btn#83a2f1) — sightmap: src/Checkout.tsx
CancelButton (button, uid=btn#7c4d2e) — sightmap: src/Checkout.tsx
```

Agents can now say `click $CONN SubmitButton` and know exactly what they're clicking.

### Three Sightmap Sections

**1. Components** — Named UI elements with source file references and memory.

```yaml
components:
  - name: SubmitButton
    selector: "button[data-testid=submit]"
    source: src/Checkout.tsx
    memory:
      - "Disabled until all required fields are filled"
      - "Shows spinner during API call"
```

The `memory` array is injected into agent context alongside each component. This is per-component tribal knowledge — agents use it to avoid known gotchas without re-discovering them every session.

**2. Views** — Route-scoped component groups.

```yaml
views:
  - name: checkout
    route: "/checkout"
    source: src/Checkout.tsx
    components:
      - name: CheckoutForm
        selector: "form.checkout-form"
```

Views tell Subtext which components to expect at which routes. Snapshot output is scoped to the current route — agents get a focused, relevant component list instead of the entire app's DOM.

**3. Requests** — Labeled API calls.

When a sightmap is loaded, Subtext classifies network requests against your component/view structure. Instead of `POST /api/v1/order`, agents see `[checkout.CheckoutForm] POST /api/v1/order`. This makes `assert network` meaningful — you can assert the checkout form actually submitted an order.

### Why This Matters

Other agentic browser tools give you the raw DOM. With Subtext + sightmap:

- **Semantic names**: no positional guesses that break on every deploy
- **Source references**: agent reasons about which code produced each element
- **Memory**: tribal knowledge survives across sessions
- **Route scoping**: snapshot is focused, not the entire app's DOM
- **Request labeling**: network assertions reference your component names

This is what makes Subtext the right tool for code review validation — testing YOUR app with YOUR sightmap, not scraping public websites.

### Sightmap YAML Schema (version 1)

```yaml
version: 1

memory:
  - "Global note: this app uses Tailwind CSS"

components:
  - name: NavBar
    selector: "nav.main-nav"
    source: src/NavBar.tsx
    description: "Main navigation"
    memory:
      - "Has 5 links on desktop, hamburger on mobile"
    children:
      - name: NavLink
        selector: "a.nav-link"

views:
  - name: checkout
    route: "/checkout"
    source: src/Checkout.tsx
    components:
      - name: CheckoutForm
        selector: "form.checkout"
        children:
          - name: SubmitButton
            selector: "button.submit"
```

**Schema fields:**

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `version` | `number` | yes | Schema version, currently `1` |
| `memory` | `string[]` | no | Global notes injected into agent context |
| `components` | `SightmapComponent[]` | no | Top-level named components |
| `views` | `SightmapView[]` | no | Route-scoped component groups |

**SightmapComponent:**

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | `string` | yes | Semantic name (agent uses this as a handle) |
| `selector` | `string \| string[]` | yes | CSS selector(s) |
| `source` | `string` | no | Source file path |
| `description` | `string` | no | Human-readable description |
| `memory` | `string[]` | no | Component-specific notes for agent context |
| `children` | `SightmapComponent[]` | no | Nested child components |

**Auto-upload:** On `connect`, the CLI walks up from cwd (max 5 levels) looking for `.sightmap/` with YAML files. All components are flattened and uploaded. Disable with `--no-hooks`.

---

## Commands

### Connection

#### `connect <url>`

```bash
subtext connect https://staging.myapp.com --json
subtext connect http://localhost:3000 --json          # auto-tunnels localhost
subtext connect http://localhost:3000 --no-tunnel --json
subtext connect http://localhost:3000 --no-hooks --json
```

| Flag | Default | Description |
|------|---------|-------------|
| `--json` | `false` | Emit structured JSON envelope |
| `--hooks` / `--no-hooks` | `true` | Run post-connect hooks (sightmap upload) |
| `--no-tunnel` | `false` | Disable auto-tunnel for localhost URLs |

#### `disconnect <connection_id>`

```bash
subtext disconnect conn_abc123 --json
```

#### `navigate <connection_id> <url>`

```bash
subtext navigate conn_abc123 https://example.com/page2 --json
```

### Inspection

#### `snapshot <connection_id> [view_id]`

```bash
subtext snapshot $CONN --json
subtext snapshot $CONN --interactive --json   # recommended for agents
subtext snapshot $CONN --brief --json
subtext snapshot $CONN --full --json
subtext snapshot $CONN --diff <handle> --json
```

| Flag | Description |
|------|-------------|
| `--json` | Emit structured JSON |
| `--interactive` | Only interactive/visible components |
| `--brief` | Names only (minimal tokens) |
| `--full` | Full tree with all metadata |
| `--diff <handle>` | Diff vs stored snapshot handle |

#### `screenshot <connection_id>`

```bash
subtext screenshot $CONN -o before.png --json
```

#### `logs <connection_id> [level] [limit]`

```bash
subtext logs $CONN --json
subtext logs $CONN error --json
subtext logs $CONN error 50 --json
```

#### `network <connection_id> [pattern] [limit]`

```bash
subtext network $CONN --json
subtext network $CONN "api/" --json
```

#### `eval <connection_id> <expression>`

```bash
subtext eval $CONN "document.title" --json
```

### Assertions

All 10 types return `{ passed, message, expected, actual }` in the `--json` envelope.

```bash
subtext assert text $CONN "Welcome back" --json
subtext assert url $CONN "/dashboard" --json
subtext assert title $CONN "My App" --json
subtext assert visible $CONN <uid> --json
subtext assert absent $CONN <uid> --json
subtext assert value $CONN <uid> "expected@email.com" --json
subtext assert count $CONN ".list-item" 5 --json
subtext assert modal-closed $CONN "Confirm Delete" --json
subtext assert network $CONN --url-contains "/api/users" --status 200 --json
subtext assert console $CONN --severity error --json
```

**Shared assert flags:**

| Flag | Default | Description |
|------|---------|-------------|
| `--json` | `false` | Emit structured JSON assertion result |
| `--timeout <ms>` | `0` | Poll until pass or timeout. 0 = no polling. |
| `--screenshot-on-fail` | `false` | Capture screenshot+snapshot on failure |

Exit 0 = pass, exit 1 = fail.

### Report

```bash
subtext report $CONN --case smoke --out ./evidence --json
```

| Flag | Required | Description |
|------|----------|-------------|
| `--case <name>` | yes | Test case name |
| `--out <dir>` | yes | Output directory |
| `--json` | no | Emit structured JSON |

Requires `SUBTEXT_EVIDENCE_DIR` to be set.

### Doctor

```bash
subtext doctor --json
subtext doctor   # human-readable text output
```

Runs 6 checks. Exit 0 if all pass (warnings ok), exit 1 if any fail.

### Benchmark

```bash
subtext benchmark compare ./benchmarks/results/before ./benchmarks/results/after --out compare.md
subtext benchmark compare ./run-a ./run-b   # print to stdout
```

**Results directory convention:**

```
benchmarks/results/
  <run-label>/
    <caseId>.json    ← BenchmarkResult per case
    run-meta.json    ← BenchmarkRunMeta
```

**BenchmarkResult schema:**

```jsonc
{
  "tool": "subtext", "version": "0.3.3",
  "caseId": "checkout-flow", "runNumber": 1, "status": "pass",
  "durationMs": 4832, "passCount": 5, "failCount": 0,
  "evidenceScore": 87,
  "truthsProved": [{ "label": "checkout page loads", "assertionType": "url", "proved": true }],
  "truthsNotProved": [], "startedAt": "2025-04-09T12:00:00Z"
}
```

### Interaction

```bash
subtext click $CONN <uid> --json
subtext fill $CONN <uid> "value" --json
subtext fill-multi $CONN '[{"component_id":"email","value":"a@b.com"}]' --json
subtext hover $CONN <uid> --json
subtext keypress $CONN Enter --json
subtext drag $CONN <uid> 200 0 --json
subtext wait $CONN selector ".loading-done" --json
subtext wait $CONN text "Welcome back" --json
subtext dialog $CONN accept --json
subtext upload $CONN <uid> ./photo.png --json
```

### Viewport & Device

```bash
subtext resize $CONN 1440 900 --json   # desktop
subtext resize $CONN 768 1024 --json   # tablet
subtext resize $CONN 375 812 --json    # mobile
subtext emulate $CONN "iPhone 14" --json
```

### Tabs

```bash
subtext tabs $CONN --json
subtext new-tab $CONN https://example.com --json
subtext close-tab $CONN <view_id> --json
```

### Review

```bash
subtext review open "https://app.fullstory.com/session/..." --json
subtext review view <client_id> <page_id> <timestamp_ms> --json
subtext review view <client_id> <page_id> <timestamp_ms> --upload --json
subtext review diff <client_id> <page_id> <from_ms> <to_ms> --json
subtext review close <client_id> --json
```

### Comments

```bash
subtext comments list <session_id>
subtext comments add <session_id> "VERIFIED: login works" --intent looks-good
subtext comments add <session_id> "ISSUE: button broken" --intent bug --screenshot-url "..."
subtext comments reply <session_id> <comment_id> "Fixed in latest commit"
subtext comments resolve <session_id> <comment_id>
subtext comments watch <session_id>   # long-running poller
```

### Sightmap

```bash
subtext sightmap show
subtext sightmap upload <upload_url>
```

### Advanced

```bash
subtext tools
subtext raw <tool_name> <json>
subtext tunnel <relayUrl> --target http://localhost:8080
subtext get-skill
subtext get-skill --json
```

---

## Comment Templates

### VERIFIED (intent: `looks-good`)

```bash
subtext raw comment-add '{"session_id":"<session_id>","intent":"looks-good","screenshot_url":"<signed_url>","text":"VERIFIED: Dashboard renders correctly. Zero console errors."}'
```

### ISSUE (intent: `bug`)

```bash
subtext raw comment-add '{"session_id":"<session_id>","intent":"bug","screenshot_url":"<signed_url>","text":"ISSUE [p1]: Submit button unresponsive.\\nSteps: 1. Fill form 2. Click submit 3. Nothing happens"}'
```

### SESSION SUMMARY (intent: `looks-good`)

```bash
subtext raw comment-add '{"session_id":"<session_id>","intent":"looks-good","text":"SESSION SUMMARY:\\nTested: login, dashboard, logout\\nPassed: 3/3\\nFailed: 0\\nNext steps: None — ready for PR"}'
```

---

## Environment Variables

| Variable | Description |
|----------|-------------|
| `SECRET_SUBTEXT_API_KEY` | API key (preferred, auto-set by MCP hosts) |
| `SUBTEXT_API_KEY` | API key (fallback) |
| `SUBTEXT_API_URL` | Override the MCP endpoint |
| `SUBTEXT_SCREENSHOT_DIR` | Directory to auto-save screenshots |
| `SUBTEXT_EVIDENCE_DIR` | Directory for session evidence recording |
| `SUBTEXT_NO_HOOKS` | Set to `1` to disable post-connect hooks |

---

## SDK Usage

```typescript
import { SubtextClient } from "@subtextdev/subtext-cli";

const client = new SubtextClient({ apiKey: process.env.SUBTEXT_API_KEY! });
const conn = await client.connect("https://staging.myapp.com");
const snap = await client.snapshot(conn.connection_id);
await client.click(conn.connection_id, "btn_submit");
await client.disconnect(conn.connection_id);
```

### SDK Methods

| Method | Description |
|--------|-------------|
| `connect(url)` | Open browser and navigate |
| `disconnect(connectionId)` | Close session |
| `snapshot(connectionId, viewId?)` | Screenshot + component tree |
| `screenshot(connectionId, viewId?)` | Screenshot only |
| `navigate(connectionId, url)` | Navigate to URL |
| `newTab(connectionId, url?)` | Open new tab |
| `closeTab(connectionId, viewId)` | Close tab |
| `tabs(connectionId)` | List tabs |
| `emulate(connectionId, device)` | Device emulation |
| `resize(connectionId, width, height)` | Resize viewport |
| `click(connectionId, componentId)` | Click component |
| `fill(connectionId, componentId, value)` | Fill input |
| `hover(connectionId, componentId)` | Hover |
| `keypress(connectionId, key, componentId?)` | Press key |
| `drag(connectionId, componentId, dx, dy)` | Drag component |
| `waitFor(connectionId, type, value)` | Wait for condition |
| `eval(connectionId, expression)` | Execute JS |
| `logs(connectionId, level?, limit?)` | Console messages |
| `network(connectionId, pattern?, limit?)` | Network requests |
| `dialog(connectionId, action, text?)` | Handle dialogs |
| `upload(connectionId, componentId, filePath)` | Upload file |
| `raw(tool, params)` | Call any MCP tool |

---

## Security

The Subtext CLI can execute arbitrary JavaScript via `eval`, tunnel to localhost, and interact with any element. Keep your API key secure — use environment variables, never commit it to source control.