npm package: @haystackeditor/verify
Description: Configure Haystack to verify PRs on your repo with fixtures, auth bypass, and visual checks
Author: Haystack
Latest version: 0.2.1

# @haystackeditor/verify

Configure Haystack to verify PRs on your repository with dev server setup, verification commands, and fixtures.

## Quick Start

The fastest way to get started:

```bash
npx @haystackeditor/cli skills install
```

Then invoke the setup in your coding CLI:

| CLI | How to invoke |
|-----|---------------|
| **Claude Code** | `/setup-haystack` |
| **Codex CLI** | `/setup-haystack` or ask "set up haystack verification" |
| **Cursor** | `/setup-haystack` (in Composer) |

## Alternative Installation

If you prefer to register the MCP server manually:

```bash
claude mcp add haystack-verify -- npx @haystackeditor/verify
```

## What `/setup-haystack` Does

This diagnoses your project and guides you through complete setup:
- Creates `.haystack.json` config
- Configures your bundler (Vite/Next.js/Webpack) for sandbox compatibility
- Fixes localhost/hostname issues in your code
- Sets up API keys and secrets
- Adds verification flows

### Available Skills

| Command | Purpose |
|---------|---------|
| `/setup-haystack` | **Start here** - diagnoses project, routes to sub-skills |
| `/prepare-haystack` | Add aria-labels and data-testid for browser automation |
| `/setup-haystack-secrets` | Configure API keys, LLM credentials, secrets |

### 2. Or Create Config Manually

Create `.haystack.json` in your project root:

```yaml
version: "1"
name: my-app

dev_server:
  command: pnpm dev
  port: 3000
  ready_pattern: "Local:"
  env:
    SKIP_AUTH: "true"  # Bypass auth in sandbox

verification:
  commands:
    - name: build
      run: pnpm build
    - name: lint
      run: pnpm lint
    - name: typecheck
      run: pnpm tsc --noEmit

  fixtures:
    "/api/users/*":
      source: "file://fixtures/users.json"
    "/api/config":
      source: "https://staging.myapp.com/api/config"
```

## How It Works

When Haystack runs in a sandbox:

1. **Clone & Install** - Your repo is cloned and dependencies installed
2. **Run Verification Commands** - Runs each command in `verification.commands` during sandbox creation
3. **Cache Results** - Command outputs are cached for instant restoration
4. **Start Dev Server** - Runs `dev_server.command` with `dev_server.env` variables
5. **Wait for Ready** - Waits for `dev_server.ready_pattern` in stdout
6. **Load Fixtures** - Intercepts API calls matching fixture patterns

The sandbox caches verification results, so subsequent sandboxes restore instantly.

## Configuration Reference

### Simple Mode (Single Dev Server)

For projects with a single service:

```yaml
version: "1"
name: my-app

dev_server:
  command: pnpm dev           # Command to start server
  port: 3000                  # Port number
  ready_pattern: "Local:"     # Stdout pattern indicating ready
  env:                        # Environment variables
    NODE_ENV: development
    SKIP_AUTH: "true"         # Bypass auth in sandbox

verification:
  commands:
    - name: build
      run: pnpm build
    - name: lint
      run: pnpm lint
```

### Monorepo Mode (Multiple Services)

For monorepos with multiple services (frontend, backend workers, etc.):

```yaml
version: "1"
name: my-monorepo

services:
  # Frontend web app
  frontend:
    root: ./                    # Directory relative to repo root
    command: pnpm dev
    port: 3000
    ready_pattern: "Local:"
    env:
      SKIP_AUTH: "true"

  # Backend API worker
  api:
    root: infra/api-worker
    command: pnpm wrangler dev --local
    port: 8787
    ready_pattern: "Ready on"
    env:
      MOCK_EXTERNAL_APIS: "true"

  # Batch job (not a long-running server)
  analysis:
    root: analysis
    type: batch               # Run once, don't wait for port
    command: pnpm start
    env:
      LOCAL_OUTPUT: "true"

verification:
  commands:
    - name: build
      run: pnpm build
    - name: lint
      run: pnpm lint
```

### Authentication Bypass

Auth bypass is configured via environment variables in `dev_server.env` or `services[].env`:

```yaml
dev_server:
  command: pnpm dev
  port: 3000
  env:
    # Common patterns - use whichever your app supports
    SKIP_AUTH: "true"
    BYPASS_AUTH: "true"
    AUTH_DISABLED: "1"
    NEXT_PUBLIC_MOCK_AUTH: "true"
```

Your app code should check these env vars and skip authentication when set:

```typescript
// Example auth bypass in your app
const skipAuth = process.env.SKIP_AUTH === 'true';
if (skipAuth) {
  // Return mock user, skip token validation, etc.
}
```

### Verification Commands

Verification commands run during sandbox creation. Results are cached.

```yaml
verification:
  commands:
    - name: build
      run: pnpm build
    - name: lint
      run: pnpm lint
    - name: typecheck
      run: pnpm tsc --noEmit
    - name: test
      run: pnpm test --passWithNoTests
```

Common command templates by project type:

| Project Type | Typical Commands |
|-------------|-----------------|
| Frontend (React/Vue) | `pnpm build`, `pnpm lint`, `pnpm tsc --noEmit` |
| Node.js API | `pnpm build`, `pnpm lint`, `pnpm test` |
| Cloudflare Worker | `pnpm wrangler deploy --dry-run`, `pnpm tsc --noEmit` |
| Python | `pytest`, `mypy .`, `ruff check .` |
| Monorepo | `pnpm -r build`, `pnpm -r lint` |

### Fixtures (Optional)

Fixtures define mock data for API endpoints. They're keyed by URL pattern:

```yaml
verification:
  fixtures:
    # Local file (relative to repo root)
    "/api/config":
      source: "file://fixtures/config.json"

    # Remote URL with API token
    "/api/users/*":
      source: "https://staging.example.com/api/users"
      headers:
        Authorization: "Bearer $STAGING_TOKEN"

    # Remote URL with session cookies (login-required endpoints)
    "/api/dashboard":
      source: "https://staging.example.com/api/dashboard"
      cookies: "$STAGING_SESSION"

    # AWS S3
    "/api/data":
      source: "s3://bucket/data.json"

    # Cloudflare R2
    "/api/reports":
      source: "r2://bucket/reports.json"

    # Dynamic generation via script
    "/api/dynamic":
      source: "script://fixtures/generate.ts"

    # Pass through to real API (no mocking)
    "/api/public/*":
      source: passthrough
```

### Feeding Fixtures to Your App

Defining fixtures in `.haystack.json` tells Haystack *what* data to use. You also need middleware to intercept requests:

**Option 1: Vite Plugin (Recommended for Vite apps)**

```typescript
// vite.config.ts
import { haystackFixtures } from '@haystackeditor/verify/vite';

export default defineConfig({
  plugins: [
    haystackFixtures(), // Reads .haystack.json, intercepts matching routes
  ],
});
```

The plugin activates when `HAYSTACK_FIXTURES=true` is set in your dev_server.env.

**Option 2: MSW (Framework-agnostic)**

```typescript
// src/mocks/handlers.ts
import { haystackHandlers } from '@haystackeditor/verify/msw';

export const handlers = [
  ...haystackHandlers(), // Auto-generated from .haystack.json
];
```

**Option 3: Manual Middleware**

```javascript
// Express example
app.use(async (req, res, next) => {
  if (process.env.HAYSTACK_FIXTURES !== 'true') return next();

  const fixture = findMatchingFixture(req.path); // from .haystack.json
  if (fixture) {
    const data = await loadFixture(fixture.source);
    return res.json(data);
  }
  next();
});
```

### Choosing a Fixture Source

| Source | Best For | Tradeoffs |
|--------|----------|-----------|
| `file://` | Small, stable data (<50KB) | Committed to repo, versions with code |
| `https://` | Dynamic data from staging | Requires network access, may be flaky |
| `s3://` | Large datasets (>50KB) | Needs AWS credentials, not in repo |
| `r2://` | Large datasets on Cloudflare | Needs R2 credentials, not in repo |
| `script://` | Generated/computed data | Runs at startup, flexible but slower |

**Rule of thumb:**
- Auth mocks, config, small responses → `file://` (commit to repo)
- Large API responses, analysis data → `s3://` or `r2://`
- Data that changes per-run → `script://`

### Login-Required Data (Cookie-Based Auth)

For endpoints that require a logged-in session (not just an API token), use the `cookies` field:

**Step 1: Log into your staging site in a browser**

**Step 2: Export your session cookies**
```
# Chrome DevTools: Application > Cookies > [your-domain]
# Copy the cookie string, e.g.: "session=abc123; csrf=xyz789"
```

**Step 3: Store as a secret**
```bash
haystack secrets set STAGING_SESSION "session=abc123; csrf=xyz789"
```

**Step 4: Reference in `.haystack.json`**
```yaml
verification:
  fixtures:
    "/api/dashboard":
      source: "https://staging.example.com/api/dashboard"
      cookies: "$STAGING_SESSION"
```

**Note:** Session cookies typically expire after some time. You'll need to re-export them periodically.

For long-lived access, consider:
- Creating a dedicated test account with non-expiring sessions
- Using API tokens instead of cookies where possible
- Using `script://` fixtures that perform OAuth programmatically

## MCP Tools

### Setup & Configuration

| Tool | Description |
|------|-------------|
| `haystack_setup_start` | Start interactive setup wizard with auto-detection |
| `haystack_setup_answer` | Provide answer to setup question |
| `haystack_load_config` | Load `.haystack.json` configuration |
| `haystack_validate_config` | Validate config for errors and warnings |
| `haystack_detect_project` | Auto-detect framework, monorepo, services |
| `haystack_get_instructions` | Get context-aware next steps |

### Fixtures

| Tool | Description |
|------|-------------|
| `haystack_load_fixtures` | Load fixtures from all sources |
| `haystack_generate_fixture` | Generate skeleton fixture file |
| `haystack_record_fixture` | Record API response as fixture file |

### Verification

| Tool | Description |
|------|-------------|
| `haystack_get_verify_targets` | Get dev server config and verification commands |
| `haystack_get_auth_config` | Get auth bypass environment variables |
| `haystack_get_flow_templates` | Get pre-built verification command templates |

### Secrets

| Tool | Description |
|------|-------------|
| `haystack_secrets_info` | Get documentation about the secrets system |
| `haystack_secrets_list` | List configured secrets (keys only, never values) |
| `haystack_secrets_set` | Store a secret with zero-knowledge encryption |
| `haystack_secrets_delete` | Delete a stored secret |

### Auto-Detection

The `haystack_setup_start` and `haystack_detect_project` tools automatically detect:

- **Framework**: Vite, Next.js, Remix, Nuxt, SvelteKit, Astro, CRA, Cloudflare Workers
- **Package Manager**: npm, pnpm, yarn, bun (from lockfiles)
- **Monorepo**: pnpm workspaces, Lerna, Nx, Turborepo, Rush
- **Services**: Scans workspace packages for runnable services
- **Auth Bypass**: Checks `.env.example` for common auth skip patterns

This provides smart defaults in the setup wizard, reducing manual configuration.

### Verification Command Templates

Use `haystack_get_flow_templates` to get pre-built command sets:

| Template | Description |
|----------|-------------|
| `frontend` | React/Vue/Svelte apps (build, lint, typecheck) |
| `node_api` | Node.js APIs (build, lint, test) |
| `cloudflare_worker` | Cloudflare Workers (deploy --dry-run, typecheck) |
| `python` | Python projects (pytest, mypy, ruff) |
| `monorepo` | Monorepo with pnpm workspaces |
| `minimal` | Just build command |

## Environment Variables & Secrets

For fixtures from external sources or other sensitive values, use Haystack Secrets:

```bash
# Store a secret (zero-knowledge encrypted)
haystack secrets set STAGING_API_TOKEN "your-token"
haystack secrets set AWS_ACCESS_KEY_ID "AKIA..."
haystack secrets set AWS_SECRET_ACCESS_KEY "..."
```

Then reference in `.haystack.json`:

```yaml
fixtures:
  "/api/data":
    source: "https://staging.example.com/api/data"
    headers:
      Authorization: "Bearer $STAGING_API_TOKEN"

  "/api/reports":
    source: "s3://bucket/reports.json"
    # Uses $AWS_ACCESS_KEY_ID and $AWS_SECRET_ACCESS_KEY automatically
```

Common secrets:

| Variable | Purpose |
|----------|---------|
| `STAGING_API_TOKEN` | Token for staging API access |
| `AWS_ACCESS_KEY_ID` | AWS S3 credentials |
| `AWS_SECRET_ACCESS_KEY` | AWS S3 credentials |
| `R2_ACCESS_KEY_ID` | Cloudflare R2 credentials |
| `R2_SECRET_ACCESS_KEY` | Cloudflare R2 credentials |
| `CLOUDFLARE_ACCOUNT_ID` | For R2 bucket access |
| `DATABASE_URL` | Database connection string |

See `/setup-haystack-secrets` skill for full documentation.

## Schema Reference

Full `.haystack.json` schema:

```yaml
version: "1"                    # Required, must be "1"
name: my-app                    # Optional display name
package_manager: pnpm           # Optional: npm, pnpm, yarn, bun (auto-detected)

# Simple mode: single dev server
dev_server:
  command: pnpm dev             # Required: command to start server
  port: 3000                    # Required: port number
  ready_pattern: "Local:"       # Optional: stdout pattern when ready
  env:                          # Optional: environment variables
    SKIP_AUTH: "true"

# Monorepo mode: multiple services (mutually exclusive with dev_server)
services:
  frontend:
    root: ./                    # Optional: directory relative to repo root
    command: pnpm dev           # Required: command to start service
    port: 3000                  # Required for servers, omit for batch jobs
    ready_pattern: "Local:"     # Optional: stdout pattern when ready
    type: server                # Optional: 'server' (default) or 'batch'
    env:                        # Optional: environment variables
      SKIP_AUTH: "true"

# Verification configuration
verification:
  commands:                     # Optional: commands run during sandbox creation
    - name: build               # Human-readable name
      run: pnpm build           # Shell command to run

  fixtures:                     # Optional: API mock data sources
    "/api/users/*":             # URL pattern to intercept
      source: "file://..."      # Data source (file://, https://, s3://, r2://, script://, passthrough)
      headers:                  # Optional: headers for remote sources
        Authorization: "Bearer $TOKEN"
```