npm package: @lucid-dreams/agent-runtime-cli
Description: Command line interface for managing Lucid agent runtime deployments
Latest version: 0.2.24

# Agent Runtime CLI Blueprint

## Purpose

- Deliver a dedicated command-line interface (`lucid agents …`) that streamlines credential provisioning, rotation, and verification for agent operators.
- Provide a zero-config deployment experience (`lucid deploy`) that ships Bun agents onto Lucid-managed GCP infrastructure without exposing Cloud Run internals to users.
- Ship as `@lucid-dreams/agent-runtime-cli` under `packages/agent-runtime-cli`, depending on:
  - `@lucid-dreams/client` for user/session-authenticated admin APIs.
  - `@lucid-dreams/agent-runtime` for agent challenge/exchange helpers.
  - Shared utility packages (logging, prompts) already present in the monorepo.
- Maintain a clean separation from the runtime SDK to avoid dragging Node-only dependencies into browser/edge environments.

## Distribution & Packaging

- `package.json` `type: "module"`, exports `bin: { "lucid-agents": "dist/index.js" }`.
- Installable globally (`npm i -g @lucid-dreams/agent-runtime-cli`) or via `npx`.
- Built with `tsup` (matching other packages) targeting Node 18+.
- Includes telemetry/analytics opt-in flag (`--telemetry`) aligning with company standards.

## Command Overview

| Command                                | Summary                                                                                                                       | Key Flags                                                                          |
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| `lucid agents init`                    | Provision a new agent credential, run challenge/exchange, persist config.                                                     | `--agent <ref>`, `--wallet <id>`, `--scopes`, `--config-path`, `--non-interactive` |
| `lucid agents rotate`                  | Create a replacement credential, warm it, update config, optionally revoke old one.                                           | `--keep-old`, `--primary`, `--credential <id>`                                     |
| `lucid agents revoke`                  | Revoke an existing credential and clear any cached refresh handles.                                                           | `--credential <id>`, `--force`                                                     |
| `lucid agents whoami`                  | Inspect current configuration (agent id, credential id, scopes, refresh expiry).                                              | `--config-path`                                                                    |
| `lucid agents tokens list` _(stretch)_ | Debug refresh token states (active/jti/last-used) for an agent.                                                               | `--agent <ref>`                                                                    |
| `lucid deploy`                         | Zero-config deploy: packages current project, uploads to Lucid Deploy API, and tracks rollout to our managed GCP environment. | `--env KEY=VALUE` (optional overrides), `--watch`, `--dry-run`                     |
| `lucid agents deploy` _(alias)_        | Same as `lucid deploy` for backward compatibility.                                                                            | inherits                                                                           |
| `lucid login`                          | Interactive device-code login to obtain CLI session + project permissions.                                                    | `--env <environment>`, `--json`                                                    |

## Shared CLI Options

- `--api-base <url>` overrides `LUCID_API_BASE_URL`.
- `--config-path <path>` overrides default `.lucid-agent.json`.
- `--session-token <token>` supply user session explicitly (alternatively read from `LUCID_SESSION_TOKEN` env).
- `--json` toggles machine-readable output for CI scripts.
- `--yes` auto-confirms prompts in batch workflows.
- Deployment-only options (all optional): `--env KEY=VALUE` to append environment variables, `--watch` to stream build/deploy logs, `--dry-run` to validate without executing, `--no-update-config` to skip writing deployment metadata, `--local-build` to force local Docker build (advanced troubleshooting only).

## Managed Deploy Infrastructure (Lucid Cloud Run)

- Lucid operates dedicated GCP projects per environment (e.g., `lucid-prod-agents`, `lucid-sandbox-agents`) with Cloud Run, Artifact Registry, and Cloud Build enabled.
- Users never pass GCP coordinates; CLI retrieves deployment target settings from a new Auth API endpoint (`GET /v1/agents/:agentRef/deployment-target`).
- Deployments execute under a Lucid service account via short-lived impersonation tokens issued by the backend. No local `gcloud` auth or service-account files required.
- Artifact Registry repositories (`lucid-agents`) and Cloud Run services follow Lucid-managed naming conventions. CLI informs users of the resolved names post-deploy.
- Source archives are uploaded to a temporary GCS bucket managed by Lucid Deploy Service; signed URLs expire after upload completes.

### Authentication & Project Scoping

- `lucid login` launches a device-code flow against the Lucid Auth service; users confirm in the browser, granting CLI access scoped to their organization.
- CLI stores an encrypted refresh token (`~/.lucid/credentials.json`), rotated automatically. Access tokens carry user id, account id, and allowed agent refs.
- Storage uses `keytar` to persist refresh tokens securely when available; falls back to encrypted JSON (AES-256-GCM) with key derived from OS keystore.
- Every CLI command begins by exchanging the stored refresh token for a short-lived session token via `/v1/cli/session`.
- Backend maps session → account → deployment project. When a user creates an agent, `/v1/agents/:agentRef/deployment-target` responds with the correct Cloud Run environment derived from the owning account.
- Project scoping rules enforce that a user can only deploy agents belonging to accounts they administer; backend validates before issuing deployment tickets.

### Container Build Strategy

- CLI packages project source (respecting `.dockerignore` if present) into a tarball.
- Uploads tarball to signed URL provided by backend, alongside deployment manifest (env overrides, build args).
- Lucid Deploy Service triggers Cloud Build job using unified Bun Dockerfile template, outputs image to managed Artifact Registry, and records image digest.
- No local Docker dependency; fallback to local build optional behind `--local-build` (primarily for advanced operators or air-gapped debugging).

### Cloud Run Deployment Defaults

- Region, CPU/memory, concurrency, autoscaling bounds come from deployment target policy (per environment + agent tier).
- Services named `<agentRef>-runtime` unless otherwise configured centrally.
- Ingress/IAM handled server-side. If public access is allowed, backend binds `allUsers` automatically; otherwise returns instructions for authenticated invocation.
- Runtime identity set to managed service account (`agent-runtime@<project>.iam.gserviceaccount.com`) with least privileges; CLI surfaces its email in output.

### Backend Integration

- Deploy orchestration handled by new service endpoints:
  - `GET /v1/agents/:agentRef/deployment-target` → returns environment policy, Cloud Run service name, allowed env overrides, and signed upload URL.
  - `POST /v1/agents/:agentRef/deployments` → kicks off build/deploy pipeline; responds with deployment ID.
  - `GET /v1/agents/:agentRef/deployments/:id` → returns status (`queued`, `building`, `deploying`, `succeeded`, `failed`), progress logs, and output metadata.
- Backend authenticates CLI using user CLI session tokens (issued via `lucid login`), validates account permissions, and when agent actions require it, exchanges for agent credential JWTs server-side.
- CLI gracefully handles 429/503 responses by retrying with exponential backoff and surfaces queue position when available.

### Technical Architecture

1. **CLI Layer**
   - Node-based CLI built with `commander` + `prompts`.
   - `AuthManager` handles device-code login, token refresh, secure storage (leveraging OS keychain when available).
   - `DeployCommand` packages source (using `tar-stream`), uploads to signed URL via streaming `fetch`, and manages long-polling or SSE for deployment status.
   - Shared `AgentConfig` module reads/writes `.lucid-agent.json`, merging deployment metadata.
2. **Deploy API Service (backend)**
   - New service in `apps/server` under `/v1/agents/:agentRef/deployments`.
   - Persists deployment records in `agent_deployments` table (status, image digest, revision, logs pointer).
   - Generates signed upload URLs via `@google-cloud/storage` client, scoped to temporary bucket.
   - Submits Cloud Build jobs using `v1.CloudBuildClient` with templated build config (Bun multi-stage Dockerfile).
   - Tracks job status via Pub/Sub or Cloud Build webhooks → updates DB → notifies CLI through polling.
   - Once build succeeds, invokes Cloud Run Admin API to deploy new revision, applying environment policy from `agent_deployment_targets` table.
   - Emits logs to Cloud Logging; exposes summarized log stream to CLI via paginated `/logs` endpoint or SSE feed.
3. **Infrastructure Provisioning**
   - Terraform module per environment defines: Artifact Registry (`lucid-agents`), Cloud Run services (created lazily), Cloud Build service account, impersonation roles.
   - Service account `agent-deploy-runner@<project>` granted Cloud Build Editor + Cloud Run Admin; Deploy API impersonates it using `IAMCredentials.generateAccessToken`.
   - Optional Pub/Sub topic `agent-deploy-status` to push real-time status updates; CLI can subscribe via backend long-poll.
4. **Security Controls**
   - Deployment tickets contain HMAC-signed payload tying tarball upload to agent + user; expires after 10 minutes.
   - Backend validates tarball SHA256 hash before triggering Cloud Build to prevent tampering.
   - Environment variables injected server-side include secrets sourced from Lucid secret manager; CLI-provided `--env` values are namespaced and validated against allowlist.

## Configuration File Schema (`.lucid-agent.json`)

```json5
{
  agentId: "echo-agent",
  credentialId: "cred_123",
  refreshToken: "refresh_handle_secret",
  scopes: ["wallet.send", "wallet.sign"],
  apiBaseUrl: "https://api.lucid.example",
  lastRotatedAt: "2024-06-05T20:15:43.000Z",
  deployment: {
    environment: "production",
    service: "echo-agent-runtime",
    image: "us-central1-docker.pkg.dev/lucid-prod-agents/lucid-agents/echo-agent-runtime:deploy-20240605-2015",
    revision: "echo-agent-runtime-00042-abc",
    url: "https://echo-agent-runtime-abcdef-uc.a.run.app",
    deployedAt: "2024-06-05T20:16:12.000Z",
  },
}
```

- Stored alongside project source by default; `--config-path` allows custom locations.
- Refresh handles encrypted or masked depending on OS keychain availability (see storage strategy).
- `deployment` section optional; populated by `deploy` command for subsequent rotations/rollbacks.

## Storage & Security

- Default persistence via `FileStorageAdapter` (same implementation as runtime) with file-locking.
- Optional integration with system keychains (macOS Keychain, Windows Credential Manager, Linux Secret Service) for refresh handle storage—resolved via `--storage keychain`.
- CLI logs redact secrets; `--debug` prints HTTP traces with headers scrubbed.
- Supports environment-only (stateless) mode by passing `--no-write` to skip writing config (returns JSON payload for callers to manage).
- GCP interaction uses short-lived signed tokens returned by Lucid Deploy Service; tokens scoped to specific build/deploy job and never persisted.
- For compliance, CLI stores deployment audit trail (`deployment.history`) locally when `--json` flag is used, but omits credential material entirely.

## Workflow Details

### `lucid login`

1. CLI requests device code from `/v1/cli/auth/device`.
2. User visits verification URL, enters code, grants access; backend issues temporary poll token.
3. CLI polls `/v1/cli/auth/device/:token` until authorized, then receives refresh token + allowed account metadata.
4. Refresh token stored encrypted in `~/.lucid/credentials.json`; access token cached in memory per command run.
5. Login response includes default environment (`sandbox`/`production`) and accessible agent refs to optimize later lookups.

### `lucid agents init`

1. Resolve session token via `AuthManager` (uses stored refresh token or triggers `lucid login` device flow if absent/expired).
2. Call `GET /v1/agents` to list available agents; allow creation via `POST /v1/agents` if user chooses.
3. Call `GET /v1/accounts/:accountId/wallets` to show managed wallets.
4. POST to `/v1/agents/:agentRef/credentials` to create credential with chosen scopes; mark as primary if requested.
5. Execute challenge/exchange using `AgentRuntime` helpers to mint first refresh handle.
6. Persist config (encrypt refresh token if keychain mode).
7. Run smoke test via `GET /v1/agents/:agentRef` using the new access token; report success/failure.

### `lucid agents rotate`

1. Read existing config to determine agent + current credential.
2. Offer to create new credential and set it primary.
3. Warm credential (challenge/exchange) before switching.
4. Update config with new credential + refresh handle.
5. Optionally revoke previous credential and purge its refresh tokens.

### `lucid agents revoke`

1. Validate credential ownership.
2. Call `DELETE /v1/agents/:agentRef/credentials/:credentialId`.
3. Clear local refresh token cache; emit warning if active runtime tokens remain (advise redeploy).

### `lucid agents whoami`

- Read config + resolve current access token status (exp, scopes, revocation nonce).
- Validate by calling `GET /v1/agents/:agentRef/credentials/:credentialId`.
- Output human-readable summary or JSON depending on flag.

### `lucid agents deploy`

1. Load `.lucid-agent.json` (or prompt for agent + credential) to verify runtime auth info.
2. Request deployment target from backend (`GET /v1/agents/:agentRef/deployment-target`) which returns environment (`production`, `staging`), Cloud Run service name, and capability flags.
3. Package project folder into tarball (respect `.dockerignore`/`lucid.deployignore`); optionally emit default Bun Dockerfile scaffold if none present.
4. Upload tarball + metadata to signed URL provided by backend; receive `deploymentTicket` ID.
5. Trigger deployment via `POST /v1/agents/:agentRef/deployments` with ticket, environment variables, and optional `dryRun` flag.
6. Poll `GET /v1/agents/:agentRef/deployments/:deploymentId` (with `--watch` streaming logs) until Cloud Build + Cloud Run rollout complete.
7. On success, backend returns image digest, Cloud Run service URL, revision ID, and verification results. CLI runs final health check through authenticated request.
8. Update local config with deployment metadata (environment, image, revision, URL, deployedAt) unless `--no-update-config` is set. Print summary and tail command hints.

## Error Handling

- Domain-specific error codes mapped to friendly messages:
  - `credential_revoked` → instruct to run `rotate` or `init`.
  - `wallet_ownership_failed` → prompt to re-select wallet or verify orchestrator logs.
  - `session_expired` → re-authenticate using `--session-token` or login flow.
- Non-zero exit codes for failures; `--json` output includes `error.code`, `error.message`, `details`.
- Retries with exponential backoff for 5xx network errors (max 3 attempts).

## Observability

- Structured logs via `pino` (pretty by default, JSON with `--json`).
- Optional OpenTelemetry exporter: `--otel-endpoint <url>` to ship spans (`cli.init`, `cli.rotate`) with durations + HTTP status attributes.
- CLI prints concise success banner and location of written config; hides refresh tokens by default.
- `deploy` surfaces Cloud Run URLs, revision IDs, and live log stream sourced from backend (which proxies Cloud Build/Run logs) when `--watch` or `--tail` is passed.

## Testing Strategy

- Unit tests for command handlers using mocked REST + runtime helpers (set through dependency injection).
- Integration tests with local API stub verifying full flow (credential creation, challenge, config write).
- Snapshot tests for help output and failure messaging.
- Deployment tests:
  - Mocked Docker/Cloud Build interfaces to ensure correct commands + env vars emitted.
  - Contract tests against stubbed Deploy API endpoints covering queueing, failures, and log streaming.
  - End-to-end regression in CI invoking published CLI against Lucid staging deploy environment (nightly) to validate cloud integration.

## Rollout Plan

1. Scaffold package in `packages/agent-runtime-cli`, share `tsconfig` + `tsup` config with other CLIs.
2. Implement `init` command first, ensuring compatibility with new runtime config format.
3. Add `rotate`, `revoke`, and `whoami`, then publish `0.1.0-alpha`.
4. Stand up Deploy API backend (target-resolution, signed uploads, build orchestration) and integrate CLI `lucid deploy` command end-to-end.
5. Deliver internal alpha of `lucid deploy` with watch/log streaming; capture operator feedback before enabling for all users.
6. Integrate CLI into `@lucid-dreams/create-agent-kit` templates (optional flag) once stable; include deploy scaffolding for Bun agents.
7. Monitor adoption via telemetry; iterate on UX before `1.0.0`.

## Implementation Milestones

1. **Auth groundwork**
   - Add `/v1/cli/auth/device`, `/v1/cli/auth/device/:token`, `/v1/cli/session` endpoints in `apps/server`.
   - Create `cli_sessions`, `cli_refresh_tokens` tables (hashed tokens, expiration, project scope).
   - Build `AuthManager` in CLI using device-code flow + secure storage.
2. **Deployment metadata**
   - Add `agent_deployment_targets` table seeded per environment (project, region, cpu/mem, ingress policy).
   - Add `agent_deployments` table to track job lifecycle, logs pointer, revision metadata.
   - Implement `GET /v1/agents/:agentRef/deployment-target`.
3. **Deploy service orchestration**
   - Implement signed upload URL generator (GCS) + `DeploymentTicket` issuance (HMAC).
   - Build Cloud Build config template (YAML) for Bun container with arguments for repo digest, entrypoint.
   - Implement Cloud Build job submitter + status subscriber (via Pub/Sub or Build API polling).
   - Integrate Cloud Run deploy client applying environment defaults + env var allowlist.
4. **CLI deploy command**
   - Implement tarball packaging (`tar-stream`), ignore rules, streaming upload with progress.
   - Add deployment status polling + log streaming (SSE or incremental fetch).
   - Update `.lucid-agent.json` persistence with deployment metadata & history.
5. **Observability & tooling**
   - Emit structured events (`deploy.started`, `deploy.succeeded`, `deploy.failed`) to Amplitude/DataDog.
   - Provide `lucid deploy --dry-run` to validate target + build config without executing pipeline.
   - Add docs + examples to `agent-runtime-cli` README and update onboarding guides.