Skill: jbushman/pgmemory (version 1.2.0)

--- name: pgmemory version: 1.2.0 description: Persistent semantic memory for OpenClaw agents — PostgreSQL + pgvector author: jbushman tags: [memory, postgresql, pgvector, embeddings, agents] --- # pgmemory Gives OpenClaw agents persistent semantic memory backed by PostgreSQL + pgvector. Agents wake up fresh every session. pgmemory fixes that — decisions, constraints, infrastructure facts, and discoveries persist across sessions and surface automatically when relevant. ## Setup Run once after installing: ```bash python3 ~/.openclaw/skills/pgmemory/scripts/setup.py ``` The wizard handles everything: Docker/PostgreSQL, migrations, embedding provider, AGENTS.md scaffolding, and decay cron. ## When to use this skill Read this SKILL.md when: - Setting up pgmemory for the first time - Adding a new OpenClaw agent and want it to have persistent memory - Diagnosing memory issues (run `--doctor`) - Switching embedding providers - Understanding how memory decay or archiving works ## Core commands ### Write a memory ```bash python3 ~/.openclaw/skills/pgmemory/scripts/write_memory.py \ --key "unique.descriptive.key" \ --content "What to remember" \ --category decision \ --importance 3 ``` **Categories:** `decision` · `constraint` · `infrastructure` · `vision` · `preference` · `context` · `task` **Importance:** - `3` = critical — decisions, constraints, infrastructure. Never expires. Always loaded. - `2` = important — context, preferences. Expires after 180 days if unused. - `1` = transient — low-value notes. Expires after 30 days. ### Search memories ```bash # Semantic search python3 ~/.openclaw/skills/pgmemory/scripts/query_memory.py "database connection" # Load all critical memories (importance 3) python3 ~/.openclaw/skills/pgmemory/scripts/query_memory.py --importance 3 --limit 20 # Stats python3 ~/.openclaw/skills/pgmemory/scripts/query_memory.py --stats # List all keys python3 ~/.openclaw/skills/pgmemory/scripts/query_memory.py --list ``` ### Maintenance ```bash # Full health check python3 ~/.openclaw/skills/pgmemory/scripts/setup.py --doctor # Validate config python3 ~/.openclaw/skills/pgmemory/scripts/setup.py --validate # Run pending migrations python3 ~/.openclaw/skills/pgmemory/scripts/setup.py --migrate # Sync pgmemory into all OpenClaw agent workspaces python3 ~/.openclaw/skills/pgmemory/scripts/setup.py --sync-agents # Run decay cycle manually python3 ~/.openclaw/skills/pgmemory/scripts/setup.py --decay ``` ## What to write to memory Write immediately when: - You make a **decision** about architecture, tooling, or approach - You discover a **constraint** — something that will bite you if forgotten - You complete **infrastructure** work (migrations, deployments, config changes) - You identify a **preference** or **vision** that should guide future work - A sub-agent completes — harvest its important findings Skip writing for: - Casual conversation - Things already in MEMORY.md or other workspace files - Anything you'd classify as importance 1 unless it's genuinely useful ## Multi-agent setup Each OpenClaw agent gets its own namespace (= agent ID). Run `--sync-agents` after adding a new agent to scaffold pgmemory automatically: ```bash openclaw agents add code-writer python3 ~/.openclaw/skills/pgmemory/scripts/setup.py --sync-agents ``` Or add `--sync-agents` to HEARTBEAT.md for automatic pickup within 30 minutes. ### Harvest from sub-agents After a sub-agent completes, pull its important findings into the primary namespace: ```bash python3 ~/.openclaw/skills/pgmemory/scripts/query_memory.py \ --harvest shopwalk:subagent:task-label ``` ## Memory decay Memories decay based on age and category. Frequently accessed memories stay fresh. Decayed memories move to archive (never deleted). Restored automatically if matched in a future search. Decay runs daily via cron (configured during setup). Run manually anytime: ```bash python3 ~/.openclaw/skills/pgmemory/scripts/setup.py --decay ``` ## Switching embedding providers Changing providers after setup requires re-embedding all memories — you cannot mix dimensions in the same database. Run `--doctor` first to check for mismatches. ⚠️ Provider migration (`--re-embed`) is planned for v1.1. For now: set up a fresh database if you need to switch providers. ## Config reference Minimal (all that's required): ```json { "db": { "uri": "postgresql://openclaw@localhost:5432/openclaw" }, "embeddings": { "provider": "voyage", "api_key_env": "VOYAGE_API_KEY" }, "agent": { "name": "main" } } ``` Default config is at `~/.openclaw/pgmemory.json`. Override with `--config <path>`. Full config reference: see `references/schema.sql` and `CHANGELOG.md`. ## Requirements - Python 3.9+ - PostgreSQL 14+ with pgvector 0.5+ - `psycopg2-binary`, `numpy` — install via `pip install -r requirements.txt` - Embedding provider API key (or Ollama for local)

path: CHANGELOG.md
content: # Changelog

## v1.2.0 — 2026-03-03

### Added

- **`api_key` field in config** — store the embedding API key directly in `pgmemory.json` so scripts work with zero env var setup. Setup wizard prompts to paste the key; it's stored as `embeddings.api_key`. Resolves in order: config `api_key` → env var → error.
- **Setup wizard prompts for API key** — if key isn't in env or config, wizard asks "Paste your API key or leave blank". Key is stored in config for all future queries automatically.
- **No more `VOYAGE_API_KEY=...` prefixes** — startup commands in generated AGENTS.md blocks no longer require env var prefixes; key comes from config transparently.

### Changed

- All scripts (`query_memory.py`, `write_memory.py`, `setup.py`) resolve API key from config first, env var second

## v1.1.0 — 2026-03-03

### Changed

- **pgmemory replaces markdown file reads at startup** — loading MEMORY.md + daily files dumps thousands of lines into context; pgmemory returns the 20 most relevant memories via semantic search, far faster and more focused. AGENTS.md startup now says "query pgmemory instead of reading markdown files"
- **AGENTS.md integration is now startup-mandatory** — setup and `--sync-agents` inject pgmemory startup steps directly into the "Every Session" numbered list in AGENTS.md, so agents run memory queries before every task (not just when they remember to)
- **Smarter injection** — `inject_startup_into_agents_md()` finds the last numbered startup list in AGENTS.md and appends pgmemory as the next step in-place, rather than only appending to end of file
- **Section marker** — pgmemory blocks in AGENTS.md are tagged with `<!-- pgmemory:section -->` and `<!-- pgmemory:startup -->` so re-runs are idempotent and the section is easy to find/update
- **Clearer instructions** — startup block now explicitly states "This is not optional" and explains why (decisions, constraints, infrastructure facts)
- **`scaffold_agents_md` generates reference + startup separately** — `startup_steps()` for injection, `scaffold_agents_md()` for the full reference section

### Philosophy

- **pgmemory is the default** — used by default for all reads and writes
- **Markdown is the automatic backup** — always also write a brief note to `memory/YYYY-MM-DD.md`; this ensures memories survive if the DB is unavailable. Full detail in pgmemory, brief backup in markdown.

### Added

- **`--check` flag** on `query_memory.py` — fast connectivity check (exit 0=ok, 2=unreachable). Prints a clear `⚠️ pgmemory UNREACHABLE` warning with host, error, and fallback instructions. 5-second connect timeout prevents hanging on unreachable hosts.
- **Startup step now runs `--check` first** — if it fails with exit 2, agent warns the user and falls back to markdown files rather than silently continuing

### Fixed

- DB connection now uses `connect_timeout=5` to fail fast instead of hanging on unreachable hosts
- Install no longer checks `if "pgmemory" in content` (too broad — matches comments); now checks for `<!-- pgmemory:section -->` marker

## v1.0.0 — 2026-03-02

Initial release.

### Features

- **Persistent semantic memory** — PostgreSQL + pgvector backend; memories survive session compaction
- **Three scripts** — `setup.py`, `write_memory.py`, `query_memory.py`
- **Wizard** — interactive setup with Docker detection/install, DB provisioning, embedding provider config, AGENTS.md scaffolding, cron setup
- **Migrations** — versioned SQL files with checksum verification, rollback support, idempotent on existing schemas
- **Decay + reinforcement** — relevance scores decay by category/importance, boosted by access frequency; archive not delete
- **Doctor** — full system health check including dimension mismatch detection, index health, cap warnings
- **Validate** — config pre-flight, no DB connection needed
- **`--sync-agents`** — auto-scaffold pgmemory into all OpenClaw agent workspaces from `openclaw.json`
- **Harvest** — pull important findings from sub-agent namespaces into primary namespace
- **Embedding providers** — Voyage AI (default), OpenAI, Ollama (local)
- **Archive table** — expired/evicted/decayed memories move to `archived_memories`, never hard deleted; `restore_on_access` brings them back automatically
- **Sane defaults** — minimal config is 3 fields; all options configurable


path: README.md
content: # openclaw-pgmemory

Persistent semantic memory for [OpenClaw](https://github.com/openclaw/openclaw) agents — PostgreSQL + pgvector.

OpenClaw agents wake up fresh every session. pgmemory fixes that. Decisions, constraints, infrastructure facts, and discoveries persist across sessions and surface automatically when relevant to the current task.

```
16 result(s) for "database connection" — agent: main

[infrastructure/★★★]  rel=0.98  infra.db.ovh  (2026-02-14)
    OVH PostgreSQL at 10.10.0.1:5432 — swapi user, pgbouncer on port 5432
    similarity: 0.921

[constraint/★★★]  rel=0.94  infra.db.partial-index  (2026-02-28)
    Merchant upsert: SELECT-then-INSERT pattern required. Cannot use ON CONFLICT
    due to partial unique indexes on merchants table.
    similarity: 0.887
```

---

## Install

```bash
clawhub install pgmemory
```

Then run the setup wizard:

```bash
python3 ~/.openclaw/skills/pgmemory/scripts/setup.py
```

The wizard handles everything — Docker or existing PostgreSQL, migrations, embedding provider config, `AGENTS.md` scaffolding, and decay cron.

## Requirements

- Python 3.9+
- PostgreSQL 14+ with [pgvector](https://github.com/pgvector/pgvector) 0.5+
- An embedding provider: [Voyage AI](https://www.voyageai.com) (default), [OpenAI](https://platform.openai.com), or [Ollama](https://ollama.ai) (local, no API key)

```bash
pip install -r requirements.txt
```

## Quick start

```bash
# 1. Install and setup
clawhub install pgmemory
python3 ~/.openclaw/skills/pgmemory/scripts/setup.py

# 2. Write a memory
python3 ~/.openclaw/skills/pgmemory/scripts/write_memory.py \
  --key "infra.db.prod" \
  --content "Production DB at 10.10.0.1:5432, pgbouncer on same host" \
  --category infrastructure \
  --importance 3

# 3. Search memories
python3 ~/.openclaw/skills/pgmemory/scripts/query_memory.py "database connection"

# 4. Check health
python3 ~/.openclaw/skills/pgmemory/scripts/setup.py --doctor
```

## Minimal config

Three fields. Everything else uses sane defaults.

```json
{
  "db":         { "uri": "postgresql://openclaw@localhost:5432/openclaw" },
  "embeddings": { "provider": "voyage", "api_key_env": "VOYAGE_API_KEY" },
  "agent":      { "name": "main" }
}
```

Config lives at `~/.openclaw/pgmemory.json`. Override with `--config <path>`.

## Usage

### Write

```bash
python3 scripts/write_memory.py \
  --key "unique.descriptive.key" \
  --content "What to remember" \
  --category decision \
  --importance 3
```

**Categories:** `decision` · `constraint` · `infrastructure` · `vision` · `preference` · `context` · `task`

**Importance:**
| Level | Meaning | Default TTL |
|---|---|---|
| `3` | Critical — decisions, constraints, infrastructure | Never expires |
| `2` | Important — context, preferences | 180 days |
| `1` | Transient — low-value notes | 30 days |

Categories `decision`, `constraint`, `infrastructure`, `vision`, `preference` never expire regardless of importance.

### Search

```bash
# Semantic search
python3 scripts/query_memory.py "kubernetes deployment"

# Load all critical memories
python3 scripts/query_memory.py --importance 3 --limit 20

# Filter by category
python3 scripts/query_memory.py --category decision

# Include archived memories in search
python3 scripts/query_memory.py --include-archived "old approach"

# Restore an archived memory
python3 scripts/query_memory.py --restore "infra.old-key"

# Stats
python3 scripts/query_memory.py --stats

# JSON output
python3 scripts/query_memory.py --json "search query"
```

### Maintenance

```bash
python3 scripts/setup.py --doctor       # full health check
python3 scripts/setup.py --validate     # validate config only
python3 scripts/setup.py --migrate      # run pending migrations
python3 scripts/setup.py --rollback 1   # rollback to schema version 1
python3 scripts/setup.py --decay        # recalculate relevance scores
python3 scripts/setup.py --sync-agents  # scaffold all OpenClaw agents
```

## Memory decay

Memories don't just expire — they decay based on age and how often they're accessed. Frequently referenced memories stay fresh. Ignored ones fade. Nothing is ever hard deleted.

| Category | Half-life | Notes |
|---|---|---|
| `decision`, `constraint` | ~700 days | Effectively permanent if accessed |
| `infrastructure`, `vision`, `preference` | ~230 days | |
| `context` | ~14 days | |
| `task` | ~7 days | |

Importance multiplies the rate — a critical (`3`) memory decays much slower than a transient (`1`). Faded memories move to `archived_memories`. If a future search matches an archived memory, it's automatically restored.

Decay runs daily via cron (configured during setup). Run manually: `setup.py --decay`

## Multi-agent support

Each OpenClaw agent gets its own memory namespace (namespace = agent ID). After adding a new agent, run `--sync-agents` to scaffold pgmemory automatically:

```bash
openclaw agents add code-writer
python3 scripts/setup.py --sync-agents
```

Or add to `HEARTBEAT.md` for automatic pickup within 30 minutes.

### Sub-agent harvest

After a sub-agent completes, pull its important findings into the primary namespace:

```bash
python3 scripts/query_memory.py --harvest shopwalk:subagent:task-name
```

Harvest deduplicates against existing memories (95% similarity threshold) and attributes the source.

## Embedding providers

| Provider | Model | Dimensions | Notes |
|---|---|---|---|
| Voyage AI | voyage-3 | 1024 | Default — best retrieval quality |
| OpenAI | text-embedding-3-small | 1536 | Common fallback |
| Ollama | nomic-embed-text | 768 | Local, no API key required |

⚠️ Do not mix providers in the same database. Changing providers requires re-embedding all memories. Run `--doctor` to detect dimension mismatches before they cause problems.

## Safe migrations

Schema changes use versioned SQL files with checksum verification. Migrations never auto-run on startup — always explicit.

```bash
python3 scripts/setup.py --migrate      # apply pending
python3 scripts/setup.py --rollback 1   # roll back to version 1
```

Each migration file has both `UP` and `DOWN` sections. The `pgmemory_migrations` table tracks what's been applied and when.

## License

MIT — see [LICENSE](LICENSE)

---

Made by [@jbushman](https://github.com/jbushman) · [shopwalk.com](https://shopwalk.com)


path: _meta.json
content: ownerId: kn75ja6xgq6bh488fewf341rq58240m4
slug: pgmemory
version: 1.2.0
publishedAt: 1772487689616

path: assets/docker-compose.yml
content: # pgmemory — PostgreSQL + pgvector
# Recommended: copy this file, then run: docker compose up -d
#
# Connection URI: postgresql://openclaw:pgmemory@localhost:15432/openclaw

services:
  pgmemory:
    image: pgvector/pgvector:pg17
    container_name: pgmemory
    restart: unless-stopped
    environment:
      POSTGRES_USER: openclaw
      POSTGRES_PASSWORD: pgmemory
      POSTGRES_DB: openclaw
    ports:
      - "15432:5432"
    volumes:
      - pgmemory_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U openclaw"]
      interval: 5s
      timeout: 5s
      retries: 10

volumes:
  pgmemory_data:


path: references/schema.sql
content: -- pgmemory reference schema (documentation copy — authoritative source is migrations/)
-- This file documents the full schema as of the latest migration.

-- memories: core table
-- agent           TEXT        — OpenClaw agent ID (namespace)
-- namespace       TEXT        — sub-namespace within agent (default: 'default')
-- category        TEXT        — decision|constraint|infrastructure|vision|preference|context|task
-- key             TEXT        — unique identifier within agent (agent+key is unique)
-- content         TEXT        — the memory text
-- embedding       vector(1024)— semantic embedding (dimension from config)
-- importance      INT 1-3     — 1=transient, 2=important, 3=critical
-- relevance_score FLOAT 0-1   — decays over time, boosted by access_count
-- access_count    INT         — increments each time memory is returned in search
-- source          TEXT        — session|harvest:{namespace}|import|manual
-- tags            TEXT[]      — optional free-form tags
-- expires_at      TIMESTAMPTZ — NULL = never expires
-- last_accessed   TIMESTAMPTZ — last time returned in a search result
-- decay_rate_override FLOAT   — per-memory decay rate (NULL = use category default)

-- archived_memories: same columns + archived_at + archive_reason
-- Memories are never hard deleted — they move here on expiry/eviction.

-- session_state: one row per agent
-- current_task, summary, active_jobs[], blocked_on[], metadata JSONB

-- pgmemory_migrations: tracks applied migrations
-- version INT PK, filename TEXT, checksum TEXT, applied_at TIMESTAMPTZ

-- archive_reason values: expired | evicted | decayed | manual
-- archived_ttl_days: null = keep forever (default)
-- restore_on_access: if semantic search matches archived memory, auto-restore with relevance_score=1.0

-- Archive triggers (all default true, configurable independently):
-- on_expire  — expires_at passed
-- on_evict   — max_memories cap exceeded, lowest relevance_score evicted
-- on_decay   — relevance_score dropped below archive_threshold (default 0.1)
-- on_manual  — explicit --archive call or admin action


path: requirements.txt
content: psycopg2-binary>=2.9


path: scripts/migrations/0001_initial_schema.sql
content: -- pgmemory migration 0001
-- Initial schema — safe for both fresh installs and existing tables

-- UP

CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_trgm;

-- Create memories table (fresh install)
CREATE TABLE IF NOT EXISTS memories (
  id              BIGSERIAL   PRIMARY KEY,
  agent           TEXT        NOT NULL DEFAULT 'default',
  namespace       TEXT        NOT NULL DEFAULT 'default',
  category        TEXT        NOT NULL DEFAULT 'context',
  key             TEXT,
  content         TEXT        NOT NULL,
  embedding       vector(1024),
  importance      INT         NOT NULL DEFAULT 2 CHECK (importance BETWEEN 1 AND 3),
  relevance_score FLOAT       NOT NULL DEFAULT 1.0,
  access_count    INT         NOT NULL DEFAULT 0,
  source          TEXT        DEFAULT 'session',
  tags            TEXT[]      DEFAULT '{}',
  metadata        JSONB       DEFAULT '{}',
  expires_at      TIMESTAMPTZ,
  last_accessed   TIMESTAMPTZ,
  created_at      TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at      TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Add any columns that may be missing on existing tables
ALTER TABLE memories ADD COLUMN IF NOT EXISTS namespace       TEXT        NOT NULL DEFAULT 'default';
ALTER TABLE memories ADD COLUMN IF NOT EXISTS relevance_score FLOAT       NOT NULL DEFAULT 1.0;
ALTER TABLE memories ADD COLUMN IF NOT EXISTS access_count    INT         NOT NULL DEFAULT 0;
ALTER TABLE memories ADD COLUMN IF NOT EXISTS tags            TEXT[]      DEFAULT '{}';
ALTER TABLE memories ADD COLUMN IF NOT EXISTS expires_at      TIMESTAMPTZ;
ALTER TABLE memories ADD COLUMN IF NOT EXISTS last_accessed   TIMESTAMPTZ;

-- Ensure unique index on (agent, key) — may already exist under a different name
DO $$ BEGIN
  IF NOT EXISTS (
    SELECT 1 FROM pg_indexes WHERE tablename='memories' AND indexname='idx_memories_agent_key'
  ) THEN
    CREATE UNIQUE INDEX idx_memories_agent_key ON memories (agent, key) WHERE key IS NOT NULL;
  END IF;
END $$;

-- Vector search index
DO $$ BEGIN
  IF NOT EXISTS (
    SELECT 1 FROM pg_indexes WHERE tablename='memories' AND indexname='idx_memories_embedding'
  ) THEN
    CREATE INDEX idx_memories_embedding ON memories USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);
  END IF;
END $$;

CREATE INDEX IF NOT EXISTS idx_memories_agent_importance ON memories (agent, importance DESC);
CREATE INDEX IF NOT EXISTS idx_memories_category         ON memories (agent, category);
CREATE INDEX IF NOT EXISTS idx_memories_expires_at       ON memories (expires_at) WHERE expires_at IS NOT NULL;

-- Archived memories table
CREATE TABLE IF NOT EXISTS archived_memories (
  id              BIGSERIAL   PRIMARY KEY,
  agent           TEXT        NOT NULL DEFAULT 'default',
  namespace       TEXT        NOT NULL DEFAULT 'default',
  category        TEXT        NOT NULL DEFAULT 'context',
  key             TEXT,
  content         TEXT        NOT NULL,
  embedding       vector(1024),
  importance      INT         NOT NULL DEFAULT 2,
  relevance_score FLOAT       NOT NULL DEFAULT 1.0,
  access_count    INT         NOT NULL DEFAULT 0,
  source          TEXT        DEFAULT 'session',
  tags            TEXT[]      DEFAULT '{}',
  metadata        JSONB       DEFAULT '{}',
  expires_at      TIMESTAMPTZ,
  last_accessed   TIMESTAMPTZ,
  created_at      TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at      TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  archived_at     TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  archive_reason  TEXT        NOT NULL DEFAULT 'expired'
);

CREATE INDEX IF NOT EXISTS idx_archived_memories_agent ON archived_memories (agent, archived_at DESC);

-- Session state table
CREATE TABLE IF NOT EXISTS session_state (
  agent        TEXT        PRIMARY KEY,
  current_task TEXT,
  summary      TEXT,
  active_jobs  TEXT[]      DEFAULT '{}',
  blocked_on   TEXT[]      DEFAULT '{}',
  metadata     JSONB       DEFAULT '{}',
  last_updated TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  created_at   TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Updated_at trigger (create or replace)
CREATE OR REPLACE FUNCTION update_updated_at()
RETURNS TRIGGER AS $$
BEGIN NEW.updated_at = NOW(); RETURN NEW; END;
$$ LANGUAGE plpgsql;

DROP TRIGGER IF EXISTS memories_updated_at ON memories;
CREATE TRIGGER memories_updated_at
  BEFORE UPDATE ON memories
  FOR EACH ROW EXECUTE FUNCTION update_updated_at();

-- DOWN

DROP TRIGGER IF EXISTS memories_updated_at ON memories;
DROP FUNCTION IF EXISTS update_updated_at();
DROP TABLE IF EXISTS session_state;
DROP TABLE IF EXISTS archived_memories;
DROP TABLE IF EXISTS memories;


path: scripts/migrations/0002_add_decay_columns.sql
content: -- pgmemory migration 0002
-- Decay columns were included in 0001 (relevance_score, access_count, last_accessed)
-- This migration adds the decay_rate column for per-memory override

-- UP

ALTER TABLE memories
  ADD COLUMN decay_rate_override FLOAT DEFAULT NULL;

COMMENT ON COLUMN memories.decay_rate_override IS
  'Per-memory decay rate override. NULL means use category default from config.';

-- DOWN

ALTER TABLE memories DROP COLUMN IF EXISTS decay_rate_override;


path: scripts/query_memory.py
content: #!/usr/bin/env python3
"""
pgmemory query_memory.py — search and inspect memories

Usage:
  python3 query_memory.py "database connection"          # semantic search
  python3 query_memory.py --importance 3 --limit 20      # load critical memories
  python3 query_memory.py --category decision             # filter by category
  python3 query_memory.py --stats                         # show counts and health
  python3 query_memory.py --list                          # list all memory keys
  python3 query_memory.py --include-archived "search"    # search archive too
  python3 query_memory.py --restore "key.name"           # restore archived memory
  python3 query_memory.py --harvest <subagent-namespace> # harvest sub-agent memories
  python3 query_memory.py --check                        # connectivity check only (exit 0=ok, 1=warn, 2=down)
"""

import argparse, json, os, sys
from pathlib import Path

SKILL_DIR      = Path(__file__).parent.parent
DEFAULT_CONFIG = Path.home() / ".openclaw" / "pgmemory.json"

def load_config(path: Path) -> dict:
    if not path.exists():
        print(f"Config not found: {path}\nRun: python3 {SKILL_DIR}/scripts/setup.py", file=sys.stderr)
        sys.exit(1)
    with open(path) as f: return json.load(f)

def get_embed(text: str, config: dict):
    provider = config.get("embeddings", {}).get("provider", "voyage")
    key_env  = config.get("embeddings", {}).get("api_key_env", "VOYAGE_API_KEY")
    # Prefer api_key stored directly in config; fall back to env var
    [REDACTED_SECRET]"embeddings", {}).get("api_key") or os.environ.get(key_env, "")
    model    = config.get("embeddings", {}).get("model")
    import urllib.request
    try:
        if provider == "voyage":
            data = json.dumps({"input": [text], "model": model or "voyage-3"}).encode()
            req  = urllib.request.Request("https://api.voyageai.com/v1/embeddings", data=data,
                       headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"})
            return json.loads(urllib.request.urlopen(req, timeout=15).read())["data"][0]["embedding"]
        elif provider == "openai":
            data = json.dumps({"input": text, "model": model or "text-embedding-3-small"}).encode()
            req  = urllib.request.Request("https://api.openai.com/v1/embeddings", data=data,
                       headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"})
            return json.loads(urllib.request.urlopen(req, timeout=15).read())["data"][0]["embedding"]
        elif provider == "ollama":
            data = json.dumps({"model": model or "nomic-embed-text", "prompt": text}).encode()
            req  = urllib.request.Request("http://localhost:11434/api/embeddings", data=data,
                       headers={"Content-Type": "application/json"})
            return json.loads(urllib.request.urlopen(req, timeout=15).read())["embedding"]
    except Exception as e:
        print(f"Embedding failed: {e}", file=sys.stderr)
        return None

def fmt_row(row, show_score=True) -> str:
    key, category, importance, content, relevance, updated_at = row[:6]
    stars = "★" * importance + "☆" * (3 - importance)
    score = f"  rel={relevance:.2f}" if show_score else ""
    date  = str(updated_at)[:10] if updated_at else "?"
    short = content[:120] + ("…" if len(content) > 120 else "")
    return f"[{category}/{stars}]{score}  {key}  ({date})\n    {short}"

def main():
    p = argparse.ArgumentParser(description="Search and inspect pgmemory memories")
    p.add_argument("query",           nargs="?",  help="Semantic search query")
    p.add_argument("--agent",         default=None, help="Agent namespace (default: from config)")
    p.add_argument("--importance",    type=int,   choices=[1,2,3], help="Filter by minimum importance level")
    p.add_argument("--category",      default=None, choices=["decision","constraint","infrastructure","vision","preference","context","task","all"], help="Filter by category")
    p.add_argument("--limit",         type=int,   default=10, help="Max results to return (default: 10)")
    p.add_argument("--stats",         action="store_true", help="Show memory counts and health stats")
    p.add_argument("--list",          action="store_true", help="List all memory keys")
    p.add_argument("--include-archived", action="store_true", help="Also search archived memories")
    p.add_argument("--restore",       metavar="KEY", help="Restore archived memory by key")
    p.add_argument("--harvest",       metavar="NAMESPACE", help="Harvest memories from a sub-agent namespace")
    p.add_argument("--harvest-threshold", type=int, default=2, metavar="N", help="Only harvest memories with importance >= N (default: 2)")
    p.add_argument("--config",        default=str(DEFAULT_CONFIG), help=f"Path to pgmemory.json (default: {DEFAULT_CONFIG})")
    p.add_argument("--json",          action="store_true", help="Output results as JSON")
    p.add_argument("--check",         action="store_true", help="Connectivity check only — exit 0=ok, 2=unreachable")
    args = p.parse_args()

    config     = load_config(Path(args.config))
    agent_name = args.agent or config.get("agent", {}).get("name", "main")

    try:
        import psycopg2
        db_uri = config.get("db", {}).get("uri", "")
        # Add connect_timeout if not already in URI
        uri_with_timeout = db_uri + ("&" if "?" in db_uri else "?") + "connect_timeout=5"
        conn = psycopg2.connect(uri_with_timeout)
        conn.autocommit = True
        cur  = conn.cursor()
    except Exception as e:
        host = db_uri.split("@")[-1].split("/")[0] if "@" in db_uri else db_uri
        print(f"\n⚠️  pgmemory UNREACHABLE — {host}", file=sys.stderr)
        print(f"   Error: {e}", file=sys.stderr)
        print(f"   Falling back to MEMORY.md and daily memory files.", file=sys.stderr)
        print(f"   Fix: check DB at {host} and verify pgmemory.json config.", file=sys.stderr)
        sys.exit(2)

    if args.check:
        try:
            cur.execute("SELECT COUNT(*) FROM memories")
            count = cur.fetchone()[0]
            print(f"✓ pgmemory OK — {count:,} memories accessible")
            sys.exit(0)
        except Exception as e:
            print(f"\n⚠️  pgmemory connected but schema missing: {e}", file=sys.stderr)
            print(f"   Run: python3 {SKILL_DIR}/scripts/setup.py --migrate", file=sys.stderr)
            sys.exit(2)

    # ── Stats ──────────────────────────────────────────────────────────────────
    if args.stats:
        cur.execute("SELECT COUNT(*) FROM memories WHERE agent=%s", (agent_name,))
        active = cur.fetchone()[0]
        cur.execute("SELECT COUNT(*) FROM archived_memories WHERE agent=%s", (agent_name,))
        archived = cur.fetchone()[0]
        cur.execute("""
            SELECT category, COUNT(*), AVG(relevance_score)::numeric(4,2)
            FROM memories WHERE agent=%s GROUP BY category ORDER BY COUNT(*) DESC
        """, (agent_name,))
        by_cat = cur.fetchall()
        cur.execute("""
            SELECT importance, COUNT(*) FROM memories
            WHERE agent=%s GROUP BY importance ORDER BY importance DESC
        """, (agent_name,))
        by_imp = cur.fetchall()
        cur.execute("SELECT COUNT(*) FROM memories WHERE agent=%s AND embedding IS NULL", (agent_name,))
        no_embed = cur.fetchone()[0]

        if args.json:
            print(json.dumps({"agent": agent_name, "active": active, "archived": archived,
                               "by_category": {r[0]: {"count": r[1], "avg_relevance": float(r[2])} for r in by_cat},
                               "by_importance": {r[0]: r[1] for r in by_imp},
                               "missing_embeddings": no_embed}))
        else:
            print(f"\npgmemory stats — agent: {agent_name}")
            print(f"  Active:   {active:,}  |  Archived: {archived:,}  |  No embedding: {no_embed}")
            print(f"\n  By category:")
            for cat, count, avg_rel in by_cat:
                print(f"    {cat:<20} {count:>5}  avg_relevance={avg_rel}")
            print(f"\n  By importance:")
            for imp, count in by_imp:
                stars = "★" * imp + "☆" * (3-imp)
                print(f"    {stars}  {count:>5}")
        cur.close(); conn.close(); return

    # ── List ───────────────────────────────────────────────────────────────────
    if args.list:
        q    = "SELECT key, category, importance, updated_at FROM memories WHERE agent=%s"
        vals = [agent_name]
        if args.category and args.category != 'all':   q += " AND category=%s"; vals.append(args.category)
        if args.importance: q += " AND importance=%s"; vals.append(args.importance)
        q += " ORDER BY importance DESC, category, key"
        cur.execute(q, vals)
        rows = cur.fetchall()
        if args.json:
            print(json.dumps([{"key":r[0],"category":r[1],"importance":r[2],"updated":str(r[3])} for r in rows]))
        else:
            for key, cat, imp, upd in rows:
                print(f"  {'★'*imp+'☆'*(3-imp)}  [{cat}]  {key}  ({str(upd)[:10]})")
        cur.close(); conn.close(); return

    # ── Restore ────────────────────────────────────────────────────────────────
    if args.restore:
        cur.execute("""
            SELECT agent,key,category,content,importance,source,tags,expires_at
            FROM archived_memories WHERE agent=%s AND key=%s ORDER BY archived_at DESC LIMIT 1
        """, (agent_name, args.restore))
        row = cur.fetchone()
        if not row:
            print(f"No archived memory found: {args.restore}"); cur.close(); conn.close(); return
        conn.autocommit = False
        cur.execute("""
            INSERT INTO memories (agent,key,category,content,importance,source,tags,expires_at,relevance_score)
            VALUES (%s,%s,%s,%s,%s,%s,%s,%s,1.0)
            ON CONFLICT (agent,key) DO UPDATE SET
                content=EXCLUDED.content, relevance_score=1.0, updated_at=NOW()
        """, row)
        cur.execute("DELETE FROM archived_memories WHERE agent=%s AND key=%s", (agent_name, args.restore))
        conn.commit()
        print(f"✓ Restored: {args.restore}")
        cur.close(); conn.close(); return

    # ── Harvest ────────────────────────────────────────────────────────────────
    if args.harvest:
        src_ns = args.harvest
        cur.execute("""
            SELECT key, category, content, importance, embedding
            FROM memories WHERE agent=%s AND importance>=%s AND key NOT LIKE '%%:summary'
        """, (src_ns, args.harvest_threshold))
        rows = cur.fetchall()
        conn.autocommit = False
        promoted = skipped = 0
        for key, category, content, importance, embedding in rows:
            # Check similarity against existing primary memories
            if embedding:
                cur.execute("""
                    SELECT 1 FROM memories
                    WHERE agent=%s AND embedding IS NOT NULL
                    AND 1-(embedding<=>%s::vector) > 0.95 LIMIT 1
                """, (agent_name, embedding))
                if cur.fetchone(): skipped += 1; continue

            new_key = f"from:{src_ns}:{key}"
            if embedding:
                cur.execute("""
                    INSERT INTO memories (agent,key,category,content,embedding,importance,source)
                    VALUES (%s,%s,%s,%s,%s::vector,%s,%s)
                    ON CONFLICT (agent,key) DO UPDATE SET
                        content=EXCLUDED.content, importance=EXCLUDED.importance, updated_at=NOW()
                """, (agent_name, new_key, category, content, embedding, importance, f"harvest:{src_ns}"))
            else:
                cur.execute("""
                    INSERT INTO memories (agent,key,category,content,importance,source)
                    VALUES (%s,%s,%s,%s,%s,%s)
                    ON CONFLICT (agent,key) DO NOTHING
                """, (agent_name, new_key, category, content, importance, f"harvest:{src_ns}"))
            promoted += 1

        conn.commit()
        print(f"✓ Harvested from {src_ns}: {promoted} promoted, {skipped} skipped (duplicate)")
        cur.close(); conn.close(); return

    # ── Semantic search ────────────────────────────────────────────────────────
    results = []

    if args.query:
        vec = get_embed(args.query, config)
        if not vec:
            print("Embedding failed — cannot run semantic search", file=sys.stderr); sys.exit(1)

        emb_str = f"[{','.join(str(x) for x in vec)}]"

        # Build filter
        filters = ["agent=%s", "embedding IS NOT NULL"]
        vals    = [agent_name]
        if args.importance: filters.append("importance>=%s"); vals.append(args.importance)
        if args.category and args.category != 'all':   filters.append("category=%s");   vals.append(args.category)
        where = " AND ".join(filters)

        cur.execute(f"""
            SELECT key, category, importance, content, relevance_score, updated_at,
                   1-(embedding<=>%s::vector) AS similarity
            FROM memories WHERE {where}
            ORDER BY embedding<=>%s::vector
            LIMIT %s
        """, [emb_str] + vals + [emb_str, args.limit])
        results = [(*r[:6], r[6]) for r in cur.fetchall()]

        # Auto-update access_count + last_accessed
        if results:
            keys = [r[0] for r in results]
            cur.execute("""
                UPDATE memories SET access_count=access_count+1, last_accessed=NOW()
                WHERE agent=%s AND key=ANY(%s)
            """, (agent_name, keys))

        # Optionally search archive too
        if args.include_archived:
            cur.execute(f"""
                SELECT key, category, importance, content, 0.0, archived_at,
                       1-(embedding<=>%s::vector) AS similarity
                FROM archived_memories WHERE agent=%s
                AND embedding IS NOT NULL AND 1-(embedding<=>%s::vector) > 0.5
                ORDER BY similarity DESC LIMIT 5
            """, [emb_str, agent_name, emb_str])
            archive_rows = cur.fetchall()
            if archive_rows:
                print(f"\n{len(archive_rows)} archived result(s):")
                for r in archive_rows:
                    print(f"  [ARCHIVED] {fmt_row(r)}")
                    info_restore = f"  Restore: python3 {SKILL_DIR}/scripts/query_memory.py --restore \"{r[0]}\""
                    print(info_restore)

    else:
        # No query — load by importance/category
        filters = ["agent=%s"]
        vals    = [agent_name]
        if args.importance: filters.append("importance>=%s"); vals.append(args.importance)
        if args.category and args.category != 'all':   filters.append("category=%s");   vals.append(args.category)
        where = " AND ".join(filters)

        cur.execute(f"""
            SELECT key, category, importance, content, relevance_score, updated_at, relevance_score
            FROM memories WHERE {where}
            ORDER BY importance DESC, relevance_score DESC
            LIMIT %s
        """, vals + [args.limit])
        results = list(cur.fetchall())

    cur.close(); conn.close()

    if not results:
        print("No memories found.")
        return

    if args.json:
        out = [{"key":r[0],"category":r[1],"importance":r[2],"content":r[3],
                "relevance":float(r[4] or 0),"updated":str(r[5]),"similarity":float(r[6] or 0)}
               for r in results]
        print(json.dumps(out, indent=2))
    else:
        query_label = f'"{args.query}"' if args.query else f"importance>={args.importance or 1}"
        print(f"\n{len(results)} result(s) for {query_label} — agent: {agent_name}\n")
        for r in results:
            print(fmt_row(r))
            if len(r) > 6 and r[6]: print(f"    similarity: {r[6]:.3f}")
            print()

if __name__ == "__main__":
    main()


path: scripts/setup.py
content: #!/usr/bin/env python3
"""
pgmemory setup.py
─────────────────
Usage:
  python3 setup.py                  # interactive wizard
  python3 setup.py --validate       # validate config only
  python3 setup.py --migrate        # run pending migrations
  python3 setup.py --rollback 2     # rollback to schema version 2
  python3 setup.py --doctor         # full system health check
  python3 setup.py --decay          # recalculate relevance scores
  python3 setup.py --yes            # non-interactive (use defaults)
  python3 setup.py --config <path>  # use specific config file
"""

import argparse, hashlib, json, math, os, platform, re, subprocess, sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional

SKILL_DIR      = Path(__file__).parent.parent
MIGRATIONS_DIR = SKILL_DIR / "scripts" / "migrations"
DEFAULT_CONFIG = Path.home() / ".openclaw" / "pgmemory.json"

# ── Colours ────────────────────────────────────────────────────────────────────
USE_COLOR = sys.stdout.isatty()
def _c(code, t): return f"\033[{code}m{t}\033[0m" if USE_COLOR else t
green  = lambda t: _c("32", t)
yellow = lambda t: _c("33", t)
red    = lambda t: _c("31", t)
bold   = lambda t: _c("1",  t)
dim    = lambda t: _c("2",  t)
def ok(m):   print(f"  {green('✓')} {m}")
def warn(m): print(f"  {yellow('⚠')} {m}")
def err(m):  print(f"  {red('✗')} {m}")
def info(m): print(f"  {dim('→')} {m}")
def hdr(m):  print(f"\n{bold(m)}")

# ── Defaults ───────────────────────────────────────────────────────────────────
SANE_DEFAULTS = {
    "memory": {
        "importance_threshold_write": 2,
        "importance_threshold_load":  3,
        "save_interval_messages":     20,
        "save_interval_minutes":      30,
        "interval_write_mode":        "replace",
        "inherit_on_spawn":           True,
        "ttl_days":                   {"1": 30, "2": 180, "3": None},
        "never_expire_categories":    ["decision","constraint","infrastructure","vision","preference"],
        "dedup_window_minutes":       5,
        "similarity_dedup_threshold": 0.95,
        "max_memories":               10000,
    },
    "decay": {
        "enabled": True,
        "rates": {
            "decision": 0.001, "constraint": 0.001,
            "infrastructure": 0.003, "vision": 0.003, "preference": 0.003,
            "context": 0.05, "task": 0.1,
        },
        "reinforcement_factor": 0.1,
        "archive_threshold":    0.1,
        "restore_on_access":    True,
    },
    "archive": {
        "enabled": True, "on_expire": True, "on_evict": True,
        "on_decay": True, "archived_ttl_days": None, "restore_on_access": True,
    },
    "agents":    {"namespace_from_agent_id": True, "search_scope": "completed"},
    "subagents": {"auto_inject_context": True, "context_top_k": 5,
                  "auto_harvest_on_complete": True, "harvest_importance_threshold": 2},
}

EXPECTED_DIMS   = {"voyage": 1024, "openai": 1536, "ollama": 768}
VALID_PROVIDERS = list(EXPECTED_DIMS.keys())
VALID_CATS      = ["decision","constraint","infrastructure","vision","preference","context","task"]

# ── Config helpers ─────────────────────────────────────────────────────────────
def load_config(path: Path) -> dict:
    if not path.exists(): return {}
    with open(path) as f: return json.load(f)

def save_config(config: dict, path: Path):
    path.parent.mkdir(parents=True, exist_ok=True)
    with open(path, "w") as f: json.dump(config, f, indent=2)
    print(f"\n{green('✓')} Config saved to {path}")

def merged(config: dict) -> dict:
    result = json.loads(json.dumps(SANE_DEFAULTS))
    def deep_merge(base, over):
        for k, v in over.items():
            if isinstance(v, dict) and isinstance(base.get(k), dict): deep_merge(base[k], v)
            else: base[k] = v
    deep_merge(result, config)
    return result

# ── DB ─────────────────────────────────────────────────────────────────────────
def connect(uri: str):
    try:
        import psycopg2
        conn = psycopg2.connect(uri)
        conn.autocommit = False
        return conn
    except Exception:
        return None

def get_applied(conn) -> dict:
    cur = conn.cursor()
    try:
        cur.execute("SELECT version, checksum FROM pgmemory_migrations ORDER BY version")
        return {r[0]: r[1] for r in cur.fetchall()}
    except Exception:
        return {}
    finally:
        cur.close()

def file_md5(path: Path) -> str:
    return hashlib.md5(path.read_bytes()).hexdigest()

def extract_section(sql: str, section: str) -> str:
    m = re.search(rf"--\s*{section}\s*\n(.*?)(?=--\s*(?:UP|DOWN)|$)", sql, re.DOTALL|re.IGNORECASE)
    return m.group(1).strip() if m else sql.strip()

def migration_files() -> list:
    result = []
    for f in sorted(MIGRATIONS_DIR.glob("*.sql")):
        m = re.match(r"^(\d+)_", f.name)
        if m: result.append((int(m.group(1)), f))
    return result

# ── Validate ───────────────────────────────────────────────────────────────────
def cmd_validate(config_path: Path) -> bool:
    hdr("Validating config")
    errors, warnings = [], []

    if not config_path.exists():
        err(f"Config not found: {config_path}")
        info("Run setup.py without flags to create one")
        return False

    try:
        config = load_config(config_path)
    except json.JSONDecodeError as e:
        err(f"Invalid JSON: {e}"); return False
    ok("Valid JSON")

    db_uri = config.get("db", {}).get("uri", "")
    if not db_uri:                               errors.append("db.uri is required")
    elif not db_uri.startswith("postgresql"):    errors.append("db.uri must be a PostgreSQL URI")
    else:                                        ok(f"db.uri present")

    provider = config.get("embeddings", {}).get("provider", "")
    if not provider:                             errors.append("embeddings.provider is required")
    elif provider not in VALID_PROVIDERS:        errors.append(f"provider must be: {', '.join(VALID_PROVIDERS)}")
    else:                                        ok(f"embeddings.provider: {provider}")

    agent = config.get("agent", {}).get("name", "")
    if not agent: errors.append("agent.name is required")
    else:         ok(f"agent.name: {agent}")

    if provider and provider != "ollama":
        key_env     = config.get("embeddings", {}).get("api_key_env", "")
        stored_key  = config.get("embeddings", {}).get("api_key", "")
        env_key     = os.environ.get(key_env, "") if key_env else ""
        resolved    = stored_key or env_key
        if not resolved:
            if stored_key:  ok(f"embeddings.api_key stored in config")
            elif key_env:   warnings.append(f"${key_env} not set and no api_key in config")
            else:           warnings.append(f"No api_key_env or api_key configured for {provider}")
        else:
            src = "config" if stored_key else f"${key_env}"
            ok(f"API key available (from {src})")

    cfg = merged(config)
    m = cfg.get("memory", {})
    if not (1 <= m.get("importance_threshold_write", 2) <= 3):
        errors.append("importance_threshold_write must be 1-3")
    for cat, rate in cfg.get("decay", {}).get("rates", {}).items():
        if not (0.0 <= rate <= 1.0):
            errors.append(f"decay.rates.{cat}={rate} must be 0.0-1.0")
    known = {"db","embeddings","agent","memory","decay","archive","agents","subagents"}
    for k in config:
        if k not in known: warnings.append(f"Unknown config key: {k}")

    for w in warnings: warn(w)
    for e_ in errors:  err(e_)

    if errors:
        print(f"\n{red(str(len(errors)) + ' error(s)')} — fix before running setup")
        return False
    suf = f", {len(warnings)} warning(s)" if warnings else ""
    print(f"\n{green('Config is valid')}{suf}")
    return True

# ── Migrations ─────────────────────────────────────────────────────────────────
def cmd_migrate(conn) -> bool:
    hdr("Running migrations")
    cur = conn.cursor()
    # Bootstrap migrations table
    cur.execute("""
        CREATE TABLE IF NOT EXISTS pgmemory_migrations (
            version INT PRIMARY KEY, filename TEXT NOT NULL,
            checksum TEXT NOT NULL, applied_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
        )
    """)
    conn.commit()

    applied = get_applied(conn)
    pending = 0

    for version, path in migration_files():
        checksum = file_md5(path)
        if version in applied:
            if applied[version] != checksum:
                err(f"Migration {version:04d} checksum MISMATCH — file modified after apply")
                err("Restore the original file or run --rollback before re-applying")
                return False
            continue
        try:
            cur.execute(extract_section(path.read_text(), "UP"))
            cur.execute("INSERT INTO pgmemory_migrations (version,filename,checksum) VALUES (%s,%s,%s)",
                        (version, path.name, checksum))
            conn.commit()
            ok(f"Applied {version:04d}: {path.name}")
            pending += 1
        except Exception as e:
            conn.rollback()
            err(f"Migration {version:04d} FAILED: {e}")
            return False

    cur.close()
    files = migration_files()
    latest = max(v for v,_ in files) if files else 0
    if pending == 0: ok(f"Already at version {latest:04d} — nothing to do")
    else:            ok(f"Schema now at version {latest:04d} ({pending} applied)")
    return True

def cmd_rollback(conn, target: int) -> bool:
    hdr(f"Rolling back to version {target:04d}")
    applied = get_applied(conn)
    to_rb   = sorted([(v,p) for v,p in migration_files() if v > target and v in applied],
                     key=lambda x: x[0], reverse=True)
    if not to_rb:
        ok(f"Already at or below version {target:04d}"); return True

    cur = conn.cursor()
    for version, path in to_rb:
        sql_down = extract_section(path.read_text(), "DOWN")
        if not sql_down:
            err(f"Migration {version:04d} has no DOWN section"); return False
        try:
            cur.execute(sql_down)
            cur.execute("DELETE FROM pgmemory_migrations WHERE version = %s", (version,))
            conn.commit()
            ok(f"Rolled back {version:04d}: {path.name}")
        except Exception as e:
            conn.rollback()
            err(f"Rollback {version:04d} FAILED: {e}"); return False
    cur.close()
    ok(f"Schema at version {target:04d}")
    return True

# ── Embed helpers ──────────────────────────────────────────────────────────────
def embed_text(text: str, provider: str, api_key: str, model: str = None) -> Optional[list]:
    import urllib.request
    try:
        if provider == "voyage":
            m = model or "voyage-3"
            data = json.dumps({"input": [text], "model": m}).encode()
            req  = urllib.request.Request("https://api.voyageai.com/v1/embeddings", data=data,
                       headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"})
            resp = json.loads(urllib.request.urlopen(req, timeout=15).read())
            return resp["data"][0]["embedding"]
        elif provider == "openai":
            m = model or "text-embedding-3-small"
            data = json.dumps({"input": text, "model": m}).encode()
            req  = urllib.request.Request("https://api.openai.com/v1/embeddings", data=data,
                       headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"})
            resp = json.loads(urllib.request.urlopen(req, timeout=15).read())
            return resp["data"][0]["embedding"]
        elif provider == "ollama":
            m = model or "nomic-embed-text"
            data = json.dumps({"model": m, "prompt": text}).encode()
            req  = urllib.request.Request("http://localhost:11434/api/embeddings", data=data,
                       headers={"Content-Type": "application/json"})
            resp = json.loads(urllib.request.urlopen(req, timeout=15).read())
            return resp["embedding"]
    except Exception:
        return None

def get_stored_dims(conn) -> Optional[int]:
    cur = conn.cursor()
    try:
        cur.execute("SELECT array_length(embedding::real[], 1) FROM memories WHERE embedding IS NOT NULL LIMIT 1")
        row = cur.fetchone()
        return row[0] if row else None
    except Exception: return None
    finally: cur.close()

# ── Doctor ─────────────────────────────────────────────────────────────────────
def cmd_doctor(config: dict) -> int:
    cfg = merged(config)
    uri = config.get("db", {}).get("uri", "")
    errors, warnings = [], []

    hdr("Connectivity")
    conn = connect(uri) if uri else None
    if not conn:
        err(f"Cannot connect: {uri or '(no uri configured)'}")
        print(f"\n{red('Cannot continue — fix database connection first')}")
        return 2
    ok("Database connection OK")

    cur = conn.cursor()
    try:
        cur.execute("SELECT extversion FROM pg_extension WHERE extname = 'vector'")
        row = cur.fetchone()
        if row: ok(f"pgvector {row[0]} installed")
        else:   err("pgvector not installed"); errors.append("pgvector")
    except Exception as e:
        err(f"pgvector check failed: {e}"); errors.append("pgvector")

    cur.execute("SELECT 1 FROM pg_extension WHERE extname = 'pg_trgm'")
    if cur.fetchone(): ok("pg_trgm installed")
    else: warn("pg_trgm not installed (optional)"); warnings.append("pg_trgm")

    hdr("Schema")
    applied  = get_applied(conn)
    files    = migration_files()
    if files:
        latest  = max(v for v,_ in files)
        current = max(applied.keys()) if applied else 0
        if current >= latest: ok(f"Schema at version {current:04d} (latest)")
        else:
            err(f"Schema at version {current:04d}, latest is {latest:04d}")
            info("Run: python3 setup.py --migrate"); errors.append("schema_behind")

        bad_cs = [v for v,p in files if v in applied and applied[v] != file_md5(p)]
        if bad_cs:
            for v in bad_cs: err(f"Migration {v:04d} checksum mismatch — file modified after apply")
            errors.extend([f"checksum_{v}" for v in bad_cs])
        elif applied: ok("All migration checksums verified")

    for tbl in ["memories","archived_memories","session_state","pgmemory_migrations"]:
        cur.execute("SELECT 1 FROM information_schema.tables WHERE table_name=%s", (tbl,))
        if cur.fetchone(): ok(f"Table '{tbl}' exists")
        else:
            err(f"Table '{tbl}' missing — run: python3 setup.py --migrate")
            errors.append(f"table_{tbl}")

    hdr("Embeddings")
    provider = config.get("embeddings", {}).get("provider", "")
    key_env  = config.get("embeddings", {}).get("api_key_env", "")
    [REDACTED_SECRET], "") if key_env else ""

    if not provider:
        warn("No embedding provider configured"); warnings.append("no_provider")
    else:
        if provider != "ollama" and not api_key:
            err(f"${key_env} not set"); errors.append("api_key")
        else:
            vec = embed_text("pgmemory doctor test", provider, api_key)
            if vec:
                dims = len(vec)
                ok(f"Test embed OK ({dims}-dim) via {provider}")
                stored = get_stored_dims(conn)
                if stored and stored != dims:
                    err(f"DIMENSION MISMATCH: stored={stored}, provider returns {dims}")
                    info("Option A: revert provider in pgmemory.json")
                    info("Option B: Change embeddings.provider back to the original provider")
                    info("Option C: Contact support or open an issue for --re-embed guidance")
                    errors.append("dimension_mismatch")
            else:
                err(f"Test embed FAILED — check API key / provider config")
                errors.append("embed_failed")

    hdr("Memory")
    try:
        cur.execute("SELECT COUNT(*) FROM memories")
        count = cur.fetchone()[0]
        max_m = cfg.get("memory", {}).get("max_memories", 10000)
        pct   = int(100 * count / max_m) if max_m else 0
        if max_m and pct >= 90:
            warn(f"{count:,} / {max_m:,} memories ({pct}% of cap)")
            info("Run: python3 setup.py --decay  (will archive low-relevance memories)"); warnings.append("near_cap")
        else:
            ok(f"{count:,} active memories" + (f" ({pct}% of cap)" if max_m else ""))

        cur.execute("SELECT COUNT(*) FROM memories WHERE embedding IS NULL")
        nulls = cur.fetchone()[0]
        if nulls: warn(f"{nulls:,} memories without embeddings"); warnings.append("null_embeds")
        else:     ok("All memories have embeddings")

        cur.execute("SELECT COUNT(*) FROM archived_memories")
        ok(f"{cur.fetchone()[0]:,} archived memories")
    except Exception as e:
        warn(f"Could not query memories: {e}"); warnings.append("memory_query")

    cur.close(); conn.close()
    print()
    if errors:
        print(red(f"{len(errors)} error(s), {len(warnings)} warning(s)") + " — see above")
        return 2
    elif warnings:
        print(yellow(f"0 errors, {len(warnings)} warning(s)"))
        return 1
    else:
        print(green("All checks passed ✓"))
        return 0

# ── Decay ──────────────────────────────────────────────────────────────────────
def cmd_decay(conn, config: dict) -> bool:
    hdr("Running decay cycle")
    cfg   = merged(config)
    dcfg  = cfg.get("decay", {})
    acfg  = cfg.get("archive", {})
    if not dcfg.get("enabled", True):
        info("Decay disabled in config"); return True

    rates     = dcfg.get("rates", {})
    r_factor  = dcfg.get("reinforcement_factor", 0.1)
    threshold = dcfg.get("archive_threshold", 0.1)
    on_decay  = acfg.get("on_decay", True) and acfg.get("enabled", True)

    cur = conn.cursor()
    try:
        cur.execute("SELECT id, category, importance, relevance_score, access_count, updated_at FROM memories")
        rows    = cur.fetchall()
        now     = datetime.now(timezone.utc)
        updated = archived = 0

        for mem_id, category, importance, relevance, access_count, updated_at in rows:
            if updated_at:
                if updated_at.tzinfo is None: updated_at = updated_at.replace(tzinfo=timezone.utc)
                days = (now - updated_at).total_seconds() / 86400
            else: days = 0

            base_rate  = rates.get(category, 0.05)
            imp_weight = importance / 3.0
            rate       = base_rate * (1.0 / max(imp_weight, 0.1))
            new_score  = min(1.0, math.exp(-rate * days) * (1 + math.log1p(access_count) * r_factor) * imp_weight)

            if abs(new_score - relevance) < 0.001: continue

            if new_score < threshold and on_decay:
                cur.execute("""
                    INSERT INTO archived_memories SELECT *, NOW(), 'decayed'
                    FROM memories WHERE id = %s ON CONFLICT DO NOTHING
                """, (mem_id,))
                cur.execute("DELETE FROM memories WHERE id = %s", (mem_id,))
                archived += 1
            else:
                cur.execute("UPDATE memories SET relevance_score=%s WHERE id=%s",
                            (round(new_score, 4), mem_id))
                updated += 1

        conn.commit()
        ok(f"Updated {updated:,} scores")
        if archived: ok(f"Archived {archived:,} decayed memories")
        cur.close()
        return True
    except Exception as e:
        conn.rollback(); cur.close()
        err(f"Decay failed: {e}"); return False

# ── Docker ─────────────────────────────────────────────────────────────────────
def detect_docker() -> bool:
    try:
        return subprocess.run(["docker","info"], capture_output=True, timeout=5).returncode == 0
    except Exception: return False

def install_docker(yes: bool) -> bool:
    hdr("Installing Docker")
    system = platform.system().lower()
    if system == "linux":
        if not yes and input("  Install Docker via get.docker.com? [y/n] ").lower() != "y": return False
        if subprocess.run("curl -fsSL https://get.docker.com | sh", shell=True).returncode != 0:
            err("Docker install failed"); return False
        subprocess.run(["sudo","systemctl","enable","--now","docker"], capture_output=True)
        user = os.environ.get("USER","")
        if user:
            subprocess.run(["sudo","usermod","-aG","docker",user], capture_output=True)
            warn(f"Added {user} to docker group — re-login for this to take effect")
        ok("Docker installed"); return True
    elif system == "darwin":
        if subprocess.run(["which","brew"], capture_output=True).returncode == 0:
            if not yes and input("  Install docker + colima via Homebrew? [y/n] ").lower() != "y": return False
            subprocess.run(["brew","install","docker","colima"])
            subprocess.run(["colima","start"])
            ok("Docker + Colima ready"); return True
        else:
            err("Install Docker Desktop: https://docker.com/products/docker-desktop"); return False
    err(f"Unsupported OS for auto-install: {system}"); return False

def start_postgres_docker(config_path: Path) -> Optional[str]:
    compose_path = config_path.parent / "pgmemory-compose.yml"
    compose_path.write_text("""services:
  pgmemory:
    image: pgvector/pgvector:pg17
    container_name: pgmemory
    restart: unless-stopped
    environment:
      POSTGRES_USER: openclaw
      POSTGRES_PASSWORD: pgmemory
      POSTGRES_DB: openclaw
    ports:
      - "15432:5432"
    volumes:
      - pgmemory_data:/var/lib/postgresql/data
volumes:
  pgmemory_data:
""")
    for cmd in [["docker","compose","-f",str(compose_path),"up","-d"],
                ["docker-compose","-f",str(compose_path),"up","-d"]]:
        r = subprocess.run(cmd, capture_output=True, text=True)
        if r.returncode == 0: break
    else:
        err("Failed to start Docker container"); return None

    ok("Container started (pgvector/pgvector:pg17)")
    info("Waiting for PostgreSQL...")
    import time
    uri = "postgresql://openclaw:pgmemory@localhost:15432/openclaw"
    for _ in range(30):
        c = connect(uri)
        if c: c.close(); ok("PostgreSQL ready"); return uri
        time.sleep(1)
    err("PostgreSQL did not become ready in 30s"); return None

# ── AGENTS.md scaffold ─────────────────────────────────────────────────────────
PGMEMORY_STARTUP_MARKER = "<!-- pgmemory:startup -->"
PGMEMORY_SECTION_MARKER = "<!-- pgmemory:section -->"

def startup_steps(agent_name: str, config_path: Path) -> str:
    """The mandatory startup lines to inject into the Every Session block."""
    return f"""\
   **Query pgmemory instead of reading markdown files** — faster, focused, survives compaction. ({PGMEMORY_STARTUP_MARKER})
   ```bash
   # Check connectivity first — warns loudly if DB is down
   python3 {SKILL_DIR}/scripts/query_memory.py --check
   # Load critical memories (decisions, constraints, infrastructure)
   python3 {SKILL_DIR}/scripts/query_memory.py --importance 3 --limit 20
   # Semantic search for current task context
   python3 {SKILL_DIR}/scripts/query_memory.py "brief description of current work"
   ```
   If --check fails with exit code 2: warn the user ("⚠️ pgmemory unreachable") and fall back to MEMORY.md + daily files."""

def scaffold_agents_md(agent_name: str, db_uri: str, config_path: Path) -> str:
    """Full pgmemory reference section appended to AGENTS.md."""
    return f"""
{PGMEMORY_SECTION_MARKER}
## pgmemory — Persistent Semantic Memory

**Config:** `{config_path}`  |  **Namespace:** `{agent_name}`

pgmemory is the **default memory system**. Markdown files are the automatic backup.
Always write to both. Always read from pgmemory first.

### ⚡ Startup (MANDATORY)

```bash
# Check connectivity (warns loudly if DB is down)
python3 {SKILL_DIR}/scripts/query_memory.py --check

# Load critical memories
python3 {SKILL_DIR}/scripts/query_memory.py --importance 3 --limit 20

# Semantic search for current task context
python3 {SKILL_DIR}/scripts/query_memory.py "describe what you're working on"
```

If `--check` fails (exit 2): warn the user, fall back to reading daily markdown files.

### ✏️ Writing memories (always write to BOTH)

**1. pgmemory — primary:**
```bash
python3 {SKILL_DIR}/scripts/write_memory.py \\
  --key "unique.descriptive.key" \\
  --content "What you want to remember" \\
  --category decision \\
  --importance 3
```

**2. Daily markdown backup** — also append a brief note to `memory/YYYY-MM-DD.md`.
Full detail lives in pgmemory; markdown is the durable fallback if DB is ever unavailable.

### Categories & importance

| Category | Importance | Expires |
|---|---|---|
| decision, constraint | 3 | Never |
| infrastructure, vision, preference | 2–3 | Never or 180d |
| context | 2 | 180d |
| task | 1 | 30d |

### Maintenance

```bash
python3 {SKILL_DIR}/scripts/setup.py --doctor    # health check
python3 {SKILL_DIR}/scripts/setup.py --decay     # archive faded memories
python3 {SKILL_DIR}/scripts/query_memory.py --stats
```
"""

def inject_startup_into_agents_md(agents_md: Path, agent_name: str, config_path: Path) -> bool:
    """
    Try to inject the pgmemory startup step into the 'Every Session' numbered list.
    Returns True if injected, False if the pattern wasn't found (caller falls back to append).
    """
    content = agents_md.read_text()

    # Already injected?
    if PGMEMORY_STARTUP_MARKER in content:
        return True

    # Look for the numbered startup list — find the last "N. " item in an "Every Session" block
    # We look for a block that has "1. Read" style items and append after the last one
    import re
    # Match the last numbered item before a blank line or ## heading
    pattern = r'((?:^[ \t]*\d+\. .+\n)+)'
    matches = list(re.finditer(pattern, content, re.MULTILINE))
    if not matches:
        return False

    # Use the last numbered list (most likely to be the startup steps)
    last_match = matches[-1]
    insert_pos = last_match.end()

    step = startup_steps(agent_name, config_path)
    new_content = content[:insert_pos] + step + "\n" + content[insert_pos:]
    agents_md.write_text(new_content)
    return True

# ── Wizard ─────────────────────────────────────────────────────────────────────
def cmd_wizard(config_path: Path, yes: bool):
    print(bold("\n╔══════════════════════════════╗"))
    print(bold("║   pgmemory setup wizard      ║"))
    print(bold("╚══════════════════════════════╝\n"))

    existing = load_config(config_path)

    # ── DB ────────────────────────────────────────────────────────────────────
    hdr("Step 1: Database")
    existing_uri = existing.get("db", {}).get("uri", "")
    if existing_uri:
        info(f"Found existing config: {config_path}")
        use_it = yes or input(f"  Use existing DB? [{existing_uri[:60]}] [Y/n] ").lower() in ("","y")
        db_uri = existing_uri if use_it else input("  Enter PostgreSQL URI: ").strip()
    else:
        print("  [1] Docker — pgvector/pgvector:pg17 (recommended)")
        print("  [2] Existing PostgreSQL")
        choice = "1" if yes else (input("  Choice [1]: ").strip() or "1")
        if choice == "1":
            if not detect_docker():
                warn("Docker not found")
                do_install = yes or input("  Install Docker? [y/n] ").lower() == "y"
                if do_install:
                    if not install_docker(yes) or not detect_docker():
                        err("Docker unavailable — install manually and retry"); sys.exit(1)
                else:
                    err("Docker required for option 1"); sys.exit(1)
            db_uri = start_postgres_docker(config_path)
            if not db_uri: sys.exit(1)
        else:
            default = "postgresql://openclaw@localhost:5432/openclaw"
            db_uri  = (input(f"  URI [{default}]: ").strip() or default) if not yes else default

    conn = connect(db_uri)
    if not conn: err(f"Cannot connect to: {db_uri}"); sys.exit(1)
    ok("Connected")

    # ── Embeddings ────────────────────────────────────────────────────────────
    hdr("Step 2: Embeddings")
    existing_provider = existing.get("embeddings", {}).get("provider", "")
    if yes:
        provider = existing_provider or "voyage"
    else:
        print("  [1] Voyage AI  — voyage-3, 1024-dim (default, best)")
        print("  [2] OpenAI     — text-embedding-3-small, 1536-dim")
        print("  [3] Ollama     — nomic-embed-text, 768-dim (local, no key)")
        choice   = input("  Choice [1]: ").strip() or "1"
        provider = {"1":"voyage","2":"openai","3":"ollama"}.get(choice, "voyage")

    key_env = api_key = ""
    if provider != "ollama":
        default_env = {"voyage":"VOYAGE_API_KEY","openai":"OPENAI_API_KEY"}.get(provider,"EMBEDDING_API_KEY")
        key_env = (input(f"  API key env var [{default_env}]: ").strip() or default_env) if not yes \
                  else existing.get("embeddings",{}).get("api_key_env", default_env)
        # Try to resolve key: existing config → env var → prompt
        stored_key = existing.get("embeddings", {}).get("api_key", "")
        [REDACTED_SECRET] or os.environ.get(key_env, "")
        if not api_key and not yes:
            entered = input(f"  Paste {provider} API key (or leave blank to set ${key_env} later): ").strip()
            if entered: [REDACTED_SECRET]
        if not [REDACTED_SECRET]"No API key found — set ${key_env} or add api_key to config before querying")
        else:
            vec = embed_text("pgmemory setup test", provider, api_key)
            if vec: ok(f"Test embed OK ({len(vec)}-dim)")
            else:   warn("Test embed failed — check your API key")

    dims = EXPECTED_DIMS.get(provider, 1024)

    # ── Agent ─────────────────────────────────────────────────────────────────
    hdr("Step 3: Agent")
    default_agent = existing.get("agent", {}).get("name", "main")
    agent_name = default_agent if yes else (input(f"  OpenClaw agent name [{default_agent}]: ").strip() or default_agent)
    ok(f"Agent: {agent_name}")

    # ── Write config ──────────────────────────────────────────────────────────
    hdr("Step 4: Config")
    config = {"db": {"uri": db_uri},
              "embeddings": {"provider": provider, "dimensions": dims},
              "agent": {"name": agent_name}}
    if key_env: config["embeddings"]["api_key_env"] = key_env
    if [REDACTED_SECRET]"embeddings"]["api_key"] = api_key
    for k in ("memory","decay","archive","agents","subagents"):
        if k in existing: config[k] = existing[k]
    save_config(config, config_path)

    # ── Migrate ───────────────────────────────────────────────────────────────
    hdr("Step 5: Migrations")
    if not cmd_migrate(conn): conn.close(); sys.exit(1)
    conn.close()

    # ── AGENTS.md ─────────────────────────────────────────────────────────────
    hdr("Step 6: AGENTS.md")
    workspace  = Path.home() / ".openclaw" / "workspace"
    agents_md  = workspace / "AGENTS.md"
    section    = scaffold_agents_md(agent_name, db_uri, config_path)
    if agents_md.exists():
        content = agents_md.read_text()
        if PGMEMORY_SECTION_MARKER in content:
            ok("AGENTS.md already has pgmemory section")
        elif yes or input(f"  Update {agents_md} with pgmemory? [Y/n] ").lower() in ("","y"):
            # Try to inject startup steps into the numbered list first
            injected = inject_startup_into_agents_md(agents_md, agent_name, config_path)
            if injected:
                ok("Injected startup steps into 'Every Session' list")
            else:
                warn("Could not find numbered startup list — appending section instead")
            # Always append the full reference section
            with open(agents_md, "a") as f: f.write(section)
            ok(f"pgmemory section added to {agents_md}")
    elif yes or input(f"  Create {agents_md} with pgmemory? [Y/n] ").lower() in ("","y"):
        agents_md.parent.mkdir(parents=True, exist_ok=True)
        agents_md.write_text(section)
        ok(f"Created {agents_md}")

    # ── Cron ──────────────────────────────────────────────────────────────────
    hdr("Step 7: Decay schedule")
    decay_cmd = f"0 3 * * * python3 {SKILL_DIR}/scripts/setup.py --decay --config {config_path} >> /tmp/pgmemory-decay.log 2>&1"
    if yes:
        info(f"Add cron for daily decay:\n  {decay_cmd}")
    elif input("  Add daily decay cron (3am)? [Y/n] ").lower() in ("","y"):
        try:
            existing_cron = subprocess.run(["crontab","-l"], capture_output=True, text=True).stdout
            if "pgmemory" not in existing_cron:
                proc = subprocess.run(["crontab","-"], input=existing_cron.rstrip()+"\n"+decay_cmd+"\n",
                                      text=True, capture_output=True)
                if proc.returncode == 0: ok("Cron job added")
                else: warn(f"Could not add cron: {proc.stderr}")
            else: ok("Cron already present")
        except Exception as e:
            warn(f"Could not configure cron: {e}")
            info(f"Add manually:\n  {decay_cmd}")

    print(f"\n{bold('══════════════════════════════════════')}")
    print(f"{green('pgmemory is ready!')}")
    print(f"{'══════════════════════════════════════'}")
    print(f"  Config:   {config_path}")
    print(f"  Agent:    {agent_name} ({provider}, {dims}-dim)")
    print(f"  Write:    python3 {SKILL_DIR}/scripts/write_memory.py --help")
    print(f"  Search:   python3 {SKILL_DIR}/scripts/query_memory.py --help")
    print(f"  Health:   python3 {SKILL_DIR}/scripts/setup.py --doctor\n")

# ── Main ───────────────────────────────────────────────────────────────────────
def main():
    p = argparse.ArgumentParser(description="pgmemory — persistent semantic memory for OpenClaw")
    p.add_argument("--config",   default=str(DEFAULT_CONFIG), help=f"Path to pgmemory.json (default: {DEFAULT_CONFIG})")
    p.add_argument("--validate", action="store_true", help="Validate config file and exit")
    p.add_argument("--migrate",  action="store_true", help="Run pending schema migrations")
    p.add_argument("--rollback", type=int, metavar="N", help="Roll back schema to version N")
    p.add_argument("--doctor",   action="store_true", help="Full system health check")
    p.add_argument("--decay",    action="store_true", help="Recalculate memory relevance scores and archive faded memories")
    p.add_argument("--sync-agents", action="store_true", help="Scaffold pgmemory into all OpenClaw agent workspaces")
    p.add_argument("--yes",      action="store_true", help="Non-interactive mode, accept all defaults")
    args = p.parse_args()

    config_path = Path(args.config)

    if args.validate:
        sys.exit(0 if cmd_validate(config_path) else 1)

    if args.doctor:
        sys.exit(cmd_doctor(load_config(config_path)))

    if args.migrate or args.rollback is not None:
        if not cmd_validate(config_path): sys.exit(1)
        conn = connect(load_config(config_path)["db"]["uri"])
        if not conn: err("Cannot connect"); sys.exit(1)
        ok_ = cmd_rollback(conn, args.rollback) if args.rollback is not None else cmd_migrate(conn)
        conn.close(); sys.exit(0 if ok_ else 1)

    if args.decay:
        config = load_config(config_path)
        if not config: err(f"Config not found: {config_path}"); sys.exit(1)
        conn = connect(config["db"]["uri"])
        if not conn: err("Cannot connect"); sys.exit(1)
        ok_ = cmd_decay(conn, config); conn.close(); sys.exit(0 if ok_ else 1)

    if args.sync_agents:
        config = load_config(config_path)
        if not config:
            err(f"Config not found: {config_path}"); sys.exit(1)
        ok_ = cmd_sync_agents(config_path, config['db']['uri'])
        sys.exit(0 if ok_ else 1)

    cmd_wizard(config_path, args.yes)


# ── Sync agents ────────────────────────────────────────────────────────────────
def find_openclaw_config() -> Optional[Path]:
    """Find the openclaw.json config file."""
    env_path = os.environ.get("OPENCLAW_CONFIG_PATH", "")
    candidates = [
        Path(env_path) if env_path else None,
        Path.home() / ".openclaw" / "openclaw.json",
        Path.home() / ".openclaw" / "config.json",
    ]
    for p in candidates:
        if p and p.exists() and p.is_file(): return p
    return None

def get_openclaw_agents(openclaw_config: Path) -> list:
    """Return list of {id, workspace} dicts from openclaw.json."""
    try:
        with open(openclaw_config) as f:
            data = json.load(f)
    except Exception:
        # Try JSON5-style (strip comments)
        import re
        with open(openclaw_config) as f:
            raw = f.read()
        raw = re.sub(r'//.*', '', raw)
        raw = re.sub(r'/\*.*?\*/', '', raw, flags=re.DOTALL)
        try:
            data = json.loads(raw)
        except Exception:
            return []

    agents = []
    agent_list = data.get("agents", {}).get("list", [])
    state_dir = Path(os.environ.get("OPENCLAW_STATE_DIR", Path.home() / ".openclaw"))

    for agent in agent_list:
        agent_id = agent.get("id", "main")
        # Resolve workspace path
        workspace = agent.get("workspace")
        if workspace:
            workspace = Path(workspace.replace("~", str(Path.home())))
        else:
            workspace = state_dir / f"workspace-{agent_id}"
            if not workspace.exists() and agent_id == "main":
                workspace = state_dir / "workspace"
        agents.append({"id": agent_id, "workspace": workspace})

    # Always include main if list is empty
    if not agents:
        workspace = state_dir / "workspace"
        agents.append({"id": "main", "workspace": workspace})

    return agents

def cmd_sync_agents(config: Path, db_uri: str) -> bool:
    hdr("Syncing agents")

    oc_config = find_openclaw_config()
    if not oc_config:
        warn("Could not find openclaw.json — checking default workspace only")
        agents = [{"id": "main", "workspace": Path.home() / ".openclaw" / "workspace"}]
    else:
        info(f"Found OpenClaw config: {oc_config}")
        agents = get_openclaw_agents(oc_config)

    if not agents:
        warn("No agents found in OpenClaw config"); return True

    conn = connect(db_uri)
    if not conn:
        err(f"Cannot connect to database"); return False
    cur = conn.cursor()

    updated = 0
    for agent in agents:
        agent_id  = agent["id"]
        workspace = agent["workspace"]
        agents_md = workspace / "AGENTS.md"

        # Check if workspace exists
        if not workspace.exists():
            warn(f"{agent_id}: workspace not found ({workspace}) — skipping")
            continue

        # Check if AGENTS.md has pgmemory
        if agents_md.exists():
            content = agents_md.read_text()
            if PGMEMORY_SECTION_MARKER in content:
                ok(f"{agent_id}: already configured")
                _ensure_session_state(cur, agent_id)
                conn.commit()
                continue

        # Add pgmemory block
        section = scaffold_agents_md(agent_id, db_uri, config)
        if agents_md.exists():
            injected = inject_startup_into_agents_md(agents_md, agent_id, config)
            if injected:
                ok(f"{agent_id}: startup steps injected into 'Every Session' list")
            with open(agents_md, "a") as f: f.write(section)
        else:
            agents_md.parent.mkdir(parents=True, exist_ok=True)
            agents_md.write_text(section)

        _ensure_session_state(cur, agent_id)
        conn.commit()
        ok(f"{agent_id}: pgmemory block added → {agents_md}")
        updated += 1

    cur.close(); conn.close()
    print()
    if updated:
        ok(f"{updated} agent(s) updated, {len(agents)-updated} already configured")
    else:
        ok(f"All {len(agents)} agent(s) already have pgmemory")
    return True

def _ensure_session_state(cur, agent_id: str):
    cur.execute("""
        INSERT INTO session_state (agent) VALUES (%s)
        ON CONFLICT (agent) DO NOTHING
    """, (agent_id,))

if __name__ == "__main__":
    main()


path: scripts/write_memory.py
content: #!/usr/bin/env python3
"""
pgmemory write_memory.py — write a memory to the DB

Usage:
  python3 write_memory.py --key "infra.db" --content "OVH at 10.10.0.1:5432" --importance 3 --category infrastructure
  python3 write_memory.py --key "decision.ui" --content "AI left panel ~380px" --category decision
"""

import argparse, json, os, sys
from pathlib import Path

SKILL_DIR      = Path(__file__).parent.parent
DEFAULT_CONFIG = Path.home() / ".openclaw" / "pgmemory.json"
VALID_CATS     = ["decision","constraint","infrastructure","vision","preference","context","task"]

def load_config(path: Path) -> dict:
    if not path.exists():
        print(f"Config not found: {path}\nRun: python3 {SKILL_DIR}/scripts/setup.py", file=sys.stderr)
        sys.exit(1)
    with open(path) as f: return json.load(f)

def get_embed(text: str, config: dict):
    provider = config.get("embeddings", {}).get("provider", "voyage")
    key_env  = config.get("embeddings", {}).get("api_key_env", "VOYAGE_API_KEY")
    # Prefer api_key stored directly in config; fall back to env var
    [REDACTED_SECRET]"embeddings", {}).get("api_key") or os.environ.get(key_env, "")
    model    = config.get("embeddings", {}).get("model")

    import urllib.request
    try:
        if provider == "voyage":
            data = json.dumps({"input": [text], "model": model or "voyage-3"}).encode()
            req  = urllib.request.Request("https://api.voyageai.com/v1/embeddings", data=data,
                       headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"})
            resp = json.loads(urllib.request.urlopen(req, timeout=15).read())
            return resp["data"][0]["embedding"]
        elif provider == "openai":
            data = json.dumps({"input": text, "model": model or "text-embedding-3-small"}).encode()
            req  = urllib.request.Request("https://api.openai.com/v1/embeddings", data=data,
                       headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"})
            resp = json.loads(urllib.request.urlopen(req, timeout=15).read())
            return resp["data"][0]["embedding"]
        elif provider == "ollama":
            data = json.dumps({"model": model or "nomic-embed-text", "prompt": text}).encode()
            req  = urllib.request.Request("http://localhost:11434/api/embeddings", data=data,
                       headers={"Content-Type": "application/json"})
            resp = json.loads(urllib.request.urlopen(req, timeout=15).read())
            return resp["embedding"]
    except Exception as e:
        print(f"Embedding failed: {e}", file=sys.stderr)
        return None

def main():
    p = argparse.ArgumentParser(description="Write a memory to pgmemory")
    p.add_argument("--key",        required=True,  help="Unique key (e.g. infra.db.ovh)")
    p.add_argument("--content",    required=True,  help="Memory text")
    p.add_argument("--category",   default="context", choices=VALID_CATS)
    p.add_argument("--importance", type=int, default=2, choices=[1,2,3])
    p.add_argument("--agent",      default=None,   help="Agent name (overrides config)")
    p.add_argument("--source",     default="session")
    p.add_argument("--tags",       nargs="*",      help="Optional tags")
    p.add_argument("--config",     default=str(DEFAULT_CONFIG))
    p.add_argument("--no-embed",   action="store_true", help="Skip embedding (faster, no vector search)")
    p.add_argument("--dry-run",    action="store_true")
    args = p.parse_args()

    config     = load_config(Path(args.config))
    agent_name = args.agent or config.get("agent", {}).get("name", "main")
    cfg        = config.get("memory", {})

    # TTL calculation
    ttl_days = cfg.get("ttl_days", {}).get(str(args.importance))
    never_cats = cfg.get("never_expire_categories",
                         ["decision","constraint","infrastructure","vision","preference"])
    expires_at = None
    if args.category not in never_cats and ttl_days:
        from datetime import datetime, timezone, timedelta
        expires_at = (datetime.now(timezone.utc) + timedelta(days=ttl_days)).isoformat()

    if args.dry_run:
        print(json.dumps({
            "agent": agent_name, "key": args.key, "category": args.category,
            "importance": args.importance, "content": args.content,
            "source": args.source, "expires_at": expires_at,
        }, indent=2))
        return

    # Get embedding
    embedding = None
    if not args.no_embed:
        embedding = get_embed(args.content, config)
        if embedding is None:
            print("Warning: embedding failed — writing without vector", file=sys.stderr)

    # Connect + write
    try:
        import psycopg2
        conn = psycopg2.connect(config["db"]["uri"])
        cur  = conn.cursor()

        # Dedup check: don't rewrite same key within dedup_window_minutes
        dedup_mins = cfg.get("dedup_window_minutes", 5)
        cur.execute("""
            SELECT updated_at FROM memories
            WHERE agent=%s AND key=%s
            AND updated_at > NOW() - ((%s || ' minutes')::interval)
        """, (agent_name, args.key, dedup_mins))
        if cur.fetchone():
            print(f"Skipped (dedup window {dedup_mins}min): {args.key}")
            cur.close(); conn.close(); return

        # Upsert: update if key exists, insert if new
        if embedding:
            emb_str = f"[{','.join(str(x) for x in embedding)}]"
            cur.execute("""
                INSERT INTO memories (agent, key, category, content, embedding, importance,
                                      source, tags, expires_at)
                VALUES (%s,%s,%s,%s,%s::vector,%s,%s,%s,%s)
                ON CONFLICT (agent, key) WHERE key IS NOT NULL DO UPDATE SET
                    content=EXCLUDED.content, category=EXCLUDED.category,
                    embedding=EXCLUDED.embedding, importance=EXCLUDED.importance,
                    source=EXCLUDED.source, tags=EXCLUDED.tags,
                    expires_at=EXCLUDED.expires_at, updated_at=NOW()
            """, (agent_name, args.key, args.category, args.content, emb_str,
                  args.importance, args.source, args.tags or [], expires_at))
        else:
            cur.execute("""
                INSERT INTO memories (agent, key, category, content, importance, source, tags, expires_at)
                VALUES (%s,%s,%s,%s,%s,%s,%s,%s)
                ON CONFLICT (agent, key) WHERE key IS NOT NULL DO UPDATE SET
                    content=EXCLUDED.content, category=EXCLUDED.category,
                    importance=EXCLUDED.importance, source=EXCLUDED.source,
                    tags=EXCLUDED.tags, expires_at=EXCLUDED.expires_at, updated_at=NOW()
            """, (agent_name, args.key, args.category, args.content,
                  args.importance, args.source, args.tags or [], expires_at))

        conn.commit()
        cur.close(); conn.close()
        print(f"✓ Wrote [{args.category}/{args.importance}] {args.key} → {agent_name}")
        if expires_at: print(f"  Expires: {expires_at[:10]}")

    except Exception as e:
        print(f"Error writing memory: {e}", file=sys.stderr); sys.exit(1)

if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)