Skill: jd2005l/opencortex (version 3.6.7)

--- name: OpenCortex homepage: https://github.com/JD2005L/opencortex description: > Self-improving memory architecture for OpenClaw agents. Structured memory files, nightly distillation, weekly synthesis, enforced principles (P0 for custom, P1-P8 managed), write-ahead durability, and model-agnostic delegation — so your agent compounds knowledge instead of forgetting it. Includes opt-in metrics tracking with growth charts and compound scoring to measure effectiveness over time. All sensitive features (voice profiling, infrastructure auto-collection, git push) are OFF by default and require explicit opt-in via environment variable or flag. Safe to install: no network calls during setup, fully auditable bash scripts, isolated cron sessions scoped to workspace only. Use when: (1) setting up a new OpenClaw instance, (2) user asks to improve/organize memory, (3) user wants the agent to stop forgetting things, (4) bootstrapping a fresh agent with best practices. NOT for: runtime memory_search queries (use built-in memory tools). Triggers: "set up memory", "organize yourself", "stop forgetting", "memory architecture", "self-improving", "cortex", "bootstrap memory", "memory optimization". metadata: {"openclaw":{"requires":{"bins":["grep","sed","find"],"optionalBins":["git","gpg","openssl","openclaw","secret-tool","keyctl","file"]},"env":{"CLAWD_WORKSPACE":{"description":"Workspace directory (defaults to cwd)","required":false},"CLAWD_TZ":{"description":"Timezone for cron scheduling (defaults to UTC)","required":false},"OPENCORTEX_VAULT_PASS":{"description":"Vault passphrase via env var. Prefer system keyring.","required":false,"sensitive":true},"OPENCORTEX_VOICE_PROFILE":{"description":"Set to 1 to enable voice profiling in nightly distillation (env override; persistent default lives in .opencortex-flags).","required":false,"sensitive":false},"OPENCORTEX_INFRA_COLLECT":{"description":"Set to 1 to enable infrastructure auto-collection in nightly distillation (env override; persistent default lives in .opencortex-flags).","required":false,"sensitive":false},"OPENCORTEX_SCRUB_ALL":{"description":"Set to 1 to scrub all tracked files (not just known text types) during git backup. Off by default.","required":false,"sensitive":false},[REDACTED_SECRET]:{"description":"Set to 1 to allow vault passphrase stored in a file (.vault/.passphrase). Off by default; prefer system keyring.","required":false,"sensitive":false}},"sensitiveFiles":[".secrets-map",".vault/.passphrase"],"networkAccess":"Optional git push only (off by default, requires --push flag)"}} --- # OpenCortex — Self-Improving Memory Architecture Transform a default OpenClaw agent into one that compounds knowledge daily. 📦 [Full source on GitHub](https://github.com/JD2005L/opencortex) — review the code, file issues, or contribute. ## What This Does 1. **Structures memory** into purpose-specific files instead of one flat dump 2. **Installs nightly maintenance** that distills daily work into permanent knowledge 3. **Installs weekly synthesis** that catches patterns across days 4. **Establishes principles** that enforce good memory habits — and backs them up with nightly audits that verify tool documentation, decision capture, sub-agent debriefs, failure analysis, and unnecessary deferrals to the user. Nothing slips through the cracks. 6. **Builds a voice profile** of your human from daily conversations for authentic ghostwriting (opt-in, requires `OPENCORTEX_VOICE_PROFILE=1` or `.opencortex-flags` with `VOICE_PROFILE=1`) 7. **Encrypts sensitive data** in an AES-256 vault with key-only references in docs; supports passphrase rotation (`vault.sh rotate`) and validates key names on `vault.sh set` 8. **Enables safe git backup** with secret scrubbing (secrets never modified in your live workspace — scrubbed in an isolated copy only) 9. **Tracks growth over time** *(opt-in)* — daily metrics snapshots with compound scoring and ASCII growth charts ## Installation **Prerequisites** (install these separately if you don't have them): - [OpenClaw](https://github.com/openclaw/openclaw) 2026.2.x+ - [ClawHub CLI](https://clawhub.com) ```bash # 1. Download the skill from your OpenClaw workspace directory cd ~/clawd # or wherever your workspace is clawhub install opencortex # 2. Run the installer FROM YOUR WORKSPACE DIRECTORY (not from inside the skill folder) bash skills/opencortex/scripts/install.sh # Optional: preview what would be created without changing anything bash skills/opencortex/scripts/install.sh --dry-run ``` The installer will ask about optional features (encrypted vault, voice profiling, infrastructure collection, git backup) and your preferred AI model for cron jobs (e.g. `sonnet`, `opus`, `haiku`). Saved to `.opencortex-model`. It's safe to re-run — it skips anything that already exists. The installer itself makes no network calls — it only creates local files and registers cron jobs. ```bash # 3. Verify everything is working (read-only — checks files and cron jobs, changes nothing) bash skills/opencortex/scripts/verify.sh ``` You can also ask your OpenClaw agent "is OpenCortex working?" — it knows how to run the verification and share results. The script will: - Create the file hierarchy (non-destructively — won't overwrite existing files) - Create directory structure - Set up cron jobs (daily distillation, weekly synthesis) - Optionally set up git backup with secret scrubbing After install, review and customize: - `SOUL.md` — personality and identity (make it yours) - `USER.md` — info about your human - `MEMORY.md` — principles (add/remove as needed) - `.secrets-map` — add your actual secrets for git scrubbing ## Updating ```bash # 1. Download the latest version (run from workspace root) clawhub install opencortex --force # 2. Re-run the installer — it detects your existing install and offers to update bash skills/opencortex/scripts/install.sh ``` The installer detects your existing version and offers three options: Update (recommended), Full reinstall, or Cancel. The update path is non-destructive — it adds missing content, refreshes cron messages, and offers any new optional features without overwriting your customized files. ## Architecture ``` SOUL.md ← Identity, personality, boundaries AGENTS.md ← Operating protocol, delegation rules MEMORY.md ← Principles + memory index (< 3KB, loaded every session) TOOLS.md ← Tool shed: APIs, scripts, and access methods with abilities descriptions INFRA.md ← Infrastructure atlas: hosts, IPs, services, network USER.md ← Human's preferences, projects, communication style BOOTSTRAP.md ← First-run checklist for new sessions memory/ projects/ ← One file per project (distilled, not raw) contacts/ ← One file per person/org (role, context, preferences) workflows/ ← One file per workflow/pipeline (services, steps, issues) runbooks/ ← Step-by-step procedures (delegatable to sub-agents) preferences.md ← Cross-cutting user preferences by category archive/ ← Archived daily logs + weekly summaries YYYY-MM-DD.md ← Today's working log (distilled nightly) ``` ## Principles (installed by default) | # | Name | Purpose | |---|------|---------| | P1 | Delegate First | Assess tasks for sub-agent delegation; stay available | | P2 | Write It Down | Commit to files, not mental notes | | P3 | Ask Before External | Confirm before emails, public posts, destructive ops | | P4 | Tool Shed & Workflows | Document tools and workflows; enforced by nightly audit | | P5 | Capture Decisions & Preferences | Record decisions and preferences; enforced by nightly + weekly audit | | P6 | Sub-agent Debrief | Delegated work feeds back to daily log; orphans recovered by distillation | | P7 | Log Failures | Tag failures/corrections; root cause analysis enforced by nightly audit | | P8 | Check the Shed First | Consult TOOLS.md/INFRA.md/memory before deferring work to user; enforced by nightly audit | ## Cron Jobs (installed) | Schedule | Name | What it does | |----------|------|-------------| | Daily 3 AM (local) | Distillation | Reads daily logs → distills into project/tools/infra files → audits tools/decisions/debriefs/failures → optimizes → archives | | Weekly Sunday 5 AM | Synthesis | Reviews week for patterns, recurring problems, unfinished threads, decisions; auto-creates runbooks from repeated procedures | Both jobs use a shared lockfile (`/tmp/opencortex-distill.lock`) to prevent conflicts when daily and weekly runs overlap. Customize times by editing cron jobs: `openclaw cron list` then `openclaw cron edit <id> --cron "..."`. ## Git Backup (optional) If enabled during install, creates: - `scripts/git-backup.sh` — auto-commit every 6 hours, scrubs secrets in an isolated temp copy (workspace files never modified) - `.secrets-map` — maps secrets to placeholders (gitignored, 600 perms) Add secrets to `.secrets-map` in format: `actual_secret|{{PLACEHOLDER_NAME}}` Before each push, `git-backup.sh` verifies no raw secrets remain in the scrubbed copy. If any are found, the backup is aborted — nothing reaches the remote. ## Customization **Adding a project:** Create `memory/projects/my-project.md`, add to MEMORY.md index. **Adding a contact:** Create `memory/contacts/name.md`. Distillation auto-creates contacts from conversations. **Adding a workflow:** Create `memory/workflows/my-pipeline.md`. Distillation auto-creates workflows when described. **Adding a preference:** Append to `memory/preferences.md` under the right category. Distillation auto-captures from conversation. **Adding a principle:** Append to MEMORY.md under 🔴 PRINCIPLES. Keep it short. **Adding a runbook:** Create `memory/runbooks/my-procedure.md`. Sub-agents can follow these directly. **Adding a tool:** Add to TOOLS.md with: what it is, how to access it, and a goal-oriented abilities description (so future intent-based lookup matches). ## How It Compounds ``` Daily work → daily log → nightly distill → routes to project/tools/infra/principles files → optimization pass (dedup, prune stale, condense) → weekly synthesis → patterns, recurring problems, unfinished threads → auto-creates runbooks from repeated procedures → `memory/runbooks/` Sub-agent work → debrief (P6) → daily log → same pipeline Decisions → captured with reasoning (P5) → never re-asked New tools → documented with abilities (P4) → findable by intent ``` Each day the agent wakes up slightly more knowledgeable and better organized.

path: README.md
content: # 🧠 OpenCortex

**Self-improving memory architecture for [OpenClaw](https://github.com/openclaw/openclaw) agents.**

Stop forgetting. Start compounding.

---

## The Problem

Out of the box, OpenClaw agents dump everything into a flat `MEMORY.md`. Context fills up, compaction loses information, and the agent forgets what it learned last week. It's like having a brilliant employee with amnesia who takes notes on napkins.

## The Solution

OpenCortex transforms your agent into one that **gets smarter every day** through:

- **Structured memory** — Purpose-specific files instead of one flat dump
- **Nightly distillation** — Daily work automatically distilled into permanent knowledge
- **Weekly synthesis** — Pattern detection across days catches recurring problems and unfinished threads
- **Enforced principles** — Habits that prevent knowledge loss (decision capture, tool documentation, sub-agent debriefs)
- **Write-ahead durability** — Agent writes decisions and preferences to memory before responding, so nothing is lost if the session ends or compacts mid-conversation
- **Encrypted vault** — AES-256 encrypted secret storage with system keyring support
- **Voice profiling** *(opt-in)* — Learns how your human communicates for authentic ghostwriting
- **Infrastructure collection** *(opt-in)* — Auto-routes infrastructure details from daily logs to INFRA.md
- **Safe git backup** *(opt-in)* — Automatic secret scrubbing in an isolated copy — workspace files are never modified

## Quick Start

**Prerequisites:** [OpenClaw](https://github.com/openclaw/openclaw) 2026.2.x+ and [ClawHub CLI](https://clawhub.com)

```bash
# From your OpenClaw workspace directory (e.g. ~/clawd)
clawhub install opencortex
bash skills/opencortex/scripts/install.sh

# Preview without changing anything:
bash skills/opencortex/scripts/install.sh --dry-run

# Verify everything is working (read-only):
bash skills/opencortex/scripts/verify.sh
```

**Important:** Run the installer from your workspace root, NOT from inside the skill folder.

The installer asks about optional features, creates files (won't overwrite existing ones), and registers cron jobs. It makes zero network calls.

After install, customize:
1. `SOUL.md` — personality and identity
2. `USER.md` — info about your human
3. `MEMORY.md` — principles and project index
4. `TOOLS.md` — tools and APIs as you discover them
5. `INFRA.md` — infrastructure reference
6. `.secrets-map` — secrets for git scrubbing (if using git backup)

### From Source

```bash
git clone https://github.com/JD2005L/opencortex.git
cd opencortex && bash scripts/install.sh
```

## Updating

```bash
clawhub install opencortex --force         # Download latest
bash skills/opencortex/scripts/install.sh  # Detects existing install, offers Update
```

The installer detects your existing version and offers: **1) Update** (recommended), **2) Full reinstall**, **3) Cancel.** It never overwrites files you've customized.

### What the updater does

| Content | Update method | User data safe? |
|---------|--------------|-----------------|
| Principles (P1-P8) | Hash comparison, asks before replacing | ✅ Asks y/N per principle |
| P0 (Custom Principles) | Never touched | ✅ Your custom principles are always safe |
| Helper scripts (verify, vault, metrics, git-backup) | Checksum comparison, auto-replaced | ✅ These aren't user-edited |
| Reference docs (distillation, weekly-synthesis, architecture) | Checksum comparison, auto-replaced | ✅ These aren't user-edited |
| Cron job messages | Always updated to latest template | ✅ Only the message text changes |
| Cron model overrides | Cleared on every update | ✅ Gateway uses its configured default |
| Cron deduplication | Detects and removes duplicate crons from prior bugs | ✅ Keeps the first, deletes extras |
| Extra principles (P9+) | Detects duplicates and orphans, offers remove/migrate to P0 | ✅ Asks per principle |
| MEMORY.md bloat | Warns if >5KB, flags non-standard sections | ✅ Suggests what to move |
| Missing cron jobs | Offers to recreate with timezone auto-detection | ✅ Asks before creating |
| MEMORY.md structure (## Identity, ## Memory Index) | Adds missing core sections | ✅ Existing sections untouched |
| MEMORY.md index (### Infrastructure through ### Daily Logs) | Adds all 8 missing sub-sections | ✅ Existing sections untouched |
| preferences.md | Created if missing | ✅ Existing file untouched |
| New directories (contacts, workflows) | Created if missing | ✅ |
| AGENTS.md | Merges: regenerates standard sections, preserves custom sections | ✅ Custom sections appended |
| BOOTSTRAP.md | Merges: regenerates standard sections, preserves custom sections | ✅ Custom sections appended |
| SOUL.md | Created if missing | ✅ Existing file untouched |
| USER.md | Created if missing | ✅ Existing file untouched |
| .gitignore | Adds missing sensitive entries (.vault/, .secrets-map, etc.) | ✅ Existing entries untouched |

---

## Architecture

```
SOUL.md          ← Identity & personality
AGENTS.md        ← Operating protocol & delegation rules
MEMORY.md        ← Principles + index (< 3KB, loaded every session)
TOOLS.md         ← Tool shed: APIs, scripts with abilities descriptions
INFRA.md         ← Infrastructure atlas: hosts, IPs, services
USER.md          ← Your human's preferences
BOOTSTRAP.md     ← Session startup checklist

memory/
  projects/      ← One file per project (distilled, not raw)
  contacts/      ← One file per person/org (role, context, preferences)
  workflows/     ← One file per workflow/pipeline (services, steps, issues)
  runbooks/      ← Step-by-step procedures (delegatable to sub-agents)
  preferences.md ← Cross-cutting user preferences by category
  archive/       ← Archived daily logs + weekly summaries
  YYYY-MM-DD.md  ← Today's working log (distilled nightly)
```

## Principles (P0–P8)

| # | Principle | What It Does | Enforcement |
|---|-----------|-------------|-------------|
| P0 | Custom Principles | Your own principles (P0-A, P0-B, etc.) | Never modified by updates |
| P1 | Delegate First | Model-agnostic sub-agent delegation (Light/Medium/Heavy) | Agent protocol |
| P2 | Write It Down | Write-ahead durability: save before responding | Agent protocol |
| P3 | Ask Before External | Confirm before public/destructive actions | Agent protocol |
| P4 | Tool Shed & Workflows | Document tools and workflows | Nightly audit scans for undocumented tools and workflows |
| P5 | Capture Decisions & Preferences | Record decisions and preferences | Nightly + weekly audit for uncaptured decisions and preferences |
| P6 | Sub-agent Debrief | Delegated work feeds back to daily log | Nightly audit recovers orphaned debriefs |
| P7 | Log Failures | Tag failures with root cause analysis | Nightly audit checks for missing root causes |
| P8 | Check the Shed First | Use documented tools before deferring to user | Nightly audit flags unnecessary deferrals |

## How It Compounds

```
Week 1:  Agent knows basics, asks lots of questions
Week 4:  Agent has project history, knows tools, follows decisions
Week 12: Agent has deep institutional knowledge, patterns, runbooks
Week 52: Agent knows more about your setup than you remember
```

---

## Security Model

### Threat Model Summary

OpenCortex is a **workspace-scoped memory skill**. It creates files, registers cron jobs that run as isolated OpenClaw agent sessions, and optionally manages an encrypted vault. The primary risk surface is:

1. **Autonomous cron jobs** that read/write workspace files without human interaction
2. **Optional features** that collect sensitive data (voice patterns, infrastructure details)
3. **Optional git backup** that handles secret scrubbing before commits

OpenCortex contains **zero network operations** — no telemetry, no phone-home, no external endpoints. Every script is plain bash. [Full source is public.](https://github.com/JD2005L/opencortex)

### Default State: What's On and Off

| Feature | Default | Opt-In Required | What It Accesses | How to Disable |
|---------|---------|----------------|-----------------|----------------|
| Structured memory files | ✅ ON | — | Creates markdown files in workspace | Delete unwanted files |
| Daily distillation cron | ✅ ON | — | Reads/writes `memory/`, `MEMORY.md`, `TOOLS.md`, `USER.md` | `openclaw cron delete <id>` |
| Weekly synthesis cron | ✅ ON | — | Reads `memory/archive/`, writes summaries + runbooks | `openclaw cron delete <id>` |
| Principle enforcement audits | ✅ ON | — | Part of distillation — audits within workspace | Remove audit sections from cron message |
| Encrypted vault | Asked at install | Choose "direct" mode to skip | `.vault/` directory, system keyring | Don't init vault; delete `.vault/` |
| Voice profiling | ❌ OFF | `OPENCORTEX_VOICE_PROFILE=1` **or** `.opencortex-flags` (`VOICE_PROFILE=1`) | Reads workspace conversation logs → `memory/VOICE.md` | Set flag/env to `0`; delete `memory/VOICE.md` |
| Infrastructure collection | ❌ OFF | `OPENCORTEX_INFRA_COLLECT=1` **or** `.opencortex-flags` (`INFRA_COLLECT=1`) | Routes infra mentions from daily logs → `INFRA.md` | Set flag/env to `0` |
| Git backup | ❌ OFF | Say "yes" at install | Commits workspace to git (local only by default) | Remove from crontab; delete scripts |
| Git push to remote | ❌ OFF | `--push` flag on each run | Pushes scrubbed commits to remote | Don't pass `--push` |
| Daily metrics tracking | ❌ OFF | Say "yes" at install | Read-only file counts → `memory/metrics.log` | Remove from crontab; delete `metrics.log` |
| Broad file scrubbing | ❌ OFF | `OPENCORTEX_SCRUB_ALL=1` | Scrubs all tracked files (not just known text types) | Unset env var |
| File-based vault passphrase | ❌ OFF | [REDACTED_SECRET] | Stores passphrase at `.vault/.passphrase` | Unset env var; use system keyring |

### What Runs Autonomously

Two cron jobs, both running as **isolated OpenClaw agent sessions** scoped to the workspace:

| Job | Schedule | Reads | Writes | Network Access |
|-----|----------|-------|--------|----------------|
| Daily Distillation | Daily 3 AM (local) | `memory/*.md`, workspace `*.md` | `memory/projects/`, `memory/contacts/`, `memory/workflows/`, `memory/preferences.md`, `MEMORY.md`, `TOOLS.md`, `USER.md`, daily log audit outputs | **None** |
| Weekly Synthesis | Sunday 5 AM (local) | `memory/archive/*.md`, `memory/projects/*.md`, `memory/contacts/*.md`, `memory/workflows/*.md`, `memory/preferences.md` | `memory/archive/weekly-*.md`, project/contact/workflow/preference files, `memory/runbooks/` | **None** |

Both jobs:
- Use a shared lockfile (`/tmp/opencortex-distill.lock`) to prevent conflicts
- Contain **no** `rm`, system modifications, network calls, or external API access
- Reference **only** workspace-relative paths (`memory/`, `MEMORY.md`, `TOOLS.md`, etc.)
- Are fully inspectable: `openclaw cron list`
- Are fully removable: `openclaw cron delete <id>`

**How cron jobs work:** OpenCortex does not bundle standalone distillation scripts. Instead, the installer registers OpenClaw cron jobs (`openclaw cron add`) with detailed task instructions. At runtime, OpenClaw spawns an isolated agent session that follows those instructions to read, synthesize, and write workspace files. The LLM agent is the executor — it's far better at knowledge synthesis than any bash script could be. The cron job messages *are* the implementation, fully viewable and editable via `openclaw cron list` / `openclaw cron edit`.

**On workspace isolation:** The cron instructions themselves don't enforce sandboxing — that's the **OpenClaw platform's** responsibility. OpenClaw cron jobs run in isolated sessions scoped to the workspace directory by the runtime, the same way a Dockerfile doesn't implement kernel isolation — the container runtime does. OpenCortex's cron instructions contain no references to external filesystems, network calls, or system commands beyond `openclaw cron list` and `crontab -l` (for self-auditing cron health).

### Git Backup Security

Git backup (when enabled) uses an **isolated copy approach** — your workspace files are never modified during scrubbing:

1. All files to commit are copied to a temp directory
2. Secrets are scrubbed in the copy only (using `.secrets-map` replacements)
3. The scrubbed copy is verified — if any raw secrets remain, the backup aborts immediately
4. A git commit is built from the scrubbed copy using git plumbing (`hash-object`, `update-index`, `write-tree`, `commit-tree`)
5. The temp directory is cleaned up
6. Your original workspace files are **untouched throughout the entire process**

Additional safeguards:
- `.secrets-map` and `.vault/` are always gitignored (enforced at install)
- Pre-backup check aborts if either exists but isn't gitignored
- Push requires explicit `--push` flag — local-only by default
- `.secrets-map` has 600 permissions (owner-only read/write)

**Recommendation:** Test in a disposable repo first. Run the backup, inspect the commit diff, and confirm scrubbing works before pointing at a real remote.

### Vault Security

The encrypted vault stores secrets at rest via GPG symmetric encryption (AES-256). Passphrase storage uses the **best available backend** (auto-detected):

| Priority | Backend | Passphrase Location | On Disk? |
|----------|---------|-------------------|----------|
| 1 | secret-tool (Linux keyring) | GNOME/KDE keyring | No |
| 2 | macOS Keychain | Native macOS keystore | No |
| 3 | keyctl (Linux kernel keyring) | Kernel memory | No |
| 4 | Environment variable | `OPENCORTEX_VAULT_PASS` | No |
| 5 | File fallback | `.vault/.passphrase` (mode 600) | Yes — requires `OPENCORTEX_ALLOW_FILE_PASSPHRASE=1` |

Commands: `vault.sh init`, `vault.sh set <key> <value>`, `vault.sh get <key>`, `vault.sh rotate`, `vault.sh migrate`, `vault.sh backend`

Key names are validated on set (alphanumeric + underscores only).

### Install Mechanism

The installer (`scripts/install.sh`) is a single bash script that:
- Creates markdown files (only if they don't already exist)
- Creates directories (`memory/projects/`, `memory/contacts/`, `memory/workflows/`, `memory/runbooks/`, `memory/archive/`)
- Registers OpenClaw cron jobs via `openclaw cron add`
- Optionally copies bundled `git-backup.sh` and `vault.sh` scripts to the workspace

**No external downloads.** No package installs. No network calls. No binaries. All code is plain bash + markdown, bundled in the skill package and fully auditable.

### Credentials

OpenCortex declares **no required API keys or environment variables**. The cron jobs use your gateway's default model — OpenCortex never sees or handles model provider keys. Any model capable of reading and writing markdown files will work.

Optional environment variables (all off by default):

| Variable | Purpose | Sensitive |
|----------|---------|-----------|
| `CLAWD_WORKSPACE` | Override workspace directory (defaults to cwd) | No |
| `CLAWD_TZ` | Timezone for cron scheduling (defaults to UTC) | No |
| `OPENCORTEX_VAULT_PASS` | Vault passphrase via env var (prefer keyring) | Yes |
| `OPENCORTEX_VOICE_PROFILE` | Enable voice profiling in distillation (override) | No |
| `OPENCORTEX_INFRA_COLLECT` | Enable infrastructure auto-collection (override) | No |

Runtime note: isolated cron jobs persist opt-in switches in `.opencortex-flags` so they work even when shell env vars are not injected into cron sessions.
| `OPENCORTEX_SCRUB_ALL` | Scrub all tracked files during git backup | No |
| `OPENCORTEX_ALLOW_FILE_PASSPHRASE` | Allow file-based vault passphrase | No |

---

## What to Review Before Installing

1. **Read the scripts.** They're bundled plain bash — `install.sh`, `update.sh`, `vault.sh`, `git-backup.sh`, `verify.sh`, `metrics.sh`. You can read every line before running anything. Required binaries: `grep`, `sed`, `find`. Optional: `git`, `gpg`, `openssl`, `openclaw`, `secret-tool`, `keyctl`, `file` (for binary detection during scrubbing).
2. **Confirm workspace isolation.** OpenCortex delegates sandbox enforcement to the OpenClaw platform. Verify your OpenClaw instance enforces workspace-only behavior for cron sessions. If isolation is misconfigured, a cron session could theoretically access files outside the workspace.
3. **Inspect cron messages after install.** Run `openclaw cron list` to see the exact instructions registered. These are the actual implementation — edit or remove them freely.
4. **Prefer system keyring for vault.** Use `secret-tool`, macOS Keychain, or `keyctl` over file-based passphrase storage. Set [REDACTED_SECRET] only if no keyring is available and you accept the risk.
5. **Test git backup in a disposable repo.** Verify `.secrets-map` entries scrub correctly before using on a real remote.
6. **Opt-in features are off by default.** Voice profiling, infrastructure collection, broad scrubbing, and git push all require explicit activation. Only enable what you need.
7. **Consider disabling voice profiling** if you're uncomfortable with the agent building a persistent behavioral profile from conversations.

---

## Metrics & Growth Tracking

If enabled during install, OpenCortex tracks your agent's knowledge growth over time. A daily system cron (11:30 PM local) snapshots file counts, decision captures, tool documentation, and more into `memory/metrics.log`. No sensitive data is collected — only counts and pattern matches.

### What's Tracked

| Metric | What It Measures |
|--------|-----------------|
| Knowledge files | Total files in `memory/projects/`, `memory/contacts/`, `memory/workflows/`, `memory/runbooks/`, and `memory/` |
| Knowledge size (KB) | Total size of knowledge files |
| Decisions captured | `**Decision:**` entries across all memory files |
| Preferences captured | `**Preference:**` entries in `memory/preferences.md` |
| Contacts | People/orgs documented in `memory/contacts/` |
| Workflows | Pipelines/automations in `memory/workflows/` |
| Runbooks | Reusable procedures in `memory/runbooks/` |
| Tools documented | Entries in `TOOLS.md` |
| Failures logged | `❌ FAILURE:` and `🔧 CORRECTION:` entries |
| Debriefs | Sub-agent debrief entries in daily logs |
| Projects | Files in `memory/projects/` |
| Archive files | Distilled daily logs in `memory/archive/` |

### Commands

```bash
# Snapshot today's metrics
bash scripts/metrics.sh --collect

# Show trends with ASCII growth charts + compound score
bash scripts/metrics.sh --report

# Last 4 weeks only
bash scripts/metrics.sh --report --weeks 4

# JSON output (for integrations)
bash scripts/metrics.sh --json
```

Or just ask your agent: *"How is OpenCortex doing?"* or *"Show me OpenCortex metrics."*

### Compound Score

The report includes a 0–100 compound score reflecting knowledge depth, growth rate, and tracking consistency:

| Score | Rating |
|-------|--------|
| 80–100 | Thriving — deep knowledge, steady growth |
| 60–79 | Growing — good foundation, building momentum |
| 40–59 | Developing — basics in place, room to grow |
| 20–39 | Getting started — early days |
| 0–19 | Just installed — give it time |

A healthy OpenCortex installation trends upward over weeks. Flat or declining scores highlight specific areas to focus on.

### Weekly Summary

If metrics tracking is enabled, the weekly synthesis cron automatically includes a metrics report in its output — showing 4-week trends and flagging areas that need attention.

### Security

The metrics script (`scripts/metrics.sh`) is **read-only** — it only counts files and greps for patterns. It writes only to `memory/metrics.log` (append-only in `--collect` mode). No network access, no sensitive data captured (counts, never content), no system modifications.

---

## Customization

**Add a project:** Create `memory/projects/my-project.md`, add to MEMORY.md index. Nightly distillation routes relevant daily log entries to it.

**Add a contact:** Create `memory/contacts/name.md` with: name, role/relationship, context, communication preferences. Distillation auto-creates contacts mentioned in conversation.

**Add a workflow:** Create `memory/workflows/my-pipeline.md` with: what it does, services involved, how to operate it. Distillation auto-creates workflows when described.

**Add a preference:** Append to `memory/preferences.md` under the right category. Format: `**Preference:** [what] — [context] (date)`. Distillation auto-captures preferences stated in conversation.

**Add a principle:** Append to MEMORY.md under 🔴 PRINCIPLES. Keep it short.

**Add a runbook:** Create `memory/runbooks/my-procedure.md` with step-by-step instructions. Sub-agents follow these directly.

**Add a tool:** Add to TOOLS.md with: what it is, how to access it, goal-oriented abilities description.

**Change cron schedule:** `openclaw cron list` then `openclaw cron edit <id> --cron "..."`.

**Run fully air-gapped:** Decline all optional features during install. No voice profiling, no infrastructure collection, no git backup. The core memory architecture and distillation work entirely offline.

## Requirements

- [OpenClaw](https://github.com/openclaw/openclaw) 2026.2.x+
- **Required:** `grep`, `sed`, `find` (standard on most systems)
- **Optional:** `git` (for backup), `gpg` (for vault), `openssl` (for passphrase generation)

## License

MIT

## Credits

Created by [JD2005L](https://github.com/JD2005L)


path: _meta.json
content: ownerId: kn7e5n3qxtp49kdnhne6vr3wzd81n3sc
slug: opencortex
version: 3.6.7
publishedAt: 1772303157357

path: references/architecture.md
content: # OpenCortex Architecture Reference

## Why This Exists

Default OpenClaw memory is a flat MEMORY.md that grows unbounded. Context fills up, compaction loses information, the agent forgets what it learned. OpenCortex solves this with:

1. **Separation of concerns** — different files for different purposes
2. **Nightly distillation** — raw daily logs → permanent structured knowledge
3. **Weekly synthesis** — pattern detection across days
4. **Principles** — enforced habits that prevent knowledge loss (P0 for custom, P1-P8 managed)
5. **Sub-agent debrief loop** — delegated work feeds back into memory

## File Purposes

| File | Loaded at boot? | Purpose | Size target |
|------|-----------------|---------|-------------|
| MEMORY.md | Yes | Principles + index only | < 3KB |
| TOOLS.md | Yes | Tool/API catalog with abilities | Grows with tools |
| INFRA.md | Yes | Infrastructure reference | Grows with infra |
| SOUL.md | Yes | Identity, personality | < 1KB |
| AGENTS.md | Yes | Operating protocol | < 1KB |
| USER.md | Yes | Human's preferences | < 1KB |
| BOOTSTRAP.md | Yes | Session startup checklist | < 0.5KB |
| memory/projects/*.md | On demand | Per-project knowledge | Any |
| memory/contacts/*.md | On demand | Per-person/org knowledge | Any |
| memory/workflows/*.md | On demand | Per-workflow/pipeline knowledge | Any |
| memory/preferences.md | On demand | Cross-cutting user preferences by category | Any |
| memory/runbooks/*.md | On demand | Procedures for sub-agents | Any |
| memory/YYYY-MM-DD.md | Current day | Working log | Any |
| memory/archive/*.md | Via search | Historical logs | Any |

## Distillation Routes

The nightly cron reads daily logs and routes each piece of information:

| Information type | Destination |
|-----------------|-------------|
| Project work, features, bugs | memory/projects/{project}.md |
| New tool descriptions and capabilities | TOOLS.md (sensitive values → vault) |
| Infrastructure changes | INFRA.md (if OPENCORTEX_INFRA_COLLECT=1) |
| People and organizations mentioned | memory/contacts/{name}.md |
| Workflows and pipelines described | memory/workflows/{name}.md |
| Stated preferences and opinions | memory/preferences.md (categorized) |
| Decisions and architectural directions | Relevant project file or MEMORY.md |
| New principles, lessons | MEMORY.md |
| User info and communication style | USER.md |
| Scheduled job changes | MEMORY.md jobs table |
| Repeatable procedures | memory/runbooks/ |

## Preference Categories

Preferences in `memory/preferences.md` are organized by category:

| Category | Examples |
|----------|---------|
| Communication | "No verbose explanations", "Direct messages only" |
| Code & Technical | "Detailed commit messages", "Prefer TypeScript" |
| Workflow & Process | "Check for messages before pushing", "Batch commits" |
| Scheduling & Time | "Don't schedule before 9 AM", "Prefer async" |
| Tools & Services | "Use VS Code over Vim", "Prefer Brave over Chrome" |
| Content & Media | "720p minimum", "No dubbed content" |
| Environment & Setup | "Dark mode everywhere", "Dual monitor layout" |

Format: `**Preference:** [what] — [context/reasoning] (date)`

Preferences are auto-captured from conversation when the user says "I prefer", "always do", "I don't like", etc. Contradicted preferences are updated (not duplicated).

## Compounding Effect

```
Week 1:  Agent knows basics, asks lots of questions
Week 4:  Agent has project history, knows tools, follows decisions, remembers preferences
Week 12: Agent has deep institutional knowledge, patterns, runbooks, contact history
Week 52: Agent knows more about the setup than most humans would remember
```

The key insight: **daily distillation + weekly synthesis + decision/preference capture** means the agent gets better at a rate proportional to how much it's used. Unlike raw log accumulation which just fills context, structured knowledge compounds.

## Common Customizations

### Adding delegation tiers
Edit MEMORY.md P1 to adjust which capability tier (Light/Medium/Heavy) handles what complexity. P1 is model-agnostic and works with whatever models you have configured.

### Changing distillation schedule
`openclaw cron edit <id> --cron "0 10 * * *" --tz "Your/Timezone"`

### Adding custom principles
All custom principles go in P0 as sub-principles (P0-A, P0-B, P0-C, etc.). P1-P8 are managed by OpenCortex updates and should not be modified directly. The agent is instructed to:
- Route all new principle requests to P0, even if the user asks for P9 or beyond
- Check for conflicts with P1-P8 before adding
- Assess whether the request is truly a principle (persistent behavioral rule) or would be better suited as a preference, decision, runbook, or agent rule

### Write-ahead durability (P2)
When the user states a preference, makes a decision, gives a deadline, or corrects the agent, the agent writes it to the relevant memory file before composing a response. This prevents context loss if the session ends or compacts mid-conversation.

### Memory health monitoring
The weekly synthesis includes automated checks that maintain memory quality over time:
- **Structural integrity audit** — verifies information is in the correct file (preferences in preferences.md, tools in TOOLS.md, etc.) and moves misplaced content.
- **Memory file reorganization** — merges duplicates, groups related info, restructures growing files while preserving all detail.
- **Retrieval quality testing** — runs test queries against memory_search and verifies results are relevant. Diagnoses failures (file too large, content misplaced, stale index), fixes what it can, escalates what it can't, and tracks gaps across weeks.
- **Stale content cleanup** — flags completed projects for archival, syncs MEMORY.md cron table against actual cron jobs.

The verify.sh script also checks memory search index health, file count, and MEMORY.md boot size.

### Multi-bot setups
Each bot gets its own OpenCortex install. Share knowledge via:
- Common git repo (read-only for non-primary bots)
- SSH-based management (primary bot propagates changes)
- Shared NFS/SMB mount for common reference docs


path: references/distillation.md
content: # Daily Memory Distillation — Instructions

You are an AI assistant. Daily memory maintenance task.

**IMPORTANT:** Before writing to any file, check for /tmp/opencortex-distill.lock. If it exists and was created less than 10 minutes ago, wait 30 seconds and retry (up to 3 times). Before starting work, create this lockfile. Remove it when done. This prevents daily and weekly jobs from conflicting.

## Part 1: Distillation

1. Check memory/ for daily log files (YYYY-MM-DD.md, not in archive/).
2. Distill ALL useful information into the right file:
   - Project work → memory/projects/ (create new files if needed)
   - New tool descriptions and capabilities → TOOLS.md (names, URLs, what they do)
   - **IMPORTANT:** Never write passwords, tokens, or secrets into any file. For sensitive values, instruct the user to run: scripts/vault.sh set <key> <value>. Reference in docs as: vault:<key>
   - Infrastructure changes → INFRA.md (ONLY if OPENCORTEX_INFRA_COLLECT=1 is set OR `.opencortex-flags` contains `INFRA_COLLECT=1` — otherwise skip infrastructure routing entirely)
   - Contacts mentioned → memory/contacts/ (one file per person/org. Include: name, role/relationship, context, communication preferences, key interactions. Create new file if first mention, update existing if already known.)
   - Workflows described → memory/workflows/ (one file per workflow/pipeline. Include: what it does, services involved, how to operate it, known issues. Create new file if first description.)
   - Preferences stated → memory/preferences.md (append under the matching category: Communication, Code & Technical, Workflow & Process, Scheduling & Time, Tools & Services, Content & Media, Environment & Setup. Format: **Preference:** [what] — [context/reasoning] (date). Do NOT duplicate existing preferences — update them if the user changes their mind.)
   - Decisions → relevant project file or MEMORY.md. Format: **Decision:** [what] — [why] (date)
   - Principles, lessons → MEMORY.md
   - Scheduled jobs → MEMORY.md jobs table
   - User info and communication style → USER.md
3. Synthesize, do not copy. Extract decisions, architecture, lessons, issues, capabilities, contacts, workflows, preferences.
4. Move distilled logs to memory/archive/
5. Update MEMORY.md index if new files created.

## Part 2: Voice Profile

ONLY perform this section if OPENCORTEX_VOICE_PROFILE=1 is set OR `.opencortex-flags` contains `VOICE_PROFILE=1`. If neither is set, skip this section entirely.

6. Read memory/VOICE.md. Review today's conversations for new patterns:
   - New vocabulary, slang, shorthand the user uses
   - How they phrase requests, decisions, reactions
   - Tone shifts in different contexts
   Append new observations to VOICE.md. Do not duplicate existing entries.

## Optimization

- Review memory/projects/ for duplicates, stale info, verbose sections. Fix directly.
- Review memory/contacts/ — merge duplicates, update stale info, add missing context.
- Review memory/workflows/ — verify accuracy, update if services or steps changed.
- Review memory/preferences.md — remove contradicted preferences (user changed mind), merge duplicates, ensure categories are correct.
- Review MEMORY.md: verify index accuracy, principles concise, jobs table current.
- Review TOOLS.md and (if OPENCORTEX_INFRA_COLLECT=1) INFRA.md: remove stale entries, verify descriptions.

## Stale Content Cleanup

- Check memory/projects/ for projects marked "Complete" more than 30 days ago with no recent daily log mentions. Flag for archival in the summary (do not delete — the user decides).
- Check MEMORY.md scheduled jobs table against actual cron jobs (openclaw cron list + crontab -l). Remove entries for crons that no longer exist. Add entries for crons not yet documented.

## Tool Shed Audit (P4 Enforcement)

- Read TOOLS.md. Scan today's daily logs for any CLI tools, APIs, or services that were USED but are NOT documented in TOOLS.md. Add missing entries with: what it is, how to access it, what it can do.
- For tools already in TOOLS.md, check if today's logs reveal gotchas, failure modes, or usage notes not yet captured. Update existing entries.

## Decision & Preference Audit (P5 Enforcement)

- Scan today's daily logs for any decisions stated by the user that are NOT captured in project files, MEMORY.md, or USER.md.
- For each uncaptured decision, write it to the appropriate file. Format: **Decision:** [what] — [why] (date)
- Scan today's daily logs for any stated preferences NOT in memory/preferences.md. Phrases like 'I prefer', 'always do', 'I don't like', 'I want', 'don't ever' signal preferences.
- For each uncaptured preference, append to memory/preferences.md under the right category. Format: **Preference:** [what] — [context/reasoning] (date). If contradicts existing, UPDATE existing.

## Contact Audit

- Scan today's daily logs for any people or organizations mentioned. For each, check if a file exists in memory/contacts/. If not and relevant, create one.
- For existing contacts, update with new information from today's logs.

## Workflow Audit

- Scan today's daily logs for any workflows, pipelines, or multi-service processes. For each, check if a file exists in memory/workflows/. If not, create one.
- For existing workflows, update if today's logs reveal changes or issues.

## Debrief Recovery (P6 Enforcement)

- Check today's daily logs for any sub-agent delegations. For each, verify a debrief entry exists. If missing, write a recovery debrief.

## Shed Deferral Audit (P8 Enforcement)

- Scan today's daily logs for instances where the agent deferred to the user. Cross-reference with TOOLS.md, INFRA.md, and memory/. Flag unnecessary deferrals.

## Failure Root Cause (P7 Enforcement)

- Scan today's daily logs for ❌ FAILURE: or 🔧 CORRECTION: entries. Verify root cause analysis exists. If missing, add it.

## Cron Health

- Run openclaw cron list and crontab -l. Verify no two jobs within 15 minutes. Fix MEMORY.md jobs table if out of sync.

---

Before completing, append debrief to memory/YYYY-MM-DD.md.
Reply with brief summary.


path: references/weekly-synthesis.md
content: # Weekly Synthesis — Instructions

You are an AI assistant. Weekly synthesis — higher-altitude review.

**IMPORTANT:** Before writing to any file, check for /tmp/opencortex-distill.lock. If it exists and was created less than 10 minutes ago, wait 30 seconds and retry (up to 3 times). Before starting work, create this lockfile. Remove it when done. This prevents daily and weekly jobs from conflicting.

1. Read archived daily logs from past 7 days (memory/archive/).
2. Read all project files (memory/projects/), contact files (memory/contacts/), workflow files (memory/workflows/), and preferences (memory/preferences.md).
3. Identify and act on:
   a. Recurring problems → add to project Known Issues
   b. Unfinished threads → add to Pending with last-touched date
   c. Cross-project connections → add cross-references
   d. Decisions this week → ensure captured with reasoning
   e. New capabilities → verify in TOOLS.md with abilities (P4)
   f. **Runbook detection** — identify any multi-step procedure (3+ steps) performed more than once this week, or likely to recur. Check if a runbook exists in memory/runbooks/. If not, create one with clear steps a sub-agent could follow. Update MEMORY.md runbooks index.
   g. **Principle health** — read MEMORY.md principles section. Verify each principle has: clear intent, enforcement mechanism, and that the enforcement is actually reflected in the distillation cron. Flag any principle without enforcement.
   h. **Contact review** — check memory/contacts/ for stale entries, missing contacts, or contacts that should be merged.
   i. **Workflow review** — check memory/workflows/ for outdated descriptions or new workflows.
   j. **Preference review** — read memory/preferences.md. Check for contradictions, stale preferences, and organization.
   k. **Memory file reorganization** — review all memory files (projects, contacts, workflows, preferences, TOOLS.md) for organization quality. For files that have grown large or disorganized: merge duplicate entries, group related information together, ensure consistent formatting, and restructure sections when it would improve accessibility. Preserve ALL detail during reorganization — this is restructuring, not summarizing. Prioritize files that have had the most additions this week.
   l. **Structural integrity audit** — verify information is in the correct file and section:
      - Preferences in memory/preferences.md, NOT scattered across project files or MEMORY.md
      - Decisions in the relevant project file, NOT in preferences.md or daily logs
      - Tool/API documentation in TOOLS.md, NOT in project files or MEMORY.md
      - Infrastructure details in INFRA.md (if it exists), NOT in TOOLS.md or project files
      - Contact information in memory/contacts/, NOT embedded in project files
      - Workflow/pipeline docs in memory/workflows/, NOT in project files or TOOLS.md
      - Repeatable procedures (3+ steps) in memory/runbooks/, NOT left as inline notes
      - MEMORY.md contains ONLY principles and the index — no project details, no tool docs, no preferences
      - AGENTS.md contains ONLY operating protocol — no project-specific rules or preferences
      - If anything is misplaced, move it to the correct location. Preserve all detail during the move.
   m. **Retrieval quality check** — test memory_search with 3-5 queries based on this week's work (project names, key decisions, people mentioned). For each query, verify the top results are actually relevant. If retrieval misses information you know exists:
      1. **Diagnose** — determine the cause: file too large (>50KB, needs splitting), information in the wrong file (structural integrity issue, move it), duplicate/scattered content (needs consolidation), or embeddings not configured/stale.
      2. **Fix automatically** — for issues within the agent's control: split oversized files into focused sub-files, move misplaced content to the correct file (per item l), consolidate scattered duplicates, update MEMORY.md index to reflect new files.
      3. **Escalate to user** — for issues requiring user action: embeddings not configured (suggest setup steps), persistent retrieval failures after restructuring (may need QMD backend or manual review).
      4. **Track** — log each retrieval gap and its resolution in the weekly summary under a "Retrieval Health" section. If the same gap appears two weeks in a row without resolution, flag it prominently to the user.
      5. **Verify** — re-test previously failed queries to confirm fixes worked. Note improvements or regressions.
4. Write weekly summary to memory/archive/weekly-YYYY-MM-DD.md.

## Runbook Detection

- Review this week's daily logs for any multi-step procedure (3+ steps) that was performed more than once, or is likely to recur.
- For each candidate: check if a runbook already exists in memory/runbooks/.
- If not, create one with clear step-by-step instructions that a sub-agent could follow independently.
- Update MEMORY.md runbooks index if new runbooks created.

## Metrics Summary (if enabled)

- If scripts/metrics.sh exists, run: bash scripts/metrics.sh --report --weeks 4
- Include the output in your weekly summary.
- If the compound score is declining or flat, note specific areas that need attention.

---

Before completing, append debrief to memory/YYYY-MM-DD.md.
Reply with weekly summary.


path: scripts/git-backup.sh
content: #!/bin/bash
# OpenCortex — Safe auto-commit workspace changes
#
# SECURITY: Secrets are scrubbed in a temp copy only.
# Workspace files are NEVER modified — no exposure window.
#
# Usage:
#   git-backup.sh          # commit locally only
#   git-backup.sh --push   # commit and push to remote
set -euo pipefail

WORKSPACE="$(cd "$(dirname "$0")/.." && pwd)"
cd "$WORKSPACE"

# Nothing to commit?
if git diff --quiet && git diff --cached --quiet && \
   [ -z "$(git ls-files --others --exclude-standard)" ]; then
  exit 0
fi

PUSH="${1:-}"
SECRETS_FILE="$WORKSPACE/.secrets-map"

# Safety: critical paths must be gitignored before any git operations
for SENSITIVE in ".vault" ".secrets-map"; do
  if [ -e "$WORKSPACE/$SENSITIVE" ]; then
    if ! git check-ignore -q "$SENSITIVE" 2>/dev/null; then
      echo "❌ ABORT: $SENSITIVE is not gitignored. Add it to .gitignore first."
      exit 1
    fi
  fi
done

# Set up temp workspace
TMPDIR_BASE=$(mktemp -d)
SCRUB_DIR="$TMPDIR_BASE/scrub"
TMPINDEX="$TMPDIR_BASE/git-index"
FILE_LIST="$TMPDIR_BASE/files.txt"
DELETED_LIST="$TMPDIR_BASE/deleted.txt"
mkdir -p "$SCRUB_DIR"

cleanup() { rm -rf "$TMPDIR_BASE"; }
trap cleanup EXIT

# Collect file lists BEFORE touching any index
{ git ls-files; git ls-files --others --exclude-standard; } | sort -u > "$FILE_LIST"
git diff --name-only --diff-filter=D HEAD 2>/dev/null > "$DELETED_LIST" || touch "$DELETED_LIST"

# Step 1: Copy all files to commit into the scrub dir
# Workspace originals are NEVER modified — only this copy is scrubbed
while IFS= read -r f; do
  [ -f "$WORKSPACE/$f" ] || continue
  mkdir -p "$SCRUB_DIR/$(dirname "$f")"
  cp "$WORKSPACE/$f" "$SCRUB_DIR/$f"
done < "$FILE_LIST"

# Step 2: Scrub secrets in the copy (only SCRUB_DIR is touched)
if [ -f "$SECRETS_FILE" ]; then
  while IFS='|' read -r secret placeholder; do
    case "$secret" in '#'*|'') continue ;; esac
    [REDACTED_SECRET] '%s' "$secret" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')"
    [ -z "$secret" ] && continue

    # Scrub all non-binary tracked files (matches verification scope)
    while IFS= read -r -d '' f; do
      file -b --mime-encoding "$f" 2>/dev/null | grep -q "binary" && continue
      grep -qF "$secret" "$f" 2>/dev/null && sed -i "s|$secret|$placeholder|g" "$f"
    done < <(find "$SCRUB_DIR" -type f -print0)
  done < "$SECRETS_FILE"

  # Step 3: Verify no secrets remain in the scrubbed copy
  LEAK_FOUND=0
  while IFS='|' read -r secret placeholder; do
    case "$secret" in '#'*|'') continue ;; esac
    [REDACTED_SECRET] '%s' "$secret" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')"
    [ -z "$secret" ] && continue
    if grep -rlF "$secret" "$SCRUB_DIR" 2>/dev/null | grep -q .; then
      echo "❌ Secret found in scrubbed copy — aborting. Your workspace files are untouched."
      LEAK_FOUND=1
      break
    fi
  done < "$SECRETS_FILE"
  [ "$LEAK_FOUND" -eq 1 ] && exit 1
fi

# Step 4: Build git commit from scrubbed copy using git plumbing
# The real git index (and working tree) are NEVER touched.
# GIT_INDEX_FILE points to a temporary index file in TMPDIR_BASE.
export GIT_INDEX_FILE="$TMPINDEX"

# Seed the temp index from the current HEAD tree
git read-tree HEAD 2>/dev/null || git read-tree --empty

# Stage each scrubbed file into the temp index as a git blob
while IFS= read -r f; do
  [ -f "$SCRUB_DIR/$f" ] || continue
  mode=$(git ls-files --stage "$f" 2>/dev/null | awk '{print $1; exit}')
  [ -z "$mode" ] && mode="100644"
  blob=$(git hash-object -w "$SCRUB_DIR/$f")
  git update-index --add --cacheinfo "$mode,$blob,$f"
done < "$FILE_LIST"

# Remove deleted tracked files from the temp index
while IFS= read -r f; do
  [ -z "$f" ] && continue
  git update-index --remove "$f" 2>/dev/null || true
done < "$DELETED_LIST"

# Write tree → commit → update branch ref
TREE=$(git write-tree)
PARENT=$(git rev-parse HEAD 2>/dev/null || echo "")
COMMIT_MSG="Auto-backup: $(date '+%Y-%m-%d %H:%M')"

if [ -n "$PARENT" ]; then
  COMMIT=$(git commit-tree -p "$PARENT" -m "$COMMIT_MSG" "$TREE")
else
  COMMIT=$(git commit-tree -m "$COMMIT_MSG" "$TREE")
fi

unset GIT_INDEX_FILE

# Update the current branch to point to the new commit
BRANCH=$(git symbolic-ref --short HEAD 2>/dev/null || echo "")
if [ -n "$BRANCH" ]; then
  git update-ref "refs/heads/$BRANCH" "$COMMIT"
else
  git update-ref HEAD "$COMMIT"
fi

echo "✅ Committed (workspace files untouched — secrets scrubbed in isolated copy)."

if [ "$PUSH" = "--push" ]; then
  git push --quiet 2>/dev/null
  echo "✅ Pushed to remote."
else
  echo "   Run with --push to push to remote. Or: git push"
fi


path: scripts/install.sh
content: #!/bin/bash
# OpenCortex — Self-Improving Memory Architecture Installer
# Safe to re-run: won't overwrite existing files.
set -euo pipefail

OPENCORTEX_VERSION="3.6.7"

# --- Version check: detect existing install and offer update ---
WORKSPACE="${CLAWD_WORKSPACE:-$(pwd)}"
VERSION_FILE="$WORKSPACE/.opencortex-version"
FLAGS_FILE="$WORKSPACE/.opencortex-flags"

set_feature_flag() {
  local key="$1"
  local value="$2"

  if [ "$DRY_RUN" = "true" ]; then
    echo "   [DRY RUN] Would set feature flag: ${key}=${value} in $FLAGS_FILE"
    return 0
  fi

  touch "$FLAGS_FILE"
  if grep -q "^${key}=" "$FLAGS_FILE" 2>/dev/null; then
    sed -i "s/^${key}=.*/${key}=${value}/" "$FLAGS_FILE"
  else
    echo "${key}=${value}" >> "$FLAGS_FILE"
  fi
  chmod 600 "$FLAGS_FILE" 2>/dev/null || true
}

# Detect existing install: version file OR core files present
INSTALLED_VERSION=""
if [ -f "$VERSION_FILE" ]; then
  INSTALLED_VERSION=$(cat "$VERSION_FILE" 2>/dev/null | tr -d '[:space:]')
elif [ -f "$WORKSPACE/MEMORY.md" ] && grep -q "PRINCIPLES" "$WORKSPACE/MEMORY.md" 2>/dev/null; then
  INSTALLED_VERSION="unknown"
fi

if [ -n "$INSTALLED_VERSION" ]; then
  if [ "$INSTALLED_VERSION" = "$OPENCORTEX_VERSION" ]; then
    echo "✅ OpenCortex v$OPENCORTEX_VERSION is already installed."
    echo ""
    echo "   Options:"
    echo "   1) Reconfigure — re-run setup to enable/change optional features"
    echo "   2) Check for updates — download latest from ClawHub and update"
    echo "   3) Verify health — run verify.sh (read-only)"
    echo "   4) Exit"
    echo ""
    read -p "   Choose (1/2/3/4) [4]: " RECONFIG_CHOICE
    RECONFIG_CHOICE="${RECONFIG_CHOICE:-4}"
    case "$RECONFIG_CHOICE" in
      1) echo ""; echo "Proceeding with reconfiguration..." ;;
      2)
        echo ""
        echo "   To check for updates, run:"
        echo "     clawhub install opencortex --force"
        echo "   Then re-run this script to apply."
        exit 0
        ;;
      3)
        SKILL_DIR="$(cd "$(dirname "$0")" && pwd)"
        if [ -f "$SKILL_DIR/verify.sh" ]; then
          bash "$SKILL_DIR/verify.sh"
        else
          echo "   verify.sh not found"
        fi
        exit 0
        ;;
      *) echo "   Nothing to do."; exit 0 ;;
    esac
  else
    if [ "$INSTALLED_VERSION" = "unknown" ]; then
      echo "🔄 OpenCortex detected (version unknown) — latest is v$OPENCORTEX_VERSION"
    else
      echo "🔄 OpenCortex update available: v$INSTALLED_VERSION → v$OPENCORTEX_VERSION"
    fi
    echo ""
    echo "   Options:"
    echo "   1) Update — applies new features without overwriting your files (recommended)"
    echo "   2) Full reinstall — re-runs the installer from scratch"
    echo "   3) Cancel"
    echo ""
    read -p "   Choose (1/2/3) [1]: " UPDATE_CHOICE
    UPDATE_CHOICE="${UPDATE_CHOICE:-1}"
    case "$UPDATE_CHOICE" in
      1)
        SKILL_DIR="$(cd "$(dirname "$0")" && pwd)"
        if [ -f "$SKILL_DIR/update.sh" ]; then
          echo ""
          bash "$SKILL_DIR/update.sh"
          echo "$OPENCORTEX_VERSION" > "$VERSION_FILE"
          echo "✅ Version updated to v$OPENCORTEX_VERSION"
          exit 0
        else
          echo "⚠️  update.sh not found — falling through to full install."
        fi
        ;;
      3) echo "Cancelled."; exit 0 ;;
      *) echo "Proceeding with full install..." ;;
    esac
  fi
fi

# --- Pre-flight: check required tools ---
REQUIRED_TOOLS=(grep sed find)
OPTIONAL_TOOLS=(openclaw git gpg)
MISSING=()
for tool in "${REQUIRED_TOOLS[@]}"; do
  command -v "$tool" &>/dev/null || MISSING+=("$tool")
done
if [ ${#MISSING[@]} -gt 0 ]; then
  echo "❌ Missing required tools: ${MISSING[*]}"
  echo "   Install them and re-run."
  exit 1
fi
for tool in "${OPTIONAL_TOOLS[@]}"; do
  command -v "$tool" &>/dev/null || echo "   ⚠️  Optional tool not found: $tool (some features will be unavailable)"
done

# --- Dry-run mode ---
DRY_RUN=false
for arg in "$@"; do
  [[ "$arg" == "--dry-run" ]] && DRY_RUN=true
done
if [ "$DRY_RUN" = "true" ]; then
  echo "⚠️  DRY RUN MODE — no files will be created or modified."
  echo ""
fi

# Detect interactive terminal
INTERACTIVE=false
if [ -t 0 ]; then
  INTERACTIVE=true
fi

# Helper: ask y/n question, loop until valid answer
# In non-interactive mode: always uses default. If no default, returns 1 (no).
ask_yn() {
  local prompt="$1"
  local default="${2:-}"
  local answer

  if [ "$INTERACTIVE" != "true" ]; then
    if [ "$default" = "y" ]; then
      echo "${prompt}y (auto — non-interactive)"
      return 0
    else
      echo "${prompt}n (auto — non-interactive)"
      return 1
    fi
  fi

  while true; do
    read -p "$prompt" answer < /dev/tty
    answer=$(echo "$answer" | tr '[:upper:]' '[:lower:]')
    case "$answer" in
      y|yes) return 0 ;;
      n|no) return 1 ;;
      "")
        if [ "$default" = "y" ]; then return 0;
        elif [ "$default" = "n" ]; then return 1;
        else echo "   Please enter y or n."; fi ;;
      *) echo "   Please enter y or n." ;;
    esac
  done
}

WORKSPACE="${CLAWD_WORKSPACE:-$(pwd)}"

# Detect timezone: env var → system → ask user
if [ -n "${CLAWD_TZ:-}" ]; then
  TZ="$CLAWD_TZ"
elif [ -f /etc/timezone ]; then
  TZ=$(cat /etc/timezone 2>/dev/null | tr -d '[:space:]')
elif [ -L /etc/localtime ]; then
  TZ=$(readlink -f /etc/localtime 2>/dev/null | sed 's|.*/zoneinfo/||')
fi
TZ="${TZ:-UTC}"

if [ "$TZ" = "UTC" ] || [ "$TZ" = "Etc/UTC" ]; then
  echo "⏰ Could not detect your local timezone."
  echo "   Cron jobs need your timezone to run at the right local time."
  echo "   Examples: America/New_York, America/Edmonton, Europe/London, Asia/Tokyo"
  read -p "   Enter your timezone (or press Enter for UTC): " USER_TZ
  USER_TZ=$(echo "$USER_TZ" | tr -d '[:space:]')
  if [ -n "$USER_TZ" ]; then
    TZ="$USER_TZ"
  fi
fi

# --- Model selection ---
MODEL_FILE="$WORKSPACE/.opencortex-model"
USER_MODEL=""
if [ -f "$MODEL_FILE" ]; then
  USER_MODEL=$(cat "$MODEL_FILE" 2>/dev/null | tr -d '[:space:]')
fi
if [ -z "$USER_MODEL" ]; then
  echo "🤖 Preferred AI model for cron jobs:"
  echo "   This is the model used for nightly distillation and weekly synthesis."
  echo "   Examples: sonnet, opus, haiku, anthropic/claude-sonnet-4-5, gpt-4o"
  read -p "   Enter model name (or press Enter for 'sonnet'): " USER_MODEL_INPUT
  USER_MODEL_INPUT=$(echo "$USER_MODEL_INPUT" | tr -d '[:space:]')
  USER_MODEL="${USER_MODEL_INPUT:-sonnet}"
  if [ "$DRY_RUN" != "true" ]; then
    echo "$USER_MODEL" > "$MODEL_FILE"
    echo "   ✅ Saved to .opencortex-model"
  fi
  echo ""
fi

echo "🧠 OpenCortex — Installing self-improving memory architecture"
echo "   Workspace: $WORKSPACE"
echo "   Timezone:  $TZ"
echo "   Model:     $USER_MODEL"
echo ""

# --- Feature Selection ---
echo "Select features:"
echo ""

echo "🔒 Secret storage mode:"
echo "   secure = Sensitive values encrypted in vault, referenced by key in docs"
echo "   direct = Agent documents everything in plain workspace files"
read -p "   Choose (secure/direct) [secure]: " SECRET_MODE
SECRET_MODE=$(echo "${SECRET_MODE:-secure}" | tr '[:upper:]' '[:lower:]')

echo ""
if ask_yn "📝 Enable voice profiling? Analyzes conversation style for ghostwriting. (y/N): " n; then
  ENABLE_VOICE="y"
else
  ENABLE_VOICE="n"
fi

echo ""
if ask_yn "🗺️  Enable infrastructure auto-collection? Cron will route infra details from daily logs to INFRA.md. (y/N): " n; then
  ENABLE_INFRA="y"
else
  ENABLE_INFRA="n"
fi

echo ""
if ask_yn "📊 Enable daily metrics tracking? Tracks knowledge growth over time (read-only, no sensitive data). (y/N): " n; then
  ENABLE_METRICS="y"
else
  ENABLE_METRICS="n"
fi

# Persist feature flags so isolated cron runs can reliably detect opt-in settings
if [ "$ENABLE_VOICE" = "y" ]; then
  set_feature_flag "VOICE_PROFILE" "1"
else
  set_feature_flag "VOICE_PROFILE" "0"
fi

if [ "$ENABLE_INFRA" = "y" ]; then
  set_feature_flag "INFRA_COLLECT" "1"
else
  set_feature_flag "INFRA_COLLECT" "0"
fi

echo ""
echo "🧠 Memory loading strategy:"
echo "   1) Eager — load core files at session start for faster responses (uses more context)"
echo "   2) Lazy  — load files only when needed, using search-first approach (saves context)"
echo ""
read -p "   Choose (1/2) [1]: " LOADING_STRATEGY
LOADING_STRATEGY="${LOADING_STRATEGY:-1}"

echo ""

# --- Directory Structure ---
echo "📁 Creating directory structure..."
if [ "$DRY_RUN" = "true" ]; then
  echo "   [DRY RUN] Would mkdir: $WORKSPACE/memory/projects"
  echo "   [DRY RUN] Would mkdir: $WORKSPACE/memory/runbooks"
  echo "   [DRY RUN] Would mkdir: $WORKSPACE/memory/contacts"
  echo "   [DRY RUN] Would mkdir: $WORKSPACE/memory/workflows"
  echo "   [DRY RUN] Would mkdir: $WORKSPACE/memory/archive"
  echo "   [DRY RUN] Would mkdir: $WORKSPACE/scripts"
else
  mkdir -p "$WORKSPACE/memory/projects"
  mkdir -p "$WORKSPACE/memory/runbooks"
  mkdir -p "$WORKSPACE/memory/contacts"
  mkdir -p "$WORKSPACE/memory/workflows"
  mkdir -p "$WORKSPACE/memory/archive"
  mkdir -p "$WORKSPACE/scripts"
fi

# --- Safety: always gitignore sensitive paths ---
if [ "$DRY_RUN" != "true" ]; then
  touch "$WORKSPACE/.gitignore"
  grep -q "^\.vault/" "$WORKSPACE/.gitignore" 2>/dev/null || echo ".vault/" >> "$WORKSPACE/.gitignore"
  grep -q "^\.secrets-map" "$WORKSPACE/.gitignore" 2>/dev/null || echo ".secrets-map" >> "$WORKSPACE/.gitignore"
  echo "   🔒 Ensured .vault/ and .secrets-map are gitignored"
else
  echo "   [DRY RUN] Would ensure .vault/ and .secrets-map in .gitignore"
fi

# --- Core Files (create only if missing) ---
create_if_missing() {
  local file="$1"
  local content="$2"
  if [ "$DRY_RUN" = "true" ]; then
    echo "   [DRY RUN] Would create: $file"
    return
  fi
  if [ ! -f "$file" ]; then
    echo "   ✅ Creating $file"
    echo "$content" > "$file"
  else
    echo "   ⏭️  $file already exists (skipped)"
  fi
}

echo "📄 Creating core files..."

create_if_missing "$WORKSPACE/SOUL.md" '# SOUL.md - Who You Are

*You'"'"'re not a chatbot. You'"'"'re becoming someone.*

## Core Truths

**Be genuinely helpful, not performatively helpful.** Skip the "Great question!" — just help.

**Have opinions.** You'"'"'re allowed to disagree, prefer things, find stuff amusing or boring.

**Be resourceful before asking.** Try to figure it out. Read the file. Check the context. Search for it. *Then* ask if you'"'"'re stuck.

**Earn trust through competence.** Be careful with external actions. Be bold with internal ones.

**Remember you'"'"'re a guest.** You have access to someone'"'"'s life. Treat it with respect.

## Boundaries
- Private things stay private. Period.
- When in doubt, ask before acting externally.
- Never send half-baked replies to messaging surfaces.

## Continuity
Each session, you wake up fresh. These files *are* your memory. Read them. Update them.

---
*This file is yours to evolve. Update it as you learn who you are.*'

if [ "$LOADING_STRATEGY" = "2" ]; then
AGENTS_BOOT="## Boot Sequence
1. Read SOUL.md — who you are
2. Read MEMORY.md — principles + memory index (always small, always current)
3. Do NOT read TOOLS.md, INFRA.md, or USER.md at boot — load them only when relevant
4. Always use memory_search before memory_get — never bulk-read a file
5. Load project/contact/workflow files only when actively working on that topic

## Context Budget
Minimize context usage. Prefer targeted reads (specific lines via memory_get) over full file reads.
When searching for information: memory_search first, read only the matching lines, then act.
Only load TOOLS.md when you need a tool. Only load INFRA.md when infrastructure is discussed."
else
AGENTS_BOOT="## Boot Sequence
1. Read SOUL.md — who you are
2. Read MEMORY.md — principles + memory index (always small, always current)
3. Use memory_search for anything deeper — do not load full files unless needed"
fi

# AGENTS.md — write with heredoc to avoid quoting issues
if [ ! -f "$WORKSPACE/AGENTS.md" ]; then
  if [ "$DRY_RUN" = "true" ]; then
    echo "   [DRY RUN] Would create: AGENTS.md"
  else
    cat > "$WORKSPACE/AGENTS.md" << AGENTSEOF
# AGENTS.md — Operating Protocol

$AGENTS_BOOT

## Principles
Live in MEMORY.md under 🔴 PRINCIPLES. Follow them always.

## Delegation (P1)
**Default action: delegate.** Before doing work, ask:
1. Can a sub-agent do this? → Yes for most things
2. What calibre? → Light (simple), Medium (moderate), Heavy (complex)
3. Delegate with clear task description + relevant file paths
4. Stay available to the user

**Sub-agent debrief (P6):** Include in every sub-agent task:
"Before completing, append a brief debrief to memory/YYYY-MM-DD.md: what you did, what you learned, any issues."

**Never delegate:** Conversation, confirmations, principle changes, ambiguous decisions

## Custom Principles (P0)
When the user asks to add a new principle, even if they ask for P9, P10, or any number beyond P8:
1. All custom principles go in P0 as sub-principles (P0-A, P0-B, P0-C, etc.)
2. Explain that P1-P8 are managed by OpenCortex and P0 is the dedicated space for custom additions
3. Before adding, assess whether it truly belongs as a principle or would be better as:
   - A **preference** (memory/preferences.md) — if it is about how the user likes things done
   - A **decision** (relevant project file) — if it is a one-time choice, not an ongoing rule
   - A **runbook** (memory/runbooks/) — if it is a step-by-step procedure
   - An **AGENTS.md rule** — if it is about agent behavior during boot or delegation
4. Check for conflicts with P1-P8. If the proposed principle would contradict an existing one, explain the conflict and work with the user to resolve it before adding
5. A principle should be a persistent behavioral rule that applies across all sessions and all work

## Write Before Responding (P2)
When the user states a preference, makes a decision, gives a deadline, or corrects you:
1. Write it to the relevant memory file FIRST
2. Then compose and send your response
This ensures nothing is lost if the session ends or compacts between your response and the write.

## Memory Structure
- MEMORY.md — Principles + index (< 3KB, fast load)
- TOOLS.md — Tool shed with abilities descriptions
- INFRA.md — Infrastructure atlas
- memory/projects/*.md — Per-project knowledge
- memory/contacts/*.md — Per-person/org knowledge
- memory/workflows/*.md — Per-workflow/pipeline knowledge
- memory/preferences.md — Cross-cutting user preferences by category
- memory/runbooks/*.md — Repeatable procedures
- memory/archive/*.md — Archived daily logs
- memory/YYYY-MM-DD.md — Today'"'"'s working log

## Health Check
When the user asks if OpenCortex is installed, working, or wants a status check, run:
  bash skills/opencortex/scripts/verify.sh
Share the results and offer to fix any failures.

## Metrics
When the user asks about OpenCortex metrics, how it is doing, or wants to see growth:
1. Run: bash scripts/metrics.sh --report
2. Share the trends, compound score, and any areas that need attention.
3. If no data exists yet, run: bash scripts/metrics.sh --collect first.

## Safety
- Never exfiltrate private data
- Ask before external actions (P3)
- Private context stays out of group chats
- When in doubt, ask — do not assume permission

## Formatting
- Keep replies concise for chat surfaces (Telegram, Discord, etc.)
- Avoid markdown tables on surfaces that do not render them well
- Match the communication style documented in USER.md

## Updates
When the user asks to update OpenCortex or check for updates:
1. Run: clawhub install opencortex --force
2. Run: bash skills/opencortex/scripts/update.sh
3. Run: bash skills/opencortex/scripts/verify.sh
Share the results with the user.
AGENTSEOF
    echo "   📝 Created AGENTS.md"
  fi
else
  echo "   ⏭️  AGENTS.md already exists"
fi

# MEMORY.md needs special handling: create if missing, OR inject principles if exists without them
if [ ! -f "$WORKSPACE/MEMORY.md" ]; then
  echo "   ✅ Creating MEMORY.md"
  if [ "$DRY_RUN" != "true" ]; then
    cat > "$WORKSPACE/MEMORY.md" << 'MEMEOF'
# MEMORY.md — Core Memory
<!-- Keep this file under 5KB. Move accumulated lessons to memory/lessons.md -->

## 🔴 PRINCIPLES (always loaded, always followed)

### P0: Custom Principles
Your custom principles go here as P0-A, P0-B, P0-C, etc. All custom principles belong in P0 regardless of how they are requested. These are never modified by OpenCortex updates.

### P1: Delegate First
Assess every task for sub-agent delegation before starting. Stay available. Assign sub-agents by complexity using whatever models are configured:
- **Light:** File ops, searches, data extraction, simple scripts, monitoring, lookups
- **Medium:** Multi-step work, code writing, debugging, research, moderate complexity
- **Heavy:** Complex reasoning, architecture decisions, sensitive or destructive operations
- **Keep main thread for:** Conversation, decisions, confirmations, quick answers

### P2: Write It Down
Do not mentally note — commit to memory files. Update indexes after significant work.
Write before responding: when a user states a preference, makes a decision, gives a deadline, or corrects you, write it to the relevant memory file before composing your response. If the session ends or compacts before you save, the context is lost. Writing first ensures durability.

### P3: Ask Before External Actions
Emails, public posts, destructive ops — get confirmation first.

### P4: Tool Shed & Workflows
All tools, APIs, access methods, and capabilities SHALL be documented in TOOLS.md with goal-oriented abilities descriptions. When given a new tool during work, immediately add it. Document workflows and pipelines in memory/workflows/.
**Enforcement:** After using any CLI tool, API, or service — before ending the task — verify it exists in TOOLS.md. If not, add it immediately.

### P5: Capture Decisions & Preferences
When the user makes a decision or states a preference, immediately record it. Decisions go in the relevant project/memory file. Preferences go in memory/preferences.md under the right category. Never re-ask something already decided or stated.
**Format:** **Decision:** [what] — [why] (date). **Preference:** [what] — [context] (date).
**Enforcement:** Before ending any conversation with substantive work, scan for uncaptured decisions and preferences. Write them before closing.

### P6: Sub-agent Debrief
Sub-agents MUST write a brief debrief to memory/YYYY-MM-DD.md before completing. Include: what was done, what was learned, any issues.
**Recovery:** If a sub-agent fails or is killed before debriefing, the parent agent writes the debrief noting the failure mode.

### P7: Log Failures
When something fails or the user corrects you, immediately append to the daily log with ❌ FAILURE: or 🔧 CORRECTION: tags. Include: what happened, why it failed, what fixed it.
**Root cause:** Log *why* it happened and what would prevent it next time. Propose a fix if systemic.

### P8: Check the Shed First
Before telling the user you cannot do something, CHECK your resources: TOOLS.md, INFRA.md, memory/projects/, runbooks. If a tool or access method exists that could accomplish the task — use it.

---

## Identity
- **Name:** (set your name)
- **Human:** (set your human)
- **Channel:** (telegram/discord/etc)

## Memory Index

### Infrastructure
- INFRA.md — Network, hosts, IPs, services
- TOOLS.md — APIs, scripts, and access methods

### Projects (memory/projects/)
| Project | Status | File |
|---------|--------|------|
| (add projects as they come) | | |

### Scheduled Jobs
(populated after cron setup below)

### Contacts (memory/contacts/)
(one file per person/org — name, role, context, preferences, history)

### Workflows (memory/workflows/)
(pipelines, automations, multi-service processes)

### Preferences (memory/preferences.md)
Cross-cutting user preferences organized by category. Updated as discovered.

### Runbooks (memory/runbooks/)
(add repeatable procedures here)

### Key Lessons
See memory/lessons.md for accumulated technical lessons.

### Daily Logs
memory/archive/YYYY-MM-DD.md — Archived daily logs
memory/YYYY-MM-DD.md — Current daily log (distilled nightly)
MEMEOF
  else
    echo "   [DRY RUN] Would create MEMORY.md"
  fi
elif ! grep -q "PRINCIPLES" "$WORKSPACE/MEMORY.md" 2>/dev/null; then
  echo "   📝 MEMORY.md exists but has no PRINCIPLES section — injecting..."
  if [ "$DRY_RUN" != "true" ]; then
    PRINCIPLES_BLOCK=$(cat << 'PREOF'

## 🔴 PRINCIPLES (always loaded, always followed)
<!-- Keep MEMORY.md under 5KB. Move accumulated lessons to memory/lessons.md -->

### P0: Custom Principles
Your custom principles go here as P0-A, P0-B, P0-C, etc. All custom principles belong in P0 regardless of how they are requested. These are never modified by OpenCortex updates.

### P1: Delegate First
Assess every task for sub-agent delegation before starting. Stay available. Assign sub-agents by complexity using whatever models are configured:
- **Light:** File ops, searches, data extraction, simple scripts, monitoring, lookups
- **Medium:** Multi-step work, code writing, debugging, research, moderate complexity
- **Heavy:** Complex reasoning, architecture decisions, sensitive or destructive operations
- **Keep main thread for:** Conversation, decisions, confirmations, quick answers

### P2: Write It Down
Do not mentally note — commit to memory files. Update indexes after significant work.
Write before responding: when a user states a preference, makes a decision, gives a deadline, or corrects you, write it to the relevant memory file before composing your response. If the session ends or compacts before you save, the context is lost. Writing first ensures durability.

### P3: Ask Before External Actions
Emails, public posts, destructive ops — get confirmation first.

### P4: Tool Shed & Workflows
All tools, APIs, access methods, and capabilities SHALL be documented in TOOLS.md with goal-oriented abilities descriptions. When given a new tool during work, immediately add it. Document workflows and pipelines in memory/workflows/.
**Enforcement:** After using any CLI tool, API, or service — before ending the task — verify it exists in TOOLS.md. If not, add it immediately.

### P5: Capture Decisions & Preferences
When the user makes a decision or states a preference, immediately record it. Decisions go in the relevant project/memory file. Preferences go in memory/preferences.md under the right category. Never re-ask something already decided or stated.
**Format:** **Decision:** [what] — [why] (date). **Preference:** [what] — [context] (date).
**Enforcement:** Before ending any conversation with substantive work, scan for uncaptured decisions and preferences. Write them before closing.

### P6: Sub-agent Debrief
Sub-agents MUST write a brief debrief to memory/YYYY-MM-DD.md before completing. Include: what was done, what was learned, any issues.
**Recovery:** If a sub-agent fails or is killed before debriefing, the parent agent writes the debrief noting the failure mode.

### P7: Log Failures
When something fails or the user corrects you, immediately append to the daily log with ❌ FAILURE: or 🔧 CORRECTION: tags. Include: what happened, why it failed, what fixed it.
**Root cause:** Log *why* it happened and what would prevent it next time. Propose a fix if systemic.

### P8: Check the Shed First
Before telling the user you cannot do something, CHECK your resources: TOOLS.md, INFRA.md, memory/projects/, runbooks. If a tool or access method exists that could accomplish the task — use it.

PREOF
)
    # Inject after the first heading line
    FIRST_LINE=$(head -1 "$WORKSPACE/MEMORY.md")
    {
      echo "$FIRST_LINE"
      echo "$PRINCIPLES_BLOCK"
      echo ""
      tail -n +2 "$WORKSPACE/MEMORY.md"
    } > "$WORKSPACE/MEMORY.md.tmp" && mv "$WORKSPACE/MEMORY.md.tmp" "$WORKSPACE/MEMORY.md"
    echo "   ✅ Principles injected into existing MEMORY.md"
  else
    echo "   [DRY RUN] Would inject PRINCIPLES into existing MEMORY.md"
  fi
else
  echo "   ⏭️  MEMORY.md already exists with PRINCIPLES (skipped)"
fi

create_if_missing "$WORKSPACE/TOOLS.md" '# TOOLS.md — Tool Shed

Document every tool, API, and script here with goal-oriented abilities descriptions (P4).

**Format:** What it is → How to access → What it can do (abilities)

---

*(Add tools as they are discovered or given during work)*'

create_if_missing "$WORKSPACE/INFRA.md" '# INFRA.md — Infrastructure Atlas

Document network, hosts, IPs, VMs/CTs, services, and storage here.

---

*(Add infrastructure details as they are discovered)*'

create_if_missing "$WORKSPACE/USER.md" '# USER.md — About My Human

- **Name:** (fill in)
- **Location:** (fill in)
- **Timezone:** (fill in)
- **Channel:** (fill in)

## Communication Style
(add preferences as learned)

## Preferences
(add as stated)'

create_if_missing "$WORKSPACE/BOOTSTRAP.md" '# BOOTSTRAP.md — First-Run Checklist

On new session start:
1. Read SOUL.md — identity and personality
2. Read MEMORY.md — principles + memory index
3. Do NOT bulk-load other files — use memory_search when needed

## Silent Replies
- NO_REPLY — when you have nothing to say (must be entire message)
- HEARTBEAT_OK — when heartbeat poll finds nothing needing attention

## Sub-Agent Protocol
When delegating, always include in task message:
"Before completing, append a brief debrief to memory/YYYY-MM-DD.md: what you did, what you learned, any issues."

## Verification
If the user asks if OpenCortex is installed, working, or wants a health check:
1. Run: bash skills/opencortex/scripts/verify.sh
2. Share the results with the user
3. If any checks fail, offer to re-run the installer: bash skills/opencortex/scripts/install.sh

## Metrics
When the user asks about metrics, growth, or how OpenCortex is doing:
1. Run: bash scripts/metrics.sh --report
2. Share the compound score and growth trends.
3. If no data yet: bash scripts/metrics.sh --collect

## Updates
When the user asks to update OpenCortex or check for updates:
1. Run: clawhub install opencortex --force
2. Run: bash skills/opencortex/scripts/update.sh
3. Run: bash skills/opencortex/scripts/verify.sh
Share the results with the user.'

create_if_missing "$WORKSPACE/memory/preferences.md" '# Preferences — What My Human Prefers

Discovered preferences, organized by category. Updated by nightly distillation when new preferences are stated in conversation. Format: **Preference:** [what] — [context/reasoning] (YYYY-MM-DD)

---

## Communication
(add as discovered)

## Code & Technical
(add as discovered)

## Workflow & Process
(add as discovered)

## Scheduling & Time
(add as discovered)

## Tools & Services
(add as discovered)

## Content & Media
(add as discovered)

## Environment & Setup
(add as discovered)'

create_if_missing "$WORKSPACE/memory/lessons.md" '# Lessons Learned

Accumulated technical lessons from all projects. Referenced from MEMORY.md.
Add new lessons here, not in MEMORY.md (to keep MEMORY.md under 5KB).

---

*(Lessons added here as they are discovered — by distillation or manually)*'

if [ "$ENABLE_VOICE" = "y" ]; then
  create_if_missing "$WORKSPACE/memory/VOICE.md" '# VOICE.md — How My Human Communicates

A living profile of communication style, vocabulary, and tone. Updated nightly by analyzing conversations. Used when ghostwriting on their behalf (community posts, emails, social media) — not for regular conversation.

---

## Tone
(observations added nightly)

## Vocabulary
(observations added nightly)

## Decision Style
(observations added nightly)

## Sentence Structure
(observations added nightly)

## What They Dislike
(observations added nightly)'
fi

# --- Vault Setup ---
if [ "$SECRET_MODE" = "secure" ]; then
  SKILL_DIR="$(cd "$(dirname "$0")" && pwd)"
  if [ -f "$SKILL_DIR/vault.sh" ]; then
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would copy: $SKILL_DIR/vault.sh → $WORKSPACE/scripts/vault.sh"
    else
      cp "$SKILL_DIR/vault.sh" "$WORKSPACE/scripts/vault.sh"
      chmod +x "$WORKSPACE/scripts/vault.sh"
      echo "   📋 Copied vault.sh"
    fi
  fi
  
  if [ "$DRY_RUN" != "true" ]; then
    "$WORKSPACE/scripts/vault.sh" init 2>/dev/null || true
  fi
  
  # Add vault to gitignore
  if [ -f "$WORKSPACE/.gitignore" ]; then
    grep -q ".vault" "$WORKSPACE/.gitignore" || echo ".vault/" >> "$WORKSPACE/.gitignore"
  else
    echo ".vault/" > "$WORKSPACE/.gitignore"
  fi
  
  echo "   ✅ Vault initialized — store secrets with: scripts/vault.sh set <key> <value>"
  echo "   📖 Reference in TOOLS.md as: [REDACTED_SECRET]"
fi

# --- Cron Jobs ---
echo ""
echo "⏰ Setting up cron jobs..."

# Check if openclaw cron is available
if command -v openclaw &>/dev/null; then
  # Daily Memory Distillation
  EXISTING=$(openclaw cron list --json 2>/dev/null | grep -c "Memory Distillation" || true)
  if [ "$EXISTING" = "0" ]; then
    # Compact cron message — full instructions live in references/distillation.md
    CRON_MSG="Daily memory maintenance. Read skills/opencortex/references/distillation.md for full instructions and follow them. Workspace: $WORKSPACE"

    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would run: openclaw cron add --name 'Daily Memory Distillation' --cron '0 3 * * *'"
    else
      openclaw cron add \
        --name "Daily Memory Distillation" \
        --cron "0 3 * * *" \
        --tz "$TZ" \
        --session "isolated" \
        --timeout-seconds 600 \
        --model "$USER_MODEL" \
        --no-deliver \
        --message "$CRON_MSG" 2>/dev/null && \
        echo "   ✅ Daily Memory Distillation cron created" && \
        echo "" && \
        echo "   📋 Cron job registered: \"Daily Memory Distillation\"" && \
        echo "      Schedule: 0 3 * * * ($TZ)" && \
        echo "      Session: isolated (workspace-only)" && \
        echo "      Review full message: openclaw cron list" && \
        echo "" || \
        echo "   ⚠️  Failed to create distillation cron"
    fi
  else
    echo "   ⏭️  Daily Memory Distillation already exists"
  fi

  # Weekly Synthesis
  EXISTING=$(openclaw cron list --json 2>/dev/null | grep -c "Weekly Synthesis" || true)
  if [ "$EXISTING" = "0" ]; then
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would run: openclaw cron add --name 'Weekly Synthesis' --cron '0 5 * * 0'"
    else
      openclaw cron add \
        --name "Weekly Synthesis" \
        --cron "0 5 * * 0" \
        --tz "$TZ" \
        --session "isolated" \
        --timeout-seconds 600 \
        --model "$USER_MODEL" \
        --no-deliver \
        --message "Weekly synthesis. Read skills/opencortex/references/weekly-synthesis.md for full instructions and follow them. Workspace: $WORKSPACE" 2>/dev/null && \
        echo "   ✅ Weekly Synthesis cron created" && \
        echo "" && \
        echo "   📋 Cron job registered: \"Weekly Synthesis\"" && \
        echo "      Schedule: 0 5 * * 0 ($TZ)" && \
        echo "      Session: isolated (workspace-only)" && \
        echo "      Review full message: openclaw cron list" && \
        echo "" || \
        echo "   ⚠️  Failed to create synthesis cron"
    fi
  else
    echo "   ⏭️  Weekly Synthesis already exists"
  fi
else
  echo "   ⚠️  openclaw command not found — skipping cron setup"
  echo "   Run 'openclaw cron add' manually after install"
fi

# --- Git Backup (optional) ---
echo ""
if ask_yn "📦 Set up git backup with secret scrubbing? (y/N): " n; then

  # Copy bundled scripts (fully inspectable in the skill package)
  SKILL_DIR="$(cd "$(dirname "$0")" && pwd)"
  for script in git-backup.sh; do
    if [ -f "$SKILL_DIR/$script" ]; then
      if [ "$DRY_RUN" = "true" ]; then
        echo "   [DRY RUN] Would copy: $SKILL_DIR/$script → $WORKSPACE/scripts/$script"
      else
        cp "$SKILL_DIR/$script" "$WORKSPACE/scripts/$script"
        chmod +x "$WORKSPACE/scripts/$script"
        echo "   📋 Copied $script"
      fi
    fi
  done

  create_if_missing "$WORKSPACE/.secrets-map" '# Secrets map: SECRET_VALUE|{{PLACEHOLDER}}
# Add your secrets here. This file is gitignored.
# Example: mysecretpassword123|{{MY_PASSWORD}}'

  [ "$DRY_RUN" != "true" ] && chmod 600 "$WORKSPACE/.secrets-map"

  # Add to gitignore
  if [ -f "$WORKSPACE/.gitignore" ]; then
    grep -q "secrets-map" "$WORKSPACE/.gitignore" || echo ".secrets-map" >> "$WORKSPACE/.gitignore"
  else
    echo ".secrets-map" > "$WORKSPACE/.gitignore"
  fi

  # Add cron
  if ! crontab -l 2>/dev/null | grep -q "git-backup"; then
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would add crontab entry: 0 */6 * * * $WORKSPACE/scripts/git-backup.sh"
    else
      (crontab -l 2>/dev/null; echo "0 */6 * * * $WORKSPACE/scripts/git-backup.sh") | crontab -
      echo "   ✅ Git backup cron added (every 6 hours)"
      echo ""
      echo "   📋 Cron job registered: \"Git Backup\""
      echo "      Schedule: 0 */6 * * * (system cron)"
      echo "      Session: direct script (no network unless --push flag set)"
      echo "      Review: crontab -l"
      echo ""
    fi
  else
    echo "   ⏭️  Git backup cron already exists"
  fi

  echo "   ✅ Git backup configured — edit .secrets-map to add your secrets"
else
  echo "   Skipped git backup setup"
fi

# --- Metrics (optional) ---
if [ "$ENABLE_METRICS" = "y" ]; then
  echo ""
  echo "📊 Setting up metrics tracking..."

  # Copy metrics script
  SKILL_DIR="$(cd "$(dirname "$0")" && pwd)"
  if [ -f "$SKILL_DIR/metrics.sh" ]; then
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would copy: $SKILL_DIR/metrics.sh → $WORKSPACE/scripts/metrics.sh"
    else
      cp "$SKILL_DIR/metrics.sh" "$WORKSPACE/scripts/metrics.sh"
      chmod +x "$WORKSPACE/scripts/metrics.sh"
      echo "   📋 Copied metrics.sh"
    fi
  fi

  # Add system cron (daily at 11:30 PM local — end of day snapshot)
  if ! crontab -l 2>/dev/null | grep -q "metrics.sh"; then
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would add crontab entry: 30 23 * * * $WORKSPACE/scripts/metrics.sh --collect"
    else
      (crontab -l 2>/dev/null; echo "30 23 * * * $WORKSPACE/scripts/metrics.sh --collect") | crontab -
      echo "   ✅ Daily metrics cron added (11:30 PM local)"
      echo ""
      echo "   📋 Cron job registered: \"Daily Metrics Collection\""
      echo "      Schedule: 30 23 * * * (system cron)"
      echo "      Action: read-only file count → memory/metrics.log"
      echo "      Review: crontab -l"
      echo ""
    fi
  else
    echo "   ⏭️  Metrics cron already exists"
  fi

  # Run first collection
  if [ "$DRY_RUN" != "true" ] && [ -x "$WORKSPACE/scripts/metrics.sh" ]; then
    "$WORKSPACE/scripts/metrics.sh" --collect
    echo "   📊 First snapshot captured. View with: bash scripts/metrics.sh --report"
  fi
else
  echo ""
  echo "   Skipped metrics tracking (enable later: copy scripts/metrics.sh and add to crontab)"
fi

# --- Write version marker ---
if [ "$DRY_RUN" != "true" ]; then
  echo "$OPENCORTEX_VERSION" > "$VERSION_FILE"
fi

# --- Done ---
echo ""
echo "🧠 OpenCortex v$OPENCORTEX_VERSION installed successfully!"
echo ""
echo "Next steps:"
echo "  1. Edit SOUL.md — make it yours"
echo "  2. Edit USER.md — describe your human"
echo "  3. Edit MEMORY.md — set identity, add projects as you go"
echo "  4. Edit TOOLS.md — document tools as you discover them"
echo "  5. Edit INFRA.md — document your infrastructure"
if [ "$SECRET_MODE" = "secure" ]; then
  echo "  6. If using git backup: edit .secrets-map with your actual secrets"
fi

# --- Env var guidance for opt-in features ---
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🔑 Opt-in feature environment variables:"
echo ""
if [ "$ENABLE_VOICE" = "y" ]; then
  echo "   Voice profiling is enabled in the cron (you said yes)."
  echo "   Runtime detection uses .opencortex-flags (VOICE_PROFILE=1) by default."
  echo "   Optional override: export OPENCORTEX_VOICE_PROFILE=1"
  echo ""
else
  echo "   Voice profiling: OFF (not requested)"
  echo "   To enable later: set VOICE_PROFILE=1 in .opencortex-flags"
  echo "   Optional override: export OPENCORTEX_VOICE_PROFILE=1"
  echo ""
fi
if [ "$ENABLE_INFRA" = "y" ]; then
  echo "   Infrastructure auto-collection is enabled in the cron (you said yes)."
  echo "   Runtime detection uses .opencortex-flags (INFRA_COLLECT=1) by default."
  echo "   Optional override: export OPENCORTEX_INFRA_COLLECT=1"
  echo ""
else
  echo "   Infrastructure auto-collection: OFF (not requested)"
  echo "   To enable later: set INFRA_COLLECT=1 in .opencortex-flags"
  echo "   Optional override: export OPENCORTEX_INFRA_COLLECT=1"
  echo ""
fi
echo "   All other optional features (git push, scrub-all, file passphrase):"
echo "   See README.md → Opt-In Features for details."
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"

echo ""
echo "The system will self-improve from here. Work normally — the nightly"
echo "distillation will organize everything you learn into permanent memory."

if [ "$DRY_RUN" = "true" ]; then
  echo ""
  echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
  echo "  Dry run complete. No files were created."
  echo "  Re-run without --dry-run to install."
  echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
fi


path: scripts/metrics.sh
content: #!/bin/bash
# OpenCortex — Metrics Collection & Reporting
#
# READ-ONLY: This script only counts files and greps for patterns.
# It does not modify any workspace files (except appending to metrics.log in --collect mode).
# No network access. No sensitive data captured — only counts, never content.
#
# Usage:
#   metrics.sh --collect          # Append today's snapshot to memory/metrics.log
#   metrics.sh --report           # Show trends from collected data
#   metrics.sh --report --weeks 8 # Show last 8 weeks (default: 12)
#   metrics.sh --json             # Output today's snapshot as JSON (for programmatic use)
#   metrics.sh --help             # Show this help
set -euo pipefail

WORKSPACE="${CLAWD_WORKSPACE:-$(cd "$(dirname "$0")/.." && pwd)}"
METRICS_LOG="$WORKSPACE/memory/metrics.log"
TODAY=$(date '+%Y-%m-%d')

# --- Helpers ---

# Trim whitespace from a number
trimnum() { echo "${1:-0}" | tr -dc '0-9'; }

# Safe grep -c wrapper (returns 0 on no match)
grepcount() { grep -c "$1" "$2" 2>/dev/null | tr -d ' ' || echo 0; }

count_files() {
  local dir="$1"
  local pattern="${2:-*.md}"
  local c
  c=$([ -d "$dir" ] && find "$dir" -name "$pattern" -type f 2>/dev/null | wc -l | tr -d ' ' || echo 0)
  echo "${c:-0}"
}

count_bytes() {
  local dir="$1"
  local pattern="${2:-*.md}"
  local c
  c=$([ -d "$dir" ] && find "$dir" -name "$pattern" -type f -exec cat {} + 2>/dev/null | wc -c | tr -d ' ' || echo 0)
  echo "${c:-0}"
}

count_grep() {
  local dir="$1"
  local pattern="$2"
  local c
  c=$([ -d "$dir" ] && grep -rl "$pattern" "$dir" --include="*.md" 2>/dev/null | wc -l | tr -d ' ' || echo 0)
  echo "${c:-0}"
}

count_grep_lines() {
  local dir="$1"
  local pattern="$2"
  if [ ! -d "$dir" ]; then echo 0; return; fi
  local c
  c=$(grep -r "$pattern" "$dir" --include="*.md" 2>/dev/null | wc -l | tr -d ' ')
  echo "${c:-0}"
}

# --- Snapshot: collect all metrics into variables ---

# Sets global vars: m_knowledge_files m_knowledge_kb m_decisions m_runbooks
# m_tools m_failures m_debriefs m_projects m_archive m_principles m_core_files
# m_cron_jobs m_infra_entries
snapshot_metrics() {
  m_knowledge_files=$(( $(count_files "$WORKSPACE/memory/projects") + $(count_files "$WORKSPACE/memory/contacts") + $(count_files "$WORKSPACE/memory/workflows") + $(count_files "$WORKSPACE/memory/runbooks") ))
  if [ -d "$WORKSPACE/memory" ]; then
    m_knowledge_files=$(( m_knowledge_files + $(trimnum "$(find "$WORKSPACE/memory" -maxdepth 1 -name "*.md" -type f 2>/dev/null | wc -l)") ))
  fi

  local kb=0
  for kdir in projects contacts workflows runbooks; do
    if [ -d "$WORKSPACE/memory/$kdir" ]; then
      kb=$(( kb + $(trimnum "$(find "$WORKSPACE/memory/$kdir" -name "*.md" -type f -exec cat {} + 2>/dev/null | wc -c)") ))
    fi
  done
  kb=$(( kb + $(trimnum "$(find "$WORKSPACE/memory" -maxdepth 1 -name "*.md" -type f -exec cat {} + 2>/dev/null | wc -c)") ))
  m_knowledge_kb=$(( kb / 1024 ))

  m_decisions=$(count_grep_lines "$WORKSPACE/memory" "\*\*Decision:\*\*")
  if [ -f "$WORKSPACE/MEMORY.md" ]; then
    m_decisions=$(( m_decisions + $(trimnum "$(grepcount '\*\*Decision:\*\*' "$WORKSPACE/MEMORY.md")") ))
  fi
  if [ -f "$WORKSPACE/USER.md" ]; then
    m_decisions=$(( m_decisions + $(trimnum "$(grepcount '\*\*Decision:\*\*' "$WORKSPACE/USER.md")") ))
  fi

  m_runbooks=$(count_files "$WORKSPACE/memory/runbooks")

  m_tools=0
  if [ -f "$WORKSPACE/TOOLS.md" ]; then
    m_tools=$(trimnum "$(grep -cE '^#{2,3} ' "$WORKSPACE/TOOLS.md" 2>/dev/null || echo 0)")
  fi

  local fail=$(count_grep_lines "$WORKSPACE/memory" "❌ FAILURE:")
  local corr=$(count_grep_lines "$WORKSPACE/memory" "🔧 CORRECTION:")
  m_failures=$(( fail + corr ))

  m_debriefs=$(count_grep_lines "$WORKSPACE/memory" "[Dd]ebrief")
  m_projects=$(count_files "$WORKSPACE/memory/projects")
  m_contacts=$(count_files "$WORKSPACE/memory/contacts")
  m_workflows=$(count_files "$WORKSPACE/memory/workflows")
  m_preferences=0
  if [ -f "$WORKSPACE/memory/preferences.md" ]; then
    m_preferences=$(trimnum "$(grep -c '\*\*Preference:\*\*' "$WORKSPACE/memory/preferences.md" 2>/dev/null || echo 0)")
  fi
  m_archive=$(count_files "$WORKSPACE/memory/archive")

  m_principles=0
  if [ -f "$WORKSPACE/MEMORY.md" ]; then
    m_principles=$(trimnum "$(grep -cE '^### P[0-9]' "$WORKSPACE/MEMORY.md" 2>/dev/null || echo 0)")
  fi

  m_core_files=0
  for f in SOUL.md AGENTS.md MEMORY.md TOOLS.md INFRA.md USER.md BOOTSTRAP.md; do
    if [ -f "$WORKSPACE/$f" ]; then
      m_core_files=$(( m_core_files + 1 ))
    fi
  done

  m_cron_jobs=0
  if command -v openclaw &>/dev/null; then
    m_cron_jobs=$(trimnum "$(openclaw cron list --json 2>/dev/null | grep -c '"name"' || echo 0)")
  fi

  m_infra_entries=0
  if [ -f "$WORKSPACE/INFRA.md" ]; then
    m_infra_entries=$(trimnum "$(grep -cE '^#{2,3} ' "$WORKSPACE/INFRA.md" 2>/dev/null || echo 0)")
  fi
}

# --- Collect ---

do_collect() {
  mkdir -p "$WORKSPACE/memory"
  snapshot_metrics

  cat >> "$METRICS_LOG" <<EOF
---
date: $TODAY
knowledge_files: $m_knowledge_files
knowledge_kb: $m_knowledge_kb
decisions: $m_decisions
runbooks: $m_runbooks
tools: $m_tools
failures_logged: $m_failures
debriefs: $m_debriefs
projects: $m_projects
contacts: $m_contacts
workflows: $m_workflows
preferences: $m_preferences
archive_files: $m_archive
principles: $m_principles
core_files: $m_core_files
cron_jobs: $m_cron_jobs
infra_entries: $m_infra_entries
EOF

  echo "✅ Metrics collected for $TODAY → memory/metrics.log"
}

# --- JSON output ---

do_json() {
  mkdir -p "$WORKSPACE/memory"
  snapshot_metrics

  cat <<EOF
{
  "date": "$TODAY",
  "knowledge_files": $m_knowledge_files,
  "knowledge_kb": $m_knowledge_kb,
  "decisions": $m_decisions,
  "runbooks": $m_runbooks,
  "tools": $m_tools,
  "failures_logged": $m_failures,
  "debriefs": $m_debriefs,
  "projects": $m_projects,
  "contacts": $m_contacts,
  "workflows": $m_workflows,
  "preferences": $m_preferences,
  "archive_files": $m_archive,
  "principles": $m_principles
}
EOF
}

# --- Report ---

do_report() {
  local weeks="${1:-12}"

  if [ ! -f "$METRICS_LOG" ]; then
    echo "No metrics data yet. Run: metrics.sh --collect"
    exit 0
  fi

  # Parse all entries from the log
  local dates=()
  local kf=() kb=() dec=() rb=() tools=() fail=() deb=() proj=() cont=() wflow=() pref=() arch=()
  local cur_date="" cur_kf="" cur_kb="" cur_dec="" cur_rb="" cur_tools="" cur_fail="" cur_deb="" cur_proj="" cur_cont="" cur_wflow="" cur_pref="" cur_arch=""

  while IFS= read -r line; do
    case "$line" in
      "---") 
        if [ -n "$cur_date" ]; then
          dates+=("$cur_date")
          kf+=("${cur_kf:-0}"); kb+=("${cur_kb:-0}"); dec+=("${cur_dec:-0}")
          rb+=("${cur_rb:-0}"); tools+=("${cur_tools:-0}"); fail+=("${cur_fail:-0}")
          deb+=("${cur_deb:-0}"); proj+=("${cur_proj:-0}"); cont+=("${cur_cont:-0}")
          wflow+=("${cur_wflow:-0}"); pref+=("${cur_pref:-0}"); arch+=("${cur_arch:-0}")
        fi
        cur_date="" cur_kf="" cur_kb="" cur_dec="" cur_rb="" cur_tools="" cur_fail="" cur_deb="" cur_proj="" cur_cont="" cur_wflow="" cur_pref="" cur_arch=""
        ;;
      date:*) cur_date="${line#date: }" ;;
      knowledge_files:*) cur_kf="${line#knowledge_files: }" ;;
      knowledge_kb:*) cur_kb="${line#knowledge_kb: }" ;;
      decisions:*) cur_dec="${line#decisions: }" ;;
      runbooks:*) cur_rb="${line#runbooks: }" ;;
      tools:*) cur_tools="${line#tools: }" ;;
      failures_logged:*) cur_fail="${line#failures_logged: }" ;;
      debriefs:*) cur_deb="${line#debriefs: }" ;;
      projects:*) cur_proj="${line#projects: }" ;;
      contacts:*) cur_cont="${line#contacts: }" ;;
      workflows:*) cur_wflow="${line#workflows: }" ;;
      preferences:*) cur_pref="${line#preferences: }" ;;
      archive_files:*) cur_arch="${line#archive_files: }" ;;
    esac
  done < "$METRICS_LOG"
  # Flush last entry
  if [ -n "$cur_date" ]; then
    dates+=("$cur_date")
    kf+=("${cur_kf:-0}"); kb+=("${cur_kb:-0}"); dec+=("${cur_dec:-0}")
    rb+=("${cur_rb:-0}"); tools+=("${cur_tools:-0}"); fail+=("${cur_fail:-0}")
    deb+=("${cur_deb:-0}"); proj+=("${cur_proj:-0}"); cont+=("${cur_cont:-0}")
    wflow+=("${cur_wflow:-0}"); pref+=("${cur_pref:-0}"); arch+=("${cur_arch:-0}")
  fi

  local total=${#dates[@]}
  if [ "$total" -eq 0 ]; then
    echo "No metrics entries found."
    exit 0
  fi

  # Limit to last N days (weeks * 7)
  local limit=$(( weeks * 7 ))
  local start=0
  [ "$total" -gt "$limit" ] && start=$(( total - limit ))

  echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
  echo "  🧠 OpenCortex Metrics Report"
  echo "  Period: ${dates[$start]} → ${dates[$((total-1))]}"
  echo "  Data points: $(( total - start ))"
  echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
  echo ""

  # Summary: first vs latest
  local first=$start
  local last=$(( total - 1 ))

  printf "  %-24s %8s %8s %8s\n" "Metric" "Start" "Current" "Change"
  printf "  %-24s %8s %8s %8s\n" "────────────────────────" "────────" "────────" "────────"

  print_row() {
    local label="$1" v1="$2" v2="$3"
    local diff=$(( v2 - v1 ))
    local sign=""
    [ "$diff" -gt 0 ] && sign="+"
    printf "  %-24s %8d %8d %7s%d\n" "$label" "$v1" "$v2" "$sign" "$diff"
  }

  print_row "Knowledge files" "${kf[$first]}" "${kf[$last]}"
  print_row "Knowledge size (KB)" "${kb[$first]}" "${kb[$last]}"
  print_row "Decisions captured" "${dec[$first]}" "${dec[$last]}"
  print_row "Runbooks" "${rb[$first]}" "${rb[$last]}"
  print_row "Tools documented" "${tools[$first]}" "${tools[$last]}"
  print_row "Failures logged" "${fail[$first]}" "${fail[$last]}"
  print_row "Debriefs" "${deb[$first]}" "${deb[$last]}"
  print_row "Projects" "${proj[$first]}" "${proj[$last]}"
  print_row "Contacts" "${cont[$first]}" "${cont[$last]}"
  print_row "Workflows" "${wflow[$first]}" "${wflow[$last]}"
  print_row "Preferences" "${pref[$first]}" "${pref[$last]}"
  print_row "Archive files" "${arch[$first]}" "${arch[$last]}"

  echo ""

  # ASCII sparkline for knowledge KB
  echo "  📈 Knowledge Growth (KB):"
  draw_chart "kb" "$start" "$total"
  echo ""

  echo "  📈 Decisions Captured:"
  draw_chart "dec" "$start" "$total"
  echo ""

  echo "  📈 Tools Documented:"
  draw_chart "tools" "$start" "$total"
  echo ""

  # Health score
  local score=0
  local max_score=100

  # Score components (each out of ~12-13 points)
  [ "${kf[$last]}" -ge 5 ] && score=$(( score + 10 ))
  [ "${kf[$last]}" -ge 15 ] && score=$(( score + 5 ))
  [ "${dec[$last]}" -ge 5 ] && score=$(( score + 10 ))
  [ "${dec[$last]}" -ge 20 ] && score=$(( score + 5 ))
  [ "${rb[$last]}" -ge 1 ] && score=$(( score + 5 ))
  [ "${rb[$last]}" -ge 3 ] && score=$(( score + 5 ))
  [ "${tools[$last]}" -ge 3 ] && score=$(( score + 8 ))
  [ "${tools[$last]}" -ge 10 ] && score=$(( score + 4 ))
  [ "${proj[$last]}" -ge 2 ] && score=$(( score + 8 ))
  [ "${cont[$last]}" -ge 1 ] && score=$(( score + 3 ))
  [ "${cont[$last]}" -ge 5 ] && score=$(( score + 3 ))
  [ "${wflow[$last]}" -ge 1 ] && score=$(( score + 3 ))
  [ "${pref[$last]}" -ge 3 ] && score=$(( score + 4 ))
  [ "${pref[$last]}" -ge 10 ] && score=$(( score + 3 ))
  [ "${arch[$last]}" -ge 7 ] && score=$(( score + 7 ))
  [ "${arch[$last]}" -ge 30 ] && score=$(( score + 4 ))

  # Growth bonus (if any metric grew)
  local grew=0
  [ "${kf[$last]}" -gt "${kf[$first]}" ] && grew=1
  [ "${dec[$last]}" -gt "${dec[$first]}" ] && grew=1
  [ "${tools[$last]}" -gt "${tools[$first]}" ] && grew=1
  [ "${pref[$last]}" -gt "${pref[$first]}" ] && grew=1
  [ "$grew" -eq 1 ] && score=$(( score + 7 ))

  # Consistency bonus (more than 7 data points = at least a week of tracking)
  [ "$(( total - start ))" -ge 7 ] && score=$(( score + 7 ))

  [ "$score" -gt "$max_score" ] && score=$max_score

  echo "  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
  echo ""
  printf "  🏆 Compound Score: %d/%d" "$score" "$max_score"
  if [ "$score" -ge 80 ]; then
    echo "  — Thriving"
  elif [ "$score" -ge 60 ]; then
    echo "  — Growing"
  elif [ "$score" -ge 40 ]; then
    echo "  — Developing"
  elif [ "$score" -ge 20 ]; then
    echo "  — Getting started"
  else
    echo "  — Just installed"
  fi
  echo ""
  echo "  Score reflects knowledge depth, growth rate, and tracking consistency."
  echo "  A healthy OpenCortex installation trends upward over weeks."
  echo ""
}

draw_chart() {
  local var_name="$1"
  local start="$2"
  local total="$3"
  local -n arr="$var_name"

  # Find min/max
  local min=999999 max=0
  for (( i=start; i<total; i++ )); do
    local v="${arr[$i]}"
    [ "$v" -lt "$min" ] && min=$v
    [ "$v" -gt "$max" ] && max=$v
  done

  local range=$(( max - min ))
  [ "$range" -eq 0 ] && range=1

  local bar_chars="▁▂▃▄▅▆▇█"
  local chart=""

  # Sample up to 40 points for display width
  local count=$(( total - start ))
  local step=1
  [ "$count" -gt 40 ] && step=$(( count / 40 ))

  for (( i=start; i<total; i+=step )); do
    local v="${arr[$i]}"
    local idx=$(( (v - min) * 7 / range ))
    [ "$idx" -gt 7 ] && idx=7
    [ "$idx" -lt 0 ] && idx=0
    chart="${chart}${bar_chars:$idx:1}"
  done

  echo "     ${arr[$start]} ${chart} ${arr[$((total-1))]}"
}

# --- Main ---

case "${1:---help}" in
  --collect) do_collect ;;
  --report)  do_report "${3:-12}" ;;
  --json)    do_json ;;
  --help)
    echo "OpenCortex Metrics — track your agent's knowledge growth over time"
    echo ""
    echo "Usage:"
    echo "  metrics.sh --collect            Snapshot today's metrics → memory/metrics.log"
    echo "  metrics.sh --report             Show trends (default: last 12 weeks)"
    echo "  metrics.sh --report --weeks 8   Show last 8 weeks"
    echo "  metrics.sh --json               Today's snapshot as JSON"
    echo ""
    echo "This script is READ-ONLY (except appending to metrics.log in --collect mode)."
    echo "No network access. No sensitive data — only file counts and pattern matches."
    ;;
  *) echo "Unknown option: $1 (try --help)"; exit 1 ;;
esac


path: scripts/update.sh
content: #!/bin/bash
# OpenCortex — Non-destructive update script
# Adds missing content to your workspace. Never overwrites files you've customized.
# Cron job messages are updated to the latest templates.
# Run from your OpenClaw workspace directory: bash skills/opencortex/scripts/update.sh

set -euo pipefail

OPENCORTEX_VERSION="3.6.7"

# ─────────────────────────────────────────────────────────────────────────────
# Flags
# ─────────────────────────────────────────────────────────────────────────────
DRY_RUN=false
for arg in "$@"; do
  [[ "$arg" == "--dry-run" ]] && DRY_RUN=true
done

if [ "$DRY_RUN" = "true" ]; then
  echo "⚠️  DRY RUN MODE — nothing will be changed."
  echo ""
fi

# Detect interactive terminal
INTERACTIVE=false
if [ -t 0 ]; then
  INTERACTIVE=true
fi

# Helper: ask y/n question, loop until valid answer
# Usage: ask_yn "prompt (y/N): " [default]
# default: y or n (what empty input means). No default = must answer.
# In non-interactive mode: always uses default. If no default, returns 1 (no).
ask_yn() {
  local prompt="$1"
  local default="${2:-}"
  local answer

  if [ "$INTERACTIVE" != "true" ]; then
    if [ "$default" = "y" ]; then
      echo "${prompt}y (auto — non-interactive)"
      return 0
    else
      echo "${prompt}n (auto — non-interactive)"
      return 1
    fi
  fi

  while true; do
    read -p "$prompt" answer < /dev/tty
    answer=$(echo "$answer" | tr '[:upper:]' '[:lower:]')
    case "$answer" in
      y|yes) return 0 ;;
      n|no) return 1 ;;
      "") 
        if [ "$default" = "y" ]; then return 0;
        elif [ "$default" = "n" ]; then return 1;
        else echo "   Please enter y or n."; fi ;;
      *) echo "   Please enter y or n." ;;
    esac
  done
}

# Helper: ask m/r/k question for non-interactive mode
# Always returns 'k' (keep) when non-interactive
ask_mrk() {
  local prompt="$1"
  local answer
  if [ "$INTERACTIVE" != "true" ]; then
    echo "${prompt}k (auto — non-interactive)"
    echo "k"
    return
  fi
  while true; do
    read -p "$prompt" answer < /dev/tty
    answer=$(echo "$answer" | tr '[:upper:]' '[:lower:]')
    case "$answer" in
      m|r|k) echo "$answer"; return ;;
      *) echo "   Please enter m, r, or k." >&2 ;;
    esac
  done
}

WORKSPACE="${CLAWD_WORKSPACE:-$(pwd)}"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
FLAGS_FILE="$WORKSPACE/.opencortex-flags"

set_feature_flag() {
  local key="$1"
  local value="$2"

  if [ "$DRY_RUN" = "true" ]; then
    echo "   [DRY RUN] Would set ${key}=${value} in $FLAGS_FILE"
    return 0
  fi

  touch "$FLAGS_FILE"
  if grep -q "^${key}=" "$FLAGS_FILE" 2>/dev/null; then
    sed -i "s/^${key}=.*/${key}=${value}/" "$FLAGS_FILE"
  else
    echo "${key}=${value}" >> "$FLAGS_FILE"
  fi
  chmod 600 "$FLAGS_FILE" 2>/dev/null || true
}

echo "🔄 OpenCortex Update v${OPENCORTEX_VERSION}"
echo "   Workspace: $WORKSPACE"
echo "   Script:    $SCRIPT_DIR"
echo ""

UPDATED=0
SKIPPED=0

# ─────────────────────────────────────────────────────────────────────────────
# Part 1: Cron job messages — update to latest templates
# ─────────────────────────────────────────────────────────────────────────────
echo "⏰ Checking cron job messages..."


WORKSPACE="${CLAWD_WORKSPACE:-$(pwd)}"
DAILY_MSG="Daily memory maintenance. Read skills/opencortex/references/distillation.md for full instructions and follow them. Workspace: $WORKSPACE"
WEEKLY_MSG="Weekly synthesis. Read skills/opencortex/references/weekly-synthesis.md for full instructions and follow them. Workspace: $WORKSPACE"


if command -v openclaw &>/dev/null; then
  # Get JSON cron list and extract IDs
  CRON_JSON=$(openclaw cron list --json 2>/dev/null || echo "[]")

  get_cron_id() {
    local name="$1"
    local name_lower
    name_lower=$(echo "$name" | tr '[:upper:]' '[:lower:]')
    # Parse JSON with grep/awk — look for "name" field matching, then grab preceding "id" field
    echo "$CRON_JSON" | tr ',' '\n' | tr '{' '\n' | tr '}' '\n' | sed 's/^ *//' | awk -v search="$name_lower" '
      /^"id"/ || /^"_id"/ || /^"uuid"/ { gsub(/[" ]/, ""); split($0, a, ":"); last_id=a[2] }
      /^"name"/ { gsub(/["]/, ""); sub(/^name: */, ""); n=tolower($0); if (index(n, search) > 0 && last_id != "") { print last_id; exit } }
    ' 2>/dev/null || true
  }

  # Helper: extract current message for a cron ID from JSON
  get_cron_msg() {
    local cron_id="$1"
    echo "$CRON_JSON" | tr '\n' ' ' | grep -oP "\"id\":\\s*\"${cron_id}\"[^}]*\"message\":\\s*\"[^\"]*\"" | grep -oP '"message":\s*"\K[^"]*' || true
  }

  DAILY_ID=$(get_cron_id "Daily Memory Distillation")
  WEEKLY_ID=$(get_cron_id "Weekly Synthesis")

  if [ -n "$DAILY_ID" ]; then
    CURRENT_DAILY_MSG=$(get_cron_msg "$DAILY_ID")
    # Check if model override exists (anything other than "default" or empty)
    DAILY_MODEL=$(echo "$CRON_JSON" | tr '\n' ' ' | grep -oP "\"id\":\\s*\"${DAILY_ID}\"[^}]*\"model\":\\s*\"[^\"]*\"" | grep -oP '"model":\s*"\K[^"]*' || true)
    DAILY_HAS_MODEL=false
    if [ -n "$DAILY_MODEL" ] && [ "$DAILY_MODEL" != "default" ]; then
      DAILY_HAS_MODEL=true
    fi
    if [ "$CURRENT_DAILY_MSG" = "$DAILY_MSG" ] && [ "$DAILY_HAS_MODEL" = false ]; then
      echo "   ⏭️  'Daily Memory Distillation' already current"
      SKIPPED=$((SKIPPED + 1))
    elif [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would update 'Daily Memory Distillation' (id: $DAILY_ID) message"
      UPDATED=$((UPDATED + 1))
    else
      _SAVED_MODEL=""
      [ -f "$WORKSPACE/.opencortex-model" ] && _SAVED_MODEL=$(cat "$WORKSPACE/.opencortex-model" 2>/dev/null | tr -d '[:space:]')
      _SAVED_MODEL="${_SAVED_MODEL:-sonnet}"
      openclaw cron edit "$DAILY_ID" --message "$DAILY_MSG" --model "$_SAVED_MODEL" 2>/dev/null \
        && echo "   ✅ Updated 'Daily Memory Distillation' cron (message + model: $_SAVED_MODEL)" \
        && UPDATED=$((UPDATED + 1)) \
        || echo "   ⚠️  Could not update 'Daily Memory Distillation' — run manually: openclaw cron edit $DAILY_ID --message '...'"
    fi
  else
    echo "   ⏭️  'Daily Memory Distillation' cron not found — run install.sh to create it"
    SKIPPED=$((SKIPPED + 1))
  fi

  if [ -n "$WEEKLY_ID" ]; then
    CURRENT_WEEKLY_MSG=$(get_cron_msg "$WEEKLY_ID")
    WEEKLY_MODEL=$(echo "$CRON_JSON" | tr '\n' ' ' | grep -oP "\"id\":\\s*\"${WEEKLY_ID}\"[^}]*\"model\":\\s*\"[^\"]*\"" | grep -oP '"model":\s*"\K[^"]*' || true)
    WEEKLY_HAS_MODEL=false
    if [ -n "$WEEKLY_MODEL" ] && [ "$WEEKLY_MODEL" != "default" ]; then
      WEEKLY_HAS_MODEL=true
    fi
    if [ "$CURRENT_WEEKLY_MSG" = "$WEEKLY_MSG" ] && [ "$WEEKLY_HAS_MODEL" = false ]; then
      echo "   ⏭️  'Weekly Synthesis' already current"
      SKIPPED=$((SKIPPED + 1))
    elif [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would update 'Weekly Synthesis' (id: $WEEKLY_ID) message"
      UPDATED=$((UPDATED + 1))
    else
      _SAVED_MODEL=""
      [ -f "$WORKSPACE/.opencortex-model" ] && _SAVED_MODEL=$(cat "$WORKSPACE/.opencortex-model" 2>/dev/null | tr -d '[:space:]')
      _SAVED_MODEL="${_SAVED_MODEL:-sonnet}"
      openclaw cron edit "$WEEKLY_ID" --message "$WEEKLY_MSG" --model "$_SAVED_MODEL" 2>/dev/null \
        && echo "   ✅ Updated 'Weekly Synthesis' cron (message + model: $_SAVED_MODEL)" \
        && UPDATED=$((UPDATED + 1)) \
        || echo "   ⚠️  Could not update 'Weekly Synthesis' — run manually: openclaw cron edit $WEEKLY_ID --message '...'"
    fi
  else
    echo "   ⏭️  'Weekly Synthesis' cron not found — run install.sh to create it"
    SKIPPED=$((SKIPPED + 1))
  fi
else
  echo "   ⚠️  openclaw CLI not found — skipping cron updates"
  SKIPPED=$((SKIPPED + 1))
fi
echo ""

# ─────────────────────────────────────────────────────────────────────────────
# Part 1.5: Feature flags file for isolated cron runtime gates
# ─────────────────────────────────────────────────────────────────────────────
echo "🏷️  Ensuring feature flags are persisted..."

# Voice profiling: infer enabled if VOICE.md exists
if [ -f "$WORKSPACE/memory/VOICE.md" ]; then
  set_feature_flag "VOICE_PROFILE" "1"
  echo "   ✅ VOICE_PROFILE=1 (memory/VOICE.md exists)"
  UPDATED=$((UPDATED + 1))
else
  set_feature_flag "VOICE_PROFILE" "0"
  echo "   ⏭️  VOICE_PROFILE=0 (memory/VOICE.md not found)"
  SKIPPED=$((SKIPPED + 1))
fi

# Infrastructure collection: preserve existing value if present, else infer from env, else default 0
INFRA_CURRENT=""
if [ -f "$FLAGS_FILE" ]; then
  INFRA_CURRENT=$(grep -E '^INFRA_COLLECT=' "$FLAGS_FILE" 2>/dev/null | head -1 | cut -d'=' -f2 | tr -d '[:space:]' || true)
fi
if [ -n "$INFRA_CURRENT" ]; then
  set_feature_flag "INFRA_COLLECT" "$INFRA_CURRENT"
  echo "   ⏭️  INFRA_COLLECT=$INFRA_CURRENT (kept existing value)"
  SKIPPED=$((SKIPPED + 1))
elif [ "${OPENCORTEX_INFRA_COLLECT:-}" = "1" ]; then
  set_feature_flag "INFRA_COLLECT" "1"
  echo "   ✅ INFRA_COLLECT=1 (from environment)"
  UPDATED=$((UPDATED + 1))
else
  set_feature_flag "INFRA_COLLECT" "0"
  echo "   ⏭️  INFRA_COLLECT=0 (default)"
  SKIPPED=$((SKIPPED + 1))
fi
echo ""

# ─────────────────────────────────────────────────────────────────────────────
# Part 2: Principles — add any missing P1–P8 to MEMORY.md
# ─────────────────────────────────────────────────────────────────────────────
echo "📜 Checking principles in MEMORY.md..."

if [ ! -f "$WORKSPACE/MEMORY.md" ]; then
  echo "   ⚠️  MEMORY.md not found — skipping principles check (run install.sh first)"
  SKIPPED=$((SKIPPED + 1))
else
  # Build associative array: principle number → full block text
  declare -A PRINCIPLE_TEXTS

  PRINCIPLE_TEXTS["P1"]=$(cat <<'EOPR'
### P1: Delegate First
Assess every task for sub-agent delegation before starting. Stay available. Assign sub-agents by complexity using whatever models are configured:
- **Light:** File ops, searches, data extraction, simple scripts, monitoring, lookups
- **Medium:** Multi-step work, code writing, debugging, research, moderate complexity
- **Heavy:** Complex reasoning, architecture decisions, sensitive or destructive operations
- **Keep main thread for:** Conversation, decisions, confirmations, quick answers
EOPR
)

  PRINCIPLE_TEXTS["P2"]=$(cat <<'EOPR'
### P2: Write It Down
Do not mentally note — commit to memory files. Update indexes after significant work.
Write before responding: when a user states a preference, makes a decision, gives a deadline, or corrects you, write it to the relevant memory file before composing your response. If the session ends or compacts before you save, the context is lost. Writing first ensures durability.
EOPR
)

  PRINCIPLE_TEXTS["P3"]=$(cat <<'EOPR'
### P3: Ask Before External Actions
Emails, public posts, destructive ops — get confirmation first.
EOPR
)

  PRINCIPLE_TEXTS["P4"]=$(cat <<'EOPR'
### P4: Tool Shed & Workflows
All tools, APIs, access methods, and capabilities SHALL be documented in TOOLS.md with goal-oriented abilities descriptions. When given a new tool during work, immediately add it. Document workflows and pipelines in memory/workflows/ with clear descriptions of what they do, how they connect, and how to operate them.
**Creation:** When you access a new system, API, or resource more than once — or when given access to something that will clearly recur — proactively create the tool entry, bridge doc, or helper script. When a multi-service workflow is described or used, document it in memory/workflows/. Do not wait to be asked.
**Enforcement:** After using any CLI tool, API, or service — before ending the task — verify it exists in TOOLS.md. If not, add it immediately. Do not defer to distillation.
EOPR
)

  PRINCIPLE_TEXTS["P5"]=$(cat <<'EOPR'
### P5: Capture Decisions & Preferences
When the user makes a decision or states a preference, immediately record it. Decisions go in the relevant project/memory file. Preferences go in memory/preferences.md under the right category. Never re-ask something already decided or stated.
**Decisions format:** **Decision:** [what] — [why] (date) — in the relevant project or memory file.
**Preferences format:** **Preference:** [what] — [context/reasoning] (date) — in memory/preferences.md under the matching category (Communication, Code & Technical, Workflow & Process, Scheduling & Time, Tools & Services, Content & Media, Environment & Setup).
**Recognition:** Decisions include: explicit choices, architectural directions, and workflow rules. Preferences include: stated likes/dislikes, communication style preferences, tool preferences, formatting preferences, and any opinion that would affect future work. If the user says "I prefer X" or "always do Y" or "I don't like Z" — that is a preference. Capture it immediately.
**Enforcement:** Before ending any conversation with substantive work, scan for uncaptured decisions AND preferences. If any, write them before closing.
EOPR
)

  PRINCIPLE_TEXTS["P6"]=$(cat <<'EOPR'
### P6: Sub-agent Debrief
Sub-agents MUST write a brief debrief to memory/YYYY-MM-DD.md before completing. Include: what was done, what was learned, any issues.
**Recovery:** If a sub-agent fails, times out, or is killed before debriefing, the parent agent writes the debrief on its behalf noting the failure mode. No delegated work should vanish from memory.
EOPR
)

  PRINCIPLE_TEXTS["P7"]=$(cat <<'EOPR'
### P7: Log Failures
When something fails or the user corrects you, immediately append to the daily log with ❌ FAILURE: or 🔧 CORRECTION: tags. Include: what happened, why it failed, what fixed it. Nightly distillation routes these to the right file.
**Root cause:** Do not just log what happened — log *why* it happened and what would prevent it next time. If it is a systemic issue (missing principle, bad assumption, tool gap), propose a fix immediately.
EOPR
)

  PRINCIPLE_TEXTS["P8"]=$(cat <<'EOPR'
### P8: Check the Shed First
Before telling the user you cannot do something, or asking them to do it manually, CHECK your resources: TOOLS.md, INFRA.md, memory/projects/, runbooks, and any bridge docs. If a tool, API, credential, or access method exists that could accomplish the task — use it. The shed exists so you do not make the user do work you are equipped to handle.
**Enforcement:** Nightly audit scans for instances where the agent deferred work to the user that could have been done via documented tools.
EOPR
)

  # Ensure P0 exists (needed for migration from older versions)
  if ! grep -q "^### P0:" "$WORKSPACE/MEMORY.md" 2>/dev/null; then
    echo "   ℹ️  Adding P0 (Custom Principles) section for your own additions..."
    if [ "$DRY_RUN" != "true" ]; then
      # Insert P0 right after the PRINCIPLES header
      p_header=$(grep -n "^## .*PRINCIPLES" "$WORKSPACE/MEMORY.md" | head -1 | cut -d: -f1)
      if [ -n "$p_header" ]; then
        p0_text="\n### P0: Custom Principles\nYour custom principles go here as P0-A, P0-B, P0-C, etc. All custom principles belong in P0 regardless of how they are requested. These are never modified by OpenCortex updates.\n"
        sed -i "${p_header}a\\${p0_text}" "$WORKSPACE/MEMORY.md"
        echo "   ✅ Added P0 section"
        UPDATED=$((UPDATED + 1))
      fi
    fi
  fi

  # Collect missing or outdated principles
  MISSING_PRINCIPLES=()
  OUTDATED_PRINCIPLES=()
  for pnum in P1 P2 P3 P4 P5 P6 P7 P8; do
    if grep -q "^### ${pnum}:" "$WORKSPACE/MEMORY.md" 2>/dev/null; then
      # Check if the principle title matches the latest version
      current_title=""
      current_title=$(grep "^### ${pnum}:" "$WORKSPACE/MEMORY.md" | head -1)
      expected_title=""
      expected_title=$(echo "${PRINCIPLE_TEXTS[$pnum]}" | head -1)
      if [ "$current_title" != "$expected_title" ]; then
        echo "   🔄 ${pnum} title changed — will update"
        OUTDATED_PRINCIPLES+=("$pnum")
      else
        # Title matches — check if body content has changed
        # Extract current principle block including header (from ### Px: to line before next ### P or ---)
        current_body=$(awk "/^### ${pnum}:/{found=1} found{if(/^### P[0-9]/ && !/^### ${pnum}:/)exit; if(/^---$/)exit; if(/^## /)exit; print}" "$WORKSPACE/MEMORY.md" 2>/dev/null)
        expected_body="${PRINCIPLE_TEXTS[$pnum]}"
        current_hash=$(printf '%s' "$current_body" | tr -d '[:space:]' | md5sum | cut -d' ' -f1)
        expected_hash=$(printf '%s' "$expected_body" | tr -d '[:space:]' | md5sum | cut -d' ' -f1)
        if [ "$current_hash" != "$expected_hash" ]; then
          echo "   🔄 ${pnum} content changed — will update"
          OUTDATED_PRINCIPLES+=("$pnum")
        else
          echo "   ⏭️  ${pnum} already exists (skipped)"
          SKIPPED=$((SKIPPED + 1))
        fi
      fi
    else
      echo "   ⚠️  ${pnum} missing — will add"
      MISSING_PRINCIPLES+=("$pnum")
    fi
  done

  # Replace outdated principles in-place
  if [ ${#OUTDATED_PRINCIPLES[@]} -gt 0 ]; then
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would update principles: ${OUTDATED_PRINCIPLES[*]}"
    else
      for pnum in "${OUTDATED_PRINCIPLES[@]}"; do
        # Show what changed
        current_title=$(grep "^### ${pnum}:" "$WORKSPACE/MEMORY.md" | head -1)
        expected_title=$(echo "${PRINCIPLE_TEXTS[$pnum]}" | head -1)
        echo ""
        # Extract current principle block for display (including header)
        current_full=$(awk "/^### ${pnum}:/{found=1} found{if(/^### P[0-9]/ && !/^### ${pnum}:/)exit; if(/^---$/)exit; if(/^## /)exit; print}" "$WORKSPACE/MEMORY.md" 2>/dev/null)
        expected_full="${PRINCIPLE_TEXTS[$pnum]}"

        if [ "$current_title" != "$expected_title" ]; then
          echo "   ${pnum} title changed:"
          echo "     Current: $current_title"
          echo "     New:     $expected_title"
        else
          echo "   ${pnum} content updated (title unchanged)"
        fi
        echo ""
        echo "   ┌─ Current ──────────────────────────────"
        echo "$current_full" | sed 's/^/   │ /'
        echo "   └─────────────────────────────────────────"
        echo ""
        echo "   ┌─ New ─────────────────────────────────"
        echo "$expected_full" | sed 's/^/   │ /'
        echo "   └─────────────────────────────────────────"
        echo ""
        # Detect custom additions: lines in current that aren't in the expected version
        custom_lines=""
        if [ -n "$current_full" ] && [ -n "$expected_full" ]; then
          custom_lines=$(diff <(echo "$expected_full") <(echo "$current_full") 2>/dev/null | grep "^> " | sed 's/^> //' | sed '/^$/d' || true)
        fi

        if [ -n "$custom_lines" ]; then
          echo "   📋 Custom content detected beyond the standard ${pnum}:"
          echo "$custom_lines" | sed 's/^/      /'
          echo ""
          if ask_yn "   Migrate custom content to P0 before updating? (Y/n): " y; then
            # Find or create P0 section
            if grep -q "^### P0:" "$WORKSPACE/MEMORY.md" 2>/dev/null; then
              # Count existing P0 sub-principles to determine next letter
              existing_count=$(grep -c "^#### P0-" "$WORKSPACE/MEMORY.md" 2>/dev/null || true)
              existing_count=$(printf '%s' "$existing_count" | tr -dc '0-9')
              existing_count=${existing_count:-0}
              next_letter=$(printf "\\$(printf '%03o' $((65 + existing_count)))")
              # Insert after the P0 description line
              p0_line=$(grep -n "^### P0:" "$WORKSPACE/MEMORY.md" | head -1 | cut -d: -f1)
              # Find the next ### heading after P0
              p0_end=$(tail -n "+$((p0_line + 1))" "$WORKSPACE/MEMORY.md" | grep -n "^### P[0-9]" | head -1 | cut -d: -f1)
              if [ -n "$p0_end" ]; then
                insert_at=$((p0_line + p0_end - 1))
              else
                insert_at=$((p0_line + 1))
              fi
              # Build the sub-principle
              # Use temp file approach instead of sed to avoid special character issues
              tmp_insert=$(mktemp)
              head -n "$insert_at" "$WORKSPACE/MEMORY.md" > "$tmp_insert"
              echo "" >> "$tmp_insert"
              echo "#### P0-${next_letter}: Custom from ${pnum}" >> "$tmp_insert"
              echo "$custom_lines" >> "$tmp_insert"
              echo "" >> "$tmp_insert"
              tail -n "+$((insert_at + 1))" "$WORKSPACE/MEMORY.md" >> "$tmp_insert"
              mv "$tmp_insert" "$WORKSPACE/MEMORY.md"
              echo "   ✅ Migrated to P0-${next_letter}"
            else
              echo "   ⚠️  P0 section not found — custom content preserved in current ${pnum}"
              echo "      Skipping update to avoid data loss."
              SKIPPED=$((SKIPPED + 1))
              continue
            fi
          fi
        fi

        echo "   ⚠️  Replacing will overwrite any custom additions you made to this principle."
        if ask_yn "   Update ${pnum}? (y/N): " n; then
          # Find the start and end line of the existing principle block
          start_line=""; end_line=""; next_section=""
          start_line=$(grep -n "^### ${pnum}:" "$WORKSPACE/MEMORY.md" | head -1 | cut -d: -f1)
          # Find the next ### or ## heading after start_line
          next_section=$(tail -n +"$((start_line + 1))" "$WORKSPACE/MEMORY.md" | grep -n "^###\|^## " | head -1 | cut -d: -f1)
          if [ -n "$next_section" ]; then
            end_line=$((start_line + next_section - 1))
          else
            end_line=$(wc -l < "$WORKSPACE/MEMORY.md")
          fi
          # Replace the block
          tmp_mem=$(mktemp)
          head -n "$((start_line - 1))" "$WORKSPACE/MEMORY.md" > "$tmp_mem"
          echo "${PRINCIPLE_TEXTS[$pnum]}" >> "$tmp_mem"
          echo "" >> "$tmp_mem"
          tail -n "+$((end_line + 1))" "$WORKSPACE/MEMORY.md" >> "$tmp_mem"
          mv "$tmp_mem" "$WORKSPACE/MEMORY.md"
          echo "   ✅ Updated ${pnum}"
          UPDATED=$((UPDATED + 1))
        else
          echo "   ⏭️  Kept existing ${pnum}"
          SKIPPED=$((SKIPPED + 1))
        fi
      done
    fi
  fi

  if [ ${#MISSING_PRINCIPLES[@]} -gt 0 ]; then
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would add missing principles: ${MISSING_PRINCIPLES[*]}"
      UPDATED=$((UPDATED + ${#MISSING_PRINCIPLES[@]}))
    else
      # Write all missing principles to a temp file
      TEMP_P=$(mktemp)
      for pnum in "${MISSING_PRINCIPLES[@]}"; do
        printf '\n%s\n' "${PRINCIPLE_TEXTS[$pnum]}" >> "$TEMP_P"
      done

      # Insert before "## Identity" if it exists, otherwise append
      if grep -q "^## Identity" "$WORKSPACE/MEMORY.md"; then
        # Insert new principles before ## Identity line
        sed -i "/^## Identity/e cat $TEMP_P" "$WORKSPACE/MEMORY.md"
        # Add blank line before ## Identity if missing
        sed -i '/^## Identity/{x;/./{x;b};x;s/^/\n/}' "$WORKSPACE/MEMORY.md" 2>/dev/null || true
      elif grep -q "^---$" "$WORKSPACE/MEMORY.md"; then
        # Insert before first --- divider
        sed -i "0,/^---$/{ /^---$/e cat $TEMP_P
        }" "$WORKSPACE/MEMORY.md"
      else
        # Append to end
        cat "$TEMP_P" >> "$WORKSPACE/MEMORY.md"
      fi

      rm -f "$TEMP_P"
      for pnum in "${MISSING_PRINCIPLES[@]}"; do
        echo "   ✅ Added principle ${pnum}"
        UPDATED=$((UPDATED + 1))
      done
    fi
  fi
fi

# Clean up extra/duplicate principles beyond P0-P8
if [ -f "$WORKSPACE/MEMORY.md" ]; then
  echo ""
  echo "🧹 Checking for extra principles..."
  EXTRA_PRINCIPLES=$(grep "^### P[0-9]" "$WORKSPACE/MEMORY.md" | grep -v "^### P0:\|^### P1:\|^### P2:\|^### P3:\|^### P4:\|^### P5:\|^### P6:\|^### P7:\|^### P8:" || true)

  if [ -n "$EXTRA_PRINCIPLES" ]; then
    echo "   Found principles outside P0-P8 range:"
    echo "$EXTRA_PRINCIPLES" | sed 's/^/      /'
    echo ""

    while IFS= read -r extra_line; do
      [ -z "$extra_line" ] && continue
      extra_pnum=$(echo "$extra_line" | sed 's/^### \(P[0-9]*\):.*/\1/')
      extra_title="$extra_line"

      # Extract the full body of this extra principle
      extra_body=$(awk "/^### ${extra_pnum}:/{found=1} found{if(/^### P[0-9]/ && !/^### ${extra_pnum}:/)exit; if(/^---$/)exit; if(/^## /)exit; print}" "$WORKSPACE/MEMORY.md" 2>/dev/null)

      # Check if this is a duplicate of an existing P1-P8
      # Must match BOTH title AND body content to be considered a duplicate
      is_duplicate=false
      dup_of=""
      for std_pnum in P1 P2 P3 P4 P5 P6 P7 P8; do
        std_title=$(grep "^### ${std_pnum}:" "$WORKSPACE/MEMORY.md" 2>/dev/null | head -1)
        if [ -n "$std_title" ]; then
          extra_name=$(echo "$extra_title" | sed 's/^### P[0-9]*: //')
          std_name=$(echo "$std_title" | sed 's/^### P[0-9]*: //')
          if [ "$extra_name" = "$std_name" ]; then
            # Title matches — now compare body content
            std_body=$(awk "/^### ${std_pnum}:/{found=1} found{if(/^### P[0-9]/ && !/^### ${std_pnum}:/)exit; if(/^---$/)exit; if(/^## /)exit; print}" "$WORKSPACE/MEMORY.md" 2>/dev/null)
            # Compare using hash of whitespace-stripped content
            extra_hash=$(echo "$extra_body" | tr -d '[:space:]' | md5sum | cut -d' ' -f1)
            std_hash=$(echo "$std_body" | tr -d '[:space:]' | md5sum | cut -d' ' -f1)
            if [ "$extra_hash" = "$std_hash" ]; then
              is_duplicate=true
              dup_of="$std_pnum"
              echo "   🔄 $extra_pnum is a full duplicate of $std_pnum ($std_name) — same title AND body"
              break
            else
              echo "   ℹ️  $extra_pnum has same title as $std_pnum ($std_name) but DIFFERENT body content"
            fi
          fi
        fi
      done

      if [ "$is_duplicate" = true ]; then
        if ask_yn "   Remove duplicate $extra_pnum? (y/N): " n; then
          if [ "$DRY_RUN" != "true" ]; then
            start_line=$(grep -n "^### ${extra_pnum}:" "$WORKSPACE/MEMORY.md" | head -1 | cut -d: -f1)
            next_line=$(tail -n "+$((start_line + 1))" "$WORKSPACE/MEMORY.md" | grep -n "^### \|^## " | head -1 | cut -d: -f1)
            if [ -n "$next_line" ]; then
              end_line=$((start_line + next_line - 1))
            else
              end_line=$(wc -l < "$WORKSPACE/MEMORY.md")
            fi
            tmp_mem=$(mktemp)
            head -n "$((start_line - 1))" "$WORKSPACE/MEMORY.md" > "$tmp_mem"
            tail -n "+$((end_line + 1))" "$WORKSPACE/MEMORY.md" >> "$tmp_mem"
            mv "$tmp_mem" "$WORKSPACE/MEMORY.md"
            echo "   ✅ Removed duplicate $extra_pnum"
            UPDATED=$((UPDATED + 1))
          fi
        fi
      else
        echo "   📋 $extra_pnum has unique content:"
        echo "$extra_body" | head -5 | sed 's/^/      /'
        echo ""
        echo "   What should we do with $extra_pnum?"
        echo "     m = Move to P0 as custom sub-principle"
        echo "     r = Remove entirely"
        echo "     k = Keep as-is (no change)"
        ACTION=$(ask_mrk "   Action for $extra_pnum? (m/r/k): ")
        case "$ACTION" in
            m)
              if [ "$DRY_RUN" != "true" ]; then
                # Count existing P0 sub-principles
                existing_count=$(grep -c "^#### P0-" "$WORKSPACE/MEMORY.md" 2>/dev/null || true)
                existing_count=$(printf '%s' "$existing_count" | tr -dc '0-9')
                existing_count=${existing_count:-0}
                next_letter=$(printf "\\$(printf '%03o' $((65 + existing_count)))")

                # Add to P0
                p0_line=$(grep -n "^### P0:" "$WORKSPACE/MEMORY.md" | head -1 | cut -d: -f1)
                p0_end=$(tail -n "+$((p0_line + 1))" "$WORKSPACE/MEMORY.md" | grep -n "^### P[0-9]" | head -1 | cut -d: -f1)
                if [ -n "$p0_end" ]; then
                  insert_at=$((p0_line + p0_end - 1))
                else
                  insert_at=$((p0_line + 1))
                fi
                # Build sub-principle from body (skip the original ### header)
                sub_body=$(echo "$extra_body" | tail -n +2)
                sub_name=$(echo "$extra_title" | sed 's/^### P[0-9]*: //')
                # Use temp file approach to avoid special character issues with sed
                tmp_insert=$(mktemp)
                head -n "$insert_at" "$WORKSPACE/MEMORY.md" > "$tmp_insert"
                echo "" >> "$tmp_insert"
                echo "#### P0-${next_letter}: ${sub_name}" >> "$tmp_insert"
                echo "$sub_body" >> "$tmp_insert"
                echo "" >> "$tmp_insert"
                tail -n "+$((insert_at + 1))" "$WORKSPACE/MEMORY.md" >> "$tmp_insert"
                mv "$tmp_insert" "$WORKSPACE/MEMORY.md"

                # Remove the original
                start_line=$(grep -n "^### ${extra_pnum}:" "$WORKSPACE/MEMORY.md" | tail -1 | cut -d: -f1)
                next_line=$(tail -n "+$((start_line + 1))" "$WORKSPACE/MEMORY.md" | grep -n "^### \|^## " | head -1 | cut -d: -f1)
                if [ -n "$next_line" ]; then
                  end_line=$((start_line + next_line - 1))
                else
                  end_line=$(wc -l < "$WORKSPACE/MEMORY.md")
                fi
                tmp_mem=$(mktemp)
                head -n "$((start_line - 1))" "$WORKSPACE/MEMORY.md" > "$tmp_mem"
                tail -n "+$((end_line + 1))" "$WORKSPACE/MEMORY.md" >> "$tmp_mem"
                mv "$tmp_mem" "$WORKSPACE/MEMORY.md"
                echo "   ✅ Moved to P0-${next_letter}, removed $extra_pnum"
                UPDATED=$((UPDATED + 1))
              fi
              ;;
            r)
              if [ "$DRY_RUN" != "true" ]; then
                start_line=$(grep -n "^### ${extra_pnum}:" "$WORKSPACE/MEMORY.md" | head -1 | cut -d: -f1)
                next_line=$(tail -n "+$((start_line + 1))" "$WORKSPACE/MEMORY.md" | grep -n "^### \|^## " | head -1 | cut -d: -f1)
                if [ -n "$next_line" ]; then
                  end_line=$((start_line + next_line - 1))
                else
                  end_line=$(wc -l < "$WORKSPACE/MEMORY.md")
                fi
                tmp_mem=$(mktemp)
                head -n "$((start_line - 1))" "$WORKSPACE/MEMORY.md" > "$tmp_mem"
                tail -n "+$((end_line + 1))" "$WORKSPACE/MEMORY.md" >> "$tmp_mem"
                mv "$tmp_mem" "$WORKSPACE/MEMORY.md"
                echo "   ✅ Removed $extra_pnum"
                UPDATED=$((UPDATED + 1))
              fi
              ;;
            k)
              echo "   ⏭️  Kept $extra_pnum"
              SKIPPED=$((SKIPPED + 1))
              ;;
        esac
      fi
    done <<< "$EXTRA_PRINCIPLES"
  else
    echo "   ⏭️  No extra principles found (P0-P8 only)"
    SKIPPED=$((SKIPPED + 1))
  fi
fi

# Check for non-standard ## sections in MEMORY.md that should be moved
if [ -f "$WORKSPACE/MEMORY.md" ]; then
  echo ""
  echo "🧹 Checking MEMORY.md structure..."
  STANDARD_MEM_SECTIONS="PRINCIPLES|Identity|Memory Index"
  NON_STANDARD=""
  while IFS= read -r section_line; do
    section_name=$(echo "$section_line" | sed 's/^## //')
    if ! echo "$section_name" | grep -qE "($STANDARD_MEM_SECTIONS)"; then
      NON_STANDARD="${NON_STANDARD}${section_line}\n"
    fi
  done < <(grep "^## " "$WORKSPACE/MEMORY.md" | grep -v "^## 🔴")

  if [ -n "$NON_STANDARD" ]; then
    echo "   Found non-standard ## sections in MEMORY.md:"
    printf "   %b" "$NON_STANDARD" | sed 's/^/      /'
    echo "   MEMORY.md should only contain: 🔴 PRINCIPLES, ## Identity, ## Memory Index"
    echo ""

    # Move each non-standard section to its own file
    while IFS= read -r ns_line; do
      [ -z "$ns_line" ] && continue
      ns_name=$(echo "$ns_line" | sed 's/^## //')
      # Sanitize for filename: lowercase, spaces to hyphens, strip special chars
      ns_file=$(echo "$ns_name" | tr '[:upper:]' '[:lower:]' | tr ' ' '-' | tr -cd 'a-z0-9-')
      ns_dest="$WORKSPACE/memory/${ns_file}.md"

      # Extract section content
      ns_start=$(grep -n "^## ${ns_name}$" "$WORKSPACE/MEMORY.md" | head -1 | cut -d: -f1)
      [ -z "$ns_start" ] && continue
      ns_next=$(tail -n "+$((ns_start + 1))" "$WORKSPACE/MEMORY.md" | grep -n "^## " | head -1 | cut -d: -f1)
      if [ -n "$ns_next" ]; then
        ns_end=$((ns_start + ns_next - 1))
      else
        ns_end=$(wc -l < "$WORKSPACE/MEMORY.md")
      fi
      ns_content=$(sed -n "${ns_start},${ns_end}p" "$WORKSPACE/MEMORY.md")

      echo "   📦 \"## ${ns_name}\" → memory/${ns_file}.md"
      if ask_yn "   Move this section? (Y/n): " y; then
        if [ "$DRY_RUN" != "true" ]; then
          if [ -f "$ns_dest" ]; then
            echo "" >> "$ns_dest"
            echo "$ns_content" >> "$ns_dest"
            echo "   ✅ Appended to existing ${ns_dest##*/}"
          else
            echo "# ${ns_name}" > "$ns_dest"
            echo "" >> "$ns_dest"
            # Write body (skip the ## header line)
            echo "$ns_content" | tail -n +2 >> "$ns_dest"
            echo "   ✅ Created memory/${ns_file}.md"
          fi

          # Remove from MEMORY.md
          tmp_mem=$(mktemp)
          # Keep lines before section, skip section, keep lines after
          head -n "$((ns_start - 1))" "$WORKSPACE/MEMORY.md" > "$tmp_mem"
          if [ "$ns_end" -lt "$(wc -l < "$WORKSPACE/MEMORY.md")" ]; then
            tail -n "+$((ns_end + 1))" "$WORKSPACE/MEMORY.md" >> "$tmp_mem"
          fi
          mv "$tmp_mem" "$WORKSPACE/MEMORY.md"
          echo "   ✅ Removed from MEMORY.md"
          UPDATED=$((UPDATED + 1))
        fi
      else
        echo "   ⏭️  Kept in MEMORY.md"
        SKIPPED=$((SKIPPED + 1))
      fi
    done <<< "$(printf '%b' "$NON_STANDARD")"
  else
    echo "   ⏭️  MEMORY.md structure is clean"
    SKIPPED=$((SKIPPED + 1))
  fi

  # Size check — after non-standard section moves, re-measure
  MEM_SIZE=$(du -k "$WORKSPACE/MEMORY.md" 2>/dev/null | cut -f1)
  if [ "$MEM_SIZE" -gt 5 ]; then
    echo ""
    echo "   📏 MEMORY.md is ${MEM_SIZE}KB (target: < 5KB) — checking for bloated subsections..."

    # Find ### subsections under ## Memory Index that are over 10 lines
    # These can be extracted to dedicated files with a reference left behind
    MEM_INDEX_START=$(grep -n "^## Memory Index" "$WORKSPACE/MEMORY.md" | head -1 | cut -d: -f1 || true)
    if [ -n "$MEM_INDEX_START" ]; then
    MEM_INDEX_END=$(tail -n "+$((MEM_INDEX_START + 1))" "$WORKSPACE/MEMORY.md" | grep -n "^## " | head -1 | cut -d: -f1 || true)
      if [ -n "$MEM_INDEX_END" ]; then
        MEM_INDEX_END=$((MEM_INDEX_START + MEM_INDEX_END - 1))
      else
        MEM_INDEX_END=$(wc -l < "$WORKSPACE/MEMORY.md")
      fi

      # Extract each ### subsection and check line count
      SUBSECTIONS=$(sed -n "${MEM_INDEX_START},${MEM_INDEX_END}p" "$WORKSPACE/MEMORY.md" | grep "^### " || true)
      while IFS= read -r sub_header; do
        [ -z "$sub_header" ] && continue
        sub_name=$(echo "$sub_header" | sed 's/^### //' | sed 's/ (.*//')
        # Use fixed-string grep to handle special chars in headers like (memory/contacts/)
        sub_start=$(grep -nF "$sub_header" "$WORKSPACE/MEMORY.md" | head -1 | cut -d: -f1)
        [ -z "$sub_start" ] && continue
        sub_next=$(tail -n "+$((sub_start + 1))" "$WORKSPACE/MEMORY.md" | grep -n "^### \|^## " | head -1 | cut -d: -f1 || true)
        if [ -n "$sub_next" ]; then
          sub_end=$((sub_start + sub_next - 1))
        else
          sub_end=$MEM_INDEX_END
        fi
        sub_lines=$((sub_end - sub_start))

        if [ "$sub_lines" -gt 10 ]; then
          # Determine destination file
          sub_file=$(echo "$sub_name" | tr '[:upper:]' '[:lower:]' | tr ' ' '-' | tr -cd 'a-z0-9-')
          sub_dest="$WORKSPACE/memory/${sub_file}.md"

          echo "   📦 \"### ${sub_name}\" is ${sub_lines} lines — move to memory/${sub_file}.md?"
          if ask_yn "   Extract and replace with reference? (Y/n): " y; then
            if [ "$DRY_RUN" != "true" ]; then
              # Extract content (include header)
              sub_content=$(sed -n "${sub_start},${sub_end}p" "$WORKSPACE/MEMORY.md")

              if [ -f "$sub_dest" ]; then
                echo "" >> "$sub_dest"
                echo "$sub_content" | tail -n +2 >> "$sub_dest"
              else
                echo "# ${sub_name}" > "$sub_dest"
                echo "" >> "$sub_dest"
                echo "$sub_content" | tail -n +2 >> "$sub_dest"
              fi

              # Replace section with brief reference (keep ### header, add See reference, remove body)
              tmp_mem=$(mktemp)
              head -n "$sub_start" "$WORKSPACE/MEMORY.md" > "$tmp_mem"
              echo "See \`memory/${sub_file}.md\` for details." >> "$tmp_mem"
              echo "" >> "$tmp_mem"
              if [ "$sub_end" -lt "$(wc -l < "$WORKSPACE/MEMORY.md")" ]; then
                tail -n "+$((sub_end + 1))" "$WORKSPACE/MEMORY.md" >> "$tmp_mem"
              fi
              mv "$tmp_mem" "$WORKSPACE/MEMORY.md"

              echo "   ✅ Extracted to memory/${sub_file}.md, left reference in MEMORY.md"
              UPDATED=$((UPDATED + 1))

              # Recalculate line numbers since file changed
              MEM_INDEX_START=$(grep -n "^## Memory Index" "$WORKSPACE/MEMORY.md" | head -1 | cut -d: -f1 || true)
              if [ -n "$MEM_INDEX_START" ]; then
                MEM_INDEX_END_TMP=$(tail -n "+$((MEM_INDEX_START + 1))" "$WORKSPACE/MEMORY.md" | grep -n "^## " | head -1 | cut -d: -f1 || true)
                if [ -n "$MEM_INDEX_END_TMP" ]; then
                  MEM_INDEX_END=$((MEM_INDEX_START + MEM_INDEX_END_TMP - 1))
                else
                  MEM_INDEX_END=$(wc -l < "$WORKSPACE/MEMORY.md")
                fi
              fi
            fi
          else
            echo "   ⏭️  Kept inline"
            SKIPPED=$((SKIPPED + 1))
          fi
        fi
      done <<< "$SUBSECTIONS"
    fi

    # Final size report
    MEM_SIZE=$(du -k "$WORKSPACE/MEMORY.md" 2>/dev/null | cut -f1)
    if [ "$MEM_SIZE" -gt 5 ]; then
      echo "   ℹ️  MEMORY.md is now ${MEM_SIZE}KB — further trimming may be needed by the agent during distillation"
    else
      echo "   ✅ MEMORY.md trimmed to ${MEM_SIZE}KB (within target)"
    fi
  else
    echo "   ✅ MEMORY.md is ${MEM_SIZE}KB (within target)"
  fi
fi
echo ""

# ─────────────────────────────────────────────────────────────────────────────
# Part 3: Scripts — copy any missing helper scripts to workspace
# ─────────────────────────────────────────────────────────────────────────────
echo "📋 Checking helper scripts..."

copy_or_update_script() {
  local script_name="$1"
  local src="$SCRIPT_DIR/$script_name"
  local dst="$WORKSPACE/scripts/$script_name"

  if [ ! -f "$src" ]; then
    echo "   ⏭️  $script_name not found in skill package (skipped)"
    SKIPPED=$((SKIPPED + 1))
    return
  fi

  if [ -f "$dst" ]; then
    # Compare checksums — update if different
    local src_hash dst_hash
    src_hash=$(md5sum "$src" 2>/dev/null | cut -d' ' -f1)
    dst_hash=$(md5sum "$dst" 2>/dev/null | cut -d' ' -f1)
    if [ "$src_hash" = "$dst_hash" ]; then
      echo "   ⏭️  $script_name already current (skipped)"
      SKIPPED=$((SKIPPED + 1))
      return
    fi
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would update: $script_name"
    else
      cp "$src" "$dst"
      chmod +x "$dst"
      echo "   🔄 Updated $script_name"
    fi
    UPDATED=$((UPDATED + 1))
    return
  fi

  if [ "$DRY_RUN" = "true" ]; then
    echo "   [DRY RUN] Would copy: $src → $dst"
    UPDATED=$((UPDATED + 1))
  else
    mkdir -p "$WORKSPACE/scripts"
    cp "$src" "$dst"
    chmod +x "$dst"
    echo "   ✅ Copied $script_name to workspace scripts/"
    UPDATED=$((UPDATED + 1))
  fi
}

copy_or_update_script "verify.sh"
copy_or_update_script "vault.sh"
copy_or_update_script "metrics.sh"
copy_or_update_script "git-backup.sh"
echo ""

# ─────────────────────────────────────────────────────────────────────────────
# Part 3b: Reference docs — update to latest versions
# ─────────────────────────────────────────────────────────────────────────────
echo "📚 Checking reference documents..."

copy_or_update_ref() {
  local ref_name="$1"
  local src="$SCRIPT_DIR/../references/$ref_name"
  local dst="$WORKSPACE/skills/opencortex/references/$ref_name"

  if [ ! -f "$src" ]; then
    # Try alternate skill path
    src="$SCRIPT_DIR/../references/$ref_name"
    [ ! -f "$src" ] && return
  fi

  mkdir -p "$(dirname "$dst")"

  if [ -f "$dst" ]; then
    local src_hash dst_hash
    src_hash=$(md5sum "$src" 2>/dev/null | cut -d' ' -f1)
    dst_hash=$(md5sum "$dst" 2>/dev/null | cut -d' ' -f1)
    if [ "$src_hash" = "$dst_hash" ]; then
      echo "   ⏭️  references/$ref_name already current (skipped)"
      SKIPPED=$((SKIPPED + 1))
      return
    fi
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would update: references/$ref_name"
    else
      cp "$src" "$dst"
      echo "   🔄 Updated references/$ref_name"
    fi
    UPDATED=$((UPDATED + 1))
  else
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would copy: references/$ref_name"
    else
      cp "$src" "$dst"
      echo "   ✅ Copied references/$ref_name"
    fi
    UPDATED=$((UPDATED + 1))
  fi
}

copy_or_update_ref "distillation.md"
copy_or_update_ref "weekly-synthesis.md"
copy_or_update_ref "architecture.md"

# Create new directories if missing
for d in memory/contacts memory/workflows; do
  if [ ! -d "$WORKSPACE/$d" ]; then
    if [ "$DRY_RUN" = "true" ]; then
      echo "   [DRY RUN] Would create: $d/"
    else
      mkdir -p "$WORKSPACE/$d"
      echo "   📁 Created $d/"
      UPDATED=$((UPDATED + 1))
    fi
  fi
done

# Create preferences.md if missing
if [ ! -f "$WORKSPACE/memory/preferences.md" ]; then
  if [ "$DRY_RUN" = "true" ]; then
    echo "   [DRY RUN] Would create: memory/preferences.md"
  else
    cat > "$WORKSPACE/memory/preferences.md" <<'PREFEOF'
# Preferences — What My Human Prefers

Discovered preferences, organized by category. Updated by nightly distillation when new preferences are stated in conversation. Format: **Preference:** [what] — [context/reasoning] (YYYY-MM-DD)

---

## Communication
(add as discovered)

## Code & Technical
(add as discovered)

## Workflow & Process
(add as discovered)

## Scheduling & Time
(add as discovered)

## Tools & Services
(add as discovered)

## Content & Media
(add as discovered)

## Environment & Setup
(add as discovered)
PREFEOF
    echo "   📝 Created memory/preferences.md"
    UPDATED=$((UPDATED + 1))
  fi
fi

# Add missing MEMORY.md index sections
if [ -f "$WORKSPACE/MEMORY.md" ]; then
  echo "📋 Checking MEMORY.md index sections..."
  # Check core structure sections (## level)
  for core_section in "Identity" "Memory Index"; do
    if ! grep -q "^## ${core_section}" "$WORKSPACE/MEMORY.md" 2>/dev/null; then
      if [ "$DRY_RUN" = "true" ]; then
        echo "   [DRY RUN] Would add: ## ${core_section}"
      else
        case "$core_section" in
          Identity)
            echo -e "\n## Identity\n- **Name:** (your agent name)\n- **Human:** (your name)\n" >> "$WORKSPACE/MEMORY.md"
            ;;
          "Memory Index")
            echo -e "\n## Memory Index\n" >> "$WORKSPACE/MEMORY.md"
            ;;
        esac
        echo "   ✅ Added ## ${core_section}"
        UPDATED=$((UPDATED + 1))
      fi
    else
      echo "   ⏭️  ## ${core_section} exists (skipped)"
      SKIPPED=$((SKIPPED + 1))
    fi
  done

  # Check index sub-sections (### level)
  declare -A INDEX_TEMPLATES
  INDEX_TEMPLATES["Infrastructure"]="\n### Infrastructure\n- \`TOOLS.md\` — APIs, credentials, scripts, access methods\n"
  INDEX_TEMPLATES["Projects"]="\n### Projects (memory/projects/)\n| Project | Status | File |\n|---------|--------|------|\n| (your projects) | | |\n"
  INDEX_TEMPLATES["Scheduled Jobs"]="\n### Scheduled Jobs\n(document cron jobs and scheduled tasks here)\n"
  INDEX_TEMPLATES["Contacts"]="\n### Contacts (memory/contacts/)\n(one file per person/org — name, role, context, preferences, history)\n"
  INDEX_TEMPLATES["Workflows"]="\n### Workflows (memory/workflows/)\n(pipelines, automations, multi-service processes)\n"
  INDEX_TEMPLATES["Preferences"]="\n### Preferences (memory/preferences.md)\nCross-cutting user preferences organized by category. Updated as discovered.\n"
  INDEX_TEMPLATES["Runbooks"]="\n### Runbooks (memory/runbooks/)\nStep-by-step procedures for repeatable tasks. Sub-agents can follow these directly.\n"
  INDEX_TEMPLATES["Daily Logs"]="\n### Daily Logs\n\`memory/YYYY-MM-DD.md\` — Working daily logs. Distilled into project files periodically.\n"

  for section_name in "Infrastructure" "Projects" "Scheduled Jobs" "Contacts" "Workflows" "Preferences" "Runbooks" "Daily Logs"; do
    if ! grep -q "### ${section_name}" "$WORKSPACE/MEMORY.md" 2>/dev/null; then
      if [ "$DRY_RUN" = "true" ]; then
        echo "   [DRY RUN] Would add: ### ${section_name} section"
      else
        INDEX_TEXT="${INDEX_TEMPLATES[$section_name]}"
        # Try to insert in a sensible position
        if [ "$section_name" = "Daily Logs" ]; then
          # Always append at end
          echo -e "$INDEX_TEXT" >> "$WORKSPACE/MEMORY.md"
        elif grep -q "### Daily Logs" "$WORKSPACE/MEMORY.md"; then
          sed -i "/### Daily Logs/i\\${INDEX_TEXT}" "$WORKSPACE/MEMORY.md"
        else
          echo -e "$INDEX_TEXT" >> "$WORKSPACE/MEMORY.md"
        fi
        echo "   ✅ Added ### ${section_name} to MEMORY.md index"
        UPDATED=$((UPDATED + 1))
      fi
    else
      echo "   ⏭️  ### ${section_name} already in index (skipped)"
      SKIPPED=$((SKIPPED + 1))
    fi
  done
fi

# ─────────────────────────────────────────────────────────────────────────────
# Part 5: Template files — check existence and offer to regenerate
# ─────────────────────────────────────────────────────────────────────────────
echo "📄 Checking template files..."

# SOUL.md
if [ ! -f "$WORKSPACE/SOUL.md" ]; then
  echo "   ⚠️  SOUL.md is missing"
  if ask_yn "   Create default SOUL.md? (Y/n): " y; then
    if [ "$DRY_RUN" != "true" ]; then
      cat > "$WORKSPACE/SOUL.md" << 'SOULEOF'
# SOUL.md — Who You Are

*Customize this to define your agent's personality and boundaries.*

## Core Truths

**Be genuinely helpful, not performatively helpful.** Skip filler words and just help. Actions over ceremony.

**Have opinions.** You're allowed to disagree, prefer things, find stuff amusing or boring.

**Be resourceful before asking.** Try to figure it out first. Read the file. Check the context. Search for it. Then ask if stuck.

**Earn trust through competence.** Be careful with external actions. Be bold with internal ones.

**Remember you're a guest.** You have access to someone's life. Treat it with respect.

## Boundaries

- Private things stay private. Period.
- When in doubt, ask before acting externally.
- Never send half-baked replies to messaging surfaces.

## Continuity

Each session, you wake up fresh. Your files are your memory. Read them. Update them.

---

*This file is yours to evolve. As you learn who you are, update it.*
SOULEOF
      echo "   ✅ Created SOUL.md"
      UPDATED=$((UPDATED + 1))
    fi
  fi
else
  echo "   ⏭️  SOUL.md exists (skipped)"
  SKIPPED=$((SKIPPED + 1))
fi

# USER.md
if [ ! -f "$WORKSPACE/USER.md" ]; then
  echo "   ⚠️  USER.md is missing"
  if ask_yn "   Create default USER.md? (Y/n): " y; then
    if [ "$DRY_RUN" != "true" ]; then
      cat > "$WORKSPACE/USER.md" << 'USEREOF'
# USER.md — About Your Human

- **Name:** (your name)
- **Location:** (city, country)
- **Timezone:** (timezone)

## Communication Style

- **Prefers:** (direct? detailed? casual?)
- **Values:** (what matters to them?)

## Projects

(list active projects here as the agent learns about them)

## Preferences

(agent will fill this in over time from conversations)
USEREOF
      echo "   ✅ Created USER.md"
      UPDATED=$((UPDATED + 1))
    fi
  fi
else
  echo "   ⏭️  USER.md exists (skipped)"
  SKIPPED=$((SKIPPED + 1))
fi

# .gitignore — ensure sensitive entries
if [ -f "$WORKSPACE/.gitignore" ]; then
  GITIGNORE_ADDS=()
  for entry in ".vault/" ".secrets-map" ".env" "*.key" "*.pem"; do
    if ! grep -qF "$entry" "$WORKSPACE/.gitignore" 2>/dev/null; then
      GITIGNORE_ADDS+=("$entry")
    fi
  done
  if [ ${#GITIGNORE_ADDS[@]} -gt 0 ]; then
    if [ "$DRY_RUN" != "true" ]; then
      for entry in "${GITIGNORE_ADDS[@]}"; do
        echo "$entry" >> "$WORKSPACE/.gitignore"
      done
      echo "   ✅ Added missing .gitignore entries: ${GITIGNORE_ADDS[*]}"
      UPDATED=$((UPDATED + 1))
    else
      echo "   [DRY RUN] Would add .gitignore entries: ${GITIGNORE_ADDS[*]}"
    fi
  else
    echo "   ⏭️  .gitignore already has sensitive entries (skipped)"
    SKIPPED=$((SKIPPED + 1))
  fi
elif [ ! -f "$WORKSPACE/.gitignore" ]; then
  if [ "$DRY_RUN" != "true" ]; then
    printf '.vault/\n.secrets-map\n.env\n*.key\n*.pem\n' > "$WORKSPACE/.gitignore"
    echo "   ✅ Created .gitignore with sensitive entries"
    UPDATED=$((UPDATED + 1))
  fi
fi

# AGENTS.md — check for key sections, offer regeneration
if [ -f "$WORKSPACE/AGENTS.md" ]; then
  AGENTS_WARNINGS=()
  if ! grep -q "contacts\|Contacts" "$WORKSPACE/AGENTS.md" 2>/dev/null; then
    AGENTS_WARNINGS+=("contacts/workflows/preferences")
  fi
  if ! grep -q "Write Before Responding\|write-ahead\|Write before responding" "$WORKSPACE/AGENTS.md" 2>/dev/null; then
    AGENTS_WARNINGS+=("Write Before Responding (P2)")
  fi
  if ! grep -q "Custom Principles\|P0" "$WORKSPACE/AGENTS.md" 2>/dev/null; then
    AGENTS_WARNINGS+=("Custom Principles (P0) guidance")
  fi
  if ! grep -q "## Safety" "$WORKSPACE/AGENTS.md" 2>/dev/null; then
    AGENTS_WARNINGS+=("Safety boundaries")
  fi
  if ! grep -q "## Formatting" "$WORKSPACE/AGENTS.md" 2>/dev/null; then
    AGENTS_WARNINGS+=("Formatting guidance")
  fi
  if [ ${#AGENTS_WARNINGS[@]} -gt 0 ]; then
    echo "   ⚠️  AGENTS.md is missing: ${AGENTS_WARNINGS[*]}"
    if ask_yn "   Back up current and regenerate AGENTS.md? (y/N): " n; then
      if [ "$DRY_RUN" != "true" ]; then
        # Extract custom sections from existing AGENTS.md before regenerating
        # Standard sections that the template provides:
        STANDARD_SECTIONS="Boot Sequence|Principles|Delegation|Custom Principles|Write Before Responding|Memory Structure|Health Check|Metrics|Updates|Safety|Formatting"
        CUSTOM_SECTIONS=""
        if [ -f "$WORKSPACE/AGENTS.md" ]; then
          # Extract any ## sections whose title doesn't match a standard section
          current_section=""
          current_content=""
          while IFS= read -r line; do
            if echo "$line" | grep -q "^## "; then
              # Save previous non-standard section
              if [ -n "$current_section" ] && ! echo "$current_section" | grep -qE "($STANDARD_SECTIONS)"; then
                CUSTOM_SECTIONS="${CUSTOM_SECTIONS}${current_content}"$'\n'
              fi
              current_section=$(echo "$line" | sed 's/^## //')
              current_content="$line"$'\n'
            elif [ -n "$current_section" ]; then
              current_content="${current_content}${line}"$'\n'
            fi
          done < "$WORKSPACE/AGENTS.md"
          # Capture last section
          if [ -n "$current_section" ] && ! echo "$current_section" | grep -qE "($STANDARD_SECTIONS)"; then
            CUSTOM_SECTIONS="${CUSTOM_SECTIONS}${current_content}"$'\n'
          fi
        fi

        if [ -n "$CUSTOM_SECTIONS" ]; then
          echo "   📋 Preserving custom sections from existing AGENTS.md"
        fi

        # Determine boot sequence
        AGENTS_BOOT="## Boot Sequence"$'\n'"1. Read SOUL.md"$'\n'"2. Read MEMORY.md — principles + memory index"$'\n'"3. Use memory_search for anything deeper"
        if [ -f "$WORKSPACE/INFRA.md" ]; then
          AGENTS_BOOT="$AGENTS_BOOT"$'\n'"4. Read INFRA.md — infrastructure reference"
        fi

        cat > "$WORKSPACE/AGENTS.md" << AGENTSEOF
# AGENTS.md — Operating Protocol

$AGENTS_BOOT

## Principles
Live in MEMORY.md under 🔴 PRINCIPLES. Follow them always.

## Delegation (P1)
**Default action: delegate.** Before doing work, ask:
1. Can a sub-agent do this? → Yes for most things
2. What calibre? → Light (simple), Medium (moderate), Heavy (complex)
3. Delegate with clear task description + relevant file paths
4. Stay available to the user

**Sub-agent debrief (P6):** Include in every sub-agent task:
"Before completing, append a brief debrief to memory/YYYY-MM-DD.md: what you did, what you learned, any issues."

**Never delegate:** Conversation, confirmations, principle changes, ambiguous decisions

## Custom Principles (P0)
When the user asks to add a new principle, even if they ask for P9, P10, or any number beyond P8:
1. All custom principles go in P0 as sub-principles (P0-A, P0-B, P0-C, etc.)
2. Explain that P1-P8 are managed by OpenCortex and P0 is the dedicated space for custom additions
3. Before adding, assess whether it truly belongs as a principle or would be better as:
   - A **preference** (memory/preferences.md) — if it is about how the user likes things done
   - A **decision** (relevant project file) — if it is a one-time choice, not an ongoing rule
   - A **runbook** (memory/runbooks/) — if it is a step-by-step procedure
   - An **AGENTS.md rule** — if it is about agent behavior during boot or delegation
4. Check for conflicts with P1-P8. If the proposed principle would contradict an existing one, explain the conflict and work with the user to resolve it before adding
5. A principle should be a persistent behavioral rule that applies across all sessions and all work

## Write Before Responding (P2)
When the user states a preference, makes a decision, gives a deadline, or corrects you:
1. Write it to the relevant memory file FIRST
2. Then compose and send your response
This ensures nothing is lost if the session ends or compacts between your response and the write.

## Memory Structure
- MEMORY.md — Principles + index (< 3KB, fast load)
- TOOLS.md — Tool shed with abilities descriptions
- INFRA.md — Infrastructure atlas
- memory/projects/*.md — Per-project knowledge
- memory/contacts/*.md — Per-person/org knowledge
- memory/workflows/*.md — Per-workflow/pipeline knowledge
- memory/preferences.md — Cross-cutting user preferences by category
- memory/runbooks/*.md — Repeatable procedures
- memory/archive/*.md — Archived daily logs
- memory/YYYY-MM-DD.md — Today's working log

## Health Check
When the user asks if OpenCortex is installed, working, or wants a status check, run:
  bash skills/opencortex/scripts/verify.sh
Share the results and offer to fix any failures.

## Metrics
When the user asks about OpenCortex metrics, how it is doing, or wants to see growth:
1. Run: bash scripts/metrics.sh --report
2. Share the trends, compound score, and any areas that need attention.
3. If no data exists yet, run: bash scripts/metrics.sh --collect first.

## Safety
- Never exfiltrate private data
- Ask before external actions (P3)
- Private context stays out of group chats
- When in doubt, ask — do not assume permission

## Formatting
- Keep replies concise for chat surfaces (Telegram, Discord, etc.)
- Avoid markdown tables on surfaces that do not render them well
- Match the communication style documented in USER.md

## Updates
When the user asks to update OpenCortex or check for updates:
1. Run: clawhub install opencortex --force
2. Run: bash skills/opencortex/scripts/update.sh
3. Run: bash skills/opencortex/scripts/verify.sh
Share the results with the user.
AGENTSEOF

        # Append any custom sections that were in the old file
        if [ -n "$CUSTOM_SECTIONS" ]; then
          echo "" >> "$WORKSPACE/AGENTS.md"
          printf '%s' "$CUSTOM_SECTIONS" >> "$WORKSPACE/AGENTS.md"
          echo "   ✅ Regenerated AGENTS.md (custom sections preserved)"
        else
          echo "   ✅ Regenerated AGENTS.md"
        fi
        UPDATED=$((UPDATED + 1))
      fi
    else
      echo "   ⏭️  Kept existing AGENTS.md"
    fi
  else
    echo "   ⏭️  AGENTS.md looks current (skipped)"
    SKIPPED=$((SKIPPED + 1))
  fi
else
  echo "   ⚠️  AGENTS.md missing — run install.sh to create it"
fi

# BOOTSTRAP.md — check for key content, offer regeneration
if [ -f "$WORKSPACE/BOOTSTRAP.md" ]; then
  BOOTSTRAP_WARNINGS=()
  if ! grep -q "Sub-Agent Protocol\|sub-agent\|debrief" "$WORKSPACE/BOOTSTRAP.md" 2>/dev/null; then
    BOOTSTRAP_WARNINGS+=("sub-agent debrief protocol")
  fi
  if ! grep -q "HEARTBEAT\|heartbeat" "$WORKSPACE/BOOTSTRAP.md" 2>/dev/null; then
    BOOTSTRAP_WARNINGS+=("heartbeat handling")
  fi
  if [ ${#BOOTSTRAP_WARNINGS[@]} -gt 0 ]; then
    echo "   ⚠️  BOOTSTRAP.md is missing: ${BOOTSTRAP_WARNINGS[*]}"
    if ask_yn "   Back up current and regenerate BOOTSTRAP.md? (y/N): " n; then
      if [ "$DRY_RUN" != "true" ]; then
        # Extract custom sections from existing BOOTSTRAP.md
        BOOT_STANDARD="First-Run Checklist|Silent Replies|Sub-Agent Protocol"
        BOOT_CUSTOM=""
        if [ -f "$WORKSPACE/BOOTSTRAP.md" ]; then
          current_section=""
          current_content=""
          while IFS= read -r line; do
            if echo "$line" | grep -q "^## "; then
              if [ -n "$current_section" ] && ! echo "$current_section" | grep -qE "($BOOT_STANDARD)"; then
                BOOT_CUSTOM="${BOOT_CUSTOM}${current_content}"$'\n'
              fi
              current_section=$(echo "$line" | sed 's/^## //')
              current_content="$line"$'\n'
            elif [ -n "$current_section" ]; then
              current_content="${current_content}${line}"$'\n'
            fi
          done < "$WORKSPACE/BOOTSTRAP.md"
          if [ -n "$current_section" ] && ! echo "$current_section" | grep -qE "($BOOT_STANDARD)"; then
            BOOT_CUSTOM="${BOOT_CUSTOM}${current_content}"$'\n'
          fi
        fi
        cat > "$WORKSPACE/BOOTSTRAP.md" << 'BOOTEOF'
# BOOTSTRAP.md — First-Run Checklist

On new session start:
1. Read SOUL.md — identity and personality
2. Read MEMORY.md — principles (🔴 PRINCIPLES section) and memory index
3. Do NOT bulk-load other files — use memory_search when needed

## Silent Replies
- `NO_REPLY` — when you have nothing to say (must be entire message, nothing else)
- `HEARTBEAT_OK` — when heartbeat poll finds nothing needing attention

## Sub-Agent Protocol
When delegating, always include in task message:
"Before completing, append a brief debrief to memory/YYYY-MM-DD.md (today's date): what you did, what you learned, any issues."
BOOTEOF
        if [ -n "$BOOT_CUSTOM" ]; then
          echo "" >> "$WORKSPACE/BOOTSTRAP.md"
          printf '%s' "$BOOT_CUSTOM" >> "$WORKSPACE/BOOTSTRAP.md"
          echo "   ✅ Regenerated BOOTSTRAP.md (custom sections preserved)"
        else
          echo "   ✅ Regenerated BOOTSTRAP.md"
        fi
        UPDATED=$((UPDATED + 1))
      fi
    else
      echo "   ⏭️  Kept existing BOOTSTRAP.md"
    fi
  else
    echo "   ⏭️  BOOTSTRAP.md looks current (skipped)"
    SKIPPED=$((SKIPPED + 1))
  fi
else
  echo "   ⚠️  BOOTSTRAP.md missing — run install.sh to create it"
fi

# ─────────────────────────────────────────────────────────────────────────────
# Part 6: Cron jobs — verify they exist
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "⏰ Verifying cron jobs..."
if command -v openclaw >/dev/null 2>&1; then
  CRON_LIST=$(openclaw cron list 2>/dev/null || true)

  # Deduplicate: remove extra Daily Memory Distillation crons (keep the first)
  DAILY_MATCHES=$(echo "$CRON_LIST" | grep -i "Daily Memory Distill" | awk '{print $1}' || true)
  DAILY_COUNT=$(echo "$DAILY_MATCHES" | grep -c . 2>/dev/null || true)
  if [ "$DAILY_COUNT" -gt 1 ]; then
    echo "   🧹 Found $DAILY_COUNT 'Daily Memory Distillation' crons — removing duplicates..."
    FIRST_DAILY=true
    while IFS= read -r cron_id; do
      [ -z "$cron_id" ] && continue
      if [ "$FIRST_DAILY" = true ]; then
        FIRST_DAILY=false
        echo "   ⏭️  Keeping: $cron_id"
      else
        if [ "$DRY_RUN" != "true" ]; then
          openclaw cron delete "$cron_id" 2>/dev/null && \
          echo "   🗑️  Deleted duplicate: $cron_id" || true
        else
          echo "   [DRY RUN] Would delete duplicate: $cron_id"
        fi
      fi
    done <<< "$DAILY_MATCHES"
    UPDATED=$((UPDATED + 1))
  fi

  # Deduplicate: remove extra Weekly Synthesis crons (keep the first)
  WEEKLY_MATCHES=$(echo "$CRON_LIST" | grep -i "Weekly Synth" | awk '{print $1}' || true)
  WEEKLY_COUNT=$(echo "$WEEKLY_MATCHES" | grep -c . 2>/dev/null || true)
  if [ "$WEEKLY_COUNT" -gt 1 ]; then
    echo "   🧹 Found $WEEKLY_COUNT 'Weekly Synthesis' crons — removing duplicates..."
    FIRST_WEEKLY=true
    while IFS= read -r cron_id; do
      [ -z "$cron_id" ] && continue
      if [ "$FIRST_WEEKLY" = true ]; then
        FIRST_WEEKLY=false
        echo "   ⏭️  Keeping: $cron_id"
      else
        if [ "$DRY_RUN" != "true" ]; then
          openclaw cron delete "$cron_id" 2>/dev/null && \
          echo "   🗑️  Deleted duplicate: $cron_id" || true
        else
          echo "   [DRY RUN] Would delete duplicate: $cron_id"
        fi
      fi
    done <<< "$WEEKLY_MATCHES"
    UPDATED=$((UPDATED + 1))
  fi

  # Refresh cron list after dedup
  CRON_LIST=$(openclaw cron list 2>/dev/null || true)

  if echo "$CRON_LIST" | grep -qi "Daily Memory Distill"; then
    echo "   ⏭️  Daily Memory Distillation cron exists"
    SKIPPED=$((SKIPPED + 1))
  else
    echo "   ⚠️  Daily Memory Distillation cron not found"
    if ask_yn "   Create it now? (Y/n): " y; then
      # Detect timezone
      CRON_TZ="${CLAWD_TZ:-}"
      if [ -z "$CRON_TZ" ] && [ -f /etc/timezone ]; then
        CRON_TZ=$(cat /etc/timezone 2>/dev/null | tr -d '[:space:]')
      elif [ -z "$CRON_TZ" ] && [ -L /etc/localtime ]; then
        CRON_TZ=$(readlink -f /etc/localtime 2>/dev/null | sed 's|.*/zoneinfo/||')
      fi
      CRON_TZ="${CRON_TZ:-UTC}"
      if [ "$DRY_RUN" != "true" ]; then
        USER_MODEL=""
        [ -f "$WORKSPACE/.opencortex-model" ] && USER_MODEL=$(cat "$WORKSPACE/.opencortex-model" 2>/dev/null | tr -d '[:space:]')
        USER_MODEL="${USER_MODEL:-sonnet}"
        openclaw cron add \
          --name "Daily Memory Distillation" \
          --cron "0 3 * * *" \
          --tz "$CRON_TZ" \
          --session "isolated" \
          --timeout-seconds 600 \
          --model "$USER_MODEL" \
          --no-deliver \
          --message "$DAILY_MSG" 2>/dev/null && \
        echo "   ✅ Created Daily Memory Distillation cron (3 AM $CRON_TZ)" || \
        echo "   ❌ Failed to create cron — try: openclaw cron add --name 'Daily Memory Distillation' --cron '0 3 * * *'"
        UPDATED=$((UPDATED + 1))
      fi
    fi
  fi
  if echo "$CRON_LIST" | grep -qi "Weekly Synth"; then
    echo "   ⏭️  Weekly Synthesis cron exists"
    SKIPPED=$((SKIPPED + 1))
  else
    echo "   ⚠️  Weekly Synthesis cron not found"
    if ask_yn "   Create it now? (Y/n): " y; then
      CRON_TZ="${CRON_TZ:-${CLAWD_TZ:-UTC}}"
      if [ "$DRY_RUN" != "true" ]; then
        USER_MODEL=""
        [ -f "$WORKSPACE/.opencortex-model" ] && USER_MODEL=$(cat "$WORKSPACE/.opencortex-model" 2>/dev/null | tr -d '[:space:]')
        USER_MODEL="${USER_MODEL:-sonnet}"
        openclaw cron add \
          --name "Weekly Synthesis" \
          --cron "0 5 * * 0" \
          --tz "$CRON_TZ" \
          --session "isolated" \
          --timeout-seconds 600 \
          --model "$USER_MODEL" \
          --no-deliver \
          --message "$WEEKLY_MSG" 2>/dev/null && \
        echo "   ✅ Created Weekly Synthesis cron (Sunday 5 AM $CRON_TZ)" || \
        echo "   ❌ Failed to create cron — try: openclaw cron add --name 'Weekly Synthesis' --cron '0 5 * * 0'"
        UPDATED=$((UPDATED + 1))
      fi
    fi
  fi
else
  echo "   ⏭️  openclaw not in PATH — skipping cron existence check"
fi

echo ""

# ─────────────────────────────────────────────────────────────────────────────
# New optional features (offer if not already set up)
# ─────────────────────────────────────────────────────────────────────────────
WORKSPACE="${CLAWD_WORKSPACE:-$(pwd)}"
SKILL_DIR="$(cd "$(dirname "$0")" && pwd)"

# Metrics
if ! crontab -l 2>/dev/null | grep -q "metrics.sh"; then
  echo ""
  if ask_yn "📊 New feature: daily metrics tracking (knowledge growth over time). Enable? (y/N): " n; then
    if [ -f "$SKILL_DIR/metrics.sh" ]; then
      if [ "$DRY_RUN" != "true" ]; then
        cp "$SKILL_DIR/metrics.sh" "$WORKSPACE/scripts/metrics.sh"
        chmod +x "$WORKSPACE/scripts/metrics.sh"
        (crontab -l 2>/dev/null; echo "30 23 * * * $WORKSPACE/scripts/metrics.sh --collect") | crontab -
        "$WORKSPACE/scripts/metrics.sh" --collect
        echo "   ✅ Metrics enabled — daily snapshots at 11:30 PM"
        echo "   📊 First snapshot captured. View with: bash scripts/metrics.sh --report"
        UPDATED=$((UPDATED + 1))
      else
        echo "   [DRY RUN] Would enable metrics tracking"
      fi
    fi
  fi
else
  # Update metrics script if it exists
  if [ -f "$SKILL_DIR/metrics.sh" ] && [ -f "$WORKSPACE/scripts/metrics.sh" ]; then
    if [ "$DRY_RUN" != "true" ]; then
      cp "$SKILL_DIR/metrics.sh" "$WORKSPACE/scripts/metrics.sh"
      chmod +x "$WORKSPACE/scripts/metrics.sh"
      echo "   📊 Metrics script updated to latest version"
    fi
  fi
fi

# ─────────────────────────────────────────────────────────────────────────────
# Memory search health
# ─────────────────────────────────────────────────────────────────────────────
echo "🔍 Checking memory search..."
MEM_DB=$(find "$HOME/.openclaw" -name "memory*.sqlite" -o -name "memory*.db" 2>/dev/null | head -1)
if [ -n "$MEM_DB" ]; then
  DB_SIZE=$(du -k "$MEM_DB" 2>/dev/null | cut -f1)
  echo "   ✅ Memory search index found (${DB_SIZE}KB)"
  SKIPPED=$((SKIPPED + 1))
else
  echo "   ℹ️  No memory search index found (optional)."
  echo "   OpenCortex works fine without it — memory files are still searchable by the agent."
  echo "   For improved semantic search, configure an embedding provider in OpenClaw:"
  echo "   Docs: https://docs.openclaw.ai/concepts/memory"
fi
echo ""

echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "   ✅ Updated: $UPDATED"
echo "   ⏭️  Skipped (already current): $SKIPPED"
echo ""

if [ "$DRY_RUN" = "true" ]; then
  echo "   Dry run complete. Re-run without --dry-run to apply changes."
else
  # Update version marker
  WORKSPACE="${CLAWD_WORKSPACE:-$(pwd)}"
  echo "$OPENCORTEX_VERSION" > "$WORKSPACE/.opencortex-version"
  echo "   Update complete (v$OPENCORTEX_VERSION). Run verify.sh to confirm everything is healthy:"
  echo "   bash skills/opencortex/scripts/verify.sh"
fi
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"


path: scripts/vault.sh
content: #!/bin/bash
# OpenCortex Vault — Encrypted key-value store for sensitive data
# Uses GPG symmetric encryption (AES-256).
# Passphrase storage (in order of preference):
#   1. System keyring (secret-tool / macOS Keychain / keyctl)
#   2. Environment variable OPENCORTEX_VAULT_PASS
#   3. File at .vault/.passphrase (mode 600) — fallback
#
# Usage:
#   vault.sh init              — Set up vault (creates encrypted store + stores passphrase)
#   vault.sh set <key> <value> — Store a secret
#   vault.sh get <key>         — Retrieve a secret (stdout only, no logging)
#   vault.sh list              — List stored keys (not values)
#   vault.sh delete <key>      — Remove a secret
#   vault.sh rotate            — Rotate passphrase (re-encrypts all secrets)

set -euo pipefail

WORKSPACE="${CLAWD_WORKSPACE:-$(cd "$(dirname "$0")/.." && pwd)}"
VAULT_DIR="$WORKSPACE/.vault"
VAULT_FILE="$VAULT_DIR/secrets.gpg"
VAULT_PASS_FILE="$VAULT_DIR/.passphrase"
KEYRING_LABEL="opencortex-vault"

# --- Passphrase backend detection ---

_keyring_backend() {
  # Returns: secret-tool, keychain, keyctl, env, file, none
  if command -v secret-tool &>/dev/null; then
    echo "secret-tool"
  elif command -v security &>/dev/null && [[ "$OSTYPE" == darwin* ]]; then
    echo "keychain"
  elif command -v keyctl &>/dev/null; then
    echo "keyctl"
  elif [ -n "${OPENCORTEX_VAULT_PASS:-}" ]; then
    echo "env"
  elif [ -f "$VAULT_PASS_FILE" ]; then
    echo "file"
  else
    echo "none"
  fi
}

_store_passphrase() {
  local pass="$1"
  local backend=$(_keyring_backend)
  
  case "$backend" in
    secret-tool)
      echo "$pass" | secret-tool store --label="$KEYRING_LABEL" service opencortex key vault-passphrase 2>/dev/null
      echo "   🔐 Passphrase stored in system keyring (secret-tool)"
      ;;
    keychain)
      security add-generic-password -a opencortex -s "$KEYRING_LABEL" -w "$pass" 2>/dev/null || \
        security delete-generic-password -a opencortex -s "$KEYRING_LABEL" 2>/dev/null && \
        security add-generic-password -a opencortex -s "$KEYRING_LABEL" -w "$pass"
      echo "   🔐 Passphrase stored in macOS Keychain"
      ;;
    keyctl)
      local ring=$(keyctl newring opencortex @u 2>/dev/null || keyctl search @u keyring opencortex 2>/dev/null)
      echo "$pass" | keyctl padd user vault-passphrase "$ring" >/dev/null 2>&1
      echo "   🔐 Passphrase stored in kernel keyring (keyctl)"
      ;;
    *)
      # Fallback to file — only if explicitly allowed
      if [ "${OPENCORTEX_ALLOW_FILE_PASSPHRASE:-0}" = "1" ]; then
        echo "$pass" > "$VAULT_PASS_FILE"
        chmod 600 "$VAULT_PASS_FILE"
        echo "   📁 Passphrase stored in $VAULT_PASS_FILE (mode 600)"
        echo "   ⚠️  File-based storage is less secure than a system keyring"
      else
        echo "   ❌ No system keyring available (need secret-tool, macOS Keychain, or keyctl)."
        echo "   💡 Install a keyring tool, or set OPENCORTEX_ALLOW_FILE_PASSPHRASE=1 to allow file-based storage."
        return 1
      fi
      ;;
  esac
}

_get_passphrase() {
  local backend=$(_keyring_backend)
  
  case "$backend" in
    secret-tool)
      secret-tool lookup service opencortex key vault-passphrase 2>/dev/null
      ;;
    keychain)
      security find-generic-password -a opencortex -s "$KEYRING_LABEL" -w 2>/dev/null
      ;;
    keyctl)
      local ring=$(keyctl search @u keyring opencortex 2>/dev/null) || { echo ""; return 1; }
      local key=$(keyctl search "$ring" user vault-passphrase 2>/dev/null) || { echo ""; return 1; }
      keyctl print "$key" 2>/dev/null
      ;;
    env)
      echo "${OPENCORTEX_VAULT_PASS:-}"
      ;;
    file)
      cat "$VAULT_PASS_FILE" 2>/dev/null
      ;;
    *)
      echo ""
      return 1
      ;;
  esac
}

_ensure_vault() {
  if [ ! -f "$VAULT_FILE" ]; then
    echo "Vault not initialized. Run: vault.sh init"
    exit 1
  fi
  local pass=$(_get_passphrase)
  if [ -z "$pass" ]; then
    echo "❌ Cannot retrieve vault passphrase."
    echo "   Checked: system keyring, OPENCORTEX_VAULT_PASS env, $VAULT_PASS_FILE"
    echo "   Run 'vault.sh init' to set up, or set OPENCORTEX_VAULT_PASS."
    exit 1
  fi
}

# Passphrase passed via fd 3 to avoid exposure in process list
_encrypt() {
  local content="$1"
  local pass=$(_get_passphrase)
  local tmpfile=$(mktemp)
  echo "$content" > "$tmpfile"
  gpg --batch --yes --passphrase-fd 3 --quiet --symmetric --cipher-algo AES256 --output "$VAULT_FILE" "$tmpfile" 3<<< "$pass" 2>/dev/null
  rm -f "$tmpfile"
  chmod 600 "$VAULT_FILE"
}

_decrypt() {
  local pass=$(_get_passphrase)
  gpg --batch --yes --passphrase-fd 3 --quiet --decrypt "$VAULT_FILE" 3<<< "$pass" 2>/dev/null
}

case "${1:-help}" in
  init)
    mkdir -p "$VAULT_DIR"
    chmod 700 "$VAULT_DIR"
    
    # Check if vault already exists with a retrievable passphrase
    if [ -f "$VAULT_FILE" ]; then
      local_pass=$(_get_passphrase 2>/dev/null || true)
      if [ -n "$local_pass" ]; then
        echo "Vault already initialized at $VAULT_DIR"
        echo "   Backend: $(_keyring_backend)"
        exit 0
      fi
    fi
    
    # Generate random passphrase
    PASS=$(openssl rand -base64 32)
    
    # Store in best available backend
    _store_passphrase "$PASS"
    
    # Create empty vault
    # Temporarily set file backend for initial encrypt
    echo "$PASS" > "$VAULT_PASS_FILE"
    chmod 600 "$VAULT_PASS_FILE"
    _encrypt ""
    
    # If we stored in keyring, remove the file
    local backend=$(_keyring_backend)
    if [ "$backend" != "file" ] && [ "$backend" != "none" ]; then
      rm -f "$VAULT_PASS_FILE"
      echo "   🗑️  Removed file-based passphrase (using $backend instead)"
    fi
    
    echo "✅ Vault initialized at $VAULT_DIR"
    ;;
    
  set)
    _ensure_vault
    KEY="${2:-}"
    VALUE="${3:-}"
    
    if [ -z "$KEY" ] || [ -z "$VALUE" ]; then
      echo "Usage: vault.sh set <key> <value>"
      exit 1
    fi

    # Validate key name
    if ! echo "$KEY" | grep -qE '^[a-zA-Z_][a-zA-Z0-9_]*$'; then
      echo "❌ Invalid key name: '$KEY'"
      echo "   Key must start with a letter or underscore, and contain only"
      echo "   letters, digits, and underscores."
      exit 1
    fi

    CONTENT=$(_decrypt | grep -v "^${KEY}=" || true)
    CONTENT="${CONTENT}
${KEY}=${VALUE}"
    _encrypt "$CONTENT"
    
    echo "✅ Stored: $KEY"
    ;;
    
  get)
    _ensure_vault
    KEY="${2:-}"
    
    if [ -z "$KEY" ]; then
      echo "Usage: vault.sh get <key>"
      exit 1
    fi
    
    VALUE=$(_decrypt | grep "^${KEY}=" | head -1 | cut -d= -f2-)
    
    if [ -z "$VALUE" ]; then
      echo "Key not found: $KEY" >&2
      exit 1
    fi
    
    echo "$VALUE"
    ;;
    
  list)
    _ensure_vault
    _decrypt | grep -v "^$" | cut -d= -f1 | sort
    ;;
    
  delete)
    _ensure_vault
    KEY="${2:-}"
    
    if [ -z "$KEY" ]; then
      echo "Usage: vault.sh delete <key>"
      exit 1
    fi
    
    CONTENT=$(_decrypt | grep -v "^${KEY}=" || true)
    _encrypt "$CONTENT"
    
    echo "✅ Deleted: $KEY"
    ;;
    
  rotate)
    _ensure_vault
    echo "🔄 Rotating vault passphrase..."

    # Decrypt all secrets with current passphrase
    CONTENT=$(_decrypt)

    # Generate new passphrase
    NEW_PASS=$(openssl rand -base64 32)

    # Store new passphrase
    _store_passphrase "$NEW_PASS"
    
    # If file backend, update directly
    if [ -f "$VAULT_PASS_FILE" ]; then
      echo "$NEW_PASS" > "$VAULT_PASS_FILE"
      chmod 600 "$VAULT_PASS_FILE"
    fi

    # Re-encrypt with new passphrase (fd 3 to avoid process list exposure)
    local tmpfile=$(mktemp)
    echo "$CONTENT" > "$tmpfile"
    gpg --batch --yes --passphrase-fd 3 --quiet --symmetric --cipher-algo AES256 --output "$VAULT_FILE" "$tmpfile" 3<<< "$NEW_PASS" 2>/dev/null
    rm -f "$tmpfile"
    chmod 600 "$VAULT_FILE"

    echo "✅ Passphrase rotated. All secrets re-encrypted."
    echo "   Backend: $(_keyring_backend)"
    ;;

  migrate)
    # Migrate passphrase from file to system keyring
    _ensure_vault
    if [ ! -f "$VAULT_PASS_FILE" ]; then
      echo "No file-based passphrase to migrate."
      echo "Current backend: $(_keyring_backend)"
      exit 0
    fi
    
    PASS=$(cat "$VAULT_PASS_FILE")
    OLD_BACKEND=$(_keyring_backend)
    
    # Try to store in keyring (will use best available non-file backend)
    if command -v secret-tool &>/dev/null || { command -v security &>/dev/null && [[ "$OSTYPE" == darwin* ]]; } || command -v keyctl &>/dev/null; then
      _store_passphrase "$PASS"
      # Verify we can retrieve it
      VERIFY=$(_get_passphrase)
      if [ "$VERIFY" = "$PASS" ]; then
        rm -f "$VAULT_PASS_FILE"
        echo "✅ Migrated passphrase from file → $(_keyring_backend)"
        echo "   🗑️  Removed $VAULT_PASS_FILE"
      else
        echo "❌ Migration failed — keyring retrieval mismatch. File kept."
      fi
    else
      echo "❌ No system keyring available (need secret-tool, macOS Keychain, or keyctl)."
      echo "   Passphrase remains in $VAULT_PASS_FILE"
    fi
    ;;

  backend)
    echo "Passphrase backend: $(_keyring_backend)"
    if [ -f "$VAULT_PASS_FILE" ]; then
      echo "⚠️  File-based passphrase exists at $VAULT_PASS_FILE"
      echo "   Run 'vault.sh migrate' to move to system keyring"
    fi
    ;;

  help|*)
    echo "OpenCortex Vault — Encrypted secret storage"
    echo ""
    echo "Commands:"
    echo "  init              Set up vault (auto-detects best passphrase storage)"
    echo "  set <key> <value> Store a secret"
    echo "  get <key>         Retrieve a secret"
    echo "  list              List keys (not values)"
    echo "  delete <key>      Remove a secret"
    echo "  rotate            Rotate passphrase (re-encrypts all secrets)"
    echo "  migrate           Move passphrase from file → system keyring"
    echo "  backend           Show current passphrase storage backend"
    echo ""
    echo "Passphrase storage (in order of preference):"
    echo "  1. secret-tool    Linux keyring (GNOME/KDE)"
    echo "  2. security       macOS Keychain"
    echo "  3. keyctl         Linux kernel keyring"
    echo "  4. env var        OPENCORTEX_VAULT_PASS"
    echo "  5. file           .vault/.passphrase (mode 600, fallback)"
    echo ""
    echo "Current backend: $(_keyring_backend)"
    ;;
esac


path: scripts/verify.sh
content: #!/bin/bash
# OpenCortex — Verify installation is working correctly
# Run from your OpenClaw workspace directory: bash skills/opencortex/scripts/verify.sh

set -euo pipefail
WORKSPACE="${CLAWD_WORKSPACE:-$(pwd)}"
PASS=0
FAIL=0
WARN=0

echo "🔍 OpenCortex Installation Verification"
INSTALLED_VER="unknown"
[ -f "$WORKSPACE/.opencortex-version" ] && INSTALLED_VER="v$(cat "$WORKSPACE/.opencortex-version" | tr -d '[:space:]')"
echo "   Workspace: $WORKSPACE"
echo "   Version:   $INSTALLED_VER"
echo ""

check() {
  local label="$1"
  local result="$2"
  if [ "$result" = "ok" ]; then
    echo "   ✅ $label"
    PASS=$((PASS + 1))
  elif [ "$result" = "warn" ]; then
    echo "   ⚠️  $label"
    WARN=$((WARN + 1))
  else
    echo "   ❌ $label"
    FAIL=$((FAIL + 1))
  fi
}

# --- Core files ---
echo "📁 Core files:"
for f in MEMORY.md SOUL.md USER.md TOOLS.md INFRA.md BOOTSTRAP.md AGENTS.md; do
  if [ -f "$WORKSPACE/$f" ]; then
    check "$f exists" "ok"
  else
    check "$f missing" "fail"
  fi
done
echo ""

# --- Directories ---
echo "📂 Directories:"
for d in memory memory/projects memory/runbooks memory/contacts memory/workflows memory/archive; do
  if [ -d "$WORKSPACE/$d" ]; then
    check "$d/ exists" "ok"
  else
    check "$d/ missing" "fail"
  fi
done
echo ""

# --- Voice profile ---
echo "🎙️ Voice profile:"
FLAGS_FILE="$WORKSPACE/.opencortex-flags"
VOICE_FLAG=""
if [ -f "$FLAGS_FILE" ]; then
  VOICE_FLAG=$(grep -E '^VOICE_PROFILE=' "$FLAGS_FILE" 2>/dev/null | head -1 | cut -d'=' -f2 | tr -d '[:space:]' || true)
fi
VOICE_ENV="${OPENCORTEX_VOICE_PROFILE:-}"
if [ -f "$WORKSPACE/memory/VOICE.md" ]; then
  check "VOICE.md exists (voice profiling enabled)" "ok"
  if [ "$VOICE_FLAG" = "1" ] || [ "$VOICE_ENV" = "1" ]; then
    check "Voice profiling runtime gate is enabled (flag/env)" "ok"
  else
    check "VOICE.md exists but runtime voice gate is OFF (set VOICE_PROFILE=1 in .opencortex-flags or OPENCORTEX_VOICE_PROFILE=1)" "warn"
  fi
else
  check "VOICE.md not found (voice profiling not enabled — this is fine if you skipped it)" "warn"
fi
echo ""

# --- Cron jobs ---
echo "⏰ Cron jobs:"
if command -v openclaw &>/dev/null; then
  CRON_LIST=$(openclaw cron list 2>/dev/null || echo "")
  if echo "$CRON_LIST" | grep -qi "distill"; then
    check "Daily Memory Distillation cron found" "ok"
  else
    check "Daily Memory Distillation cron NOT found" "fail"
  fi
  if echo "$CRON_LIST" | grep -qi "synth"; then
    check "Weekly Synthesis cron found" "ok"
  else
    check "Weekly Synthesis cron NOT found" "fail"
  fi
  # Check for problematic model overrides (models that don't exist)
  CRON_JSON=$(openclaw cron list --json 2>/dev/null || echo "")
  if [ -n "$CRON_JSON" ]; then
    # Check distillation/weekly prompts are not empty
    if command -v python3 &>/dev/null; then
      DISTILL_MSG=$(echo "$CRON_JSON" | python3 -c "
import sys, json
try:
  data = json.load(sys.stdin)
  jobs = data.get('jobs', []) if isinstance(data, dict) else (data if isinstance(data, list) else [])
  for c in jobs:
    n = (c.get('name') or '').lower()
    if 'distill' in n:
      payload = c.get('payload') or {}
      msg = c.get('message') or payload.get('message') or ''
      print(msg.strip())
      break
except: pass
" 2>/dev/null || echo "")
      WEEKLY_MSG=$(echo "$CRON_JSON" | python3 -c "
import sys, json
try:
  data = json.load(sys.stdin)
  jobs = data.get('jobs', []) if isinstance(data, dict) else (data if isinstance(data, list) else [])
  for c in jobs:
    n = (c.get('name') or '').lower()
    if 'synth' in n:
      payload = c.get('payload') or {}
      msg = c.get('message') or payload.get('message') or ''
      print(msg.strip())
      break
except: pass
" 2>/dev/null || echo "")
      if [ -n "$DISTILL_MSG" ]; then
        check "Distillation cron prompt is set" "ok"
      else
        check "Distillation cron prompt is empty (distillation will no-op)" "warn"
      fi
      if [ -n "$WEEKLY_MSG" ]; then
        check "Weekly synthesis cron prompt is set" "ok"
      else
        check "Weekly synthesis cron prompt is empty (synthesis will no-op)" "warn"
      fi
    fi

    BAD_MODELS=$(echo "$CRON_JSON" | grep -o '"model":\s*"[^"]*"' | grep -iE '"(anthropic/default|default)"' || true)
    if [ -n "$BAD_MODELS" ]; then
      check "Cron jobs have invalid model overrides (e.g. 'default' is not a real model)" "warn"
    else
      check "No invalid model overrides in crons" "ok"
    fi
    # Check cron timeouts are sufficient
    if command -v python3 &>/dev/null; then
      DISTILL_TIMEOUT=$(echo "$CRON_JSON" | python3 -c "
import sys, json
try:
  data = json.load(sys.stdin)
  if isinstance(data, list):
    for c in data:
      name = c.get('name','')
      if 'distill' in name.lower():
        print(c.get('timeoutSeconds', c.get('timeout_seconds', c.get('timeout', 0))))
        break
except: pass
" 2>/dev/null || echo "")
      SYNTH_TIMEOUT=$(echo "$CRON_JSON" | python3 -c "
import sys, json
try:
  data = json.load(sys.stdin)
  if isinstance(data, list):
    for c in data:
      name = c.get('name','')
      if 'synth' in name.lower():
        print(c.get('timeoutSeconds', c.get('timeout_seconds', c.get('timeout', 0))))
        break
except: pass
" 2>/dev/null || echo "")
      if [ -n "$DISTILL_TIMEOUT" ] && [ "$DISTILL_TIMEOUT" -gt 0 ] 2>/dev/null; then
        if [ "$DISTILL_TIMEOUT" -lt 300 ]; then
          check "Distillation cron timeout is ${DISTILL_TIMEOUT}s — recommended minimum is 300s (set to 600s for best results)" "warn"
        else
          check "Distillation cron timeout: ${DISTILL_TIMEOUT}s (ok)" "ok"
        fi
      fi
      if [ -n "$SYNTH_TIMEOUT" ] && [ "$SYNTH_TIMEOUT" -gt 0 ] 2>/dev/null; then
        if [ "$SYNTH_TIMEOUT" -lt 300 ]; then
          check "Weekly synthesis cron timeout is ${SYNTH_TIMEOUT}s — recommended minimum is 300s for distillation, 600s for synthesis" "warn"
        else
          check "Weekly synthesis cron timeout: ${SYNTH_TIMEOUT}s (ok)" "ok"
        fi
      fi
    fi
  fi
else
  check "openclaw CLI not found — cannot verify cron jobs" "fail"
fi
echo ""

# --- Principles ---
echo "📜 Principles:"
if [ -f "$WORKSPACE/MEMORY.md" ]; then
  P_COUNT=$(grep -c "^### P[0-9]" "$WORKSPACE/MEMORY.md" 2>/dev/null || true)
  P_COUNT=$(printf '%s' "$P_COUNT" | tr -dc '0-9')
  P_COUNT=${P_COUNT:-0}
  if [ "$P_COUNT" -ge 7 ]; then
    check "$P_COUNT principles found in MEMORY.md" "ok"
  elif [ "$P_COUNT" -ge 1 ]; then
    check "Only $P_COUNT principles found (expected 7+)" "warn"
  else
    check "No principles found in MEMORY.md" "fail"
  fi
  # Context budget: check MEMORY.md size
  MEM_SIZE=$(wc -c < "$WORKSPACE/MEMORY.md" 2>/dev/null | tr -d ' ')
  MEM_KB=$(( MEM_SIZE / 1024 ))
  if [ "$MEM_SIZE" -le 5120 ]; then
    check "MEMORY.md size: ${MEM_KB}KB (within 5KB target)" "ok"
  elif [ "$MEM_SIZE" -le 8192 ]; then
    LESSONS_FILE="$WORKSPACE/memory/lessons.md"
    if [ -f "$LESSONS_FILE" ] && [ "$(wc -l < "$LESSONS_FILE" 2>/dev/null)" -gt 5 ]; then
      check "MEMORY.md size: ${MEM_KB}KB (lessons already in memory/lessons.md — size is ok)" "ok"
    else
      check "MEMORY.md size: ${MEM_KB}KB — consider moving lessons to memory/lessons.md" "warn"
    fi
  else
    check "MEMORY.md size: ${MEM_KB}KB — too large, will slow every session boot" "fail"
  fi
else
  check "MEMORY.md not found" "fail"
fi
echo ""

# --- Gitignore ---
echo "🔒 Security:"
if [ -f "$WORKSPACE/.gitignore" ]; then
  if grep -q "^\.vault/" "$WORKSPACE/.gitignore" 2>/dev/null; then
    check ".vault/ is gitignored" "ok"
  else
    check ".vault/ is NOT gitignored" "warn"
  fi
  if grep -q "^\.secrets-map" "$WORKSPACE/.gitignore" 2>/dev/null; then
    check ".secrets-map is gitignored" "ok"
  else
    check ".secrets-map is NOT gitignored" "warn"
  fi
else
  check "No .gitignore found" "warn"
fi
echo ""

# --- Memory Search ---
echo "🔍 Memory Search:"
if command -v openclaw >/dev/null 2>&1; then
  # Check if memory_search is functional by looking for the memory index
  MEM_DB=$(find "$HOME/.openclaw" -name "memory*.sqlite" -o -name "memory*.db" 2>/dev/null | head -1)
  if [ -n "$MEM_DB" ]; then
    DB_SIZE=$(du -k "$MEM_DB" 2>/dev/null | cut -f1)
    check "Memory search index exists (${DB_SIZE}KB)" "ok"
  else
    check "No memory search index (optional — see docs.openclaw.ai/concepts/memory)" "ok"
  fi

  # Check if memory files are being indexed (non-empty memory dir)
  MEM_FILES=$(find "$WORKSPACE/memory" -name "*.md" -type f 2>/dev/null | wc -l | tr -d ' ')
  if [ "$MEM_FILES" -gt 0 ]; then
    check "$MEM_FILES memory files available for indexing" "ok"
  else
    check "No memory files found in memory/" "warn"
  fi

  # Check MEMORY.md size (should stay small for boot performance)
  if [ -f "$WORKSPACE/MEMORY.md" ]; then
    MEM_SIZE_KB=$(( $(wc -c < "$WORKSPACE/MEMORY.md" 2>/dev/null | tr -d ' ') / 1024 ))
    if [ "$MEM_SIZE_KB" -le 5 ]; then
      check "MEMORY.md is ${MEM_SIZE_KB}KB (target: < 5KB)" "ok"
    elif [ "$MEM_SIZE_KB" -le 8 ]; then
      # Check if lessons are already split out before warning
      LESSONS_FILE="$WORKSPACE/memory/lessons.md"
      if [ -f "$LESSONS_FILE" ] && [ "$(wc -l < "$LESSONS_FILE" 2>/dev/null)" -gt 5 ]; then
        check "MEMORY.md is ${MEM_SIZE_KB}KB (lessons already in memory/lessons.md — size is ok)" "ok"
      else
        check "MEMORY.md is ${MEM_SIZE_KB}KB — consider moving lessons to memory/lessons.md" "warn"
      fi
    else
      check "MEMORY.md is ${MEM_SIZE_KB}KB — too large, will slow every session boot" "fail"
    fi
  fi
else
  check "openclaw not in PATH — cannot verify memory search" "warn"
fi
echo ""

# --- Vault ---
echo "🔐 Vault:"
if [ -f "$WORKSPACE/scripts/vault.sh" ]; then
  check "vault.sh installed" "ok"
  BACKEND=$(bash "$WORKSPACE/scripts/vault.sh" backend 2>/dev/null | head -1 || echo "unknown")
  check "Passphrase backend: $BACKEND" "ok"
else
  check "vault.sh not installed (vault feature not enabled — this is fine if you skipped it)" "warn"
fi
echo ""

# --- Summary ---
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "   ✅ Passed: $PASS"
echo "   ⚠️  Warnings: $WARN"
echo "   ❌ Failed: $FAIL"
echo ""

if [ "$FAIL" -eq 0 ]; then
  echo "🎉 OpenCortex is installed and ready!"
  echo "   Work normally — the nightly distillation will start organizing your knowledge automatically."
else
  echo "⚠️  Some checks failed. Re-run the installer:"
  echo "   cd $WORKSPACE && bash skills/opencortex/scripts/install.sh"
fi


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)