Skill: robbyczgw-cla/lucid-dreamer (version 0.7.8)

--- name: lucid-dreamer version: 0.7.8 description: "Nightly AI memory reasoning system. Lucid runs every night while you sleep — it reads your daily notes and memory files, detects stale facts, unresolved todos, recurring problems, forgotten decisions, and can optionally perform aggressive cleanup and contradiction detection. Includes optional session debrief for quick end-of-day memory capture. Zero dependencies, no database, no embeddings. Just a cron job and markdown files. Use when you want your AI agent to automatically maintain and improve its long-term memory over time. Triggers on \"memory dreamer\", \"nightly memory review\", \"lucid\", \"auto memory\", \"memory cleanup\", \"memory hygiene\"." metadata: {"openclaw":{"requires":{"bins":["git","date","python3"],"note":"Set CLAWD_DIR to your workspace path before running. Auto-apply and aggressive cleanup are disabled by default. python3 required for trend detection."}}} --- # Lucid Dreamer 🧠 > *Your AI sleeps. Lucid dreams.* Lucid keeps your AI's memory clean. Every night, it reads what happened, checks what your AI already knows, and suggests what's outdated, missing, or forgotten. See `README.md` for full setup, `ARCHITECTURE.md` for internals, and `config/` for configuration. ## Quick Setup 1. Set your workspace path in the config: ```bash export CLAWD_DIR=/path/to/your/workspace ``` 2. Create a nightly cron job using OpenClaw's cron tool — run the prompt in `prompts/nightly-review.md` at 3 AM. Optional: add a lightweight session debrief cron around 18:00 using `prompts/session-debrief.md`. This is a faster daily capture pass than the nightly review — it reads today's daily note and writes durable decisions/facts straight into memory without creating a review report. 3. Wake up to a review report in `memory/review/YYYY-MM-DD.md`. 4. Approve or reject suggestions — Lucid tracks state in `memory/review/state.json`. ## Optional Session Debrief Cron Use `prompts/session-debrief.md` for a quick end-of-day memory pass around 18:00. It is designed to run faster than the nightly review: read today's daily note, capture durable decisions/facts/action items, and write them directly into memory. Recommended OpenClaw cron settings: ```bash openclaw cron add \ --name "lucid-debrief" \ --cron "0 18 * * *" \ --tz "Europe/Vienna" \ --model "your-preferred-model" \ # e.g. anthropic/claude-haiku-4-5 or opencode-go/minimax-m2.7 --session isolated \ --wake-mode now \ --message "$(cat prompts/session-debrief.md)" ``` What it does: - Reads today's daily note (`memory/TODAY.md`) - Captures key decisions, durable facts, and concrete action items - Writes those updates directly into long-term memory - Skips the full review report to stay quick and cheap ## Files - `prompts/nightly-review.md` — the main nightly review prompt - `prompts/session-debrief.md` — optional quick-capture prompt for ~18:00 - `config/` — thresholds and behavior settings - `examples/` — sample review output and state file ## Security **Files read at runtime:** - `MEMORY.md` — long-term agent memory summary - `USER.md` — user profile and preferences - Last 7 daily notes (`memory/YYYY-MM-DD.md`) **Files written at runtime:** - `memory/review/YYYY-MM-DD.md` — the generated review report - `memory/review/state.json` — approval/rejection tracking state **What this skill is designed to avoid:** - Avoid suggesting or outputting passwords, API keys, tokens, or other credentials in generated memory updates - Never accesses files outside the configured workspace directory - Never pushes to remote git automatically — all commits are local only, and no `git push` is performed unless you explicitly run it - Announce/notification delivery is opt-in and off by default — no messages are sent without explicit configuration **Recommendations:** - Set `CLAWD_DIR` explicitly in your environment to ensure the skill operates on the correct workspace - This skill reads workspace markdown files such as `MEMORY.md`, `USER.md`, and recent daily notes. Do not run it on a workspace containing unencrypted API keys or other secrets in plain markdown files. - Review generated reports before approving suggestions — Lucid proposes changes, but you remain in control

path: ARCHITECTURE.md
content: # Architecture

Technical deep-dive into Lucid's design decisions and internals.

## System Overview

Lucid is a **prompt-driven memory curator** that runs as a scheduled job. There is no runtime, no daemon, no persistent process. Each run is stateless except for the suggestion ledger (`state.json`).

```
┌─────────────────────────────────────────────────────────────────┐
│                        NIGHTLY REVIEW                           │
│                                                                 │
│  Inputs:                    Process:           Outputs:          │
│  ┌──────────────┐                             ┌──────────────┐ │
│  │ MEMORY.md    │──┐     ┌──────────────┐     │ review/      │ │
│  │ USER.md      │──┼────▶│  LLM + Prompt │────▶│ YYYY-MM-DD.md│ │
│  │ 7 daily notes│──┤     │  (Sonnet)     │     │ state.json   │ │
│  │ state.json   │──┘     └──────────────┘     │ .last-success│ │
│  └──────────────┘                             └──────────────┘ │
│                                                                 │
│  Anti-circular: review/*.md is NEVER read as input              │
└─────────────────────────────────────────────────────────────────┘
```

## Design Principles

### 1. Conservative by Default

The biggest risk in automated memory maintenance is **false authority** — the LLM writes polished, confident suggestions that are subtly wrong. Over time, bad suggestions leak into `MEMORY.md` and poison downstream recall.

Lucid mitigates this with:
- **2-day threshold**: Facts must appear on 2+ separate days before being suggested
- **Confidence labels**: Every suggestion tagged `high`, `medium`, or `low`
- **Human approval**: Nothing auto-applied in V1
- **Source citations**: Every claim linked to file + approximate line

### 2. Anti-Circular Reasoning

If the nightly review reads its own previous reviews, it creates a feedback loop:
1. Night 1: "Hermes seems stuck" (weak inference)
2. Night 2: Reads Night 1 review → "Hermes is definitely stuck" (reinforced)
3. Night 3: "Hermes has been stuck for 3 days" (snowball)

**Solution:** The prompt explicitly forbids reading `memory/review/*.md`. Only raw daily notes and curated memory are valid inputs.

### 3. Memory Classes

Not all facts are equal. A family member's birthday needs much higher confidence to change than a project's current status:

| Class | Examples | Required Confidence | Change Frequency |
|-------|----------|-------------------|------------------|
| Personal/Family | Birthdays, relationships, preferences | HIGH | Rarely |
| Operational Policy | Security rules, workflow conventions | HIGH + explicit decision marker | Rarely |
| Project State | Status, current version, active branches | MEDIUM | Often |
| Open Loops | Todos, pending decisions | MEDIUM | Resolved or carried |
| Temporary/Tactical | Debug sessions, one-off incidents | Should NOT be promoted | N/A |

### 4. Idempotent Suggestions

Without tracking, the same suggestion appears every night with slightly different wording. Hash-based dedup fails because the LLM rephrases.

**Solution:** `state.json` tracks suggestions by human-assigned slug IDs with explicit status:

```json
{
  "id": "add-hermes-project",
  "firstSeen": "2026-03-16",
  "lastSeen": "2026-03-17",
  "status": "pending",
  "confidence": "high"
}
```

The prompt reads `state.json` before generating and skips any suggestion with `rejected` or `deferred` status.

**Future (V2):** Embedding-based similarity check for suggestions that the LLM assigns a new slug but which are semantically identical to an existing one.

## Negative Rules

The prompt includes explicit prohibitions to prevent common failure modes:

1. **No credentials in memory** — API keys, tokens, passwords must never be suggested for `MEMORY.md`
2. **No volatile URLs** — Dev server ports, temporary endpoints, debugging URLs
3. **No ephemeral session info** — Container IDs, process PIDs, temporary workarounds
4. **No output without evidence** — Every suggestion must cite a source file and line

## Suggestion Lifecycle

```
  New suggestion found
         │
         ▼
    ┌─────────┐
    │ pending  │ ← appears in nightly review
    └────┬────┘
         │
    ┌────┴────────────────┬──────────────┐
    ▼                     ▼              ▼
┌──────────┐      ┌───────────┐   ┌──────────┐
│ accepted │      │ rejected  │   │ deferred │
│ (applied)│      │ (gone)    │   │ (14 days)│
└──────────┘      └───────────┘   └────┬─────┘
                                       │
                                  after 14 days
                                       │
                                       ▼
                                  ┌─────────┐
                                  │ pending  │ (resurfaced once)
                                  └─────────┘
```

## Health Monitoring

`.last-success` contains an ISO timestamp updated after every successful run. If this file is stale (>36 hours), the cron has likely failed.

**Future (V1.5):** Morning send job validates `.last-success` before delivering context. If stale, it sends a warning instead.

## Failure Modes

| Risk | Severity | Mitigation |
|------|----------|------------|
| False authority (plausible but wrong suggestions) | HIGH | Conservative policy, human approval, source citations |
| Circular reasoning (reading own reviews) | HIGH | Anti-circular rule in prompt |
| Memory pollution (junk promoted to MEMORY.md) | HIGH | Negative rules, memory classes, confidence thresholds |
| Suggestion fatigue (too many suggestions, user ignores all) | MEDIUM | Output cap (~50 lines/section), idempotent ledger |
| Hallucinated patterns (sparse data → invented trends) | MEDIUM | 2-day threshold, evidence requirement |
| Silent cron failure | LOW | `.last-success` sentinel |

## Cost Model

| Model | Est. Input Tokens | Est. Output Tokens | Cost/Night |
|-------|------------------|--------------------|------------|
| Sonnet 4.6 | ~15,000 | ~3,000 | ~$0.07 |
| Opus 4.6 | ~15,000 | ~3,000 | ~$0.30 |
| Haiku 4.5 | ~15,000 | ~3,000 | ~$0.02 |

Costs scale with daily note volume. 7 days × ~300 lines = ~15k tokens typical input.

## Relationship to Embeddings

Lucid does **not** use embeddings. It reads files directly via the LLM's tool use (file read).

If your platform has embedding-based memory search (e.g., OpenClaw's `memory_search`), that system operates independently. Lucid complements it:

- **Embedding search** = real-time recall during conversations ("what did I say about X?")
- **Lucid** = offline analysis at night ("what's missing, stale, or forgotten in memory?")

They solve different problems and don't depend on each other.


path: CHANGELOG.md
content: # Changelog

All notable changes to Lucid will be documented in this file.

Format based on [Keep a Changelog](https://keepachangelog.com/).

## [Unreleased] — "Sharper Dreams"

### Planned
- **Memory Section Split**: MEMORY.md → `memory/sections/*.md` via `scripts/migrate_memory.py`. Selective loading, reduced context bloat.

## [0.7.8] - 2026-04-01

### Fixed
- Removed CLAWD_DIR from env metadata entirely — mentioned in note only, eliminating optional/required mismatch

## [0.7.7] - 2026-04-01

### Fixed
- Cleaned up CHANGELOG (removed debugging history, consolidated v0.7.x entries)

## [0.7.0] - 2026-04-01

### Added
- **Session Debrief**: Documented optional ~18:00 quick-capture cron (`prompts/session-debrief.md`) in `SKILL.md` and `README.md`. Reads today's daily note and writes key decisions/facts directly to memory — faster than nightly review.
- **Contradiction Detection**: Step 6b in nightly review scans MEMORY.md vs last 7 daily notes for version/status/existence/value/decision conflicts. Outputs `## ⚡ Contradictions Detected` section. High-confidence factual contradictions auto-apply.

### Changed
- **Auto-apply config**: Added factual contradictions as auto-apply category in `config/auto-apply.md`.

## [0.6.7] - 2026-03-30

### Security
- **CLAWD_DIR hard requirement:** `nightly-review.md` now validates `CLAWD_DIR` is set before any file operations and aborts with a clear error if missing — eliminates silent fallback to cwd causing unintended reads/writes/commits in unknown directories
- **Removed `${CLAWD_DIR:-.}` fallback pattern** from all three git commit calls — replaced with strict `${CLAWD_DIR}` (no cwd fallback)
- **Trend script workspace fix:** `trend_detection.py` invocation now passes `${CLAWD_DIR}` instead of `$(pwd)` for consistent workspace resolution

## [0.6.6] - 2026-03-30

### Security
- **Shell injection hardening:** Git commit messages in `prompts/nightly-review.md` are now static strings instead of AI-generated dynamic content, eliminating potential shell injection via indirect prompt injection from markdown notes.

## [0.6.6] - 2026-03-30

### Fixed
- Version bump (previous version already published)

## [0.6.5] - 2026-03-30

### Added
- **Pre-flight flush prompt** (`prompts/pre-flight-flush.md`): Optional cron at 02:45 Vienna time that appends today's session context to the daily memory file before Lucid runs at 03:00. Ensures Lucid always reads fresh data even when the main session wasn't manually flushed. Use `append` mode — never overwrites existing content.

- **Memory Index**: Auto-generated `memory/index.md` manifest with section descriptions and last-updated timestamps.
- **Memory Index**: Auto-generated `memory/index.md` manifest with section descriptions and last-updated timestamps.
- **Session Debrief** (`prompts/session-debrief.md`): Lightweight end-of-day quick-capture prompt (target: <2 min, <5k tokens).
- **Contradiction Detection**: Step 6b in nightly review — compares memory vs daily notes, classifies as factual vs judgment.
- **Memory Loading Guidance** (`prompts/memory-loading-guidance.md`): Instructions for selective section loading.

### Attribution (for planned features)
Architecture inspired by [ByteRover](https://github.com/openclaw/openclaw/pull/50848)'s Context Engine approach — hierarchical memory tree, after-turn learning, contradiction detection. Lucid adapts these as a zero-dependency skill.

## [0.6.6] - 2026-03-30

### Fixed
- Version bump (previous version already published)

## [0.6.5] - 2026-03-30

### Fixed
- Version bump (previous version already published)

## [0.6.4] - 2026-03-29

### Changed
- **README rewrite:** Cleaner structure, removed bloat, sectioned memory moved to Roadmap, no more false claims about unreleased features. Honest about what's shipped vs planned.

## [0.6.3] - 2026-03-29

### Fixed
- **README honesty:** Sectioned Memory architecture section now clearly marked as "Planned / upcoming" instead of implying it's released. Features exist as experimental scripts but are not part of the default workflow.

## [0.6.2] - 2026-03-29

### Fixed
- Removed fake Telegram token from example review file

## [0.6.1] - 2026-03-29

### Fixed
- **Config defaults now match documentation:** `autoApply.enabled`, `aggressiveCleanup.enabled`, and `announceOnDelivery` all default to `false` in shipped config. Previously shipped as `true`, contradicting README/SKILL.md claims of "opt-in, default off."
- Removed hardcoded notification channel from default config

## [0.6.0] - 2026-03-29

### Added
- **Aggressive Cleanup** (opt-in): When enabled via `aggressiveCleanup.enabled` in config, Lucid auto-removes resolved Open Loops, closed Blockers, and confirmed-stale entries from MEMORY.md. Each removal is a separate git commit for easy rollback. Removed items are listed in the review file under `## 🗑️ Removed (Auto-Cleanup)` with git hashes. Default: off.

### Changed
- `prompts/nightly-review.md`: New Step 7a for aggressive cleanup workflow
- `config/auto-apply.md`: Added aggressive cleanup section with criteria and rollback docs

## [0.5.0] — 2026-03-27

### Fixed
- Portability fixes: removed hardcoded workspace paths, fixed config loading in `trend_detection.py` to look relative to script dir first, fixed loop bug where `args.days` (could be None) was used instead of resolved `days` variable, cleared hardcoded telegram topic from default config

## [0.4.0] — 2026-03-25

### Added
- **Trend Detection** module (`scripts/trend_detection.py`) — analyzes patterns across 14 days of daily notes:
  - **Recurring Issues** — flags topics/problems appearing on 3+ separate days (e.g., "backup failed", "tailscale down")
  - **Stale Project Detection** — finds projects in MEMORY.md not mentioned in any daily note for 30+ days
  - **Escalated Patterns** — detects the same lesson/mistake appearing in 3+ daily notes (repeated mistakes not yet fixed)
- New `## Trends` section in nightly review output (`memory/review/YYYY-MM-DD.md`)
- Trend history tracking in `state.json` under `trends` key (accumulates across runs, keeps last 30 entries)
- Nightly review prompt updated with Step 7b to invoke trend detection script
- Example review and state files updated with trend output samples

### Technical Details
- Standalone Python script with no external dependencies (stdlib only)
- Uses difflib SequenceMatcher for fuzzy clustering of similar issues/lessons
- Configurable via CLI flags: `--days`, `--stale-days`, `--min-recurrence`

## [0.3.0] — 2026-03-21

### Added
- **Extended Auto-Apply categories** (5 new safe classes):
  - Infrastructure facts — cron IDs, script paths, service names, port assignments
  - Lessons Learned — purely factual technical lessons
  - Model Strategy — agent counts, new agent entries when fully documented
  - Closed Open Loops — remove resolved items with explicit closure signal on 2+ days
  - Stale project status — update planning/in-progress to live when URL + service confirmed
- **`config/auto-apply.md`** — configurable file to enable/disable auto-apply categories per project. Edit it to go conservative, aggressive, or anywhere in between.
- **Stricter confidence gate**: medium/low confidence items never auto-applied (only high)
- Prompt reads `config/auto-apply.md` at runtime if present (Step 4b)

## [0.2.0] — 2026-03-18

### Added
- **Auto-Apply** for high-confidence, low-risk suggestions:
  - Version numbers in skill/plugin tables
  - Stale Cron IDs where removal is documented
  - New project entries with complete details (2+ days mentioned)
- **Git integration**: auto-apply commits to local git with descriptive message (`dreamer: auto-apply — DESCRIPTION`)
- **Rollback**: `git revert <hash>` to undo any auto-applied change
- Review file now has `## ✅ Auto-Applied` section showing what was applied automatically

### Changed
- Review format updated: auto-applied changes shown separately from pending decisions
- State ledger: auto-applied suggestions tracked as `accepted` immediately

### Never Auto-Applied (by design)
- Belief Updates, Key Decisions, Family/personal facts
- Model Strategy preferences
- Anything with uncertainty

## [0.1.0] — 2026-03-16

### Added
- Initial release of Lucid V1
- Nightly review prompt with 6 analysis categories:
  - Candidate Updates, Open Loops, Blockers, Belief Updates, Stale Facts, Duplicate/Merge Suggestions
- Suggestion ledger (`state.json`) with status tracking (pending/accepted/rejected/deferred)
- Health sentinel (`.last-success` timestamp)
- Anti-circular reasoning rule (never reads own previous reviews)
- Decision policy with 2-day threshold, confidence labels, output cap
- Negative rules (no credentials, tokens, volatile URLs in memory)
- Memory class awareness (personal facts vs project state vs policies)
- Example review output and state ledger
- Full architecture documentation

### Design process
- Concept reviewed by 4 AI agents across 2 rounds:
  - Round 1: Codex (GPT-5.3) + Claude Code (Sonnet 4.6) — pragmatic + architectural feedback
  - Round 2: Codex (GPT-5.4) + Claude Code (Opus 4.6) — deeper failure mode analysis
- Key insights incorporated:
  - State ledger for idempotency (GPT-5.4)
  - Anti-circular reasoning (Opus)
  - Memory classes (GPT-5.4)
  - "Curator, not journalist" framing (GPT-5.4)
  - Drop morning send from V1 (both)


path: README.md
content: # Lucid 🧠

> *Your AI sleeps. Lucid dreams.*

Lucid keeps your AI's memory clean. Every night, it reads what happened, checks what your AI already knows, and tells you what's outdated, missing, or forgotten.

No database. No embeddings. No dependencies. Just a prompt, a cron job, and markdown files.

## The Problem

AI agents forget. Between sessions, context is lost. `MEMORY.md` gets stale. Todos slip through the cracks. Nobody notices until it's too late.

Lucid fixes this automatically. While you sleep, it:

- 🔍 Finds facts worth remembering that aren't saved yet
- ⏳ Spots todos and decisions that were never resolved
- 🚧 Flags problems that keep coming back
- 🔄 Tracks when your opinions or plans changed
- 🗑️ Catches outdated entries in your memory files
- 📈 Detects recurring issues, stale projects, and repeated mistakes over 14 days
- 🧹 Optionally removes resolved items automatically (v0.6.0+)

You wake up to a short report. Approve what's useful, reject what's not. Done.

## How It Works

```
Daily Notes (7 days)     MEMORY.md + USER.md
         \                    /
          \                  /
       ┌──────────────────────┐
       │   Nightly LLM Review │  ← runs at 3 AM
       └──────────────────────┘
                  │
          ┌───────┴───────┐
          │               │
    review file        state.json
   (for you to read)   (tracks suggestions)
          │
          ▼
    you approve/reject
          │
          ▼
    memory updated ✅
```

**Cost:** One LLM call per night (~$0.20 with Sonnet, less with Haiku).

## Quick Start

### With OpenClaw

```bash
mkdir -p memory/review

openclaw cron add \
  --name "lucid" \
  --cron "0 3 * * *" \
  --tz "Your/Timezone" \
  --model "anthropic/claude-sonnet-4-6" \
  --announce \
  --session isolated \
  --timeout-seconds 120 \
  --message "$(cat prompts/nightly-review.md)"
```

Check results in the morning:
```bash
cat memory/review/YYYY-MM-DD.md
```

Tell your agent what to accept or reject. It handles the rest.

### Optional: Pre-flight Flush (Recommended)

Add a cron at 02:45 Vienna time using `prompts/pre-flight-flush.md` as the message. This ensures your daily memory file is up-to-date before Lucid runs at 03:00, even if you forgot to manually flush your session.

Cron schedule: `45 2 * * *` tz: Europe/Vienna

### Optional: Session Debrief at ~18:00

Session debrief is a quick daily capture pass that runs before the full nightly review. It is meant for a fast end-of-day sweep: read today's daily note, pull out key decisions and durable facts, and write them directly into memory without generating a review report.

Recommended OpenClaw cron:

```bash
openclaw cron add \
  --name "lucid-debrief" \
  --cron "0 18 * * *" \
  --tz "Europe/Vienna" \
  --model "your-preferred-model" \  # e.g. anthropic/claude-haiku-4-5 or opencode-go/minimax-m2.7
  --session isolated \
  --wake-mode now \
  --message "$(cat prompts/session-debrief.md)"
```

Why use it:
- Faster and cheaper than the nightly review
- Reads today's daily note only
- Writes key decisions/facts directly to memory
- Keeps memory fresher before the 03:00 full review

### Without OpenClaw

Lucid is just a prompt. Copy `prompts/nightly-review.md`, give it to any LLM that can read and write files, and run it on a schedule.

## What You Need

```
your-workspace/
├── MEMORY.md              # Your AI's long-term memory
├── USER.md                # User profile (optional but helpful)
└── memory/
    ├── 2026-03-15.md      # Daily notes (Lucid reads last 7 days)
    ├── 2026-03-16.md
    └── review/            # Lucid writes here
        ├── 2026-03-17.md  # The review
        ├── state.json     # Suggestion tracker
        └── .last-success  # Health check timestamp
```

No vector database, no embeddings, no SQLite, no external APIs, no runtime dependencies.

## Features

### Suggestion Tracking

Every suggestion gets tracked in `state.json`:

- **pending** → Waiting for your review
- **accepted** → Applied to memory
- **rejected** → Won't come back
- **deferred** → Come back in 14 days

### Auto-Apply (opt-in)

Automatically apply high-confidence changes without review. Enable in `config/lucid.config.json`:

```json
"autoApply": { "enabled": true }
```

Configure categories in `config/auto-apply.md`:
- Version numbers, new project entries, infrastructure facts, lessons learned, closed open loops, stale project status, factual contradictions

Never auto-applied (hardcoded): belief updates, key decisions, family/personal facts, anything medium/low confidence.

All auto-applied changes are git-committed with a `dreamer: auto-apply —` prefix. Undo with `git revert`.

### Aggressive Cleanup (v0.6.0, opt-in)

Go beyond flagging — actually **remove** resolved entries from memory. Enable in `config/lucid.config.json`:

```json
"aggressiveCleanup": { "enabled": true }
```

Removes resolved Open Loops, closed Blockers, and confirmed-stale entries — but only with **high confidence** and explicit closure evidence from the last 7 days.

Each removal is a **separate git commit** listed in the review under `## 🗑️ Removed (Auto-Cleanup)` with the commit hash. Undo: `git revert <hash>`.

### Trend Detection (v0.5.0)

Analyzes patterns across the last 14 days:

- **🔁 Recurring Issues** — same problem on 3+ days
- **🕸️ Stale Projects** — in MEMORY.md but not mentioned for 30+ days
- **⚠️ Escalated Patterns** — same mistake in 3+ daily notes

Standalone usage:
```bash
python3 scripts/trend_detection.py \
  --workspace /path/to/workspace \
  --days 14 \
  --stale-days 30 \
  --state memory/review/state.json
```

## Safety

- Only suggests new facts if mentioned on **2+ separate days**
- Only flags stale entries with **clear** evidence
- Never suggests passwords, API keys, tokens, or temp debug info
- Auto-apply/cleanup only on **high confidence**
- Every change is git-committed and revertable
- ⚠️ Don't run on workspaces with unencrypted secrets in markdown files

## Inspiration

- [Honcho.dev](https://honcho.dev) — Background "dreaming" over conversations
- [Gigabrain](https://github.com/legendaryvibecoder/gigabrain) — World model with beliefs and suggestion ledger
- [Nuggets](https://github.com/NeoVertex1/nuggets) — Fact promotion to permanent memory

## Roadmap

- [x] Nightly review, state ledger, human approval
- [x] Automatic delivery via announce
- [x] Auto-apply with configurable categories (git-backed)
- [x] Trend Detection (recurring issues, stale projects, escalated patterns)
- [x] Aggressive Cleanup (opt-in, git-backed rollback)
- [ ] Sectioned memory (`memory/sections/*.md` with selective loading)
- [x] Session debrief (lightweight end-of-day capture)
- [x] Contradiction detection (memory vs daily notes)
- [ ] Embedding-based dedup
- [ ] Weekly consolidation

## License

MIT


path: config/auto-apply.md
content: # Auto-Apply Configuration

This file defines which categories Lucid is allowed to auto-apply without human review.

Only changes with **HIGH confidence** are ever auto-applied — regardless of category.
Medium and low confidence always require human approval.

When sectioned memory is enabled, Lucid should apply safe edits directly to the relevant `memory/sections/*.md` file(s) and then update the matching `Last Updated` value in `memory/index.md`.
If `memory/sections/` does not exist yet, fall back to editing `MEMORY.md`.

## ✅ Enabled Categories

Edit this list to match your project's needs.

### Safe (factual, no opinions)
- **Version numbers** — skill/plugin tables clearly outdated
- **Stale Cron IDs** — cron was explicitly removed/replaced in notes
- **Service/port deletions** — clearly documented as removed
- **New project entries** — complete details (URL, repo, path, service) mentioned on 2+ days
- **Infrastructure facts** — new cron IDs, script paths, service names, port assignments
- **Lessons Learned** — purely factual technical lessons (not preferences or opinions)
- **Model Strategy** — agent counts, new agent entries when clearly documented with model + alias
- **Closed Open Loops** — remove resolved items when explicit closure signal exists on 2+ days
- **Stale project status** — update "planning"/"in progress" to "live" when URL + service confirmed on 2+ days
- **Factual contradictions** — objective memory-vs-note conflicts where newer daily note evidence clearly supersedes the old value and is safe to auto-apply

## ❌ Never Auto-Apply (hardcoded)

These are always manual regardless of confidence:

- Belief Updates (opinions, model preferences, strategy changes)
- Key Decisions
- Family/personal facts
- User preferences or communication style
- Judgment contradictions
- Anything you are uncertain about
- Anything with medium or low confidence

## 🗑️ Aggressive Cleanup (opt-in)

When `aggressiveCleanup.enabled` is `true` in `lucid.config.json`, Lucid will also **remove** entries from long-term memory that meet ALL of these criteria:

- The item is an Open Loop, Blocker, or factual entry
- Daily notes from the last 7 days contain **explicit closure signal** (e.g., "done", "fixed", "deployed", "removed", "cancelled", "no longer needed")
- The closure signal appears on **2+ separate days** OR is unambiguously final (e.g., "project deleted", "service decommissioned")
- Confidence is HIGH

**What gets removed:**
- Open Loops with confirmed resolution
- Blockers that are no longer blocking (fix deployed, workaround in place)
- Infrastructure entries for removed services/ports/crons
- Project entries for abandoned/deleted projects (only if explicitly stated)

**What NEVER gets removed (even with aggressive cleanup on):**
- Key Decisions (historical record)
- Belief Updates (trajectory matters)
- Family/personal facts
- Lessons Learned (knowledge is permanent)
- Active projects (even if not mentioned recently)

**Rollback:** Every removal is a separate git commit. Use `git revert <hash>` to restore any removed entry.

## Target Files

When a safe change is accepted, prefer these targets:
- `memory/sections/identity.md`
- `memory/sections/operations.md`
- `memory/sections/projects.md`
- `memory/sections/infrastructure.md`
- `memory/sections/skills.md`
- `memory/sections/decisions.md`
- `memory/sections/lessons.md`
- `memory/sections/models.md`
- `memory/index.md` (timestamp updates only)

Fallback target when sectioned memory is unavailable:
- `MEMORY.md`

## Customization Examples

**Minimal setup** (only version numbers + new projects):
Remove all categories except "Version numbers" and "New project entries".

**Aggressive setup** (trust the AI more):
Add "Open Loops closure", "Stale project status", and "Factual contradictions" — but watch for false positives.

**Conservative setup** (human reviews everything):
Remove all categories. Lucid will still generate suggestions but never apply them.


path: config/lucid.config.json
content: {
  "_comment": "Lucid configuration \u2014 all features can be enabled/disabled per preference. Defaults are conservative (off).",
  "trends": {
    "enabled": true,
    "recurringIssues": {
      "enabled": true,
      "lookbackDays": 14,
      "minOccurrences": 3
    },
    "staleProjects": {
      "enabled": true,
      "staleAfterDays": 30
    },
    "escalatedPatterns": {
      "enabled": true,
      "minOccurrences": 3
    }
  },
  "autoApply": {
    "enabled": false,
    "categories": {
      "versionNumbers": true,
      "staleCronIds": true,
      "servicePortDeletions": true,
      "newProjectEntries": true,
      "infrastructureFacts": true,
      "lessonsLearned": true,
      "modelStrategy": true,
      "closedOpenLoops": true,
      "staleProjectStatus": true
    },
    "_note": "Auto-apply is opt-in. Set enabled: true to activate. Categories control what gets auto-applied when enabled."
  },
  "review": {
    "maxLinesPerSection": 50,
    "confidenceThreshold": "high",
    "requireMultipleDays": true
  },
  "_comment_topic": "Set this to your notification target, e.g. telegram:-1234567890:topic:5. Leave null to use main chat.",
  "notifications": {
    "announceOnDelivery": false,
    "channel": null,
    "topic": null
  },
  "aggressiveCleanup": {
    "enabled": false,
    "_comment": "When enabled, Lucid auto-removes resolved Open Loops, closed Blockers, and confirmed-stale entries from MEMORY.md instead of just flagging them. Removals are git-committed for easy rollback. Default: off."
  }
}

path: examples/index-template.md
content: # Memory Index

Generated from `MEMORY.md` on YYYY-MM-DD by `scripts/migrate_memory.py`.

Read this file first. Always load `sections/identity.md` and `sections/operations.md`, then load other section files only when the task needs them.

| Section | File | Description | Last Updated | Default Load |
|---|---|---|---|---|
| Identity | sections/identity.md | Personal identity, household context, and stable human facts. | YYYY-MM-DD | always |
| Operations | sections/operations.md | Operational policy, cron jobs, workflows, and maintenance rules. | YYYY-MM-DD | always |
| Projects | sections/projects.md | Active projects, repos, URLs, and current project state. | YYYY-MM-DD | on-demand |
| Infrastructure | sections/infrastructure.md | Servers, services, ports, paths, and infrastructure facts. | YYYY-MM-DD | on-demand |
| Decisions | sections/decisions.md | Key decisions, reversals, and rationale worth keeping. | YYYY-MM-DD | on-demand |
| Lessons | sections/lessons.md | Durable technical lessons learned. | YYYY-MM-DD | on-demand |
| Models | sections/models.md | Model choices, providers, agents, and routing notes. | YYYY-MM-DD | on-demand |
| Skills | sections/skills.md | Published skills, plugins, and package metadata. | YYYY-MM-DD | on-demand |

## Maintenance Notes

- Update `Last Updated` whenever a section file changes.
- Keep descriptions short and explicit so agents know what to load.
- Add new rows when you introduce new section files.


path: examples/review-sample.md
content: # Memory Review — 2026-03-16

## Candidate Updates

### 1. Atlas Agent — New Infrastructure (high confidence)
- **Add to Projects section:** `atlas-node` VM (Tailscale `10.0.0.42`) running Atlas agent with GPT-5.4 via OAuth, systemd service `atlas-gateway.service`, lightweight vector memory enabled
- **Source:** `memory/2026-03-15.md` (Atlas Agent Setup) + `memory/2026-03-13.md` (initial install)
- **Context:** Replaces `old-orbit` (retired 2026-03-13), `Pilot` vs `Atlas` decision still pending as of 2026-03-15

### 2. atlas-webchat Project (medium confidence)
- **Add to Projects section:** `atlas-webchat` — Next.js 15 + FastAPI mock backend, repo `alexchen-labs/atlas-webchat`, deployed on atlas-node ports 8444/8445 (currently disabled), **no real Atlas agent connected yet**
- **Source:** `memory/2026-03-16.md` (atlas-webchat project section)

### 3. Morning Briefing Model — Pending Change (medium confidence)
- **Update Model Strategy table:** `Morning Briefing` row — currently shows Kimi, but 2026-03-14 benchmarks found Sonnet wins. Pending Alex decision.
- **Source:** `memory/2026-03-14.md` (~"Morning Briefing Model" section)

### 4. Backup System Overhaul (high confidence)
- **Update MEMORY.md:** Add note that `scripts/backup-remote.sh` was rewritten 2026-03-15 to use native `openclaw backup create`. Old split-workspace + separate-config approach replaced.
- **Source:** `memory/2026-03-15.md` (Backup System section)

### 5. web-search-plus Skill Version Mismatch (medium confidence)
- **MEMORY.md Published Skills table** shows `web-search-plus 2.7.2` but notes show it was bumped to 2.9.0 (2026-03-12) and the plugin to 1.2.2 (2026-03-11). Table is stale.
- **Source:** `memory/2026-03-12.md` (web-search-plus v2.9.0) + `memory/2026-03-11.md` (plugin v1.2.2)

### 6. QUERY_API_KEY TODO (low confidence)
- **Lessons Learned / Operational:** `QUERY_API_KEY` still missing from `skills/web-search-plus/config.json` — noted as TODO since 2026-03-12. Worth adding as a formal TODO or note.
- **Source:** `memory/2026-03-12.md` (web-search-plus v2.9.0 section, end)

---

## Open Loops

### 1. Pilot vs Atlas Decision
- **Started:** 2026-03-15 — Plan was "test Atlas this week → if stable, shut down Pilot"
- **Missing:** No closure signal. Pilot still running (6€/mo). Atlas tested on 2026-03-15/16 but decision not final.
- **Source:** `memory/2026-03-15.md` ("Decision: Pilot vs Atlas")

### 2. atlas-webchat — Real Agent Integration
- **Started:** 2026-03-16 — Mock backend only, no real Atlas agent connected
- **Missing:** Actual integration + UI polish ("still rough" per Alex)
- **Source:** `memory/2026-03-16.md` ("Next Steps")

### 3. Skill Selection — Life OS / GitClaw / others
- **Started:** 2026-03-12 brainstorm → carried through 2026-03-15/16
- **Missing:** No decision made after 4+ days of discussion
- **Source:** `memory/2026-03-16.md` (Skill Ideas section) + `memory/2026-03-15.md` (Top 3 section)

### 4. X/Twitter Login on browser-node
- **Started:** 2026-03-14 — Manual login via noVNC needed, bot detection blocks automated flow
- **Missing:** Still not logged in as of 2026-03-15
- **Source:** `memory/2026-03-15.md` + `memory/2026-03-14.md`

### 5. awesome-mcp-servers PR #3156
- **Started:** 2026-03-13 — Submitted, waiting on merge after AAA score (2026-03-14)
- **Missing:** No confirmation of merge
- **Source:** `memory/2026-03-14.md` (web-search-plus-mcp section)

### 6. Morning Briefing Model Switch (Kimi → Sonnet)
- **Started:** 2026-03-14 — Sonnet identified as winner but not yet changed
- **Missing:** Config update never happened; explicit decision pending
- **Source:** `memory/2026-03-14.md` (Morning Briefing Model section)

### 7. agent-chronicle 0.6.2 → ClawHub publish
- **Started:** 2026-03-10 (Open TODOs) — local ahead of ClawHub 0.6.0
- **Missing:** Still listed as TODO across 6+ days with no resolution
- **Source:** `memory/2026-03-10.md` + `memory/2026-03-12.md`

### 8. personas --force update to 2.2.6
- **Started:** 2026-03-10 — local 2.2.4, ClawHub 2.2.6
- **Missing:** Never resolved despite appearing in every TODO list
- **Source:** `memory/2026-03-10.md` through `memory/2026-03-12.md`

### 9. Partner Follow-up (Jamie / Casey)
- **Started:** 2026-03-09 email sent, reminder cron set for 2026-03-16 09:00 Vienna
- **Missing:** As of 2026-03-15, no reply yet. Today (2026-03-16) is the follow-up date — status unknown.
- **Source:** `memory/2026-03-14.md` + `memory/2026-03-15.md`

### 10. Gigabrain Fork Decision
- **Started:** 2026-03-12 — "Tomorrow decide if we start"
- **Missing:** No decision noted in 2026-03-13/14/15/16 notes
- **Source:** `memory/2026-03-12.md` (Memory System Research section)

---

## Blockers

### 1. atlas-webchat UI quality ("still rough")
- Alex explicitly noted the UI is still rough after two code passes
- Mentioned on 2026-03-16 with no resolution in same session
- **Source:** `memory/2026-03-16.md`

### 2. Persistent low-priority TODOs (personas, agent-chronicle)
- `personas --force update to 2.2.6` and `publish agent-chronicle 0.6.2` appear repeatedly from 2026-03-10 through 2026-03-16 — never acted on
- Both are trivial 5-minute tasks that keep getting deprioritized
- **Source:** `memory/2026-03-10.md`, `2026-03-12.md`, `2026-03-14.md`, `2026-03-15.md`

---

## Belief Updates

### 1. Browser Tool Default Changed
- **Previously (2026-03-11):** `browser` tool used `profile: "browserless"` as explicit default
- **Now (2026-03-14+):** `browser-node` is the new default — no profile parameter needed
- AGENTS.md was updated; MEMORY.md references are consistent with this
- **Source:** `memory/2026-03-14.md` (Infrastructure Notes) + `memory/2026-03-11.md`

### 2. Old Orbit Server — Deleted
- **Previously:** Multiple references to `10.0.0.43` / `orbit.tail12345.ts.net` / `oldorbit.tail12345.ts.net` as experimental servers
- **Now:** Deleted by Alex 2026-03-13 — no active server at that IP
- MEMORY.md appears clean of these references already
- **Source:** `memory/2026-03-13.md` (end: "Old Orbit — deleted")

### 3. Vendor Trial — Evaluation Stopped
- **Previously (2026-03-10/15):** External agent vendor being tested, credits received
- **Now (2026-03-15):** Uninstalled after repeated disconnect issues, decided not a fit
- MEMORY.md doesn't mention it — no update needed
- **Source:** `memory/2026-03-15.md` (Vendor Trial section)

---

## Stale Facts

### 1. MEMORY.md Model Strategy Table — Morning Briefing
- Table shows `Morning Briefing → Kimi`
- 2026-03-14 benchmarks clearly show Sonnet wins Morning Briefing
- However: decision to switch is explicitly pending → confidence medium (don't change until Alex decides)
- **Source:** MEMORY.md "Model Strategy" section vs `memory/2026-03-14.md`

### 2. MEMORY.md Published Skills Table — Versions
- `web-search-plus 2.7.2` is stale → actually 2.9.0 (since 2026-03-12)
- `web-search-plus-plugin 1.0.1` stale → actually 1.2.2 (since 2026-03-11)
- These are high-confidence staleness findings
- **Source:** `memory/2026-03-12.md` + `memory/2026-03-11.md`

### 3. Partner Reminder Cron — One-shot, likely fired
- MEMORY.md still shows: "Reminder Cron: set for 2026-03-16 09:00 Vienna (ID: `demo-cron-1234`)"
- Today is 2026-03-16 — this cron has already fired (or was due to). Should be removed from MEMORY.md once the follow-up loop closes.
- **Source:** MEMORY.md Partner Follow-up section + `memory/2026-03-14.md`

---

## Trends

### 🔁 Recurring Issues
_Same issue/topic flagged on 3+ separate days in the last 14 days._

1. **tailscale down / disconnect** — appeared on 4 days (2026-03-10, 2026-03-12, 2026-03-14, 2026-03-15)
2. **personas --force update pending** — appeared on 7 days (2026-03-10, 2026-03-11, 2026-03-12, 2026-03-13, 2026-03-14, 2026-03-15, 2026-03-16)

### 🕸️ Possibly Stale Projects
_Projects in MEMORY.md not mentioned in any daily note for 30+ days._

_All tracked projects appear active._

### ⚠️ Escalated Patterns (Repeated Mistakes)
_Same lesson/mistake appearing in 3+ daily notes — pattern not yet broken._

_No escalated patterns detected._

---

## Duplicate/Merge Suggestions

### 1. Atlas Agent — Scattered Across Notes
- Atlas setup info appears across 2026-03-13 and 2026-03-15 daily notes
- No single coherent MEMORY.md entry exists yet for `atlas-node`
- Suggest adding a consolidated "Pilot / Atlas" section to MEMORY.md
- **Source:** `memory/2026-03-13.md` + `memory/2026-03-15.md`

### 2. Pilot section in MEMORY.md — Merge with Atlas when decided
- Current Pilot section is standalone; once the Pilot vs Atlas decision is made, these should be consolidated
- Low urgency until decision is final
- **Source:** MEMORY.md "Pilot" section


path: examples/state-sample.json
content: {
  "suggestions": [
    {
      "id": "add-hermes-node-project",
      "firstSeen": "2026-03-16",
      "lastSeen": "2026-03-16",
      "status": "accepted",
      "section": "candidate-updates",
      "summary": "Add hermes-node container as project entry",
      "targetSection": "Projects",
      "confidence": "high"
    },
    {
      "id": "update-morning-briefing-model",
      "firstSeen": "2026-03-16",
      "lastSeen": "2026-03-16",
      "status": "rejected",
      "section": "candidate-updates",
      "summary": "Update Model Strategy: Morning Briefing Kimi \u2192 Sonnet",
      "targetSection": "Model Strategy",
      "confidence": "medium"
    },
    {
      "id": "open-loop-skill-selection",
      "firstSeen": "2026-03-16",
      "lastSeen": "2026-03-17",
      "status": "pending",
      "section": "open-loops",
      "summary": "Next skill to build \u2014 undecided for 5+ days",
      "targetSection": "Projects",
      "confidence": "high"
    }
  ],
  "lastRun": "2026-03-17T03:00:00+01:00",
  "lastSuccess": true,
  "trends": {
    "recurringIssues": [
      {
        "topic": "tailscale down / disconnect",
        "days": ["2026-03-10", "2026-03-12", "2026-03-14", "2026-03-15"],
        "count": 4
      }
    ],
    "staleProjects": [],
    "escalatedPatterns": [],
    "history": [
      {
        "date": "2026-03-17",
        "recurringCount": 1,
        "staleCount": 0,
        "escalatedCount": 0
      }
    ]
  },
  "lastTrendRun": "2026-03-17"
}

path: prompts/memory-loading-guidance.md
content: # Memory Loading Guidance

Use sectioned memory selectively.

## Load order
1. Read `memory/index.md` first.
2. Always load:
   - `memory/sections/identity.md`
   - `memory/sections/operations.md`
3. Load additional section files only when the current request needs them.

## How to choose extra sections
- Project, repo, deployment, roadmap work → `memory/sections/projects.md`
- Services, ports, cron jobs, infra debugging → `memory/sections/infrastructure.md`
- Published skills, plugins, packaging → `memory/sections/skills.md`
- Decisions, reversals, rationale → `memory/sections/decisions.md`
- Lessons learned / recurring mistakes → `memory/sections/lessons.md`
- Models, agents, routing strategy → `memory/sections/models.md`

## Rules
- Do not preload every section by default.
- If `memory/index.md` or `memory/sections/` does not exist yet, fall back to `MEMORY.md`.
- When you update a section file, also update its `Last Updated` date in `memory/index.md`.
- Prefer reading one relevant section over the whole memory set when context is narrow.


path: prompts/nightly-review.md
content: You are the Memory Dreamer — a nightly review agent. Your job is to analyze recent daily notes against long-term memory, auto-apply high-confidence safe changes, and surface remaining suggestions for human review.

## INSTRUCTIONS

### Step 0: Validate workspace
Run `echo "${CLAWD_DIR:-MISSING}"` to check if CLAWD_DIR is set.
- If output is `MISSING`: **ABORT immediately.** Write a single error file at `memory/review/ERROR.md` with content: "CLAWD_DIR is not set. Set CLAWD_DIR=/path/to/workspace and re-run." Then stop.
- If set: use `$CLAWD_DIR` as your base directory for all subsequent steps.
- If output is a valid path: confirm it looks like a real workspace directory (not `/`, not empty), then continue.

### Step 1: Determine today's date
Run `date +%Y-%m-%d` to get today's date. Store it as TODAY.

### Step 2: Read core memory files
Use the `read` tool to read these files:
1. `USER.md` — user profile
2. Memory source:
   - Prefer sectioned memory when available: read `memory/index.md` first, then read `memory/sections/identity.md` and `memory/sections/operations.md`
   - Read additional `memory/sections/*.md` files as needed to fully understand the current memory state
   - If `memory/sections/` does not exist, fall back to reading `MEMORY.md`

### Step 3: Read last 7 days of daily notes
Calculate the dates for the last 7 days from TODAY. For each date in YYYY-MM-DD format, attempt to read `memory/YYYY-MM-DD.md`. Skip missing files silently.

### Step 4: Read the suggestion ledger
Try to read `memory/review/state.json`. If it does not exist, start with an empty suggestions array.

### Step 4b: Read auto-apply config (optional)
If `config/auto-apply.md` exists in the Lucid directory, read it to determine which categories are enabled for auto-apply. If it doesn't exist, use the defaults defined in Step 7 below.

### Step 5: NEVER read previous reviews
Do NOT read any files matching `memory/review/*.md`. This prevents circular reasoning.

### Step 6: Analyze

Look for:
- New facts/decisions that should be in long-term memory but aren't
- Open todos/decisions that were never resolved
- Recurring problems across multiple days
- Intentional course corrections (belief updates)
- Memory entries that are now outdated
- Duplicate/overlapping entries across memory sections (or within `MEMORY.md` fallback)

### Step 6b: Contradiction Scan
Compare long-term memory against the last 7 days of daily notes and look specifically for contradictions.

Contradiction types:
1. **Version conflicts** — memory says one version, notes say it was upgraded/downgraded
2. **Status conflicts** — memory says planning/in progress, notes say deployed/live/removed
3. **Existence conflicts** — memory lists a service/project/cron that notes say was removed
4. **Value conflicts** — ports, paths, names, IDs, URLs, or counts changed
5. **Decision reversals** — memory records decision X, notes say the choice changed to Y

For each contradiction found:
- Quote both the memory text and the daily note evidence
- Classify it as:
  - **Factual contradiction** — objective and safe to auto-apply when confidence is HIGH
  - **Judgment contradiction** — involves preference, strategy, opinion, or ambiguous intent; human review required
- If the contradiction is factual, explicit, and current, include it in auto-apply consideration for Step 7
- If the contradiction is judgment-based or uncertain, include it in the review output only

### Step 7: AUTO-APPLY high-confidence safe changes

For suggestions that are HIGH confidence AND fall into these safe categories, apply them directly to long-term memory:
- **Version numbers** in published skill/plugin tables that are clearly outdated
- **Stale Cron IDs** where the note clearly states the cron was removed/replaced
- **Service/port deletions** clearly documented as removed
- **New project entries** with complete details (URL, repo, path, service) mentioned on 2+ days
- **Infrastructure facts** — new cron IDs, script paths, service names, port assignments
- **Lessons Learned** — purely factual technical lessons (not preferences or opinions)
- **Model Strategy** — agent counts, new agent entries when clearly documented with model + alias
- **Closed Open Loops** — remove resolved items when there is explicit closure signal on 2+ days
- **Stale project status** — update "planning" or "in progress" to "live" when URL + service are confirmed on 2+ days
- **Factual contradictions** — explicit memory-vs-note conflicts where the newer daily note evidence is objective and HIGH confidence

For AUTO-APPLY:
1. Edit the relevant `memory/sections/*.md` file(s) directly when sectioned memory exists; otherwise edit `MEMORY.md`
2. Update `memory/index.md` `Last Updated` values for any changed section files
3. Run `cd "${CLAWD_DIR}" && git add MEMORY.md memory/index.md memory/sections && git commit -m "dreamer: auto-apply"`
4. Track these in state.json with status `accepted`

Do NOT auto-apply:
- Belief Updates (opinions, model preferences, strategy)
- Key Decisions
- Family/personal facts
- Anything about Robby's preferences or communication style
- OpenCami status changes
- Judgment contradictions
- Anything you are uncertain about
- Anything with medium or low confidence

### Step 7a: Aggressive Cleanup (if enabled)

Read `config/lucid.config.json`. If `aggressiveCleanup.enabled` is `true`:

1. Scan Open Loops and Blockers from the previous review (or from MEMORY.md directly)
2. For each item, check if the last 7 days of daily notes contain an explicit closure signal
3. Closure signals include: "done", "fixed", "deployed", "merged", "removed", "cancelled", "resolved", "no longer needed", "erledigt", "gefixt", "fertig"
4. If HIGH confidence that the item is resolved:
   a. Remove it from the relevant memory file (`MEMORY.md` or `memory/sections/*.md`)
   b. Git commit: `cd "${CLAWD_DIR}" && git add MEMORY.md memory/sections && git commit -m "dreamer: cleanup"`
   c. Track in `state.json` with status `removed` and include the original text for reference
5. Add all removals to the review file under `## 🗑️ Removed (Auto-Cleanup)`

The removed section format:

```
## 🗑️ Removed (Auto-Cleanup)

| # | What | Why | Git Hash |
|---|------|-----|----------|
| 1 | Open Loop: OAuth regression blocker | Fixed in v2026.3.28, deployed 03-29 | abc1234 |
| 2 | Blocker: Discord Gateway Crasher | Plugin disabled, no crashes for 5 days | def5678 |

To undo: `git revert <hash>` or tell your agent "revert cleanup #1"
```

If `aggressiveCleanup.enabled` is `false` or not set, skip this step entirely.

### Step 7b: Run Trend Detection

Run the trend detection script to analyze patterns across the last 14 days:

```bash
python3 scripts/trend_detection.py \
  --workspace "${CLAWD_DIR}" \
  --date "$(date +%Y-%m-%d)" \
  --days 14 \
  --stale-days 30 \
  --state memory/review/state.json
```

Capture the stdout output — this is the `## Trends` section to include in the review file.
If the script fails or no daily notes exist, skip the Trends section gracefully.

### Step 8: Generate review file
Create `memory/review/TODAY.md` with:

```
# Memory Review — TODAY

## ✅ Auto-Applied
<!-- List what was automatically applied to long-term memory with brief description -->

## 🗑️ Removed (Auto-Cleanup)
<!-- Only present when aggressiveCleanup is enabled -->
<!-- List removed entries with: #, what was removed, why (closure evidence), git commit hash -->
<!-- "To undo: git revert <hash>" -->

## Candidate Updates (needs your decision)
<!-- Remaining proposals not auto-applied -->
<!-- Each with: confidence, what, source citation -->

## Open Loops
<!-- Todos/decisions with no closure signal -->

## Blockers
<!-- Recurring friction across 2+ days -->

## Belief Updates
<!-- "Previously X, now Y" — always manual, never auto-applied -->

## Stale Facts (needs your decision)
<!-- Non-safe stale entries -->

## ⚡ Contradictions Detected
<!-- Memory says X but daily notes say Y -->
<!-- Each with: type, classification, memory citation, daily-note citation, recommendation -->

## Trends
<!-- Output from trend_detection.py — recurring issues, stale projects, escalated patterns -->
<!-- If script produced no output, write: "No trends detected." -->
```

### Decision Policy (MANDATORY):
- Only suggest additions if mentioned on 2+ separate days OR explicitly marked as decision/fact
- Only flag stale if newer note CLEARLY supersedes with explicit evidence
- Only create open loop if unresolved action with no closure signal
- Confidence: high / medium / low only
- Hard cap: ~50 lines per section
- Memory Classes: family facts = very high threshold; project state = medium; operational = explicit decision marker required
- NEGATIVE RULES: NEVER add credentials, tokens, API keys, volatile URLs, temp debug info, ephemeral session info
- Skip suggestions with status `rejected` or `deferred` in state.json

### Step 9: Update state.json
Preserve all existing entries. Add new suggestions as `pending`. Auto-applied ones as `accepted`.

### Step 10: Write health sentinel
Write ISO timestamp to `memory/review/.last-success`

### IMPORTANT:
- You run in an isolated session with no prior context
- Use read tool for files, write tool to create/update files
- Be conservative with auto-apply — when in doubt, leave for human review
- Every suggestion needs source citation
- Keep backward compatibility: if sectioned memory has not been initialized yet, operate fully on `MEMORY.md`


path: prompts/pre-flight-flush.md
content: # Pre-Flight Flush

You are doing a lightweight pre-flight memory flush before Lucid's nightly review.

Goal: make sure Lucid's input files are current before the 03:00 run.

Important timing detail:
- This runs at 02:45 Vienna time
- At that point, the calendar date has already rolled over
- Lucid's nightly review still needs **yesterday's** daily file as the main source for the prior day
- If there has been any after-midnight activity, a **today** file may also exist and should be updated if needed

Be conservative:
- Only write factual things supported by files you can read
- Do not speculate, infer motives, or invent missing details
- Do not rewrite or remove existing content unless you are creating a missing file from scratch
- If a file already has content, append only genuinely new information
- Avoid duplicates and near-duplicates

## Steps

1. Get Vienna dates.

   On Linux:
   ```bash
   YESTERDAY=$(TZ=Europe/Vienna date -d yesterday +%Y-%m-%d)
   TODAY=$(TZ=Europe/Vienna date +%Y-%m-%d)
   ```

   On macOS:
   ```bash
   YESTERDAY=$(TZ=Europe/Vienna date -v-1d +%Y-%m-%d)
   TODAY=$(TZ=Europe/Vienna date +%Y-%m-%d)
   ```

2. Read yesterday's daily file if it exists:
   ```bash
   cat memory/$YESTERDAY.md 2>/dev/null
   ```

3. Check whether a today's file already exists from after-midnight activity:
   ```bash
   cat memory/$TODAY.md 2>/dev/null
   ```

4. Read recent session context from `memory/` and check what was written recently. Prefer factual sources created or updated late in the day or after midnight.

5. Treat **yesterday's file** as the primary target.
   - Determine whether `memory/$YESTERDAY.md` is missing or sparse.
   - Treat it as sparse if it has fewer than 200 characters of meaningful content.

6. For `memory/$YESTERDAY.md`:
   - If missing or sparse, write a basic factual summary of what is known about yesterday from available sources.
   - If it already has meaningful content, append only new context not already covered.
   - Preserve existing content exactly; use append mode only when adding to an existing file.

7. For `memory/$TODAY.md`:
   - Only touch it if the file already exists.
   - If it exists, append only factual after-midnight context that is not already covered.
   - Do not create a new today file just because the date rolled over.

## Output rules

- Yesterday's file is the main target Lucid needs for the 03:00 run.
- Today's file is optional and only updated if it already exists.
- Do not touch other memory files unless needed as read-only sources.
- Do not overwrite an existing non-sparse daily file.
- If there is nothing new to add, leave the file unchanged.
- Reply `NO_REPLY` when done.


path: prompts/session-debrief.md
content: # Session Debrief

You are the quick-capture memory pass. Run fast, stay narrow, and write only durable updates.

## Goal
In under 2 minutes and under 5k tokens, scan today's daily note and write any durable changes directly into the appropriate memory section files. Do **not** generate a review report.

## Scope
Only capture:
1. Explicit decisions
2. New factual information that changes long-term memory
3. Action items with deadlines or clear follow-up owners

Ignore everything else: chatter, brainstorming, temporary debugging details, duplicate notes, and vibes.

## Steps

### Step 1: Determine today's date
Run `date +%Y-%m-%d` and store it as TODAY.

### Step 2: Read the current daily note
Read `memory/TODAY.md`.
If the file does not exist, stop without making changes.

### Step 3: Load memory structure
- Read `memory/index.md` first.
- Read `memory/sections/identity.md` and `memory/sections/operations.md`.
- Read additional `memory/sections/*.md` files only if today's note suggests they are relevant.
- If `memory/index.md` or `memory/sections/` does not exist, fall back to reading `MEMORY.md` and stop after reporting that sectioned memory has not been initialized yet.

### Step 4: Extract only durable updates
Scan today's note for:
- **Decisions** — explicit choice, reversal, commitment, or settled direction
- **New facts** — changed versions, new services, ports, repos, project state, stable personal facts
- **Action items** — concrete next steps with a date, deadline, or owner

Skip:
- Temporary logs, shell snippets, raw stack traces
- One-off experiments with no follow-through
- Rewording existing memory without a real change
- Opinions unless they are an explicit durable decision

### Step 5: Write directly to section files
For each accepted update:
- Edit the most appropriate `memory/sections/*.md` file directly
- Keep edits small and local
- Preserve the file's structure and tone
- Update the matching `Last Updated` date in `memory/index.md`

If a needed section file does not exist yet but the index clearly implies it should, create it with a short heading and the new durable entry.

### Step 6: Commit only if something changed
If you changed any memory files, run:

```bash
git add memory/index.md memory/sections && git commit -m "debrief: quick capture for TODAY"
```

If nothing durable changed, do not create an empty commit.

## Hard rules
- Do **not** create a review report or summary file
- Do **not** scan older daily notes
- Do **not** invent new facts to make sections look complete
- Do **not** rewrite large sections when a line edit will do
- Do **not** touch unrelated files
- Be conservative: when in doubt, leave it for the nightly review


path: scripts/migrate_memory.py
content: #!/usr/bin/env python3
"""Lucid memory migration: split MEMORY.md into section files + index manifest.

Usage:
    python3 scripts/migrate_memory.py
    python3 scripts/migrate_memory.py --source path/to/MEMORY.md --output-dir memory/sections/
    python3 scripts/migrate_memory.py --dry-run
"""

from __future__ import annotations

import argparse
import re
from dataclasses import dataclass
from datetime import date
from pathlib import Path
from typing import Iterable


HEADER_RE = re.compile(r"^##\s+(.+?)\s*$", re.MULTILINE)
SLUG_RE = re.compile(r"[^a-z0-9]+")
DEFAULT_DESCRIPTION = "Migrated from MEMORY.md; refine this description after first review."
DESCRIPTION_HINTS = {
    "identity": "Personal identity, household context, and stable human facts.",
    "infrastructure": "Servers, services, ports, paths, and operational infrastructure facts.",
    "projects": "Active projects, repos, URLs, and current project state.",
    "skills": "Published skills, plugins, and related package metadata.",
    "decisions": "Key decisions, reversals, and rationale worth keeping long-term.",
    "lessons": "Technical lessons learned that should remain easy to reload.",
    "models": "Model choices, agent strategy, providers, and routing notes.",
    "operations": "Operational policy, cron jobs, workflows, and maintenance rules.",
}
ALWAYS_SECTIONS = {"identity", "operations"}


@dataclass
class Section:
    title: str
    slug: str
    body: str


def slugify(title: str) -> str:
    slug = SLUG_RE.sub("-", title.strip().lower()).strip("-")
    return slug or "section"


def extract_sections(text: str) -> list[Section]:
    matches = list(HEADER_RE.finditer(text))
    if not matches:
        raise ValueError("No level-2 (##) section headers found in source file.")

    sections: list[Section] = []
    for i, match in enumerate(matches):
        title = match.group(1).strip()
        start = match.end()
        end = matches[i + 1].start() if i + 1 < len(matches) else len(text)
        body = text[start:end].strip("\n")
        slug = slugify(title)
        sections.append(Section(title=title, slug=slug, body=body))
    return sections


def ensure_unique_slugs(sections: Iterable[Section]) -> list[Section]:
    seen: dict[str, int] = {}
    unique: list[Section] = []
    for section in sections:
        count = seen.get(section.slug, 0)
        if count:
            slug = f"{section.slug}-{count + 1}"
        else:
            slug = section.slug
        seen[section.slug] = count + 1
        unique.append(Section(title=section.title, slug=slug, body=section.body))
    return unique


def section_markdown(section: Section) -> str:
    body = section.body.strip()
    if body:
        return f"# {section.title}\n\n{body}\n"
    return f"# {section.title}\n\n<!-- Empty section created by migrate_memory.py -->\n"


def build_index(sections: list[Section], output_dir: Path, source: Path) -> str:
    today = date.today().isoformat()
    rows = [
        "| Section | File | Description | Last Updated | Default Load |",
        "|---|---|---|---|---|",
    ]
    for section in sections:
        description = DESCRIPTION_HINTS.get(section.slug, DEFAULT_DESCRIPTION)
        default_load = "always" if section.slug in ALWAYS_SECTIONS else "on-demand"
        rows.append(
            f"| {section.title} | sections/{section.slug}.md | {description} | {today} | {default_load} |"
        )

    return "\n".join(
        [
            "# Memory Index",
            "",
            f"Generated from `{source.name}` on {today} by `scripts/migrate_memory.py`.",
            "",
            "Read this file first. Always load `sections/identity.md` and `sections/operations.md`, then load other section files only when the current task needs them.",
            "",
            *rows,
            "",
            "## Maintenance Notes",
            "",
            "- Update `Last Updated` whenever a section file changes.",
            "- Refine section descriptions after migration so agents can route reads accurately.",
            "- Keep file names stable once other prompts start referencing them.",
            "",
        ]
    )


def main() -> int:
    parser = argparse.ArgumentParser(description="Split MEMORY.md into memory/sections/*.md files.")
    parser.add_argument("--source", default="MEMORY.md", help="Source MEMORY markdown file (default: MEMORY.md)")
    parser.add_argument(
        "--output-dir",
        default="memory/sections/",
        help="Directory for generated section files (default: memory/sections/)",
    )
    parser.add_argument("--dry-run", action="store_true", help="Show planned outputs without writing files")
    args = parser.parse_args()

    source = Path(args.source).expanduser().resolve()
    output_dir = Path(args.output_dir).expanduser().resolve()
    if not source.exists():
        raise SystemExit(f"Source file not found: {source}")

    text = source.read_text(encoding="utf-8")
    sections = ensure_unique_slugs(extract_sections(text))
    index_path = output_dir.parent / "index.md"

    if args.dry_run:
        print(f"[dry-run] source: {source}")
        print(f"[dry-run] output directory: {output_dir}")
        for section in sections:
            print(f"[dry-run] would write: {output_dir / f'{section.slug}.md'}")
        print(f"[dry-run] would write: {index_path}")
        return 0

    output_dir.mkdir(parents=True, exist_ok=True)
    index_path.parent.mkdir(parents=True, exist_ok=True)

    for section in sections:
        path = output_dir / f"{section.slug}.md"
        path.write_text(section_markdown(section), encoding="utf-8")

    index_path.write_text(build_index(sections, output_dir, source), encoding="utf-8")

    print(f"Wrote {len(sections)} section files to {output_dir}")
    print(f"Wrote manifest to {index_path}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())


path: scripts/trend_detection.py
content: #!/usr/bin/env python3
"""
Lucid Trend Detection — v0.4.0

Analyzes daily notes for recurring patterns, stale projects, and repeated mistakes.
Designed to be invoked by the nightly review prompt (or standalone).

Usage:
    python3 scripts/trend_detection.py --workspace /path/to/workspace [--memory MEMORY.md] [--days 14] [--state memory/review/state.json]

Output: Markdown trends section printed to stdout, state updates written to state.json.
"""

import argparse
import json
import os
import re
import sys
from collections import Counter, defaultdict
from datetime import datetime, timedelta
from pathlib import Path


# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------

def parse_date(s: str) -> datetime:
    return datetime.strptime(s, "%Y-%m-%d")


def date_range(end: datetime, days: int):
    """Yield YYYY-MM-DD strings for the last *days* days ending at *end* (inclusive)."""
    for i in range(days):
        yield (end - timedelta(days=i)).strftime("%Y-%m-%d")


def read_file(path: str) -> str | None:
    try:
        return Path(path).read_text(encoding="utf-8")
    except FileNotFoundError:
        return None


def normalise(text: str) -> str:
    """Lowercase, collapse whitespace."""
    return re.sub(r"\s+", " ", text.lower().strip())


# ---------------------------------------------------------------------------
# 1. Recurring Issues Detection
# ---------------------------------------------------------------------------

# Common issue-signal phrases (lowercase).  Extend as needed.
ISSUE_PATTERNS = [
    r"fail(?:ed|ure|s)?",
    r"error(?:s)?",
    r"broken",
    r"down",
    r"crash(?:ed|es)?",
    r"timeout(?:s)?",
    r"refused",
    r"stuck",
    r"bug(?:s)?",
    r"not working",
    r"issue(?:s)?",
    r"problem(?:s)?",
    r"missing",
    r"stale",
    r"disconnect(?:ed)?",
    r"unreachable",
    r"OOM|out of memory",
    r"permission denied",
    r"401|403|404|500|502|503",
]

# Compile a single big pattern.
_ISSUE_RE = re.compile(
    r"(?:^|\W)(" + "|".join(ISSUE_PATTERNS) + r")(?:\W|$)",
    re.IGNORECASE,
)


def _extract_issue_phrases(text: str, context_words: int = 4) -> list[str]:
    """Return short context snippets around issue-signal words.

    Works line-by-line to avoid cross-section contamination, and strips
    markdown headings / bullet markers for cleaner output.
    """
    phrases: list[str] = []
    for line in text.splitlines():
        # Skip headings and empty lines
        stripped = line.strip()
        if not stripped or stripped.startswith("#"):
            continue
        # Remove bullet/list markers
        stripped = re.sub(r"^[-*•]\s+", "", stripped)
        words = stripped.split()
        for i, w in enumerate(words):
            if _ISSUE_RE.search(w) or _ISSUE_RE.search(" ".join(words[max(0, i - 1):i + 2])):
                start = max(0, i - context_words)
                end = min(len(words), i + context_words + 1)
                snippet = " ".join(words[start:end])
                phrases.append(normalise(snippet))
    return phrases


def _cluster_phrases(all_phrases: dict[str, list[str]], min_days: int = 3) -> list[dict]:
    """
    Cluster similar issue phrases across days.

    all_phrases: {date: [phrase, ...]}
    Returns list of {topic, days: [date, ...], count}.
    """
    # Simple approach: extract a 3-gram fingerprint from each phrase
    # and cluster by shared n-grams.
    from difflib import SequenceMatcher

    # Flatten into (date, phrase) pairs
    dated: list[tuple[str, str]] = []
    for d, phrases in all_phrases.items():
        for p in phrases:
            dated.append((d, p))

    if not dated:
        return []

    # Greedy clustering: for each phrase find existing cluster with similarity > 0.55
    clusters: list[dict] = []  # {rep: str, dates: set, phrases: list}
    for date, phrase in dated:
        best_idx = -1
        best_score = 0.0
        for idx, cl in enumerate(clusters):
            score = SequenceMatcher(None, phrase, cl["rep"]).ratio()
            if score > best_score:
                best_score = score
                best_idx = idx
        if best_score >= 0.55 and best_idx >= 0:
            clusters[best_idx]["dates"].add(date)
            clusters[best_idx]["phrases"].append(phrase)
        else:
            clusters.append({"rep": phrase, "dates": {date}, "phrases": [phrase]})

    # Filter to those appearing on min_days+ distinct days
    results = []
    for cl in clusters:
        if len(cl["dates"]) >= min_days:
            # Pick shortest phrase as representative topic label
            topic = min(cl["phrases"], key=len)
            # Trim to max 60 chars
            if len(topic) > 60:
                topic = topic[:57] + "..."
            results.append({
                "topic": topic,
                "days": sorted(cl["dates"]),
                "count": len(cl["dates"]),
            })
    # Sort by count desc
    results.sort(key=lambda r: r["count"], reverse=True)
    return results


def detect_recurring_issues(daily_notes: dict[str, str], min_days: int = 3) -> list[dict]:
    """
    Scan daily notes for recurring issue patterns.

    daily_notes: {date_str: file_content}
    Returns list of {topic, days, count}.
    """
    all_phrases: dict[str, list[str]] = {}
    for date, content in daily_notes.items():
        phrases = _extract_issue_phrases(content)
        if phrases:
            all_phrases[date] = phrases
    return _cluster_phrases(all_phrases, min_days=min_days)


# ---------------------------------------------------------------------------
# 2. Stale Project Detection
# ---------------------------------------------------------------------------

def _extract_projects_from_memory(memory_text: str) -> list[str]:
    """
    Extract project names from MEMORY.md.

    Heuristics:
    - Lines under ## Projects / ## Active Projects / ## Projekte headings
    - Table rows with project-like names
    - Bold items in list entries (- **ProjectName** ...)
    """
    projects: list[str] = []

    # Strategy 1: Bold list items under project-ish headings
    in_project_section = False
    for line in memory_text.splitlines():
        stripped = line.strip()
        if re.match(r"^#{1,3}\s.*(project|projekt|active|repo)", stripped, re.IGNORECASE):
            in_project_section = True
            continue
        if re.match(r"^#{1,3}\s", stripped) and in_project_section:
            in_project_section = False
            continue
        if in_project_section:
            # Bold items: - **name** or | name |
            m = re.match(r"[-*]\s+\*\*(.+?)\*\*", stripped)
            if m:
                projects.append(m.group(1).strip())
                continue
            # Table rows (skip header separators)
            if stripped.startswith("|") and "---" not in stripped:
                cells = [c.strip().strip("*").strip() for c in stripped.split("|") if c.strip()]
                if cells:
                    projects.append(cells[0])

    # Deduplicate, preserve order
    seen: set[str] = set()
    unique: list[str] = []
    for p in projects:
        key = p.lower()
        if key not in seen and len(p) > 1:
            seen.add(key)
            unique.append(p)
    return unique


def detect_stale_projects(
    memory_text: str,
    daily_notes: dict[str, str],
    stale_days: int = 30,
    today: datetime | None = None,
) -> list[dict]:
    """
    Find projects in MEMORY.md not mentioned in recent daily notes.

    Returns list of {project, last_mention, days_since}.
    """
    if today is None:
        today = datetime.now()

    projects = _extract_projects_from_memory(memory_text)
    if not projects:
        return []

    results: list[dict] = []
    for project in projects:
        pattern = re.compile(re.escape(project), re.IGNORECASE)
        last_mention: str | None = None
        for date in sorted(daily_notes.keys(), reverse=True):
            if pattern.search(daily_notes[date]):
                last_mention = date
                break

        if last_mention is None:
            # Never mentioned in scanned window — flag with unknown last mention
            results.append({
                "project": project,
                "last_mention": "unknown (not in scanned window)",
                "days_since": stale_days + 1,
            })
        else:
            days_since = (today - parse_date(last_mention)).days
            if days_since >= stale_days:
                results.append({
                    "project": project,
                    "last_mention": last_mention,
                    "days_since": days_since,
                })

    results.sort(key=lambda r: r["days_since"], reverse=True)
    return results


# ---------------------------------------------------------------------------
# 3. Repeated Mistakes / Escalated Patterns
# ---------------------------------------------------------------------------

def _extract_lessons(text: str) -> list[str]:
    """
    Extract lesson-learned entries from a daily note.

    Looks for:
    - Sections named "Lessons", "Lesson Learned", "Takeaway", "Gelernt"
    - Bullet items under those sections
    - Lines containing "lesson:", "takeaway:", "note to self"
    """
    lessons: list[str] = []
    in_lesson_section = False

    for line in text.splitlines():
        stripped = line.strip()

        # Check for lesson section headers
        if re.match(
            r"^#{1,4}\s.*(lesson|takeaway|gelernt|erkenntn|mistake|learning)",
            stripped,
            re.IGNORECASE,
        ):
            in_lesson_section = True
            continue
        if re.match(r"^#{1,4}\s", stripped) and in_lesson_section:
            in_lesson_section = False
            continue

        # Bullet items in lesson section
        if in_lesson_section and re.match(r"^[-*]\s", stripped):
            lessons.append(normalise(stripped.lstrip("-* ")))
            continue

        # Inline lesson markers anywhere
        m = re.match(
            r"^[-*]?\s*(?:lesson|takeaway|note to self|gelernt)\s*:\s*(.+)",
            stripped,
            re.IGNORECASE,
        )
        if m:
            lessons.append(normalise(m.group(1)))

    return lessons


def detect_repeated_mistakes(daily_notes: dict[str, str], min_days: int = 3) -> list[dict]:
    """
    Find the same lesson/mistake appearing across 3+ daily notes.

    Returns list of {pattern, days, count}.
    """
    from difflib import SequenceMatcher

    # Collect all (date, lesson) pairs
    dated_lessons: list[tuple[str, str]] = []
    for date, content in daily_notes.items():
        for lesson in _extract_lessons(content):
            dated_lessons.append((date, lesson))

    if not dated_lessons:
        return []

    # Cluster similar lessons
    clusters: list[dict] = []
    for date, lesson in dated_lessons:
        best_idx = -1
        best_score = 0.0
        for idx, cl in enumerate(clusters):
            score = SequenceMatcher(None, lesson, cl["rep"]).ratio()
            if score > best_score:
                best_score = score
                best_idx = idx
        if best_score >= 0.6 and best_idx >= 0:
            clusters[best_idx]["dates"].add(date)
            clusters[best_idx]["lessons"].append(lesson)
        else:
            clusters.append({"rep": lesson, "dates": {date}, "lessons": [lesson]})

    results = []
    for cl in clusters:
        if len(cl["dates"]) >= min_days:
            pattern = min(cl["lessons"], key=len)
            if len(pattern) > 80:
                pattern = pattern[:77] + "..."
            results.append({
                "pattern": pattern,
                "days": sorted(cl["dates"]),
                "count": len(cl["dates"]),
            })
    results.sort(key=lambda r: r["count"], reverse=True)
    return results


# ---------------------------------------------------------------------------
# Output Formatting
# ---------------------------------------------------------------------------

def format_trends_section(
    recurring: list[dict],
    stale: list[dict],
    escalated: list[dict],
) -> str:
    """Format all trend results as a Markdown section."""
    lines: list[str] = ["## Trends\n"]

    # Recurring Issues
    lines.append("### 🔁 Recurring Issues")
    if recurring:
        lines.append("_Same issue/topic flagged on 3+ separate days in the last 14 days._\n")
        for i, r in enumerate(recurring, 1):
            days_str = ", ".join(r["days"])
            lines.append(f"{i}. **{r['topic']}** — appeared on {r['count']} days ({days_str})")
    else:
        lines.append("_No recurring issues detected._")
    lines.append("")

    # Stale Projects
    lines.append("### 🕸️ Possibly Stale Projects")
    if stale:
        lines.append("_Projects in MEMORY.md not mentioned in any daily note for 30+ days._\n")
        for i, s in enumerate(stale, 1):
            lines.append(f"{i}. **{s['project']}** — last mentioned: {s['last_mention']} ({s['days_since']} days ago)")
    else:
        lines.append("_All tracked projects appear active._")
    lines.append("")

    # Escalated Patterns
    lines.append("### ⚠️ Escalated Patterns (Repeated Mistakes)")
    if escalated:
        lines.append("_Same lesson/mistake appearing in 3+ daily notes — pattern not yet broken._\n")
        for i, e in enumerate(escalated, 1):
            days_str = ", ".join(e["days"])
            lines.append(f"{i}. **{e['pattern']}** — repeated on {e['count']} days ({days_str})")
    else:
        lines.append("_No escalated patterns detected._")
    lines.append("")

    return "\n".join(lines)


# ---------------------------------------------------------------------------
# State Management
# ---------------------------------------------------------------------------

def update_state_trends(state_path: str, recurring: list, stale: list, escalated: list, today_str: str):
    """Merge trend data into state.json under a 'trends' key."""
    state: dict = {}
    if os.path.exists(state_path):
        with open(state_path, "r", encoding="utf-8") as f:
            state = json.load(f)

    trends = state.get("trends", {
        "recurringIssues": [],
        "staleProjects": [],
        "escalatedPatterns": [],
        "history": [],
    })

    # Update current snapshot
    trends["recurringIssues"] = recurring
    trends["staleProjects"] = stale
    trends["escalatedPatterns"] = escalated

    # Append to history (keep last 30 entries)
    history = trends.get("history", [])
    history.append({
        "date": today_str,
        "recurringCount": len(recurring),
        "staleCount": len(stale),
        "escalatedCount": len(escalated),
    })
    trends["history"] = history[-30:]

    state["trends"] = trends
    state["lastTrendRun"] = today_str

    with open(state_path, "w", encoding="utf-8") as f:
        json.dump(state, f, indent=2, ensure_ascii=False)
        f.write("\n")


# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------

def load_config(workspace: Path) -> dict:
    """Load lucid.config.json if it exists, return defaults otherwise."""
    script_dir = Path(__file__).resolve().parent.parent
    config_path = script_dir / "config" / "lucid.config.json"
    if not config_path.exists():
        config_path = workspace / "config" / "lucid.config.json"
    if config_path.exists():
        try:
            with open(config_path, encoding="utf-8") as f:
                return json.load(f)
        except Exception:
            pass
    return {}


def get_cfg(config: dict, *keys, default=None):
    """Safe nested config access."""
    val = config
    for k in keys:
        if not isinstance(val, dict):
            return default
        val = val.get(k, default)
    return val if val is not None else default


def main():
    parser = argparse.ArgumentParser(description="Lucid Trend Detection")
    parser.add_argument("--workspace", required=True, help="Path to agent workspace root")
    parser.add_argument("--memory", default="MEMORY.md", help="Relative path to MEMORY.md (from workspace)")
    parser.add_argument("--days", type=int, default=None, help="Number of days to scan (default: from config or 14)")
    parser.add_argument("--stale-days", type=int, default=None, help="Days before a project is flagged stale (default: from config or 30)")
    parser.add_argument("--min-recurrence", type=int, default=None, help="Min days for recurring issue (default: from config or 3)")
    parser.add_argument("--state", default="memory/review/state.json", help="Relative path to state.json")
    parser.add_argument("--date", default=None, help="Override today's date (YYYY-MM-DD)")
    parser.add_argument("--config", default=None, help="Path to lucid.config.json (default: auto-detect)")
    args = parser.parse_args()

    workspace = Path(args.workspace).resolve()
    config = load_config(workspace)

    # Check if trends are enabled at all
    if not get_cfg(config, "trends", "enabled", default=True):
        print("## Trends\n\n_Trend detection disabled via config._")
        return

    # Apply config defaults (CLI args override config which overrides hardcoded defaults)
    days = args.days or get_cfg(config, "trends", "recurringIssues", "lookbackDays", default=14)
    stale_days = args.stale_days or get_cfg(config, "trends", "staleProjects", "staleAfterDays", default=30)
    min_recurrence = args.min_recurrence or get_cfg(config, "trends", "recurringIssues", "minOccurrences", default=3)

    # Per-detector enable flags
    recurring_enabled = get_cfg(config, "trends", "recurringIssues", "enabled", default=True)
    stale_enabled = get_cfg(config, "trends", "staleProjects", "enabled", default=True)
    escalated_enabled = get_cfg(config, "trends", "escalatedPatterns", "enabled", default=True)

    today = parse_date(args.date) if args.date else datetime.now()
    today_str = today.strftime("%Y-%m-%d")

    # Read daily notes
    daily_notes: dict[str, str] = {}
    for date_str in date_range(today, days):
        content = read_file(workspace / "memory" / f"{date_str}.md")
        if content:
            daily_notes[date_str] = content

    if not daily_notes:
        print("## Trends\n\n_No daily notes found for the last {} days. Skipping trend analysis._".format(days))
        return

    # Read MEMORY.md
    memory_text = read_file(workspace / args.memory) or ""

    # Run detections (respects per-detector config)
    recurring = detect_recurring_issues(daily_notes, min_days=min_recurrence) if recurring_enabled else []
    stale = detect_stale_projects(memory_text, daily_notes, stale_days=stale_days, today=today) if stale_enabled else []
    escalated = detect_repeated_mistakes(daily_notes, min_days=min_recurrence) if escalated_enabled else []

    # Output markdown
    print(format_trends_section(recurring, stale, escalated))

    # Update state
    state_path = str(workspace / args.state)
    try:
        update_state_trends(state_path, recurring, stale, escalated, today_str)
        print(f"<!-- trend data written to {args.state} -->", file=sys.stderr)
    except Exception as e:
        print(f"<!-- warning: could not update state.json: {e} -->", file=sys.stderr)


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)