Skill: danavfrost/openmem (version 1.3.0)

--- name: openmem description: "SQLite long-term memory compression system for extended memory life. Adds tools for agents to control their memory functions." metadata: { "openclaw": { "emoji": "🧠", "requires": { "bins": ["python3"] }, "optionalEnv": [ "OPENMEM_DB", "OPENMEM_SESSIONS_DIR", "OPENMEM_BOOTSTRAP_LIMIT", "OPENMEM_COMPRESS_INACTIVITY" ], "reads": [ "~/.openclaw/agents/main/sessions/*.jsonl" ], "writes": [ "~/.openclaw/workspace/memory/openmem.db", "~/.openclaw/workspace/memory/openmem-cache.json", "~/.openclaw/workspace/memory/auto_compress_state.json", "~/.openclaw/logs/openmem-compress.log" ], "credentialAccess": "none — compression uses `openclaw capability model run` which handles provider auth internally" } } --- # OpenMem v1.3.0 SQLite-backed long-term memory. MIT License — free to use, modify, redistribute. No attribution required. ## Privacy & Scope **Read this before installing.** - **Full session transcripts are read during compression.** Both manual (`compress.py read`) and automated (`auto_compress.py`) compression read the complete raw JSONL session files. The full content is passed to the extractor — everything in that session is visible. Only 3–10 selected items are written to the DB; the raw content is never stored permanently. This is intentional but privacy-sensitive: sessions may contain passwords, keys, or personal data entered during a conversation. - **Automated compression destroys original session files.** `auto_compress.py` replaces session JSONL files older than 24h with a one-line stub after memories are confirmed. This is irreversible. Run with `--no-wipe` to disable this behaviour, or skip auto-compression entirely. - **Network calls are made during auto-compression.** `auto_compress.py` calls `openclaw capability model run` to extract memories using the user's configured model. OpenClaw handles all provider routing and auth — session excerpts are passed as a prompt through the local gateway. The MCP server itself makes no network calls. - **Cache file is plaintext.** After every memory write, top memories are written to `openmem-cache.json` in the same directory as the DB, unencrypted. Trust level is identical to the DB file. - **Persistent presence.** `setup.py` registers an MCP server and an hourly cron job with your OpenClaw gateway. OpenMem has ongoing read access to session files and write access to the local DB while enabled. Run `uninstall.py` to remove all registered components. ## Filesystem & Credential Access What this skill reads and writes — declared up-front. **Reads:** - `~/.openclaw/agents/main/sessions/*.jsonl` — session transcripts, read in full during compression (see Privacy above). **Writes:** - `~/.openclaw/workspace/memory/openmem.db` — SQLite memory database - `~/.openclaw/workspace/memory/openmem-cache.json` — plaintext top-memory cache, written after every memory write - `~/.openclaw/workspace/memory/auto_compress_state.json` — last-run state for the daily guard - `~/.openclaw/logs/openmem-compress.log` — compression log All paths are overridable via env vars (see Environment Variables). ## MCP Tool Calls When the OpenMem MCP server is registered, use these native tool calls directly: | Tool | Purpose | |---|---| | `memory_add` | Store a new memory (content, category, importance, source) | | `memory_search` | FTS search with relevance + importance + recency ranking | | `memory_update` | Change content, category, or importance by ID | | `memory_delete` | Remove a memory by ID | | `memory_list` | List memories sorted by importance / recency / access | | `memory_stats` | Total count, breakdown by category, date range | ## Installation After `openclaw skills install openmem`, run setup once: ```bash python3 ~/.openclaw/workspace/skills/openmem/scripts/setup.py ``` This creates the database, checks requirements, registers the MCP server, registers the auto-compression launchd job (macOS), and prints the next steps (hook enable). ## Uninstall Removes the cron job, MCP server, and bootstrap hook. **Your database is not deleted** — its path is printed so you can export or remove it yourself. ```bash python3 ~/.openclaw/workspace/skills/openmem/scripts/uninstall.py ``` ## CLI Quick Reference ```bash SCRIPTS=~/.openclaw/workspace/skills/openmem/scripts # Add a memory python3 $SCRIPTS/mem.py add "User prefers concise responses" --category preference # Search python3 $SCRIPTS/mem.py search "response style" # List top memories python3 $SCRIPTS/mem.py list --limit 20 # Stats python3 $SCRIPTS/mem.py stats # --- Session compression --- # List uncompressed sessions python3 $SCRIPTS/compress.py pending # Read a session (then you summarize it into mem.py add calls) python3 $SCRIPTS/compress.py read <session-id> # Mark compressed after adding memories python3 $SCRIPTS/compress.py mark-done <session-id> --memory-count 5 ``` ## Categories `fact` · `insight` · `preference` · `correction` · `event` · `general` ## Compression Workflow ### Automatic (default) After setup, an OpenClaw cron job runs `auto_compress.py` every hour (via `--tools exec`, so it can only run the script — no loops possible). The script itself handles all guards: 1. Checks it hasn't already run today 2. Checks the agent has been inactive for 2+ hours 3. Finds all uncompressed sessions (skipping the most recently active one) 4. Calls Ollama to extract 3–8 memories per session (falls back to heuristics if Ollama is unavailable) 5. Deduplicates each candidate against existing memories (word-overlap check) 6. Inserts and **confirms** each memory is in the DB before proceeding 7. Wipes sessions older than 24h to a stub after memories are confirmed 8. Logs everything to `~/.openclaw/logs/openmem-compress.log` To run it manually: ```bash python3 $SCRIPTS/auto_compress.py --force # skip guards, compress now python3 $SCRIPTS/auto_compress.py --dry-run # preview without writing python3 $SCRIPTS/auto_compress.py --no-wipe # compress but keep session files ``` ### On-demand (agent) You can also trigger compression by saying: - *"compress my sessions"* - *"save this to long-term memory"* **Step by step:** 1. `python3 $SCRIPTS/compress.py pending` — find sessions not yet compressed 2. `python3 $SCRIPTS/compress.py read <id>` — read the conversation 3. Extract 3–10 key facts, corrections, preferences, and insights 4. Use `memory_add` tool call (or `mem.py add`) for each memory 5. `python3 $SCRIPTS/compress.py mark-done <id> --memory-count <N>` **What to extract:** - Facts the user stated about their system, preferences, or projects - Mistakes made and the correct approach - Decisions reached and why - Important events (deploys, incidents, milestones) **What to skip:** Raw command output, transient errors, small talk. ### Deduplication `memory_add` automatically checks for similar existing memories before inserting (word-overlap similarity ≥ 65%). If a near-duplicate is found, it returns the existing memory ID instead of inserting. Pass `"check_duplicate": false` to force an insert. ## Importance Guide | Score | Meaning | |---|---| | 0.9–1.0 | Critical — always surface (key preferences, major corrections) | | 0.7–0.8 | Important — surface often | | 0.5–0.6 | Normal (default) | | 0.3–0.4 | Low — background context | | 0.0–0.2 | Archive only | ## OpenAuto Integration If [OpenAuto](https://clawhub.ai/halthelobster/openauto) is installed alongside OpenMem, the two work together: - **OpenAuto defers long-term writes to OpenMem** — when `memory_add` is available, OpenAuto uses it instead of writing directly to `MEMORY.md` - **OpenMem replaces `MEMORY.md` searches** — `memory_search` handles ranked FTS lookup so OpenAuto doesn't need to grep flat files - **Session compression feeds OpenMem** — OpenAuto's Working Buffer and daily notes are the source material; saying "compress my sessions" extracts durable memories into OpenMem - **Bootstrap injection bridges both** — OpenMem injects top memories as `OPENMEM.md` at session start, which OpenAuto reads alongside its own workspace files Both skills work independently. If both are installed and OpenMem's MCP server is registered, the integration activates automatically — no extra setup required. ## Bootstrap Hook The bootstrap hook auto-injects your top memories at session start. Enable once: ```bash openclaw hooks enable openmem ``` Memories appear in `OPENMEM.md` at the start of every session. Control injection count with `OPENMEM_BOOTSTRAP_LIMIT` (default: 12). ## Database Schema See [references/schema.md](references/schema.md) for full schema details. ## Environment Variables | Variable | Default | Description | |---|---|---| | `OPENMEM_DB` | `~/.openclaw/workspace/memory/openmem.db` | Database path | | `OPENMEM_BOOTSTRAP_LIMIT` | `12` | Memories injected at bootstrap | | `OPENMEM_SESSIONS_DIR` | `~/.openclaw/agents/main/sessions` | Session files location | | `OPENMEM_COMPRESS_INACTIVITY` | `2` | Minimum inactivity hours before auto-compress runs |

path: references/schema.md
content: # OpenMem Database Schema

Default path: `~/.openclaw/workspace/memory/openmem.db`
Override with `OPENMEM_DB` env var.

## Table: memories

The primary store. Every row is one discrete memory.

| Column | Type | Description |
|---|---|---|
| `id` | INTEGER PK | Auto-increment |
| `content` | TEXT UNIQUE | The memory text. Duplicate content is silently deduplicated. |
| `category` | TEXT | One of: `fact`, `insight`, `preference`, `correction`, `event`, `general` |
| `source` | TEXT | Where it came from: `manual`, `session:<id>`, `hook`, etc. |
| `importance` | REAL | 0.0–1.0. Higher = more likely to surface in search/bootstrap. |
| `session_id` | TEXT | OpenClaw session ID this came from (nullable) |
| `created_at` | INTEGER | Unix timestamp |
| `updated_at` | INTEGER | Unix timestamp |
| `access_count` | INTEGER | Incremented on every read. Used for future LRU eviction. |

## Table: memories_fts

FTS5 virtual table mirroring `memories.content`. Uses Porter stemmer.
Kept in sync automatically via triggers on INSERT/UPDATE/DELETE.

Search uses `bm25()` ranking combined with importance and recency:

```
score = bm25_rank - (importance × 2.0) - (age_in_days × 0.05)
```

Lower score = better match (bm25 returns negative values).

## Table: compressed_sessions

Tracks which OpenClaw sessions have been compressed into memories.

| Column | Type | Description |
|---|---|---|
| `session_id` | TEXT PK | OpenClaw session UUID |
| `session_file` | TEXT | Filename |
| `compressed_at` | INTEGER | Unix timestamp |
| `memory_count` | INTEGER | How many memories were extracted |

## Category Guide

| Category | Use for |
|---|---|
| `fact` | Objective facts about the world, system, or codebase |
| `insight` | Learned patterns, conclusions, or non-obvious observations |
| `preference` | User preferences, style choices, or stated desires |
| `correction` | Mistakes made and the correct approach |
| `event` | Things that happened (deployments, incidents, milestones) |
| `general` | Anything that doesn't fit above |


path: hooks/openclaw/handler.js
content: /**
 * OpenMem - Bootstrap Hook
 *
 * Injects long-term memories into the agent context at session start.
 * Fires on agent:bootstrap before workspace files are loaded.
 *
 * Reads from openmem-cache.json — a plain JSON file written by mem.py
 * and mcp_server.py after every memory write. No process spawning required.
 *
 * Compression is on-demand only — say "compress my sessions" or "save to memory".
 */

"use strict";

const path = require("path");
const os = require("os");
const fs = require("fs");

const DEFAULT_DB_DIR = path.join(os.homedir(), ".openclaw", "workspace", "memory");
const DB_DIR = process.env.OPENMEM_DB
  ? path.dirname(process.env.OPENMEM_DB)
  : DEFAULT_DB_DIR;
const CACHE_FILE = path.join(DB_DIR, "openmem-cache.json");
const INJECT_LIMIT = parseInt(process.env.OPENMEM_BOOTSTRAP_LIMIT || "12", 10);

function loadCache() {
  try {
    const raw = fs.readFileSync(CACHE_FILE, "utf8");
    const cache = JSON.parse(raw);
    if (!Array.isArray(cache.memories)) return null;
    return cache.memories.slice(0, INJECT_LIMIT);
  } catch {
    // Cache doesn't exist yet or is unreadable — not an error
    return null;
  }
}

function formatMemories(memories) {
  if (!memories || memories.length === 0) return null;

  const lines = [
    "## Long-Term Memory (OpenMem)",
    "",
    "Memories from previous sessions (highest importance first):",
    "",
  ];

  for (const mem of memories) {
    const cat = mem.category || "general";
    const imp = typeof mem.importance === "number" ? mem.importance.toFixed(1) : "0.5";
    const content = (mem.content || "").trim();
    if (!content) continue;
    lines.push(`**[${cat}]** (importance: ${imp})`);
    lines.push(content);
    lines.push("");
  }

  lines.push("---");
  lines.push("*Tools: memory_add · memory_search · memory_update · memory_delete · memory_list · memory_stats*");

  return lines.join("\n");
}

const handler = async (event) => {
  if (!event || typeof event !== "object") return;
  if (event.type !== "agent" || event.action !== "bootstrap") return;
  if (!event.context || !Array.isArray(event.context.bootstrapFiles)) return;

  // Skip sub-agent sessions
  const sessionKey = event.sessionKey || "";
  if (sessionKey.includes(":subagent:")) return;

  const memories = loadCache();
  if (!memories || memories.length === 0) return;

  const content = formatMemories(memories);
  if (!content) return;

  event.context.bootstrapFiles.push({
    path: "OPENMEM.md",
    content,
    virtual: true,
  });
};

module.exports = handler;
module.exports.default = handler;


path: hooks/openclaw/HOOK.md
content: ---
name: openmem
description: "Injects long-term memories from OpenMem into the agent context at session start"
metadata: {"openclaw":{"emoji":"🧠","events":["agent:bootstrap"]}}
---

# OpenMem Bootstrap Hook

Fires on `agent:bootstrap` and injects an `OPENMEM.md` virtual file containing
your most important long-term memories. No-ops silently if the database doesn't
exist yet.

## Setup

```bash
# Install the skill first, then enable the hook:
openclaw hooks enable openmem

# Initialize the database:
python3 ~/.openclaw/workspace/skills/openmem/scripts/mem.py init
```

## Configuration (environment variables)

| Variable | Default | Description |
|---|---|---|
| `OPENMEM_DB` | `~/.openclaw/workspace/memory/openmem.db` | Database path |
| `OPENMEM_BOOTSTRAP_LIMIT` | `12` | Max memories to inject at startup |
| `OPENMEM_PYTHON` | `python3` | Python interpreter to use |


path: scripts/auto_compress.py
content: #!/usr/bin/env python3
"""
OpenMem - auto_compress.py
Automated daily session compression. Designed to run unattended via OpenClaw cron.

Guards:
  - Runs at most once per calendar day
  - Only runs after 2+ hours of agent inactivity
  - Skips the most recently active session (may still be in use)

Extraction:
  - Calls `openclaw capability model run` to use the user's configured model
  - No credentials or config files are read — OpenClaw handles provider routing
  - Falls back to heuristic extraction if openclaw CLI is unavailable

Deduplication:
  - FTS5 word-overlap check before every insert
  - Skips memories too similar to existing ones

Session wipe:
  - After all memories confirmed in DB, replaces old session JSONL with a
    one-line stub. Only wipes sessions that are >24h old and not the
    most recently active session.

Usage:
  python3 auto_compress.py              # normal run (respects guards)
  python3 auto_compress.py --force      # skip date/inactivity guards
  python3 auto_compress.py --dry-run    # show what would happen, no writes
  python3 auto_compress.py --no-wipe    # compress but do not wipe session files

Env vars:
  OPENMEM_DB                   Path to openmem.db
  OPENMEM_SESSIONS_DIR         Path to OpenClaw sessions directory
  OPENMEM_COMPRESS_INACTIVITY  Minimum inactivity hours before running (default: 2)
"""

import argparse
import json
import os
import re
import shutil
import sqlite3
import subprocess
import sys
import time
from datetime import date
from pathlib import Path
from typing import Optional

# ---------------------------------------------------------------------------
# Config
# ---------------------------------------------------------------------------

DEFAULT_DB = Path(
    os.environ.get("OPENMEM_DB", "~/.openclaw/workspace/memory/openmem.db")
).expanduser()

DEFAULT_SESSIONS_DIR = Path(
    os.environ.get("OPENMEM_SESSIONS_DIR", "~/.openclaw/agents/main/sessions")
).expanduser()

STATE_FILE = DEFAULT_DB.parent / "auto_compress_state.json"
LOG_FILE = Path("~/.openclaw/logs/openmem-compress.log").expanduser()

INACTIVITY_HOURS = float(os.environ.get("OPENMEM_COMPRESS_INACTIVITY", "2"))

EXTRACT_PROMPT_TEMPLATE = """\
You are extracting long-term memories from an AI agent conversation log.

Extract 3-8 key facts, preferences, decisions, and corrections worth keeping.
Focus on:
- User preferences and working style
- Decisions made and why
- Corrections (what was wrong, what's right)
- Key facts about the user's projects, systems, and setup
- Important events or milestones

Respond with ONLY a JSON array, no explanation:
[
  {{"content": "...", "category": "preference|fact|insight|correction|event", "importance": 0.7}}
]

Rules:
- Skip routine status updates, small talk, greetings, tool output
- Each memory must be standalone (understandable without context)
- Maximum 100 words per memory
- importance: 0.9 = critical preference/correction, 0.7 = important, 0.5 = normal

Conversation (most recent messages):
{text}

JSON array:"""


# ---------------------------------------------------------------------------
# Logging
# ---------------------------------------------------------------------------

def log(msg: str):
    LOG_FILE.parent.mkdir(parents=True, exist_ok=True)
    ts = time.strftime("%Y-%m-%d %H:%M:%S")
    line = f"[{ts}] {msg}"
    print(line)
    try:
        with open(LOG_FILE, "a") as f:
            f.write(line + "\n")
    except OSError:
        pass


# ---------------------------------------------------------------------------
# State
# ---------------------------------------------------------------------------

def load_state() -> dict:
    try:
        return json.loads(STATE_FILE.read_text())
    except (OSError, json.JSONDecodeError):
        return {}


def save_state(state: dict):
    STATE_FILE.parent.mkdir(parents=True, exist_ok=True)
    STATE_FILE.write_text(json.dumps(state, indent=2))


def already_ran_today(state: dict) -> bool:
    return state.get("last_run_date") == str(date.today())


# ---------------------------------------------------------------------------
# Inactivity check
# ---------------------------------------------------------------------------

def latest_session_mtime(sessions_dir: Path) -> Optional[float]:
    """Return the mtime of the most recently modified session file."""
    best = None
    for p in sessions_dir.glob("*.jsonl"):
        if ".checkpoint." in p.name:
            continue
        mtime = p.stat().st_mtime
        if best is None or mtime > best:
            best = mtime
    return best


def agent_is_inactive(sessions_dir: Path, min_hours: float) -> bool:
    mtime = latest_session_mtime(sessions_dir)
    if mtime is None:
        return True  # no sessions at all → treat as inactive
    inactive_seconds = time.time() - mtime
    return inactive_seconds >= (min_hours * 3600)


# ---------------------------------------------------------------------------
# Session reading (mirrors compress.py)
# ---------------------------------------------------------------------------

def read_session_messages(session_file: Path) -> list[dict]:
    messages = []
    try:
        with open(session_file, encoding="utf-8") as f:
            for line in f:
                line = line.strip()
                if not line:
                    continue
                try:
                    obj = json.loads(line)
                except json.JSONDecodeError:
                    continue
                if obj.get("type") != "message":
                    continue
                msg = obj.get("message", {})
                role = msg.get("role")
                if role not in ("user", "assistant"):
                    continue
                content = msg.get("content", "")
                if isinstance(content, list):
                    parts = []
                    for block in content:
                        if isinstance(block, dict) and block.get("type") == "text":
                            parts.append(block.get("text", ""))
                    content = "\n".join(parts)
                content = _strip_metadata(content.strip())
                if content:
                    messages.append({
                        "role": role,
                        "content": content,
                        "timestamp": obj.get("timestamp"),
                    })
    except (OSError, PermissionError):
        pass
    return messages


def _strip_metadata(text: str) -> str:
    text = re.sub(
        r"^Sender \(untrusted metadata\):\s*```json\s*\{.*?\}\s*```\s*\n",
        "", text, flags=re.DOTALL
    )
    text = re.sub(r"^\[[^\]]+\]\s*", "", text)
    # Strip thinking blocks
    text = re.sub(r"<think>.*?</think>", "", text, flags=re.DOTALL)
    return text.strip()


def messages_to_text(messages: list[dict], max_chars: int = 12000) -> str:
    """Convert messages to a compact text block for the LLM prompt."""
    parts = []
    total = 0
    # Take the most recent messages first (up to max_chars)
    for msg in reversed(messages):
        role = "USER" if msg["role"] == "user" else "AGENT"
        chunk = f"{role}: {msg['content'][:500]}\n"
        if total + len(chunk) > max_chars:
            break
        parts.append(chunk)
        total += len(chunk)
    parts.reverse()
    return "\n".join(parts)


# ---------------------------------------------------------------------------
# Memory extraction — via openclaw CLI (no credential access required)
# ---------------------------------------------------------------------------

def _openclaw_bin() -> Optional[str]:
    """Return path to the openclaw binary, or None if not found."""
    return shutil.which("openclaw")


INFER_CHUNK_CHARS = 6000   # max chars per infer call
INFER_MAX_MEMORIES = 8     # cap total memories per session across all chunks


def _infer_one(bin_path: str, text: str) -> list[dict]:
    """Run a single `openclaw capability model run` call. Returns parsed memories or []."""
    prompt = EXTRACT_PROMPT_TEMPLATE.format(text=text)
    try:
        proc = subprocess.run(
            [bin_path, "--no-color", "capability", "model", "run", "--prompt", prompt, "--json"],
            capture_output=True,
            text=True,
            timeout=120,
        )
        if proc.returncode != 0:
            return []
        stdout = proc.stdout.strip()
        obj_match = re.search(r"\{.*\}", stdout, re.DOTALL)
        if not obj_match:
            return []
        data = json.loads(obj_match.group(0))
        if not data.get("ok") or not data.get("outputs"):
            return []
        content = data["outputs"][0].get("text", "").strip()
        arr_match = re.search(r"\[.*\]", content, re.DOTALL)
        if not arr_match:
            return []
        memories = json.loads(arr_match.group(0))
        result = []
        for m in memories:
            if not isinstance(m, dict) or not m.get("content"):
                continue
            result.append({
                "content": str(m["content"])[:500].strip(),
                "category": m.get("category", "general"),
                "importance": float(m.get("importance", 0.5)),
            })
        return result
    except Exception:
        return []


def extract_via_infer(text: str) -> list[dict]:
    """
    Call `openclaw capability model run` to extract memories using whatever
    model the user has configured. OpenClaw handles all provider routing and
    auth internally — this function reads no credentials and makes no direct
    network calls.

    Large sessions are split into chunks of INFER_CHUNK_CHARS characters.
    Results are merged and deduplicated by content prefix.
    Returns list of {content, category, importance}.
    """
    bin_path = _openclaw_bin()
    if not bin_path:
        log("  openclaw not found in PATH — falling back to heuristic")
        return []

    # Split into chunks if needed
    chunks = []
    if len(text) <= INFER_CHUNK_CHARS:
        chunks = [text]
    else:
        # Split on newlines at chunk boundaries to avoid cutting mid-message
        lines = text.split("\n")
        current, current_len = [], 0
        for line in lines:
            line_len = len(line) + 1
            if current_len + line_len > INFER_CHUNK_CHARS and current:
                chunks.append("\n".join(current))
                current, current_len = [], 0
            current.append(line)
            current_len += line_len
        if current:
            chunks.append("\n".join(current))

    if len(chunks) > 1:
        log(f"  large session — splitting into {len(chunks)} chunks")

    all_memories: list[dict] = []
    seen_prefixes: set = set()

    for i, chunk in enumerate(chunks):
        chunk_memories = _infer_one(bin_path, chunk)
        if len(chunks) > 1:
            log(f"  chunk {i + 1}/{len(chunks)}: {len(chunk_memories)} candidates")
        for mem in chunk_memories:
            prefix = mem["content"][:60].lower()
            if prefix not in seen_prefixes:
                seen_prefixes.add(prefix)
                all_memories.append(mem)
            if len(all_memories) >= INFER_MAX_MEMORIES:
                break
        if len(all_memories) >= INFER_MAX_MEMORIES:
            break

    if not all_memories and len(chunks) > 1:
        log("  all chunks failed infer — falling back to heuristic")

    return all_memories


def extract_heuristic(messages: list[dict]) -> list[dict]:
    """Keyword-pattern extraction when Ollama is unavailable."""
    patterns = [
        (r"(?:I|we)\s+(?:prefer|like|always use|never use|hate)\s+.{15,100}", "preference", 0.6),
        (r"(?:actually|no,?|wrong[,.]|correction[,:]|mistake[,.])\s+.{15,100}", "correction", 0.75),
        (r"(?:decided|going with|will use|the plan is|we chose)[: ].{15,100}", "insight", 0.6),
        (r"(?:the project|this repo|our stack|the system)\s+.{15,100}", "fact", 0.5),
        (r"(?:important|remember|note that|keep in mind)\s*[:-]?\s*.{15,100}", "fact", 0.55),
    ]
    memories = []
    seen = set()
    for msg in messages:
        if msg["role"] != "user":
            continue
        for pattern, category, importance in patterns:
            for match in re.finditer(pattern, msg["content"], re.IGNORECASE):
                content = match.group(0).strip().rstrip(".,")
                key = content.lower()[:50]
                if key not in seen and len(content) > 20:
                    seen.add(key)
                    memories.append({
                        "content": content,
                        "category": category,
                        "importance": importance,
                    })
    return memories[:8]


def extract_memories(messages: list[dict]) -> list[dict]:
    """Try openclaw infer first, fall back to heuristic."""
    if not messages:
        return []
    text = messages_to_text(messages)
    memories = extract_via_infer(text)
    if not memories:
        memories = extract_heuristic(messages)
    return memories


# ---------------------------------------------------------------------------
# Deduplication
# ---------------------------------------------------------------------------

def _significant_words(text: str) -> set:
    stopwords = {"the", "that", "this", "with", "from", "have", "will", "been",
                 "they", "were", "when", "what", "which", "there", "their",
                 "them", "then", "than", "your", "also", "into", "some",
                 "just", "over", "like", "such", "only", "more", "very"}
    words = set(re.findall(r"\b[a-zA-Z]{4,}\b", text.lower()))
    return words - stopwords


def find_duplicate(db: sqlite3.Connection, content: str, threshold: float = 0.65) -> Optional[int]:
    """Return ID of an existing memory with high word overlap, or None."""
    words = _significant_words(content)
    if len(words) < 3:
        return None

    # Build FTS query from up to 6 significant words
    query_words = list(words)[:6]
    fts_query = " OR ".join(f'"{w}"' for w in query_words)

    try:
        rows = db.execute(
            "SELECT m.id, m.content FROM memories m "
            "JOIN memories_fts ON m.id = memories_fts.rowid "
            "WHERE memories_fts MATCH ? LIMIT 10",
            (fts_query,)
        ).fetchall()
    except Exception:
        return None

    for row in rows:
        existing_words = _significant_words(row["content"])
        if not existing_words:
            continue
        overlap = len(words & existing_words) / min(len(words), len(existing_words))
        if overlap >= threshold:
            return row["id"]

    return None


# ---------------------------------------------------------------------------
# Database
# ---------------------------------------------------------------------------

def open_db(db_path: Path) -> sqlite3.Connection:
    db = sqlite3.connect(str(db_path), timeout=10)
    db.row_factory = sqlite3.Row
    db.execute("PRAGMA journal_mode=WAL")
    # Ensure compressed_sessions table exists with wipe column
    db.executescript("""
        CREATE TABLE IF NOT EXISTS compressed_sessions (
            session_id    TEXT PRIMARY KEY,
            session_file  TEXT NOT NULL,
            compressed_at INTEGER NOT NULL,
            memory_count  INTEGER NOT NULL DEFAULT 0,
            wiped_at      INTEGER
        );
        CREATE TABLE IF NOT EXISTS memories (
            id           INTEGER PRIMARY KEY AUTOINCREMENT,
            content      TEXT    NOT NULL,
            category     TEXT    NOT NULL DEFAULT 'general',
            source       TEXT    NOT NULL DEFAULT 'manual',
            importance   REAL    NOT NULL DEFAULT 0.5,
            session_id   TEXT,
            created_at   INTEGER NOT NULL,
            updated_at   INTEGER NOT NULL,
            access_count INTEGER NOT NULL DEFAULT 0
        );
        CREATE UNIQUE INDEX IF NOT EXISTS idx_memories_content ON memories(content);
    """)
    # Add wiped_at column if it doesn't exist (migration)
    try:
        db.execute("ALTER TABLE compressed_sessions ADD COLUMN wiped_at INTEGER")
        db.commit()
    except sqlite3.OperationalError:
        pass  # column already exists
    db.commit()
    return db


def insert_memory(db: sqlite3.Connection, content: str, category: str,
                  importance: float, session_id: str, source: str) -> Optional[int]:
    """Insert a memory and return its ID, or None if it failed."""
    now = int(time.time())
    try:
        cur = db.execute(
            "INSERT INTO memories (content, category, source, importance, session_id, "
            "created_at, updated_at) VALUES (?,?,?,?,?,?,?)",
            (content, category, source, importance, session_id, now, now)
        )
        db.commit()
        return cur.lastrowid
    except sqlite3.IntegrityError:
        # content unique constraint — exact duplicate
        row = db.execute("SELECT id FROM memories WHERE content = ?", (content,)).fetchone()
        return row["id"] if row else None
    except Exception as e:
        log(f"    DB insert error: {e}")
        return None


def confirm_memory_in_db(db: sqlite3.Connection, memory_id: int) -> bool:
    row = db.execute("SELECT id FROM memories WHERE id = ?", (memory_id,)).fetchone()
    return row is not None


def verify_session_memories(db: sqlite3.Connection, session_id: str,
                             expected_count: int) -> tuple[bool, int]:
    """
    Re-query the DB for memories sourced from this session and verify:
      1. The count matches expected_count
      2. Each memory is readable and has non-empty content
      3. The FTS index contains each memory (search by ID)
    Returns (ok, actual_count).
    """
    short_id = session_id[:8]
    rows = db.execute(
        "SELECT id, content FROM memories WHERE source LIKE ?",
        (f"%{short_id}%",)
    ).fetchall()
    actual = len(rows)

    if actual != expected_count:
        return False, actual

    # Verify content is non-empty and FTS index has each row
    for row in rows:
        if not row["content"] or not row["content"].strip():
            return False, actual
        # Confirm FTS index entry exists
        fts_row = db.execute(
            "SELECT rowid FROM memories_fts WHERE rowid = ?", (row["id"],)
        ).fetchone()
        if fts_row is None:
            return False, actual

    return True, actual


def get_compressed_ids(db: sqlite3.Connection) -> set:
    rows = db.execute("SELECT session_id FROM compressed_sessions").fetchall()
    return {r["session_id"] for r in rows}


def mark_session_compressed(db: sqlite3.Connection, session_id: str,
                             session_file: str, memory_count: int):
    db.execute(
        "INSERT OR REPLACE INTO compressed_sessions "
        "(session_id, session_file, compressed_at, memory_count) VALUES (?,?,?,?)",
        (session_id, session_file, int(time.time()), memory_count)
    )
    db.commit()


def mark_session_wiped(db: sqlite3.Connection, session_id: str):
    db.execute(
        "UPDATE compressed_sessions SET wiped_at = ? WHERE session_id = ?",
        (int(time.time()), session_id)
    )
    db.commit()


# ---------------------------------------------------------------------------
# Session file wipe
# ---------------------------------------------------------------------------

WIPE_MIN_AGE_HOURS = 24  # only wipe sessions older than this


def should_wipe_session(session_file: Path, active_session_file: Optional[Path]) -> bool:
    """Check if it's safe to wipe this session file."""
    if session_file == active_session_file:
        return False
    age_hours = (time.time() - session_file.stat().st_mtime) / 3600
    return age_hours >= WIPE_MIN_AGE_HOURS


def wipe_session_file(session_file: Path, session_id: str,
                      memory_count: int, dry_run: bool) -> bool:
    """Replace session JSONL with a compressed stub. Returns True on success."""
    stub = json.dumps({
        "type": "custom",
        "customType": "openmem:compressed-stub",
        "data": {
            "session_id": session_id,
            "compressed_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
            "memory_count": memory_count,
        },
        "id": "compressed-stub",
        "parentId": None,
        "timestamp": time.strftime("%Y-%m-%dT%H:%M:%S.000Z", time.gmtime()),
    })

    if dry_run:
        log(f"    [dry-run] would wipe {session_file.name} ({session_file.stat().st_size // 1024}KB)")
        return True

    try:
        session_file.write_text(stub + "\n", encoding="utf-8")
        return True
    except OSError as e:
        log(f"    Wipe failed: {e}")
        return False


# ---------------------------------------------------------------------------
# Main compression loop
# ---------------------------------------------------------------------------

def compress_session(session_file: Path, db: sqlite3.Connection,
                     dry_run: bool) -> int:
    """Compress one session. Returns number of memories added."""
    session_id = session_file.stem

    messages = read_session_messages(session_file)
    if not messages:
        log(f"  {session_id[:8]}: no messages, skipping")
        return 0

    log(f"  {session_id[:8]}: {len(messages)} messages → extracting...")
    candidates = extract_memories(messages)
    log(f"  {session_id[:8]}: {len(candidates)} candidates found")

    added = 0
    for mem in candidates:
        content = mem["content"]
        category = mem.get("category", "general")
        importance = float(mem.get("importance", 0.5))

        # Dedup check
        dup_id = find_duplicate(db, content)
        if dup_id is not None:
            log(f"    skip (dup #{dup_id}): {content[:60]}...")
            continue

        # Insert
        if dry_run:
            log(f"    [dry-run] would add [{category}] {content[:80]}")
            added += 1
            continue

        mem_id = insert_memory(db, content, category, importance,
                               session_id=session_id,
                               source=f"auto_compress:{session_id[:8]}")
        if mem_id is None:
            log(f"    insert failed: {content[:60]}")
            continue

        # Confirm it's in DB before counting it
        if confirm_memory_in_db(db, mem_id):
            log(f"    added #{mem_id} [{category} {importance:.1f}]: {content[:70]}")
            added += 1
        else:
            log(f"    confirmation failed for #{mem_id}: {content[:60]}")

    return added


def run(force: bool = False, dry_run: bool = False, wipe: bool = True,
        sessions_dir: Path = DEFAULT_SESSIONS_DIR,
        db_path: Path = DEFAULT_DB) -> int:
    """
    Main entry point. Returns exit code (0 = success/skipped, 1 = error).
    """
    state = load_state()

    if not force and already_ran_today(state):
        log("Already ran today — skipping")
        return 0

    if not force and not agent_is_inactive(sessions_dir, INACTIVITY_HOURS):
        log(f"Agent active within last {INACTIVITY_HOURS}h — skipping")
        return 0

    if not db_path.exists():
        log(f"Database not found: {db_path}")
        return 1

    db = open_db(db_path)
    compressed_ids = get_compressed_ids(db)

    # Find all session files, sorted oldest-first
    all_sessions = sorted(
        [p for p in sessions_dir.glob("*.jsonl") if ".checkpoint." not in p.name],
        key=lambda p: p.stat().st_mtime
    )
    pending = [f for f in all_sessions if f.stem not in compressed_ids]

    # The most recently modified session may be the active one — skip it
    # unless --force is set (covers the case where it was already the "latest")
    active_session = all_sessions[-1] if all_sessions else None
    if not force and active_session in pending:
        pending.remove(active_session)

    if not pending:
        log("No sessions pending compression")
        save_state({**state, "last_run_date": str(date.today()), "last_run_ts": int(time.time())})
        return 0

    log(f"Compressing {len(pending)} session(s)...")
    total_added = 0
    total_wiped = 0

    for session_file in pending:
        added = compress_session(session_file, db, dry_run)
        total_added += added

        if not dry_run:
            mark_session_compressed(db, session_file.stem, session_file.name, added)

        # Wipe if safe and requested — but only after verifying memories are in DB
        if wipe and not dry_run and should_wipe_session(session_file, active_session):
            if added > 0:
                ok, actual = verify_session_memories(db, session_file.stem, added)
                if not ok:
                    log(f"  {session_file.stem[:8]}: verification FAILED "
                        f"(expected {added}, confirmed {actual}) — skipping wipe")
                else:
                    wiped = wipe_session_file(session_file, session_file.stem, added, dry_run)
                    if wiped:
                        mark_session_wiped(db, session_file.stem)
                        total_wiped += 1
                        log(f"  {session_file.stem[:8]}: verified {actual} memories, wiped")
            elif session_file.stat().st_size > 50_000:
                # Large session with 0 memories extracted — still safe to wipe
                # (nothing to verify, no data loss)
                wiped = wipe_session_file(session_file, session_file.stem, 0, dry_run)
                if wiped:
                    mark_session_wiped(db, session_file.stem)
                    total_wiped += 1
                    log(f"  {session_file.stem[:8]}: 0 memories, wiped large session")

    log(f"Done: {total_added} memories added, {total_wiped} sessions wiped")

    if not dry_run:
        save_state({
            "last_run_date": str(date.today()),
            "last_run_ts": int(time.time()),
            "sessions_compressed": len(pending),
            "memories_added": total_added,
        })

    return 0


# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------

def main():
    p = argparse.ArgumentParser(
        prog="auto_compress.py",
        description="OpenMem automated session compression"
    )
    p.add_argument("--force", action="store_true",
                   help="Skip date and inactivity guards")
    p.add_argument("--dry-run", action="store_true",
                   help="Show what would happen without writing anything")
    p.add_argument("--no-wipe", action="store_true",
                   help="Compress sessions but do not wipe session files")
    p.add_argument("--sessions-dir", type=Path, default=DEFAULT_SESSIONS_DIR)
    p.add_argument("--db", type=Path, default=DEFAULT_DB)
    args = p.parse_args()

    model_label = "openclaw-infer" if _openclaw_bin() else "heuristic"
    log(f"auto_compress starting (force={args.force}, dry_run={args.dry_run}, wipe={not args.no_wipe}, model={model_label})")
    sys.exit(run(
        force=args.force,
        dry_run=args.dry_run,
        wipe=not args.no_wipe,
        sessions_dir=args.sessions_dir,
        db_path=args.db,
    ))


if __name__ == "__main__":
    main()


path: scripts/setup.py
content: #!/usr/bin/env python3
"""
OpenMem - setup.py
First-run setup: verifies requirements and creates the database.

Run once after installing the skill:
  python3 ~/.openclaw/workspace/skills/openmem/scripts/setup.py

Safe to re-run — all operations are idempotent.
"""

import json
import os
import shutil
import sqlite3
import subprocess
import sys
from pathlib import Path

DEFAULT_DB = Path(
    os.environ.get("OPENMEM_DB", "~/.openclaw/workspace/memory/openmem.db")
).expanduser()


def check_python_version():
    if sys.version_info < (3, 8):
        return False, f"Python 3.8+ required, got {sys.version}"
    return True, f"Python {sys.version.split()[0]}"


def check_sqlite_fts5():
    try:
        db = sqlite3.connect(":memory:")
        db.execute("CREATE VIRTUAL TABLE _fts5_test USING fts5(content)")
        db.close()
        return True, f"SQLite {sqlite3.sqlite_version} with FTS5"
    except Exception as e:
        return False, f"SQLite FTS5 not available: {e}"


def check_sqlite_wal():
    """WAL mode requires sqlite 3.7+, which is universal, but verify."""
    try:
        db = sqlite3.connect(":memory:")
        db.execute("PRAGMA journal_mode=WAL")
        db.close()
        return True, "WAL mode supported"
    except Exception as e:
        return False, f"WAL mode unavailable: {e}"


def init_database(db_path: Path):
    db_path.parent.mkdir(parents=True, exist_ok=True)
    db = sqlite3.connect(str(db_path))
    db.execute("PRAGMA journal_mode=WAL")
    db.executescript("""
        CREATE TABLE IF NOT EXISTS meta (
            key   TEXT PRIMARY KEY,
            value TEXT NOT NULL
        );
        CREATE TABLE IF NOT EXISTS memories (
            id           INTEGER PRIMARY KEY AUTOINCREMENT,
            content      TEXT    NOT NULL,
            category     TEXT    NOT NULL DEFAULT 'general',
            source       TEXT    NOT NULL DEFAULT 'manual',
            importance   REAL    NOT NULL DEFAULT 0.5,
            session_id   TEXT,
            created_at   INTEGER NOT NULL,
            updated_at   INTEGER NOT NULL,
            access_count INTEGER NOT NULL DEFAULT 0
        );
        CREATE UNIQUE INDEX IF NOT EXISTS idx_memories_content ON memories(content);
        CREATE INDEX IF NOT EXISTS idx_memories_category ON memories(category, created_at DESC);
        CREATE INDEX IF NOT EXISTS idx_memories_importance ON memories(importance DESC, created_at DESC);
        CREATE VIRTUAL TABLE IF NOT EXISTS memories_fts USING fts5(
            content, content='memories', content_rowid='id', tokenize='porter ascii'
        );
        CREATE TRIGGER IF NOT EXISTS memories_fts_insert
            AFTER INSERT ON memories BEGIN
                INSERT INTO memories_fts(rowid, content) VALUES (new.id, new.content);
            END;
        CREATE TRIGGER IF NOT EXISTS memories_fts_delete
            AFTER DELETE ON memories BEGIN
                INSERT INTO memories_fts(memories_fts, rowid, content)
                    VALUES ('delete', old.id, old.content);
            END;
        CREATE TRIGGER IF NOT EXISTS memories_fts_update
            AFTER UPDATE OF content ON memories BEGIN
                INSERT INTO memories_fts(memories_fts, rowid, content)
                    VALUES ('delete', old.id, old.content);
                INSERT INTO memories_fts(rowid, content) VALUES (new.id, new.content);
            END;
        CREATE TABLE IF NOT EXISTS compressed_sessions (
            session_id   TEXT PRIMARY KEY,
            session_file TEXT NOT NULL,
            compressed_at INTEGER NOT NULL,
            memory_count  INTEGER NOT NULL DEFAULT 0
        );
    """)
    db.execute("INSERT OR IGNORE INTO meta (key, value) VALUES ('schema_version', '1')")
    db.execute("INSERT OR IGNORE INTO meta (key, value) VALUES ('created_at', CAST(strftime('%s','now') AS TEXT))")
    db.commit()
    db.close()


def _openclaw(*args) -> subprocess.CompletedProcess:
    """Run an openclaw CLI command. Returns CompletedProcess or raises."""
    openclaw_bin = shutil.which("openclaw")
    if not openclaw_bin:
        raise FileNotFoundError("openclaw not found in PATH")
    return subprocess.run(
        [openclaw_bin, *args],
        capture_output=True, text=True, timeout=10
    )


def register_mcp(mcp_script: Path) -> bool:
    """Register the MCP server with openclaw. Returns True on success."""
    try:
        config = json.dumps({"command": "python3", "args": [str(mcp_script)]})
        r = _openclaw("mcp", "set", "openmem", config)
        return r.returncode == 0
    except Exception:
        return False


CRON_JOB_NAME = "openmem-compress"


def register_cron(auto_compress_script: Path) -> tuple[bool, str]:
    """Register an OpenClaw cron job to run auto_compress.py every hour.

    Uses --tools exec to prevent the agent from doing anything except running
    the script. The script itself handles all guards (inactivity, daily limit).
    """
    python3 = shutil.which("python3") or sys.executable
    message = (
        f"Run this command exactly as written and report the output:\n"
        f"python3 {auto_compress_script}"
    )
    try:
        r = _openclaw(
            "cron", "add",
            "--name", CRON_JOB_NAME,
            "--every", "1h",
            "--message", message,
            "--tools", "exec",
            "--session", "isolated",
            "--light-context",
            "--timeout-seconds", "180",
            "--no-deliver",
        )
        if r.returncode == 0:
            return True, "registered (openclaw cron, every 1h)"
        return False, r.stderr.strip() or r.stdout.strip()
    except FileNotFoundError:
        return False, "openclaw not in PATH"
    except Exception as e:
        return False, str(e)


def remove_cron() -> str:
    """Remove the OpenClaw cron job by name."""
    try:
        # Find the job ID by name
        r = _openclaw("cron", "list", "--json")
        if r.returncode != 0:
            return f"list failed: {r.stderr.strip()}"
        jobs = json.loads(r.stdout)
        job_ids = [j["id"] for j in jobs if j.get("name") == CRON_JOB_NAME]
        if not job_ids:
            return "not found"
        for jid in job_ids:
            _openclaw("cron", "rm", jid)
        return f"removed ({len(job_ids)} job(s))"
    except FileNotFoundError:
        return "skipped (openclaw not in PATH)"
    except Exception as e:
        return str(e)


def main():
    print("OpenMem Setup\n")

    checks = [
        ("Python version", check_python_version),
        ("SQLite FTS5",    check_sqlite_fts5),
        ("SQLite WAL",     check_sqlite_wal),
    ]

    all_ok = True
    for label, fn in checks:
        ok, msg = fn()
        status = "OK " if ok else "FAIL"
        print(f"  [{status}] {label}: {msg}")
        if not ok:
            all_ok = False

    if not all_ok:
        print("\nSetup failed. Resolve the issues above before using OpenMem.")
        sys.exit(1)

    print()

    # Init database
    db_path = Path(os.environ.get("OPENMEM_DB", str(DEFAULT_DB))).expanduser()
    existed = db_path.exists()
    init_database(db_path)

    if existed:
        print(f"  [OK ] Database already exists: {db_path}")
    else:
        print(f"  [OK ] Database created: {db_path}")

    scripts_dir = Path(__file__).parent

    # Register MCP server
    mcp_script = scripts_dir / "mcp_server.py"
    mcp_ok = register_mcp(mcp_script)
    if mcp_ok:
        print(f"  [OK ] MCP server registered (openmem)")
    else:
        print(f"  [SKIP] MCP registration failed or openclaw not in PATH")

    # Register OpenClaw cron job for auto-compression
    auto_compress_script = scripts_dir / "auto_compress.py"
    cron_ok, cron_msg = register_cron(auto_compress_script)
    if cron_ok:
        print(f"  [OK ] Auto-compression cron registered ({cron_msg})")
    else:
        print(f"  [SKIP] Auto-compression cron: {cron_msg}")

    print("\nOpenMem is ready.")
    print("\nRemaining steps:")
    print("  1. Enable bootstrap hook:  openclaw hooks enable openmem")
    print("  2. Restart gateway:        openclaw gateway restart")
    print()


if __name__ == "__main__":
    main()


path: scripts/mem.py
content: #!/usr/bin/env python3
"""
OpenMem - mem.py
Long-term memory store for OpenClaw agents.

Usage:
  mem.py init                          Init or migrate the database
  mem.py add TEXT [options]            Add a memory
  mem.py search QUERY [options]        Search memories (FTS + recency rank)
  mem.py get ID                        Get a single memory by ID
  mem.py list [options]                List recent memories
  mem.py update ID TEXT [options]      Update memory content
  mem.py delete ID                     Delete a memory
  mem.py stats                         Show database statistics
  mem.py export [--format json|md]     Export all memories

Options:
  --db PATH           Database path (default: ~/.openclaw/workspace/memory/openmem.db
                      or OPENMEM_DB env var)
  --category CAT      Category: fact|insight|preference|correction|event|general
  --importance FLOAT  Importance score 0.0-1.0 (default: 0.5)
  --source TEXT       Source label (default: manual)
  --limit N           Max results (default: 10)
  --json              Output as JSON lines
"""

import argparse
import json
import os
import re
import sqlite3
import sys
import time
from pathlib import Path

CATEGORIES = ("fact", "insight", "preference", "correction", "event", "general")
DEFAULT_DB = Path(os.environ.get("OPENMEM_DB", "~/.openclaw/workspace/memory/openmem.db")).expanduser()
CACHE_LIMIT = int(os.environ.get("OPENMEM_BOOTSTRAP_LIMIT", "20"))
SCHEMA_VERSION = 1


def write_cache(db: sqlite3.Connection, db_path: Path) -> None:
    """Write top memories to a JSON cache file the bootstrap hook reads directly.
    Called after every write operation so the hook never needs to spawn a process."""
    cache_path = db_path.parent / "openmem-cache.json"
    try:
        rows = db.execute(
            "SELECT id, content, category, importance FROM memories "
            "ORDER BY importance DESC, created_at DESC LIMIT ?",
            (CACHE_LIMIT,)
        ).fetchall()
        cache = {
            "updated_at": int(time.time()),
            "memories": [dict(r) for r in rows],
        }
        cache_path.write_text(json.dumps(cache, indent=2))
    except Exception:
        pass  # Cache write failure must never break normal operations


# ---------------------------------------------------------------------------
# Database
# ---------------------------------------------------------------------------

def open_db(path: Path) -> sqlite3.Connection:
    path.parent.mkdir(parents=True, exist_ok=True)
    db = sqlite3.connect(str(path))
    db.row_factory = sqlite3.Row
    db.execute("PRAGMA journal_mode=WAL")
    db.execute("PRAGMA foreign_keys=ON")
    return db


def init_db(db: sqlite3.Connection) -> None:
    db.executescript("""
        CREATE TABLE IF NOT EXISTS meta (
            key   TEXT PRIMARY KEY,
            value TEXT NOT NULL
        );

        CREATE TABLE IF NOT EXISTS memories (
            id           INTEGER PRIMARY KEY AUTOINCREMENT,
            content      TEXT    NOT NULL,
            category     TEXT    NOT NULL DEFAULT 'general',
            source       TEXT    NOT NULL DEFAULT 'manual',
            importance   REAL    NOT NULL DEFAULT 0.5,
            session_id   TEXT,
            created_at   INTEGER NOT NULL,
            updated_at   INTEGER NOT NULL,
            access_count INTEGER NOT NULL DEFAULT 0
        );

        CREATE UNIQUE INDEX IF NOT EXISTS idx_memories_content
            ON memories(content);

        CREATE INDEX IF NOT EXISTS idx_memories_category
            ON memories(category, created_at DESC);

        CREATE INDEX IF NOT EXISTS idx_memories_importance
            ON memories(importance DESC, created_at DESC);

        CREATE VIRTUAL TABLE IF NOT EXISTS memories_fts USING fts5(
            content,
            content='memories',
            content_rowid='id',
            tokenize='porter ascii'
        );

        CREATE TRIGGER IF NOT EXISTS memories_fts_insert
            AFTER INSERT ON memories BEGIN
                INSERT INTO memories_fts(rowid, content) VALUES (new.id, new.content);
            END;

        CREATE TRIGGER IF NOT EXISTS memories_fts_delete
            AFTER DELETE ON memories BEGIN
                INSERT INTO memories_fts(memories_fts, rowid, content)
                    VALUES ('delete', old.id, old.content);
            END;

        CREATE TRIGGER IF NOT EXISTS memories_fts_update
            AFTER UPDATE OF content ON memories BEGIN
                INSERT INTO memories_fts(memories_fts, rowid, content)
                    VALUES ('delete', old.id, old.content);
                INSERT INTO memories_fts(rowid, content) VALUES (new.id, new.content);
            END;
    """)

    # Track schema version
    db.execute(
        "INSERT OR IGNORE INTO meta (key, value) VALUES ('schema_version', ?)",
        (str(SCHEMA_VERSION),)
    )
    db.commit()


def row_to_dict(row: sqlite3.Row) -> dict:
    return dict(row)


# ---------------------------------------------------------------------------
# Commands
# ---------------------------------------------------------------------------

def cmd_init(args, db):
    init_db(db)
    print(f"OK: database ready at {args.db}")


def cmd_add(args, db):
    content = args.text.strip()
    if not content:
        print("ERROR: content cannot be empty", file=sys.stderr)
        sys.exit(1)

    category = args.category or "general"
    if category not in CATEGORIES:
        print(f"ERROR: category must be one of: {', '.join(CATEGORIES)}", file=sys.stderr)
        sys.exit(1)

    importance = float(args.importance or 0.5)
    importance = max(0.0, min(1.0, importance))
    now = int(time.time())

    try:
        cur = db.execute(
            """INSERT INTO memories (content, category, source, importance, session_id, created_at, updated_at)
               VALUES (?, ?, ?, ?, ?, ?, ?)""",
            (content, category, args.source or "manual", importance, args.session_id, now, now)
        )
        db.commit()
        write_cache(db, args.db)
        mem_id = cur.lastrowid
    except sqlite3.IntegrityError:
        # Duplicate content — update importance if higher
        row = db.execute("SELECT id, importance FROM memories WHERE content = ?", (content,)).fetchone()
        if row and importance > row["importance"]:
            db.execute(
                "UPDATE memories SET importance = ?, updated_at = ? WHERE id = ?",
                (importance, now, row["id"])
            )
            db.commit()
            write_cache(db, args.db)
            mem_id = row["id"]
            print(f"UPDATED: #{mem_id} (importance raised to {importance:.2f})")
        else:
            print(f"SKIP: duplicate content (existing id={row['id'] if row else '?'})")
        return

    if args.json:
        print(json.dumps({"id": mem_id, "category": category, "importance": importance}))
    else:
        print(f"OK: added memory #{mem_id} [{category}]")


def _is_wildcard_query(query: str) -> bool:
    return "*" in query or "?" in query


def _wildcard_to_like(query: str) -> str:
    """Convert shell-style wildcards (* and ?) to SQL LIKE pattern."""
    # Escape existing LIKE metacharacters first
    pattern = query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
    pattern = pattern.replace("*", "%").replace("?", "_")
    return pattern


def cmd_search(args, db):
    query = args.query.strip()
    if not query:
        print("ERROR: query cannot be empty", file=sys.stderr)
        sys.exit(1)

    limit = int(args.limit or 10)
    category_filter = args.category

    if _is_wildcard_query(query):
        # LIKE search — no relevance ranking, sort by importance + recency
        like_pattern = _wildcard_to_like(query)
        sql = """
            SELECT id, content, category, source, importance,
                   created_at, access_count
            FROM memories
            WHERE content LIKE ? ESCAPE '\\' COLLATE NOCASE
        """
        params = [like_pattern]

        if category_filter:
            sql += " AND category = ?"
            params.append(category_filter)

        sql += " ORDER BY importance DESC, created_at DESC LIMIT ?"
        params.append(limit)
    else:
        # FTS search ranked by relevance + importance + recency
        sql = """
            SELECT m.id, m.content, m.category, m.source, m.importance,
                   m.created_at, m.access_count,
                   bm25(memories_fts) AS fts_rank
            FROM memories_fts
            JOIN memories m ON memories_fts.rowid = m.id
            WHERE memories_fts MATCH ?
        """
        params = [query]

        if category_filter:
            sql += " AND m.category = ?"
            params.append(category_filter)

        sql += """
            ORDER BY (bm25(memories_fts) - m.importance * 2.0
                      - (CAST(strftime('%s','now') AS REAL) - m.created_at) / 86400.0 * 0.05)
            LIMIT ?
        """
        params.append(limit)

    rows = db.execute(sql, params).fetchall()

    # Bump access count
    for row in rows:
        db.execute(
            "UPDATE memories SET access_count = access_count + 1 WHERE id = ?",
            (row["id"],)
        )
    if rows:
        db.commit()

    _print_rows(rows, args.json)


def cmd_get(args, db):
    row = db.execute("SELECT * FROM memories WHERE id = ?", (args.id,)).fetchone()
    if not row:
        print(f"ERROR: memory #{args.id} not found", file=sys.stderr)
        sys.exit(1)
    db.execute("UPDATE memories SET access_count = access_count + 1 WHERE id = ?", (args.id,))
    db.commit()
    _print_rows([row], args.json)


def cmd_list(args, db):
    limit = int(args.limit or 10)
    category_filter = args.category

    sql = "SELECT * FROM memories"
    params = []

    if category_filter:
        sql += " WHERE category = ?"
        params.append(category_filter)

    sql += " ORDER BY importance DESC, created_at DESC LIMIT ?"
    params.append(limit)

    rows = db.execute(sql, params).fetchall()
    _print_rows(rows, args.json)


def cmd_update(args, db):
    content = args.text.strip()
    if not content:
        print("ERROR: content cannot be empty", file=sys.stderr)
        sys.exit(1)

    now = int(time.time())
    fields, params = ["content = ?", "updated_at = ?"], [content, now]

    if args.category:
        if args.category not in CATEGORIES:
            print(f"ERROR: invalid category", file=sys.stderr)
            sys.exit(1)
        fields.append("category = ?")
        params.append(args.category)

    if args.importance is not None:
        fields.append("importance = ?")
        params.append(max(0.0, min(1.0, float(args.importance))))

    params.append(args.id)
    db.execute(f"UPDATE memories SET {', '.join(fields)} WHERE id = ?", params)
    db.commit()
    write_cache(db, args.db)
    print(f"OK: updated memory #{args.id}")


def cmd_delete(args, db):
    cur = db.execute("DELETE FROM memories WHERE id = ?", (args.id,))
    db.commit()
    write_cache(db, args.db)
    if cur.rowcount == 0:
        print(f"ERROR: memory #{args.id} not found", file=sys.stderr)
        sys.exit(1)
    print(f"OK: deleted memory #{args.id}")


def cmd_stats(args, db):
    total = db.execute("SELECT COUNT(*) FROM memories").fetchone()[0]
    by_cat = db.execute(
        "SELECT category, COUNT(*) AS n, AVG(importance) AS avg_imp "
        "FROM memories GROUP BY category ORDER BY n DESC"
    ).fetchall()
    oldest = db.execute("SELECT MIN(created_at) FROM memories").fetchone()[0]
    newest = db.execute("SELECT MAX(created_at) FROM memories").fetchone()[0]

    if args.json:
        print(json.dumps({
            "total": total,
            "by_category": [dict(r) for r in by_cat],
            "oldest": oldest,
            "newest": newest,
        }))
        return

    print(f"Total memories: {total}")
    if oldest:
        import datetime
        print(f"Date range: {_ts(oldest)} to {_ts(newest)}")
    print()
    print(f"{'Category':<14} {'Count':>6}  {'Avg Importance':>14}")
    print("-" * 38)
    for r in by_cat:
        print(f"{r['category']:<14} {r['n']:>6}  {r['avg_imp']:>14.2f}")


def cmd_export(args, db):
    rows = db.execute("SELECT * FROM memories ORDER BY importance DESC, created_at DESC").fetchall()
    fmt = args.format or "json"

    if fmt == "json":
        for row in rows:
            print(json.dumps(row_to_dict(row)))
    elif fmt == "md":
        print("# OpenMem Export\n")
        for row in rows:
            d = row_to_dict(row)
            ts = _ts(d["created_at"])
            print(f"## [{d['category']}] #{d['id']} (importance: {d['importance']:.2f})")
            print(f"*{ts} · source: {d['source']}*\n")
            print(d["content"])
            print()
    else:
        print(f"ERROR: unknown format '{fmt}'", file=sys.stderr)
        sys.exit(1)


# ---------------------------------------------------------------------------
# Output helpers
# ---------------------------------------------------------------------------

def _ts(epoch: int) -> str:
    import datetime
    return datetime.datetime.fromtimestamp(epoch).strftime("%Y-%m-%d %H:%M")


def _print_rows(rows, as_json: bool) -> None:
    if not rows:
        print("(no results)")
        return

    if as_json:
        for row in rows:
            print(json.dumps(row_to_dict(row)))
        return

    for row in rows:
        d = row_to_dict(row)
        ts = _ts(d["created_at"])
        importance_bar = "█" * int(d["importance"] * 10)
        print(f"#{d['id']:>5}  [{d['category']:<12}]  {d['importance']:.1f} {importance_bar:<10}  {ts}")
        # Wrap content at 80 chars
        content = d["content"]
        if len(content) > 200:
            content = content[:197] + "..."
        for line in content.splitlines()[:3]:
            print(f"        {line}")
        print()


# ---------------------------------------------------------------------------
# Argument parsing
# ---------------------------------------------------------------------------

def build_parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(prog="mem.py", description="OpenMem - long-term memory CLI")
    p.add_argument("--db", type=Path, default=DEFAULT_DB, help="Database path")
    p.add_argument("--json", action="store_true", help="JSON output")

    sub = p.add_subparsers(dest="command", required=True)

    # init
    sub.add_parser("init", help="Initialize the database")

    # add
    a = sub.add_parser("add", help="Add a memory")
    a.add_argument("text", help="Memory content")
    a.add_argument("--category", choices=CATEGORIES)
    a.add_argument("--importance", type=float, default=0.5)
    a.add_argument("--source", default="manual")
    a.add_argument("--session-id", dest="session_id")

    # search
    s = sub.add_parser("search", help="Search memories")
    s.add_argument("query")
    s.add_argument("--category", choices=CATEGORIES)
    s.add_argument("--limit", type=int, default=10)

    # get
    g = sub.add_parser("get", help="Get a memory by ID")
    g.add_argument("id", type=int)

    # list
    li = sub.add_parser("list", help="List recent memories")
    li.add_argument("--category", choices=CATEGORIES)
    li.add_argument("--limit", type=int, default=10)

    # update
    u = sub.add_parser("update", help="Update a memory")
    u.add_argument("id", type=int)
    u.add_argument("text")
    u.add_argument("--category", choices=CATEGORIES)
    u.add_argument("--importance", type=float)

    # delete
    d = sub.add_parser("delete", help="Delete a memory")
    d.add_argument("id", type=int)

    # stats
    sub.add_parser("stats", help="Database statistics")

    # export
    e = sub.add_parser("export", help="Export memories")
    e.add_argument("--format", choices=["json", "md"], default="json")

    return p


# ---------------------------------------------------------------------------
# Entry point
# ---------------------------------------------------------------------------

def main():
    parser = build_parser()
    args = parser.parse_args()

    db = open_db(args.db)
    init_db(db)  # idempotent — safe to call on every run

    commands = {
        "init": cmd_init,
        "add": cmd_add,
        "search": cmd_search,
        "get": cmd_get,
        "list": cmd_list,
        "update": cmd_update,
        "delete": cmd_delete,
        "stats": cmd_stats,
        "export": cmd_export,
    }

    commands[args.command](args, db)
    db.close()


if __name__ == "__main__":
    main()


path: scripts/compress.py
content: #!/usr/bin/env python3
"""
OpenMem - compress.py
Import and inspect OpenClaw session files for memory compression.

This script handles the mechanical part: reading JSONL session files and
extracting conversation text. The actual summarization/compression is done
by the AI agent using mem.py to store the results.

Usage:
  compress.py pending [--sessions-dir PATH]    List sessions not yet compressed
  compress.py read SESSION_ID [--sessions-dir PATH]
                                               Print session messages as text
  compress.py mark-done SESSION_ID [--db PATH] Mark session as compressed
  compress.py history [--db PATH]              Show compression history
"""

import argparse
import json
import os
import sys
import time
from pathlib import Path

DEFAULT_SESSIONS_DIR = Path(
    os.environ.get(
        "OPENMEM_SESSIONS_DIR",
        "~/.openclaw/agents/main/sessions"
    )
).expanduser()

DEFAULT_DB = Path(
    os.environ.get("OPENMEM_DB", "~/.openclaw/workspace/memory/openmem.db")
).expanduser()


# ---------------------------------------------------------------------------
# Session reading
# ---------------------------------------------------------------------------

def read_session_messages(session_file: Path) -> list[dict]:
    """Extract role+content messages from an OpenClaw JSONL session file."""
    messages = []
    try:
        with open(session_file, encoding="utf-8") as f:
            for line in f:
                line = line.strip()
                if not line:
                    continue
                try:
                    obj = json.loads(line)
                except json.JSONDecodeError:
                    continue

                if obj.get("type") != "message":
                    continue

                msg = obj.get("message", {})
                role = msg.get("role")
                if role not in ("user", "assistant"):
                    continue

                # Content can be a string or list of content blocks
                content = msg.get("content", "")
                if isinstance(content, list):
                    parts = []
                    for block in content:
                        if isinstance(block, dict) and block.get("type") == "text":
                            parts.append(block.get("text", ""))
                    content = "\n".join(parts)

                content = content.strip()
                if not content:
                    continue

                # Strip internal sender metadata injected by OpenClaw
                content = _strip_sender_metadata(content)
                if not content:
                    continue

                messages.append({
                    "role": role,
                    "content": content,
                    "timestamp": obj.get("timestamp"),
                })
    except (OSError, PermissionError) as e:
        print(f"ERROR: cannot read {session_file}: {e}", file=sys.stderr)

    return messages


def _strip_sender_metadata(text: str) -> str:
    """Remove the sender metadata block OpenClaw prepends to user messages."""
    # Pattern: ```json\n{...}\n```\n\n[timestamp] actual message
    import re
    text = re.sub(
        r"^Sender \(untrusted metadata\):\s*```json\s*\{.*?\}\s*```\s*\n",
        "",
        text,
        flags=re.DOTALL
    )
    # Strip leading timestamp like "[Wed 2026-04-08 21:59 CDT] "
    text = re.sub(r"^\[[^\]]+\]\s*", "", text)
    return text.strip()


def list_session_files(sessions_dir: Path) -> list[Path]:
    """List all main session JSONL files (excludes checkpoints)."""
    if not sessions_dir.exists():
        return []
    files = []
    for p in sessions_dir.glob("*.jsonl"):
        # Skip checkpoint files (contain .checkpoint. in name)
        if ".checkpoint." not in p.name:
            files.append(p)
    return sorted(files, key=lambda p: p.stat().st_mtime)


def session_id_from_path(path: Path) -> str:
    return path.stem  # filename without .jsonl


# ---------------------------------------------------------------------------
# Compression tracking (stored in openmem DB)
# ---------------------------------------------------------------------------

def open_tracking_db(db_path: Path):
    """Open the openmem DB just for compression tracking."""
    import sqlite3
    db_path.parent.mkdir(parents=True, exist_ok=True)
    db = sqlite3.connect(str(db_path))
    db.row_factory = sqlite3.Row
    db.execute("""
        CREATE TABLE IF NOT EXISTS compressed_sessions (
            session_id   TEXT PRIMARY KEY,
            session_file TEXT NOT NULL,
            compressed_at INTEGER NOT NULL,
            memory_count  INTEGER NOT NULL DEFAULT 0
        )
    """)
    db.commit()
    return db


def get_compressed_ids(db) -> set:
    rows = db.execute("SELECT session_id FROM compressed_sessions").fetchall()
    return {r["session_id"] for r in rows}


# ---------------------------------------------------------------------------
# Commands
# ---------------------------------------------------------------------------

def cmd_pending(args):
    sessions_dir = args.sessions_dir or DEFAULT_SESSIONS_DIR
    db = open_tracking_db(args.db)
    done = get_compressed_ids(db)

    files = list_session_files(sessions_dir)
    pending = [f for f in files if session_id_from_path(f) not in done]

    if not pending:
        print("All sessions have been compressed.")
        return

    print(f"Pending sessions ({len(pending)}):\n")
    for f in pending:
        sid = session_id_from_path(f)
        mtime = time.strftime("%Y-%m-%d %H:%M", time.localtime(f.stat().st_mtime))
        size_kb = f.stat().st_size // 1024
        # Count messages (parse each line to avoid false string matches)
        msg_count = 0
        try:
            with open(f, encoding="utf-8", errors="ignore") as fh:
                for line in fh:
                    try:
                        if json.loads(line).get("type") == "message":
                            msg_count += 1
                    except json.JSONDecodeError:
                        pass
        except OSError:
            pass
        print(f"  {sid}")
        print(f"    modified: {mtime}  size: {size_kb}KB  messages: ~{msg_count}")
        print()

    print(f"To read a session:")
    print(f"  compress.py read <session-id>")
    print(f"\nAfter compressing, mark it done:")
    print(f"  compress.py mark-done <session-id>")


def cmd_read(args):
    sessions_dir = args.sessions_dir or DEFAULT_SESSIONS_DIR

    # Accept full UUID or prefix
    session_id = args.session_id
    session_file = sessions_dir / f"{session_id}.jsonl"

    if not session_file.exists():
        # Try prefix match
        matches = list(sessions_dir.glob(f"{session_id}*.jsonl"))
        matches = [m for m in matches if ".checkpoint." not in m.name]
        if len(matches) == 1:
            session_file = matches[0]
            session_id = session_id_from_path(session_file)
        elif len(matches) > 1:
            print(f"ERROR: ambiguous prefix '{session_id}', matches:", file=sys.stderr)
            for m in matches:
                print(f"  {m.name}", file=sys.stderr)
            sys.exit(1)
        else:
            print(f"ERROR: session '{session_id}' not found in {sessions_dir}", file=sys.stderr)
            sys.exit(1)

    messages = read_session_messages(session_file)
    if not messages:
        print("(no messages found)")
        return

    if args.json:
        for m in messages:
            print(json.dumps(m))
        return

    print(f"Session: {session_id}")
    print(f"Messages: {len(messages)}\n")
    print("=" * 60)

    for m in messages:
        ts = ""
        if m.get("timestamp"):
            try:
                import datetime
                ts = datetime.datetime.fromisoformat(
                    m["timestamp"].replace("Z", "+00:00")
                ).strftime(" [%Y-%m-%d %H:%M]")
            except Exception:
                pass
        role_label = "USER" if m["role"] == "user" else "AGENT"
        print(f"\n{role_label}{ts}:")
        print(m["content"])

    print("\n" + "=" * 60)
    print("\nCompress this session by reading the above and running:")
    print(f"  mem.py add \"<summary>\" --category insight --source session:{session_id}")
    print(f"  compress.py mark-done {session_id}")


def cmd_mark_done(args):
    db = open_tracking_db(args.db)
    now = int(time.time())
    db.execute(
        """INSERT OR REPLACE INTO compressed_sessions
           (session_id, session_file, compressed_at, memory_count)
           VALUES (?, ?, ?, ?)""",
        (args.session_id, str(args.session_id) + ".jsonl", now, args.memory_count or 0)
    )
    db.commit()
    print(f"OK: marked session {args.session_id} as compressed")


def cmd_history(args):
    db = open_tracking_db(args.db)
    rows = db.execute(
        "SELECT * FROM compressed_sessions ORDER BY compressed_at DESC"
    ).fetchall()

    if not rows:
        print("No sessions compressed yet.")
        return

    if args.json:
        for r in rows:
            print(json.dumps(dict(r)))
        return

    print(f"{'Session ID':<40} {'Compressed At':<20} {'Memories'}")
    print("-" * 70)
    for r in rows:
        ts = time.strftime("%Y-%m-%d %H:%M", time.localtime(r["compressed_at"]))
        print(f"{r['session_id']:<40} {ts:<20} {r['memory_count']}")


# ---------------------------------------------------------------------------
# Argument parsing
# ---------------------------------------------------------------------------

def build_parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(prog="compress.py", description="OpenMem session compression helper")
    p.add_argument("--db", type=Path, default=DEFAULT_DB)
    p.add_argument("--json", action="store_true")

    sub = p.add_subparsers(dest="command", required=True)

    pending = sub.add_parser("pending", help="List sessions not yet compressed")
    pending.add_argument("--sessions-dir", type=Path, dest="sessions_dir")

    read = sub.add_parser("read", help="Print session messages")
    read.add_argument("session_id")
    read.add_argument("--sessions-dir", type=Path, dest="sessions_dir")

    done = sub.add_parser("mark-done", help="Mark a session as compressed")
    done.add_argument("session_id")
    done.add_argument("--memory-count", type=int, default=0, dest="memory_count")

    sub.add_parser("history", help="Show compression history")

    return p


def main():
    parser = build_parser()
    args = parser.parse_args()

    commands = {
        "pending": cmd_pending,
        "read": cmd_read,
        "mark-done": cmd_mark_done,
        "history": cmd_history,
    }
    commands[args.command](args)


if __name__ == "__main__":
    main()


path: scripts/uninstall.py
content: #!/usr/bin/env python3
"""
OpenMem - uninstall.py
Removes OpenMem's registered components from OpenClaw.

Your memory database is NOT deleted. Its location is printed at the end
so you can back it up, migrate it, or delete it yourself.

Run:
  python3 ~/.openclaw/workspace/skills/openmem/scripts/uninstall.py
"""

import json
import os
import shutil
import subprocess
from pathlib import Path

DEFAULT_DB = Path(
    os.environ.get("OPENMEM_DB", "~/.openclaw/workspace/memory/openmem.db")
).expanduser()

MCP_NAME = "openmem"
HOOK_NAME = "openmem"
CRON_JOB_NAME = "openmem-compress"


def _openclaw(*args) -> subprocess.CompletedProcess:
    openclaw_bin = shutil.which("openclaw")
    if not openclaw_bin:
        raise FileNotFoundError("openclaw not found in PATH")
    return subprocess.run(
        [openclaw_bin, *args],
        capture_output=True, text=True, timeout=10
    )


def remove_mcp() -> str:
    try:
        r = _openclaw("mcp", "unset", MCP_NAME)
        return "removed" if r.returncode == 0 else f"failed ({r.stderr.strip()})"
    except FileNotFoundError:
        return "skipped (openclaw not in PATH)"
    except Exception as e:
        return f"failed ({e})"


def disable_hook() -> str:
    try:
        r = _openclaw("hooks", "disable", HOOK_NAME)
        if r.returncode == 0:
            return "disabled"
        return f"failed ({r.stderr.strip()})" if r.stderr.strip() else "not enabled"
    except FileNotFoundError:
        return "skipped (openclaw not in PATH)"
    except Exception as e:
        return f"failed ({e})"


def _openclaw(*args) -> subprocess.CompletedProcess:
    openclaw_bin = shutil.which("openclaw")
    if not openclaw_bin:
        raise FileNotFoundError("openclaw not found in PATH")
    return subprocess.run([openclaw_bin, *args], capture_output=True, text=True, timeout=10)


def remove_cron() -> str:
    try:
        r = _openclaw("cron", "list", "--json")
        if r.returncode != 0:
            return f"list failed: {r.stderr.strip()}"
        jobs = json.loads(r.stdout)
        job_ids = [j["id"] for j in jobs if j.get("name") == CRON_JOB_NAME]
        if not job_ids:
            return "not found"
        for jid in job_ids:
            _openclaw("cron", "rm", jid)
        return f"removed ({len(job_ids)} job(s))"
    except FileNotFoundError:
        return "skipped (openclaw not in PATH)"
    except Exception as e:
        return str(e)


def main():
    print("OpenMem Uninstall\n")

    mcp_status   = remove_mcp()
    hook_status  = disable_hook()
    cron_status  = remove_cron()

    print(f"  MCP server ({MCP_NAME}):   {mcp_status}")
    print(f"  Bootstrap hook:            {hook_status}")
    print(f"  Auto-compression cron:     {cron_status}")

    db_path = Path(os.environ.get("OPENMEM_DB", str(DEFAULT_DB))).expanduser()
    cache_path = db_path.parent / "openmem-cache.json"

    print()
    print("Your memory database was NOT deleted.")
    print(f"  Database:  {db_path}")
    if cache_path.exists():
        print(f"  Cache:     {cache_path}")
    print()
    print("To export your memories before deleting:")
    skill_dir = Path(__file__).parent
    print(f"  python3 {skill_dir / 'mem.py'} export --format md > memories.md")
    print()
    print("To delete the database permanently:")
    print(f"  rm {db_path}")
    if cache_path.exists():
        print(f"  rm {cache_path}")
    print()
    skill_path = Path(__file__).parent.parent
    print("OpenMem components have been removed from OpenClaw.")
    print("To remove the skill directory itself:")
    print(f"  rm -rf {skill_path}")
    print()
    print("Restart the gateway for changes to take effect:")
    print("  openclaw gateway restart")
    print()


if __name__ == "__main__":
    main()


path: scripts/mcp_server.py
content: #!/usr/bin/env python3
"""
OpenMem - MCP Server
Exposes memory operations as structured tool calls via the Model Context Protocol.

Registered in OpenClaw via:
  openclaw mcp set openmem '{"command":"python3","args":["<path>/mcp_server.py"]}'

Tools exposed:
  memory_search  - Full-text search memories
  memory_add     - Add or update a memory
  memory_update  - Update content/category/importance of an existing memory
  memory_delete  - Delete a memory by ID
  memory_list    - List recent memories
  memory_stats   - Database statistics

Transport: MCP stdio (newline-delimited JSON-RPC 2.0)
"""

import json
import os
import re
import sqlite3
import sys
import time
from pathlib import Path
from typing import Optional

# ---------------------------------------------------------------------------
# Config
# ---------------------------------------------------------------------------

DEFAULT_DB = Path(
    os.environ.get("OPENMEM_DB", "~/.openclaw/workspace/memory/openmem.db")
).expanduser()

CATEGORIES = ("fact", "insight", "preference", "correction", "event", "general")
SERVER_NAME = "openmem"
SERVER_VERSION = "1.3.0"
CACHE_LIMIT = int(os.environ.get("OPENMEM_BOOTSTRAP_LIMIT", "20"))


def write_cache(db: sqlite3.Connection) -> None:
    """Write top memories to JSON cache for the bootstrap hook."""
    cache_path = DEFAULT_DB.parent / "openmem-cache.json"
    try:
        rows = db.execute(
            "SELECT id, content, category, importance FROM memories "
            "ORDER BY importance DESC, created_at DESC LIMIT ?",
            (CACHE_LIMIT,)
        ).fetchall()
        cache = {"updated_at": int(time.time()), "memories": [dict(r) for r in rows]}
        cache_path.write_text(json.dumps(cache, indent=2))
    except Exception:
        pass

# ---------------------------------------------------------------------------
# Database (inline — no dep on mem.py to keep server self-contained)
# ---------------------------------------------------------------------------

def open_db() -> sqlite3.Connection:
    DEFAULT_DB.parent.mkdir(parents=True, exist_ok=True)
    db = sqlite3.connect(str(DEFAULT_DB))
    db.row_factory = sqlite3.Row
    db.execute("PRAGMA journal_mode=WAL")
    _init_schema(db)
    return db


def _init_schema(db: sqlite3.Connection) -> None:
    db.executescript("""
        CREATE TABLE IF NOT EXISTS memories (
            id           INTEGER PRIMARY KEY AUTOINCREMENT,
            content      TEXT    NOT NULL,
            category     TEXT    NOT NULL DEFAULT 'general',
            source       TEXT    NOT NULL DEFAULT 'manual',
            importance   REAL    NOT NULL DEFAULT 0.5,
            session_id   TEXT,
            created_at   INTEGER NOT NULL,
            updated_at   INTEGER NOT NULL,
            access_count INTEGER NOT NULL DEFAULT 0
        );
        CREATE UNIQUE INDEX IF NOT EXISTS idx_memories_content ON memories(content);
        CREATE INDEX IF NOT EXISTS idx_memories_category ON memories(category, created_at DESC);
        CREATE INDEX IF NOT EXISTS idx_memories_importance ON memories(importance DESC, created_at DESC);
        CREATE VIRTUAL TABLE IF NOT EXISTS memories_fts USING fts5(
            content, content='memories', content_rowid='id', tokenize='porter ascii'
        );
        CREATE TRIGGER IF NOT EXISTS memories_fts_insert
            AFTER INSERT ON memories BEGIN
                INSERT INTO memories_fts(rowid, content) VALUES (new.id, new.content);
            END;
        CREATE TRIGGER IF NOT EXISTS memories_fts_delete
            AFTER DELETE ON memories BEGIN
                INSERT INTO memories_fts(memories_fts, rowid, content)
                    VALUES ('delete', old.id, old.content);
            END;
        CREATE TRIGGER IF NOT EXISTS memories_fts_update
            AFTER UPDATE OF content ON memories BEGIN
                INSERT INTO memories_fts(memories_fts, rowid, content)
                    VALUES ('delete', old.id, old.content);
                INSERT INTO memories_fts(rowid, content) VALUES (new.id, new.content);
            END;
    """)
    db.commit()


def _row(row: sqlite3.Row) -> dict:
    d = dict(row)
    # Humanize timestamps for LLM readability
    for key in ("created_at", "updated_at"):
        if d.get(key):
            import datetime
            d[key + "_human"] = datetime.datetime.fromtimestamp(d[key]).strftime("%Y-%m-%d %H:%M")
    return d

# ---------------------------------------------------------------------------
# Tool implementations
# ---------------------------------------------------------------------------

def _is_wildcard_query(query: str) -> bool:
    return "*" in query or "?" in query


def _wildcard_to_like(query: str) -> str:
    """Convert shell-style wildcards (* and ?) to SQL LIKE pattern."""
    pattern = query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
    pattern = pattern.replace("*", "%").replace("?", "_")
    return pattern


def tool_memory_search(db, args: dict) -> dict:
    query = (args.get("query") or "").strip()
    if not query:
        return {"error": "query is required"}

    limit = min(int(args.get("limit", 10)), 50)
    category = args.get("category")

    if _is_wildcard_query(query):
        # LIKE search — no relevance ranking, sort by importance + recency
        like_pattern = _wildcard_to_like(query)
        sql = """
            SELECT id, content, category, source, importance,
                   created_at, updated_at, access_count
            FROM memories
            WHERE content LIKE ? ESCAPE '\\' COLLATE NOCASE
        """
        params = [like_pattern]

        if category:
            sql += " AND category = ?"
            params.append(category)

        sql += " ORDER BY importance DESC, created_at DESC LIMIT ?"
        params.append(limit)
    else:
        # FTS search ranked by relevance + importance + recency
        sql = """
            SELECT m.id, m.content, m.category, m.source, m.importance,
                   m.created_at, m.updated_at, m.access_count,
                   bm25(memories_fts) AS fts_rank
            FROM memories_fts
            JOIN memories m ON memories_fts.rowid = m.id
            WHERE memories_fts MATCH ?
        """
        params = [query]

        if category:
            sql += " AND m.category = ?"
            params.append(category)

        sql += """
            ORDER BY (bm25(memories_fts) - m.importance * 2.0
                      - (CAST(strftime('%s','now') AS REAL) - m.created_at) / 86400.0 * 0.05)
            LIMIT ?
        """
        params.append(limit)

    rows = db.execute(sql, params).fetchall()

    for row in rows:
        db.execute("UPDATE memories SET access_count = access_count + 1 WHERE id = ?", (row["id"],))
    if rows:
        db.commit()

    return {
        "count": len(rows),
        "memories": [_row(r) for r in rows],
    }


def _significant_words(text: str) -> set:
    """Extract meaningful words for duplicate detection."""
    stopwords = {"the", "that", "this", "with", "from", "have", "will", "been",
                 "they", "were", "when", "what", "which", "there", "their",
                 "them", "then", "than", "your", "also", "into", "some",
                 "just", "over", "like", "such", "only", "more", "very"}
    words = set(re.findall(r"\b[a-zA-Z]{4,}\b", text.lower()))
    return words - stopwords


def _find_similar(db, content: str, threshold: float = 0.65) -> Optional[dict]:
    """Return existing memory if word-overlap with content exceeds threshold."""
    words = _significant_words(content)
    if len(words) < 3:
        return None
    query_words = list(words)[:6]
    fts_query = " OR ".join(f'"{w}"' for w in query_words)
    try:
        rows = db.execute(
            "SELECT m.id, m.content, m.importance FROM memories m "
            "JOIN memories_fts ON m.id = memories_fts.rowid "
            "WHERE memories_fts MATCH ? LIMIT 10",
            (fts_query,)
        ).fetchall()
    except Exception:
        return None
    for row in rows:
        existing_words = _significant_words(row["content"])
        if not existing_words:
            continue
        overlap = len(words & existing_words) / min(len(words), len(existing_words))
        if overlap >= threshold:
            return dict(row)
    return None


def tool_memory_add(db, args: dict) -> dict:
    content = (args.get("content") or "").strip()
    if not content:
        return {"error": "content is required"}

    category = args.get("category", "general")
    if category not in CATEGORIES:
        return {"error": f"category must be one of: {', '.join(CATEGORIES)}"}

    importance = float(args.get("importance", 0.5))
    importance = max(0.0, min(1.0, importance))
    source = str(args.get("source", "agent"))
    session_id = args.get("session_id")
    now = int(time.time())

    # Semantic duplicate check (unless caller opts out)
    if args.get("check_duplicate", True):
        similar = _find_similar(db, content)
        if similar:
            # Bump importance if new value is higher
            if importance > similar["importance"]:
                db.execute("UPDATE memories SET importance = ?, updated_at = ? WHERE id = ?",
                           (importance, now, similar["id"]))
                db.commit()
                write_cache(db)
                return {"added": False, "updated": True, "id": similar["id"],
                        "note": "similar memory exists — importance raised",
                        "existing": similar["content"][:120]}
            return {"added": False, "updated": False, "id": similar["id"],
                    "note": "similar memory already exists",
                    "existing": similar["content"][:120]}

    try:
        cur = db.execute(
            """INSERT INTO memories (content, category, source, importance, session_id, created_at, updated_at)
               VALUES (?, ?, ?, ?, ?, ?, ?)""",
            (content, category, source, importance, session_id, now, now)
        )
        db.commit()
        write_cache(db)
        return {"added": True, "id": cur.lastrowid, "category": category, "importance": importance}
    except sqlite3.IntegrityError:
        # Exact duplicate on content column
        row = db.execute("SELECT id, importance FROM memories WHERE content = ?", (content,)).fetchone()
        if row and importance > row["importance"]:
            db.execute("UPDATE memories SET importance = ?, updated_at = ? WHERE id = ?",
                       (importance, now, row["id"]))
            db.commit()
            write_cache(db)
            return {"added": False, "updated": True, "id": row["id"],
                    "note": "exact duplicate — importance raised"}
        return {"added": False, "updated": False, "id": row["id"] if row else None,
                "note": "exact duplicate — no change needed"}


def tool_memory_update(db, args: dict) -> dict:
    mem_id = args.get("id")
    if not mem_id:
        return {"error": "id is required"}

    row = db.execute("SELECT * FROM memories WHERE id = ?", (int(mem_id),)).fetchone()
    if not row:
        return {"error": f"memory #{mem_id} not found"}

    now = int(time.time())
    fields, params = ["updated_at = ?"], [now]

    if "content" in args:
        content = args["content"].strip()
        if not content:
            return {"error": "content cannot be empty"}
        fields.append("content = ?")
        params.append(content)

    if "category" in args:
        if args["category"] not in CATEGORIES:
            return {"error": f"category must be one of: {', '.join(CATEGORIES)}"}
        fields.append("category = ?")
        params.append(args["category"])

    if "importance" in args:
        imp = max(0.0, min(1.0, float(args["importance"])))
        fields.append("importance = ?")
        params.append(imp)

    if len(fields) == 1:
        return {"error": "no fields to update (provide content, category, or importance)"}

    params.append(int(mem_id))
    db.execute(f"UPDATE memories SET {', '.join(fields)} WHERE id = ?", params)
    db.commit()
    write_cache(db)

    updated = db.execute("SELECT * FROM memories WHERE id = ?", (int(mem_id),)).fetchone()
    return {"updated": True, "memory": _row(updated)}


def tool_memory_delete(db, args: dict) -> dict:
    mem_id = args.get("id")
    if not mem_id:
        return {"error": "id is required"}

    cur = db.execute("DELETE FROM memories WHERE id = ?", (int(mem_id),))
    db.commit()
    write_cache(db)

    if cur.rowcount == 0:
        return {"error": f"memory #{mem_id} not found"}
    return {"deleted": True, "id": int(mem_id)}


def tool_memory_list(db, args: dict) -> dict:
    limit = min(int(args.get("limit", 10)), 100)
    category = args.get("category")
    order = args.get("order", "importance")  # importance | recent | access

    order_clause = {
        "importance": "importance DESC, created_at DESC",
        "recent": "created_at DESC",
        "access": "access_count DESC, importance DESC",
    }.get(order, "importance DESC, created_at DESC")

    sql = "SELECT * FROM memories"
    params = []

    if category:
        sql += " WHERE category = ?"
        params.append(category)

    sql += f" ORDER BY {order_clause} LIMIT ?"
    params.append(limit)

    rows = db.execute(sql, params).fetchall()
    return {
        "count": len(rows),
        "memories": [_row(r) for r in rows],
    }


def tool_memory_stats(db, _args: dict) -> dict:
    total = db.execute("SELECT COUNT(*) FROM memories").fetchone()[0]
    by_cat = db.execute(
        "SELECT category, COUNT(*) AS count, AVG(importance) AS avg_importance, "
        "MAX(created_at) AS latest "
        "FROM memories GROUP BY category ORDER BY count DESC"
    ).fetchall()
    oldest = db.execute("SELECT MIN(created_at) FROM memories").fetchone()[0]
    newest = db.execute("SELECT MAX(created_at) FROM memories").fetchone()[0]

    import datetime
    return {
        "total_memories": total,
        "by_category": [dict(r) for r in by_cat],
        "oldest_memory": datetime.datetime.fromtimestamp(oldest).strftime("%Y-%m-%d") if oldest else None,
        "newest_memory": datetime.datetime.fromtimestamp(newest).strftime("%Y-%m-%d") if newest else None,
        "db_path": str(DEFAULT_DB),
    }


# ---------------------------------------------------------------------------
# Tool registry
# ---------------------------------------------------------------------------

TOOLS = {
    "memory_search": {
        "description": "Search long-term memories using full-text search. Returns memories ranked by relevance, importance, and recency.",
        "inputSchema": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "Search query. Supports FTS5 syntax: phrases in quotes, AND/OR/NOT operators, prefix* wildcards."
                },
                "limit": {
                    "type": "integer",
                    "description": "Max results to return (default: 10, max: 50)",
                    "default": 10
                },
                "category": {
                    "type": "string",
                    "enum": list(CATEGORIES),
                    "description": "Filter by category (optional)"
                }
            },
            "required": ["query"]
        },
        "fn": tool_memory_search,
    },
    "memory_add": {
        "description": "Add a new long-term memory. Automatically checks for similar existing memories (word-overlap deduplication) — returns the existing memory ID if a near-duplicate is found instead of inserting.",
        "inputSchema": {
            "type": "object",
            "properties": {
                "content": {
                    "type": "string",
                    "description": "The memory text. Be specific and self-contained — this will be read without conversation context."
                },
                "category": {
                    "type": "string",
                    "enum": list(CATEGORIES),
                    "description": "fact=objective info | insight=learned pattern | preference=user desire | correction=mistake+fix | event=something that happened | general=other",
                    "default": "general"
                },
                "importance": {
                    "type": "number",
                    "description": "0.0-1.0. 0.9+=critical, 0.7-0.8=important, 0.5=normal, 0.3=low",
                    "default": 0.5,
                    "minimum": 0.0,
                    "maximum": 1.0
                },
                "source": {
                    "type": "string",
                    "description": "Where this came from (e.g. 'agent', 'session:uuid', 'user')",
                    "default": "agent"
                },
                "check_duplicate": {
                    "type": "boolean",
                    "description": "Check for semantically similar memories before inserting (default: true). Set false to force insert.",
                    "default": True
                }
            },
            "required": ["content"]
        },
        "fn": tool_memory_add,
    },
    "memory_update": {
        "description": "Update an existing memory's content, category, or importance by ID.",
        "inputSchema": {
            "type": "object",
            "properties": {
                "id": {
                    "type": "integer",
                    "description": "Memory ID (from search or list results)"
                },
                "content": {
                    "type": "string",
                    "description": "New content (optional)"
                },
                "category": {
                    "type": "string",
                    "enum": list(CATEGORIES),
                    "description": "New category (optional)"
                },
                "importance": {
                    "type": "number",
                    "description": "New importance 0.0-1.0 (optional)",
                    "minimum": 0.0,
                    "maximum": 1.0
                }
            },
            "required": ["id"]
        },
        "fn": tool_memory_update,
    },
    "memory_delete": {
        "description": "Permanently delete a memory by ID.",
        "inputSchema": {
            "type": "object",
            "properties": {
                "id": {
                    "type": "integer",
                    "description": "Memory ID to delete"
                }
            },
            "required": ["id"]
        },
        "fn": tool_memory_delete,
    },
    "memory_list": {
        "description": "List memories sorted by importance, recency, or access count.",
        "inputSchema": {
            "type": "object",
            "properties": {
                "limit": {
                    "type": "integer",
                    "description": "Max results (default: 10, max: 100)",
                    "default": 10
                },
                "category": {
                    "type": "string",
                    "enum": list(CATEGORIES),
                    "description": "Filter by category (optional)"
                },
                "order": {
                    "type": "string",
                    "enum": ["importance", "recent", "access"],
                    "description": "Sort order: importance (default), recent, access",
                    "default": "importance"
                }
            },
        },
        "fn": tool_memory_list,
    },
    "memory_stats": {
        "description": "Get statistics about the memory database: total count, breakdown by category, date range.",
        "inputSchema": {
            "type": "object",
            "properties": {},
        },
        "fn": tool_memory_stats,
    },
}

# ---------------------------------------------------------------------------
# MCP stdio transport (newline-delimited JSON-RPC 2.0)
# ---------------------------------------------------------------------------

def read_message():
    """Read one newline-delimited JSON-RPC message from stdin.

    Returns the parsed message dict, or None on EOF. Blank lines and
    unparseable lines are skipped so a single bad line does not kill
    the server.
    """
    while True:
        line = sys.stdin.readline()
        if not line:
            return None  # EOF
        line = line.strip()
        if not line:
            continue  # blank line — skip
        try:
            return json.loads(line)
        except json.JSONDecodeError:
            continue  # malformed line — skip


def write_message(obj: dict) -> None:
    """Write one newline-delimited JSON-RPC message to stdout."""
    sys.stdout.write(json.dumps(obj) + "\n")
    sys.stdout.flush()


def make_response(req_id, result) -> dict:
    return {"jsonrpc": "2.0", "id": req_id, "result": result}


def make_error(req_id, code: int, message: str) -> dict:
    return {"jsonrpc": "2.0", "id": req_id, "error": {"code": code, "message": message}}


# ---------------------------------------------------------------------------
# Request handlers
# ---------------------------------------------------------------------------

def handle_initialize(req_id, _params) -> dict:
    return make_response(req_id, {
        "protocolVersion": "2024-11-05",
        "capabilities": {"tools": {}},
        "serverInfo": {"name": SERVER_NAME, "version": SERVER_VERSION},
    })


def handle_tools_list(req_id, _params) -> dict:
    tools = []
    for name, defn in TOOLS.items():
        tools.append({
            "name": name,
            "description": defn["description"],
            "inputSchema": defn["inputSchema"],
        })
    return make_response(req_id, {"tools": tools})


def handle_tools_call(req_id, params, db) -> dict:
    name = params.get("name", "")
    args = params.get("arguments", {})

    if name not in TOOLS:
        return make_error(req_id, -32601, f"Unknown tool: {name}")

    try:
        result = TOOLS[name]["fn"](db, args)
    except Exception as e:
        return make_error(req_id, -32603, f"Tool error: {e}")

    # MCP tool result format
    return make_response(req_id, {
        "content": [
            {
                "type": "text",
                "text": json.dumps(result, indent=2, default=str),
            }
        ],
        "isError": "error" in result,
    })


# ---------------------------------------------------------------------------
# Main loop
# ---------------------------------------------------------------------------

def main():
    db = open_db()

    while True:
        msg = read_message()
        if msg is None:
            break

        req_id = msg.get("id")
        method = msg.get("method", "")
        params = msg.get("params", {})

        if method == "initialize":
            write_message(handle_initialize(req_id, params))
        elif method == "initialized":
            pass  # notification, no response
        elif method == "tools/list":
            write_message(handle_tools_list(req_id, params))
        elif method == "tools/call":
            write_message(handle_tools_call(req_id, params, db))
        elif method == "ping":
            write_message(make_response(req_id, {}))
        elif req_id is not None:
            # Unknown method with an ID — must respond
            write_message(make_error(req_id, -32601, f"Method not found: {method}"))
        # Notifications (no id) are silently ignored

    db.close()


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)