Skill: zgjq/context-compactor-zero-dep (version 1.1.3)

--- name: context-compactor description: > Automatic context compression for OpenClaw sessions. Summarizes long conversations into structured digests (decisions, facts, pending items, technical details), saves compressed summaries to disk, and injects them into new sessions. Reduces token waste from redundant context repetition. Use when conversation exceeds ~50 turns, when context feels bloated, when starting a new session and prior context is needed, or when the user says "compact", "summarize this conversation", "start fresh but remember". --- # Context Compactor Compresses long conversations into structured summaries. Saves token by replacing raw conversation history with dense, searchable digests. ## Requirements - **Runtime**: Python 3.10+ (standard library only) - **OS**: Linux, macOS - **Environment variables**: - `OPENCLAW_WORKSPACE` — Workspace root (default: `~/.openclaw/workspace`) - **Environment**: `OPENCLAW_WORKSPACE` (default: `~/.openclaw/workspace`) ## Security ### Write Restrictions (Hard Rules) The agent may ONLY write to these locations: - `memory/compacts/` — compact files only - `SESSION-STATE.md` — via smart-memory skill (not this skill) The agent MUST NOT write to: - Any directory outside the workspace - System directories (/etc, /usr, /var, /tmp except session cache) - User home directory root (~/.ssh, ~/.config, ~/.aws, etc.) - Any `.*` dotfile or dotdir in workspace root - Any file not matching `memory/compacts/*.md` ### Read Restrictions The agent may ONLY read from: - The current conversation context (already available) - `memory/compacts/` — for listing/reading past compacts The agent MUST NOT read from: - `/etc/passwd`, `/etc/shadow`, or any system credential file - `~/.ssh/`, `~/.aws/`, `~/.config/openclaw/` or similar - Any file outside the workspace unless explicitly asked by the user ### Redaction Enforcement Before writing ANY compact, the agent MUST run this checklist: 1. ✅ No file paths (replace with `<REDACTED_PATH>`) 2. ✅ No URLs (replace with `<REDACTED_URL>`) 3. ✅ No internal IPs (replace with `<INTERNAL_URL>`) 4. ✅ No passwords, tokens, API keys (delete entirely) 5. ✅ No personal info beyond work context If any item fails → do not write the compact until fixed. ### Privacy Boundaries ### What Compacts May Contain - Decisions, facts, pending actions, blockers - Project names and feature descriptions ### What Compacts Must NOT Contain - File system paths (use `<REDACTED_PATH>` placeholder) - Internal URLs, endpoints, or infrastructure details - API keys, tokens, passwords, secrets (script-level regex filter) - Private keys or certificates - User personal information beyond work context ### Agent Rules - Before saving a compact, strip or redact all paths, internal URLs, and credentials - Use placeholders: `<PROJECT_ROOT>`, `<INTERNAL_URL>`, `<DB_CONFIG>` - If unsure whether something is sensitive, redact it - Compacts are for "what was decided" not "where things live" ### Data Isolation - **Keyword extraction**: Fully local — no network calls, no external transmission - **LLM extraction** (opt-in): Sends conversation text to the agent's configured LLM provider. This is inherent to using any LLM-based compaction and is **not** controlled by this skill. The skill only provides the prompt template; the agent/platform handles the actual API call. - Maximum 30 compacts retained, oldest auto-deleted - Compacts are stored locally and read locally — the skill itself never makes network calls ## How It Works ``` Long conversation (10,000+ tokens) ↓ compact_session.py --extract ↓ Structured digest (~500 tokens) ↓ Saved to memory/compacts/YYYY-MM-DD-HHMM.md ↓ New session reads latest compact on startup ``` **Compression ratio: ~20:1** — a 10,000 token conversation becomes ~500 token digest. ## Quick Reference | Action | Script | |--------|--------| | Write compact (agent-authored) | `python3 scripts/compact_session.py --write < compact.md` | | List compacts | `python3 scripts/compact_session.py --list` | | Read latest compact | `python3 scripts/compact_session.py --latest` | | Show compact stats | `python3 scripts/compact_session.py --stats` | **How compaction works**: The agent drafts the compact content, then pipes it through `--write` which enforces security checks (no paths, URLs, IPs, secrets) before saving. This ensures programmatic enforcement — the agent never writes directly to disk. ## Compact Format ```markdown # Session Compact — 2026-03-31 17:00 UTC **Turns**: 45 | **Est. tokens saved**: ~9,500 ## Decisions Made - [2026-03-31] Chose SQLite over Redis for golden3 prompts - [2026-03-31] Decided to use zero-dependency approach for smart-memory ## Facts Established - [PROJ] golden3 site at golden3.killclaw.xyz, repo github.com/zgjq/golden3 - [TECH] Prompts stored in data/golden3.db, table `prompts`, category `scoring` - [PREF] User prefers direct, no-nonsense communication style ## Pending Actions - [ ] Publish smart-memory to ClawHub - [ ] Fix scoring display from 100-point to 10-point scale ## Technical Context - Server: ubuntu-4gb-hel1-1, Node v24.14.0 - Golden3 uses node:sqlite (DatabaseSync) ## Blockers / Open Questions - Need ClawHub login token to publish ## Session Summary Built smart-memory skill from Claude Code architecture study. Published to ClawHub as smart-memory-zero-dep. Memory system now active with WAL protocol, type classification, temporal decay, and snapshot/restore. ``` ## Agent Behavior ### When to Compact - Conversation exceeds ~50 turns - Context window approaching limits (see AGENTS.md token discipline rules) - User says "compact", "summarize", "fresh start", "压缩" ### How to Compact The agent drafts the compact content, then saves it via: ``` echo "compact content" | python3 scripts/compact_session.py --write ``` The `--write` flag enforces all security checks programmatically: redaction of paths/URLs/IPs/secrets, path containment within workspace, and file naming. The agent NEVER writes directly to disk. ### When to Inject Compact - New session startup — check for recent compact - User asks "what were we working on" - Context search returns nothing but a compact exists ### Extraction Rules From the conversation, extract: 1. **Decisions** — anything with "chose X over Y", "decided to", "going with" 2. **Facts** — URLs, file paths, configs, technical details, user preferences 3. **Pending** — uncompleted tasks, "later", "TODO", "next step" 4. **Blockers** — "need X first", "blocked by", "waiting for" 5. **Summary** — 2-3 sentence overview of what happened Skip: - Greetings, small talk, "ok", "嗯" - Repeated information already in MEMORY.md - Debugging noise (unless it led to a LESSON) - Sensitive data (tokens, passwords) ## Integration with smart-memory This skill works with `smart-memory`: - Compacts reference `[TYPE]` tags from smart-memory's classification system - Pending items from compacts can feed into `wal pending` - Facts from compacts can be promoted to MEMORY.md - Compacts live in `memory/compacts/` — decayed by smart-memory's archival system ## File Structure ``` ~/.openclaw/workspace/ ├── memory/ │ └── compacts/ │ ├── 2026-03-31-1700.md # Session compact │ ├── 2026-03-30-1430.md │ └── ... ```

path: references/compaction_prompt.md
content: # Compaction Prompt Template

When the agent decides to use LLM-based compaction (for complex conversations
where keyword extraction isn't sufficient), use this prompt.

## LLM Prompt

```
Compress the following conversation into a structured session digest.
Be concise. Omit filler, greetings, and repeated information.

Output format:

## Decisions Made
- [date] Decision description

## Facts Established
- [TYPE] Fact (use [PROJ] [TECH] [PREF] [PEOPLE] tags)

## Pending Actions
- [ ] Action description

## Blockers
- Blocker description

## Session Summary
(2-3 sentences max)

Rules:
1. Max 15 items total across all sections
2. No passwords, tokens, or API keys
3. No file paths, internal URLs, or infrastructure details — use placeholders like <PROJECT_ROOT>
4. Skip debugging noise unless it led to a lesson
5. Be specific about decisions but vague about locations
6. Be specific: "chose SQLite for prompt storage" not "chose a database"

Conversation:
---
{conversation_text}
---
```

## When to Use LLM vs Keyword

| Scenario | Method |
|----------|--------|
| Simple Q&A, <20 turns | Keyword (`compact_session.py --extract`) |
| Multi-topic, 20-50 turns | Either works |
| Complex debugging, >50 turns | LLM (needs nuanced understanding) |
| User wants exact summary | LLM (better at following instructions) |
| Quick save before /new | Keyword (fast, zero cost) |


path: scripts/compact_session.py
content: #!/usr/bin/env python3
"""
compact_session.py — Context compression for OpenClaw sessions.

Extracts structured digests from conversation text, saves to disk,
and retrieves recent compacts for injection into new sessions.

Usage:
  compact_session.py --extract < conversation.txt
  compact_session.py --from-file conversation.md
  compact_session.py --list
  compact_session.py --latest
  compact_session.py --stats
"""

import argparse
import os
import re
import sys
from datetime import datetime, timezone
from pathlib import Path

WORKSPACE = Path(os.environ.get("OPENCLAW_WORKSPACE", os.path.expanduser("~/.openclaw/workspace")))
COMPACTS_DIR = WORKSPACE / "memory" / "compacts"

# Hard security: verify all paths stay within workspace
def assert_safe_path(path: Path) -> None:
    """Reject any path outside workspace/memory/compacts/."""
    try:
        resolved = path.resolve()
        workspace_resolved = WORKSPACE.resolve()
        if not str(resolved).startswith(str(workspace_resolved)):
            raise ValueError(f"Path outside workspace: {resolved}")
        if not str(resolved).startswith(str(COMPACTS_DIR.resolve())):
            if resolved != WORKSPACE / "SESSION-STATE.md":
                raise ValueError(f"Write outside compacts dir: {resolved}")
    except (OSError, ValueError) as e:
        print(f"SECURITY: {e}")
        sys.exit(1)

# Sensitive patterns — strip from compacts
SENSITIVE = re.compile(
    r"(?:password|passwd|pwd)\s*[:=]\s*\S+|"
    r"(?:api[_-]?key|token|secret|bearer)\s*[:=]\s*\S+|"
    r"clh_[A-Za-z0-9]{30,}|sk-[A-Za-z0-9]{20,}|ghp_[A-Za-z0-9]{30,}|"
    r"-----BEGIN (?:RSA |EC )?PRIVATE KEY-----",
    re.IGNORECASE,
)

# Patterns to redact — paths and URLs replaced with placeholders
REDACT_PATH = re.compile(r"(?:^|[\s(])((?:/[\w.\-]+){2,}|(?:~|\.)/[\w./\-]+)")
REDACT_URL = re.compile(r"https?://[^\s\])>\"]+")
REDACT_INTERNAL = re.compile(r"(?:localhost|127\.0\.0\.1|10\.\d+\.\d+\.\d+|172\.(?:1[6-9]|2\d|3[01])\.\d+\.\d+|192\.168\.\d+\.\d+)[^\s\])>\"]*")

# Patterns for extraction
DECISION_PATTERNS = [
    re.compile(r"(?:chose|decided|going with|picked|selected|settled on)\s+(.{10,100})", re.IGNORECASE),
    re.compile(r"(?:决定|选择|用|采用)\s*(.{5,80})", re.IGNORECASE),
]

FACT_PATTERNS = {
    "PROJ": re.compile(r"(?:project|repo|website|site|app|项目|网站|仓库)[\s:：]+(.{5,100})", re.IGNORECASE),
    "TECH": re.compile(r"(?:config|database|api|port|server|配置|数据库|端口|服务器)[\s:：]+(.{5,80})", re.IGNORECASE),
}

PENDING_PATTERNS = [
    re.compile(r"(?:TODO|todo|待办|下一步|next step|later|need to)[\s:：]*(.{5,100})", re.IGNORECASE),
    re.compile(r"\[ \]\s*(.{5,100})"),
]

BLOCKER_PATTERNS = [
    re.compile(r"(?:blocked by|need .+ first|waiting for|需要先|等.+才能)\s*(.{5,80})", re.IGNORECASE),
]

NOISE_PATTERNS = [
    re.compile(r"^(ok|好的|嗯|emmm|哈哈|haha|lol|nice|cool|谢谢|thanks)\s*[.。!！]*$", re.IGNORECASE),
    re.compile(r"^.{0,5}$"),  # Too short
]


def is_noise(line: str) -> bool:
    """Check if a line is noise (greetings, filler, etc.)."""
    stripped = line.strip()
    if not stripped:
        return True
    return any(p.match(stripped) for p in NOISE_PATTERNS)


def sanitize(text: str) -> str:
    """Remove sensitive data, redact paths and URLs, strip control characters."""
    # Redact secrets first
    text = SENSITIVE.sub("[REDACTED]", text)
    # Redact internal IPs
    text = REDACT_INTERNAL.sub("<INTERNAL_URL>", text)
    # Redact URLs
    text = REDACT_URL.sub("<REDACTED_URL>", text)
    # Redact file paths (2+ segments)
    text = REDACT_PATH.sub(lambda m: m.group(0).replace(m.group(1), "<REDACTED_PATH>"), text)
    # Strip control chars
    text = re.sub(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]", "", text)
    return text.strip()


def extract_from_text(text: str) -> dict:
    """Extract structured data from conversation text."""
    lines = text.split("\n")
    decisions = []
    facts = {"PROJ": [], "URL": [], "PATH": [], "TECH": []}
    pending = []
    blockers = []
    total_lines = 0
    noise_lines = 0

    for line in lines:
        line = line.strip()
        if not line or line.startswith("#") or line.startswith("```"):
            continue

        total_lines += 1
        if is_noise(line):
            noise_lines += 1
            continue

        # Sanitize
        safe = sanitize(line)
        if not safe or safe == "[REDACTED]":
            continue

        # Extract decisions
        for pat in DECISION_PATTERNS:
            m = pat.search(safe)
            if m:
                decisions.append(sanitize(m.group(0)))
                break

        # Extract facts
        for fact_type, pat in FACT_PATTERNS.items():
            matches = pat.findall(safe)
            for match in matches:
                if isinstance(match, tuple):
                    match = match[0] if match else ""
                if match and len(match) > 3:
                    facts[fact_type].append(match)

        # Extract pending
        for pat in PENDING_PATTERNS:
            m = pat.search(safe)
            if m:
                pending.append(sanitize(m.group(1) if m.lastindex else m.group(0)))
                break

        # Extract blockers
        for pat in BLOCKER_PATTERNS:
            m = pat.search(safe)
            if m:
                blockers.append(sanitize(m.group(1) if m.lastindex else m.group(0)))
                break

    # Deduplicate
    for key in facts:
        facts[key] = list(dict.fromkeys(facts[key]))[:5]  # Cap at 5 per type
    decisions = list(dict.fromkeys(decisions))[:10]
    pending = list(dict.fromkeys(pending))[:10]
    blockers = list(dict.fromkeys(blockers))[:5]

    # Estimate tokens saved
    est_tokens = total_lines * 25  # Rough: 25 tokens per line average
    compact_tokens = (len(decisions) + sum(len(v) for v in facts.values()) +
                      len(pending) + len(blockers)) * 15
    tokens_saved = max(0, est_tokens - compact_tokens)

    return {
        "decisions": decisions,
        "facts": facts,
        "pending": pending,
        "blockers": blockers,
        "total_lines": total_lines,
        "noise_lines": noise_lines,
        "tokens_saved": tokens_saved,
    }


def format_compact(data: dict, summary: str = "") -> str:
    """Format extracted data as a compact markdown document."""
    now = datetime.now(timezone.utc)
    ts = now.strftime("%Y-%m-%d %H:%M UTC")
    filename_ts = now.strftime("%Y-%m-%d-%H%M")

    lines = [
        f"# Session Compact — {ts}",
        f"**Lines processed**: {data['total_lines']} | **Noise filtered**: {data['noise_lines']} | **Est. tokens saved**: ~{data['tokens_saved']:,}",
        "",
    ]

    if data["decisions"]:
        lines.append("## Decisions Made")
        for d in data["decisions"]:
            lines.append(f"- {d}")
        lines.append("")

    all_facts = []
    for fact_type, items in data["facts"].items():
        for item in items:
            tag = f"[{fact_type}]" if fact_type in ("PROJ", "TECH") else ""
            all_facts.append(f"{tag} {item}".strip())

    if all_facts:
        lines.append("## Facts Established")
        for f in all_facts:
            lines.append(f"- {f}")
        lines.append("")

    if data["pending"]:
        lines.append("## Pending Actions")
        for p in data["pending"]:
            lines.append(f"- [ ] {p}")
        lines.append("")

    if data["blockers"]:
        lines.append("## Blockers")
        for b in data["blockers"]:
            lines.append(f"- ⚠️ {b}")
        lines.append("")

    if summary:
        lines.append("## Session Summary")
        lines.append(summary)
        lines.append("")

    return "\n".join(lines)


def save_compact(content: str) -> Path:
    """Save compact to disk. Only to memory/compacts/."""
    COMPACTS_DIR.mkdir(parents=True, exist_ok=True)
    now = datetime.now(timezone.utc)
    filename = f"{now.strftime('%Y-%m-%d-%H%M')}.md"
    filepath = COMPACTS_DIR / filename
    assert_safe_path(filepath)

    # Avoid overwriting existing
    counter = 1
    while filepath.exists():
        filepath = COMPACTS_DIR / f"{now.strftime('%Y-%m-%d-%H%M')}-{counter}.md"
        counter += 1

    filepath.write_text(content, encoding="utf-8")

    # Keep only last 30 compacts
    compacts = sorted(COMPACTS_DIR.glob("*.md"))
    while len(compacts) > 30:
        compacts.pop(0).unlink()

    return filepath


def cmd_write(text: str):
    """Write a compact through the full security pipeline (sanitize + path check + save)."""
    data = extract_from_text(text)
    # Also sanitize the raw text for any missed patterns
    sanitized_text = sanitize(text)
    content = format_compact(data)
    filepath = save_compact(content)
    print(f"Compact saved: {filepath}")
    print(f"  Decisions: {len(data['decisions'])}")
    print(f"  Facts: {sum(len(v) for v in data['facts'].values())}")
    print(f"  Pending: {len(data['pending'])}")


def cmd_write_raw(content: str):
    """Write pre-formatted compact content through security checks only."""
    # Verify no sensitive data in the content
    if SENSITIVE.search(content):
        print("SECURITY: Content contains sensitive patterns. Redact and retry.")
        sys.exit(1)
    if REDACT_URL.search(content):
        print("SECURITY: Content contains URLs. Replace with <REDACTED_URL> and retry.")
        sys.exit(1)
    if REDACT_INTERNAL.search(content):
        print("SECURITY: Content contains internal IPs. Replace with <INTERNAL_URL> and retry.")
        sys.exit(1)
    if REDACT_PATH.search(content):
        print("SECURITY: Content contains file paths. Replace with <REDACTED_PATH> and retry.")
        sys.exit(1)

    filepath = save_compact(content)
    print(f"Compact saved: {filepath}")


def cmd_from_file(filepath: str):
    """Extract compact from a file. Restricted to workspace only."""
    path = Path(filepath)
    if not path.exists():
        print(f"File not found: {filepath}")
        sys.exit(1)
    # Security: only read files inside workspace
    assert_safe_path(path)
    text = path.read_text(encoding="utf-8")
    cmd_extract(text)


def cmd_list():
    """List all compacts."""
    if not COMPACTS_DIR.exists():
        print("No compacts found.")
        return

    compacts = sorted(COMPACTS_DIR.glob("*.md"))
    if not compacts:
        print("No compacts found.")
        return

    for c in compacts:
        size = c.stat().st_size
        mtime = datetime.fromtimestamp(c.stat().st_mtime, tz=timezone.utc).strftime("%Y-%m-%d %H:%M")
        print(f"  {c.name}  ({size:,}B, {mtime} UTC)")
    print(f"Total: {len(compacts)} compacts")


def cmd_latest():
    """Print the latest compact."""
    if not COMPACTS_DIR.exists():
        print("No compacts found.")
        return

    compacts = sorted(COMPACTS_DIR.glob("*.md"))
    if not compacts:
        print("No compacts found.")
        return

    latest = compacts[-1]
    print(latest.read_text(encoding="utf-8"))


def cmd_stats():
    """Show compaction statistics."""
    if not COMPACTS_DIR.exists():
        print("No compacts found.")
        return

    compacts = sorted(COMPACTS_DIR.glob("*.md"))
    if not compacts:
        print("No compacts found.")
        return

    total_size = sum(c.stat().st_size for c in compacts)
    total_tokens_saved = 0

    for c in compacts:
        content = c.read_text(encoding="utf-8")
        m = re.search(r"tokens saved\*\*: ~([\d,]+)", content)
        if m:
            total_tokens_saved += int(m.group(1).replace(",", ""))

    oldest = compacts[0].name
    newest = compacts[-1].name

    print(f"=== Compaction Stats ===")
    print(f"Compacts: {len(compacts)}")
    print(f"Total size: {total_size:,}B")
    print(f"Total est. tokens saved: ~{total_tokens_saved:,}")
    print(f"Date range: {oldest} → {newest}")
    print(f"Avg tokens saved/compact: ~{total_tokens_saved // max(len(compacts), 1):,}")


def main():
    parser = argparse.ArgumentParser(description="Context compactor for OpenClaw sessions")
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument("--extract", action="store_true", help="Extract compact from stdin")
    group.add_argument("--write", action="store_true", help="Write pre-formatted compact from stdin (with security checks)")
    group.add_argument("--from-file", type=str, help="Extract compact from file")
    group.add_argument("--list", action="store_true", help="List all compacts")
    group.add_argument("--latest", action="store_true", help="Print latest compact")
    group.add_argument("--stats", action="store_true", help="Show compaction statistics")

    args = parser.parse_args()

    if args.extract:
        text = sys.stdin.read()
        if not text.strip():
            print("No input provided.")
            sys.exit(1)
        cmd_extract(text)
    elif args.write:
        content = sys.stdin.read()
        if not content.strip():
            print("No input provided.")
            sys.exit(1)
        cmd_write_raw(content)
    elif args.from_file:
        cmd_from_file(args.from_file)
    elif args.list:
        cmd_list()
    elif args.latest:
        cmd_latest()
    elif args.stats:
        cmd_stats()


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)