Skill: halfdeadcat/session-janitor (version 1.6.0)

--- name: janitor description: Automated transcript trimming, LLM memory extraction, and session hygiene for OpenClaw gateways. Keeps transcripts from bloating, extracts structured memories before archiving, and prunes stale sessions. metadata: {"openclaw": {"requires": {"bins": ["python3", "jq"]}}} --- # Session Janitor Automated transcript and session hygiene for OpenClaw gateways. ## What It Does - **Sidecar offloader** — extracts large tool outputs (≥5KB) to `.toolcache/` files, replacing inline content with lightweight stubs. Runs before trim on every turn. - **Trims oversized transcripts** — keeps recent exchanges + synthetic compaction summary, archives the rest - **LLM memory extraction** — uses the gateway's chat completions API to extract structured memories (facts, decisions, lessons) from trimmed content before discarding - **Prunes stale sessions** — removes old subagent/cron session entries from sessions.json - **Archives orphan transcripts** — files with no active session after a grace period - **Cleans old archives** — removes pre-trim, reset, and checkpoint files after retention period - **Cleans orphaned checkpoints** — removes `*.checkpoint.*.jsonl` files left behind by auto-compaction once the parent session no longer has an active transcript ## Setup ```bash bash skills/session-janitor/scripts/setup.sh ``` Setup auto-discovers all gateway installations (`~/.openclaw/`, `~/.openclaw-*/`), generates `config.json`, installs a cron job, and installs the watcher service (systemd on Linux, launchd on macOS). ### macOS Prerequisites Install `fswatch` for the per-turn watcher: ```bash brew install fswatch ``` The cron-based janitor works without it (trimming every 15 min instead of within ~3s of each turn). For architecture details, trimming mechanics, and token math, see `ARCHITECTURE.md`. ## Configuration Edit `config.json` after setup to tune thresholds: | Key | Default | Description | |-----|---------|-------------| | `trimMaxKB` | 250 | Trim transcripts larger than this. Trim always fires when exceeded — there is no minimum pair count. | | `keepPairs` | 10 | Number of recent user/assistant pairs to keep after trim. If fewer pairs exist than this, all are kept (no skip). | | `keepFullPairs` | 2 | Most recent N assistant turns that keep full toolResult bodies (older turns are collapsed to summary lines) | | `minArchivePairs` | 5 | Informational only — no longer blocks trim. Sessions exceeding `trimMaxKB` are always trimmed. | | `trimFullThresholdPct` | 50 | Two-stage aggressive reduction when trimmed output still exceeds this % of `trimMaxKB`: (1) strip all assistant turns (tool args + thinking removed); (2) if still over threshold, drop all toolResult entries entirely. Set to 100 to disable. | | `archiveRetentionDays` | 7 | Delete old archives after this many days (applies to `.reset.*`, `.deleted.*`, `.pre-trim.*`, `.bak-*`, `.purged.*`, `.emergency-*`) | | `keepPreTrimFiles` | 3 | Max `.pre-trim.*` files to keep per session (oldest deleted immediately, regardless of retention period) | | `orphanGraceMinutes` | 30 | Wait before archiving orphan transcripts | | `staleSubagentHours` | 24 | Prune subagent session entries older than this | | `staleCronSessionHours` | 24 | Prune cron session entries older than this | | `llmExtraction.enabled` | true | Use LLM to extract memories from trimmed content | | `llmExtraction.model` | `openclaw` | Model identifier to use for extraction calls | | `llmExtraction.maxInputChars` | 20000 | Max characters of archived content sent to LLM | | `llmExtraction.timeoutSecs` | 60 | LLM API call timeout in seconds | | `llmExtraction.maxMemories` | 15 | Max memories to accept from a single LLM extraction | | `llmExtraction.minArchived` | 3 | Minimum archived messages required before running LLM extraction | | `llmExtraction.maxPerRun` | 1 | Max LLM extractions per cron cycle (cost control) | | `memCli.enabled` | false | Store extracted memories via `mem` CLI (requires mem) | | `cronSchedule` | `*/15 * * * *` | How often to run | | `sidecar.enabled` | true | Offload large toolResult entries to `.toolcache/` files | | `sidecar.minEntryBytes` | 5120 | Minimum toolResult content size (bytes) to trigger offload | | `extractOnTrim.enabled` | false | Fire async LLM memory extraction after each watcher-triggered trim | | `extractOnTrim.minArchivedPairs` | 3 | Minimum archived message pairs required before running extraction | | `extractOnTrim.scene` | `auto` | Memory scene tag to use (`auto` lets the LLM infer from context) | | `extractOnTrim.salience` | 0.5 | Default salience for extracted memories (0.1–1.0) | | `watchdog.enabled` | false | Run hung-session detector after each janitor pass | | `watchdog.staleMinutes` | 5 | Session `updatedAt` age threshold to consider stuck | | `watchdog.alertSlack` | true | Send Slack DM when a stuck session is detected | | `watchdog.slackTarget` | — | Slack user ID or channel to notify | | `watchdog.autoRestart` | false | Auto-trigger safe-restart script on detection (use with caution) | | `watchdog.restartScriptSlack` | `~/bin/safe-slack-restart.sh` | Safe restart script path for Slack gateway | | `watchdog.restartScriptDiscord` | `~/bin/safe-gateway-restart.sh` | Safe restart script path for Discord gateway | ## Gateway Discovery Setup scans for `~/.openclaw/openclaw.json` and `~/.openclaw-*/openclaw.json`. Each discovered gateway gets its own entry in `config.json` with: - Port (from `gateway.port` in config) - Auth token (from `gateway.auth.token`) - Sessions directory path Works with single-gateway installs, multi-gateway (Discord + Slack), or any custom layout. ## LLM Extraction Guardrails - **Lockfile** — only one extraction at a time across all gateways - **Max per run** — configurable cap per cron cycle (default: 1) - **20K char input cap** — truncates middle of very long conversations - **60s hard timeout** — SIGALRM kills hung API calls - **Dedup** — state file tracks processed trims, never re-runs - **No context inflation** — memories go to `mem` DB only, never to files that get auto-loaded into session context ## Manual Run ```bash bash skills/session-janitor/scripts/janitor.sh ``` ## Logs ```bash tail -f /tmp/session-janitor.log ``` ## Watcher Service The watcher fires trim within ~3 seconds of each turn completing (vs. waiting for the next cron cycle). `setup.sh` installs it automatically as a system service. *Linux (systemd):* ```bash systemctl --user status session-janitor-watcher systemctl --user restart session-janitor-watcher tail -f /tmp/session-janitor-watcher.log ``` *macOS (launchd):* ```bash launchctl list | grep session-janitor launchctl unload ~/Library/LaunchAgents/ai.openclaw.session-janitor-watcher.plist launchctl load ~/Library/LaunchAgents/ai.openclaw.session-janitor-watcher.plist tail -f /tmp/session-janitor-watcher.log ``` - *Linux* uses `inotifywait` (from `inotify-tools`) and watches via inotify kernel events. - *macOS* uses `fswatch` (via Homebrew) and watches via FSEvents. ## Watchdog The watchdog runs after every janitor pass and checks `updatedAt` on all active sessions. If any session is stale longer than `watchdog.staleMinutes`, it fires a Slack alert. Optional `autoRestart` will trigger the appropriate safe-restart script. Alert cooldown is 10 minutes per session to prevent spam. State is tracked in the janitor state file. ```bash # Run standalone bash skills/session-janitor/scripts/watchdog.sh 5 ~/.openclaw/session-janitor-state.json ``` ## Sub-Agent Reporting Conventions Workers spawned via Foreman should use prefixed `sessions_send` messages to prevent the parent agent from spawning new sub-agents in response to status updates: ``` # Intermediate status (fire-and-forget — parent relays but does NOT spawn) sessions_send(sessionKey="{PARENT}", message="STATUS: 🤖 [label]: doing X (60%)", timeoutSeconds=0) # Errors/failures (normal delivery — parent may intervene) sessions_send(sessionKey="{PARENT}", message="ERROR: 🤖 [label]: what failed — details") # Completion (no prefix, normal delivery) sessions_send(sessionKey="{PARENT}", message="🤖 [label]: ✅ done — summary") ``` - `STATUS:` → relay-only; `timeoutSeconds=0` prevents reply-back loops - `ERROR:` → parent receives and considers recovery action - No prefix → completion; parent handles and relays to channel ## Files ``` skills/session-janitor/ ├── SKILL.md # This file ├── ARCHITECTURE.md # Deep dive: trimming mechanics, token math, examples ├── config.json # Generated by setup (gitignored) ├── config.example.json # Reference config ├── session-janitor-watcher.service # Linux: systemd user service template ├── session-janitor-watcher.plist # macOS: launchd LaunchAgent template └── scripts/ ├── setup.sh # Discovery + config gen + cron + service install (Linux + macOS) ├── janitor.sh # Main cron entry point ├── trim.py # Transcript trimming (thinking, toolCall args, toolResult collapse) ├── sidecar.py # Large toolResult offloader (→ .toolcache/ files) ├── watcher.sh # Per-turn trigger (sidecar + trim; inotifywait/fswatch) ├── extract-llm.py # LLM memory extraction ├── prune-sessions.py # Stale session pruning ├── watchdog.sh # Hung-session detector + Slack alert ├── test-sidecar.py # Sidecar test suite (8 scenarios, 22 assertions) └── test-sidecar.sh # Shell wrapper for sidecar tests ```

path: ARCHITECTURE.md
content: # Session Janitor — Architecture & Trimming Deep Dive

## Overview

Session Janitor is a transcript hygiene system for OpenClaw gateways. It reduces context token usage by aggressively trimming JSONL session files while preserving enough history for the agent to remain coherent.

Three complementary mechanisms:
- **Sidecar (sidecar.py)** — offloads large tool outputs to `.toolcache/` files on every turn
- **Watcher (watcher.sh)** — `inotifywait` fires sidecar + trim within ~3 seconds of any turn completing
- **Cron (janitor.sh)** — runs every 15 minutes as a sweep across all sessions

---

## How Trimming Works

### What's in a Session File

Each JSONL file (`~/.openclaw*/agents/main/sessions/<uuid>.jsonl`) contains one JSON object per line:

| Type | Description |
|------|-------------|
| `session` | Header — session ID, metadata (one per file, always first) |
| `message` (user) | User message with text content |
| `message` (assistant) | Assistant response with text + tool calls |
| `message` (toolResult) | Output from tool executions |
| `compaction` | Summary stub inserted by trim or gateway compaction |

### What Gets Stripped

**Thinking blocks** (from `assistant` messages, non-recent turns):
```json
{ "type": "thinking", "thinking": "...57KB of internal reasoning...", "thinkingSignature": "..." }
```
→ Dropped entirely. The model doesn't need its old reasoning chain.

**toolCall arguments** (from `assistant` messages, non-recent turns):
```json
{ "type": "toolCall", "id": "abc123", "name": "exec", "arguments": "{\"command\":\"...long script...\"}" }
```
→ Reduced to `{ "type": "toolCall", "id": "abc123", "name": "exec" }`. The name is sufficient for context; the gateway doesn't need to re-read old arguments.

**toolResult bodies** (all but the most recent `keepFullPairs` assistant turns, default 2):
Full tool output entries (often 5-50KB each) are collapsed into a single summary line per turn:
```
exec ✓(0): first line of output | Read ✓(0) | exec ✗(1): error text
```

### What's Preserved

- **Session header** — always kept (required by gateway)
- **Last N user/assistant pairs** (default: 10, configurable via `keepPairs`) — kept verbatim. Trim fires unconditionally when over threshold — if fewer than `keepPairs` exist, all are kept.
- **Recent 2 assistant turns** — full toolResult entries preserved (configurable via `keepFullPairs`)
- **Synthetic compaction entry** — describes what was archived and why

### Two-Stage Aggressive Reduction

If the trimmed transcript still exceeds `trimFullThresholdPct`% of `trimMaxKB` (default: 50%), a second pass fires:

**Stage 1 — Strip all assistant turns:** thinking blocks and toolCall arguments dropped from every assistant entry (not just older ones).

**Stage 2 — Drop all toolResult entries:** if still over threshold after Stage 1, every `toolResult` entry is removed entirely. Only the session header, compaction stub, and raw user/assistant message text remain.

This handles sessions with very few user messages but massive tool output accumulation (e.g. stuck-loop sessions that generate hundreds of KB of tool results with only 3-4 user turns).

### Size Reduction Examples

| Session | Before | After | Reduction |
|---------|--------|-------|-----------|
| Heavy tool use + extended thinking | 401KB | 91KB | 77% |
| Normal conversation | 266KB | 100KB | 62% |
| Moderate tool use | 316KB | 102KB | 68% |
| Post-trim steady state | 157KB | 91KB | 42% |

### Token Math

Raw JSONL → tokens is roughly 4 chars/token, but JSON overhead makes it worse.

| Component | Approx tokens |
|-----------|--------------|
| System prompt + tool schemas | ~20–25k |
| Injected workspace files (SOUL, MEMORY, AGENTS, etc.) | ~5–8k |
| Total overhead before transcript | ~28–33k |
| Remaining budget (80k limit) | ~47–52k |
| 150KB trimmed transcript | ~12–15k |
| **Total at steady state** | **~40–45k / 80k** |

Before trimming, sessions routinely exceeded 80k tokens, triggering gateway compaction. After trimming, steady-state context stays around 40–45k.

---

## Sidecar Offloader

`sidecar.py` runs on every watcher-triggered write, *before* trim. It targets `toolResult` entries that exceed a size threshold (default 5KB).

### How It Works

1. Scan the JSONL for `toolResult` entries ≥ `sidecar.minEntryBytes`
2. Write the full content to `<session-id>.toolcache/<tool-call-id>.txt` (adjacent to the transcript)
3. Replace the inline content with a stub:
   ```
   [tool output offloaded to .toolcache/<tool-call-id>.txt — <N> bytes. Use the Read tool to access it if needed.]
   ```
4. If the file shrinks below the trim threshold after sidecar, skip trim entirely (and ping the gateway to reload)

### Properties

- **Idempotent** — already-stubbed entries are skipped on subsequent runs
- **Restart-safe** — `.toolcache/` files live alongside the transcript; no temp files or external state
- **Selective** — only `toolResult` entries are offloaded; user/assistant messages and thinking blocks are untouched
- **Agent-friendly** — stubs tell the agent where to find the original content if needed

### Configuration

| Key | Default | Description |
|-----|---------|-------------|
| `sidecar.enabled` | true | Enable/disable the sidecar offloader |
| `sidecar.minEntryBytes` | 5120 | Minimum toolResult content size to trigger offload |

---

## The Three Triggers

### Cron (every 15 min)

`janitor.sh` sweeps all active sessions across all gateways. Also runs:
- LLM memory extraction (extracts structured facts/decisions from archived content)
- Stale session pruning (removes orphan subagent entries)
- Archive cleanup (deletes `.pre-trim.*` files older than 7 days)

### Watcher (per-turn, ~3s latency)

`watcher.sh` uses an OS-appropriate file watcher:
- *Linux:* `inotifywait` in monitor mode (`inotify-tools` package)
- *macOS:* `fswatch` with FSEvents backend (`brew install fswatch`)

Both output one absolute path per line on write events. The rest of the flow is identical.

Flow per JSONL write event:
1. JSONL file closes after a turn completes
2. inotifywait/fswatch emits the event
3. 3-second debounce (waits for follow-on writes to finish)
4. **Run sidecar offloader** — extracts large toolResult bodies to `.toolcache/` files
5. Recheck file size — if sidecar alone brought it below threshold, skip trim and ping gateway
6. If still over threshold, verify it's an active session (check sessions.json)
7. Run `trim.py` in background
8. Ping gateway via Chat Completions API with `x-openclaw-session-key` to force in-memory reload

The reload ping is critical — without it the gateway keeps its old in-memory context until the next session load.

---

## Gateway Reload Mechanism

After trim, the watcher sends a no-op message to the gateway:

```bash
curl http://127.0.0.1:<port>/v1/chat/completions \
  -H "Authorization: Bearer <token>" \
  -H "x-openclaw-session-key: <session-key>" \
  -d '{"model":"openclaw","messages":[{"role":"user","content":"[session trimmed by maintenance — acknowledge with NO_REPLY]"}]}'
```

The `x-openclaw-session-key` header routes the request to the specific session. The gateway re-reads the JSONL file from disk before processing, so the trimmed context becomes active immediately.

The agent acknowledges with `NO_REPLY` (a special token instructing it not to send a visible reply).

---

## File Layout

```
skills/session-janitor/
├── SKILL.md                          # Agent instructions
├── ARCHITECTURE.md                   # This file
├── config.json                       # Generated by setup (gitignored)
├── config.example.json               # Reference config
├── session-janitor-watcher.service   # Systemd user service template
└── scripts/
    ├── setup.sh           # Gateway discovery, config gen, cron + service install
    ├── janitor.sh         # Cron entry point (trim + extract + prune)
    ├── trim.py            # Core transcript trimming logic
    ├── sidecar.py         # Large toolResult offloader (→ .toolcache/ files)
    ├── extract-llm.py     # LLM memory extraction from archived content
    ├── prune-sessions.py  # Stale subagent/cron session pruning
    ├── watcher.sh         # inotifywait per-turn trigger (sidecar + trim)
    ├── test-sidecar.py    # Sidecar test suite (8 scenarios, 22 assertions)
    └── test-sidecar.sh    # Shell wrapper for sidecar tests
```

### Runtime File Layout

For each session with offloaded tool outputs:
```
~/.openclaw-*/agents/main/sessions/
├── <session-id>.jsonl                # Transcript (stubs reference .toolcache/)
├── <session-id>.toolcache/            # Offloaded tool outputs
│   ├── <tool-call-id-1>.txt
│   ├── <tool-call-id-2>.txt
│   └── ...
├── <session-id>.jsonl.pre-trim.<ts>   # Pre-trim archive (if trimmed)
└── ...
```

---

## Operational Notes

- `trim.py` archives the original as `<uuid>.jsonl.pre-trim.<timestamp>` before writing
- Archives are cleaned up after 7 days (configurable via `archiveRetentionDays`)
- The watcher service restarts automatically on failure (`Restart=on-failure`)
- Watcher log: `/tmp/session-janitor-watcher.log`
- Cron log: `/tmp/session-janitor.log`
- State (dedup tracking): `~/.openclaw/session-janitor-state.json`


path: README.md
content: # session-janitor

Automated transcript and session hygiene for [OpenClaw](https://github.com/openclaw/openclaw) gateways.

## What It Does

- **Trims oversized transcripts** — keeps recent exchanges + synthetic compaction summary, archives the rest
- **LLM memory extraction** — uses the gateway's chat completions API to extract structured memories from trimmed content
- **Prunes stale sessions** — removes old subagent/cron session entries from `sessions.json`
- **Archives orphan transcripts** — cleans up files with no active session
- **Sidecar offloader** — extracts large tool outputs (≥5KB) to `.toolcache/` files, replacing them with lightweight stubs
- **Multi-gateway** — auto-discovers all gateway installations (`~/.openclaw/`, `~/.openclaw-*/`)

## Install

```bash
git clone https://github.com/sam-ueckert/session-janitor
cd session-janitor
bash scripts/setup.sh
```

Setup auto-discovers your gateway(s), generates `config.json`, and installs a cron job.

## Configuration

Edit `config.json` after setup:

| Key | Default | Description |
|-----|---------|-------------|
| `trimMaxKB` | 250 | Trim transcripts larger than this |
| `keepPairs` | 10 | Recent user/assistant pairs to keep after trim |
| `archiveRetentionDays` | 7 | Delete old archives after N days |
| `orphanGraceMinutes` | 30 | Wait before archiving orphan transcripts |
| `staleSubagentHours` | 24 | Prune stuck subagent entries older than this |
| `sidecar.enabled` | true | Offload large toolResult entries to `.toolcache/` files |
| `sidecar.minEntryBytes` | 5120 | Minimum toolResult size (bytes) to trigger offload |
| `llmExtraction.enabled` | true | Extract memories from trimmed content via LLM |
| `llmExtraction.maxPerRun` | 1 | Max LLM extractions per cron cycle |
| `memCli.enabled` | false | Store memories via `mem` CLI (requires mem) |
| `cronSchedule` | `*/15 * * * *` | How often to run |

## Manual Run

```bash
bash scripts/janitor.sh
```

## Logs

```bash
tail -f /tmp/session-janitor.log
```

## Requirements

- OpenClaw gateway installed (`~/.openclaw/openclaw.json`)
- Python 3
- `mem` CLI optional (for structured memory storage)

## License

MIT


path: config.example.json
content: {
  "gateways": [
    {
      "name": "default",
      "configDir": "~/.openclaw",
      "sessionsDir": "~/.openclaw/agents/main/sessions",
      "port": 18789,
      "token": ""
    }
  ],
  // Trim a session transcript when it exceeds this size (KB)
  "trimMaxKB": 250,

  // Number of recent user/assistant exchange pairs to retain after trimming
  "keepPairs": 10,

  // Most recent N assistant turns that keep full toolResult bodies.
  // Older turns have tool args stripped and thinking blocks removed to save space.
  "keepFullPairs": 2,

  // Informational only — no longer blocks trim. Sessions exceeding trimMaxKB are always trimmed.
  "minArchivePairs": 5,

  // Two-stage aggressive reduction when trimmed transcript still exceeds this % of trimMaxKB:
  //   Stage 1: strip all assistant turns (tool args + thinking removed)
  //   Stage 2: if still over threshold, drop ALL toolResult entries entirely
  // Handles sessions with massive tool output but few user turns (e.g. stuck-loop sessions).
  // Set to 100 to disable.
  "trimFullThresholdPct": 50,
  "archiveRetentionDays": 7,
  "orphanGraceMinutes": 30,
  "staleSubagentHours": 24,
  "staleCronSessionHours": 24,
  "llmExtraction": {
    "enabled": true,
    "maxPerRun": 1,
    "model": "openclaw",
    "maxInputChars": 20000,
    "timeoutSecs": 60,
    "maxMemories": 15,
    "minArchived": 3
  },
  // Seconds to wait after a file-change event before firing a trim (watcher mode only).
  // Debounce prevents redundant trims during rapid sequential writes.
  "watcherDebounceSecs": 3,
  "memCli": {
    "enabled": false,
    "path": "mem"
  },
  "stateFile": "~/.openclaw/session-janitor-state.json",
  "cronSchedule": "*/15 * * * *",
  "logFile": "/tmp/session-janitor.log",
  // Watchdog: detects sessions stuck longer than staleMinutes and optionally restarts the gateway.
  // alertSlack: send a Slack DM when a stuck session is detected.
  // slackTarget: Slack user ID or channel to notify.
  // autoRestart: if true, automatically trigger the safe restart script (USE WITH CAUTION).
  // restartScriptSlack/Discord: path to safe-restart scripts (defaults shown).
  "watchdog": {
    "enabled": false,
    "staleMinutes": 5,
    "alertSlack": true,
    "slackTarget": "U0AM4BLBUUW",
    "autoRestart": false,
    "restartScriptSlack": "~/bin/safe-slack-restart.sh",
    "restartScriptDiscord": "~/bin/safe-gateway-restart.sh"
  }
}


path: scripts/extract-llm.py
content: #!/usr/bin/env python3
"""LLM-powered extraction of archived transcript content into structured memory.

Usage: extract-llm.py <pre_trim> <trimmed> <sid> <gateway> <state_file> <api_url> <api_token> <mem_enabled> <mem_path>

Guardrails: lockfile, 20K char cap, 60s timeout, dedup via state file.
Exit 0 = success/skip, 2 = LLM failure (caller should fall back).
"""
import json, sys, os, subprocess, fcntl, signal
from datetime import datetime

# Defaults — overridden by config values passed via argv
DEFAULT_MAX_INPUT_CHARS = 20000
DEFAULT_API_TIMEOUT_SECS = 60
DEFAULT_MAX_MEMORIES = 15
DEFAULT_MIN_ARCHIVED = 3
LOCKFILE = "/tmp/session-janitor-extract.lock"
DEFAULT_MODEL = "openclaw"

EXTRACTION_PROMPT = """You are a memory extraction system. Analyze this conversation transcript and extract structured memories.

For each memory, output a JSON object on its own line with these fields:
- "type": one of "fact", "decision", "preference", "task", "risk", "plan", "lesson"
- "salience": 0.1 (trivia) to 1.0 (critical)
- "scene": short label for the topic area (e.g. "infrastructure", "k8s", "personal", "workflow")
- "content": concise statement of the memory (1-2 sentences max)

Rules:
- Extract 3-15 memories. Quality over quantity.
- Focus on: decisions made, facts learned, preferences expressed, tasks assigned, lessons learned, risks identified
- Skip: routine tool output, heartbeat checks, pleasantries, obvious/temporary context
- Each memory should stand alone — readable without the original conversation
- Use present tense for facts/preferences, past tense for events/decisions

Output ONLY the JSON lines, nothing else.

TRANSCRIPT:
"""

def timeout_handler(signum, frame):
    raise TimeoutError("API call exceeded timeout")

def extract_archived_content(pre_trim_file, trimmed_file):
    pre_entries = []
    with open(pre_trim_file) as f:
        for line in f:
            line = line.strip()
            if not line: continue
            try: pre_entries.append(json.loads(line))
            except: continue

    kept_ids = set()
    with open(trimmed_file) as f:
        for line in f:
            line = line.strip()
            if not line: continue
            try:
                e = json.loads(line)
                if e.get("id"): kept_ids.add(e["id"])
            except: continue

    archived = []
    for e in pre_entries:
        if e.get("type") != "message": continue
        if e.get("id") in kept_ids: continue
        role = e.get("message", {}).get("role", "")
        if role not in ("user", "assistant"): continue
        content = e.get("message", {}).get("content", "")
        if isinstance(content, list):
            content = " ".join(c.get("text", "") for c in content if c.get("type") == "text")
        content = content.strip()
        if not content or content in ("NO_REPLY", "HEARTBEAT_OK"): continue
        if content.startswith("Read HEARTBEAT"): continue
        archived.append({"role": role, "content": content})
    return archived

def format_for_llm(archived_messages, max_input_chars=DEFAULT_MAX_INPUT_CHARS):
    lines = []
    for msg in archived_messages:
        role = "User" if msg["role"] == "user" else "Assistant"
        lines.append(f"[{role}]: {msg['content']}\n")
    text = "".join(lines)
    if len(text) > max_input_chars:
        head = int(max_input_chars * 0.3)
        tail = max_input_chars - head - 50
        text = text[:head] + "\n[... middle truncated ...]\n" + text[-tail:]
    return text

def call_llm(api_url, api_token, transcript_text, model=DEFAULT_MODEL, api_timeout_secs=DEFAULT_API_TIMEOUT_SECS):
    import urllib.request
    payload = {
        "model": model, "max_tokens": 2000, "temperature": 0.3,
        "messages": [{"role": "user", "content": EXTRACTION_PROMPT + transcript_text}]
    }
    req = urllib.request.Request(
        f"{api_url}/v1/chat/completions",
        data=json.dumps(payload).encode(),
        headers={"Authorization": f"Bearer {api_token}", "Content-Type": "application/json"}
    )
    signal.signal(signal.SIGALRM, timeout_handler)
    signal.alarm(api_timeout_secs)
    try:
        with urllib.request.urlopen(req, timeout=api_timeout_secs) as resp:
            result = json.loads(resp.read())
        signal.alarm(0)
        return result["choices"][0]["message"]["content"]
    except Exception as e:
        signal.alarm(0)
        raise RuntimeError(f"LLM API call failed: {e}")

def parse_memories(llm_output, max_memories=DEFAULT_MAX_MEMORIES):
    memories = []
    for line in llm_output.strip().split("\n"):
        line = line.strip()
        if not line or not line.startswith("{"): continue
        try:
            mem = json.loads(line)
            if not all(k in mem for k in ("type", "salience", "scene", "content")): continue
            if mem["type"] not in ("fact", "decision", "preference", "task", "risk", "plan", "lesson"): continue
            if not (0.0 <= float(mem["salience"]) <= 1.0): continue
            if len(mem["content"]) < 5 or len(mem["content"]) > 500: continue
            memories.append(mem)
        except: continue
    return memories[:max_memories]

# Scene label → file mapping for scene file writes
SCENE_FILE_MAP = {
    "infra": "scene-infrastructure.md",
    "infrastructure": "scene-infrastructure.md",
    "k8s": "scene-infrastructure.md",
    "kubernetes": "scene-infrastructure.md",
    "networking": "scene-infrastructure.md",
    "network": "scene-infrastructure.md",
    "config": "scene-config.md",
    "gateway": "scene-config.md",
    "openclaw": "scene-config.md",
    "configuration": "scene-config.md",
    "projects": "scene-projects.md",
    "project": "scene-projects.md",
    "cyrano": "scene-projects.md",
    "groomer": "scene-projects.md",
    "archy": "scene-projects.md",
    "lessons": "scene-lessons.md",
    "lesson": "scene-lessons.md",
    "commodore": "scene-commodore.md",
    "personal": "scene-commodore.md",
    "preferences": "scene-commodore.md",
    "accounts": "scene-accounts.md",
    "account": "scene-accounts.md",
    "auth": "scene-accounts.md",
}
SCENE_FILE_DEFAULT = "scene-infrastructure.md"

def write_to_scene_files(memories, scene_dir):
    """Append extracted memories to git-backed scene files for durable recovery."""
    if not scene_dir or not os.path.isdir(scene_dir):
        return 0
    from datetime import date
    today = date.today().isoformat()
    written = 0
    for mem in memories:
        scene_label = mem.get("scene", "").lower()
        filename = SCENE_FILE_MAP.get(scene_label, SCENE_FILE_DEFAULT)
        filepath = os.path.join(scene_dir, filename)
        sal = mem.get("salience", 0.5)
        mtype = mem.get("type", "fact")
        content = mem.get("content", "").replace('\n', ' ').strip()
        if not content:
            continue
        line = f"- [{mtype}] (sal: {sal:.1f}) {content}  # janitor {today}\n"
        try:
            with open(filepath, "a") as f:
                f.write(line)
            written += 1
        except Exception as e:
            print(f"scene file write error ({filepath}): {e}", file=sys.stderr)
    return written

def git_commit_scene_files(scene_dir):
    """Commit and push scene file updates to git repo."""
    repo_dir = os.path.dirname(os.path.dirname(scene_dir))  # memory/private/ -> repo root
    try:
        # Check if anything changed
        result = subprocess.run(
            ["git", "-C", repo_dir, "status", "--porcelain", "memory/private/scene-*.md"],
            capture_output=True, text=True, timeout=10
        )
        # Use glob since git status porcelain doesn't expand globs
        import glob
        scene_files = glob.glob(os.path.join(scene_dir, "scene-*.md"))
        changed = []
        for sf in scene_files:
            rel = os.path.relpath(sf, repo_dir)
            r2 = subprocess.run(
                ["git", "-C", repo_dir, "diff", "--quiet", rel],
                capture_output=True, timeout=5
            )
            if r2.returncode != 0:
                changed.append(rel)
        if not changed:
            print("No scene file changes to commit")
            return
        for rel in changed:
            subprocess.run(["git", "-C", repo_dir, "add", rel], timeout=5, check=True)
        from datetime import date
        msg = f"janitor: scene file update {date.today().isoformat()}"
        subprocess.run(["git", "-C", repo_dir, "commit", "-m", msg], timeout=10, check=True)
        subprocess.run(["git", "-C", repo_dir, "push"], timeout=30, check=True)
        print(f"Committed and pushed {len(changed)} scene file(s)")
    except subprocess.CalledProcessError as e:
        print(f"git commit/push failed (non-fatal): {e}", file=sys.stderr)
    except Exception as e:
        print(f"git error (non-fatal): {e}", file=sys.stderr)

def store_memories(memories, mem_enabled, mem_path, scene_dir=None):
    stored = 0
    if mem_enabled:
        for mem in memories:
            try:
                result = subprocess.run(
                    [mem_path, "quick-store", mem["scene"], mem["type"], str(mem["salience"]), mem["content"]],
                    capture_output=True, text=True, timeout=10
                )
                if result.returncode == 0: stored += 1
                else: print(f"mem store failed: {result.stderr.strip()}", file=sys.stderr)
            except Exception as e:
                print(f"mem store error: {e}", file=sys.stderr)
    # Write to scene files (always, if path provided — DB is derived, files are durable)
    scene_written = 0
    if scene_dir:
        scene_written = write_to_scene_files(memories, scene_dir)
        if scene_written > 0:
            git_commit_scene_files(scene_dir)
            print(f"Wrote {scene_written} memories to scene files")
    return stored

def main():
    if len(sys.argv) < 8:
        print("Usage: extract-llm.py <pre_trim> <trimmed> <sid> <gateway> <state> <api_url> <api_token> [mem_enabled] [mem_path]", file=sys.stderr)
        sys.exit(1)

    pre_trim_file, trimmed_file, session_id, gateway = sys.argv[1:5]
    state_file, api_url, api_token = sys.argv[5:8]
    mem_enabled = sys.argv[8].lower() == "true" if len(sys.argv) > 8 else False
    mem_path = sys.argv[9] if len(sys.argv) > 9 else "mem"
    scene_dir = sys.argv[10] if len(sys.argv) > 10 else None
    model = sys.argv[11] if len(sys.argv) > 11 else DEFAULT_MODEL
    max_input_chars = int(sys.argv[12]) if len(sys.argv) > 12 else DEFAULT_MAX_INPUT_CHARS
    api_timeout_secs = int(sys.argv[13]) if len(sys.argv) > 13 else DEFAULT_API_TIMEOUT_SECS
    max_memories = int(sys.argv[14]) if len(sys.argv) > 14 else DEFAULT_MAX_MEMORIES
    min_archived = int(sys.argv[15]) if len(sys.argv) > 15 else DEFAULT_MIN_ARCHIVED

    # Lockfile
    lock_fd = None
    try:
        lock_fd = open(LOCKFILE, "w")
        fcntl.flock(lock_fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
    except (IOError, OSError):
        print("Another LLM extraction is running — skipping")
        sys.exit(0)

    try:
        state = {}
        if os.path.exists(state_file):
            try: state = json.load(open(state_file))
            except: pass

        llm_key = f"llm-{session_id}-{os.path.basename(pre_trim_file)}"
        if llm_key in state.get("llm_extracted", {}):
            print(f"Already extracted {llm_key} — skipping")
            sys.exit(0)

        if not os.path.exists(pre_trim_file):
            print(f"Pre-trim file not found: {pre_trim_file}", file=sys.stderr)
            sys.exit(1)

        archived = extract_archived_content(pre_trim_file, trimmed_file)
        if len(archived) < min_archived:
            print(f"Only {len(archived)} archived messages (need {min_archived}) — not enough")
            sys.exit(0)

        transcript_text = format_for_llm(archived, max_input_chars)
        print(f"Sending {len(transcript_text)} chars ({len(archived)} messages) to LLM...")

        try:
            llm_output = call_llm(api_url, api_token, transcript_text, model, api_timeout_secs)
        except Exception as e:
            print(f"LLM extraction failed: {e}", file=sys.stderr)
            sys.exit(2)

        memories = parse_memories(llm_output, max_memories)
        if not memories:
            print("LLM returned no valid memories")
            sys.exit(0)

        stored = store_memories(memories, mem_enabled, mem_path, scene_dir)
        print(f"Extracted {len(memories)} memories, stored {stored} to mem DB")

        # Update dedup state
        llm_extracted = state.get("llm_extracted", {})
        llm_extracted[llm_key] = {
            "timestamp": datetime.now().isoformat(),
            "memories": len(memories),
            "stored": stored,
            "input_chars": len(transcript_text),
        }
        if len(llm_extracted) > 100:
            for k in sorted(llm_extracted, key=lambda k: llm_extracted[k].get("timestamp", ""))[:50]:
                del llm_extracted[k]
        state["llm_extracted"] = llm_extracted
        state["lastLlmRun"] = datetime.now().isoformat()
        json.dump(state, open(state_file, "w"), indent=2)
    finally:
        if lock_fd:
            fcntl.flock(lock_fd, fcntl.LOCK_UN)
            lock_fd.close()
            try: os.unlink(LOCKFILE)
            except: pass

if __name__ == "__main__":
    main()


path: scripts/janitor.sh
content: #!/usr/bin/env bash
# session-janitor — config-driven transcript and session hygiene
# Reads config.json for gateway locations, thresholds, and features.
set -euo pipefail

export PATH="/home/${USER:-$(whoami)}/.npm-global/bin:/usr/local/bin:/usr/bin:/bin:$PATH"
export XDG_RUNTIME_DIR="/run/user/$(id -u)"
export DBUS_SESSION_BUS_ADDRESS="unix:path=${XDG_RUNTIME_DIR}/bus"

SKILL_DIR="$(cd "$(dirname "$0")/.." && pwd)"
SCRIPTS_DIR="$SKILL_DIR/scripts"
CONFIG_FILE="$SKILL_DIR/config.json"

if [[ ! -f "$CONFIG_FILE" ]]; then
    echo "ERROR: $CONFIG_FILE not found — run setup.sh first"
    exit 1
fi

# --- Load config ---
read_config() {
    python3 -c "import json,sys; c=json.load(open('$CONFIG_FILE')); exec(sys.stdin.read())" <<PYEOF
import os
vals = {
    'TRIM_MAX_KB': c.get('trimMaxKB', 250),
    'KEEP_PAIRS': c.get('keepPairs', 10),
    'KEEP_FULL_PAIRS': c.get('keepFullPairs', 2),
    'MIN_ARCHIVE_PAIRS': c.get('minArchivePairs', 5),
    'TRIM_FULL_THRESHOLD_PCT': c.get('trimFullThresholdPct', 50),
    'ARCHIVE_RETENTION_DAYS': c.get('archiveRetentionDays', 7),
    'KEEP_PRE_TRIM_FILES': c.get('keepPreTrimFiles', 3),
    'ORPHAN_GRACE_MINUTES': c.get('orphanGraceMinutes', 30),
    'STALE_SUBAGENT_HOURS': c.get('staleSubagentHours', 24),
    'STALE_CRON_SESSION_HOURS': c.get('staleCronSessionHours', 24),
    'LLM_ENABLED': str(c.get('llmExtraction',{}).get('enabled', False)).lower(),
    'LLM_MAX_PER_RUN': c.get('llmExtraction',{}).get('maxPerRun', 1),
    'LLM_GATEWAY': c.get('llmExtraction',{}).get('gateway', ''),
    'LLM_MODEL': c.get('llmExtraction',{}).get('model', 'openclaw'),
    'LLM_MAX_INPUT_CHARS': c.get('llmExtraction',{}).get('maxInputChars', 20000),
    'LLM_TIMEOUT_SECS': c.get('llmExtraction',{}).get('timeoutSecs', 60),
    'LLM_MAX_MEMORIES': c.get('llmExtraction',{}).get('maxMemories', 15),
    'LLM_MIN_ARCHIVED': c.get('llmExtraction',{}).get('minArchived', 3),
    'MEM_ENABLED': str(c.get('memCli',{}).get('enabled', False)).lower(),
    'MEM_PATH': c.get('memCli',{}).get('path', 'mem'),
    'SCENE_FILES_PATH': os.path.expanduser(c.get('sceneFilesPath', '')),
    'STATE_FILE': os.path.expanduser(c.get('stateFile', '~/.openclaw/session-janitor-state.json')),
    'GATEWAY_COUNT': len(c.get('gateways', [])),
}
for k, v in vals.items():
    print(f'{k}={v}')
PYEOF
}

eval "$(read_config)"

# Gateway details
get_gateway() {
    local idx="$1" field="$2"
    python3 -c "
import json, os
c = json.load(open('$CONFIG_FILE'))
gw = c['gateways'][$idx]
v = gw.get('$field', '')
if isinstance(v, str) and '~' in v:
    v = os.path.expanduser(v)
print(v)
"
}

LLM_EXTRACTIONS_THIS_RUN=0

# Resolve LLM gateway port/token (prefer llmExtraction.gateway name, fall back to first gateway)
get_llm_gateway_field() {
    local field="$1"
    python3 -c "
import json, os
c = json.load(open('$CONFIG_FILE'))
gws = c.get('gateways', [])
pref = c.get('llmExtraction', {}).get('gateway', '')
gw = next((g for g in gws if g.get('name') == pref), gws[0] if gws else {})
v = gw.get('$field', '')
if isinstance(v, str) and '~' in v: v = os.path.expanduser(v)
print(v)
"
}

LLM_PORT=$(get_llm_gateway_field port)
LLM_TOKEN=$(get_llm_gateway_field token)

log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*"; }

# --- Process one gateway ---
process_gateway() {
    local idx="$1"
    local name port token sessions_dir sessions_json
    name=$(get_gateway "$idx" "name")
    port=$(get_gateway "$idx" "port")
    [REDACTED_SECRET] "$idx" "token")
    sessions_dir=$(get_gateway "$idx" "sessionsDir")
    sessions_json="$sessions_dir/sessions.json"

    if [[ ! -f "$sessions_json" ]]; then
        log "$name: no sessions.json — skipping"
        return
    fi

    # Get active session IDs
    local active_sids
    active_sids=$(python3 -c "
import json
d = json.load(open('$sessions_json'))
for v in d.get('sessions', d).values():
    sid = v.get('sessionId', '')
    if sid: print(sid)
" 2>/dev/null)

    local orphan_count=0 archive_rm_count=0 reset_count=0

    shopt -s nullglob
    for jsonl in "$sessions_dir"/*.jsonl; do
        [[ "$jsonl" == *".reset."* ]] && continue
        [[ "$jsonl" == *".deleted."* ]] && continue
        [[ "$jsonl" == *".pre-trim."* ]] && continue
        [[ "$jsonl" == *".trajectory."* ]] && continue  # OC runtime telemetry, not a transcript

        local sid size_kb
        sid=$(basename "$jsonl" .jsonl)
        size_kb=$(( $(stat -c%s "$jsonl" 2>/dev/null || echo 0) / 1024 ))

        if echo "$active_sids" | grep -qF "$sid"; then
            # ACTIVE session — always clear modelOverride + compactionCheckpoints regardless of size
            python3 -c "
import json
path = '$sessions_json'
sid = '$sid'
d = json.load(open(path))
sessions = d.get('sessions', d)
changed = False
for k, v in sessions.items():
    if v.get('sessionId') == sid:
        if 'compactionCheckpoints' in v:
            del v['compactionCheckpoints']
            changed = True
        if 'modelOverride' in v:
            del v['modelOverride']
            changed = True
if changed:
    json.dump(d, open(path, 'w'), indent=2)
    print('cleared')
" 2>/dev/null | grep -q cleared && log "$name: cleared modelOverride/compactionCheckpoints for $sid" || true

            # Trim if oversized
            if (( size_kb > TRIM_MAX_KB )); then
                log "$name: active transcript $sid is ${size_kb}KB — trimming to last $KEEP_PAIRS pairs"
                if python3 "$SCRIPTS_DIR/trim.py" "$jsonl" "$sid" "$name" "$STATE_FILE" "$KEEP_PAIRS" "$KEEP_FULL_PAIRS" "$MIN_ARCHIVE_PAIRS" "$TRIM_FULL_THRESHOLD_PCT" "$TRIM_MAX_KB" 2>&1; then
                    reset_count=$((reset_count + 1))

                    # LLM extraction of archived content
                    if [[ "$LLM_ENABLED" == "true" ]] && (( LLM_EXTRACTIONS_THIS_RUN < LLM_MAX_PER_RUN )); then
                        local pre_trim_file
                        pre_trim_file=$(ls -t "${jsonl}.pre-trim."* 2>/dev/null | head -1)
                        if [[ -n "$pre_trim_file" ]]; then
                            local llm_api_url="http://127.0.0.1:${LLM_PORT}"
                            if python3 "$SCRIPTS_DIR/extract-llm.py" \
                                "$pre_trim_file" "$jsonl" "$sid" "$name" "$STATE_FILE" \
                                "$llm_api_url" "$LLM_TOKEN" "$MEM_ENABLED" "$MEM_PATH" \
                                "$SCENE_FILES_PATH" \
                                "$LLM_MODEL" "$LLM_MAX_INPUT_CHARS" "$LLM_TIMEOUT_SECS" \
                                "$LLM_MAX_MEMORIES" "$LLM_MIN_ARCHIVED" 2>&1; then
                                log "$name: LLM extraction complete for $sid"
                                LLM_EXTRACTIONS_THIS_RUN=$((LLM_EXTRACTIONS_THIS_RUN + 1))
                            else
                                local exit_code=$?
                                if [[ $exit_code -eq 2 ]]; then
                                    log "$name: LLM extraction failed for $sid"
                                else
                                    log "$name: LLM extraction skipped for $sid (dedup/lock/insufficient)"
                                fi
                            fi
                        fi
                    fi

                    # Reset the session JSONL so the gateway starts fresh on next message.
                    # Avoids broken-transcript state caused by curl-triggered LLM tool use
                    # getting interrupted mid-execution. LLM extraction above already archived
                    # the important context into mem.
                    local reset_ts
                    reset_ts=$(date -u +%Y-%m-%dT%H-%M-%SZ)
                    mv "$jsonl" "${jsonl}.reset.${reset_ts}" 2>/dev/null || true

                    # Remove the session entry from sessions.json so the gateway doesn't
                    # hold a dangling pointer to the renamed file.
                    python3 -c "
import json, sys
path = '$sessions_json'
sid = '$sid'
d = json.load(open(path))
sessions = d.get('sessions', d)
to_del = [k for k, v in sessions.items() if v.get('sessionId') == sid]
for k in to_del: del sessions[k]
if to_del: json.dump(d, open(path, 'w'), indent=2)
" 2>/dev/null || true
                    log "$name: session $sid reset after trim (clean slate for next message)"
                else
                    log "$name: trim failed for $sid"
                fi
            fi
        else
            # ORPHAN — grace period then archive
            local mod_min
            mod_min=$(( ( $(date +%s) - $(stat -c%Y "$jsonl" 2>/dev/null || echo 0) ) / 60 ))
            if (( mod_min < ORPHAN_GRACE_MINUTES )); then
                continue
            fi
            mv "$jsonl" "${jsonl}.reset.$(date -u +%Y-%m-%dT%H-%M-%SZ)"
            orphan_count=$((orphan_count + 1))
        fi
    done
    shopt -u nullglob

    # --- Prune compaction checkpoints from all active sessions ---
    local checkpoint_pruned
    checkpoint_pruned=$(python3 -c "
import json
path = '$sessions_json'
d = json.load(open(path))
sessions = d.get('sessions', d)
count = 0
for k, v in sessions.items():
    if 'compactionCheckpoints' in v:
        del v['compactionCheckpoints']
        count += 1
if count:
    json.dump(d, open(path, 'w'), indent=2)
print(count)
" 2>/dev/null || echo 0)
    (( checkpoint_pruned > 0 )) && log "$name: pruned compaction checkpoints from $checkpoint_pruned session(s)"

    # --- Prune stale session entries ---
    local pruned_count
    pruned_count=$(python3 "$SCRIPTS_DIR/prune-sessions.py" "$sessions_json" "$STALE_SUBAGENT_HOURS" "$STALE_CRON_SESSION_HOURS")

    # --- Per-session pre-trim cap: keep only the N most recent, delete the rest immediately ---
    shopt -s nullglob
    declare -A seen_bases
    for pt in "$sessions_dir"/*.jsonl.pre-trim.*; do
        base=$(echo "$pt" | sed 's|\.jsonl\.pre-trim\..*||')
        seen_bases["$base"]=1
    done
    for base in "${!seen_bases[@]}"; do
        mapfile -t pts < <(ls -t "${base}.jsonl.pre-trim."* 2>/dev/null)
        if (( ${#pts[@]} > KEEP_PRE_TRIM_FILES )); then
            excess=("${pts[@]:$KEEP_PRE_TRIM_FILES}")
            for f in "${excess[@]}"; do
                rm -f "$f"
                archive_rm_count=$((archive_rm_count + 1))
            done
        fi
    done
    unset seen_bases
    shopt -u nullglob

    # --- Remove old archives (all known patterns) ---
    while IFS= read -r archive; do
        rm -f "$archive"
        archive_rm_count=$((archive_rm_count + 1))
    done < <(find "$sessions_dir" -maxdepth 1 \
        \( -name "*.reset.*" -o -name "*.deleted.*" -o -name "*.pre-trim.*" \
           -o -name "*.bak-*" -o -name "*.purged.*" -o -name "*.emergency-*" \) \
        -mtime +${ARCHIVE_RETENTION_DAYS} 2>/dev/null)

    # --- Orphan checkpoint cleanup ---
    # Checkpoint files (*.checkpoint.*.jsonl) are created by auto-compaction.
    # Remove them when their parent session no longer has an active .jsonl,
    # or when they are older than archiveRetentionDays.
    shopt -s nullglob
    local cp_rm_count=0
    for cpf in "$sessions_dir"/*.checkpoint.*.jsonl; do
        [[ -f "$cpf" ]] || continue
        # Extract the session ID (everything before .checkpoint.)
        local cp_sid
        cp_sid=$(basename "$cpf" | sed 's|\.checkpoint\..*||')
        # If the active transcript still exists, skip (compaction may be in flight)
        if [[ -f "$sessions_dir/${cp_sid}.jsonl" ]]; then
            # Only delete if it's older than retention (gives in-flight compaction plenty of time)
            local cp_age_days
            cp_age_days=$(( ( $(date +%s) - $(stat -c%Y "$cpf" 2>/dev/null || echo 0) ) / 86400 ))
            (( cp_age_days < ARCHIVE_RETENTION_DAYS )) && continue
        fi
        rm -f "$cpf"
        cp_rm_count=$((cp_rm_count + 1))
        log "$name: removed orphaned checkpoint for $cp_sid"
    done
    shopt -u nullglob
    (( cp_rm_count > 0 )) && { archive_rm_count=$((archive_rm_count + cp_rm_count)); }

    # --- Orphan toolcache cleanup ---
    # Remove .toolcache/ dirs whose session is dead (no active .jsonl) and old enough
    shopt -s nullglob
    local tc_rm_count=0
    for tc in "$sessions_dir"/*.toolcache; do
        [[ -d "$tc" ]] || continue
        local tc_sid
        tc_sid=$(basename "$tc" .toolcache)
        # Skip if active transcript exists
        [[ -f "$sessions_dir/${tc_sid}.jsonl" ]] && continue
        # Skip if the toolcache itself is too fresh (within orphan grace)
        local tc_age_min
        tc_age_min=$(( ( $(date +%s) - $(stat -c%Y "$tc" 2>/dev/null || echo 0) ) / 60 ))
        (( tc_age_min < ORPHAN_GRACE_MINUTES )) && continue
        rm -rf "$tc"
        tc_rm_count=$((tc_rm_count + 1))
        log "$name: removed orphan toolcache ${tc_sid}"
    done
    shopt -u nullglob
    (( tc_rm_count > 0 )) && { archive_rm_count=$((archive_rm_count + tc_rm_count)); }

    # --- Report ---
    local changes=0
    (( reset_count > 0 )) && { log "$name: trimmed $reset_count transcript(s)"; changes=1; }
    (( orphan_count > 0 )) && { log "$name: archived $orphan_count orphan(s)"; changes=1; }
    (( pruned_count > 0 )) && { log "$name: pruned $pruned_count stale session entries"; changes=1; }
    (( archive_rm_count > 0 )) && { log "$name: removed $archive_rm_count old archives"; changes=1; }
    (( changes == 0 )) && log "$name: clean" || true
}

# --- Watchdog ---
run_watchdog() {
    local enabled stale_min
    enabled=$(python3 -c "import json; c=json.load(open('$CONFIG_FILE')); print(str(c.get('watchdog',{}).get('enabled',False)).lower())" 2>/dev/null || echo false)
    [[ "$enabled" != "true" ]] && return
    stale_min=$(python3 -c "import json; c=json.load(open('$CONFIG_FILE')); print(c.get('watchdog',{}).get('staleMinutes',5))" 2>/dev/null || echo 5)
    log "running watchdog (stale threshold: ${stale_min}min)"
    bash "$SCRIPTS_DIR/watchdog.sh" "$stale_min" "$STATE_FILE" 2>&1 || log "watchdog exited non-zero"
}

# --- Main ---
for (( i=0; i<GATEWAY_COUNT; i++ )); do
    process_gateway "$i"
done

run_watchdog

log "Done"


path: scripts/prune-sessions.py
content: #!/usr/bin/env python3
"""Prune stale/done cron and subagent session entries from sessions.json."""
import json, sys, time

path = sys.argv[1]
stale_sub_h = float(sys.argv[2])
stale_cron_h = float(sys.argv[3])
now = time.time()

d = json.load(open(path))
sessions = d.get("sessions", d)
to_prune = []

for k, v in sessions.items():
    updated = v.get("updatedAt", 0)
    if updated > 1e12: updated /= 1000
    if not updated: continue
    age_h = (now - updated) / 3600
    status = v.get("status", "")

    if ":subagent:" in k:
        # Done sub-agents: prune immediately (no grace period needed)
        if status == "done":
            to_prune.append(k)
        # Running/unknown sub-agents: prune after stale threshold
        elif age_h > stale_sub_h:
            to_prune.append(k)
    elif ":cron:" in k and age_h > stale_cron_h:
        to_prune.append(k)

if to_prune:
    for k in to_prune:
        del sessions[k]
    json.dump(d, open(path, "w"), indent=2)

print(len(to_prune))


path: scripts/setup.sh
content: #!/usr/bin/env bash
# session-janitor setup — discovers gateways, generates config, installs cron
set -euo pipefail

SKILL_DIR="$(cd "$(dirname "$0")/.." && pwd)"
CONFIG_FILE="$SKILL_DIR/config.json"
OS="$(uname -s)"

echo "=== Session Janitor Setup ==="
echo ""

# --- Discover gateways ---
echo "Scanning for OpenClaw gateway installations..."
gateways=()
gateway_dirs=()

# Check standard locations
for candidate in "$HOME/.openclaw" "$HOME/.openclaw-"*; do
    [[ -d "$candidate" ]] || continue
    [[ -f "$candidate/openclaw.json" ]] || continue

    config_json="$candidate/openclaw.json"
    sessions_dir="$candidate/agents/main/sessions"

    if [[ ! -d "$sessions_dir" ]]; then
        echo "  ⚠ Found $candidate but no sessions dir — skipping"
        continue
    fi

    # Extract name from dir
    dirname=$(basename "$candidate")
    if [[ "$dirname" == ".openclaw" ]]; then
        name="default"
    else
        name="${dirname#.openclaw-}"
    fi

    # Extract port from config
    port=$(python3 -c "
import json
c = json.load(open('$config_json'))
print(c.get('gateway',{}).get('port', c.get('port', 18789)))
" 2>/dev/null || echo "18789")

    # Extract auth token
    [REDACTED_SECRET] -c "
import json
c = json.load(open('$config_json'))
print(c.get('gateway',{}).get('auth',{}).get('token',''))
" 2>/dev/null || echo "")

    echo "  ✓ Found gateway: $name (${candidate}, port $port)"
    gateways+=("$name")
    gateway_dirs+=("$candidate")
done

if [[ ${#gateways[@]} -eq 0 ]]; then
    echo ""
    echo "❌ No OpenClaw gateway installations found."
    echo "   Expected: ~/.openclaw/openclaw.json or ~/.openclaw-*/openclaw.json"
    exit 1
fi

echo ""
echo "Found ${#gateways[@]} gateway(s): ${gateways[*]}"
echo ""

# --- Check for mem CLI ---
mem_available=false
mem_path=""
if command -v mem &>/dev/null; then
    mem_available=true
    mem_path=$(command -v mem)
    echo "✓ mem CLI found at $mem_path"
elif [[ -x "$HOME/bin/mem" ]]; then
    mem_available=true
    mem_path="$HOME/bin/mem"
    echo "✓ mem CLI found at $mem_path"
elif [[ -x "$HOME/.npm-global/bin/mem" ]]; then
    mem_available=true
    mem_path="$HOME/.npm-global/bin/mem"
    echo "✓ mem CLI found at $mem_path"
else
    echo "ℹ mem CLI not found — LLM extraction will skip structured memory storage"
    echo "  (Memories will still be extracted, just not stored in a searchable DB)"
fi

# --- Check for Python 3 ---
if ! command -v python3 &>/dev/null; then
    echo "❌ python3 is required but not found"
    exit 1
fi
echo "✓ python3 found"

# --- Check OS-specific watcher dependency ---
if [[ "$OS" == "Darwin" ]]; then
    if command -v fswatch &>/dev/null; then
        echo "✓ fswatch found (per-turn watcher)"
    else
        echo "⚠ fswatch not found — per-turn watcher will not work"
        echo "  Install with: brew install fswatch"
        echo "  (cron-based trimming every 15 min will still work without it)"
    fi
else
    if command -v inotifywait &>/dev/null; then
        echo "✓ inotifywait found (per-turn watcher)"
    else
        echo "⚠ inotifywait not found — per-turn watcher will not work"
        echo "  Install with: sudo apt install inotify-tools (or equivalent)"
        echo "  (cron-based trimming every 15 min will still work without it)"
    fi
fi

# --- Generate config ---
echo ""
echo "Generating config.json..."

python3 - "$CONFIG_FILE" "${gateways[@]}" <<'PYEOF'
import json, sys, os, subprocess

config_file = sys.argv[1]
gateway_names = sys.argv[2:]

gateways = []
for name in gateway_names:
    if name == "default":
        config_dir = os.path.expanduser("~/.openclaw")
    else:
        config_dir = os.path.expanduser(f"~/.openclaw-{name}")

    config_json = os.path.join(config_dir, "openclaw.json")
    sessions_dir = os.path.join(config_dir, "agents/main/sessions")

    port = 18789
    token = ""
    try:
        c = json.load(open(config_json))
        port = c.get("gateway", {}).get("port", c.get("port", 18789))
        [REDACTED_SECRET]"gateway", {}).get("auth", {}).get("token", "")
    except:
        pass

    gateways.append({
        "name": name,
        "configDir": config_dir,
        "sessionsDir": sessions_dir,
        "port": port,
        "token": token,
    })

# Check for mem
mem_path = ""
for candidate in [
    subprocess.run(["which", "mem"], capture_output=True, text=True).stdout.strip(),
    os.path.expanduser("~/bin/mem"),
    os.path.expanduser("~/.npm-global/bin/mem"),
]:
    if candidate and os.path.isfile(candidate) and os.access(candidate, os.X_OK):
        mem_path = candidate
        break

config = {
    "gateways": gateways,
    "trimMaxKB": 250,
    "keepPairs": 10,
    "keepFullPairs": 2,
    "minArchivePairs": 5,
    "trimFullThresholdPct": 50,
    "archiveRetentionDays": 7,
    "orphanGraceMinutes": 30,
    "staleSubagentHours": 24,
    "staleCronSessionHours": 24,
    "llmExtraction": {
        "enabled": True,
        "maxPerRun": 1,
        "model": "openclaw",
        "maxInputChars": 20000,
        "timeoutSecs": 60,
        "maxMemories": 15,
        "minArchived": 3,
    },
    "memCli": {
        "enabled": bool(mem_path),
        "path": mem_path,
    },
    "stateFile": os.path.expanduser("~/.openclaw/session-janitor-state.json"),
    "watcherDebounceSecs": 3,
    "cronSchedule": "*/15 * * * *",
    "logFile": "/tmp/session-janitor.log",
}

json.dump(config, open(config_file, "w"), indent=2)
print(f"  Written to {config_file}")
PYEOF

echo ""

# --- Install cron ---
JANITOR_SCRIPT="$SKILL_DIR/scripts/janitor.sh"
CRON_SCHEDULE=$(python3 -c "import json; print(json.load(open('$CONFIG_FILE')).get('cronSchedule','*/15 * * * *'))")
LOG_FILE=$(python3 -c "import json; print(json.load(open('$CONFIG_FILE')).get('logFile','/tmp/session-janitor.log'))")

CRON_LINE="$CRON_SCHEDULE $JANITOR_SCRIPT >> $LOG_FILE 2>&1"

# Check if already installed
if crontab -l 2>/dev/null | grep -qF "session-janitor"; then
    echo "Updating existing cron entry..."
    crontab -l 2>/dev/null | grep -v "session-janitor" | { cat; echo "$CRON_LINE  # session-janitor"; } | crontab -
else
    echo "Installing cron entry..."
    (crontab -l 2>/dev/null; echo "$CRON_LINE  # session-janitor") | crontab -
fi

echo "  ✓ Cron installed: $CRON_SCHEDULE"
echo ""

# --- Install watcher service (systemd on Linux, launchd on macOS) ---
if [[ "$OS" == "Darwin" ]]; then
    PLIST_TEMPLATE="$SKILL_DIR/session-janitor-watcher.plist"
    LAUNCH_AGENTS_DIR="$HOME/Library/LaunchAgents"
    PLIST_LABEL="ai.openclaw.session-janitor-watcher"
    PLIST_DEST="$LAUNCH_AGENTS_DIR/${PLIST_LABEL}.plist"

    if [[ -f "$PLIST_TEMPLATE" ]]; then
        mkdir -p "$LAUNCH_AGENTS_DIR"
        sed "s|SKILL_DIR_PLACEHOLDER|$SKILL_DIR|g" "$PLIST_TEMPLATE" > "$PLIST_DEST"
        launchctl unload "$PLIST_DEST" 2>/dev/null || true
        launchctl load "$PLIST_DEST" 2>/dev/null && \
            echo "  ✓ Watcher LaunchAgent installed and started" || \
            echo "  ⚠ LaunchAgent load failed — check: launchctl list | grep session-janitor"
        echo "  Location: $PLIST_DEST"
        echo "  Stop:     launchctl unload \"$PLIST_DEST\""
        echo "  Start:    launchctl load \"$PLIST_DEST\""
    else
        echo "  ⚠ Plist template not found at $PLIST_TEMPLATE — skipping watcher service"
    fi
else
    # Linux — systemd user service
    SERVICE_TEMPLATE="$SKILL_DIR/session-janitor-watcher.service"
    SERVICE_DIR="$HOME/.config/systemd/user"
    SERVICE_DEST="$SERVICE_DIR/session-janitor-watcher.service"

    if [[ -f "$SERVICE_TEMPLATE" ]]; then
        mkdir -p "$SERVICE_DIR"
        sed "s|SKILL_DIR_PLACEHOLDER|$SKILL_DIR|g" "$SERVICE_TEMPLATE" > "$SERVICE_DEST"
        systemctl --user daemon-reload
        systemctl --user enable --now session-janitor-watcher.service 2>/dev/null && \
            echo "  ✓ Watcher service enabled and started" || \
            echo "  ⚠ Watcher service install failed — check: systemctl --user status session-janitor-watcher"
    else
        echo "  ⚠ Service template not found at $SERVICE_TEMPLATE — skipping watcher service"
    fi
fi

echo ""

# --- Verify ---
echo "=== Setup Complete ==="
echo ""
echo "Config:    $CONFIG_FILE"
echo "Script:    $JANITOR_SCRIPT"
echo "Log:       $LOG_FILE"
echo "Schedule:  $CRON_SCHEDULE"
echo ""
echo "Test with: bash $JANITOR_SCRIPT"
echo "Logs:      tail -f $LOG_FILE"
if [[ "$OS" == "Darwin" ]]; then
    echo "Watcher:   tail -f /tmp/session-janitor-watcher.log"
    echo "           launchctl list | grep session-janitor"
else
    echo "Watcher:   tail -f /tmp/session-janitor-watcher.log"
    echo "           systemctl --user status session-janitor-watcher"
fi


path: scripts/sidecar.py
content: #!/usr/bin/env python3
"""Sidecar tool-output offloader.

Scans a JSONL transcript for large toolResult entries and moves their content
to sidecar files, replacing the inline content with a compact stub.

Usage:
    sidecar.py <jsonl_file> <session_id> [min_entry_bytes]

Sidecar layout:
    <jsonl_dir>/<session_id>.toolcache/<tool-call-id>.txt

Stub format (replaces inline content):
    [tool output offloaded to .toolcache/<tool-call-id>.txt — <N> bytes.
     Use the Read tool to access it if needed.]

- Idempotent: entries already containing a stub are skipped.
- Atomic write: original is overwritten only if ≥1 entry was changed.
- Restart-safe: sidecar files are adjacent to the transcript, survive gateway
  restarts and config changes.

Exits:
    0 — success (even if nothing was sidecared)
    1 — error
"""
import json
import os
import sys
import shutil
from pathlib import Path
from datetime import datetime, timezone

STUB_MARKER = "[tool output offloaded to"
IMAGE_STUB_MARKER = "[image offloaded to"
DEFAULT_MIN_BYTES = 5120  # 5 KB per entry
IMAGE_MIN_BYTES = 1024  # 1 KB — offload any non-trivial base64 image


def entry_content_size(entry: dict) -> int:
    """Return approximate byte size of a toolResult entry's content."""
    raw = entry.get("message", {}).get("content", [])
    if isinstance(raw, str):
        return len(raw.encode("utf-8"))
    total = 0
    for block in raw:
        if isinstance(block, dict) and block.get("type") == "text":
            total += len(block.get("text", "").encode("utf-8"))
    return total


def is_already_stubbed(entry: dict) -> bool:
    raw = entry.get("message", {}).get("content", [])
    if isinstance(raw, str):
        return STUB_MARKER in raw
    for block in raw:
        if isinstance(block, dict) and block.get("type") == "text":
            if STUB_MARKER in block.get("text", ""):
                return True
    return False


def extract_text(entry: dict) -> str:
    """Extract full text content from a toolResult entry."""
    raw = entry.get("message", {}).get("content", [])
    if isinstance(raw, str):
        return raw
    parts = []
    for block in raw:
        if isinstance(block, dict) and block.get("type") == "text":
            parts.append(block.get("text", ""))
    return "\n".join(parts)


def make_stub_content(sidecar_filename: str, byte_count: int) -> list:
    return [
        {
            "type": "text",
            "text": (
                f"{STUB_MARKER} .toolcache/{sidecar_filename} — {byte_count} bytes. "
                "Use the Read tool to access it if needed.]"
            ),
        }
    ]


def offload_images_in_entry(entry, toolcache_dir, session_id, img_counter):
    """Scan any entry's content for inline base64 images and offload them.

    Returns (modified_entry, count_offloaded, new_img_counter).
    """
    import copy
    import base64

    msg = entry.get("message", {})
    content = msg.get("content", [])
    if not isinstance(content, list):
        return entry, 0, img_counter

    modified = False
    new_content = []
    offloaded = 0

    for block in content:
        if not isinstance(block, dict):
            new_content.append(block)
            continue

        # Check for inline base64 image: {type: "image", data: "<base64>"}
        if block.get("type") == "image" and block.get("data"):
            b64_data = block["data"]
            if len(b64_data) < IMAGE_MIN_BYTES:
                new_content.append(block)
                continue

            # Already offloaded?
            if IMAGE_STUB_MARKER in b64_data:
                new_content.append(block)
                continue

            # Determine extension from mimeType
            mime = block.get("mimeType", block.get("media_type", "image/jpeg"))
            ext = {"image/jpeg": ".jpg", "image/png": ".png",
                   "image/gif": ".gif", "image/webp": ".webp"}.get(mime, ".jpg")

            entry_id = entry.get("id", f"unk-{img_counter}")
            safe_id = "".join(c if c.isalnum() or c in "-." else "_" for c in entry_id)
            sidecar_filename = f"{safe_id}_img_{img_counter}{ext}"
            sidecar_path = toolcache_dir / sidecar_filename

            # Decode and write binary image
            toolcache_dir.mkdir(parents=True, exist_ok=True)
            try:
                img_bytes = base64.b64decode(b64_data)
                sidecar_path.write_bytes(img_bytes)
            except Exception:
                # Fallback: write raw base64 as text
                sidecar_path.write_text(b64_data, encoding="utf-8")
            actual_bytes = sidecar_path.stat().st_size

            # Replace image block with text stub
            stub_text = (
                f"{IMAGE_STUB_MARKER} .toolcache/{sidecar_filename} — "
                f"{actual_bytes} bytes ({mime}). "
                f"Use the Read/image tool to access it if needed.]"
            )
            new_content.append({"type": "text", "text": stub_text})
            offloaded += 1
            img_counter += 1

            b64_kb = len(b64_data) // 1024
            print(f"  offloaded image ({entry_id[:12]}): {b64_kb}KB b64 → .toolcache/{sidecar_filename}")
            modified = True
            continue

        # Also handle Anthropic-style source.data images:
        # {type: "image", source: {type: "base64", media_type: "...", data: "..."}}
        if block.get("type") == "image" and isinstance(block.get("source"), dict):
            src = block["source"]
            b64_data = src.get("data", "")
            if len(b64_data) < IMAGE_MIN_BYTES:
                new_content.append(block)
                continue
            if IMAGE_STUB_MARKER in b64_data:
                new_content.append(block)
                continue

            mime = src.get("media_type", "image/jpeg")
            ext = {"image/jpeg": ".jpg", "image/png": ".png",
                   "image/gif": ".gif", "image/webp": ".webp"}.get(mime, ".jpg")

            entry_id = entry.get("id", f"unk-{img_counter}")
            safe_id = "".join(c if c.isalnum() or c in "-." else "_" for c in entry_id)
            sidecar_filename = f"{safe_id}_img_{img_counter}{ext}"
            sidecar_path = toolcache_dir / sidecar_filename

            toolcache_dir.mkdir(parents=True, exist_ok=True)
            try:
                img_bytes = base64.b64decode(b64_data)
                sidecar_path.write_bytes(img_bytes)
            except Exception:
                sidecar_path.write_text(b64_data, encoding="utf-8")
            actual_bytes = sidecar_path.stat().st_size

            stub_text = (
                f"{IMAGE_STUB_MARKER} .toolcache/{sidecar_filename} — "
                f"{actual_bytes} bytes ({mime}). "
                f"Use the Read/image tool to access it if needed.]"
            )
            new_content.append({"type": "text", "text": stub_text})
            offloaded += 1
            img_counter += 1

            b64_kb = len(b64_data) // 1024
            print(f"  offloaded image ({entry_id[:12]}): {b64_kb}KB b64 → .toolcache/{sidecar_filename}")
            modified = True
            continue

        new_content.append(block)

    if not modified:
        return entry, 0, img_counter

    new_entry = copy.deepcopy(entry)
    new_entry["message"]["content"] = new_content
    return new_entry, offloaded, img_counter


def main():
    if len(sys.argv) < 3:
        print(f"Usage: {sys.argv[0]} <jsonl_file> <session_id> [min_entry_bytes]", file=sys.stderr)
        sys.exit(1)

    jsonl_path = Path(sys.argv[1])
    session_id = sys.argv[2]
    min_bytes = int(sys.argv[3]) if len(sys.argv) > 3 else DEFAULT_MIN_BYTES

    if not jsonl_path.exists():
        print(f"ERROR: {jsonl_path} not found", file=sys.stderr)
        sys.exit(1)

    # Read transcript
    entries = []
    with open(jsonl_path, encoding="utf-8") as f:
        for line in f:
            line = line.strip()
            if not line:
                continue
            try:
                entries.append(json.loads(line))
            except json.JSONDecodeError as e:
                print(f"WARN: skipping malformed line: {e}", file=sys.stderr)
                continue

    if not entries:
        print("No entries — nothing to do")
        sys.exit(0)

    # Sidecar dir adjacent to the transcript
    toolcache_dir = jsonl_path.parent / f"{session_id}.toolcache"
    sidecared = 0
    images_offloaded = 0
    img_counter = 0
    changed = []

    for entry in entries:
        # --- Phase 1: offload inline base64 images from ANY entry ---
        entry, img_count, img_counter = offload_images_in_entry(
            entry, toolcache_dir, session_id, img_counter
        )
        images_offloaded += img_count

        # --- Phase 2: offload large toolResult text (existing behavior) ---
        msg = entry.get("message", {})
        if msg.get("role") != "toolResult":
            changed.append(entry)
            continue

        if is_already_stubbed(entry):
            changed.append(entry)
            continue

        size = entry_content_size(entry)
        if size < min_bytes:
            changed.append(entry)
            continue

        # Determine sidecar filename
        tcid = msg.get("toolCallId") or msg.get("id") or f"unknown-{sidecared}"
        # Sanitize for filesystem
        safe_id = "".join(c if c.isalnum() or c in "-_." else "_" for c in tcid)
        sidecar_filename = f"{safe_id}.txt"
        sidecar_path = toolcache_dir / sidecar_filename

        # Extract text and write to sidecar
        text = extract_text(entry)
        toolcache_dir.mkdir(parents=True, exist_ok=True)
        sidecar_path.write_text(text, encoding="utf-8")
        actual_bytes = sidecar_path.stat().st_size

        # Replace content with stub
        import copy
        new_entry = copy.deepcopy(entry)
        new_entry["message"]["content"] = make_stub_content(sidecar_filename, actual_bytes)
        new_entry["message"]["_sidecaredBytes"] = actual_bytes
        changed.append(new_entry)
        sidecared += 1

        tool_name = msg.get("toolName", "tool")
        print(
            f"  sidecared {tool_name} ({tcid[:12]}): "
            f"{actual_bytes // 1024}KB → .toolcache/{sidecar_filename}"
        )

    total_offloaded = sidecared + images_offloaded
    if total_offloaded == 0:
        print(f"No entries exceeded {min_bytes} bytes — nothing sidecared")
        sys.exit(0)

    # Atomic write
    tmp_path = jsonl_path.with_suffix(".jsonl.sidecar-tmp")
    with open(tmp_path, "w", encoding="utf-8") as f:
        for entry in changed:
            f.write(json.dumps(entry) + "\n")

    shutil.move(str(tmp_path), str(jsonl_path))

    orig_kb = sum(len(json.dumps(e)) for e in entries) // 1024
    new_kb = jsonl_path.stat().st_size // 1024
    parts = []
    if sidecared:
        parts.append(f"{sidecared} tool outputs")
    if images_offloaded:
        parts.append(f"{images_offloaded} images")
    print(
        f"Sidecar complete: {' + '.join(parts)} offloaded, "
        f"{orig_kb}KB → {new_kb}KB "
        f"(sidecar dir: {toolcache_dir})"
    )
    sys.exit(0)


if __name__ == "__main__":
    main()


path: scripts/test-sidecar.py
content: #!/usr/bin/env python3
"""Integration tests for sidecar.py.

Tests: basic offload, idempotency, restart resilience, trim interaction,
       restart scenarios (fresh session header after compaction).
"""
import json
import os
import shutil
import subprocess
import sys
import tempfile
import time
from pathlib import Path
from datetime import datetime, timezone

SCRIPT_DIR = Path(__file__).parent
SIDECAR_PY = SCRIPT_DIR / "sidecar.py"
TRIM_PY = SCRIPT_DIR / "trim.py"

TEST_SESSIONS_DIR = Path.home() / ".openclaw-test/agents/main/sessions"

PASS = 0
FAIL = 0


def log(msg):
    print(f"[{datetime.now().strftime('%H:%M:%S')}] {msg}")


def ok(label):
    global PASS
    print(f"  ✅ {label}")
    PASS += 1


def fail(label):
    global FAIL
    print(f"  ❌ {label}")
    FAIL += 1


def ts():
    return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.000Z")


def make_session_header(sid):
    return {"type": "session", "id": "hdr-001", "sessionId": sid,
            "timestamp": ts(), "agentId": "main", "channelId": "test"}


def make_user(id_, content, parent="hdr-001"):
    return {"type": "message", "id": id_, "parentId": parent,
            "timestamp": ts(), "message": {"role": "user", "content": content}}


def make_assistant(id_, text, tool_ids=None, parent="hdr-001"):
    content = [{"type": "text", "text": text}]
    for tid in (tool_ids or []):
        content.append({"type": "toolCall", "id": tid, "name": "exec",
                         "input": {"command": "ls"}})
    return {"type": "message", "id": id_, "parentId": parent,
            "timestamp": ts(), "message": {"role": "assistant", "content": content}}


def make_tool_result(id_, tcid, tool_name, content_text, parent="hdr-001"):
    return {"type": "message", "id": id_, "parentId": parent, "timestamp": ts(),
            "message": {"role": "toolResult", "toolCallId": tcid,
                        "toolName": tool_name,
                        "content": [{"type": "text", "text": content_text}]}}


def big_text(kb=6):
    line = "output data here " * 4 + "\n"  # ~80 chars
    lines = (kb * 1024) // len(line)
    return "".join(f"Line {i:05d}: {line}" for i in range(lines))


def small_text():
    return "exit 0\nok"


def write_jsonl(path, entries):
    path.parent.mkdir(parents=True, exist_ok=True)
    with open(path, "w") as f:
        for e in entries:
            f.write(json.dumps(e) + "\n")


def read_jsonl(path):
    entries = []
    with open(path) as f:
        for line in f:
            line = line.strip()
            if line:
                try:
                    entries.append(json.loads(line))
                except Exception:
                    pass
    return entries


def run_sidecar(jsonl, sid, min_bytes=4096):
    r = subprocess.run(
        [sys.executable, str(SIDECAR_PY), str(jsonl), sid, str(min_bytes)],
        capture_output=True, text=True
    )
    if r.stdout.strip():
        for line in r.stdout.strip().splitlines():
            print(f"    [sidecar] {line}")
    if r.returncode != 0 and r.stderr.strip():
        print(f"    [sidecar STDERR] {r.stderr.strip()}", file=sys.stderr)
    return r.returncode


def run_trim(jsonl, sid, state_file, keep_pairs=10):
    r = subprocess.run(
        [sys.executable, str(TRIM_PY), str(jsonl), sid, "test", str(state_file),
         str(keep_pairs), "2", "5", "50", "250"],
        capture_output=True, text=True
    )
    if r.stdout.strip():
        for line in r.stdout.strip().splitlines():
            print(f"    [trim] {line}")
    return r.returncode


def has_stub(entries):
    for e in entries:
        raw = e.get("message", {}).get("content", [])
        if isinstance(raw, str) and "[tool output offloaded to" in raw:
            return True
        for b in (raw if isinstance(raw, list) else []):
            if isinstance(b, dict) and "[tool output offloaded to" in b.get("text", ""):
                return True
    return False


def count_stubs(entries):
    count = 0
    for e in entries:
        raw = e.get("message", {}).get("content", [])
        if isinstance(raw, str) and "[tool output offloaded to" in raw:
            count += 1
            continue
        for b in (raw if isinstance(raw, list) else []):
            if isinstance(b, dict) and "[tool output offloaded to" in b.get("text", ""):
                count += 1
                break
    return count


def make_sid():
    return f"test-sidecar-{int(time.time() * 1000)}"


# ──────────────────────────────────────────────────────────
# TEST 1: Basic sidecar offload
# ──────────────────────────────────────────────────────────
def test_basic_offload():
    log("TEST 1: Basic sidecar offload (large toolResult → sidecar file)")
    sid = make_sid()
    jsonl = TEST_SESSIONS_DIR / f"{sid}.jsonl"
    cache = TEST_SESSIONS_DIR / f"{sid}.toolcache"
    try:
        entries = [
            make_session_header(sid),
            make_user("u-001", "run something"),
            make_assistant("a-001", "sure", ["tc-001"]),
            make_tool_result("tr-001", "tc-001", "exec", big_text(6), "a-001"),
        ]
        write_jsonl(jsonl, entries)
        orig_kb = jsonl.stat().st_size // 1024

        rc = run_sidecar(jsonl, sid)
        new_kb = jsonl.stat().st_size // 1024

        if rc == 0:
            ok(f"sidecar.py exited 0")
        else:
            fail(f"sidecar.py exited {rc}")

        ok("toolcache dir created") if cache.is_dir() else fail("toolcache dir missing")

        sidecar_file = cache / "tc-001.txt"
        ok("sidecar file tc-001.txt created") if sidecar_file.exists() else fail("sidecar file missing")

        new_entries = read_jsonl(jsonl)
        ok("stub injected into transcript") if has_stub(new_entries) else fail("no stub in transcript")

        ok(f"transcript shrank ({orig_kb}KB → {new_kb}KB)") if new_kb < orig_kb else fail("transcript did not shrink")

    finally:
        jsonl.unlink(missing_ok=True)
        shutil.rmtree(cache, ignore_errors=True)


# ──────────────────────────────────────────────────────────
# TEST 2: Idempotency
# ──────────────────────────────────────────────────────────
def test_idempotency():
    log("TEST 2: Idempotency (run sidecar twice on same file)")
    sid = make_sid()
    jsonl = TEST_SESSIONS_DIR / f"{sid}.jsonl"
    cache = TEST_SESSIONS_DIR / f"{sid}.toolcache"
    try:
        entries = [
            make_session_header(sid),
            make_user("u-001", "run"),
            make_assistant("a-001", "ok", ["tc-001"]),
            make_tool_result("tr-001", "tc-001", "exec", big_text(6), "a-001"),
        ]
        write_jsonl(jsonl, entries)
        run_sidecar(jsonl, sid)

        snapshot = jsonl.read_text()
        run_sidecar(jsonl, sid)
        after = jsonl.read_text()

        ok("transcript unchanged on second run") if snapshot == after else fail("transcript changed on second run (not idempotent)")

        file_count = len(list(cache.iterdir())) if cache.exists() else 0
        ok(f"no duplicate sidecar files ({file_count})") if file_count == 1 else fail(f"unexpected file count: {file_count}")

    finally:
        jsonl.unlink(missing_ok=True)
        shutil.rmtree(cache, ignore_errors=True)


# ──────────────────────────────────────────────────────────
# TEST 3: Small results NOT sidecared
# ──────────────────────────────────────────────────────────
def test_small_not_sidecared():
    log("TEST 3: Small toolResult (below threshold) — should NOT be sidecared")
    sid = make_sid()
    jsonl = TEST_SESSIONS_DIR / f"{sid}.jsonl"
    cache = TEST_SESSIONS_DIR / f"{sid}.toolcache"
    try:
        entries = [
            make_session_header(sid),
            make_user("u-001", "hello"),
            make_assistant("a-001", "hi", ["tc-001"]),
            make_tool_result("tr-001", "tc-001", "exec", small_text(), "a-001"),
        ]
        write_jsonl(jsonl, entries)
        run_sidecar(jsonl, sid)

        ok("no toolcache dir for small result") if not cache.exists() else fail("toolcache dir created unexpectedly")
        new_entries = read_jsonl(jsonl)
        ok("no stub in transcript") if not has_stub(new_entries) else fail("stub found unexpectedly")
    finally:
        jsonl.unlink(missing_ok=True)
        shutil.rmtree(cache, ignore_errors=True)


# ──────────────────────────────────────────────────────────
# TEST 4: Mixed large + small
# ──────────────────────────────────────────────────────────
def test_mixed():
    log("TEST 4: Mixed large + small results")
    sid = make_sid()
    jsonl = TEST_SESSIONS_DIR / f"{sid}.jsonl"
    cache = TEST_SESSIONS_DIR / f"{sid}.toolcache"
    try:
        entries = [
            make_session_header(sid),
            make_user("u-001", "step 1"),
            make_assistant("a-001", "ok", ["tc-001"]),
            make_tool_result("tr-001", "tc-001", "exec", big_text(7), "a-001"),
            make_user("u-002", "step 2", "a-001"),
            make_assistant("a-002", "done", ["tc-002"], "u-002"),
            make_tool_result("tr-002", "tc-002", "Read", small_text(), "a-002"),
        ]
        write_jsonl(jsonl, entries)
        run_sidecar(jsonl, sid)

        ok("large result sidecared") if (cache / "tc-001.txt").exists() else fail("large result not sidecared")
        ok("small result NOT sidecared") if not (cache / "tc-002.txt").exists() else fail("small result incorrectly sidecared")
    finally:
        jsonl.unlink(missing_ok=True)
        shutil.rmtree(cache, ignore_errors=True)


# ──────────────────────────────────────────────────────────
# TEST 5: Restart resilience
# Sidecar files survive; stubs remain readable after restart
# ──────────────────────────────────────────────────────────
def test_restart_resilience():
    log("TEST 5: Restart resilience (sidecar files persist across simulated restart)")
    sid = make_sid()
    jsonl = TEST_SESSIONS_DIR / f"{sid}.jsonl"
    cache = TEST_SESSIONS_DIR / f"{sid}.toolcache"
    try:
        entries = [
            make_session_header(sid),
            make_user("u-001", "run"),
            make_assistant("a-001", "ok", ["tc-001"]),
            make_tool_result("tr-001", "tc-001", "exec", big_text(6), "a-001"),
        ]
        write_jsonl(jsonl, entries)
        run_sidecar(jsonl, sid)

        sidecar_file = cache / "tc-001.txt"
        pre_restart_size = sidecar_file.stat().st_size if sidecar_file.exists() else 0

        # Simulate gateway restart: re-read transcript from disk (no state)
        restored_entries = read_jsonl(jsonl)

        # Sidecar file still there
        ok(f"sidecar file intact ({pre_restart_size} bytes)") if pre_restart_size > 1000 else fail(f"sidecar file too small: {pre_restart_size}")

        # Stub references the filename
        ok("transcript stub references filename") if has_stub(restored_entries) else fail("stub missing after restart")

        # Agent can Read the sidecar file to recover content
        recovered = sidecar_file.read_text()
        ok(f"sidecar content recoverable ({len(recovered)} chars)") if len(recovered) > 1000 else fail("sidecar content empty")

        # New session header scenario: OC writes a fresh header, stubs stay
        # (new session means different SID — old sidecar files remain on disk)
        new_sid = make_sid()
        new_entries = [make_session_header(new_sid)] + restored_entries[1:]
        new_jsonl = TEST_SESSIONS_DIR / f"{new_sid}.jsonl"
        write_jsonl(new_jsonl, new_entries)
        ok("new session header written without losing stubs") if has_stub(read_jsonl(new_jsonl)) else fail("stubs lost after new session header")
        new_jsonl.unlink(missing_ok=True)

    finally:
        jsonl.unlink(missing_ok=True)
        shutil.rmtree(cache, ignore_errors=True)


# ──────────────────────────────────────────────────────────
# TEST 6: Sidecar + trim interaction
# ──────────────────────────────────────────────────────────
def test_sidecar_plus_trim():
    log("TEST 6: Sidecar + trim interaction (sidecar runs before trim)")
    sid = make_sid()
    jsonl = TEST_SESSIONS_DIR / f"{sid}.jsonl"
    cache = TEST_SESSIONS_DIR / f"{sid}.toolcache"
    state_file = Path(tempfile.mktemp(suffix=".json"))
    state_file.write_text("{}")
    try:
        entries = [make_session_header(sid)]
        for i in range(15):
            uid = f"u-{i:02d}"
            aid = f"a-{i:02d}"
            tid = f"tc-{i:02d}"
            entries.append(make_user(uid, f"query {i}"))
            entries.append(make_assistant(aid, f"response {i}", [tid]))
            entries.append(make_tool_result(f"tr-{i:02d}", tid, "exec", big_text(8), aid))

        write_jsonl(jsonl, entries)
        before_kb = jsonl.stat().st_size // 1024

        run_sidecar(jsonl, sid)
        after_sidecar_kb = jsonl.stat().st_size // 1024

        files = list(cache.iterdir()) if cache.exists() else []
        ok(f"sidecar offloaded {len(files)} entries (expected ~15)") if len(files) >= 14 else fail(f"only {len(files)} sidecar files")

        ok(f"sidecar reduced size ({before_kb}KB → {after_sidecar_kb}KB)") if after_sidecar_kb < before_kb else fail("sidecar did not reduce size")

        rc = run_trim(jsonl, sid, state_file)
        after_trim_kb = jsonl.stat().st_size // 1024
        log(f"  After trim: {after_trim_kb}KB")

        # Trim may return 1 if "nothing to do" — both 0 and 1 are OK
        ok(f"trim ran (rc={rc})") if rc in (0, 1) else fail(f"trim failed with rc={rc}")

        # Sidecar files survive trim
        files_after = list(cache.iterdir()) if cache.exists() else []
        ok(f"sidecar files intact after trim ({len(files_after)})") if len(files_after) >= len(files) else fail(f"sidecar files lost: before={len(files)}, after={len(files_after)}")

    finally:
        jsonl.unlink(missing_ok=True)
        for pre in TEST_SESSIONS_DIR.glob(f"{sid}.jsonl.pre-trim.*"):
            pre.unlink(missing_ok=True)
        shutil.rmtree(cache, ignore_errors=True)
        state_file.unlink(missing_ok=True)


# ──────────────────────────────────────────────────────────
# TEST 7: Multiple tool calls in one assistant turn
# ──────────────────────────────────────────────────────────
def test_multi_tool_turn():
    log("TEST 7: Multiple large tool calls in one assistant turn")
    sid = make_sid()
    jsonl = TEST_SESSIONS_DIR / f"{sid}.jsonl"
    cache = TEST_SESSIONS_DIR / f"{sid}.toolcache"
    try:
        entries = [
            make_session_header(sid),
            make_user("u-001", "run two things"),
            make_assistant("a-001", "running both", ["tc-001", "tc-002"]),
            make_tool_result("tr-001", "tc-001", "exec", big_text(6), "a-001"),
            make_tool_result("tr-002", "tc-002", "exec", big_text(6), "a-001"),
        ]
        write_jsonl(jsonl, entries)
        run_sidecar(jsonl, sid)

        ok("both results sidecared") if (cache / "tc-001.txt").exists() and (cache / "tc-002.txt").exists() else fail("missing sidecar files for multi-tool turn")

        new_entries = read_jsonl(jsonl)
        n = count_stubs(new_entries)
        ok(f"two stubs in transcript ({n})") if n == 2 else fail(f"expected 2 stubs, got {n}")

    finally:
        jsonl.unlink(missing_ok=True)
        shutil.rmtree(cache, ignore_errors=True)


# ──────────────────────────────────────────────────────────
# TEST 8: Post-restart compaction scenario
# After a gateway restart, OC writes a new compaction entry.
# Sidecar stubs are preserved through compaction + re-trim cycle.
# ──────────────────────────────────────────────────────────
def test_restart_then_compaction():
    log("TEST 8: Post-restart compaction — stubs survive trim of post-restart transcript")
    sid = make_sid()
    jsonl = TEST_SESSIONS_DIR / f"{sid}.jsonl"
    cache = TEST_SESSIONS_DIR / f"{sid}.toolcache"
    state_file = Path(tempfile.mktemp(suffix=".json"))
    state_file.write_text("{}")
    try:
        # Phase 1: pre-restart session with large tool output
        entries = [make_session_header(sid)]
        for i in range(12):
            uid, aid, tid = f"u-{i:02d}", f"a-{i:02d}", f"tc-{i:02d}"
            entries.append(make_user(uid, f"query {i}"))
            entries.append(make_assistant(aid, f"ok {i}", [tid]))
            entries.append(make_tool_result(f"tr-{i:02d}", tid, "exec", big_text(6), aid))
        write_jsonl(jsonl, entries)
        run_sidecar(jsonl, sid)

        # Phase 2: simulate gateway restart — OC rewrites transcript with new session header
        # In OC, restart writes a .reset. copy; we simulate by adding a new compaction entry
        post_restart_entries = read_jsonl(jsonl)
        compaction = {
            "type": "compaction",
            "id": "compact-after-restart",
            "parentId": "hdr-001",
            "timestamp": ts(),
            "summary": "Gateway restarted. Prior context above.",
            "firstKeptEntryId": None,
            "tokensBefore": 0,
        }
        # Insert compaction after header
        post_restart_entries.insert(1, compaction)
        write_jsonl(jsonl, post_restart_entries)

        # Phase 3: Add more conversation after restart
        last_id = post_restart_entries[-1]["id"] if post_restart_entries else "compact-after-restart"
        for i in range(3):
            uid, aid = f"post-u-{i}", f"post-a-{i}"
            post_restart_entries.append(make_user(uid, f"post-restart query {i}", last_id))
            post_restart_entries.append(make_assistant(aid, f"post-restart reply {i}", [], uid))
            last_id = aid
        write_jsonl(jsonl, post_restart_entries)

        # Phase 4: Run trim on the post-restart transcript
        rc = run_trim(jsonl, sid, state_file, keep_pairs=5)
        log(f"  trim rc={rc}")

        # Sidecar files must still be intact
        if cache.exists():
            file_count = len(list(cache.iterdir()))
            ok(f"sidecar files intact after restart+compaction+trim ({file_count})") if file_count >= 10 else fail(f"only {file_count} sidecar files remain")
        else:
            fail("toolcache dir missing after restart+trim")

    finally:
        jsonl.unlink(missing_ok=True)
        for pre in TEST_SESSIONS_DIR.glob(f"{sid}.jsonl.pre-trim.*"):
            pre.unlink(missing_ok=True)
        shutil.rmtree(cache, ignore_errors=True)
        state_file.unlink(missing_ok=True)


# ──────────────────────────────────────────────────────────
# Run all tests
# ──────────────────────────────────────────────────────────
if __name__ == "__main__":
    TEST_SESSIONS_DIR.mkdir(parents=True, exist_ok=True)

    test_basic_offload()
    test_idempotency()
    test_small_not_sidecared()
    test_mixed()
    test_restart_resilience()
    test_sidecar_plus_trim()
    test_multi_tool_turn()
    test_restart_then_compaction()

    print()
    print("══════════════════════════════════════")
    print(f"  Results: ✅ {PASS} passed  ❌ {FAIL} failed")
    print("══════════════════════════════════════")

    sys.exit(1 if FAIL > 0 else 0)


path: scripts/test-sidecar.sh
content: #!/usr/bin/env bash
# test-sidecar.sh — Wrapper that runs the Python test suite
set -euo pipefail
SKILL_DIR="$(cd "$(dirname "$0")/.." && pwd)"
exec python3 "$SKILL_DIR/scripts/test-sidecar.py" "$@"


path: scripts/trim.py
content: #!/usr/bin/env python3
"""Trim an oversized JSONL transcript: keep session header + synthetic compaction + recent exchanges.

Usage: trim.py <jsonl_file> <session_id> <gateway_name> <state_file> [keep_pairs [keep_full_pairs [min_archive_pairs [trim_full_threshold_pct [trim_max_kb]]]]]

- Writes a trimmed transcript in-place with:
  - Original session header
  - A synthetic compaction entry summarizing what was archived
  - The last `keep_pairs` user/assistant exchange pairs (default 10)
  - Full toolResult bodies preserved for the most recent `keep_full_pairs` assistant turns (default 2)
- If the retained pairs still exceed `trim_full_threshold_pct`% of `trim_max_kb`, all pairs are
  further stripped (tool args removed, thinking dropped) to minimize size.
- Archives the original as .pre-trim.<timestamp>

Exits 0 on success, 1 if nothing to do.
"""
import json, sys, os, shutil, hashlib
from datetime import datetime

def random_id():
    return hashlib.md5(os.urandom(8)).hexdigest()[:8]

def main():
    jsonl_file = sys.argv[1]
    session_id = sys.argv[2]
    gateway = sys.argv[3]
    state_file = sys.argv[4]
    keep_pairs = int(sys.argv[5]) if len(sys.argv) > 5 else 10
    keep_full_pairs = int(sys.argv[6]) if len(sys.argv) > 6 else 2
    min_archive_pairs = int(sys.argv[7]) if len(sys.argv) > 7 else 5  # kept for backward compat; no longer blocks trim
    trim_full_threshold_pct = float(sys.argv[8]) if len(sys.argv) > 8 else 50.0
    trim_max_kb = int(sys.argv[9]) if len(sys.argv) > 9 else 250

    entries = []
    with open(jsonl_file) as f:
        for line in f:
            line = line.strip()
            if not line: continue
            try: entries.append(json.loads(line))
            except: continue

    if not entries: sys.exit(1)

    header = None
    content_entries = []
    for e in entries:
        if e.get("type") == "session":
            header = e
        else:
            content_entries.append(e)

    if not header:
        print(f"No session header in {jsonl_file}", file=sys.stderr)
        sys.exit(1)

    message_entries = []
    tool_result_entries = []
    other_entries = []
    for e in content_entries:
        if e.get("type") == "message":
            role = e.get("message", {}).get("role", "")
            if role in ("user", "assistant"):
                message_entries.append(e)
            elif role == "toolResult":
                tool_result_entries.append(e)
            else:
                other_entries.append(e)
        else:
            other_entries.append(e)

    # Build a map from toolCallId → entry index in tool_result_entries for fast lookup
    tool_result_map = {}
    for e in tool_result_entries:
        tcid = e.get("message", {}).get("toolCallId")
        if tcid:
            tool_result_map[tcid] = e

    total_user = sum(1 for e in message_entries if e.get("message", {}).get("role") == "user")
    if total_user < keep_pairs:
        print(f"Only {total_user} user messages (need {keep_pairs}) — proceeding with shallow trim (keep all)")

    user_count = 0
    cut_index = 0  # default: keep everything (shallow trim)
    if total_user >= keep_pairs:
        cut_index = len(message_entries)
        for i in range(len(message_entries) - 1, -1, -1):
            if message_entries[i].get("message", {}).get("role") == "user":
                user_count += 1
                if user_count >= keep_pairs:
                    cut_index = i
                    break

    kept_messages = message_entries[cut_index:]
    archived_messages = message_entries[:cut_index]

    # Identify toolCallIds referenced in kept vs archived messages
    def get_tool_call_ids(msgs):
        ids = set()
        for e in msgs:
            content = e.get("message", {}).get("content", [])
            if not isinstance(content, list):
                continue
            for b in content:
                if not isinstance(b, dict):
                    continue
                if b.get("type") == "tool_use" and b.get("id"):
                    ids.add(b["id"])
                elif b.get("type") == "toolCall" and b.get("id"):
                    ids.add(b["id"])
        return ids

    # Find the most recent N assistant turns (for full tool result preservation)
    recent_tool_ids = set()
    asst_count = 0
    for e in reversed(kept_messages):
        if e.get("message", {}).get("role") == "assistant":
            asst_count += 1
            if asst_count <= keep_full_pairs:
                recent_tool_ids.update(get_tool_call_ids([e]))

    kept_tool_ids = get_tool_call_ids(kept_messages)
    archived_tool_ids = get_tool_call_ids(archived_messages)

    if len(kept_messages) == 0:
        print(f"Would keep 0 messages — aborting trim")
        sys.exit(1)

    # Note: min_archive_pairs no longer blocks trim — session always gets trimmed if oversized.
    # (The check is kept as an informational log only.)
    if len(archived_messages) < min_archive_pairs:
        print(f"Only {len(archived_messages)} messages to archive — proceeding with shallow trim")

    # Build topic summary for compaction entry
    user_topics = []
    for e in archived_messages:
        if e.get("message", {}).get("role") != "user": continue
        content = e.get("message", {}).get("content", "")
        if isinstance(content, list):
            content = " ".join(c.get("text", "") for c in content if c.get("type") == "text")
        content = content.strip()
        if content and len(content) > 5:
            if len(content) > 200: content = content[:200] + "..."
            user_topics.append(content)

    # Build trimmed transcript
    trimmed = [header]

    summary_lines = [
        "## Archived Context", "",
        f"{len(archived_messages)} older messages were archived to memory files by transcript maintenance.",
        f"Key topics: {'; '.join(t[:80] for t in user_topics[:5])}{'...' if len(user_topics) > 5 else ''}",
        "", "Recent conversation continues below.",
    ]

    first_kept_id = kept_messages[0].get("id") if kept_messages else None
    last_parent = header.get("id", random_id())

    compaction_entry = {
        "type": "compaction",
        "id": random_id(),
        "parentId": last_parent,
        "timestamp": datetime.now(tz=__import__('datetime').timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.000Z"),
        "summary": "\n".join(summary_lines),
        "firstKeptEntryId": first_kept_id,
        "tokensBefore": 0,
    }
    trimmed.append(compaction_entry)

    def strip_assistant_entry(e):
        """For non-recent assistant turns: strip thinking blocks entirely,
        strip arguments from toolCall blocks (keep id + name only)."""
        import copy
        e = copy.deepcopy(e)
        msg = e["message"]
        content = msg.get("content", [])
        if not isinstance(content, list):
            return e  # String content — nothing to strip
        new_content = []
        for b in content:
            if not isinstance(b, dict):
                new_content.append(b)
                continue
            t = b.get("type", "")
            if t == "thinking":
                continue  # Drop entirely
            elif t == "toolCall":
                new_content.append({"type": "toolCall", "id": b["id"], "name": b["name"]})
            else:
                new_content.append(b)
        msg["content"] = new_content
        return e

    prev_id = compaction_entry["id"]
    for e in kept_messages:
        e["parentId"] = prev_id
        if "id" not in e: e["id"] = random_id()
        prev_id = e["id"]
        # Strip thinking + toolCall args from older assistant turns
        if (e.get("message", {}).get("role") == "assistant" and
                not get_tool_call_ids([e]).intersection(recent_tool_ids)):
            e = strip_assistant_entry(e)
        trimmed.append(e)

    def make_turn_summary_entry(asst_entry, results_for_turn, parent_id):
        """Collapse all toolResults for one assistant turn into a single synthetic entry.
        Format: 'exec ✓(0): first line | Read ✓(0) | exec ✗(1): error text'
        """
        parts = []
        for r in results_for_turn:
            msg = r.get("message", {})
            tool_name = msg.get("toolName", "tool")
            is_error = msg.get("isError", False)
            details = msg.get("details", {})
            exit_code = details.get("exitCode", "?")
            mark = "\u2713" if not is_error and exit_code in (0, "0", None, "?") else "\u2717"
            # Get first line of output
            text = ""
            raw_content = msg.get("content", [])
            if isinstance(raw_content, str):
                text = raw_content.strip()
            else:
                for b in raw_content:
                    if isinstance(b, dict) and b.get("type") == "text":
                        text = b["text"].strip()
                        break
            first_line = text.splitlines()[0][:100] if text else ""
            if first_line:
                parts.append(f"{tool_name} {mark}({exit_code}): {first_line}")
            else:
                parts.append(f"{tool_name} {mark}({exit_code})")

        summary_text = " | ".join(parts)
        asst_id = asst_entry.get("id", random_id())
        ts = asst_entry.get("timestamp",
            datetime.now(tz=__import__('datetime').timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.000Z"))
        return {
            "type": "message",
            "id": random_id(),
            "parentId": parent_id,
            "timestamp": ts,
            "message": {
                "role": "toolResult",
                "toolCallId": f"summary-{asst_id[:8]}",
                "toolName": "[tool summary]",
                "content": [{"type": "text", "text": summary_text}],
                "details": {"summarized": True, "count": len(results_for_turn)},
            }
        }

    # Build mapping: toolCallId → toolResult entry
    tcid_to_result = {}
    for e in tool_result_entries:
        tcid = e.get("message", {}).get("toolCallId")
        if tcid:
            tcid_to_result[tcid] = e

    # For kept messages, emit toolResults per assistant turn:
    # - Recent 2 turns: individual entries (full output)
    # - Older turns: single collapsed summary entry per turn
    # - Archived turns: dropped
    for asst_entry in kept_messages:
        if asst_entry.get("message", {}).get("role") != "assistant":
            continue
        turn_ids = get_tool_call_ids([asst_entry])
        if not turn_ids:
            continue
        turn_results = [tcid_to_result[tid] for tid in turn_ids if tid in tcid_to_result]
        if not turn_results:
            continue
        asst_id = asst_entry.get("id", "")
        if any(tid in recent_tool_ids for tid in turn_ids):
            # Recent turn: keep full individual entries
            trimmed.extend(turn_results)
        else:
            # Older turn: collapse to single summary entry
            summary_entry = make_turn_summary_entry(asst_entry, turn_results, asst_entry.get("id", random_id()))
            trimmed.append(summary_entry)

    kept_ids = {e.get("id") for e in kept_messages}
    for e in other_entries:
        if e.get("parentId") and e["parentId"] in kept_ids:
            trimmed.append(e)

    # Post-trim size check: if still over threshold, first strip all assistant turns,
    # then if STILL over threshold, drop all toolResult entries entirely.
    trimmed_size_kb = sum(len(json.dumps(e)) for e in trimmed) / 1024
    threshold_kb = trim_max_kb * (trim_full_threshold_pct / 100.0)
    if trimmed_size_kb > threshold_kb:
        print(f"Post-trim size {trimmed_size_kb:.0f}KB exceeds {trim_full_threshold_pct:.0f}% of limit "
              f"({threshold_kb:.0f}KB) — stripping all assistant pairs")
        stripped_trimmed = []
        for e in trimmed:
            if e.get("type") == "message" and e.get("message", {}).get("role") == "assistant":
                stripped_trimmed.append(strip_assistant_entry(e))
            else:
                stripped_trimmed.append(e)
        trimmed = stripped_trimmed
        new_size_kb = sum(len(json.dumps(e)) for e in trimmed) / 1024
        print(f"After assistant strip: {new_size_kb:.0f}KB")

        # Still over threshold — drop ALL toolResult entries
        if new_size_kb > threshold_kb:
            print(f"Still {new_size_kb:.0f}KB over threshold — dropping all toolResult entries")
            trimmed = [
                e for e in trimmed
                if not (e.get("type") == "message" and
                        e.get("message", {}).get("role") == "toolResult")
            ]
            new_size_kb = sum(len(json.dumps(e)) for e in trimmed) / 1024
            print(f"After toolResult drop: {new_size_kb:.0f}KB")

    # Archive original, write trimmed
    archive_path = f"{jsonl_file}.pre-trim.{datetime.now(tz=__import__('datetime').timezone.utc).strftime('%Y-%m-%dT%H-%M-%SZ')}"
    shutil.move(jsonl_file, archive_path)

    with open(jsonl_file, "w") as f:
        for entry in trimmed:
            f.write(json.dumps(entry) + "\n")

    orig_kb = os.path.getsize(archive_path) // 1024
    new_kb = os.path.getsize(jsonl_file) // 1024
    print(f"Trimmed {session_id[:12]}: {orig_kb}KB → {new_kb}KB "
          f"({len(archived_messages)} archived, {len(kept_messages)} kept)")

if __name__ == "__main__":
    main()


path: scripts/watchdog.sh
content: #!/usr/bin/env bash
# session-janitor watchdog — detects hung/stale gateway sessions and alerts via Slack
# Invoked by janitor.sh or run standalone via cron.
# A "stuck" session = gateway process alive, but active session updatedAt hasn't changed in STALE_MINUTES.
set -euo pipefail

export PATH="/home/${USER:-$(whoami)}/.npm-global/bin:/usr/local/bin:/usr/bin:/bin:$PATH"

SKILL_DIR="$(cd "$(dirname "$0")/.." && pwd)"
CONFIG_FILE="$SKILL_DIR/config.json"
STALE_MINUTES="${1:-5}"   # session must be stale this long to alert
STATE_FILE="${2:-}"        # optional: suppress duplicate alerts

log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] watchdog: $*"; }

if [[ ! -f "$CONFIG_FILE" ]]; then
    log "ERROR: $CONFIG_FILE not found"
    exit 1
fi

# Load alert config
read_alert_config() {
    python3 -c "
import json, os, sys
c = json.load(open('$CONFIG_FILE'))
w = c.get('watchdog', {})
print('ALERT_SLACK=' + str(w.get('alertSlack', True)).lower())
print('SLACK_CHANNEL=' + w.get('slackChannel', ''))
print('SLACK_TARGET=' + w.get('slackTarget', ''))
print('AUTO_RESTART=' + str(w.get('autoRestart', False)).lower())
print('RESTART_SCRIPT_SLACK=' + os.path.expanduser(w.get('restartScriptSlack', '~/bin/safe-slack-restart.sh')))
print('RESTART_SCRIPT_DISCORD=' + os.path.expanduser(w.get('restartScriptDiscord', '~/bin/safe-gateway-restart.sh')))
gws = c.get('gateways', [])
print('GATEWAY_COUNT=' + str(len(gws)))
"
}
eval "$(read_alert_config)"

get_gateway() {
    local idx="$1" field="$2"
    python3 -c "
import json, os
c = json.load(open('$CONFIG_FILE'))
gw = c['gateways'][$idx]
v = gw.get('$field', '')
if isinstance(v, str) and '~' in v:
    v = os.path.expanduser(v)
print(v)
"
}

NOW=$(date +%s)
ALERTED=0

check_gateway() {
    local idx="$1"
    local name port token sessions_dir sessions_json service
    name=$(get_gateway "$idx" "name")
    port=$(get_gateway "$idx" "port")
    [REDACTED_SECRET] "$idx" "token")
    sessions_dir=$(get_gateway "$idx" "sessionsDir")
    sessions_json="$sessions_dir/sessions.json"
    service=$(get_gateway "$idx" "service")

    [[ -f "$sessions_json" ]] || return

    # Is gateway process alive?
    if [[ -n "$service" ]]; then
        systemctl --user is-active "$service" >/dev/null 2>&1 || {
            log "$name: service $service is not active — skipping"
            return
        }
    fi

    # Find stale active sessions
    local stale_sids
    stale_sids=$(python3 -c "
import json, time, sys
d = json.load(open('$sessions_json'))
now_ms = int(time.time() * 1000)
stale_ms = $STALE_MINUTES * 60 * 1000
for key, v in d.get('sessions', d).items():
    updated = v.get('updatedAt', 0)
    sid = v.get('sessionId', '')
    if not sid or not updated: continue
    age_ms = now_ms - updated
    if age_ms > stale_ms:
        age_min = age_ms // 60000
        print(f'{sid}|{key}|{age_min}')
" 2>/dev/null)

    [[ -z "$stale_sids" ]] && { log "$name: all sessions current"; return; }

    while IFS='|' read -r sid key age_min; do
        # Check if transcript file exists and is large (implies active work)
        local jsonl="$sessions_dir/$sid.jsonl"
        [[ -f "$jsonl" ]] || continue
        local size_kb=$(( $(stat -c%s "$jsonl" 2>/dev/null || echo 0) / 1024 ))
        (( size_kb < 10 )) && continue  # tiny/empty sessions, not interesting

        # Dedup: skip if we already alerted for this sid+age window (within 10 min)
        local state_key="${name}_${sid}_stuck"
        if [[ -n "$STATE_FILE" ]] && [[ -f "$STATE_FILE" ]]; then
            local last_alert
            last_alert=$(python3 -c "
import json
d = json.load(open('$STATE_FILE'))
print(d.get('watchdog_alerts', {}).get('$state_key', 0))
" 2>/dev/null || echo 0)
            local alert_age=$(( NOW - last_alert ))
            (( alert_age < 600 )) && { log "$name: $sid stuck ${age_min}min — alert suppressed (cooldown)"; continue; }
        fi

        log "$name: SESSION STUCK — $sid stale ${age_min}min, ${size_kb}KB"
        ALERTED=1

        # Send Slack alert
        if [[ "$ALERT_SLACK" == "true" ]]; then
            local msg="⚠️ *Gateway hang detected* — \`$name\` session stale *${age_min} min* (${size_kb}KB transcript)\nSession: \`$sid\`"
            if [[ -n "$SLACK_TARGET" ]]; then
                openclaw message send --channel slack --target "$SLACK_TARGET" -m "$msg" 2>/dev/null || true
            fi
        fi

        # Optionally auto-restart
        if [[ "$AUTO_RESTART" == "true" ]]; then
            log "$name: auto-restart enabled — triggering restart"
            if [[ "$name" == *"slack"* ]]; then
                bash "$RESTART_SCRIPT_SLACK" "watchdog: session stale ${age_min}min" 2>&1 | tail -5 || true
            else
                bash "$RESTART_SCRIPT_DISCORD" "watchdog: session stale ${age_min}min" 2>&1 | tail -5 || true
            fi
        fi

        # Record alert time in state file
        if [[ -n "$STATE_FILE" ]]; then
            python3 -c "
import json, time, os
path = '$STATE_FILE'
d = json.load(open(path)) if os.path.exists(path) else {}
d.setdefault('watchdog_alerts', {})['$state_key'] = int(time.time())
json.dump(d, open(path, 'w'), indent=2)
" 2>/dev/null || true
        fi

    done <<< "$stale_sids"
}

for (( i=0; i<GATEWAY_COUNT; i++ )); do
    check_gateway "$i"
done

(( ALERTED == 0 )) && log "all gateways clean"
exit 0


path: scripts/watcher.sh
content: #!/usr/bin/env bash
# session-janitor watcher — per-turn trim trigger (Linux: inotifywait, macOS: fswatch)
# Watches all gateway session dirs. On JSONL write, debounces 3s then fires trim.
set -euo pipefail

OS="$(uname -s)"

export PATH="$HOME/.npm-global/bin:/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:$PATH"
if [[ "$OS" == "Linux" ]]; then
    export XDG_RUNTIME_DIR="/run/user/$(id -u)"
    export DBUS_SESSION_BUS_ADDRESS="unix:path=${XDG_RUNTIME_DIR}/bus"
fi

SKILL_DIR="$(cd "$(dirname "$0")/.." && pwd)"
SCRIPTS_DIR="$SKILL_DIR/scripts"
CONFIG_FILE="$SKILL_DIR/config.json"

if [[ ! -f "$CONFIG_FILE" ]]; then
    echo "ERROR: $CONFIG_FILE not found" >&2
    exit 1
fi

log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [watcher] $*"; }

# Check if a session_key belongs to an active foreman worker (skip reload to avoid aborting in-flight requests)
ACTIVE_WORKERS_FILE="$HOME/repos/swabby-brain/memory/active-workers.json"
is_active_worker_session() {
    local sk="$1"
    [[ -f "$ACTIVE_WORKERS_FILE" ]] || return 1
    python3 -c "
import json, sys
sk = sys.argv[1]
try:
    d = json.load(open('$ACTIVE_WORKERS_FILE'))
    for w in d.get('workers', []):
        if w.get('session_key') == sk and w.get('status') in ('starting', 'running', 'blocked'):
            sys.exit(0)
except Exception:
    pass
sys.exit(1)
" "$sk" 2>/dev/null
}

# Load config values
read_config_val() {
    python3 -c "
import json, os
c = json.load(open('$CONFIG_FILE'))
print(c.get('$1', '$2'))
"
}

# Load extractOnTrim config (re-read each time for live config updates)
read_extract_on_trim_val() {
    python3 -c "
import json, os
c = json.load(open('$CONFIG_FILE'))
print(c.get('extractOnTrim', {}).get('$1', '$2'))
"
}

TRIM_MAX_KB=$(read_config_val trimMaxKB 250)
KEEP_PAIRS=$(read_config_val keepPairs 10)
KEEP_FULL_PAIRS=$(read_config_val keepFullPairs 2)
MIN_ARCHIVE_PAIRS=$(read_config_val minArchivePairs 5)
TRIM_FULL_THRESHOLD_PCT=$(read_config_val trimFullThresholdPct 50)
DEBOUNCE_SECS=$(read_config_val watcherDebounceSecs 3)
STATE_FILE=$(python3 -c "import json,os; c=json.load(open('$CONFIG_FILE')); print(os.path.expanduser(c.get('stateFile','~/.openclaw/session-janitor-state.json')))")
SIDECAR_ENABLED=$(python3 -c "import json; c=json.load(open('$CONFIG_FILE')); print(str(c.get('sidecar',{}).get('enabled',True)).lower())")
SIDECAR_MIN_BYTES=$(python3 -c "import json; c=json.load(open('$CONFIG_FILE')); print(c.get('sidecar',{}).get('minEntryBytes',5120))")

# Build list of watch dirs + gateway metadata
get_gateways() {
    python3 -c "
import json, os
c = json.load(open('$CONFIG_FILE'))
for gw in c.get('gateways', []):
    sd = os.path.expanduser(gw.get('sessionsDir', ''))
    name = gw.get('name', '')
    port = gw.get('port', '')
    [REDACTED_SECRET]'token', '')
    cfgdir = os.path.expanduser(gw.get('configDir', ''))
    if sd and os.path.isdir(sd):
        print(f'{name}|{sd}|{port}|{token}|{cfgdir}')
"
}

# Associative maps: dir -> gateway info
declare -A DIR_NAME DIR_PORT DIR_TOKEN DIR_CFG

while IFS='|' read -r name sd port token cfgdir; do
    DIR_NAME["$sd"]="$name"
    DIR_PORT["$sd"]="$port"
    DIR_TOKEN["$sd"]="$token"
    DIR_CFG["$sd"]="$cfgdir"
done < <(get_gateways)

WATCH_DIRS=("${!DIR_NAME[@]}")

if [[ ${#WATCH_DIRS[@]} -eq 0 ]]; then
    log "No valid session dirs found — exiting"
    exit 1
fi

log "Watching ${#WATCH_DIRS[@]} session dir(s): ${WATCH_DIRS[*]}"
log "Threshold: ${TRIM_MAX_KB}KB | Debounce: ${DEBOUNCE_SECS}s | Keep pairs: ${KEEP_PAIRS} | Keep full pairs: ${KEEP_FULL_PAIRS}"

# Pending file => timestamp of last event
declare -A PENDING

# Portable file size in KB (GNU stat -c vs BSD stat -f)
get_file_size_kb() {
    local f="$1"
    if [[ "$OS" == "Darwin" ]]; then
        echo $(( $(stat -f%z "$f" 2>/dev/null || echo 0) / 1024 ))
    else
        echo $(( $(stat -c%s "$f" 2>/dev/null || echo 0) / 1024 ))
    fi
}

fire_trim() {
    local jsonl="$1"
    local sessions_dir
    sessions_dir="$(dirname "$jsonl")"
    local name="${DIR_NAME[$sessions_dir]:-unknown}"
    local port="${DIR_PORT[$sessions_dir]:-}"
    local [REDACTED_SECRET]"
    local cfgdir="${DIR_CFG[$sessions_dir]:-}"

    # Skip non-active files
    [[ "$jsonl" == *".reset."* ]] && return
    [[ "$jsonl" == *".deleted."* ]] && return
    [[ "$jsonl" == *".pre-trim."* ]] && return
    [[ "$jsonl" == *".trajectory."* ]] && return  # OpenClaw runtime telemetry, not a transcript
    [[ ! -f "$jsonl" ]] && return

    local size_kb
    size_kb=$(get_file_size_kb "$jsonl")

    if (( size_kb <= TRIM_MAX_KB )); then
        return  # Under threshold — skip
    fi

    local sid
    sid=$(basename "$jsonl" .jsonl)
    sid="${sid%.trajectory}"  # Strip .trajectory suffix so UUID matches sessions.json

    # Verify it's an active session
    local sessions_json="$sessions_dir/sessions.json"
    if [[ -f "$sessions_json" ]]; then
        if ! python3 -c "
import json, sys
d = json.load(open('$sessions_json'))
active = [v.get('sessionId','') for v in d.get('sessions',d).values()]
sys.exit(0 if '$sid' in active else 1)
" 2>/dev/null; then
            return  # Not an active session
        fi
    fi

    log "$name: $sid is ${size_kb}KB — trimming"

    # Run sidecar offloader first (before trim so trim sees smaller file)
    if [[ "$SIDECAR_ENABLED" == "true" ]]; then
        if python3 "$SCRIPTS_DIR/sidecar.py" "$jsonl" "$sid" "$SIDECAR_MIN_BYTES" 2>&1 | while IFS= read -r line; do log "$name: [sidecar] $line"; done; then
            : # sidecar ran (even if nothing was offloaded, exit 0)
        else
            log "$name: sidecar failed for $sid (exit $?) — continuing to trim"
        fi
        # Recheck size after sidecar (it may have shrunk the file below threshold)
        size_kb=$(get_file_size_kb "$jsonl")
        if (( size_kb <= TRIM_MAX_KB )); then
            log "$name: $sid shrank to ${size_kb}KB after sidecar — skipping trim"
            # Still ping gateway so it reloads the updated transcript
            if [[ -n "$token" && -n "$port" && -n "$sessions_json" ]]; then
                local session_key
                session_key=$(python3 -c "
import json
d = json.load(open('$sessions_json'))
for k, v in d.get('sessions', d).items():
    if v.get('sessionId','') == '$sid':
        print(k)
        break
" 2>/dev/null)
                if [[ -n "$session_key" ]]; then
                    if is_active_worker_session "$session_key"; then
                        log "$name: skipping sidecar reload ping — active foreman worker session $session_key"
                    elif [[ "$session_key" == *":direct:"* ]]; then
                        curl -sS --max-time 10 "http://127.0.0.1:${port}/v1/chat/completions" \
                            -H "Authorization: Bearer $token" \
                            -H "Content-Type: application/json" \
                            -H "x-openclaw-session-key: $session_key" \
                            -d '{"model":"openclaw","messages":[{"role":"user","content":"[tool outputs sidecared — acknowledge with NO_REPLY]"}]}' \
                            >/dev/null 2>&1 && log "$name: gateway reload pinged after sidecar for $session_key" || true
                    else
                        log "$name: skipping sidecar reload ping for non-direct session $session_key"
                    fi
                fi
            fi
            return
        fi
    fi

    if python3 "$SCRIPTS_DIR/trim.py" "$jsonl" "$sid" "$name" "$STATE_FILE" "$KEEP_PAIRS" "$KEEP_FULL_PAIRS" "$MIN_ARCHIVE_PAIRS" "$TRIM_FULL_THRESHOLD_PCT" "$TRIM_MAX_KB" 2>&1; then
        log "$name: trim complete for $sid"

        # Async memory extraction from archived content (if enabled)
        local extract_enabled
        extract_enabled=$(python3 -c "import json; c=json.load(open('$CONFIG_FILE')); print(str(c.get('extractOnTrim',{}).get('enabled',False)).lower())" 2>/dev/null)
        if [[ "$extract_enabled" == "true" ]]; then
            # Find the .pre-trim.* archive written by trim.py
            local pre_trim_file
            pre_trim_file=$(ls -t "${jsonl}.pre-trim."* 2>/dev/null | head -1)
            if [[ -n "$pre_trim_file" ]]; then
                local extract_scene extract_salience extract_min
                extract_scene=$(python3 -c "import json; c=json.load(open('$CONFIG_FILE')); print(c.get('extractOnTrim',{}).get('scene','auto'))" 2>/dev/null)
                extract_salience=$(python3 -c "import json; c=json.load(open('$CONFIG_FILE')); print(c.get('extractOnTrim',{}).get('salience',0.5))" 2>/dev/null)
                extract_min=$(python3 -c "import json; c=json.load(open('$CONFIG_FILE')); print(c.get('extractOnTrim',{}).get('minArchivedPairs',3))" 2>/dev/null)
                local gw_url="http://127.0.0.1:${port}"
                log "$name: firing async extract-llm for $sid (pre-trim: $pre_trim_file)"
                python3 "$SCRIPTS_DIR/extract-llm.py" \
                    "$pre_trim_file" "$jsonl" "$sid" "$gw_url" "$STATE_FILE" \
                    "${gw_url}/v1/chat/completions" "$token" \
                    "true" "mem" "$extract_scene" "openclaw" "20000" "60" "15" "$extract_min" \
                    >> /tmp/janitor-extract.log 2>&1 &
                log "$name: extract-llm launched async (pid $!)"
            else
                log "$name: extractOnTrim enabled but no pre-trim file found for $sid"
            fi
        fi

        # Ping gateway to reload
        if [[ -n "$token" && -n "$port" ]]; then
            local session_key
            session_key=$(python3 -c "
import json
d = json.load(open('$sessions_json'))
for k, v in d.get('sessions', d).items():
    if v.get('sessionId','') == '$sid':
        print(k)
        break
" 2>/dev/null)
            if [[ -n "$session_key" ]]; then
                # Skip active foreman worker sessions — reload aborts in-flight API requests
                # Root-caused 2026-05-02: janitor reload POST killed workers mid-turn
                if is_active_worker_session "$session_key"; then
                    log "$name: skipping reload ping — active foreman worker session $session_key (trim still applied on disk)"
                elif [[ "$session_key" == *":direct:"* ]]; then
                    curl -sS --max-time 10 "http://127.0.0.1:${port}/v1/chat/completions" \
                        -H "Authorization: Bearer $token" \
                        -H "Content-Type: application/json" \
                        -H "x-openclaw-session-key: $session_key" \
                        -d '{"model":"openclaw","messages":[{"role":"user","content":"[session trimmed by maintenance — acknowledge with NO_REPLY]"}]}' \
                        >/dev/null 2>&1 && log "$name: gateway reload pinged for $session_key" || true
                else
                    log "$name: skipping reload ping for non-direct session $session_key (trim still applied on disk)"
                fi
            fi
        fi
    else
        log "$name: trim failed for $sid (exit $?)"
    fi
}

# Main loop: inotifywait feeds events, we debounce and fire
process_pending() {
    local now
    now=$(date +%s)
    for jsonl in "${!PENDING[@]}"; do
        local ts="${PENDING[$jsonl]}"
        if (( now - ts >= DEBOUNCE_SECS )); then
            unset "PENDING[$jsonl]"
            fire_trim "$jsonl" &
        fi
    done
}

# OS-specific file watcher — outputs one absolute path per line
start_watcher() {
    if [[ "$OS" == "Darwin" ]]; then
        if ! command -v fswatch &>/dev/null; then
            log "ERROR: fswatch not found — install with: brew install fswatch"
            exit 1
        fi
        # FSEvents backend: Updated covers writes, MovedTo covers atomic renames
        fswatch --event Updated --event MovedTo --latency 1 "${WATCH_DIRS[@]}"
    else
        inotifywait -m -e close_write,moved_to --format '%w%f' "${WATCH_DIRS[@]}" 2>/dev/null
    fi
}

start_watcher | while IFS= read -r filepath; do
    # Only care about .jsonl files (not .pre-trim., .reset., trajectory, etc.)
    [[ "$filepath" == *.jsonl ]] || continue
    [[ "$filepath" == *".pre-trim."* ]] && continue
    [[ "$filepath" == *".reset."* ]] && continue
    [[ "$filepath" == *".deleted."* ]] && continue
    [[ "$filepath" == *".trajectory."* ]] && continue  # OpenClaw runtime telemetry, not a transcript

    PENDING["$filepath"]=$(date +%s)

    # Check if any pending items are ready to fire
    process_pending
done


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)