Skill: donovanpankratz-del/agent-observability (version 1.0.0)

--- name: agent-observability description: Full observability stack for OpenClaw agents. Installs four tools: (1) weekly throughput dashboard (tasks/cost/quality), (2) decision audit log (why decisions were made), (3) failure trace capture (what went wrong when subagents fail), (4) drift-guard auto-scoring (weekly INTENT.md compliance check). Use when you want visibility into agent behavior, when debugging subagent failures, or when setting up production monitoring. Works standalone or alongside intent-engineering skill. --- # Agent Observability ## What Gets Installed | File | Purpose | Location | |---|---|---| | `throughput-dashboard.js` | Weekly productivity metrics | `scripts/` | | `decision-audit.js` | Append-only decision log with reasoning | `lib/` | | `failure-tracer.js` | Captures traces when quality score < 7 | `lib/` | | `drift-guard-auto.js` | Weekly INTENT.md compliance scoring | `scripts/` | ## Installation ### Step 1 — Copy files ```bash WORKSPACE="${OPENCLAW_WORKSPACE:-$(pwd)}" cp references/throughput-dashboard.js "$WORKSPACE/scripts/" cp references/decision-audit.js "$WORKSPACE/lib/" cp references/failure-tracer.js "$WORKSPACE/lib/" cp references/drift-guard-auto.js "$WORKSPACE/scripts/" ``` Or manually copy each file from the `references/` directory in this skill. ### Step 2 — Add to heartbeat/cron (weekly) In your heartbeat or weekly cron script: ```bash node "$WORKSPACE/scripts/throughput-dashboard.js" "$WORKSPACE" node "$WORKSPACE/scripts/drift-guard-auto.js" "$WORKSPACE" ``` ### Step 3 — Wire decision-audit into high-stakes decisions ```javascript const { logDecision } = require('./lib/decision-audit'); logDecision({ task_type: 'code_generation', decision: 'spawn CoderAgent', reasoning_summary: 'Multi-file edit blocks chat >5s', session_channel: 'discord' // optional }, workspaceRoot); ``` ### Step 4 — Wire failure-tracer into quality validation (optional) The failure-tracer fires automatically when you call it after scoring subagent output: ```javascript const { captureFailureTrace } = require('./lib/failure-tracer'); // Call after scoring any subagent output if (qualityScore < 7) { captureFailureTrace('AgentLabel-task', qualityScore, agentOutput, workspaceRoot); } ``` ## Reading the Data | Path | Contents | |---|---| | `memory/dashboards/YYYY-MM-DD.md` | Weekly throughput snapshot | | `memory/drift-reports/YYYY-MM-DD.md` | Drift compliance report | | `memory/decisions-audit.jsonl` | Full decision log (JSONL) | | `memory/traces/[label]-[timestamp].json` | Failure traces | ### Query examples ```bash # Recent decisions tail -20 memory/decisions-audit.jsonl | jq . # All failure traces ls memory/traces/ # Latest drift report cat memory/drift-reports/$(ls memory/drift-reports/ | tail -1) ``` ## Tool Descriptions ### throughput-dashboard.js Aggregates weekly metrics: tasks routed, subagents spawned, estimated cost, quality ratio, routing distribution. Reads from `session-metrics.js` (if installed) and `drift-guard-auto.js`. Degrades gracefully if data sources are missing — every section is independent. ### decision-audit.js Append-only JSONL log at `memory/decisions-audit.jsonl`. Each entry: `{ id, ts, task_type, decision, reasoning_summary, outcome, session_channel }`. Use `updateOutcome(id, 'success', workspaceRoot)` to close the loop after a decision resolves. ### failure-tracer.js Fires when quality score < 7. Writes structured JSON to `memory/traces/`. Each trace includes: tool call sequence hints, output snippet, inferred failure reason. Use to post-mortem why a subagent underperformed. ### drift-guard-auto.js Scores recent agent outputs against behavioral rules (sycophancy, social cushioning, unprompted explanations, hallucination hedges). Reads INTENT.md for custom criteria if installed. Writes weekly report to `memory/drift-reports/`. ## References - `references/throughput-dashboard.js` — Full script implementation - `references/decision-audit.js` — Full lib implementation - `references/failure-tracer.js` — Full lib implementation - `references/drift-guard-auto.js` — Full script implementation


Security scan verdicts — ClawScan: clean | static analysis:  | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)