Skill: fortunto2/solo-retro (version 2.1.0)

--- name: solo-retro description: Post-pipeline retrospective — parse logs, score process quality, find waste patterns, suggest skill/script patches. Use after pipeline completes or when user says "retro", "evaluate pipeline", "what went wrong", "pipeline review", "check pipeline logs". license: MIT metadata: author: fortunto2 version: "2.1.0" openclaw: emoji: "🔮" allowed-tools: Read, Grep, Bash, Glob, Write, Edit, AskUserQuestion, mcp__solograph__session_search, mcp__solograph__codegraph_explain, mcp__solograph__codegraph_query argument-hint: "[project-name]" --- # /retro This skill is self-contained — follow the phases below instead of delegating to other skills (/review, /audit, /build) or spawning Task subagents. Run all analysis directly. Post-pipeline retrospective. Parses pipeline logs, counts productive vs wasted iterations, identifies recurring failure patterns, scores the pipeline run, and suggests concrete patches to skills/scripts to prevent the same failures next time. ## When to use After a pipeline completes (or gets cancelled). This is the process quality check — `/review` checks **code quality**, `/retro` checks **pipeline process quality**. Can also be used standalone on any project — with or without pipeline logs. ## MCP Tools (use if available) - `session_search(query)` — find past pipeline runs and known issues - `codegraph_explain(project)` — understand project architecture context - `codegraph_query(query)` — query code graph for project metadata If MCP tools are not available, fall back to Glob + Grep + Read. ## Phase 1: Locate Artifacts 1. **Detect project** from `$ARGUMENTS` or CWD: - If argument provided: use it as project name - Otherwise: extract from CWD basename (e.g., `~/projects/my-app` -> `my-app`) 2. **Find pipeline state file:** `.solo/pipelines/solo-pipeline-{project}.local.md` (project-local) or `~/.solo/pipelines/solo-pipeline-{project}.local.md` (global fallback) - If it exists: pipeline is still running or wasn't cleaned up — read YAML frontmatter for `project_root:` - If not: pipeline completed — use CWD as project root 3. **Verify artifacts exist (parallel reads):** - Pipeline log: `{project_root}/.solo/pipelines/pipeline.log` - Iter logs: `{project_root}/.solo/pipelines/iter-*.log` - Progress file: `{project_root}/.solo/pipelines/progress.md` - Plan-done directory: `{project_root}/docs/plan-done/` - Active plan: `{project_root}/docs/plan/` 4. **Determine analysis mode:** - If pipeline log exists: proceed with full log-based analysis (Phases 2-4) - If NO pipeline log: switch to **fallback mode** (see Fallback Analysis below) 5. **Count iter logs** (if they exist): `ls {project_root}/.solo/pipelines/iter-*.log | wc -l` - Report: "Found {N} iteration logs" ## Fallback Analysis (No Pipeline Logs) If no pipeline logs exist, the retro can still provide value by analyzing: 1. **Git history:** `git log --oneline --since="1 week ago"` — commit frequency, patterns, conventional format 2. **Test results:** run test suite if configured in CLAUDE.md or package.json 3. **Build status:** run build if configured 4. **CLAUDE.md changes:** `git log --oneline -- CLAUDE.md` — how docs evolved 5. **Code quality metrics:** file counts, TODO/FIXME density, dead code indicators 6. **Project structure:** completeness of docs/, tests/, CI config Skip Phases 2-4 and proceed directly to Phase 5 (Plan Fidelity) and Phase 6 (Git & Code Quality). Adjust Phase 7 scoring to weight available data more heavily. ## Phase 2: Parse Pipeline Log (quantitative) Read `pipeline.log` in full. Parse line-by-line, extracting structured data from log tags: **Log format:** `[HH:MM:SS] TAG | message` **Extract by tag:** | Tag | What to extract | |-----|----------------| | `START` | Pipeline run boundary — count restarts (multiple START lines = restarts) | | `STAGE` | `iter N/M \| stage S/T: {stage_id}` — iteration count per stage | | `SIGNAL` | `<solo:done/>` or `<solo:redo/>` — which stages got completion signals | | `INVOKE` | Skill invoked — extract skill name, check for wrong names | | `ITER` | `commit: {sha} \| result: {stage complete\|continuing}` — per-iteration outcome | | `CHECK` | `{stage} \| {path} -> FOUND\|NOT FOUND` — marker file checks | | `FINISH` | `Duration: {N}m` — total duration per run | | `MAXITER` | `Reached max iterations ({N})` — hit iteration ceiling | | `QUEUE` | Plan cycling events (activating, archiving) | | `CIRCUIT` | Circuit breaker triggered (if present) | | `CWD` | Working directory changes | | `CTRL` | Control signals (pause/stop/skip) | **Compute metrics:** ``` total_runs = count of START lines total_iterations = count of ITER lines productive_iters = count of ITER lines with "stage complete" wasted_iters = total_iterations - productive_iters waste_pct = wasted_iters / total_iterations * 100 maxiter_hits = count of MAXITER lines plan_cycles = count of QUEUE lines with "Cycling" per_stage = { stage_id: { attempts: count of STAGE lines for this stage, successes: count of ITER lines with "stage complete" for this stage, waste_ratio: (attempts - successes) / attempts * 100, } } ``` ## Phase 3: Parse Progress.md (qualitative) Read `progress.md` and scan for error patterns: 1. **Unknown skill errors:** grep for `Unknown skill:` — extract which skill name was wrong 2. **Empty iterations:** iterations where "Last 5 lines" show only errors or session header (no actual work done) 3. **Repeated errors:** same error appearing in consecutive iterations -> spin-loop indicator 4. **Doubled signals:** `<solo:done/><solo:done/>` in same iteration -> minor noise (note but don't penalize) 5. **Redo loops:** count how many times build->review->redo->build cycles occurred For each error pattern found, record: - Pattern name - First occurrence (iteration number) - Total occurrences - Consecutive streak (max) ## Phase 4: Analyze Iter Logs (sample-based) Do NOT read all iter logs — could be 60+. Use smart sampling: 1. **First failed iter per pattern:** For each failure pattern found in Phase 3, read the first iter log that shows it - Strip ANSI codes when reading: `sed 's/\x1b\[[0-9;]*m//g' < iter-NNN-stage.log | head -100` 2. **First successful iter per stage:** For each stage that eventually succeeded, read the first successful iter log - Look for `<solo:done/>` in the output 3. **Final review iter:** Read the last `iter-*-review.log` (the verdict) 4. **Extract from each sampled log:** - Tools called (count of tool_use blocks) - Errors encountered (grep for `Error`, `error`, `Unknown`, `failed`) - Signal output (`<solo:done/>` or `<solo:redo/>` present?) - First 5 and last 10 meaningful lines (skip blank lines) ## Phase 5: Plan Fidelity Check For each track directory in `docs/plan-done/` and `docs/plan/`: 1. **Read spec.md** (if exists): - Count acceptance criteria: total `- [ ]` and `- [x]` checkboxes - Calculate: `criteria_met = checked / total * 100` 2. **Read plan.md** (if exists): - Count tasks: total `- [ ]` and `- [x]` checkboxes - Count phases (## headers) - Check for SHA annotations (`<!-- sha:... -->`) - Calculate: `tasks_done = checked / total * 100` 3. **Compile per-track summary:** - Track ID, criteria met %, tasks done %, has SHAs ## Phase 6: Git & Code Quality (lightweight) Quick checks only — NOT a full /review: 1. **Commit count and format:** ```bash git -C {project_root} log --oneline | wc -l git -C {project_root} log --oneline | head -30 ``` - Count commits with conventional format (`feat:`, `fix:`, `chore:`, `test:`, `docs:`, `refactor:`, `build:`, `ci:`, `perf:`) - Calculate: `conventional_pct = conventional / total * 100` 2. **Committer breakdown:** ```bash git -C {project_root} shortlog -sn --no-merges | head -10 ``` 3. **Test status** (if test command exists in CLAUDE.md or package.json): - Run test suite, capture pass/fail count - If no test command found, skip and note "no tests configured" 4. **Build status** (if build command exists): - Run build, capture success/fail - If no build command found, skip and note "no build configured" ## Phase 7: Score & Report Load scoring rubric from `${CLAUDE_PLUGIN_ROOT}/skills/retro/references/eval-dimensions.md`. If plugin root not available, use the embedded weights: **Scoring weights:** - Efficiency (waste %): 25% - Stability (restarts): 20% - Fidelity (criteria met): 20% - Quality (test pass rate): 15% - Commits (conventional %): 5% - Docs (plan staleness): 5% - Signals (clean signals): 5% - Speed (total duration): 5% **Note:** In fallback mode (no pipeline logs), redistribute Efficiency and Stability weights to Fidelity, Quality, and Commits. **Generate report** at `{project_root}/docs/retro/{date}-retro.md`: ```markdown # Pipeline Retro: {project} ({date}) ## Overall Score: {N}/10 ## Pipeline Efficiency | Metric | Value | Rating | |--------|-------|--------| | Total iterations | {N} | | | Productive iterations | {N} ({pct}%) | {emoji} | | Wasted iterations | {N} ({pct}%) | {emoji} | | Pipeline restarts | {N} | {emoji} | | Max-iter hits | {N} | {emoji} | | Total duration | {time} | {emoji} | ## Per-Stage Breakdown | Stage | Attempts | Successes | Waste % | Notes | |-------|----------|-----------|---------|-------| | scaffold | | | | | | setup | | | | | | plan | | | | | | build | | | | | | deploy | | | | | | review | | | | | ## Failure Patterns ### Pattern 1: {name} - **Occurrences:** {N} iterations - **Root cause:** {analysis} - **Wasted:** {N} iterations - **Fix:** {concrete suggestion with file reference} ### Pattern 2: ... ## Plan Fidelity | Track | Criteria Met | Tasks Done | SHAs | Rating | |-------|-------------|------------|------|--------| | {track-id} | {N}% | {N}% | {yes/no} | {emoji} | ## Code Quality (Quick) - **Tests:** {N} pass, {N} fail (or "not configured") - **Build:** PASS / FAIL (or "not configured") - **Commits:** {N} total, {pct}% conventional format ## Three-Axis Growth | Axis | Score | Evidence | |------|-------|----------| | **Technical** (code, tools, architecture) | {0-10} | {what changed} | | **Cognitive** (understanding, strategy, decisions) | {0-10} | {what improved} | | **Process** (harness, skills, pipeline, docs) | {0-10} | {what evolved} | If only one axis is served — note what's missing. ## Recommendations 1. **[CRITICAL]** {patch suggestion with file:line reference} 2. **[HIGH]** {improvement} 3. **[MEDIUM]** {optimization} 4. **[LOW]** {nice-to-have} ## Suggested Patches ### Patch 1: {file} — {description} **What:** {one-line description} **Why:** {root cause reference from Failure Patterns} \```diff - old line + new line \``` ``` **Rating guide (use these emojis):** - GREEN = excellent - YELLOW = acceptable - RED = needs attention ## Phase 8: Interactive Patching After generating the report: 1. **Show summary** to user: overall score, top 3 failure patterns, top 3 recommendations 2. **For each suggested patch** (if any), use `AskUserQuestion`: - Question: "Apply patch to {file}? {one-line description}" - Options: "Apply" / "Skip" / "Show diff first" 3. **If "Show diff first":** display the full diff, then ask again (Apply / Skip) 4. **If "Apply":** use Edit tool to apply the change directly 5. **After all patches processed:** - If any patches were applied: suggest committing with `fix(retro): {description}` - Do NOT auto-commit — just suggest the command ## Phase 9: CLAUDE.md Revision After patching, revise the project's CLAUDE.md to keep it lean and useful for future agents. ### Steps: 1. **Read CLAUDE.md** and check size: `wc -c CLAUDE.md` 2. **Add learnings from this retro:** - Pipeline failure patterns worth remembering (avoid next time) - New workflow rules or process improvements - Updated commands or tooling changes - Architecture decisions that emerged during the pipeline run 3. **If over 40,000 characters — trim ruthlessly:** - Collapse completed phase/milestone histories into one line each - Remove verbose explanations — keep terse, actionable notes - Remove duplicate info (same thing explained in multiple sections) - Remove historical migration notes, old debugging context - Remove examples that are obvious from code or covered by skill/doc files - Remove outdated troubleshooting for resolved issues 4. **Verify result <= 40,000 characters** — if still over, cut least actionable content 5. **Write updated CLAUDE.md**, update "Last updated" date ### Priority (keep -> cut): 1. **ALWAYS KEEP:** Tech stack, directory structure, Do/Don't rules, common commands, architecture decisions 2. **KEEP:** Workflow instructions, troubleshooting for active issues, key file references 3. **CONDENSE:** Phase histories (one line each), detailed examples, tool/MCP listings 4. **CUT FIRST:** Historical notes, verbose explanations, duplicated content, resolved issues ### Rules: - Never remove Do/Don't sections — critical guardrails - Preserve overall section structure and ordering - Every line must earn its place: "would a future agent need this to do their job?" - Commit the update: `git add CLAUDE.md && git commit -m "docs: revise CLAUDE.md (post-retro)"` ## Phase 10: Factory Critic (optional) **Run this phase only if `${CLAUDE_PLUGIN_ROOT}` is available** (i.e., solo-factory is installed). Skip if running as a standalone skill without the factory context. After evaluating the project pipeline, step back and evaluate **the factory itself** — the skills, scripts, and pipeline logic that produced this result. Be a harsh critic. ### What to evaluate: 1. **Read the skills that were invoked** in this pipeline run (from INVOKE lines in pipeline.log): - For each skill: `${CLAUDE_PLUGIN_ROOT}/skills/{stage}/SKILL.md` - Did the skill have the right instructions for this project's needs? - Did it miss context it should have had? 2. **Read pipeline script** signal handling and stage logic: - `${CLAUDE_PLUGIN_ROOT}/scripts/solo-dev.sh` - Were there structural issues (wrong stage order, missing re-exec, broken redo)? 3. **Cross-reference with failure patterns** from Phase 3: - For each failure: was the root cause in the skill, the script, or the project? - Skills that caused waste = factory defects ### Score the factory (not the project): ``` Factory Score: {N}/10 Skill quality: - {skill}: {score}/10 — {why} - {skill}: {score}/10 — {why} Pipeline reliability: {N}/10 — {why} Missing capabilities: - {what the factory couldn't do that it should have} Top factory defects: 1. {defect} → {which file to fix} → {concrete fix} 2. {defect} → {which file to fix} → {concrete fix} ``` ### Harness Evolution — think about the bigger picture After scoring the factory, step back further and think about the **harness** — the entire system that guides agents (CLAUDE.md, docs/, linters, skills, templates). Ask: 1. **Context engineering:** Did the agent have everything it needed in-repo? Or did it struggle because knowledge was missing / scattered / stale? - Missing docs -> add to `docs/` or CLAUDE.md - Stale docs -> flag for doc-gardening - Knowledge only in your head -> encode it 2. **Architectural constraints:** Did the agent break module boundaries, produce inconsistent patterns, or ignore conventions? - Repeated boundary violations -> need a linter or structural test - Inconsistent patterns -> need golden principle in CLAUDE.md - Data shape errors -> need parse-at-boundary enforcement 3. **Decision traces:** What worked well that future agents should reuse? What failed that they should avoid? - Good patterns -> capture as precedent in docs or CLAUDE.md - Bad patterns -> encode as anti-pattern or lint rule - Think: "if another agent hits this same problem tomorrow, what should it find?" 4. **Skill gaps:** Which skills need better instructions? Which new skills should exist? - Skill that caused waste -> concrete SKILL.md patch - Missing capability -> new skill idea for evolution log ### Write to evolution log: Append findings to `{project_root}/docs/evolution.md` (create if not exists). If `~/.solo/evolution.md` exists, append there as well for cross-project tracking. ```markdown ## {YYYY-MM-DD} | {project} | Factory Score: {N}/10 Pipeline: {stages run} | Iters: {total} | Waste: {pct}% ### Defects - **{severity}** | {skill/script}: {description} - Fix: {concrete file:change} ### Harness Gaps - **Context:** {what knowledge was missing or stale for the agent} - **Constraints:** {what boundary violations or inconsistencies occurred} - **Precedents:** {patterns worth capturing for future agents — good or bad} ### Missing - {capability the factory lacked} ### What worked well - {skill/pattern that performed efficiently} ``` **Rules:** - Be brutally honest — if a skill is broken, say so - Every defect must have a concrete fix (file + what to change) - Track what works well too — don't regress good patterns - Keep entries compact — this file accumulates over time ## Signal Output **Output signal:** `<solo:done/>` **Important:** `/retro` always outputs `<solo:done/>` — it never needs redo. Even if pipeline was terrible, the retro itself always completes. ## Edge Cases - **No pipeline.log and no git history:** abort with clear message — "No pipeline log or git history found. Nothing to analyze." - **No pipeline.log but git history exists:** switch to fallback mode (see Fallback Analysis) - **Empty pipeline.log:** report "Pipeline log is empty — was the pipeline cancelled before any iteration?" - **No iter logs:** skip Phase 4 sampling, note in report - **No plan-done:** skip Phase 5, note "No completed plans found" - **No test/build commands:** skip those checks in Phase 6, note in report - **Pipeline still running:** warn user — "State file exists, pipeline may still be running. Retro on partial data." ## Reference Files - `${CLAUDE_PLUGIN_ROOT}/skills/retro/references/eval-dimensions.md` — scoring rubric (8 axes, weights) - `${CLAUDE_PLUGIN_ROOT}/skills/retro/references/failure-catalog.md` — known failure patterns and fixes


Security scan verdicts — ClawScan: suspicious | static analysis:  | VirusTotal: suspicious | SkillSpector: suspicious (CRITICAL)