Skill: brasco05/deep-debugging (version 2.2.0)

--- name: deep-debugging description: Evidence-first debugging and incident triage for unclear, recurring, production-like, or high-risk software bugs. Use when the user asks for root cause analysis, says a fix did not work, reports 401/403/500 with context, deploy/runtime failures, broken integrations, or needs investigation before code changes. Do not use for obvious typos, missing installs, or trivial one-line fixes. version: "2.2.0" requiredEnv: [] permissions: - read:logs - read:env-keys - run:bash-readonly security: restriction: read-only-by-default note: > This skill may inspect logs, commit metadata, dependency versions, endpoint status, and environment key names. Never print secrets, tokens, cookies, customer data, or raw production payloads. Any production write, deploy, rollback, database action, or external API change requires explicit user approval. --- # Deep Debugging **No guessing. No random fixes. Stabilize incidents first, then prove the root cause.** ## When to use Use this skill for: - unclear or recurring bugs - user-facing/prod-like failures: `500`, broken login, failed deploy, red healthcheck - auth/session bugs: `401`, `403`, cookie/JWT weirdness - integrations/webhooks/rate limits/signature failures - “still broken”, “same error”, “find root cause”, “debug this properly” Do **not** use it for obvious compiler errors, typos, missing install/setup steps, or cosmetic UI tweaks. ## Operating contract 1. **Observe before editing.** No code/config changes before evidence + hypothesis. 2. **One hypothesis at a time.** If you cannot state the proof, you do not know the cause. 3. **Binary search the chain.** Split request → app → service → DB/API → response. 4. **Minimal reversible fix.** No drive-by refactors. 5. **Verify and prevent.** Test the exact failure path and document recurrence prevention. ## Workflow ```text 0. Incident Gate → user/prod impact? stabilize first 1. Quick Triage → obvious setup/runtime misses 2. Evidence → exact error, repro, affected path, last change 3. Hypothesis → one testable cause + one test 4. Narrow → binary-search the failure chain 5. Fix → smallest reversible change 6. Verify → exact repro/test/build/log evidence 7. Prevent → regression/monitoring/learning when recurring or prod-like ``` ## Phase 0 — Incident Gate If users, production, money flows, auth, data integrity, or external integrations are affected, switch to incident mode before debugging. Output first: ```text INCIDENT SNAPSHOT Impact: [who/what is affected] Severity: [low/medium/high/critical + why] Started: [time/commit/deploy if known] Evidence: [logs/status/metrics; redacted] Stabilize: [rollback, feature flag, pause job, monitor, or no-op] Next step: [one concrete diagnostic action] ``` Rules: - High/critical production incidents: check rollback/feature flag before hotfix. - Preserve evidence before redeploy/restart when possible. - Production writes, rollbacks, migrations, credential changes, or third-party changes require explicit user approval. For detailed incident checklists read `references/incident-first.md`. ## Phase 1 — Quick Triage Check these before deeper analysis: ```text □ Server/process restarted after config/code change? □ Correct env file/keys present? Key names only, never values. □ Dependencies installed/generated after package/schema changes? □ Migration/schema state matches runtime? □ Browser/client cache or stale build ruled out? □ Repro uses test data, not live credentials/customer data? ``` If a quick triage item explains the issue, fix that minimally and still verify. ## Phase 2 — Evidence Collect real proof: ```text Error: exact message/status/stack excerpt Path: endpoint/function/job/component Repro: minimal steps or request shape Scope: all users vs specific role/input/tenant/environment Expected: what should happen Actual: what happens Last change: commit/deploy/config/schema/provider change ``` Optional helper: run `scripts/incident_snapshot.sh` locally to collect safe environment metadata. It prints env **key names only**, not values. ## Phase 3 — Hypothesis State exactly one hypothesis before touching code: ```text HYPOTHESIS: The failure happens because [specific cause], which I will prove/disprove by [specific test]. ``` Bad: “Something is wrong with auth.” Good: “The 401 happens because the login token is set but not sent on `/me`, which I will prove by comparing the login response headers with the follow-up request headers.” ## Phase 4 — Narrow with binary search Pick the chain and split it: ```text Frontend → request creation → network → API gateway/middleware → controller → service → DB/external API → response → UI ``` After each test report: ```text ✅ Ruled out: [component] because [evidence] ❌ Found: [component] fails because [evidence] ``` For stack-specific checklists read `references/stack-checklists.md`. ## Phase 5 — Fix Only after evidence supports the hypothesis: - change the smallest surface area - do not refactor unrelated code - prefer reversible config/code changes - keep secrets and production data out of logs/reports - stop after 3 failed fix attempts and restart from evidence/hypothesis ## Phase 6 — Verify Before saying done, provide evidence: ```text DEBUG REPORT Failure: [exact issue] Root cause: [specific cause] Proof: [test/log/code evidence] Fix: [minimal change] Verified: [command/test/repro result] Prevention: [test/monitoring/doc/learning, or "not needed" + why] Remaining: [risk/blocker, or "none known"] ``` For report variants read `references/output-templates.md`. ## Phase 7 — Prevent recurrence Required when the bug is production-like, recurring, security-adjacent, or took more than one hypothesis: ```text □ Regression test or smoke test added/identified □ Monitoring/logging improved or gap named □ Runbook/rollback note captured for future incidents □ Durable learning written if likely to recur ``` ## Hard rules - Never fix before hypothesis. - Never test with real credentials or copy raw tokens/cookies/secrets. - Never mix unrelated refactor with bug fix. - Never claim root cause without proof. - Never hide remaining risk. - If meaningful optimization potential remains, mention it as `Next optimization:`.

path: README.md
content: # 🕵️ Deep Debugging

**Stop blind fixes. Stabilize incidents first. Prove root cause before changing code.**

Deep Debugging is an evidence-first protocol for AI coding agents and developers. It prevents random patch loops by forcing a clear sequence: impact → evidence → hypothesis → binary search → minimal fix → verification → prevention.

## v2.2.0 — 9.5 polish

- Shorter core `SKILL.md` for lower context cost.
- Progressive references for incident, stack-specific, and report templates.
- Safe `scripts/incident_snapshot.sh` helper that prints metadata and env **key names only**, never secrets.
- More stack-neutral workflow while keeping optional NestJS/Next.js/Prisma/Auth/Webhook checklists.
- Explicit `Next optimization:` rule so remaining improvement potential is not hidden.

## Use it for

- unclear or recurring bugs
- failed deploys, red healthchecks, production-like failures
- `401`, `403`, `500`, broken login/session flows
- external API/webhook failures
- bugs where previous fixes did not hold

## Do not use it for

- obvious typos/compiler-pointed syntax errors
- missing install/setup steps
- trivial one-line route/config fixes
- cosmetic UI tweaks

## Core workflow

```text
Incident Gate → Quick Triage → Evidence → Hypothesis → Binary Search → Minimal Fix → Verification → Prevention
```

**Rule:** if you cannot state the proof, you do not know the cause yet. Never copy real tokens, cookies, credentials, customer data, or production secrets into reports.

---

*by brasco05 · evidence-first debugging for OpenClaw/ClawHub agents*


path: package.json
content: {
  "name": "deep-debugging",
  "version": "2.2.0",
  "description": "Evidence-first debugging and incident triage for AI coding agents with concise core workflow, progressive checklists, and safe local snapshots.",
  "keywords": [
    "debugging",
    "incident-response",
    "production-debugging",
    "root-cause-analysis",
    "hypothesis-driven",
    "binary-search",
    "rollback",
    "nestjs",
    "nextjs",
    "react",
    "jwt",
    "prisma",
    "webhooks",
    "agent-safety"
  ],
  "author": "brasco05",
  "license": "MIT"
}


path: references/incident-first.md
content: # Incident-First Debugging Reference

Use this when the failure is production-like, user-facing, money/auth/data related, or caused by a deploy/config/provider change.

## Severity guide

| Severity | Use when | Default posture |
|---|---|---|
| low | local/staging, easy workaround | debug normally |
| medium | limited users/flow affected | minimal fix, monitor |
| high | major user flow broken, auth broken, repeated 500s | stabilize first, approval for risky actions |
| critical | data loss/security/payment outage/broad outage | rollback/disable first, preserve evidence |

## 5-minute incident triage

```text
□ Environment: prod, staging, local?
□ Impact: users, tenants, money, auth, data, integrations?
□ Start: exact time, commit, deploy, config, migration, provider event?
□ Current state: error rate, healthcheck, logs, queue backlog?
□ Safety: data loss, auth leak, credential exposure, duplicate side effects?
□ Rollback path: last known good version/config? feature flag? pause job?
□ Evidence preserved: request ID, deploy ID, commit SHA, redacted log snippet?
```

## Stabilization choices

Prefer the least risky action that reduces harm:

1. **No-op + monitor** — if impact is low and evidence collection matters more.
2. **Feature flag / disable job** — when one feature/job causes damage.
3. **Rollback** — when a recent deploy/config is strongly correlated.
4. **Hotfix** — only when root cause is proven or rollback is worse.
5. **Manual repair** — database/data fixes require explicit approval and backup plan.

## Deploy/runtime failure checklist

```text
□ Build passed but runtime crashes? inspect start logs.
□ Healthcheck path/port/protocol changed?
□ Required env key missing or renamed? print key names only.
□ Runtime version changed? Node/Python/package manager mismatch?
□ Migration required before code? migration backward-compatible?
□ Container/host lacks file, permission, binary, or native dependency?
□ Recent dependency update changed behavior?
□ Last green deploy exists and can be restored?
```

## Live HTTP 500 checklist

```text
□ Exact endpoint/job/component and request ID captured?
□ Does stack trace point to app code, DB, network, or provider?
□ All users or specific role/input/tenant?
□ Any correlation with last deploy/config/migration/provider event?
□ Is the error deterministic with safe test data?
□ Are retries causing duplicate writes or queue explosions?
□ Is there a safe fallback/disable path?
```

## Auth/session outage checklist

```text
□ 401 vs 403 separated?
□ Login creates token/session/cookie? Do not print token.
□ Follow-up request sends token/cookie? inspect presence/header names only.
□ Signing/validation config uses same key source? never print values.
□ Cookie SameSite/Secure/domain/path changed?
□ CORS/credentials mode changed?
□ Clock skew/token expiry changed?
□ Role/tenant guard changed independently from auth?
```

## Webhook/API integration checklist

```text
□ Provider status page checked?
□ Rate limit/quota/exhaustion visible?
□ Signature verification failing after secret rotation? key version only.
□ Payload schema changed? store redacted shape, not raw data.
□ Retry policy and idempotency key behavior safe?
□ First failing event compared with last successful event?
□ Network/DNS/TLS/proxy changed?
□ Backfill/replay needed? requires explicit approval.
```

## Incident closeout

```text
Root cause:
Detection gap:
Prevention:
Rollback/runbook note:
Remaining risk:
```


path: references/output-templates.md
content: # Output Templates

## Standard debug report

```text
DEBUG REPORT
Failure:      [exact issue]
Root cause:   [specific cause]
Proof:        [test/log/code evidence]
Fix:          [minimal change]
Verified:     [command/test/repro result]
Prevention:   [test/monitoring/doc/learning, or "not needed" + why]
Remaining:    [risk/blocker, or "none known"]
Next optimization: [optional meaningful improvement potential]
```

## Incident snapshot

```text
INCIDENT SNAPSHOT
Impact:       [who/what is affected]
Severity:     [low/medium/high/critical + why]
Started:      [time/commit/deploy if known]
Evidence:     [logs/status/metrics; redacted]
Hypothesis:   [testable cause]
Stabilize:    [rollback, feature flag, hotfix, monitor, no-op]
Next action:  [one concrete diagnostic action]
```

## Hypothesis update

```text
HYPOTHESIS
Current:      [specific suspected cause]
Test:         [how to prove/disprove]
Result:       [passed/failed + evidence]
Decision:     [narrow further / fix / new hypothesis]
```

## After 3 failed fixes

```text
ESCALATION REPORT
What failed:      [3 attempted fixes]
Why insufficient: [evidence each did not address root cause]
Known facts:      [confirmed evidence]
Unknowns:         [missing data]
Recommended next: [one safer diagnostic path]
```

## User-facing concise summary

```text
Gefunden: [root cause in one sentence]
Gefixt: [minimal change]
Beweis: [test/log result]
Offen: [remaining risk or none]
```


path: references/stack-checklists.md
content: # Stack Checklists

Load only the relevant checklist for the current hypothesis.

## Generic API chain

```text
Client action → request construction → network → proxy/CORS → middleware → auth/guard → controller/route → service/domain → DB/cache/external API → response serialization → client handling
```

Split the chain and prove which half contains the failure.

## Auth / JWT / session

```text
□ Login endpoint returns expected status?
□ Token/session/cookie created? presence only, no raw token.
□ Client stores/sends credential correctly? header/cookie name only.
□ Server extracts credential from expected location?
□ Validation config matches signing config? key source names only.
□ Expiry/clock/skew/audience/issuer mismatch?
□ Role/permission/tenant guard failing after authentication?
□ Browser cookie rules: SameSite, Secure, domain, path, credentials mode?
```

Common root causes:
- token generated but not sent on follow-up request
- signing and validation read different env keys
- cross-site cookie blocked
- `401` auth failure confused with `403` authorization failure
- tenant/role guard throws after valid auth

## Frontend / React / Next.js

```text
□ Event handler runs?
□ State update happens and component re-renders as expected?
□ useEffect dependencies correct? no stale closure/infinite loop?
□ Network request payload correct?
□ Response handled correctly, including error path?
□ SSR/client boundary issue? window/localStorage/date/random mismatch?
□ Hooks called before any early return?
□ Cache/revalidation/stale data involved?
```

## Database / ORM

```text
□ Runtime schema matches migrations/generated client?
□ Query returns null/empty unexpectedly?
□ Relation/include/join missing?
□ Transaction boundary correct?
□ Unique constraint or FK error hidden by generic 500?
□ Connection pool exhausted or leaked?
□ Slow query has index? use EXPLAIN/ANALYZE safely.
□ Migration is backward-compatible with deployed code?
```

## External API / webhook

```text
□ Provider request built as docs require?
□ Auth header/key present by name only?
□ Status code and response body category captured redacted?
□ Rate limit, quota, or plan restriction?
□ Pagination/filter/timezone issue?
□ Signature verification and timestamp tolerance?
□ Retry/idempotency behavior prevents duplicates?
□ Provider SDK version changed?
```

## Deploy / CI / runtime

```text
□ CI failure is install, lint/typecheck, test, build, or deploy step?
□ Lockfile/package manager mismatch?
□ Node/Python/runtime version mismatch?
□ Env key present in target environment? names only.
□ Build-time vs runtime env confusion?
□ Port/host/healthcheck route mismatch?
□ Native dependency or binary unavailable in target image?
□ Cache artifact stale?
```

## Performance / memory

```text
□ Repro has baseline timing/memory/CPU evidence?
□ N+1 queries or looped network calls?
□ Missing DB index or inefficient query plan?
□ Large bundle or expensive hydration?
□ Unbounded queue, recursion, pagination, or stream buffering?
□ Event listener/interval/subscription cleanup missing?
□ Cache miss storm or retry storm?
```


path: scripts/incident_snapshot.sh
content: #!/usr/bin/env bash
set -euo pipefail

# Safe local diagnostic snapshot for debugging/incident triage.
# It prints metadata and environment KEY NAMES only. It must never print env values.

redact_path() {
  sed -E "s#${HOME}#~#g"
}

section() {
  printf '\n## %s\n' "$1"
}

section "time"
date -u '+utc=%Y-%m-%dT%H:%M:%SZ'
date '+local=%Y-%m-%dT%H:%M:%S%z' 2>/dev/null || true

section "system"
printf 'pwd=%s\n' "$(pwd | redact_path)"
printf 'kernel=%s\n' "$(uname -srm 2>/dev/null || true)"
command -v node >/dev/null 2>&1 && printf 'node=%s\n' "$(node -v)"
command -v npm >/dev/null 2>&1 && printf 'npm=%s\n' "$(npm -v)"
command -v pnpm >/dev/null 2>&1 && printf 'pnpm=%s\n' "$(pnpm -v)"
command -v yarn >/dev/null 2>&1 && printf 'yarn=%s\n' "$(yarn -v)"
command -v python3 >/dev/null 2>&1 && printf 'python3=%s\n' "$(python3 --version 2>&1)"

section "git"
if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
  printf 'branch=%s\n' "$(git branch --show-current 2>/dev/null || true)"
  printf 'commit=%s\n' "$(git rev-parse --short HEAD 2>/dev/null || true)"
  printf 'dirty_files=%s\n' "$(git status --short 2>/dev/null | wc -l | tr -d ' ')"
  echo 'recent_commits='
  git log --oneline -5 2>/dev/null || true
else
  echo 'not_a_git_repo=true'
fi

section "project_files"
for f in package.json pnpm-lock.yaml package-lock.json yarn.lock tsconfig.json next.config.js next.config.mjs prisma/schema.prisma Dockerfile docker-compose.yml railway.json vercel.json; do
  [ -e "$f" ] && printf '%s\n' "$f"
done

section "env_key_names"
# Print only names likely relevant to runtime/debugging. Never values.
env | cut -d= -f1 | grep -E '(^NODE_ENV$|^APP_ENV$|^PORT$|DATABASE|REDIS|JWT|AUTH|COOKIE|CORS|API|WEBHOOK|STRIPE|OPENAI|ANTHROPIC|RAILWAY|VERCEL|NEXT_PUBLIC)' | sort -u || true

section "listening_ports"
if command -v lsof >/dev/null 2>&1; then
  lsof -nP -iTCP -sTCP:LISTEN 2>/dev/null | awk 'NR==1 || /node|python|bun|deno|next|npm|pnpm/ {print $1,$2,$9}' || true
else
  echo 'lsof_unavailable=true'
fi

section "notes"
echo 'Secrets, tokens, cookies, env values, and customer data intentionally omitted.'


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)