Skill: andreagriffiths11/proof-agent (version 1.1.0)

--- name: proof-agent description: Adversarial verification of AI-generated work. Spawns an independent verifier to check for false claims, broken code, and security issues. --- # Proof Agent Independent adversarial verification for AI work. The worker and the verifier are always separate agents — self-verification is not verification. ## When to Verify Verify automatically when: - Subagent changed **3+ files** - ANY changed file matches: `*auth*`, `*secret*`, `*permission*`, `Dockerfile`, `*.env*` - User explicitly asks for verification Skip verification for: - Formatting-only changes (whitespace, linting fixes) - `.gitignore` changes ## How to Verify 1. **Spawn an independent verifier subagent** — the worker CANNOT verify its own work 2. Give the verifier ONLY: the original request, files changed, and approach taken 3. Do NOT share the worker's self-assessment or test results 4. The verifier must run its own commands and provide evidence 5. If no subagent ran (manual changes or user says "verify this"), use `git diff` output as the approach summary ### Verification Prompt Use this prompt when spawning the verifier subagent: ``` VERIFICATION REQUEST ## Original Request {what was asked} ## Files Changed {list of files} ## Approach Taken {what the worker did — or git diff summary if no subagent ran} ## Your Job You are an independent verifier. The worker who made these changes CANNOT verify their own work — only you can assign a verdict. ### Review Checklist 1. Correctness: Does the code actually do what was requested? 2. Bugs & Edge Cases: Regressions, unhandled errors, missed cases? 3. Security: Vulnerabilities, exposed secrets, permission issues? 4. Build: Does it build/compile/lint cleanly? 5. Facts: Are any claims, version numbers, or URLs verifiable? Check them. ### Rules - For EVERY check, include the actual command you ran and its output - Do NOT take the worker's word for anything - Do NOT give PASS without running at least 3 verification commands - You have NO information about the worker's test results — verify independently ## Verdict Assign EXACTLY ONE verdict as a markdown heading (### PASS, ### FAIL, or ### PARTIAL): ### PASS All checks passed. Every claim backed by command output. ### FAIL Issues found. List each as a bullet (- file, line, what's wrong, severity: critical/major/minor). ### PARTIAL Some passed, some unverifiable. List both with evidence. ``` ## Verdicts - **PASS** — All checks passed with evidence - **FAIL** — Issues found. Report to user with specifics. Retry up to 3 times if fixable. - **PARTIAL** — Some checks passed, others couldn't be verified. Report what's unverifiable. ## After Verification - **PASS**: Report summary to user, proceed - **FAIL**: Report issues to user. If auto-fixable, spawn worker to fix, then re-verify (max 3 attempts) - **PARTIAL**: Report to user, let them decide whether to proceed ## Scripts ### `scripts/verify.sh [base-ref]` Auto-extracts git diff, changed files, commit messages, and sensitive file detection. Outputs a filled verification prompt ready to send to the verifier subagent. Default base: `HEAD~1`. ```bash bash scripts/verify.sh # verify last commit bash scripts/verify.sh main # verify all changes since main ``` ### `scripts/fact-check.sh <file> [file2 ...]` Extracts and validates factual claims from files: - URLs → HTTP status check - npm packages → registry version lookup - GitHub Actions → tag/SHA existence check ```bash bash scripts/fact-check.sh src/content/articles/en/my-article.md bash scripts/fact-check.sh .github/workflows/*.yml ``` Returns exit code 1 if any checks fail. ## Configuration Projects can customize via `proof-agent.yaml` in the repo root (loaded by `proof_agent/config.py`): ```yaml thresholds: min_files_changed: 3 always_verify: - "**/*auth*" - "**/*secret*" - "**/*permission*" - "**/Dockerfile" - "**/*.env*" never_verify: - "**/.gitignore" retry: max_attempts: 3 escalate_on_max: true ``` ## Key Principle > The worker and verifier must be separate agents. Self-verification is not verification.

path: scripts/fact-check.sh
content: #!/usr/bin/env bash
# fact-check.sh — Extract and validate factual claims from files
# Usage: ./fact-check.sh <file> [file2 ...]
#
# Checks:
# - URLs (HTTP status)
# - npm package versions (registry lookup)
# - GitHub action versions (tag existence)
# - Python package versions (PyPI lookup)
#
# Outputs a report of claims found and their validation status.

set -euo pipefail

RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
NC='\033[0m'

PASS_COUNT=0
FAIL_COUNT=0
WARN_COUNT=0

check_url() {
  local url="$1"
  local status
  status=$(curl -sL -o /dev/null -w "%{http_code}" --max-time 10 "$url" 2>/dev/null || echo "000")
  if [[ "$status" =~ ^[23] ]]; then
    echo -e "  ${GREEN}✓${NC} URL $url → $status"
    ((PASS_COUNT++))
  elif [ "$status" = "000" ]; then
    echo -e "  ${YELLOW}?${NC} URL $url → timeout/unreachable"
    ((WARN_COUNT++))
  else
    echo -e "  ${RED}✗${NC} URL $url → $status"
    ((FAIL_COUNT++))
  fi
}

check_npm_version() {
  local pkg="$1"
  local ver="$2"
  local registry_ver
  registry_ver=$(curl -s "https://registry.npmjs.org/$pkg/$ver" 2>/dev/null | grep -o '"version":"[^"]*"' | head -1 | cut -d'"' -f4)
  if [ "$registry_ver" = "$ver" ]; then
    echo -e "  ${GREEN}✓${NC} npm $pkg@$ver exists"
    ((PASS_COUNT++))
  elif [ -z "$registry_ver" ]; then
    echo -e "  ${RED}✗${NC} npm $pkg@$ver not found on registry"
    ((FAIL_COUNT++))
  else
    echo -e "  ${YELLOW}?${NC} npm $pkg@$ver — registry returned $registry_ver"
    ((WARN_COUNT++))
  fi
}

check_github_action() {
  local action="$1"
  local ref="$2"
  local repo
  repo=$(echo "$action" | cut -d'@' -f1)
  local status
  status=$(curl -sL -o /dev/null -w "%{http_code}" "https://github.com/$repo/tree/$ref" 2>/dev/null || echo "000")
  if [[ "$status" =~ ^[23] ]]; then
    echo -e "  ${GREEN}✓${NC} Action $repo@$ref exists"
    ((PASS_COUNT++))
  else
    echo -e "  ${RED}✗${NC} Action $repo@$ref → $status"
    ((FAIL_COUNT++))
  fi
}

if [ $# -eq 0 ]; then
  echo "Usage: fact-check.sh <file> [file2 ...]"
  exit 1
fi

for file in "$@"; do
  if [ ! -f "$file" ]; then
    echo -e "${YELLOW}Skipping $file (not found)${NC}"
    continue
  fi

  echo ""
  echo "=== Checking: $file ==="

  # Extract and check URLs
  URLS=$(grep -oE 'https?://[^ '\''")<>\]]+' "$file" 2>/dev/null | sort -u || true)
  if [ -n "$URLS" ]; then
    echo ""
    echo "URLs found:"
    while IFS= read -r url; do
      # Skip localhost and example URLs
      if echo "$url" | grep -qE 'localhost|127\.0\.0\.1|example\.com|your-'; then
        continue
      fi
      check_url "$url"
    done <<< "$URLS"
  fi

  # Extract and check npm packages (pattern: package@version)
  NPM_PKGS=$(grep -oE '[a-zA-Z@][a-zA-Z0-9./_-]*@[0-9]+\.[0-9]+\.[0-9]+[a-zA-Z0-9.-]*' "$file" 2>/dev/null | grep -v '^@' | sort -u || true)
  # Also catch scoped packages
  SCOPED_PKGS=$(grep -oE '@[a-zA-Z0-9-]+/[a-zA-Z0-9-]+@[0-9]+\.[0-9]+\.[0-9]+' "$file" 2>/dev/null | sort -u || true)
  ALL_PKGS=$(printf '%s\n%s' "$NPM_PKGS" "$SCOPED_PKGS" | grep -v '^$' | sort -u || true)
  if [ -n "$ALL_PKGS" ]; then
    echo ""
    echo "npm packages found:"
    while IFS= read -r pkg_ver; do
      pkg=$(echo "$pkg_ver" | sed 's/@[0-9]*\.[0-9]*\.[0-9]*.*$//' | sed 's/\^//g')
      ver=$(echo "$pkg_ver" | grep -oE '[0-9]+\.[0-9]+\.[0-9]+[a-zA-Z0-9.-]*')
      if [ -n "$pkg" ] && [ -n "$ver" ]; then
        check_npm_version "$pkg" "$ver"
      fi
    done <<< "$ALL_PKGS"
  fi

  # Extract GitHub Actions (pattern: uses: owner/repo@ref)
  ACTIONS=$(grep -oE 'uses: *[a-zA-Z0-9_-]+/[a-zA-Z0-9_-]+@[a-zA-Z0-9._-]+' "$file" 2>/dev/null | sed 's/uses: *//' | sort -u || true)
  if [ -n "$ACTIONS" ]; then
    echo ""
    echo "GitHub Actions found:"
    while IFS= read -r action; do
      ref=$(echo "$action" | cut -d'@' -f2)
      check_github_action "$action" "$ref"
    done <<< "$ACTIONS"
  fi
done

echo ""
echo "=== Summary ==="
echo -e "${GREEN}Pass: $PASS_COUNT${NC}  ${RED}Fail: $FAIL_COUNT${NC}  ${YELLOW}Warn: $WARN_COUNT${NC}"

if [ "$FAIL_COUNT" -gt 0 ]; then
  exit 1
fi


path: scripts/verify.sh
content: #!/usr/bin/env bash
# verify.sh — Auto-extract context for proof-agent verification
# Usage: ./verify.sh [base-ref]
# Default base-ref: HEAD~1 (last commit)
#
# Outputs a filled verification prompt to stdout.

set -euo pipefail

BASE="${1:-HEAD~1}"

# Get changed files
FILES=$(git diff --name-only "$BASE" HEAD 2>/dev/null || git diff --name-only --cached 2>/dev/null || echo "(no git changes detected)")

# Count
FILE_COUNT=$(echo "$FILES" | grep -c '.' || echo 0)

# Get diff summary
DIFF=$(git diff --stat "$BASE" HEAD 2>/dev/null || echo "(no diff available)")

# Get commit messages since base
COMMITS=$(git log --oneline "$BASE"..HEAD 2>/dev/null || echo "(no commits)")

# Check for sensitive files
SENSITIVE=""
for pattern in auth secret permission Dockerfile .env; do
  MATCHES=$(echo "$FILES" | grep -i "$pattern" || true)
  if [ -n "$MATCHES" ]; then
    SENSITIVE="${SENSITIVE}${MATCHES}\n"
  fi
done

# Determine if verification is needed
SHOULD_VERIFY=false
if [ "$FILE_COUNT" -ge 3 ]; then
  SHOULD_VERIFY=true
fi
if [ -n "$SENSITIVE" ]; then
  SHOULD_VERIFY=true
fi

if [ "$SHOULD_VERIFY" = false ]; then
  echo "SKIP: Only $FILE_COUNT file(s) changed, no sensitive files detected."
  echo "Use --force or run manually to verify anyway."
  exit 0
fi

# Build the prompt
cat <<EOF
VERIFICATION REQUEST

## Files Changed ($FILE_COUNT files)
$FILES

## Diff Summary
$DIFF

## Commits
$COMMITS

## Sensitive Files Detected
$([ -n "$SENSITIVE" ] && echo -e "$SENSITIVE" || echo "None")

## Your Job

You are an independent verifier. The worker who made these changes CANNOT verify their own work — only you can assign a verdict.

### Review Checklist
1. Correctness: Does the code actually do what was requested?
2. Bugs & Edge Cases: Regressions, unhandled errors, missed cases?
3. Security: Vulnerabilities, exposed secrets, permission issues?
4. Build: Does it build/compile/lint cleanly?
5. Facts: Are any claims, version numbers, or URLs verifiable? Check them.

### Rules
- For EVERY check, include the actual command you ran and its output
- Do NOT take the worker's word for anything
- Do NOT give PASS without running at least 3 verification commands
- You have NO information about the worker's test results — verify independently

## Verdict

Assign EXACTLY ONE:

PASS — All checks passed. Every claim backed by command output.
FAIL — Issues found. List each: file, line, what's wrong, severity (critical/major/minor).
PARTIAL — Some passed, some unverifiable. List both with evidence.
EOF


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (HIGH)