Skill: charlie-morrison/test-impact-analyzer (version 1.0.1)

--- name: test-impact-analyzer description: Determine which tests need to run for a given code change — trace file dependencies, map source-to-test relationships, identify untested changes, and prioritize test execution order for faster CI feedback. --- # Test Impact Analyzer Don't run all tests for every change. Analyze which source files changed, trace their dependencies, find the corresponding tests, and produce a targeted test execution plan. Faster CI, focused testing, immediate feedback. Use when: "which tests should I run", "what does this change affect", "test impact analysis", "optimize CI test time", "what tests cover this file", "skip unrelated tests", or speeding up CI pipelines. ## Commands ### 1. `affected` — Find Tests Affected by Changes Given a set of changed files (from git diff), find all tests that should run. ```bash # Get changed files (compared to main/master) BASE_BRANCH="${1:-main}" CHANGED_FILES=$(git diff --name-only "$BASE_BRANCH"...HEAD 2>/dev/null || git diff --name-only HEAD~1 2>/dev/null) if [ -z "$CHANGED_FILES" ]; then echo "No changed files detected. Specify base branch or ensure you're on a feature branch." exit 0 fi echo "Changed files:" echo "$CHANGED_FILES" | sed 's/^/ /' echo "" ``` #### Step 1: Direct Test Matches ```bash echo "=== Direct Test Matches ===" echo "$CHANGED_FILES" | while read f; do # Skip non-source files echo "$f" | grep -qE '\.(ts|js|tsx|jsx|py|go|rs|java)$' || continue # Skip test files themselves echo "$f" | grep -qE '\.(test|spec)\.' && continue BASE=$(basename "$f" | sed 's/\.[^.]*$//') DIR=$(dirname "$f") # Find corresponding test file for pattern in "${BASE}.test.ts" "${BASE}.test.js" "${BASE}.test.tsx" "${BASE}.test.jsx" \ "${BASE}.spec.ts" "${BASE}.spec.js" "${BASE}.spec.tsx" \ "test_${BASE}.py" "${BASE}_test.py" "${BASE}_test.go"; do FOUND=$(find "$DIR" -maxdepth 2 -name "$pattern" -not -path '*/node_modules/*' 2>/dev/null | head -1) if [ -n "$FOUND" ]; then echo " $f → $FOUND" break fi done done ``` #### Step 2: Import Chain Analysis ```bash echo "" echo "=== Import Chain (files that import changed files) ===" echo "$CHANGED_FILES" | while read f; do echo "$f" | grep -qE '\.(ts|js|tsx|jsx)$' || continue echo "$f" | grep -qE '\.(test|spec)\.' && continue BASE=$(basename "$f" | sed 's/\.[^.]*$//') DIR=$(dirname "$f") # Find files that import this module # Match relative imports like: from './module' or from '../utils/module' IMPORTERS=$(rg -l "from ['\"]\..*/${BASE}['\"]|from ['\"]\./${BASE}['\"]|require\(['\"]\..*/${BASE}['\"]\)" \ -g '*.{ts,js,tsx,jsx}' -g '!node_modules' -g '!dist' 2>/dev/null) if [ -n "$IMPORTERS" ]; then echo " $f is imported by:" echo "$IMPORTERS" | while read imp; do IMP_BASE=$(basename "$imp" | sed 's/\.[^.]*$//') # Is the importer a test file? if echo "$imp" | grep -qE '\.(test|spec)\.'; then echo " 🧪 $imp (test — should run)" else # Check if the importer has its own test TEST=$(find "$(dirname "$imp")" -maxdepth 2 \ -name "${IMP_BASE}.test.*" -o -name "${IMP_BASE}.spec.*" 2>/dev/null | head -1) if [ -n "$TEST" ]; then echo " 📄 $imp → 🧪 $TEST (transitive)" else echo " 📄 $imp (no test found)" fi fi done fi done ``` #### Step 3: Collect All Tests to Run ```bash echo "" echo "=== Test Execution Plan ===" # Collect unique test files TESTS_TO_RUN=$(mktemp) echo "$CHANGED_FILES" | while read f; do echo "$f" | grep -qE '\.(test|spec)\.' && echo "$f" >> "$TESTS_TO_RUN" BASE=$(basename "$f" | sed 's/\.[^.]*$//') DIR=$(dirname "$f") # Direct test matches find "$DIR" -maxdepth 2 \( -name "${BASE}.test.*" -o -name "${BASE}.spec.*" -o -name "test_${BASE}.*" -o -name "${BASE}_test.*" \) \ -not -path '*/node_modules/*' 2>/dev/null >> "$TESTS_TO_RUN" # Tests from importers IMPORTERS=$(rg -l "from ['\"]\..*/${BASE}['\"]|from ['\"]\./${BASE}['\"]" \ -g '*.{test.*,spec.*}' -g '!node_modules' 2>/dev/null) echo "$IMPORTERS" >> "$TESTS_TO_RUN" 2>/dev/null done UNIQUE_TESTS=$(sort -u "$TESTS_TO_RUN" | grep -v '^$') TOTAL=$(echo "$UNIQUE_TESTS" | grep -c "." 2>/dev/null || echo "0") ALL_TESTS=$(find . -type f \( -name "*.test.*" -o -name "*.spec.*" -o -name "test_*" \) \ -not -path '*/node_modules/*' 2>/dev/null | wc -l) echo "Tests to run: $TOTAL / $ALL_TESTS total ($(echo "scale=0; $TOTAL * 100 / ($ALL_TESTS + 1)" | bc)%)" echo "" echo "$UNIQUE_TESTS" | sed 's/^/ /' rm -f "$TESTS_TO_RUN" ``` ### 2. `map` — Source-to-Test Mapping Build a complete map of which source files are covered by which tests. ```bash echo "=== Source → Test Map ===" find . -type f \( -name "*.ts" -o -name "*.js" -o -name "*.tsx" -o -name "*.jsx" -o -name "*.py" -o -name "*.go" \) \ -not -path '*/node_modules/*' -not -path '*/vendor/*' -not -path '*/dist/*' \ -not -name '*.test.*' -not -name '*.spec.*' -not -name 'test_*' 2>/dev/null | sort | while read f; do BASE=$(basename "$f" | sed 's/\.[^.]*$//') DIR=$(dirname "$f") TEST=$(find "$DIR" -maxdepth 2 \( -name "${BASE}.test.*" -o -name "${BASE}.spec.*" -o -name "test_${BASE}.*" -o -name "${BASE}_test.*" \) \ -not -path '*/node_modules/*' 2>/dev/null | head -1) if [ -n "$TEST" ]; then echo "✅ $f → $TEST" else echo "�� $f → (no test)" fi done ``` Summary stats: ``` Coverage map: X/Y files have corresponding tests (Z%) Untested critical files (by size): 1. src/core/engine.ts (450 lines) — NO TEST 2. src/api/handlers.ts (320 lines) — NO TEST ``` ### 3. `gaps` — Find Untested Code Paths Identify source files without tests, prioritized by risk. ```bash echo "=== Untested Files (by risk) ===" python3 -c " import subprocess, os # Get all source files src = subprocess.run( ['find', '.', '-type', 'f', '(', '-name', '*.ts', '-o', '-name', '*.js', '-o', '-name', '*.py', ')', '-not', '-path', '*/node_modules/*', '-not', '-path', '*/dist/*', '-not', '-name', '*.test.*', '-not', '-name', '*.spec.*', '-not', '-name', 'test_*'], capture_output=True, text=True ).stdout.strip().split('\n') untested = [] for f in src: if not f: continue base = os.path.splitext(os.path.basename(f))[0] d = os.path.dirname(f) # Check for test file has_test = False for pattern in [f'{base}.test', f'{base}.spec', f'test_{base}', f'{base}_test']: result = subprocess.run( ['find', d, '-maxdepth', '2', '-name', f'{pattern}.*'], capture_output=True, text=True ) if result.stdout.strip(): has_test = True break if not has_test: # Get file size and git churn try: lines = sum(1 for _ in open(f)) except: lines = 0 try: churn = int(subprocess.run( ['git', 'log', '--since=90 days ago', '--oneline', '--', f], capture_output=True, text=True ).stdout.strip().count('\n')) + 1 except: churn = 0 risk = lines * 0.5 + churn * 10 # larger + more active = higher risk untested.append((risk, lines, churn, f)) untested.sort(reverse=True) print(f'Untested files: {len(untested)} / {len(src)} ({len(untested)*100//max(len(src),1)}%)') print() print('Top 20 by risk (size × activity):') for risk, lines, churn, f in untested[:20]: print(f' [{lines:>4} lines, {churn:>2} commits] {f}') " 2>/dev/null ``` ### 4. `order` — Optimal Test Execution Order Prioritize test execution for fastest feedback: 1. **Tests for changed files** — most likely to fail, run first 2. **Tests for files importing changed files** — transitive impact 3. **Integration/E2E tests** — broader coverage, run if unit tests pass 4. **Everything else** — only in full CI run ```bash echo "=== Recommended Test Order ===" echo "" echo "Phase 1 — Direct (run immediately, ~seconds):" # Changed files' tests echo "Phase 2 — Transitive (if Phase 1 passes, ~minutes):" # Tests that import changed modules echo "Phase 3 ��� Integration (if Phase 2 passes, ~minutes):" # E2E/integration tests in affected areas echo "Phase 4 — Full suite (nightly/merge, ~minutes-hours):" # Everything ``` ### 5. `ci` — Generate CI Test Commands Output the exact commands to run only affected tests. ```bash # For Jest JEST_TESTS=$(echo "$UNIQUE_TESTS" | grep -E '\.(test|spec)\.(ts|js|tsx|jsx)$' | tr '\n' ' ') if [ -n "$JEST_TESTS" ]; then echo "Jest command:" echo " npx jest --passWithNoTests $JEST_TESTS" fi # For pytest PYTEST_TESTS=$(echo "$UNIQUE_TESTS" | grep -E '(test_.*\.py|.*_test\.py)$' | tr '\n' ' ') if [ -n "$PYTEST_TESTS" ]; then echo "Pytest command:" echo " pytest $PYTEST_TESTS" fi # For Go GO_TESTS=$(echo "$UNIQUE_TESTS" | grep -E '_test\.go$' | xargs -I{} dirname {} | sort -u | tr '\n' ' ') if [ -n "$GO_TESTS" ]; then echo "Go test command:" echo " go test $GO_TESTS" fi ``` ## Output Formats - **text** (default): Human-readable with file tree - **json**: `{changed_files: [], affected_tests: [], untested_changes: [], execution_plan: {phases: []}}` - **paths**: Plain list of test file paths (pipe to test runner) ## CI Integration ```yaml # GitHub Actions — run only affected tests - name: Find affected tests id: tests run: | # Agent runs: test-impact-analyzer affected ${{ github.event.pull_request.base.ref }} --format paths > affected-tests.txt echo "count=$(wc -l < affected-tests.txt)" >> $GITHUB_OUTPUT - name: Run affected tests if: steps.tests.outputs.count > 0 run: npx jest $(cat affected-tests.txt | tr '\n' ' ') - name: Full test suite if: steps.tests.outputs.count == 0 run: npm test ``` Exit codes: - 0: All changed code has test coverage - 1: Some changed code is untested (lists the files) - 2: No tests found at all ## Notes - Uses file naming conventions for test matching (file.test.ts, test_file.py, file_test.go) - Import chain analysis is 1 level deep by default — use `--depth 2` for transitive imports - Does not parse code ASTs — uses grep/ripgrep patterns for speed - Works best when test files are co-located or follow naming conventions - For monorepos: respects workspace boundaries when tracing imports - Not a replacement for code coverage tools — this is pre-execution analysis, not runtime coverage


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)