crackerjack 0.54.2
Crackerjack Python project management tool
License: BSD-3-CLAUSE
Requires Python: >=3.13

# Crackerjack: Advanced AI-Driven Python Development Platform

[![Code style: crackerjack](https://img.shields.io/badge/code%20style-crackerjack-000042)](https://github.com/lesleslie/crackerjack)
[![Python: 3.13+](https://img.shields.io/badge/python-3.13%2B-green)](https://www.python.org/downloads/)
[![pytest](https://img.shields.io/badge/pytest-coverage%20ratchet-blue)](https://pytest.org)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)
[![Quality Hooks](https://img.shields.io/badge/quality%20hooks-17%20tools-brightgreen)](https://github.com/lesleslie/crackerjack)
[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
![Coverage](https://img.shields.io/badge/coverage-21.6%25-red)

## 🎯 Purpose

**Crackerjack** transforms Python development from reactive firefighting to proactive excellence. This sophisticated platform empowers developers to create exceptional code through intelligent automation, comprehensive quality enforcement, and AI-powered assistance. Experience the confidence that comes from knowing your code meets the highest standards before it ever runs in production.

### What is "Crackerjack"?

**crack·​er·​jack** ˈkra-kər-ˌjak (noun): *A person or thing of marked excellence or ability; first-rate; exceptional.*

Just as the name suggests, Crackerjack makes your Python projects first-rate through:

- **🧠 Proactive AI Architecture**: 12 specialized AI agents prevent issues before they occur
- **⚡ Autonomous Quality**: Intelligent auto-fixing with architectural planning
- **🛡️ Zero-Compromise Standards**: 100% test coverage, complexity ≤15, security-first patterns
- **🔄 Learning System**: Gets smarter with every project, caching successful patterns
- **🌟 One Command Excellence**: From setup to PyPI publishing with a single command

**The Crackerjack Philosophy**: If your code needs fixing after it's written, you're doing it wrong. We prevent problems through intelligent architecture and proactive patterns, making exceptional code the natural outcome, not a lucky accident.

## What Problem Does Crackerjack Solve?

**Instead of configuring multiple tools separately:**

```bash
# Traditional workflow
pip install black isort flake8 mypy pytest
# Configure each tool individually
# Set up git hooks manually
# Remember different commands for each tool
```

**Crackerjack provides unified commands:**

```bash
pip install crackerjack
python -m crackerjack run        # Setup + quality checks
python -m crackerjack run --run-tests        # Add testing
python -m crackerjack run --all patch # Full release workflow
```

**Key differentiators:**

- **Single command** replaces 6+ separate tools
- **Pre-configured** with Python best practices
- **UV integration** for fast dependency management
- **Automated publishing** with PyPI authentication
- **MCP server** for AI agent integration

## The Crackerjack Philosophy

Crackerjack is built on the following core principles:

- **Code Clarity:** Code should be easy to read, understand, and maintain
- **Automation:** Tedious tasks should be automated, allowing developers to focus on solving problems
- **Consistency:** Code style, formatting, and project structure should be consistent across projects
- **Reliability:** Tests are essential, and code should be checked rigorously
- **Tool Integration:** Leverage powerful existing tools instead of reinventing the wheel
- **Auto-Discovery:** Prefer intelligent auto-discovery of configurations and settings over manual configuration whenever possible, reducing setup friction and configuration errors
- **Static Typing:** Static typing is essential for all development

## Crackerjack vs Pre-commit: Architecture & Features

Crackerjack and pre-commit solve related but different problems. While pre-commit is a language-agnostic git hook manager, Crackerjack is a comprehensive Python development platform with quality enforcement built-in.

### Architectural Differences

| Aspect | Pre-commit | Crackerjack |
|--------|-----------|-------------|
| **Execution Model** | Wrapper framework that spawns subprocesses for each hook | Direct tool invocation with adapter architecture |
| **Concurrency** | Synchronous sequential execution (one hook at a time) | **Async-first with 11 concurrent adapters** - true parallel execution |
| **Performance** | Overhead from framework wrapper + subprocess spawning | Zero wrapper overhead, 70% cache hit rate, 50% faster workflows |
| **Language Focus** | Language-agnostic (Python, Go, Rust, Docker, etc.) | Python-first with native tool implementations |
| **Configuration** | YAML-based `.pre-commit-config.yaml` with repo URLs | Python-based configuration with intelligent defaults |
| **Hook Management** | Clones repos, manages environments per hook | Native Python tools + direct UV invocation |

### Feature Comparison

#### Quality Hooks & Tools

| Feature | Pre-commit | Crackerjack |
|---------|-----------|-------------|
| **Code Formatting** | ✅ Via hooks (black, ruff, etc.) | ✅ Native Ruff integration + mdformat |
| **Linting** | ✅ Via hooks (flake8, pylint, etc.) | ✅ Native Ruff + codespell |
| **Type Checking** | ✅ Via hooks (mypy, pyright) | ✅ **Zuban** (20-200x faster than pyright) |
| **Security Scanning** | ✅ Via hooks (bandit, gitleaks) | ✅ Native bandit + gitleaks integration |
| **Dead Code Detection** | ✅ Via vulture hook | ✅ **Skylos** (20x faster than vulture) |
| **Complexity Analysis** | ❌ Not built-in | ✅ Native complexipy integration |
| **Dependency Validation** | ❌ Not built-in | ✅ Native creosote unused dependency detection |
| **Custom Python Tools** | ✅ Via `repo: local` hooks | ✅ 6 native tools in `crackerjack/tools/` |

#### Development Workflow

| Feature | Pre-commit | Crackerjack |
|---------|-----------|-------------|
| **Git Integration** | ✅ Pre-commit, pre-push, commit-msg hooks | ✅ Git hooks + intelligent commit messages |
| **Testing Framework** | ❌ Not included | ✅ Built-in pytest with coverage ratchet |
| **CI/CD Integration** | ✅ Via `pre-commit run --all-files` | ✅ Unified `--ci` mode with quality + tests |
| **Version Management** | ❌ Not included | ✅ Intelligent version bumping + AI recommendations |
| **Publishing** | ❌ Not included | ✅ PyPI publishing with UV authentication |
| **Hook Stages** | ✅ Multiple stages (commit, push, merge, manual) | ✅ Fast (~5s) vs Comprehensive (~30s) strategies |
| **Retry Logic** | ❌ No built-in retry | ✅ Automatic retry for formatting hooks |
| **Parallel Execution** | ✅ Limited parallelism (sequential by default) | ✅ **Async-first architecture**: 11 concurrent adapters, 76% speedup |

#### Advanced Features

| Feature | Pre-commit | Crackerjack |
|---------|-----------|-------------|
| **AI Integration** | ❌ Not built-in | ✅ 12 specialized AI agents + auto-fixing |
| **Dependency Injection** | ❌ Not applicable | ✅ legacy framework with protocol-based DI |
| **Caching** | ✅ Per-file hash caching | ✅ Content-based caching (70% hit rate) |
| **MCP Server** | ❌ Not included | ✅ Built-in MCP server for Claude integration |
| **Monitoring** | ❌ Not included | ✅ MCP status + progress monitors |
| **Configuration Management** | ✅ YAML + `--config` flag | ✅ settings with YAML + local overrides |
| **Auto-Update** | ✅ `pre-commit autoupdate` | ⚠️ Manual UV dependency updates |
| **Language Support** | ✅ 15+ languages (Python, Go, Rust, Docker, etc.) | ✅ Python + external tools (gitleaks, etc.) |

#### Configuration & Ease of Use

| Feature | Pre-commit | Crackerjack |
|---------|-----------|-------------|
| **Setup Complexity** | Medium (YAML config + `pre-commit install`) | Low (single `python -m crackerjack run`) |
| **Configuration Format** | YAML with repo URLs and hook IDs | Python settings with intelligent defaults |
| **Hook Discovery** | Manual (add repos to `.pre-commit-config.yaml`) | Automatic (17 tools pre-configured) |
| **Tool Installation** | Auto (pre-commit manages environments) | UV-based (one virtual environment) |
| **Learning Curve** | Medium (understand repos, hooks, stages) | Low (unified Python commands) |

### When to Use Each

**Choose Pre-commit when:**

- ✅ Working with multiple languages (Go, Rust, Docker, etc.)
- ✅ Need language-agnostic hook framework
- ✅ Want to use hooks from community repositories
- ✅ Polyglot projects requiring diverse tooling
- ✅ Simple YAML-based configuration preferred

**Choose Crackerjack when:**

- ✅ Python-focused development (Python 3.13+)
- ✅ Want comprehensive development platform (testing, publishing, AI)
- ✅ Need maximum performance (async architecture, Rust tools, caching, 11x parallelism)
- ✅ Desire AI-powered auto-fixing and recommendations
- ✅ Want unified workflow (quality + tests + publishing in one command)
- ✅ Prefer Python-based configuration over YAML
- ✅ Need advanced features (coverage ratchet, MCP integration, monitoring)

### Migration from Pre-commit

Crackerjack can **coexist** with pre-commit if needed, but most Python projects can fully migrate:

```bash
# Remove pre-commit (optional)
pre-commit uninstall
rm .pre-commit-config.yaml

# Install crackerjack
uv tool install crackerjack

# Run quality checks (replaces pre-commit run --all-files)
python -m crackerjack run

# With tests (comprehensive workflow)
python -m crackerjack run --run-tests
```

**Note**: Crackerjack Phase 8 successfully migrated from pre-commit framework to direct tool invocation, achieving 50% performance improvement while maintaining full compatibility with existing quality standards.

## Table of Contents

- [Crackerjack vs Pre-commit](#crackerjack-vs-pre-commit-architecture--features)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [AI Auto-Fix Features](#ai-auto-fix-features)
- [Core Workflow](#core-workflow)
- [Core Features](#core-features)
- [legacy Architecture & Performance](#-legacy-architecture--performance)
- [Adapters](#adapters)
- [Configuration Management](#-configuration-management-legacy-settings--configuration-templates)
- [MCP Server Configuration](#mcp-server-configuration)
- [Quality Hook Modes](#quality-hook-modes)
- [Command Reference](#command-reference)
- [Style Guide](#style-guide)
- [Publishing & Version Management](#publishing--version-management)
- [Troubleshooting](#-troubleshooting)

## Installation

### Prerequisites

- Python 3.13+
- [UV](https://github.com/astral-sh/uv) package manager

### Install UV

```bash
# Recommended: Official installer script
curl -LsSf https://astral.sh/uv/install.sh | sh

# Alternative: Using pipx
pipx install uv

# Alternative: Using Homebrew (macOS)
brew install uv
```

### Install Crackerjack

```bash
# Recommended: Using UV (fastest)
uv tool install crackerjack

# Alternative: Using pip
pip install crackerjack

# For existing project: Add as dependency
uv add crackerjack
```

## Quick Start

### Initialize a Project

```bash
# Navigate to your project directory
cd your-project

# Initialize with Crackerjack
python -m crackerjack run

# Or use interactive mode
python -m crackerjack run -i
```

## AI Auto-Fix Features

![AI Agent Orchestration](docs/diagrams/ai-agent-orchestration.png)
*12 specialized AI agents with confidence-based routing and batch processing*

Crackerjack provides two distinct approaches to automatic error fixing:

### 1. Hook Auto-Fix Modes (Basic Formatting)

Limited tool-specific auto-fixes for simple formatting issues:

- `ruff --fix`: Import sorting, basic formatting
- `trailing-whitespace --fix`: Removes trailing whitespace
- `end-of-file-fixer --fix`: Ensures files end with newline

**Limitations:** Only handles simple style issues, cannot fix type errors, security issues, test failures, or complex code quality problems.

### 2. AI Agent Auto-Fixing (Comprehensive Intelligence)

**Revolutionary AI-powered code quality enforcement** that automatically fixes ALL types of issues:

#### How AI Agent Auto-Fixing Works

1. **🚀 Run All Checks**: Fast hooks, comprehensive hooks, full test suite
1. **🔍 Analyze Failures**: AI parses error messages, identifies root causes
1. **🤖 Intelligent Fixes**: AI reads source code and makes targeted modifications
1. **🔄 Repeat**: Continue until ALL checks pass (up to 8 iterations)
1. **🎉 Perfect Quality**: Zero manual intervention required

#### Comprehensive Coverage

The AI agent intelligently fixes:

- **Type Errors (zuban)**: Adds missing annotations, fixes type mismatches
- **🔒 Security Issues (bandit)**: Comprehensive security hardening including:
  - **Shell Injection Prevention**: Removes `shell=True` from subprocess calls
  - **Weak Cryptography**: Replaces MD5/SHA1 with SHA256
  - **Insecure Random Functions**: Replaces `random.choice` with `secrets.choice`
  - **Unsafe YAML Loading**: Replaces `yaml.load` with `yaml.safe_load`
  - **Token Exposure**: Masks PyPI tokens, GitHub PATs, and sensitive credentials
  - **Debug Print Removal**: Eliminates debug prints containing sensitive information
- **Dead Code (vulture)**: Removes unused imports, variables, functions
- **Performance Issues**: Transforms inefficient patterns (list concatenation, string building, nested loops)
- **Documentation Issues**: Auto-generates changelogs, maintains consistency across .md files
- **Test Failures**: Fixes missing fixtures, import errors, assertions
- **Code Quality (refurb)**: Applies refactoring, reduces complexity
- **All Hook Failures**: Formatting, linting, style issues

#### AI Agent Commands

```bash
# Standard AI agent mode (recommended)
python -m crackerjack run --ai-fix --run-tests --verbose

# Preview fixes without applying (dry-run mode)
python -m crackerjack run --dry-run --run-tests --verbose

# Custom iteration limit
python -m crackerjack run --ai-fix --max-iterations 15

# MCP server
python -m crackerjack start

# Lifecycle commands (start/stop/restart/status/health) are available via MCPServerCLIFactory.
```

#### MCP Integration

When using crackerjack via MCP tools (session-mgmt-mcp):

```python
# ✅ CORRECT - Use semantic command + ai_agent_mode parameter
crackerjack_run(command="test", ai_agent_mode=True)

# ✅ CORRECT - With additional arguments
crackerjack_run(command="check", args="--verbose", ai_agent_mode=True, timeout=600)

# ✅ CORRECT - Dry-run mode
crackerjack_run(command="test", args="--dry-run", ai_agent_mode=True)

# ❌ WRONG - Don't put flags in command parameter
crackerjack_run(command="--ai-fix -t")  # This will error!

# ❌ WRONG - Don't use --ai-fix in args
crackerjack_run(command="test", args="--ai-fix")  # Use ai_agent_mode=True instead
```

#### Configuration

Auto-fix requires:

1. **Anthropic API key**: Set environment variable

   ```bash
   export ANTHROPIC_API_KEY=sk-ant-...
   ```

1. **Configuration file**: `settings/adapters.yml`

   ```yaml
   ai: claude
   ```

#### Key Benefits

- **Zero Configuration**: No complex flag combinations needed
- **Complete Automation**: Handles entire quality workflow automatically
- **Intelligent Analysis**: Understands code context and business logic
- **Comprehensive Coverage**: Fixes ALL error types, not just formatting
- **Perfect Results**: Achieves 100% code quality compliance

#### 🤖 Specialized Agent Architecture

**12 Specialized AI Agents** for comprehensive code quality improvements:

- **🔒 SecurityAgent**: Fixes shell injections, weak crypto, token exposure, unsafe library usage
- **♻️ RefactoringAgent**: Reduces complexity ≤15, extracts helper methods, applies SOLID principles
- **🚀 PerformanceAgent**: Optimizes algorithms, fixes O(n²) patterns, improves string building
- **📝 DocumentationAgent**: Auto-generates changelogs, maintains .md file consistency
- **🧹 DRYAgent**: Eliminates code duplication, extracts common patterns to utilities
- **✨ FormattingAgent**: Handles code style, import organization, formatting violations
- **🧪 TestCreationAgent**: Fixes test failures, missing fixtures, dependency issues
- **📦 ImportOptimizationAgent**: Removes unused imports, restructures import statements
- **🔬 TestSpecialistAgent**: Advanced testing scenarios, fixture management
- **🔍 SemanticAgent**: Advanced semantic analysis, code comprehension, intelligent refactoring suggestions based on business logic understanding
- **🏗️ ArchitectAgent**: High-level architectural patterns, design recommendations, system-level optimization strategies
- **🎯 EnhancedProactiveAgent**: Proactive issue prevention, predictive quality monitoring, optimization before problems occur

**Agent Coordination Features**:

- **Confidence Scoring**: Routes issues to best-match agent (≥0.7 confidence)
- **Batch Processing**: Groups related issues for efficient parallel processing
- **Collaborative Mode**: Multiple agents handle complex cross-cutting concerns

#### Security & Safety Features

- **Command Validation**: All AI modifications are validated for safety
- **Advanced-Grade Regex**: Centralized pattern system eliminates dangerous regex issues
- **No Shell Injection**: Uses secure subprocess execution with validated patterns
- **Rollback Support**: All changes can be reverted via git
- **Human Review**: Review AI-generated changes before commit

#### ⚡ High-Performance Rust Tool Integration

**Ultra-Fast Static Analysis Tools**:

- **🦅 Skylos** (Dead Code Detection): Replaces vulture with **20x performance improvement**

  - Rust-powered dead code detection and import analysis
  - Seamlessly integrates with crackerjack's quality workflow
  - Zero configuration changes required

- **🔍 Zuban** (Type Checking): Replaces pyright with **20-200x performance improvement**

  - Lightning-fast type checking and static analysis
  - Drop-in replacement for slower Python-based tools
  - Maintains full compatibility with existing configurations

**Performance Benefits**:

- **Faster Development Cycles**: Quality hooks complete in seconds, not minutes
- **Improved Developer Experience**: Near-instantaneous feedback during development
- **Seamless Integration**: Works transparently with existing crackerjack workflows
- **Zero Breaking Changes**: Same CLI interface, dramatically better performance

**Implementation Details**:

```bash
# These commands now benefit from Rust tool speed improvements:
python -m crackerjack run                    # Dead code detection 20x faster
python -m crackerjack run --run-tests        # Type checking 20-200x faster
python -m crackerjack run --ai-fix --run-tests # Complete workflow optimized
```

**Benchmark Results**: Real-world performance measurements show consistent **6,000+ operations/second** throughput with **600KB+/second** data processing capabilities during comprehensive quality checks.

## 🎯 Skills Tracking Integration (Session-Buddy)

Crackerjack integrates with **session-buddy** for comprehensive AI agent metrics tracking and intelligent skill recommendations.

### What is Skills Tracking?

**Automated metrics collection** for all AI agent invocations:

- **Which agents were selected** - Track agent choices and why
- **User queries** - Record problems that triggered agent selection
- **Alternatives considered** - Log which other agents were evaluated
- **Success/failure rates** - Measure agent effectiveness by context
- **Performance metrics** - Duration, completion rates, by workflow phase
- **Semantic discovery** - Find best agents for problems using vector similarity

### Why It Matters

**Learn from Every Agent Invocation**:

- 🎯 **Better Agent Selection**: Learn which agents work best for specific problems
- 📊 **Performance Insights**: Identify bottlenecks and optimization opportunities
- 🧠 **Semantic Discovery**: Find agents using natural language queries
- 🔄 **Continuous Improvement**: System gets smarter with every invocation
- 📈 **Workflow Correlation**: Understand agent effectiveness by Oneiric phase

### Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    Crackerjack                              │
│  ┌───────────────┐    ┌───────────────┐    ┌───────────────┐│
│  │  Agent        │    │  Agent        │    │  Agent        ││
│  │  Orchestrator │───▶│  Context      │───▶│  Skills       ││
│  │              │    │              │    │  Tracker      ││
│  └───────────────┘    └───────────────┘    └───────┬───────┘│
│                                                 │          │
└─────────────────────────────────────────────────┼───────────┘
                                                  │
                              ┌─────────────────────┴─────────────────┐
                              │   Skills Tracking Protocol           │
                              │  (track_invocation, get_recommendations)│
                              └─────────────────────┬─────────────────┘
                                                  │
                    ┌─────────────────────────────┼─────────────────────┐
                    │                             │                     │
            ┌───────▼────────┐          ┌────────▼────────┐    ┌───────▼──────┐
            │  Direct API   │          │   MCP Bridge    │    │   No-Op      │
            │  (Tight Coupling)│       │  (Loose Coupling)│    │  (Disabled)  │
            │  session-buddy│          │  session-buddy  │    │              │
            └───────────────┘          └─────────────────┘    └──────────────┘
                    │
            ┌───────▼────────┐
            │ Dhruva Storage│
            │ (SQLite + WAL) │
            └────────────────┘
```

### Configuration

**Enable/Disable in `settings/local.yaml` or `settings/crackerjack.yaml`**:

```yaml
# Enable skills tracking (default: true)
skills:
  enabled: true

  # Backend choice: "direct", "mcp", "auto"
  backend: auto  # Tries MCP first, falls back to direct

  # Database location (default: .session-buddy/skills.db)
  db_path: null

  # MCP server URL (for MCP bridge)
  mcp_server_url: "http://localhost:8678"

  # Recommendation settings
  min_similarity: 0.3          # Minimum similarity for recommendations (0.0-1.0)
  max_recommendations: 5       # Max agents to recommend
  enable_phase_aware: true     # Consider workflow phase in recommendations
  phase_weight: 0.3           # Weight for phase effectiveness (0.0-1.0)
```

### Backend Options

| Backend | Pros | Cons | Best For |
|---------|------|------|----------|
| **`direct`** | • Fast (direct API)<br>• Simple setup<br>• Low latency | • Tight coupling<br>• Requires session-buddy in Python path | • Local development<br>• Single-machine setups |
| **`mcp`** | • Loose coupling<br>• Remote deployment<br>• Easy testing | • Higher latency<br>• More complex | • Distributed systems<br>• Microservices<br>• Multi-project setups |
| **`auto`** (default) | • Tries MCP first<br>• Automatic fallback<br>• Best of both | • Slightly slower initial connection | • Most scenarios (recommended) |

### Usage Patterns

#### 1. Automatic Tracking (Default)

All agent invocations are **automatically tracked** via `AgentOrchestrator`:

```python
# In agent_orchestrator.py
async def _execute_crackerjack_agent(agent, request):
    # Automatic tracking
    completer = request.context.track_skill_invocation(
        skill_name=agent.metadata.name,
        user_query=request.task.description,
        workflow_phase=request.task.category,
    )

    try:
        result = await agent.agent.analyze_and_fix(issue)
        completer(completed=True)  # Record success
    except Exception as e:
        completer(completed=False, error_type=str(e))  # Record failure
        raise
```

#### 2. Manual Tracking in Custom Code

```python
from crackerjack.agents.base import AgentContext

# Track with manual control
context = AgentContext(
    project_path=Path("/my/project"),
    skills_tracker=tracker,  # From dependency injection
)

# Track invocation
completer = context.track_skill_invocation(
    skill_name="MyCustomAgent",
    user_query="Fix complexity issues",
    workflow_phase="comprehensive_hooks",
)

# ... do work ...

# Complete tracking
completer(completed=True)
```

#### 3. Get Recommendations

```python
# Get agent recommendations for a problem
recommendations = context.get_skill_recommendations(
    user_query="How do I fix type errors in async code?",
    limit=5,
    workflow_phase="comprehensive_hooks",
)

# Returns:
# [
#   {
#     "skill_name": "RefactoringAgent",
#     "similarity_score": 0.92,
#     "completed": True,
#     "duration_seconds": 45.2,
#     "workflow_phase": "comprehensive_hooks"
#   },
#   ...
# ]
```

### Data Migration

**Migrate from JSON-based metrics to Dhruva database**:

```bash
# 1. Backup existing JSON
cp .crackerjack/metrics.json .crackerjack/metrics.json.backup

# 2. Run migration (dry-run first)
python scripts/migrate_skills_to_sessionbuddy.py --dry-run

# 3. Actual migration
python scripts/migrate_skills_to_sessionbuddy.py

# 4. Validate migration
python scripts/validate_skills_migration.py

# 5. Rollback if needed
python scripts/rollback_skills_migration.py
```

**Migration Features**:

- ✅ **Automatic backup** - Creates `.pre-migration.backup` files
- ✅ **Dry-run mode** - Preview changes without modifying database
- ✅ **Validation** - Checks JSON structure and required fields
- ✅ **Rollback support** - Restore from backup if issues occur
- ✅ **Progress tracking** - See migration status in real-time

### Performance Considerations

**Direct API (Tight Coupling)**:

- **Latency**: < 1ms per invocation (in-process)
- **Throughput**: 10,000+ invocations/second
- **Memory**: ~5MB per session
- **Best for**: Local development, single-machine setups

**MCP Bridge (Loose Coupling)**:

- **Latency**: 5-10ms per invocation (network round-trip)
- **Throughput**: 1,000+ invocations/second
- **Memory**: ~10MB per session (includes client)
- **Best for**: Distributed systems, microservices

**Overhead**:

- **No-op (disabled)**: Zero overhead (~0.001µs per check)
- **Direct**: ~0.5% overhead in typical workflows
- **MCP**: ~2% overhead in typical workflows

### Advanced Features

#### Semantic Skill Discovery

```python
# Find agents using natural language
recommendations = tracker.get_recommendations(
    user_query="I need help with memory leaks in async code", limit=5
)

# Semantic search finds:
# - PerformanceAgent (specializes in leaks)
# - RefactoringAgent (async patterns)
# - TestSpecialistAgent (memory testing)
```

#### Workflow-Phase-Aware Recommendations

```python
# Get recommendations for specific Oneiric phase
recommendations = tracker.get_recommendations(
    user_query="Fix import errors",
    workflow_phase="fast_hooks",  # Only agents effective in fast_hooks
    limit=3,
)

# Considers:
# - Which agents work best in fast_hooks phase
# - Historical completion rates by phase
# - Average duration by phase
```

#### Selection Ranking

```python
# Track which alternative agents were considered
completer = context.track_skill_invocation(
    skill_name="RefactoringAgent",  # Selected agent
    user_query="Fix complexity",
    alternatives_considered=["PerformanceAgent", "DRYAgent"],
    selection_rank=1,  # First choice
)
```

### Troubleshooting

**Skills tracking not working**:

```bash
# Check if session-buddy is available
python -c "from session_buddy.core.skills_tracker import get_session_tracker; print('OK')"

# Verify configuration
python -c "from crackerjack.config import CrackerjackSettings; s = CrackerjackSettings.load(); print(s.skills)"

# Check database
ls -la .session-buddy/skills.db
```

**MCP connection failures**:

```bash
# Verify MCP server is running
python -m crackerjack status

# Test MCP connection
curl http://localhost:8678/health

# Check fallback to direct tracking
# MCP failures automatically fall back to direct API
```

**Migration issues**:

```bash
# Validate JSON before migration
python scripts/validate_skills_migration.py --json-only

# Run with verbose output
python scripts/migrate_skills_to_sessionbuddy.py --verbose

# Rollback if needed
python scripts/rollback_skills_migration.py --force
```

### See Also

- **CLAUDE.md**: Complete developer documentation with integration examples
- **`docs/features/SKILLS_INTEGRATION.md`**: Detailed feature documentation
- **`scripts/migrate_skills_to_sessionbuddy.py`**: Migration tool source code

## Core Workflow

**Enhanced three-stage quality enforcement with intelligent code cleaning:**

1. **Fast Hooks** (~5 seconds): Essential formatting and security checks
1. **🧹 Code Cleaning Stage** (between fast and comprehensive): AI-powered cleanup for optimal comprehensive hook results
1. **Comprehensive Hooks** (~30 seconds): Complete static analysis on cleaned code

**Optimal Execution Order**:

- **Fast hooks first** # → **retry once if any fail** (formatting fixes cascade to other issues)
- **Code cleaning** # → Remove TODO detection, apply standardized patterns
- **Post-cleaning fast hooks sanity check** # → Ensure cleaning didn't introduce issues
- **Full test suite** # → Collect ALL test failures (don't stop on first)
- **Comprehensive hooks** # → Collect ALL quality issues on clean codebase
- **AI batch fixing** # → Process all collected issues intelligently

**With AI integration:**

- `--ai-fix` flag enables automatic error resolution with specialized sub-agents
- MCP server allows AI agents to run crackerjack commands with real-time progress tracking
- Structured error output for programmatic fixes with confidence scoring
- Advanced-grade regex pattern system ensures safe automated text transformations

## Core Features

### Project Management

- **Effortless Project Setup:** Initializes new Python projects with a standard directory structure, `pyproject.toml`, and essential configuration files
- **UV Integration:** Manages dependencies and virtual environments using [UV](https://github.com/astral-sh/uv) for lightning-fast package operations
- **Dependency Management:** Automatically detects and manages project dependencies

### Code Quality

- **Automated Code Cleaning:** Removes unnecessary docstrings, line comments, and trailing whitespace
- **Consistent Code Formatting:** Enforces a unified style using [Ruff](https://github.com/astral-sh/ruff), the lightning-fast Python linter and formatter
- **Comprehensive Quality Hooks:** Direct tool invocation with no wrapper overhead - runs Python tools, Rust analyzers, and security scanners efficiently
- **Interactive Checks:** Supports interactive quality checks (like `refurb`, `bandit`, and `pyright`) to fix issues in real-time
- **Static Type Checking:** Enforces type safety with Pyright integration

### Testing & Coverage Ratchet System

- **Built-in Testing:** Automatically runs tests using `pytest` with intelligent parallelization
- **Coverage Ratchet:** Revolutionary coverage system that targets 100% - coverage can only increase, never decrease
- **Milestone Celebrations:** Progress tracking with milestone achievements (15%, 20%, 25%... # → 100%)
- **No Arbitrary Limits:** Replaced traditional hard limits with continuous improvement toward perfection
- **Visual Progress:** Rich terminal displays showing journey to 100% coverage
- **Benchmark Testing:** Performance regression detection and monitoring
- **Easy Version Bumping:** Provides commands to bump the project version (patch, minor, or major)
- **Simplified Publishing:** Automates publishing to PyPI via UV with enhanced authentication

#### Coverage Ratchet Philosophy

🎯 **Target: 100% Coverage** - Not an arbitrary number, but true comprehensive testing
📈 **Continuous Improvement** - Each test run can only maintain or improve coverage
🏆 **Milestone System** - Celebrate achievements at 15%, 25%, 50%, 75%, 90%, and 100%
🚫 **No Regression** - Once you achieve a coverage level, you can't go backward

```bash
# Show coverage progress
python -m crackerjack run --coverage-report

# Run tests with ratchet system
python -m crackerjack run --run-tests

# Example output:
# 🎉 Coverage improved from 10.11% to 15.50%!
# 🏆 Milestone achieved: 15% coverage!
# 📈 Progress: [███░░░░░░░░░░░░░░░░░] 15.50% # → 100%
# 🎯 Next milestone: 20% (+4.50% needed)
```

### Git Integration

- **Intelligent Commit Messages:** Analyzes git changes and suggests descriptive commit messages based on file types and modifications
- **Commit and Push:** Commits and pushes your changes with standardized commit messages
- **Pull Request Creation:** Creates pull requests to upstream repositories on GitHub or GitLab
- **Git Hook Integration:** Ensures code quality before commits with fast, direct tool execution

## ⚡ legacy Architecture & Performance

![Oneiric Workflow DAG](docs/diagrams/oneiric-workflow-dag.png)
*Complete execution pipeline: CLI → Workflow Selection → Fast/Comprehensive Hooks → Tests → AI Batch Fixing*

Crackerjack is built on the **legacy DI framework** framework, providing advanced-grade dependency injection, intelligent caching, and parallel execution.

### What is legacy?

[legacy](https://github.com/lesleslie/crackerjack) is a lightweight dependency injection framework that enables:

- **Module-level registration** via `depends.set()` for clean dependency management
- **Runtime-checkable protocols** ensuring type safety across all components
- **Async-first design** with lifecycle management and timeout strategies
- **Clean separation of concerns** through adapters, orchestrators, and services

### Architecture Overview

**legacy Workflow Engine (Default since Phase 4.2)**

```
User Command # → BasicWorkflowEngine (legacy)
    ↓
Workflow Selection (Standard/Fast/Comprehensive/Test)
    ↓
Action Handlers (run_fast_hooks, run_code_cleaning, run_comprehensive_hooks, run_test_workflow)
    ↓
asyncio.to_thread() for non-blocking execution
    ↓
WorkflowPipeline (DI-injected via context)
    ↓
Phase Execution (_run_fast_hooks_phase, _run_comprehensive_hooks_phase, etc.)
    ↓
HookManager + TestManager (Manager Layer: 80% compliant)
    ↓
Direct adapter.check() calls (No subprocess overhead)
    ↓
ToolProxyCacheAdapter (Content-based caching, 70% hit rate)
    ↓
Parallel Execution (Up to 11 concurrent adapters)
    ↓
Results Aggregation with real-time console output
```

**Legacy Orchestrator Path** (opt-out with `--use-legacy-orchestrator`)

```
User Command # → WorkflowOrchestrator (Legacy)
    ↓
SessionCoordinator (@depends.inject + protocols)
    ↓
PhaseCoordinator (Orchestration Layer)
    ↓
HookManager + TestManager
    ↓
[Same execution path as legacy from here...]
```

**Architecture Compliance (Phase 2-4.2 Audit Results)**

| Layer | Compliance | Status | Notes |
|-------|-----------|--------|-------|
| **legacy Workflows** | 95% | ✅ Production | **Default since Phase 4.2** - Real-time output, non-blocking |
| **CLI Handlers** | 90% | ✅ Excellent | Gold standard: `@depends.inject` + `Inject[Protocol]` |
| **Services** | 95% | ✅ Excellent | Phase 3 refactored, consistent constructors |
| **Managers** | 80% | ✅ Good | Protocol-based injection, minor improvements needed |
| **Legacy Orchestration** | 70% | ⚠️ Opt-out | Available with `--use-legacy-orchestrator` |
| **Coordinators** | 70% | ⚠️ Mixed | Phase coordinators ✅, async needs standardization |
| **Agent System** | 40% | 📋 Legacy | Uses `AgentContext` pattern (predates legacy) |

**Key Architectural Patterns**

```python
# ✅ GOLD STANDARD Pattern (from CLI Handlers)
from legacy.depends import depends, Inject
from crackerjack.models.protocols import Console


@depends.inject
def setup_environment(console: Inject[Console] = None, verbose: bool = False) -> None:
    """Protocol-based injection with @depends.inject decorator."""
    console.print("[green]Environment ready[/green]")


# ❌ ANTI-PATTERN: Avoid manual fallbacks
def setup_environment_wrong(console: Console | None = None):
    self.console = console or Console()  # Bypasses DI container
```

### Performance Benefits

| Metric | Legacy | legacy Workflows (Phase 4.2) | Improvement |
|--------|--------|----------------------------|-------------|
| **Fast Hooks** | ~45s | ~48s | Comparable |
| **Full Workflow** | ~60s | ~90s | Real-time output |
| **Console Output** | Buffered | **Real-time streaming** | UX improvement |
| **Event Loop** | Sync (blocking) | **Async (non-blocking)** | Responsive |
| **Cache Hit Rate** | 0% | **70%** | New capability |
| **Concurrent Adapters** | 1 | **11** | 11x parallelism |
| **DI Context** | Manual | **Protocol-based injection** | Type safety |

### Core Components

#### 1. Quality Assurance Adapters

**Location:** `crackerjack/adapters/`

legacy-registered adapters for all quality checks:

- **Format:** Ruff formatting, mdformat
- **Lint:** Codespell, complexity analysis
- **Security:** Bandit security scanning, Gitleaks secret detection
- **Type:** Zuban type checking (20-200x faster than Pyright)
- **Refactor:** Creosote (unused dependencies), Refurb (Python idioms)
- **Complexity:** Complexipy analysis
- **Utility:** Various validation checks
- **AI:** Claude integration for intelligent auto-fixing

#### 2. Hook Orchestrator

**Location:** `crackerjack/orchestration/hook_orchestrator.py`

Features:

- **Dual execution mode:** Legacy (pre-commit CLI) + legacy (direct adapters)
- **Dependency resolution:** Intelligent hook ordering (e.g., format before lint)
- **Adaptive strategies:** Fast, comprehensive, or dependency-aware execution
- **Graceful degradation:** Timeout strategies prevent hanging

#### 3. Cache Adapters

**Location:** `crackerjack/orchestration/cache/`

Two caching strategies:

- **ToolProxyCache:** Content-based caching with file hash verification
- **MemoryCache:** In-memory LRU cache for testing

Benefits:

- **70% cache hit rate** in typical workflows
- **Content-aware invalidation:** Only re-runs when files actually change
- **Configurable TTL:** Default 3600s (1 hour)

#### 4. MCP Server Integration

**Location:** `crackerjack/mcp/`

legacy-registered services:

- **MCPServerService:** FastMCP server for AI agent integration
- **ErrorCache:** Pattern tracking for AI fix recommendations
- **JobManager:** WebSocket job tracking and progress streaming
- **WebSocketSecurityConfig:** Security hardening (localhost-only, rate limiting)

### Mi