efficiency-tools 1.0.0
Spec-Driven Development workflow and Claude Agent SDK integration toolkit
Requires Python: <4.0,>=3.11
Keywords: claude, ai, agents, sdd, spec-driven-development, documentation

# Efficiency Tools

> Spec-Driven Development workflow and Claude Agent SDK integration toolkit

Quickly integrate SDD methodology, Claude agents, and intelligent document management into any existing codebase.

[![Python Version](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
[![Version](https://img.shields.io/badge/version-1.0.0-orange)](https://github.com/username/efficiency/releases)

## Features

✨ **Claude Agent SDK Integration**
- 🤖 Pre-configured agents (document-curator, personal-secretary, epic-validator, etc.)
- ⚡ Custom slash commands (/doc-cluster, /secretary, /gen-changedoc)
- 📝 Agent configuration templates

📚 **Intelligent Document Management**
- 🔍 Semantic clustering with ChromaDB vector embeddings
- 🏷️ TF-IDF based cluster naming (not path-based!)
- 🚫 `.docignore` support for excluding system files
- 🔗 Automatic link updates after reorganization

🎯 **Spec-Driven Development (SDD)**
- 📋 5-phase methodology (Constitution → Specify → Plan → Tasks → Implement)
- 💬 Chat simulation frameworks for each phase
- 📊 Session-based workflow persistence

🛠️ **Developer Experience**
- 📦 One-command installation
- 🔄 Smart configuration merging (preserves user customizations)
- ⬆️ Seamless upgrades with backup/rollback
- ✅ Installation verification tools

---

## Quick Start

### Option 1: UV (Recommended - Fastest)

```bash
# Install UV if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install and initialize efficiency-tools
uvx efficiency-tools init

# Or install to specific directory
uvx efficiency-tools init /path/to/project
```

### Option 2: PIPX (Standard)

```bash
# Install from GitHub
pipx install git+https://github.com/username/efficiency.git

# Initialize in current directory
efficiency init

# Or specify target
efficiency init /path/to/project
```

### Option 3: Shell Script (Zero Dependencies)

```bash
# One-liner installation
curl -fsSL https://raw.githubusercontent.com/user/efficiency/main/install.sh | bash

# Or download and run
wget https://raw.githubusercontent.com/user/efficiency/main/install.sh
bash install.sh /path/to/project
```

### Option 4: Poetry (For Poetry Projects)

```bash
# Add as dev dependency
poetry add --group dev efficiency-tools

# Run installation
poetry run efficiency init
```

---

## Installation

### Interactive Installation

The default installation is interactive and guides you through component selection:

```bash
efficiency init
```

You'll be prompted to select:
- ✅ Claude Agent SDK Agents
- ✅ Slash Commands
- ✅ Document Management Scripts
- ✅ ChromaDB Integration
- ✅ Workflow Documentation
- ⬜ SDD Metaprompting Simulations (optional)

### Non-Interactive Installation

For automated/CI environments:

```bash
efficiency init \
  --no-interactive \
  --components agents \
  --components commands \
  --components clustering \
  --min-clusters 10 \
  --max-clusters 15
```

---

## What Gets Installed

### Directory Structure

```
your-project/
├── .claude/                  # Claude Code configuration
│   ├── agents/              # Agent definitions (document-curator, personal-secretary, etc.)
│   ├── commands/            # Slash commands (/doc-cluster, /secretary, etc.)
│   ├── docs/                # Reference documentation
│   └── settings.local.json  # User settings (merged, not overwritten)
│
├── .efficiency/             # Efficiency tools installation
│   ├── cluster_documents.py # Semantic clustering script
│   ├── docs/                # Documentation templates
│   ├── sdd/                 # SDD chat simulations (optional)
│   ├── quickstart.sh        # Quick start guide
│   └── config.json          # Installation configuration
│
├── .docindex/               # ChromaDB vector database
│   └── chroma/             # Document embeddings storage
│
├── .docignore               # Exclusion patterns for document indexing
│
└── CLAUDE.md                # Project instructions for Claude (template)
```

### Components

| Component | Description | Files |
|-----------|-------------|-------|
| **Claude Agents** | Pre-configured agents for document management, development assistance | `.claude/agents/*` |
| **Slash Commands** | Custom commands accessible in Claude Code | `.claude/commands/*` |
| **Document Scripts** | Semantic clustering and indexing tools | `.efficiency/cluster_documents.py` |
| **ChromaDB** | Vector database for document embeddings | `.docindex/chroma/` |
| **SDD Simulations** | Chat simulations for each SDD phase | `.efficiency/sdd/*` |
| **Documentation** | Templates and guides | `.efficiency/docs/*` |

---

## Usage

### 1. Index Your Documentation

```bash
# Run semantic clustering
poetry run python .efficiency/cluster_documents.py

# Or using wrapper script (from shell installer)
./efficiency python .efficiency/cluster_documents.py
```

This will:
- ✅ Index all markdown files in your project
- ✅ Generate semantic clusters (10-15 groups)
- ✅ Create human-readable cluster names using TF-IDF
- ✅ Exclude files matching `.docignore` patterns
- ✅ Generate `CLUSTERING_ANALYSIS_REPORT.md`

### 2. Use Slash Commands in Claude Code

Open Claude Code and use the following commands:

```
/doc-cluster       - Run semantic clustering on your docs
/secretary         - Activate personal secretary agent
/gen-changedoc     - Generate change documentation from git diff
/debug-hypothesis  - Validate EPIC hypotheses with lean testing
/context7-api-check - Check for outdated API usage patterns
```

### 3. Run SDD Chat Simulations

```bash
# Constitution phase (establish architectural principles)
poetry run python .efficiency/sdd/constitution-chat-simulation.py

# Plan phase (create implementation roadmap)
poetry run python .efficiency/sdd/plan-chat-simulation.py

# Tasks phase (break down into executable units)
poetry run python .efficiency/sdd/tasks-chat-simulation.py

# Implement phase (TDD execution)
poetry run python .efficiency/sdd/implement-chat-simulation.py
```

### 4. Verify Installation

```bash
efficiency verify

# Example output:
# ✅ PASS - Claude config
# ✅ PASS - Agents
# ✅ PASS - Commands
# ✅ PASS - ChromaDB
# ✅ PASS - Efficiency scripts
# ✅ PASS - cluster_documents.py
# ✅ PASS - .docignore
```

---

## CLI Reference

### Commands

```bash
efficiency init [TARGET_DIR]
  Initialize Efficiency tools in a project directory

efficiency verify [TARGET_DIR]
  Verify installation integrity

efficiency upgrade [TARGET_DIR] [--version VERSION] [--dry-run]
  Upgrade to a newer version

efficiency check-updates [TARGET_DIR]
  Check for available updates

efficiency quickstart
  Display quick start guide and common commands
```

### Options

```
--interactive / --no-interactive
  Enable/disable interactive configuration wizard

--components [agents|commands|sdd|clustering|docs]
  Components to install (repeatable, non-interactive mode)

--min-clusters N
  Minimum number of clusters for document clustering (default: 10)

--max-clusters N
  Maximum number of clusters for document clustering (default: 15)

--version
  Show version and exit

--help
  Show help message
```

---

## Configuration

### .docignore Patterns

The `.docignore` file uses gitignore-style patterns to exclude files from indexing:

```
# System directories
.git/
.docindex/
.claude/
node_modules/
__pycache__/

# Build artifacts
dist/
build/
.next/
coverage/

# Generated reports
*_REPORT.md
*_ANALYSIS.md
CLUSTERING_*.md

# Virtual environments
.venv/
venv/
```

**Test exclusion patterns:**

```bash
# Check if file is ignored (from installed tools)
cat > test_docignore.py << 'EOF'
import sys
sys.path.insert(0, '.efficiency/document-curator')
from docignore_parser import DocIgnoreParser

parser = DocIgnoreParser('.')
parser.load_rules()

test_path = sys.argv[1]
if parser.should_ignore(test_path):
    print(f"✅ Excluded: {test_path}")
else:
    print(f"❌ Not excluded: {test_path}")
EOF

python test_docignore.py .claude/agents/test.md
# Output: ✅ Excluded: .claude/agents/test.md
```

### Clustering Configuration

Edit `.efficiency/cluster_documents.py` to customize clustering:

```python
# Lines 25-29: Cluster count range
MIN_CLUSTERS = 10  # Minimum number of clusters
MAX_CLUSTERS = 15  # Maximum number of clusters

# Lines 216-223: TF-IDF parameters
vectorizer = TfidfVectorizer(
    max_features=50,      # Top N keywords to consider
    min_df=1,             # Minimum document frequency
    max_df=0.9,           # Maximum document frequency
    ngram_range=(1, 3),   # Use unigrams, bigrams, trigrams
)
```

---

## Upgrading

### Check for Updates

```bash
efficiency check-updates

# Output:
# 📦 Update available!
# Current version: 1.0.0
# Latest version: 1.2.0
```

### Upgrade to Latest

```bash
# Preview changes (dry-run)
efficiency upgrade --dry-run

# Upgrade to latest version
efficiency upgrade

# Upgrade to specific version
efficiency upgrade --version 1.2.0
```

Upgrades automatically:
- ✅ Create backup of current configuration
- ✅ Merge new agent configurations
- ✅ Preserve user customizations
- ✅ Update dependencies
- ✅ Provide rollback path if needed

---

## Examples

### Use Case 1: Organize 100+ Markdown Docs

```bash
# 1. Install efficiency-tools
uvx efficiency-tools init

# 2. Index and cluster documents
poetry run python .efficiency/cluster_documents.py

# Expected output:
# ✓ Retrieved 127 documents
# ✓ Excluded 28 system files (.claude/, __pycache__/)
# ✓ Optimal clusters: 14 (silhouette score: 0.2238)
# ✓ Report saved: CLUSTERING_ANALYSIS_REPORT.md

# 3. Review clustering report
cat CLUSTERING_ANALYSIS_REPORT.md | head -50

# Example clusters:
# - Agent Architecture / SDK Documentation (12 files)
# - Epic Debugging & Hypothesis Validation (8 files)
# - Workflow Setup & Configuration (6 files)
```

### Use Case 2: Add to Existing Poetry Project

```bash
# 1. Add as dev dependency
poetry add --group dev efficiency-tools

# 2. Initialize in project root
poetry run efficiency init

# 3. Select components interactively
# ✅ Claude Agent SDK Agents
# ✅ Slash Commands
# ✅ Document Management Scripts
# ⬜ SDD Simulations (skip for now)

# 4. Verify installation
poetry run efficiency verify

# 5. Index documentation
poetry run python .efficiency/cluster_documents.py
```

### Use Case 3: CI/CD Integration

```yaml
# .github/workflows/docs.yml
name: Document Clustering

on:
  push:
    paths:
      - 'docs/**'
      - '**.md'

jobs:
  cluster:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install UV
        run: curl -LsSf https://astral.sh/uv/install.sh | sh

      - name: Install Efficiency Tools
        run: uvx efficiency-tools init --no-interactive --components clustering

      - name: Run clustering
        run: poetry run python .efficiency/cluster_documents.py

      - name: Commit clustering report
        run: |
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          git add CLUSTERING_ANALYSIS_REPORT.md
          git commit -m "Update document clustering report"
          git push
```

---

## Troubleshooting

### Issue 1: "Collection 'project_docs' not found"

**Solution:**
```bash
# Re-initialize ChromaDB
poetry run python -c "
import chromadb
client = chromadb.PersistentClient(path='.docindex/chroma')
client.create_collection('project_docs')
print('✅ Collection created')
"
```

### Issue 2: "ModuleNotFoundError: No module named 'chromadb'"

**Solution:**
```bash
# Install dependencies
poetry install

# Or if using venv from shell installer
source .efficiency/.venv/bin/activate
pip install chromadb scikit-learn
```

### Issue 3: System files included in clustering

**Solution:**
```bash
# Check .docignore rules
cat .docignore | grep ".claude"

# Add missing patterns
echo ".claude/" >> .docignore

# Re-run clustering
poetry run python .efficiency/cluster_documents.py
```

---

## Documentation

- 📘 [Document Agents Quick Start Guide](.efficiency/docs/DOCUMENT_AGENTS_QUICK_START_GUIDE.md)
- 📗 [Installation Script Ideas](.efficiency/docs/INSTALLATION_SCRIPT_IDEAS_OUTLINE.md)
- 📕 [CLAUDE.md Template](CLAUDE.md)
- 📙 [Agent Configuration Guide](.claude/docs/)

---

## Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

---

## License

MIT License - see [LICENSE](LICENSE) file for details

---

## Support

- 📧 Email: brent.tudas@gmail.com
- 🐛 Issues: [GitHub Issues](https://github.com/username/efficiency/issues)
- 💬 Discussions: [GitHub Discussions](https://github.com/username/efficiency/discussions)

---

## Roadmap

### v1.1 (Next)
- [ ] Web UI for cluster visualization
- [ ] VSCode extension for agent management
- [ ] Plugin system for custom agents
- [ ] Cloud-hosted ChromaDB support

### v1.2 (Future)
- [ ] Auto-update service with GitHub webhooks
- [ ] Team collaboration features
- [ ] Agent marketplace
- [ ] IntelliJ/Cursor integrations

---

**Made with ❤️ using Claude Code and the Efficiency methodology**