ctx-gen 0.3.0
An intelligent code context generator that converts project structure and file contents into LLM-friendly formats.
License: MIT
Requires Python: >=3.8
Keywords: ai, artificial intelligence, chatgpt, claude, cli, code analysis, code context, code generation, context generator, copilot, dev productivity, developer tools, file contents, gpt, large language model, llm, lmarena.ai, markdown, project structure, prompt, prompt engineering, text formatting

# ctx-gen 🚀

[![Python Version](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![GitHub Stars](https://img.shields.io/github/stars/foxmcp/ctx-gen.svg)](https://github.com/foxmcp/ctx-gen/stargazers)
[![Issues](https://img.shields.io/github/issues/foxmcp/ctx-gen.svg)](https://github.com/foxmcp/ctx-gen/issues)

An intelligent code context generator that converts project structures and file contents into LLM-friendly formats. Perfect for seamlessly sharing your codebase with AI assistants like ChatGPT, Claude, and other Large Language Models.

## ✨ Features

- 🔍 **Smart Scanning**: Intelligently scans directories with customizable ignore rules
- 📊 **Token Estimation**: Calculates token counts using OpenAI's tiktoken library
- 🎯 **Binary Detection**: Automatically detects and handles binary files
- 📋 **Clipboard Ready**: One-click copy to clipboard for instant LLM sharing
- 🎨 **Multiple Formats**: Supports Standard, Markdown, JSON, and XML output formats
- 🚀 **Performance**: Fast parallel processing with progress indicators
- 🔧 **Highly Configurable**: Flexible configuration through CLI options or config files
- 📈 **Project Analytics**: Built-in statistics and analysis tools

## 📦 Installation

### Using pip

```bash
pip install ctx-gen
```

### From source

```bash
git clone https://github.com/foxmcp/ctx-gen.git
cd ctx-gen
pip install -e .
```

## 🚀 Quick Start

### Basic Usage

```bash
# Scan current directory and copy to clipboard
ctx-gen generate

# Scan specific directory
ctx-gen generate /path/to/project

# Save output to file
ctx-gen generate -o context.md

# Show only tree structure
ctx-gen generate --tree-only
```

### Initialize Project Configuration

```bash
# Interactive setup
ctx-gen init

# This creates:
# - .ctxgenrc: Configuration file
# - .promptignore: Ignore rules file
```

## 📖 Commands

### `generate` - Main Command

Generate project context for LLM consumption.

```bash
ctx-gen generate [OPTIONS] [TARGET_DIR]
```

**Options:**

- `-i, --ignore-file PATH`: Specify custom ignore file
- `--no-ignore`: Disable all ignore rules
- `-d, --use-default-ignore`: Force use built-in default ignore rules
- `--skip-binary/--include-binary`: Skip or include binary files (default: skip)
- `-a, --include-hidden`: Include hidden files and directories
- `-s, --max-file-size INT`: Maximum file size in bytes (default: 1MB)
- `-t, --max-tokens INT`: Maximum total tokens (default: 100,000)
- `--tokenizer MODEL`: Choose tokenizer model (cl100k_base/p50k_base/r50k_base)
- `-f, --format FORMAT`: Output format (standard/markdown/json/xml)
- `-o, --output PATH`: Save to file instead of clipboard
- `--tree-only`: Show only file tree structure
- `--show-skipped`: Display list of skipped files
- `--save-config`: Save current settings to .ctxgenrc

### `init` - Initialize Configuration

```bash
ctx-gen init [--interactive/--no-interactive]
```

Create project configuration files with interactive setup wizard.

### `stats` - Project Statistics

```bash
ctx-gen stats [TARGET_DIR]
```

Display detailed project statistics including file types, sizes, and distributions.

### `version` - Version Information

```bash
ctx-gen version
```

## ⚙️ Configuration

### Priority Order

ctx-gen follows this priority order for configuration:

1. CLI flags (highest priority)
2. `.ctxgenrc` configuration file
3. `.promptignore` / `.ctxignore` files
4. `.gitignore` file
5. Built-in defaults (lowest priority)

### .promptignore File

Uses gitignore syntax to exclude files:

```gitignore
# Dependencies
node_modules/
venv/
*.pyc

# Build artifacts
dist/
build/
*.o

# Sensitive files
.env
*.key
secrets/

# Large files
*.mp4
*.zip
data/
```

### .ctxgenrc Configuration

JSON configuration file for project settings:

```json
{
  "max_file_size": 1048576,
  "max_total_tokens": 100000,
  "skip_binary": true,
  "include_hidden": false,
  "tokenizer_model": "cl100k_base",
  "output_format": "markdown"
}
```

## 📊 Output Formats

### Standard Format

Plain text with clear file separators, ideal for direct LLM input.

### Markdown Format

Formatted with syntax highlighting, perfect for documentation and readable sharing.

### JSON Format

Structured data format for programmatic processing.

### XML Format

Hierarchical format for systems requiring XML input.

## 🎯 Use Cases

- **AI Code Reviews**: Share your entire codebase with ChatGPT or Claude for comprehensive reviews
- **Documentation Generation**: Create instant project overviews for documentation
- **Code Migration**: Provide context for AI-assisted code refactoring or migration
- **Learning & Teaching**: Share complete project context for educational purposes
- **Debugging Assistance**: Give LLMs full context for better debugging help

## 💡 Examples

### Example 1: Prepare Context for ChatGPT

```bash
# Generate markdown format optimized for ChatGPT
ctx-gen generate --format markdown --max-tokens 50000
```

### Example 2: Analyze Large Project

```bash
# Show statistics first
ctx-gen stats ./my-project

# Generate with custom limits
ctx-gen generate ./my-project -s 2097152 -t 150000 --show-skipped
```

### Example 3: Include Specific Files Only

Create a `.promptignore` file:

```gitignore
# Ignore everything
*

# But include specific patterns
!src/**/*.py
!tests/**/*.py
!README.md
!requirements.txt
```

Then run:

```bash
ctx-gen generate
```

## 🛠️ Advanced Features

### Token Estimation

ctx-gen uses OpenAI's tiktoken library to accurately estimate token counts for different models:

- **cl100k_base**: GPT-4, GPT-3.5-turbo, text-embedding-ada-002
- **p50k_base**: Codex models
- **r50k_base**: GPT-3 models (davinci, curie, babbage, ada)

### Binary File Detection

Intelligent detection system that:

- Checks file extensions against known binary formats
- Analyzes file content for binary signatures
- Handles edge cases with multiple encoding attempts

### Performance Optimization

- Parallel file scanning
- Streaming processing for large files
- Memory-efficient handling of large projects
- Progress indicators for long operations

## 🤝 Contributing

We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request

## 🐛 Bug Reports & Feature Requests

Please use the [GitHub Issues](https://github.com/foxmcp/ctx-gen/issues) page to report bugs or request features.

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## 📧 Contact

- **GitHub**: [https://github.com/foxmcp/ctx-gen](https://github.com/foxmcp/ctx-gen)
- **Email**: [ctx-gen@foxmcp.com](mailto:ctx-gen@foxmcp.com)
- **Issues**: [GitHub Issues](https://github.com/foxmcp/ctx-gen/issues)

## 🙏 Acknowledgments

- Built with [Typer](https://typer.tiangolo.com/) for elegant CLI interactions
- Uses [Rich](https://rich.readthedocs.io/) for beautiful terminal formatting
- Token counting powered by [tiktoken](https://github.com/openai/tiktoken)
- Ignore patterns handled by [pathspec](https://github.com/cpburnz/python-pathspec)

## 📈 Roadmap

- [ ] Support for more LLM platforms and their specific formats
- [ ] Integration with popular IDEs (VSCode, PyCharm)
- [ ] Web interface for non-technical users
- [ ] Custom preprocessing rules for specific file types
- [ ] Incremental context updates for large projects
- [ ] Team collaboration features

---

<p align="center">
  Made with ❤️ by <a href="https://github.com/foxmcp">foxmcp</a>
</p>

<p align="center">
  <a href="https://github.com/foxmcp/ctx-gen/stargazers">⭐ Star us on GitHub!</a>
</p>