ragnet-mcp 0.2.0
Documentation RAG pipeline with a Textual dashboard and MCP server
Requires Python: >=3.10

# RAGNet MCP

**Give Claude access to any documentation you want.**

RAGNet crawls documentation websites, stores them in a searchable database, and lets Claude search through them when you're coding. Instead of copy-pasting docs into your prompts, Claude can look things up itself.

---

## How It Works (Plain English)

```
┌─────────────────┐      stdio      ┌─────────────────┐
│   Claude Code   │ ◄─────────────► │   RAGNet MCP    │
│   (or Desktop)  │   (messages)    │   (this tool)   │
└─────────────────┘                 └────────┬────────┘
                                             │
                                    HTTP     │
                                             ▼
                                    ┌─────────────────┐
                                    │     Qdrant      │
                                    │ (vector database)│
                                    └─────────────────┘
```

**Three pieces:**

1. **Claude** (the AI) - asks questions like "how do I use AsyncWebCrawler?"
2. **RAGNet MCP** (this project) - receives the question, searches the database, returns relevant docs
3. **Qdrant** (vector database) - stores all the documentation chunks and finds similar content

**Important:** RAGNet is NOT a web server. It's a subprocess that Claude spawns and talks to via stdin/stdout (like two programs chatting through a pipe). Qdrant is the only actual server running on a port.

---

## Quick Start

### What You'll Need

- **Python 3.10 or newer** - [Download here](https://www.python.org/downloads/)
- **Docker Desktop** - [Download here](https://www.docker.com/products/docker-desktop/) (for running Qdrant)
- **OpenAI API key** - [Get one here](https://platform.openai.com/api-keys) (for generating embeddings)

### Install from PyPI (Recommended)

```bash
pip install ragnet-mcp
ragnet init
```

### Install from Source

```bash
git clone https://github.com/orro3790/ragnet.git
cd ragnet
pip install -e .
ragnet init
```

The `ragnet init` command will:
1. Ask for your OpenAI API key
2. Auto-generate a Qdrant API key
3. Create the `.env` file
4. Start Qdrant via Docker
5. Create the database collection

### Crawl Documentation

```bash
ragnet dashboard
```

This opens the TUI where you can crawl documentation sources. Select a source from `crawlist/` and start indexing.

---

## CLI Reference

| Command | Description |
|---------|-------------|
| `ragnet init` | First-time setup (API keys, Docker, collection) |
| `ragnet start` | Start Qdrant |
| `ragnet stop` | Stop Qdrant |
| `ragnet status` | Check service status |
| `ragnet dashboard` | Launch the TUI |
| `ragnet mcp` | Run the MCP server directly |
| `ragnet update` | Update to the latest version |

---

## Connect Claude to RAGNet

**For Claude Desktop:**

Edit your config file:
- **macOS:** `~/Library/Application Support/Claude/claude_desktop_config.json`
- **Windows:** `%APPDATA%\Claude\claude_desktop_config.json`

```json
{
  "mcpServers": {
    "ragnet-mcp": {
      "command": "C:/full/path/to/ragnet/venv/Scripts/python.exe",
      "args": ["C:/full/path/to/ragnet/ragnet.py"],
      "env": {
        "QDRANT_URL": "http://localhost:6333",
        "QDRANT__SERVICE__API_KEY": "your-qdrant-api-key"
      },
      "alwaysAllow": ["find_code_examples", "get_context_chain", "list_sources", "query"],
      "timeout": 30
    }
  }
}
```

> **Note:** `OPENAI_API_KEY` is loaded from the `.env` file in the ragnet directory. The `alwaysAllow` list pre-approves these tools so Claude doesn't ask for permission each time.

Restart Claude Desktop to see the RAGNet tools.

**For Claude Code:**

Add to your project's `.mcp.json`:

```json
{
  "mcpServers": {
    "ragnet-mcp": {
      "command": "C:/full/path/to/ragnet/venv/Scripts/python.exe",
      "args": ["C:/full/path/to/ragnet/ragnet.py"],
      "env": {
        "QDRANT_URL": "http://localhost:6333",
        "QDRANT__SERVICE__API_KEY": "your-qdrant-api-key"
      },
      "alwaysAllow": ["find_code_examples", "get_context_chain", "list_sources", "query"],
      "timeout": 30
    }
  }
}
```

> **Tip:** Use forward slashes in paths even on Windows, or escape backslashes (`C:\\Users\\...`).

---

## Using RAGNet

Once connected, Claude can use these tools:

| Tool | What it does | When to use it |
|------|--------------|----------------|
| `query` | Basic semantic search | Looking up specific things: "AsyncWebCrawler class", "BrowserConfig options" |
| `query_with_hyde` | Smarter conceptual search | Asking "how" or "why" questions: "how do I handle authentication?" |
| `find_code_examples` | Find code snippets | "Show me examples of using X" |
| `query_complex` | Multi-part questions | "What is X and how does it compare to Y?" |
| `get_context_chain` | Get surrounding chunks | When you need more context around a result |
| `list_sources` | See indexed docs | Check what documentation is available |

You don't need to call these manually - just ask Claude questions and it'll use the right tool.

---

## Adding Your Own Documentation

1. Create a file in `crawlist/` named `your-docs.md`
2. Add URLs (one per line) or a sitemap URL
3. Run `ragnet dashboard` and select your new source

Example `crawlist/my-library.md`:
```markdown
# My Library Docs

https://my-library.dev/docs/getting-started
https://my-library.dev/docs/api-reference
https://my-library.dev/docs/examples
```

---

## Troubleshooting

### Check status first
```bash
ragnet status
```
This shows if Docker, Qdrant, and the collection are properly configured.

### "Qdrant connection refused"
Make sure Docker is running:
```bash
ragnet start
```

### "OpenAI API error"
Check that your `OPENAI_API_KEY` in `.env` is valid and has credits.

### "No results found"
Run `ragnet dashboard` and crawl some documentation first.

### Claude doesn't see the tools
- For Claude Code: Make sure your `.mcp.json` is configured correctly
- For Claude Desktop: Restart the app after editing config

---

## Project Structure

```
ragnet/
├── cli.py                  # Unified CLI (ragnet command)
├── ragnet.py               # The MCP server (what Claude talks to)
├── dashboard.py            # Textual TUI for crawling/management
├── rag_pipeline.py         # Crawls and indexes documentation
├── config.py               # Configuration management
├── chunk_processor.py      # Splits docs into searchable chunks
├── crawlist/               # URL lists for documentation sources
├── qdrant_admin_utils/     # Database management scripts
├── docker-compose.yml      # Qdrant container config
└── venv/                   # Python virtual environment
```

---

## How the Search Works (For the Curious)

When you search, RAGNet can use multiple strategies:

1. **Dense search** (default): Converts your query to a vector and finds similar vectors. Good for semantic matching.

2. **Hybrid search**: Combines dense vectors with keyword matching (BM25). Better when exact terms matter.

3. **HyDE** (Hypothetical Document Embeddings): First generates what a good answer *would* look like, then searches for docs similar to that answer. Better for conceptual questions.

```
Your Question → [Generate hypothetical answer] → [Search for similar docs] → Results
```

---

## Using Qdrant Cloud (Optional)

Instead of running Qdrant locally with Docker, you can use their free cloud tier:

1. Sign up at [cloud.qdrant.io](https://cloud.qdrant.io)
2. Create a cluster
3. Get your URL and API key
4. Run init with `--skip-docker`:
   ```bash
   ragnet init --skip-docker --qdrant-key your-cloud-api-key
   ```
5. Update `.env` with your cloud URL:
   ```bash
   QDRANT_URL=https://your-cluster-id.aws.cloud.qdrant.io
   ```

---

## License

MIT