llmpromptvault 0.1.0
Version, compare, and manage your LLM prompts. No API keys required. Bring your own model.
License: MIT
Requires Python: >=3.9
Keywords: llm, prompts, prompt-engineering, prompt-management, versioning, mlops, ai, comparison, testing

# PromptVault 🔐

> **Version and compare your LLM prompts. No API key required. Bring your own model.**

[![PyPI version](https://badge.fury.io/py/promptvault.svg)](https://pypi.org/project/promptvault/)
[![Python](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

---

## What PromptVault Does

PromptVault is a **prompt management library** — not an LLM client. It does three things:

1. **Versions your prompts** — tracks every change like Git
2. **Logs responses** — stores what each prompt returned, with latency and token counts
3. **Compares prompts** — shows you side-by-side how two prompt versions perform

**PromptVault never calls any LLM.** You call your own model however you like — OpenAI, Claude, Gemini, Ollama, a local model, anything. You pass the response to PromptVault and it handles the rest.

---

## Installation

```bash
pip install promptvault
```

Only one dependency: `pyyaml`. Everything else uses Python's standard library.

---

## Quick Start

```python
from promptvault import Prompt, Compare

# 1. Define two versions of a prompt
v1 = Prompt("summarize", template="Summarize this: {text}", version="v1")
v2 = Prompt("summarize", template="Summarize in 3 bullet points: {text}", version="v2")

# 2. YOU call your LLM (any model, any way you like)
r1 = your_llm(v1.render(text="Some article content..."))
r2 = your_llm(v2.render(text="Some article content..."))

# 3. PromptVault compares and logs them
cmp = Compare(v1, v2)
cmp.log(r1, r2)
cmp.show()
```

Output:
```
────────────────────────────────────────────────────────────
  PROMPTVAULT COMPARISON
────────────────────────────────────────────────────────────
  Prompt A                       summarize (v1)
  Prompt B                       summarize (v2)
────────────────────────────────────────────────────────────

  ── Response A ──
  Here is a summary of the article...

  ── Response B ──
  • Key point one
  • Key point two
  • Key point three

────────────────────────────────────────────────────────────
  Metric                           Prompt A     Prompt B
  ──────────────────────────── ──────────── ────────────
  Word count                             12           18
  Char count                             68          112
  Latency (ms)                        820.0        950.0
  Tokens                                 45           62
────────────────────────────────────────────────────────────
```

---

## Core API

### `Prompt` — Define and version your prompts

```python
from promptvault import Prompt

# Create a prompt
p = Prompt(
    name="classify",
    template="Classify this text as positive or negative: {text}",
    version="v1",
    description="Sentiment classifier",
    tags=["classify", "sentiment"],
)

# See required variables
p.variables()         # ['text']

# Render it (just string formatting — no LLM call)
rendered = p.render(text="I love this product!")

# YOU call your LLM
response = your_llm(rendered)

# Log the response
p.log(
    rendered_prompt=rendered,
    response=response,
    model="gpt-4o-mini",        # optional
    latency_ms=820,             # optional
    tokens=45,                  # optional
)

# View stats across all logged runs
p.stats()
# {
#   'run_count': 10,
#   'avg_latency_ms': 750.0,
#   'avg_tokens': 42.0,
#   'models_used': ['gpt-4o-mini'],
#   ...
# }

# View raw run history
p.runs(last_n=10)
```

### Versioning

```python
# Create v2 — v1 is automatically preserved in history
v2 = p.update(
    new_template="You are a sentiment expert. Classify as positive/negative/neutral: {text}"
)
# Auto-increments: v1 → v2. Or pass version="my-version"

# See all versions
p.history()   # [{'version': 'v1', 'template': ..., ...}, {'version': 'v2', ...}]
```

### Save & Load YAML

```python
# Export to a human-readable file
p.save("prompts/classify.yaml")

# Load it back anywhere
p = Prompt.load("prompts/classify.yaml")
```

```yaml
# prompts/classify.yaml
name: classify
version: v1
description: Sentiment classifier
template: 'Classify this text as positive or negative: {text}'
tags:
  - classify
  - sentiment
```

---

### `Compare` — Side-by-side prompt comparison

```python
from promptvault import Compare

cmp = Compare(v1, v2)

# Log one comparison pair
cmp.log(
    response_a=response_v1,
    response_b=response_v2,
    model="llama3",           # optional
    latency_ms_a=820,         # optional
    latency_ms_b=950,         # optional
    tokens_a=45,              # optional
    tokens_b=62,              # optional
)

# Print side-by-side to terminal
cmp.show()

# Get structured diff dict
cmp.diff()
# {
#   'response_a': '...', 'response_b': '...',
#   'words_a': 12, 'words_b': 18,
#   'latency_ms_a': 820, 'latency_ms_b': 950,
#   ...
# }

# Aggregate summary across multiple comparison runs
cmp.summary()
# {
#   'total_comparisons': 5,
#   'avg_words_a': 12.0, 'avg_words_b': 18.2,
#   'avg_latency_ms_a': 810.0, 'avg_latency_ms_b': 940.0,
#   ...
# }
```

---

### `Registry` — Share prompts with your team

```python
from promptvault import Registry

reg = Registry("./shared_prompts")   # any folder path

# Push prompts in
reg.push(v1)
reg.push(v2)

# Pull them out anywhere
p = reg.pull("classify")           # latest version
p = reg.pull("classify", "v1")     # specific version

reg.list()                 # ['classify', 'summarize', ...]
reg.versions("classify")   # ['v1', 'v2']
reg.delete("classify", "v1")
```

---

## Works with Any LLM

Because PromptVault never makes LLM calls itself, it works with literally any model:

```python
# OpenAI
import openai
client = openai.OpenAI(api_key="...")
response = client.chat.completions.create(model="gpt-4o", messages=[{"role": "user", "content": rendered}])
p.log(rendered, response.choices[0].message.content, model="gpt-4o")

# Anthropic
import anthropic
client = anthropic.Anthropic(api_key="...")
response = client.messages.create(model="claude-haiku-4-5-20251001", max_tokens=1024, messages=[{"role": "user", "content": rendered}])
p.log(rendered, response.content[0].text, model="claude-haiku-4-5-20251001")

# Ollama (local, free, no key)
import requests
response = requests.post("http://localhost:11434/api/generate", json={"model": "llama3", "prompt": rendered, "stream": False})
p.log(rendered, response.json()["response"], model="llama3")

# Any other LLM, API, or local model
response = any_llm_you_want(rendered)
p.log(rendered, response)
```

---

## File Structure

```
your_project/
├── prompts/
│   └── classify.yaml       ← exported prompt templates
├── .promptvault/
│   ├── history.json         ← version history
│   └── runs.db              ← run logs (SQLite)
└── main.py
```

Add `.promptvault/` to `.gitignore` to keep run logs local, or commit it to share analytics with your team.

---

## Run the Demo

```bash
git clone https://github.com/your-username/promptvault
cd promptvault
pip install pyyaml
python examples/demo.py
```

No API key needed — the demo uses simulated responses.

---

## Publish to PyPI

```bash
pip install build twine
python -m build
twine upload dist/*
```

See `PUBLISHING_GUIDE.md` for the full step-by-step.

---

## License

MIT © PromptVault Contributors