Skill: theshadowrose/token-tamer (version 1.1.0)

--- name: "Token Tamer — AI API Cost Control" description: "Monitor, budget, and optimize AI API spending across any provider. Tracks every call, enforces budgets, detects waste, provides optimization recommendations." author: "@TheShadowRose" version: "1.1.0" tags: ["cost-control", "budget", "optimization", "api-costs", "monitoring", "spending"] license: "MIT" --- # Token Tamer — AI API Cost Control Use this skill to track AI API costs, enforce spending budgets, and get optimization recommendations. Works with any provider: OpenAI, Anthropic, Gemini, Ollama, or custom. ## When to Use This Skill Invoke this skill when a user asks about: - "How much am I spending on API calls?" - Setting daily/weekly/monthly budget limits - Getting a cost breakdown by model or provider - Detecting wasteful API usage patterns - Receiving an alert when spending approaches a budget limit ## Setup 1. Copy `config_example.py` to `token_config.py` 2. Set your budget limits and model pricing 3. Configure your storage file path ```python # token_config.py USAGE_FILE = "./token_usage.json" BUDGETS = { 'daily': 10.00, 'weekly': 50.00, 'monthly': 150.00, } ALERT_THRESHOLDS = { 'warning': 80, # alert at 80% of budget 'critical': 95, # alert at 95% of budget } ``` ## Recording API Usage Call this after every API request to track the cost: ```python import token_config as config from token_tamer import TokenTamer tamer = TokenTamer(config) # Log a call — returns (cost, status) cost, status = tamer.log_usage( provider='anthropic', model='claude-sonnet-4-5', input_tokens=1500, output_tokens=800, task='summarize', # optional session='sess_001' # optional ) if status == 'KILL': raise Exception("Kill switch activated — daily budget exceeded") ``` ## Checking Budget Status ```python status = tamer.get_status() # Returns: { # 'daily': { 'cost': 2.40, 'budget': 10.00, 'status': 'OK', 'message': None }, # 'weekly': { 'cost': 8.10, 'budget': 50.00, 'status': 'OK', 'message': None }, # 'monthly': { 'cost': 8.10, 'budget': 150.00, 'status': 'OK', 'message': None }, # 'kill_switch': False # } if status['daily']['status'] == 'CRITICAL': print("Daily budget nearly exhausted — switch to cheaper model") ``` ## Pre-Call Budget Check ```python # Check before making an expensive call (returns True = OK, False = blocked) if not tamer.check_before_call(estimated_cost=0.10): raise Exception("Kill switch active — API calls blocked") ``` ## Cost Reports ```python import token_config as config from token_reports import ReportGenerator reporter = ReportGenerator(config) # Daily report report = reporter.generate_daily_report() reporter.print_report(report) # Weekly report report = reporter.generate_weekly_report() reporter.print_report(report) # Breakdown by provider report = reporter.generate_by_provider_report() reporter.print_report(report) # Breakdown by model report = reporter.generate_by_model_report() reporter.print_report(report) # Breakdown by task report = reporter.generate_by_task_report() reporter.print_report(report) ``` ## Waste Detection & Optimization ```python import token_config as config from token_optimizer import WasteDetector, Optimizer # Detect waste (last 7 days) detector = WasteDetector(config) waste = detector.detect_all(days=7) # Get optimization recommendations optimizer = Optimizer(config) recommendations = optimizer.generate_recommendations(days=7) ``` ## CLI Usage ```bash # Check current spend status python token_tamer.py --status # Log a call via CLI python token_tamer.py --log --provider openai --model gpt-4o \ --input-tokens 1500 --output-tokens 300 --task "code_generation" # Check if API calls should proceed (exit code 0 = OK, 1 = blocked) python token_tamer.py --check # Generate daily report python token_reports.py --daily # Detect waste (last 7 days) python token_optimizer.py --detect-waste --days 7 # Get optimization recommendations python token_optimizer.py --recommendations --days 7 ``` See README.md for full API reference, model pricing configuration, and advanced usage.

path: config_example.py
content: """
Token Tamer Configuration Example
Copy this file to token_config.py and customize for your environment.

Author: Shadow Rose
License: MIT
"""

# ============================================================================
# STORAGE SETTINGS
# ============================================================================

# File where usage data is stored
USAGE_FILE = "/path/to/token_usage.json"

# ============================================================================
# BUDGET LIMITS
# ============================================================================

# Daily, weekly, and monthly budget limits (in USD)
# Set to 0 to disable budget tracking for that period
BUDGETS = {
    'daily': 10.00,      # $10/day
    'weekly': 50.00,     # $50/week
    'monthly': 150.00,   # $150/month
}

# ============================================================================
# ALERT THRESHOLDS
# ============================================================================

# Percentage thresholds for alerts
ALERT_THRESHOLDS = {
    'warning': 80,    # Warn at 80% of budget
    'critical': 95,   # Critical at 95% of budget
    # At 100%, kill switch activates automatically
}

# ============================================================================
# MODEL PRICING
# ============================================================================

# Pricing per million tokens (input, output)
# Format: "provider/model": {"input": price, "output": price}
# Use "provider/*" for wildcard (applies to all models from that provider)

MODEL_PRICING = {
    # OpenAI
    'openai/gpt-4': {
        'input': 30.00,
        'output': 60.00
    },
    'openai/gpt-4-turbo': {
        'input': 10.00,
        'output': 30.00
    },
    'openai/gpt-3.5-turbo': {
        'input': 0.50,
        'output': 1.50
    },
    'openai/gpt-4o': {
        'input': 2.50,
        'output': 10.00
    },
    'openai/gpt-4o-mini': {
        'input': 0.15,
        'output': 0.60
    },
    
    # Anthropic
    'anthropic/claude-3-opus': {
        'input': 15.00,
        'output': 75.00
    },
    'anthropic/claude-3-sonnet': {
        'input': 3.00,
        'output': 15.00
    },
    'anthropic/claude-3-haiku': {
        'input': 0.25,
        'output': 1.25
    },
    'anthropic/claude-sonnet-4': {
        'input': 3.00,
        'output': 15.00
    },
    'anthropic/claude-sonnet-4-5': {
        'input': 3.00,
        'output': 15.00
    },
    
    # Google
    'google/gemini-1.5-pro': {
        'input': 1.25,
        'output': 5.00
    },
    'google/gemini-1.5-flash': {
        'input': 0.075,
        'output': 0.30
    },
    'google/gemini-2.0-flash': {
        'input': 0.10,
        'output': 0.40
    },
    
    # xAI
    'xai/grok-beta': {
        'input': 5.00,
        'output': 15.00
    },
    'xai/grok-2': {
        'input': 2.00,
        'output': 10.00
    },
    
    # OpenRouter (average pricing)
    'openrouter/*': {
        'input': 5.00,
        'output': 15.00
    },
    
    # Custom provider example
    'custom/my-model': {
        'input': 1.00,
        'output': 2.00
    },
}

# ============================================================================
# WASTE DETECTION THRESHOLDS
# ============================================================================

WASTE_THRESHOLDS = {
    # Time window for detecting duplicate calls (minutes)
    'duplicate_window_minutes': 5,
    
    # Number of retries before considering excessive
    'excessive_retries_count': 3,
    
    # Input token count considered "large" (might have unnecessary context)
    'large_context_tokens': 50000,
    
    # Output/input ratio below this is suspicious (sending huge context, getting little output)
    'low_output_ratio': 0.01,  # 1%
}

# ============================================================================
# OPTIMIZATION SETTINGS
# ============================================================================

# Suggest cheaper model when expensive model is used for small requests
SUGGEST_CHEAPER_MODEL_THRESHOLD_TOKENS = 1000

# Models considered "expensive" (will trigger optimization suggestions)
EXPENSIVE_MODELS = [
    'gpt-4',
    'claude-3-opus',
    'claude-opus',
]

# Recommended cheaper alternatives
CHEAPER_ALTERNATIVES = {
    'gpt-4': 'gpt-4o-mini',
    'claude-3-opus': 'claude-3-haiku',
}

# ============================================================================
# ADVANCED SETTINGS
# ============================================================================

# Automatically activate kill switch when daily budget exceeded
AUTO_KILL_SWITCH = True

# Log every API call to console (verbose mode)
VERBOSE_LOGGING = False

# Retention period for usage data (days)
# Set to 0 to keep all data indefinitely
USAGE_DATA_RETENTION_DAYS = 90

# ============================================================================
# INTEGRATION SETTINGS
# ============================================================================

# Export usage data to external systems
# Options: None, 'webhook', 'file'
EXPORT_MODE = None

# Webhook URL for cost alerts (optional)
WEBHOOK_URL = None

# Webhook authentication header (optional)
WEBHOOK_AUTH_HEADER = None  # e.g., "Authorization: Bearer TOKEN"

# Export file path (if EXPORT_MODE = 'file')
EXPORT_FILE = "/path/to/usage_export.json"

# ============================================================================
# PROVIDER-SPECIFIC SETTINGS
# ============================================================================

# Some providers report usage differently
# Configure parsing rules here if needed

PROVIDER_CONFIG = {
    'openai': {
        'token_field': 'usage',
        'input_field': 'prompt_tokens',
        'output_field': 'completion_tokens'
    },
    'anthropic': {
        'token_field': 'usage',
        'input_field': 'input_tokens',
        'output_field': 'output_tokens'
    },
    # Add custom providers as needed
}


path: LIMITATIONS.md
content: # Token Tamer Limitations

**What Token Tamer doesn't do — honest about the gaps.**

---

## Not Automatic Integration

Token Tamer is a **logging and budgeting tool**, not an API proxy:

- **Manual logging required** — You must call `tamer.log_usage()` after each API call
- **No automatic interception** — Doesn't automatically wrap OpenAI/Anthropic SDKs
- **No middleware** — Can't drop-in to existing code without modification
- **Your responsibility** — You must remember to log usage

**Recommendation:** Build a thin wrapper around your API calls that logs to Token Tamer. Centralizethe wrapper so you can't forget to log.

---

## Pricing Data Is Manual

Token Tamer doesn't fetch live pricing from providers:

- **Static config** — Pricing is in `config_example.py`, not auto-updated
- **You update it** — When providers change pricing, you update the config
- **No API pricing queries** — Doesn't call provider APIs to fetch current rates
- **Stale data risk** — If you don't update config, cost calculations will be wrong

**Recommendation:** Check provider pricing pages monthly. Update `MODEL_PRICING` when changes occur. Subscribe to provider pricing update emails.

---

## Kill Switch Requires Integration

The kill switch doesn't magically block API calls:

- **You must call `check_before_call()`** — Kill switch only works if you check it
- **Not enforced by providers** — Providers don't know about your budget
- **Application-level only** — If your code bypasses Token Tamer, budget is ignored
- **No network-level blocking** — Can't block at firewall/proxy level
- **Resets on restart** — Kill switch state is in-memory only. If the process restarts after budget is exceeded, the kill switch resets to inactive. Persist kill switch state to file if you need it to survive restarts.

**Recommendation:** Wrap all API calls in a function that checks kill switch before proceeding.

---

## No Real-Time Provider Sync

Token Tamer doesn't query provider usage APIs:

- **Self-reported only** — Trusts the token counts you log
- **No validation** — Doesn't verify against OpenAI/Anthropic dashboards
- **Drift possible** — If you forget to log some calls, totals will be wrong
- **No reconciliation** — Can't auto-reconcile with provider bills

**Recommendation:** Periodically compare Token Tamer totals with provider dashboards. Investigate discrepancies.

---

## Waste Detection Is Heuristic

Waste detection uses pattern matching, not AI:

- **False positives** — Might flag legitimate retries as waste
- **False negatives** — Might miss wasteful patterns it doesn't know about
- **No semantic understanding** — Can't tell if large context is necessary
- **Fixed thresholds** — Doesn't learn what "normal" is for your workload

**Recommendation:** Review waste detection results manually. Tune `WASTE_THRESHOLDS` based on your usage patterns.

---

## No Multi-Currency Support

Token Tamer assumes USD:

- **USD only** — All costs stored and reported in dollars
- **No conversion** — Can't handle providers billing in other currencies
- **Manual conversion required** — You must convert to USD before logging

**Recommendation:** If your provider bills in EUR/GBP/etc., convert to USD using exchange rate when logging.

---

## No Predictive Budgeting

Token Tamer tracks historical costs, not future projections:

- **No forecasting** — Can't predict end-of-month costs based on current trajectory
- **No anomaly detection** — Can't alert "spending is 3x higher than usual today"
- **No trending** — Doesn't show if costs are increasing/decreasing over time

**Recommendation:** Export data to spreadsheet or BI tool for trending/forecasting.

---

## Limited Retry Detection

Retry detection is basic:

- **Time-based only** — Detects retries by clustering calls within 1 hour
- **No error code tracking** — Can't tell if retry was due to rate limit vs. timeout
- **No exponential backoff analysis** — Can't verify if backoff strategy is optimal

**Recommendation:** Add metadata to `log_usage()` calls (e.g., `metadata={'retry': True}`) for better tracking.

---

## No Team/User Attribution

Token Tamer tracks tasks and sessions, but not users:

- **No user_id field** — Can't attribute costs to specific team members
- **No access control** — Anyone can log usage under any task name
- **No audit trail** — Can't prove who initiated a specific API call

**Recommendation:** Use `session` field to encode user info if needed (`session='user_alice'`).

---

## Storage Is Local JSON

Token Tamer stores data in a local JSON file:

- **No database** — Can't scale to millions of records efficiently
- **No replication** — If file is lost, all history is lost
- **No concurrency control** — Multiple processes writing simultaneously can corrupt file
- **No compression** — Large usage files consume disk space

**Recommendation:** Backup `USAGE_FILE` regularly. For massive scale, export to database (Postgres, SQLite).

---

## No Cloud Sync

Token Tamer is local-only:

- **No multi-machine tracking** — Can't aggregate costs across dev/staging/prod
- **No team dashboard** — Can't share cost visibility with team
- **No webhooks** — Can't send alerts to Slack/Discord (config exists, not implemented)

**Recommendation:** Export reports to shared storage (S3, Google Drive) for team visibility.

---

## Optimization Recommendations Are Generic

Token Tamer suggests optimizations but doesn't know your use case:

- **"Use cheaper model"** — Might not work if you need quality
- **"Reduce context"** — Might break your prompts
- **"Cache results"** — Might not be possible for your workflow

**Recommendation:** Treat recommendations as starting points. Test before implementing.

---

## No Integration with CI/CD

Token Tamer doesn't block deploys based on cost:

- **Manual checks only** — You must run `--check` in CI pipeline yourself
- **No GitHub Action** — No pre-built integration with GitHub/GitLab
- **No automatic rollback** — If deploy causes cost spike, no auto-revert

**Recommendation:** Add Token Tamer budget check to CI pipeline as manual step.

---

## No Cost Alerts (Yet)

Token Tamer logs alerts but doesn't send them:

- **No email alerts** — Won't email you when budget is exceeded
- **No Slack/Discord** — Config exists for webhooks, not yet implemented
- **Console only** — Alerts written to stderr, easy to miss

**Recommendation:** Grep logs for alert keywords. Planned for v1.1.

---

## Provider-Specific Edge Cases

Token Tamer assumes standard token counting:

- **Cached tokens** — Some providers (OpenAI) offer discounted cached input tokens
- **Batching** — Doesn't account for batch API pricing (different rates)
- **Streaming** — Assumes you log final token counts (not per-chunk)
- **Fine-tuned models** — Pricing may differ for custom/fine-tuned models

**Recommendation:** Add provider-specific logic in your API wrapper if needed.

---

## What Token Tamer Is

Token Tamer is a **cost tracking and budgeting tool**, not a cost management platform. It helps you monitor spending, enforce budgets, and detect waste. It doesn't auto-optimize your code, negotiate better pricing, or predict future costs.

**Use Token Tamer for:** Real-time cost tracking, budget enforcement, waste detection, cost reporting.

**Don't use Token Tamer for:** Automatic optimization, multi-team dashboards, predictive budgeting, provider bill reconciliation.

---

## Planned Improvements

Features we're considering for future releases:

- Webhook alerts (Slack, Discord, email)
- Provider API integration (fetch usage from OpenAI/Anthropic dashboards)
- Forecasting (predict end-of-month costs)
- Multi-currency support
- Team/user attribution
- Database backend option (SQLite, Postgres)
- GitHub Action for CI/CD integration
- Cached token discount tracking

No promises. Token Tamer is open source (MIT). Contributions welcome.


path: README.md
content: # Token Tamer — AI API Cost Control

**Monitor, budget, and optimize AI API spending across ANY provider.**

Token Tamer tracks every AI API call, enforces budgets, detects waste, and provides actionable optimization recommendations. Works with OpenAI, Anthropic, Google, xAI, OpenRouter, or any custom provider. Stop runaway costs before they happen.

---

## The Problem

AI API costs are unpredictable and can spiral out of control:

- **No visibility** — You don't know how much you're spending until the bill arrives
- **Runaway costs** — One buggy loop can burn $100 in minutes
- **No budgets** — APIs don't stop when you hit a limit; they just charge more
- **Hidden waste** — Duplicate calls, oversized contexts, unnecessary retries
- **No attribution** — Can't tell which task/model/session is costing the most

Manual cost tracking doesn't work. Spreadsheets are always outdated. Provider dashboards are too slow.

## What Token Tamer Does

### 1. **Real-Time Cost Tracking** (`token_tamer.py`)
- Log every API call: provider, model, tokens, cost
- Per-task and per-session cost attribution
- Configurable model pricing database (all major providers included)
- Custom provider support (bring your own pricing)
- Persistent storage (JSON-based, survives restarts)

### 2. **Budget Enforcement**
- Daily, weekly, and monthly budget limits
- Configurable alert thresholds (warning at 80%, critical at 95%)
- **Kill switch** — Hard stop at 100% of budget (blocks all API calls)
- Pre-call budget check (prevent calls that would exceed budget)
- Status dashboard (see current spend vs. budget at any time)

### 3. **Cost Reports** (`token_reports.py`)
- Daily, weekly, monthly summaries
- Breakdown by provider (which provider costs the most?)
- Breakdown by task (which tasks burn budget fastest?)
- Breakdown by model (are you using expensive models unnecessarily?)
- JSON and text output formats

### 4. **Waste Detection** (`token_optimizer.py`)
- **Duplicate calls** — Detects identical calls within time window
- **Excessive retries** — Flags tasks with too many retry attempts
- **Large contexts** — Identifies calls with bloated input (>50K tokens)
- **Low output ratio** — Finds calls sending huge context, getting tiny output
- **Overqualified models** — Detects expensive models used for simple tasks
- Estimates cost savings from eliminating waste

### 5. **Optimization Recommendations**
- Model selection advice (cheaper alternatives for routine tasks)
- Task-level optimization (which tasks need caching/refactoring)
- Potential savings calculation (how much you could save)
- Prioritized recommendations (fix high-impact issues first)

---

## Quick Start

```bash
# 1. Configure Token Tamer
cp config_example.py token_config.py
# Edit token_config.py with your budgets and model pricing

# 2. Log an API call
python3 token_tamer.py --log \
    --provider openai \
    --model gpt-4o \
    --input-tokens 1500 \
    --output-tokens 300 \
    --task "code_generation" \
    --session "sess_001"

# 3. Check status
python3 token_tamer.py --status

# 4. Generate daily report
python3 token_reports.py --daily

# 5. Detect waste
python3 token_optimizer.py --detect-waste --days 7

# 6. Get optimization recommendations
python3 token_optimizer.py --recommendations --days 7
```

---

## Design Philosophy

### Budget First, Optimize Second
Set budgets before you start spending. Kill switch ensures you never exceed limits, even if code goes haywire.

### Provider-Agnostic
Token Tamer doesn't care which AI provider you use. Configure pricing, log usage, track costs. Works with any API that returns token counts.

### Zero External Dependencies
Pure Python stdlib. No pip installs, no network calls (except your API usage). Runs anywhere Python runs.

### Actionable Insights
Reports aren't just numbers. Token Tamer tells you **what** is expensive and **how** to fix it.

### Safety by Default
Kill switch activates automatically when budget exceeded. Better to block one API call than burn $1000.

---

## Use Cases

### For Indie Developers
- **Cost control** — Never exceed your budget (critical when using personal credit card)
- **Waste detection** — Find and fix expensive bugs before they drain your account
- **Model selection** — Use expensive models only when necessary, cheaper models for routine tasks

### For Startups
- **Budget allocation** — Track costs per feature/customer/session
- **Scalability testing** — Project costs as you scale (how much will 10x users cost?)
- **Investor reporting** — Show AI cost metrics to investors/stakeholders

### For Enterprises
- **Multi-team tracking** — Attribute costs to specific teams/projects/tasks
- **Compliance** — Audit trail of all API usage
- **Cost optimization** — Identify waste at scale, optimize before it compounds

### For AI Researchers
- **Experiment tracking** — Cost per experiment/run
- **Model comparison** — Which model gives best quality/cost ratio?
- **Hyperparameter tuning** — How much does each tuning run cost?

---

## What's Included

| File | Purpose |
|------|---------|
| `token_tamer.py` | Main cost tracking and budget engine |
| `token_reports.py` | Report generation (daily, weekly, by-task) |
| `token_optimizer.py` | Waste detection and optimization suggestions |
| `config_example.py` | Configuration template with all settings |
| `LIMITATIONS.md` | What Token Tamer doesn't do |
| `LICENSE` | MIT License |

---

## Configuration

See `config_example.py` for full details. Key settings:

### Budgets
```python
BUDGETS = {
    'daily': 10.00,      # $10/day
    'weekly': 50.00,     # $50/week
    'monthly': 150.00,   # $150/month
}

ALERT_THRESHOLDS = {
    'warning': 80,    # Warn at 80%
    'critical': 95,   # Critical at 95%
    # Kill switch at 100%
}
```

### Model Pricing
```python
MODEL_PRICING = {
    'openai/gpt-4o': {
        'input': 2.50,   # Per million tokens
        'output': 10.00
    },
    'anthropic/claude-sonnet-4': {
        'input': 3.00,
        'output': 15.00
    },
    # Add your models here
}
```

### Waste Detection
```python
WASTE_THRESHOLDS = {
    'duplicate_window_minutes': 5,
    'excessive_retries_count': 3,
    'large_context_tokens': 50000,
    'low_output_ratio': 0.01,
}
```

---

## Advanced Usage

### Integration with API Wrapper

```python
from token_tamer import TokenTamer
import token_config as config

tamer = TokenTamer(config)

def call_openai_api(prompt):
    # Check budget before calling
    if not tamer.check_before_call(estimated_cost=0.10):
        raise Exception("Budget exceeded — kill switch active")
    
    # Make API call
    response = openai.ChatCompletion.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}]
    )
    
    # Log usage
    cost, status = tamer.log_usage(
        provider="openai",
        model="gpt-4o",
        input_tokens=response['usage']['prompt_tokens'],
        output_tokens=response['usage']['completion_tokens'],
        task="chat"
    )
    
    if status == 'KILL':
        raise Exception("Kill switch activated after this call")
    
    return response
```

### Daily Cost Report (cron)

```bash
# Run daily at 9 AM
0 9 * * * cd /path/to/token-tamer && python3 token_reports.py --daily --format text | mail -s "Daily AI Cost Report" [REDACTED_SECRET]
```

### Weekly Optimization Review

```bash
# Run weekly on Monday morning
0 9 * * 1 cd /path/to/token-tamer && python3 token_optimizer.py --recommendations --days 7
```

### Pre-Deployment Budget Check

```bash
# Check if we're within budget before deploying
python3 token_tamer.py --check || exit 1
# If kill switch is active, deployment is blocked
```

---

## Reports Examples

### Daily Report
```
============================================================
            Daily Report: 2026-02-21
============================================================

Total calls:           127
Total tokens:      1,234,567
  Input:             987,654
  Output:            246,913
Total cost:          $  12.34
Avg cost/call:       $   0.0972
Avg tokens/call:          9,721
```

### By Provider Report
```
BY PROVIDER
────────────────────────────────────────────────────────────

openai               $   8.50     89 calls   1,000,000 tokens
anthropic            $   3.20     28 calls     200,000 tokens
google               $   0.64     10 calls      34,567 tokens
```

### Waste Detection
```
WASTE DETECTION REPORT
============================================================

Period: Last 7 days
Total records analyzed: 1,234
Total waste detected: $45.67

Waste Detected:

  • 12 groups of duplicate API calls within 5 minutes
    Estimated waste: $23.45

  • 5 tasks with >3 retry attempts
    Estimated waste: $12.34

  • 34 calls with input >50,000 tokens
    Estimated waste: $9.88

Recommendations:

  → Implement caching to avoid duplicate calls
  → Add exponential backoff and better error handling
  → Implement context pruning and summarization
```

---

## Requirements

- Python 3.8+
- No external dependencies (stdlib only)
- Works on Linux, macOS, Windows
- ~100KB disk space for code
- Usage data storage grows with usage (JSON file)

---

## License

MIT — See `LICENSE` file.


---

## ⚠️ Disclaimer

This software is provided "AS IS", without warranty of any kind, express or implied.

**USE AT YOUR OWN RISK.**

- The author(s) are NOT liable for any damages, losses, or consequences arising from 
  the use or misuse of this software — including but not limited to financial loss, 
  data loss, security breaches, business interruption, or any indirect/consequential damages.
- This software does NOT constitute financial, legal, trading, or professional advice.
- Users are solely responsible for evaluating whether this software is suitable for 
  their use case, environment, and risk tolerance.
- No guarantee is made regarding accuracy, reliability, completeness, or fitness 
  for any particular purpose.
- The author(s) are not responsible for how third parties use, modify, or distribute 
  this software after purchase.

By downloading, installing, or using this software, you acknowledge that you have read 
this disclaimer and agree to use the software entirely at your own risk.

---

?? **GitHub:** https://github.com/TheShadowRose
?? **Found a bug?** Email [REDACTED_SECRET]
? **Ko-fi:** https://ko-fi.com/theshadowrose
?? **Twitter:** https://x.com/TheShadowyRose
---

## Support & Links

| | |
|---|---|
| 🐛 **Bug Reports** | [REDACTED_SECRET] |
| ☕ **Ko-fi** | [ko-fi.com/theshadowrose](https://ko-fi.com/theshadowrose) |
| 🛒 **Gumroad** | [shadowyrose.gumroad.com](https://shadowyrose.gumroad.com) |
| 🐦 **Twitter** | [@TheShadowyRose](https://twitter.com/TheShadowyRose) |
| 🐙 **GitHub** | [github.com/TheShadowRose](https://github.com/TheShadowRose) |
| 🧠 **PromptBase** | [promptbase.com/profile/shadowrose](https://promptbase.com/profile/shadowrose) |

*Built with [OpenClaw](https://github.com/openclaw/openclaw) — thank you for making this possible.*

---

🛠️ **Need something custom?** Custom OpenClaw agents & skills starting at . If you can describe it, I can build it. → [Hire me on Fiverr](https://www.fiverr.com/s/jjmlZ0v)



path: token_optimizer.py
content: #!/usr/bin/env python3
"""
Token Optimizer — Waste detection and optimization suggestions
Part of Token Tamer — AI API Cost Control

Author: Shadow Rose
License: MIT
"""

import os
import sys
import json
from pathlib import Path
from datetime import datetime, timedelta
from typing import Dict, List, Optional, Tuple
from collections import defaultdict, Counter

# Import config
try:
    import token_config as config
except ImportError:
    print("ERROR: token_config.py not found. Copy config_example.py to token_config.py and configure it.")
    sys.exit(1)

# Import from main module
from token_tamer import UsageStore, UsageRecord


class WasteDetector:
    """Detect wasteful API usage patterns."""
    
    def __init__(self, config_obj):
        self.config = config_obj
        self.storage = UsageStore(Path(config_obj.USAGE_FILE))
        self.thresholds = getattr(config_obj, 'WASTE_THRESHOLDS', {
            'duplicate_window_minutes': 5,
            'excessive_retries_count': 3,
            'large_context_tokens': 50000,
            'low_output_ratio': 0.01  # Output/input < 1%
        })
    
    def detect_all(self, days: int = 7) -> Dict:
        """Run all waste detection algorithms."""
        start_date = datetime.now() - timedelta(days=days)
        records = self.storage.get_records(start_date=start_date)
        
        results = {
            'period': f"Last {days} days",
            'total_records': len(records),
            'waste_detected': [],
            'total_waste_cost': 0.0,
            'recommendations': []
        }
        
        # Detect duplicates
        duplicates = self.detect_duplicates(records)
        if duplicates:
            waste_cost = sum(sum(r.cost for r in group[1:]) for group in duplicates)
            results['waste_detected'].append({
                'type': 'duplicate_calls',
                'count': len(duplicates),
                'waste_cost': waste_cost,
                'description': f"{len(duplicates)} groups of duplicate API calls within {self.thresholds['duplicate_window_minutes']} minutes"
            })
            results['total_waste_cost'] += waste_cost
            results['recommendations'].append("Implement caching to avoid duplicate calls")
        
        # Detect excessive retries
        retries = self.detect_excessive_retries(records)
        if retries:
            waste_cost = sum(sum(r.cost for r in group[self.thresholds['excessive_retries_count']:]) for group in retries)
            results['waste_detected'].append({
                'type': 'excessive_retries',
                'count': len(retries),
                'waste_cost': waste_cost,
                'description': f"{len(retries)} tasks with >{self.thresholds['excessive_retries_count']} retry attempts"
            })
            results['total_waste_cost'] += waste_cost
            results['recommendations'].append("Add exponential backoff and better error handling")
        
        # Detect large contexts
        large_contexts = self.detect_large_contexts(records)
        if large_contexts:
            waste_cost = sum(r.cost * 0.5 for r in large_contexts)  # Assume 50% could be trimmed
            results['waste_detected'].append({
                'type': 'large_contexts',
                'count': len(large_contexts),
                'waste_cost': waste_cost,
                'description': f"{len(large_contexts)} calls with input >{self.thresholds['large_context_tokens']:,} tokens"
            })
            results['total_waste_cost'] += waste_cost
            results['recommendations'].append("Implement context pruning and summarization")
        
        # Detect low output ratio (mostly input, little output)
        low_output = self.detect_low_output_ratio(records)
        if low_output:
            waste_cost = sum(r.cost * 0.3 for r in low_output)  # Assume 30% could be saved
            results['waste_detected'].append({
                'type': 'low_output_ratio',
                'count': len(low_output),
                'waste_cost': waste_cost,
                'description': f"{len(low_output)} calls with output/input ratio <{self.thresholds['low_output_ratio']:.1%}"
            })
            results['total_waste_cost'] += waste_cost
            results['recommendations'].append("Review prompts — may be sending unnecessary context")
        
        # Detect expensive model usage for simple tasks
        expensive_simple = self.detect_expensive_models_simple_tasks(records)
        if expensive_simple:
            waste_cost = sum(r.cost * 0.7 for r in expensive_simple)  # Assume 70% could be saved with cheaper model
            results['waste_detected'].append({
                'type': 'overqualified_models',
                'count': len(expensive_simple),
                'waste_cost': waste_cost,
                'description': f"{len(expensive_simple)} calls using expensive models for likely simple tasks"
            })
            results['total_waste_cost'] += waste_cost
            results['recommendations'].append("Use cheaper models for simple/routine tasks")
        
        return results
    
    def detect_duplicates(self, records: List[UsageRecord]) -> List[List[UsageRecord]]:
        """Detect duplicate or near-duplicate API calls."""
        # Group by task and session
        by_key = defaultdict(list)
        for record in records:
            key = (record.task, record.session, record.model)
            by_key[key].append(record)
        
        duplicates = []
        window = timedelta(minutes=self.thresholds['duplicate_window_minutes'])
        
        for key, key_records in by_key.items():
            # Sort by timestamp
            sorted_records = sorted(key_records, key=lambda r: r.timestamp)
            
            # Find clusters within time window
            current_cluster = [sorted_records[0]]
            
            for record in sorted_records[1:]:
                if record.timestamp - current_cluster[-1].timestamp <= window:
                    # Similar token counts (within 10%)
                    if abs(record.total_tokens - current_cluster[0].total_tokens) / current_cluster[0].total_tokens < 0.1:
                        current_cluster.append(record)
                    else:
                        if len(current_cluster) > 1:
                            duplicates.append(current_cluster)
                        current_cluster = [record]
                else:
                    if len(current_cluster) > 1:
                        duplicates.append(current_cluster)
                    current_cluster = [record]
            
            if len(current_cluster) > 1:
                duplicates.append(current_cluster)
        
        return duplicates
    
    def detect_excessive_retries(self, records: List[UsageRecord]) -> List[List[UsageRecord]]:
        """Detect tasks with too many retry attempts."""
        by_task = defaultdict(list)
        for record in records:
            by_task[record.task].append(record)
        
        excessive = []
        threshold = self.thresholds['excessive_retries_count']
        
        for task, task_records in by_task.items():
            if len(task_records) > threshold:
                # Check if they're within a short time period (likely retries)
                sorted_records = sorted(task_records, key=lambda r: r.timestamp)
                time_span = (sorted_records[-1].timestamp - sorted_records[0].timestamp).total_seconds() / 3600
                
                if time_span < 1:  # All within 1 hour
                    excessive.append(task_records)
        
        return excessive
    
    def detect_large_contexts(self, records: List[UsageRecord]) -> List[UsageRecord]:
        """Detect calls with excessively large input contexts."""
        threshold = self.thresholds['large_context_tokens']
        return [r for r in records if r.input_tokens > threshold]
    
    def detect_low_output_ratio(self, records: List[UsageRecord]) -> List[UsageRecord]:
        """Detect calls with very low output relative to input (possible waste)."""
        threshold = self.thresholds['low_output_ratio']
        low_output = []
        
        for record in records:
            if record.input_tokens > 0:
                ratio = record.output_tokens / record.input_tokens
                if ratio < threshold:
                    low_output.append(record)
        
        return low_output
    
    def detect_expensive_models_simple_tasks(self, records: List[UsageRecord]) -> List[UsageRecord]:
        """Detect use of expensive models for likely simple tasks."""
        # Define expensive models (these are just examples, configure in real config)
        expensive_models = [
            'gpt-4', 'claude-opus', 'claude-3-opus', 'gpt-4-turbo'
        ]
        
        expensive = []
        
        for record in records:
            is_expensive = any(em in record.model.lower() for em in expensive_models)
            
            if is_expensive and record.total_tokens < 1000:
                # Expensive model used for small request (likely simple)
                expensive.append(record)
        
        return expensive


class Optimizer:
    """Generate optimization recommendations."""
    
    def __init__(self, config_obj):
        self.config = config_obj
        self.storage = UsageStore(Path(config_obj.USAGE_FILE))
        self.detector = WasteDetector(config_obj)
    
    def generate_recommendations(self, days: int = 7) -> Dict:
        """Generate comprehensive optimization recommendations."""
        waste = self.detector.detect_all(days)
        start_date = datetime.now() - timedelta(days=days)
        records = self.storage.get_records(start_date=start_date)
        
        recommendations = {
            'period': f"Last {days} days",
            'total_cost': sum(r.cost for r in records),
            'potential_savings': waste['total_waste_cost'],
            'savings_percent': (waste['total_waste_cost'] / sum(r.cost for r in records) * 100) if records else 0,
            'waste_summary': waste,
            'optimization_opportunities': []
        }
        
        # Model usage analysis
        model_usage = self._analyze_model_usage(records)
        if model_usage['recommendations']:
            recommendations['optimization_opportunities'].append({
                'category': 'Model Selection',
                'recommendations': model_usage['recommendations']
            })
        
        # Task analysis
        task_analysis = self._analyze_task_costs(records)
        if task_analysis['recommendations']:
            recommendations['optimization_opportunities'].append({
                'category': 'Task Optimization',
                'recommendations': task_analysis['recommendations']
            })
        
        return recommendations
    
    def _analyze_model_usage(self, records: List[UsageRecord]) -> Dict:
        """Analyze model usage patterns."""
        by_model = defaultdict(list)
        for record in records:
            key = f"{record.provider}/{record.model}"
            by_model[key].append(record)
        
        model_stats = {}
        for model, model_records in by_model.items():
            total_cost = sum(r.cost for r in model_records)
            avg_tokens = sum(r.total_tokens for r in model_records) / len(model_records)
            
            model_stats[model] = {
                'count': len(model_records),
                'total_cost': total_cost,
                'avg_tokens': avg_tokens
            }
        
        recommendations = []
        
        # Find most expensive model
        if model_stats:
            most_expensive = max(model_stats.items(), key=lambda x: x[1]['total_cost'])
            recommendations.append(
                f"Top spend: {most_expensive[0]} (${most_expensive[1]['total_cost']:.2f}, {most_expensive[1]['count']} calls)"
            )
            
            # Check if cheaper alternatives exist
            if 'gpt-4' in most_expensive[0].lower() or 'opus' in most_expensive[0].lower():
                recommendations.append(
                    f"Consider using cheaper model for routine tasks (potential 70% cost reduction)"
                )
        
        return {'model_stats': model_stats, 'recommendations': recommendations}
    
    def _analyze_task_costs(self, records: List[UsageRecord]) -> Dict:
        """Analyze task-level costs."""
        by_task = defaultdict(list)
        for record in records:
            by_task[record.task].append(record)
        
        task_stats = {}
        for task, task_records in by_task.items():
            total_cost = sum(r.cost for r in task_records)
            task_stats[task] = {
                'count': len(task_records),
                'total_cost': total_cost
            }
        
        recommendations = []
        
        # Find most expensive task
        if task_stats:
            most_expensive = max(task_stats.items(), key=lambda x: x[1]['total_cost'])
            recommendations.append(
                f"Most expensive task: '{most_expensive[0]}' (${most_expensive[1]['total_cost']:.2f}, {most_expensive[1]['count']} calls)"
            )
            
            if most_expensive[1]['count'] > 10:
                recommendations.append(
                    f"Task '{most_expensive[0]}' called {most_expensive[1]['count']} times — consider caching results"
                )
        
        return {'task_stats': task_stats, 'recommendations': recommendations}


def main():
    """CLI entry point."""
    import argparse
    
    parser = argparse.ArgumentParser(description="Token Optimizer — Waste detection and optimization")
    parser.add_argument('--detect-waste', action='store_true', help='Detect wasteful usage patterns')
    parser.add_argument('--recommendations', action='store_true', help='Generate optimization recommendations')
    parser.add_argument('--days', type=int, default=7, help='Number of days to analyze (default: 7)')
    parser.add_argument('--format', choices=['text', 'json'], default='text', help='Output format')
    
    args = parser.parse_args()
    
    if args.detect_waste:
        detector = WasteDetector(config)
        results = detector.detect_all(days=args.days)
        
        if args.format == 'json':
            print(json.dumps(results, indent=2))
        else:
            print("\n" + "=" * 60)
            print("WASTE DETECTION REPORT".center(60))
            print("=" * 60 + "\n")
            print(f"Period: {results['period']}")
            print(f"Total records analyzed: {results['total_records']:,}")
            print(f"Total waste detected: ${results['total_waste_cost']:.2f}\n")
            
            if results['waste_detected']:
                print("Waste Detected:\n")
                for waste in results['waste_detected']:
                    print(f"  • {waste['description']}")
                    print(f"    Estimated waste: ${waste['waste_cost']:.2f}\n")
            
            if results['recommendations']:
                print("Recommendations:\n")
                for rec in results['recommendations']:
                    print(f"  → {rec}")
                print()
    
    elif args.recommendations:
        optimizer = Optimizer(config)
        results = optimizer.generate_recommendations(days=args.days)
        
        if args.format == 'json':
            print(json.dumps(results, indent=2))
        else:
            print("\n" + "=" * 60)
            print("OPTIMIZATION RECOMMENDATIONS".center(60))
            print("=" * 60 + "\n")
            print(f"Period: {results['period']}")
            print(f"Total cost: ${results['total_cost']:.2f}")
            print(f"Potential savings: ${results['potential_savings']:.2f} ({results['savings_percent']:.1f}%)\n")
            
            for opp in results['optimization_opportunities']:
                print(f"{opp['category']}:")
                for rec in opp['recommendations']:
                    print(f"  → {rec}")
                print()
    
    else:
        parser.print_help()


if __name__ == '__main__':
    main()


path: token_reports.py
content: #!/usr/bin/env python3
"""
Token Reports — Cost report generation
Part of Token Tamer — AI API Cost Control

Author: Shadow Rose
License: MIT
"""

import os
import sys
import json
from pathlib import Path
from datetime import datetime, timedelta
from typing import Dict, List, Optional
from collections import defaultdict

# Import config
try:
    import token_config as config
except ImportError:
    print("ERROR: token_config.py not found. Copy config_example.py to token_config.py and configure it.")
    sys.exit(1)

# Import from main module
from token_tamer import UsageStore, UsageRecord


class ReportGenerator:
    """Generate cost reports."""
    
    def __init__(self, config_obj):
        self.config = config_obj
        self.storage = UsageStore(Path(config_obj.USAGE_FILE))
    
    def generate_daily_report(self, date: Optional[datetime] = None) -> Dict:
        """Generate report for a specific day."""
        if date is None:
            date = datetime.now()
        
        start = date.replace(hour=0, minute=0, second=0, microsecond=0)
        end = start + timedelta(days=1)
        
        records = self.storage.get_records(start_date=start, end_date=end)
        
        return self._aggregate_records(records, f"Daily Report: {date.date()}")
    
    def generate_weekly_report(self, week_start: Optional[datetime] = None) -> Dict:
        """Generate report for a week."""
        if week_start is None:
            now = datetime.now()
            week_start = now - timedelta(days=now.weekday())
        
        start = week_start.replace(hour=0, minute=0, second=0, microsecond=0)
        end = start + timedelta(days=7)
        
        records = self.storage.get_records(start_date=start, end_date=end)
        
        return self._aggregate_records(records, f"Weekly Report: {start.date()} to {end.date()}")
    
    def generate_monthly_report(self, year: Optional[int] = None, month: Optional[int] = None) -> Dict:
        """Generate report for a month."""
        if year is None or month is None:
            now = datetime.now()
            year = now.year
            month = now.month
        
        start = datetime(year, month, 1)
        if month == 12:
            end = datetime(year + 1, 1, 1)
        else:
            end = datetime(year, month + 1, 1)
        
        records = self.storage.get_records(start_date=start, end_date=end)
        
        return self._aggregate_records(records, f"Monthly Report: {year}-{month:02d}")
    
    def generate_by_provider_report(self, start_date: Optional[datetime] = None,
                                   end_date: Optional[datetime] = None) -> Dict:
        """Generate report grouped by provider."""
        records = self.storage.get_records(start_date, end_date)
        
        by_provider = defaultdict(list)
        for record in records:
            by_provider[record.provider].append(record)
        
        report = {
            'title': 'Report by Provider',
            'period': f"{start_date or 'beginning'} to {end_date or 'now'}",
            'providers': {}
        }
        
        for provider, provider_records in by_provider.items():
            report['providers'][provider] = self._aggregate_records(provider_records, provider)
        
        # Add totals
        report['total_cost'] = sum(p['total_cost'] for p in report['providers'].values())
        report['total_tokens'] = sum(p['total_tokens'] for p in report['providers'].values())
        report['call_count'] = sum(p['call_count'] for p in report['providers'].values())
        
        return report
    
    def generate_by_task_report(self, start_date: Optional[datetime] = None,
                               end_date: Optional[datetime] = None) -> Dict:
        """Generate report grouped by task."""
        records = self.storage.get_records(start_date, end_date)
        
        by_task = defaultdict(list)
        for record in records:
            by_task[record.task].append(record)
        
        report = {
            'title': 'Report by Task',
            'period': f"{start_date or 'beginning'} to {end_date or 'now'}",
            'tasks': {}
        }
        
        for task, task_records in by_task.items():
            report['tasks'][task] = self._aggregate_records(task_records, task)
        
        # Add totals
        report['total_cost'] = sum(t['total_cost'] for t in report['tasks'].values())
        report['total_tokens'] = sum(t['total_tokens'] for t in report['tasks'].values())
        report['call_count'] = sum(t['call_count'] for t in report['tasks'].values())
        
        # Sort tasks by cost
        report['top_tasks'] = sorted(
            [(task, data['total_cost']) for task, data in report['tasks'].items()],
            key=lambda x: x[1],
            reverse=True
        )[:10]
        
        return report
    
    def generate_by_model_report(self, start_date: Optional[datetime] = None,
                                end_date: Optional[datetime] = None) -> Dict:
        """Generate report grouped by model."""
        records = self.storage.get_records(start_date, end_date)
        
        by_model = defaultdict(list)
        for record in records:
            key = f"{record.provider}/{record.model}"
            by_model[key].append(record)
        
        report = {
            'title': 'Report by Model',
            'period': f"{start_date or 'beginning'} to {end_date or 'now'}",
            'models': {}
        }
        
        for model, model_records in by_model.items():
            report['models'][model] = self._aggregate_records(model_records, model)
        
        # Add totals
        report['total_cost'] = sum(m['total_cost'] for m in report['models'].values())
        report['total_tokens'] = sum(m['total_tokens'] for m in report['models'].values())
        report['call_count'] = sum(m['call_count'] for m in report['models'].values())
        
        # Sort models by cost
        report['top_models'] = sorted(
            [(model, data['total_cost']) for model, data in report['models'].items()],
            key=lambda x: x[1],
            reverse=True
        )[:10]
        
        return report
    
    def _aggregate_records(self, records: List[UsageRecord], title: str = "") -> Dict:
        """Aggregate statistics from a list of records."""
        if not records:
            return {
                'title': title,
                'call_count': 0,
                'total_tokens': 0,
                'input_tokens': 0,
                'output_tokens': 0,
                'total_cost': 0.0,
                'avg_cost_per_call': 0.0,
                'avg_tokens_per_call': 0.0
            }
        
        total_cost = sum(r.cost for r in records)
        total_tokens = sum(r.total_tokens for r in records)
        input_tokens = sum(r.input_tokens for r in records)
        output_tokens = sum(r.output_tokens for r in records)
        call_count = len(records)
        
        return {
            'title': title,
            'call_count': call_count,
            'total_tokens': total_tokens,
            'input_tokens': input_tokens,
            'output_tokens': output_tokens,
            'total_cost': total_cost,
            'avg_cost_per_call': total_cost / call_count,
            'avg_tokens_per_call': total_tokens / call_count
        }
    
    def print_report(self, report: Dict, format: str = 'text'):
        """Print report in specified format."""
        if format == 'json':
            print(json.dumps(report, indent=2))
        else:
            self._print_text_report(report)
    
    def _print_text_report(self, report: Dict):
        """Print report in human-readable text format."""
        print(f"\n{'=' * 60}")
        print(f"{report.get('title', 'Report').center(60)}")
        print(f"{'=' * 60}\n")
        
        if 'period' in report:
            print(f"Period: {report['period']}\n")
        
        # Simple aggregation report
        if 'call_count' in report:
            print(f"Total calls:       {report['call_count']:8,}")
            print(f"Total tokens:      {report['total_tokens']:8,}")
            print(f"  Input:           {report['input_tokens']:8,}")
            print(f"  Output:          {report['output_tokens']:8,}")
            print(f"Total cost:        ${report['total_cost']:8.2f}")
            print(f"Avg cost/call:     ${report['avg_cost_per_call']:8.4f}")
            print(f"Avg tokens/call:   {report['avg_tokens_per_call']:8.0f}")
        
        # Provider breakdown
        if 'providers' in report:
            print("\n" + "─" * 60)
            print("BY PROVIDER".center(60))
            print("─" * 60 + "\n")
            
            for provider, data in sorted(report['providers'].items(), 
                                        key=lambda x: x[1]['total_cost'], 
                                        reverse=True):
                print(f"{provider:20} ${data['total_cost']:8.2f}  {data['call_count']:6,} calls  {data['total_tokens']:10,} tokens")
        
        # Task breakdown
        if 'tasks' in report:
            print("\n" + "─" * 60)
            print("BY TASK".center(60))
            print("─" * 60 + "\n")
            
            for task, cost in report.get('top_tasks', [])[:10]:
                print(f"{task:30} ${cost:8.2f}")
        
        # Model breakdown
        if 'models' in report:
            print("\n" + "─" * 60)
            print("BY MODEL".center(60))
            print("─" * 60 + "\n")
            
            for model, cost in report.get('top_models', [])[:10]:
                print(f"{model:40} ${cost:8.2f}")
        
        print()


def main():
    """CLI entry point."""
    import argparse
    
    parser = argparse.ArgumentParser(description="Token Reports — Cost report generation")
    parser.add_argument('--daily', action='store_true', help='Generate daily report')
    parser.add_argument('--weekly', action='store_true', help='Generate weekly report')
    parser.add_argument('--monthly', action='store_true', help='Generate monthly report')
    parser.add_argument('--by-provider', action='store_true', help='Report by provider')
    parser.add_argument('--by-task', action='store_true', help='Report by task')
    parser.add_argument('--by-model', action='store_true', help='Report by model')
    parser.add_argument('--date', type=str, help='Date for daily report (YYYY-MM-DD)')
    parser.add_argument('--from', dest='date_from', type=str, help='Start date (YYYY-MM-DD)')
    parser.add_argument('--to', dest='date_to', type=str, help='End date (YYYY-MM-DD)')
    parser.add_argument('--format', choices=['text', 'json'], default='text', help='Output format')
    
    args = parser.parse_args()
    
    generator = ReportGenerator(config)
    
    # Parse dates
    date = None
    date_from = None
    date_to = None
    
    if args.date:
        try:
            date = datetime.strptime(args.date, '%Y-%m-%d')
        except ValueError:
            print(f"Invalid date format: {args.date}", file=sys.stderr)
            sys.exit(1)
    
    if args.date_from:
        try:
            date_from = datetime.strptime(args.date_from, '%Y-%m-%d')
        except ValueError:
            print(f"Invalid date format: {args.date_from}", file=sys.stderr)
            sys.exit(1)
    
    if args.date_to:
        try:
            date_to = datetime.strptime(args.date_to, '%Y-%m-%d')
        except ValueError:
            print(f"Invalid date format: {args.date_to}", file=sys.stderr)
            sys.exit(1)
    
    # Generate report
    if args.daily:
        report = generator.generate_daily_report(date)
    elif args.weekly:
        report = generator.generate_weekly_report(date)
    elif args.monthly:
        if date:
            report = generator.generate_monthly_report(date.year, date.month)
        else:
            report = generator.generate_monthly_report()
    elif args.by_provider:
        report = generator.generate_by_provider_report(date_from, date_to)
    elif args.by_task:
        report = generator.generate_by_task_report(date_from, date_to)
    elif args.by_model:
        report = generator.generate_by_model_report(date_from, date_to)
    else:
        parser.print_help()
        return
    
    generator.print_report(report, format=args.format)


if __name__ == '__main__':
    main()


path: token_tamer.py
content: #!/usr/bin/env python3
"""
Token Tamer — AI API Cost Control
Monitor, budget, and optimize AI API spending across ANY provider.

Author: Shadow Rose
License: MIT
"""

import os
import sys
import json
import time
from pathlib import Path
from datetime import datetime, timedelta
from typing import Dict, List, Optional, Tuple
from collections import defaultdict

# Import config
try:
    import token_config as config
except ImportError:
    print("ERROR: token_config.py not found. Copy config_example.py to token_config.py and configure it.")
    sys.exit(1)


class UsageRecord:
    """Represents a single API usage event."""
    
    def __init__(self, provider: str, model: str, input_tokens: int, output_tokens: int,
                 cost: float, task: Optional[str] = None, session: Optional[str] = None,
                 metadata: Optional[Dict] = None):
        self.timestamp = datetime.now()
        self.provider = provider
        self.model = model
        self.input_tokens = input_tokens
        self.output_tokens = output_tokens
        self.total_tokens = input_tokens + output_tokens
        self.cost = cost
        self.task = task or "unknown"
        self.session = session or "unknown"
        self.metadata = metadata or {}
    
    def to_dict(self) -> Dict:
        """Convert to dictionary."""
        return {
            'timestamp': self.timestamp.isoformat(),
            'provider': self.provider,
            'model': self.model,
            'input_tokens': self.input_tokens,
            'output_tokens': self.output_tokens,
            'total_tokens': self.total_tokens,
            'cost': self.cost,
            'task': self.task,
            'session': self.session,
            'metadata': self.metadata
        }
    
    @staticmethod
    def from_dict(data: Dict) -> 'UsageRecord':
        """Create from dictionary."""
        record = UsageRecord(
            provider=data['provider'],
            model=data['model'],
            input_tokens=data['input_tokens'],
            output_tokens=data['output_tokens'],
            cost=data['cost'],
            task=data.get('task'),
            session=data.get('session'),
            metadata=data.get('metadata', {})
        )
        record.timestamp = datetime.fromisoformat(data['timestamp'])
        return record


class CostCalculator:
    """Calculate costs based on provider and model pricing."""
    
    def __init__(self, config_obj):
        self.config = config_obj
        self.pricing = getattr(config_obj, 'MODEL_PRICING', {})
    
    def calculate_cost(self, provider: str, model: str, input_tokens: int, output_tokens: int) -> float:
        """Calculate cost for a usage event."""
        # Look up pricing
        key = f"{provider}/{model}"
        if key not in self.pricing:
            # Try provider wildcard
            key = f"{provider}/*"
        
        if key not in self.pricing:
            # Unknown model, return 0 and warn
            print(f"Warning: No pricing data for {provider}/{model}", file=sys.stderr)
            return 0.0
        
        pricing = self.pricing[key]
        
        # Calculate cost (pricing is per million tokens)
        input_cost = (input_tokens / 1_000_000) * pricing['input']
        output_cost = (output_tokens / 1_000_000) * pricing['output']
        
        return input_cost + output_cost


class BudgetTracker:
    """Track spending against budgets."""
    
    def __init__(self, config_obj):
        self.config = config_obj
        self.budgets = getattr(config_obj, 'BUDGETS', {})
        self.alert_thresholds = getattr(config_obj, 'ALERT_THRESHOLDS', {})
    
    def check_budget(self, period: str, current_spend: float) -> Tuple[str, Optional[str]]:
        """Check if spending is within budget."""
        if period not in self.budgets:
            return 'OK', None
        
        budget = self.budgets[period]
        percent = (current_spend / budget) * 100 if budget > 0 else 0
        
        # Check thresholds
        if percent >= 100:
            return 'KILL', f"Budget exceeded: ${current_spend:.2f} / ${budget:.2f} ({percent:.1f}%)"
        elif percent >= self.alert_thresholds.get('critical', 95):
            return 'CRITICAL', f"Budget critical: ${current_spend:.2f} / ${budget:.2f} ({percent:.1f}%)"
        elif percent >= self.alert_thresholds.get('warning', 80):
            return 'WARNING', f"Budget warning: ${current_spend:.2f} / ${budget:.2f} ({percent:.1f}%)"
        else:
            return 'OK', None
    
    def should_throttle(self, period: str, current_spend: float) -> bool:
        """Determine if API calls should be throttled."""
        status, _ = self.check_budget(period, current_spend)
        return status in ['CRITICAL', 'KILL']
    
    def should_kill(self, period: str, current_spend: float) -> bool:
        """Determine if API calls should be completely blocked."""
        status, _ = self.check_budget(period, current_spend)
        return status == 'KILL'


class UsageStore:
    """Persistent storage for usage records."""
    
    def __init__(self, storage_file: Path):
        self.storage_file = storage_file
        self.records: List[UsageRecord] = []
        self.load()
    
    def load(self):
        """Load records from disk."""
        if self.storage_file.exists():
            try:
                with open(self.storage_file, 'r') as f:
                    data = json.load(f)
                self.records = [UsageRecord.from_dict(r) for r in data.get('records', [])]
            except Exception as e:
                print(f"Error loading usage data: {e}", file=sys.stderr)
                self.records = []
    
    def save(self):
        """Save records to disk."""
        try:
            self.storage_file.parent.mkdir(parents=True, exist_ok=True)
            with open(self.storage_file, 'w') as f:
                data = {
                    'last_updated': datetime.now().isoformat(),
                    'records': [r.to_dict() for r in self.records]
                }
                json.dump(data, f, indent=2)
        except Exception as e:
            print(f"Error saving usage data: {e}", file=sys.stderr)
    
    def add_record(self, record: UsageRecord):
        """Add a usage record."""
        self.records.append(record)
        self.save()
    
    def get_records(self, start_date: Optional[datetime] = None, 
                   end_date: Optional[datetime] = None,
                   provider: Optional[str] = None,
                   task: Optional[str] = None) -> List[UsageRecord]:
        """Query records with filters."""
        filtered = self.records
        
        if start_date:
            filtered = [r for r in filtered if r.timestamp >= start_date]
        
        if end_date:
            filtered = [r for r in filtered if r.timestamp <= end_date]
        
        if provider:
            filtered = [r for r in filtered if r.provider == provider]
        
        if task:
            filtered = [r for r in filtered if r.task == task]
        
        return filtered
    
    def get_total_cost(self, start_date: Optional[datetime] = None,
                      end_date: Optional[datetime] = None,
                      provider: Optional[str] = None) -> float:
        """Get total cost for period."""
        records = self.get_records(start_date, end_date, provider)
        return sum(r.cost for r in records)


class TokenTamer:
    """Main cost tracking and budget engine."""
    
    def __init__(self, config_obj):
        self.config = config_obj
        self.storage = UsageStore(Path(config_obj.USAGE_FILE))
        self.calculator = CostCalculator(config_obj)
        self.budget_tracker = BudgetTracker(config_obj)
        self.kill_switch_active = False
    
    def log_usage(self, provider: str, model: str, input_tokens: int, output_tokens: int,
                  task: Optional[str] = None, session: Optional[str] = None,
                  metadata: Optional[Dict] = None) -> Tuple[float, str]:
        """Log API usage and return cost + status."""
        
        # Calculate cost
        cost = self.calculator.calculate_cost(provider, model, input_tokens, output_tokens)
        
        # Create record
        record = UsageRecord(provider, model, input_tokens, output_tokens, cost, task, session, metadata)
        
        # Check budget before logging
        daily_cost = self.get_daily_cost()
        status, message = self.budget_tracker.check_budget('daily', daily_cost + cost)
        
        # Check kill switch
        if self.budget_tracker.should_kill('daily', daily_cost + cost):
            self.kill_switch_active = True
            print(f"🚨 KILL SWITCH ACTIVATED: {message}", file=sys.stderr)
            return cost, 'KILL'
        
        # Log the record
        self.storage.add_record(record)
        
        # Check throttle
        if self.budget_tracker.should_throttle('daily', daily_cost + cost):
            print(f"⚠️  THROTTLE WARNING: {message}", file=sys.stderr)
            return cost, 'THROTTLE'
        
        if message:
            print(f"💰 Budget status: {message}", file=sys.stderr)
        
        return cost, status
    
    def get_daily_cost(self) -> float:
        """Get today's total cost."""
        today_start = datetime.now().replace(hour=0, minute=0, second=0, microsecond=0)
        return self.storage.get_total_cost(start_date=today_start)
    
    def get_weekly_cost(self) -> float:
        """Get this week's total cost."""
        now = datetime.now()
        week_start = now - timedelta(days=now.weekday())
        week_start = week_start.replace(hour=0, minute=0, second=0, microsecond=0)
        return self.storage.get_total_cost(start_date=week_start)
    
    def get_monthly_cost(self) -> float:
        """Get this month's total cost."""
        now = datetime.now()
        month_start = now.replace(day=1, hour=0, minute=0, second=0, microsecond=0)
        return self.storage.get_total_cost(start_date=month_start)
    
    def get_status(self) -> Dict:
        """Get current cost status."""
        daily = self.get_daily_cost()
        weekly = self.get_weekly_cost()
        monthly = self.get_monthly_cost()
        
        daily_status, daily_msg = self.budget_tracker.check_budget('daily', daily)
        weekly_status, weekly_msg = self.budget_tracker.check_budget('weekly', weekly)
        monthly_status, monthly_msg = self.budget_tracker.check_budget('monthly', monthly)
        
        return {
            'daily': {
                'cost': daily,
                'budget': self.budget_tracker.budgets.get('daily', 0),
                'status': daily_status,
                'message': daily_msg
            },
            'weekly': {
                'cost': weekly,
                'budget': self.budget_tracker.budgets.get('weekly', 0),
                'status': weekly_status,
                'message': weekly_msg
            },
            'monthly': {
                'cost': monthly,
                'budget': self.budget_tracker.budgets.get('monthly', 0),
                'status': monthly_status,
                'message': monthly_msg
            },
            'kill_switch': self.kill_switch_active
        }
    
    def check_before_call(self, estimated_cost: float = 0.10) -> bool:
        """Check if API call should proceed (return True = OK, False = blocked)."""
        if self.kill_switch_active:
            print("🚨 KILL SWITCH ACTIVE: API calls blocked", file=sys.stderr)
            return False
        
        daily = self.get_daily_cost()
        
        if self.budget_tracker.should_kill('daily', daily + estimated_cost):
            self.kill_switch_active = True
            print("🚨 KILL SWITCH ACTIVATED: Budget exceeded", file=sys.stderr)
            return False
        
        return True


def main():
    """CLI entry point."""
    import argparse
    
    parser = argparse.ArgumentParser(description="Token Tamer — AI API Cost Control")
    parser.add_argument('--log', action='store_true', help='Log a usage event')
    parser.add_argument('--provider', type=str, help='Provider name (e.g., openai, anthropic)')
    parser.add_argument('--model', type=str, help='Model name')
    parser.add_argument('--input-tokens', type=int, help='Input tokens')
    parser.add_argument('--output-tokens', type=int, help='Output tokens')
    parser.add_argument('--task', type=str, help='Task name')
    parser.add_argument('--session', type=str, help='Session ID')
    parser.add_argument('--status', action='store_true', help='Show current cost status')
    parser.add_argument('--check', action='store_true', help='Check if API call should proceed')
    
    args = parser.parse_args()
    
    tamer = TokenTamer(config)
    
    if args.status:
        status = tamer.get_status()
        print("\n=== Token Tamer Status ===\n")
        print(f"Daily:   ${status['daily']['cost']:8.2f} / ${status['daily']['budget']:8.2f}  [{status['daily']['status']}]")
        if status['daily']['message']:
            print(f"         {status['daily']['message']}")
        print(f"Weekly:  ${status['weekly']['cost']:8.2f} / ${status['weekly']['budget']:8.2f}  [{status['weekly']['status']}]")
        if status['weekly']['message']:
            print(f"         {status['weekly']['message']}")
        print(f"Monthly: ${status['monthly']['cost']:8.2f} / ${status['monthly']['budget']:8.2f}  [{status['monthly']['status']}]")
        if status['monthly']['message']:
            print(f"         {status['monthly']['message']}")
        print(f"\nKill switch: {'🚨 ACTIVE' if status['kill_switch'] else '✓ OK'}")
    
    elif args.check:
        can_proceed = tamer.check_before_call()
        if can_proceed:
            print("✓ OK: API call can proceed")
            sys.exit(0)
        else:
            print("✗ BLOCKED: Kill switch active or budget exceeded")
            sys.exit(1)
    
    elif args.log:
        if not all([args.provider, args.model, args.input_tokens is not None, args.output_tokens is not None]):
            print("ERROR: --log requires --provider, --model, --input-tokens, --output-tokens")
            sys.exit(1)
        
        cost, status = tamer.log_usage(
            provider=args.provider,
            model=args.model,
            input_tokens=args.input_tokens,
            output_tokens=args.output_tokens,
            task=args.task,
            session=args.session
        )
        
        print(f"✓ Logged: ${cost:.4f} [{status}]")
        
        if status == 'KILL':
            sys.exit(1)
    
    else:
        parser.print_help()


if __name__ == '__main__':
    main()


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)