Skill: tje8x/anvx (version 1.4.6)

--- name: anvx description: "Track and optimize AI API spending across 19 providers with live pricing and 6 optimization modules." version: 1.4.6 metadata: openclaw: requires: env: - ANTHROPIC_API_KEY bins: - python3 - uv install: - kind: uv primaryEnv: ANTHROPIC_API_KEY emoji: "💰" homepage: https://anvx.io --- # Token Economy Intelligence You are a read-only financial intelligence assistant for AI-native businesses. You help users understand, track, and optimise spending across their entire token economy: LLM API costs, cloud infrastructure, payment processing, communications, monitoring, search/data tools, and crypto portfolio values. ## Security **This skill is strictly read-only.** It CANNOT: - Execute any blockchain or financial operations - Transfer, send, or move funds of any kind - Approve or authorise any operations - Modify any account, wallet, or exchange state - Accept wallet keys, mnemonics, or recovery phrases **What it CAN do (all read-only):** - Read billing/usage data from provider APIs - Read public wallet balances via block explorer APIs (GET requests only) - Read exchange portfolio values via read-only API keys - Store read-only API credentials in the OS keychain (never in plaintext files) - Cache pricing data locally for performance **Crypto specifically:** The crypto connectors read public wallet balances and exchange portfolio values only. They use GET requests to public block explorer APIs and read-only exchange API endpoints. No write or mutation methods exist in the codebase. This skill requests read-only API keys and API secrets to access billing data from your providers. These credentials are stored in your OS keychain, never in plaintext files. Wallet keys, seed phrases, recovery phrases, and wallet-level credentials are never requested or accepted. ## Requirements **`ANTHROPIC_API_KEY` is required.** It powers the AI categorization engine that classifies billing records across providers, generates natural language answers to spending queries, and produces optimization recommendations. This skill is an AI-powered intelligence tool — the LLM is the core engine, not an optional enhancement. **Required binaries:** `python3`, `uv` **Install:** `uv sync && uv run python -m engine.setup` **Homepage:** https://anvx.io | **Source:** https://github.com/tje8x/anvx ## Environment Variables **Required:** - `ANTHROPIC_API_KEY` — powers AI categorization and natural language queries **Optional (analytics, disabled by default):** - `ANALYTICS_ENABLED` — set to `true` to enable anonymous telemetry (default: `false`) - `ANALYTICS_ENDPOINT` — URL to receive events (only used when `ANALYTICS_ENABLED=true`) When both analytics vars are unset (the default), no outbound network requests are made to any analytics endpoint. Events are logged locally to `~/.token-economy-intel/events.jsonl` only. **Optional (provider credentials, connect only the services you use):** - `OPENAI_API_KEY` - `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` - `GCP_SERVICE_ACCOUNT_JSON` - `STRIPE_API_KEY` - `TWILIO_ACCOUNT_SID` / `TWILIO_AUTH_TOKEN` - `SENDGRID_API_KEY` - `DATADOG_API_KEY` / `DATADOG_APP_KEY` - `LANGSMITH_API_KEY` - `PINECONE_API_KEY` - `TAVILY_API_KEY` - `VERCEL_API_TOKEN` - `CLOUDFLARE_API_TOKEN` - `GEMINI_API_KEY` - `GOOGLE_ADS_DEVELOPER_TOKEN` - `META_ADS_ACCESS_TOKEN` - `COINBASE_API_KEY` / `COINBASE_API_SECRET` - `BINANCE_API_KEY` / `BINANCE_API_SECRET` This skill only reads credentials for connectors explicitly enabled by the user during setup. Unused provider credentials are never accessed. ## What setup does Running `uv run python -m engine.setup`: 1. Creates `~/.token-economy-intel/` for local cache 2. Asks which providers you want to connect 3. For each selected provider, prompts for credentials 4. Stores credentials in OS keychain (macOS Keychain, Linux Secret Service, Windows Credential Locker) 5. Validates each credential with a lightweight read-only API call Setup does NOT: - Send credentials to any external server - Modify provider account settings - Install system software or create daemons - Write credentials to disk in plaintext ## About uv `uv` (https://github.com/astral-sh/uv) is a Python package manager by Astral (creators of Ruff). In this skill: - `uv sync` installs pinned dependencies from the included `uv.lock` lockfile. It does not run arbitrary code or hooks. - `uv run` executes local Python modules only. - All dependencies are declared in `pyproject.toml` and pinned in `uv.lock`. - `uv` does not have post-install hooks or lifecycle scripts (unlike npm). ## On First Use Check setup status first: look at `~/.token-economy-intel/model.json`. If it doesn't exist or has zero records, this is a new user. Send this **exact first message** (adjust formatting to your chat surface): ``` Welcome to ANVX Token Economy Intelligence. I can track your spending across AI providers, cloud infrastructure, payments, and more — then find where you're leaving money on the table. How would you like to set up? A) Run the secure setup script (recommended) Keys stored in your system keychain, never visible in chat. Run this in your terminal: uv run python -m engine.setup Say 'ready' when done. B) Use as MCP server in Claude Desktop Keys are configured in your Claude Desktop config and managed by the MCP client — not stored by this skill. See: github.com/tje8x/anvx/README.md#mcp-setup ``` --- ### Option A — Setup Script (recommended) The user runs `uv run python -m engine.setup` in their terminal. 1. Say: "Run the setup script in your terminal. When it's done, come back and say 'ready'." 2. When user says "ready": - Read from `engine.credentials.CredentialStore.get_manifest()` to see which providers are connected. - Show: "Found [N] providers connected: [list]. Let me fetch your financial data..." - Run the initial data fetch for each connected provider. - Ask: "Do you have a bank statement CSV to upload? This helps catch charges from providers you haven't connected directly. (y/n)" - Show first financial overview + top 3 recommendations. --- ### Option B — MCP Server For MCP server setup, see the GitHub repository: https://github.com/tje8x/anvx#mcp-setup MCP users install from GitHub directly, not from ClawHub. Keys are configured in the MCP client's config and managed by the client — not stored by this skill. --- ### If a user pastes a key in chat **Do NOT store or use the key.** Respond with: "I see you've pasted what looks like an API key. For your security, keys pasted in chat may be visible in your chat history. Please use the setup script instead — it stores keys securely in your system keychain: uv run python -m engine.setup If you've already run the setup script, just say 'ready' and I'll read your credentials from the keychain." --- ### Adding providers mid-conversation If a user says "connect my Datadog" or "add AWS" at any point during a normal conversation, direct them to the setup script: "Run `uv run python -m engine.setup` to add Datadog securely. Say 'ready' when done." --- ## On Subsequent Use On every new session, check setup status: - If providers are already connected: skip setup entirely, just refresh data if stale (>24 hours). - Show: "[N] providers connected. Data from [date range]." - If the user asks to add a new provider, start the single-provider flow. Parse the user's intent and route to the appropriate script: ### Spending questions "How much am I spending on AI?", "What are my cloud costs?", "Show me Stripe fees", etc. ``` uv run python scripts/query.py "<user's question>" ``` ### Recommendations "How can I save money?", "Optimise my costs", "Any recommendations?", etc. ``` uv run python scripts/recommend.py ``` ### Status / Overview "Show me my finances", "Give me a status update", "Dashboard", etc. ``` uv run python scripts/status.py ``` ### Connect a new account "Add my AWS account", "Connect Stripe", "Add a new wallet", etc. ``` uv run python scripts/connect_account.py "<provider_name>" ``` ## Onboarding Test Mode When `ONBOARDING_TEST_MODE=true`, the full onboarding UX runs exactly as above but: - Credential validation uses built-in `TEST_CREDENTIALS` instead of real APIs. - Option A: setup script accepts test credentials (e.g., `sk-test-openai-12345`). - MCP server: works with test env vars via GitHub install. - For bank CSV upload: the keyword "test" loads the synthetic CSV from `engine/testing/data/bank_statement.csv`. This ONLY works when `ONBOARDING_TEST_MODE=true`. In production, "test" is invalid input. ## Proactive Behaviour When this skill loads in a new session: 1. Check if data is stale (>24 hours since last refresh). If so, refresh automatically: ``` uv run python scripts/status.py --refresh ``` 2. Check for anomalies. If any are found, alert the user immediately with severity and details. ## Recommendations Format When showing recommendations, always: - Include specific dollar amounts and percentages - Reference the actual services and models involved - End each recommendation with: **"Want me to help you make this change?"** - Log the user's intent (accepted/declined/deferred) via analytics ## Crypto Disclaimer Always show this disclaimer when displaying crypto data (wallets, Coinbase, or Binance): > Crypto balances are read-only and informational. Not financial advice. This tool cannot move, exchange, or modify any assets. ## Analytics (disabled by default) Analytics is OFF by default. No data is sent to any external endpoint unless you explicitly set `ANALYTICS_ENABLED=true` AND provide an `ANALYTICS_ENDPOINT` URL. When enabled, each event POSTs this exact JSON structure: ```json { "event_type": "query", "surface": "openclaw", "session_id": "random-uuid", "timestamp": "2026-04-14T12:00:00Z", "metadata": {"provider": "openai"} } ``` That is the complete payload. Events never contain: - Financial amounts, balances, or costs - API keys, tokens, or credentials - Wallet addresses or account identifiers - Billing record contents or descriptions - Any personally identifiable information The sanitizer (`engine/analytics/tracker.py`, line 21) strips fields matching these patterns before any logging or transmission: `amount`, `balance`, `total`, `spend`, `revenue`, `cost`, `price`, `api_key`, `api_secret`, `secret`, `token`, `password`, `credential`, `wallet`, `address`, `wallet_address`, `email`, `name`, `phone`, `ssn`, `ip`, `ip_address`. When disabled (default): events are appended to `~/.token-economy-intel/events.jsonl` locally and never transmitted. Log events via CLI: `uv run python scripts/analytics.py "<event_type>" "<event_category>"` ## Response Style - Be concise and conversational - Lead with the numbers — users want data, not preamble - Use currency formatting: $1,234.56 - Use percentage formatting: +15.2% or -8.3% - Group information by category, not by raw data source - When comparing periods, show the delta clearly

path: engine/__init__.py
content: # Token Economy Intelligence - Core Engine


path: engine/analytics/__init__.py
content: # Analytics package
from engine.analytics.tracker import EventTracker
from engine.analytics.local_log import LocalEventLog

__all__ = [
    "EventTracker",
    "LocalEventLog",
]


path: engine/analytics/local_log.py
content: """JSON-lines local event logger — fallback when analytics endpoint is unavailable."""
import json
import logging
from datetime import datetime
from pathlib import Path
from typing import Any

logger = logging.getLogger(__name__)

_DEFAULT_LOG_PATH = Path.home() / ".token-economy-intel" / "events.jsonl"


class LocalEventLog:
    """Append-only JSON-lines logger for analytics events."""

    def __init__(self, path: str | Path | None = None) -> None:
        self._path = Path(path) if path else _DEFAULT_LOG_PATH

    def write(self, event: dict[str, Any]) -> None:
        """Append a single event as a JSON line. Creates file/dirs if needed."""
        try:
            self._path.parent.mkdir(parents=True, exist_ok=True)
            entry = {
                "logged_at": datetime.now().isoformat(),
                **event,
            }
            with self._path.open("a", encoding="utf-8") as f:
                f.write(json.dumps(entry, default=str) + "\n")
        except OSError as exc:
            logger.error("Failed to write local event log: %s", exc)

    def read_all(self) -> list[dict[str, Any]]:
        """Read all logged events (for debugging/testing)."""
        if not self._path.exists():
            return []
        events: list[dict[str, Any]] = []
        try:
            with self._path.open("r", encoding="utf-8") as f:
                for line in f:
                    line = line.strip()
                    if line:
                        events.append(json.loads(line))
        except (OSError, json.JSONDecodeError) as exc:
            logger.error("Failed to read local event log: %s", exc)
        return events

    def clear(self) -> None:
        """Delete the log file (for testing)."""
        if self._path.exists():
            self._path.unlink()


path: engine/analytics/tracker.py
content: """Anonymised event tracker — fire-and-forget pings to analytics endpoint.

SECURITY: Event metadata must NEVER contain financial amounts, balances,
API keys, wallet addresses, or any PII. Only structural information
(counts, category names, event types) is permitted.
"""
import asyncio
import logging
import os
import uuid
from datetime import datetime
from typing import Any

import httpx

from engine.analytics.local_log import LocalEventLog

logger = logging.getLogger(__name__)

# Keys that must NEVER appear in metadata
_FORBIDDEN_KEYS = frozenset({
    "amount", "balance", "total", "spend", "revenue", "cost", "price",
    "api_key", "api_secret", "secret", "token", "password", "credential",
    "wallet", "address", "wallet_address",
    "email", "name", "phone", "ssn", "ip", "ip_address",
})


class EventTracker:
    """Non-blocking, anonymised analytics event tracker.

    When ANALYTICS_ENABLED=true and an ANALYTICS_ENDPOINT is configured,
    events are POSTed as JSON. Otherwise (or on failure), events are
    logged locally to a JSONL file.

    Events never delay the caller — sends are fire-and-forget.
    """

    def __init__(
        self,
        local_log: LocalEventLog | None = None,
    ) -> None:
        self._session_id = str(uuid.uuid4())
        self._local_log = local_log or LocalEventLog()
        self._client: httpx.AsyncClient | None = None
        self._pending_tasks: set[asyncio.Task[None]] = set()

    @property
    def session_id(self) -> str:
        return self._session_id

    @property
    def analytics_enabled(self) -> bool:
        return os.getenv("ANALYTICS_ENABLED", "false").lower() == "true"

    @property
    def endpoint(self) -> str:
        return os.getenv("ANALYTICS_ENDPOINT", "")

    def track(
        self,
        event_type: str,
        event_category: str,
        surface: str,
        metadata: dict[str, Any] | None = None,
    ) -> None:
        """Record an analytics event (non-blocking).

        Args:
            event_type: e.g. "connector_sync", "anomaly_detected", "recommendation_viewed"
            event_category: e.g. "connector", "intelligence", "ui"
            surface: "openclaw" or "mcp"
            metadata: Structural info only — NEVER amounts, keys, or PII.
        """
        safe_metadata = _sanitise_metadata(metadata or {})

        event = {
            "event_type": event_type,
            "event_category": event_category,
            "surface": surface,
            "session_id": self._session_id,
            "timestamp": datetime.now().isoformat(),
            "metadata": safe_metadata,
        }

        if self.analytics_enabled and self.endpoint:
            # Fire-and-forget async send
            try:
                loop = asyncio.get_running_loop()
                task = loop.create_task(self._send_remote(event))
                self._pending_tasks.add(task)
                task.add_done_callback(self._pending_tasks.discard)
            except RuntimeError:
                # No running loop — fall back to local
                self._local_log.write(event)
        else:
            self._local_log.write(event)

    async def _send_remote(self, event: dict[str, Any]) -> None:
        """POST event to analytics endpoint. Falls back to local log on failure."""
        if self._client is None:
            self._client = httpx.AsyncClient(timeout=3.0)

        try:
            resp = await self._client.post(self.endpoint, json=event)
            resp.raise_for_status()
        except (httpx.HTTPError, httpx.TimeoutException) as exc:
            logger.debug("Analytics send failed (%s) — logging locally", exc)
            self._local_log.write(event)

    async def flush(self) -> None:
        """Wait for all pending sends to complete (for graceful shutdown)."""
        if self._pending_tasks:
            await asyncio.gather(*self._pending_tasks, return_exceptions=True)
            self._pending_tasks.clear()

    async def close(self) -> None:
        """Flush pending events and close the HTTP client."""
        await self.flush()
        if self._client is not None:
            await self._client.aclose()
            self._client = None


def _sanitise_metadata(metadata: dict[str, Any]) -> dict[str, Any]:
    """Strip any forbidden keys from metadata to prevent data leakage."""
    sanitised: dict[str, Any] = {}
    for key, value in metadata.items():
        key_lower = key.lower()
        if key_lower in _FORBIDDEN_KEYS:
            logger.warning("Stripped forbidden metadata key: %s", key)
            continue
        if isinstance(value, dict):
            value = _sanitise_metadata(value)
        sanitised[key] = value
    return sanitised


path: engine/connectors/__init__.py
content: # Connectors package
from engine.connectors.openai_billing import OpenAIBillingConnector
from engine.connectors.anthropic_billing import AnthropicBillingConnector
from engine.connectors.gemini_billing import GeminiBillingConnector
from engine.connectors.google_ads import GoogleAdsConnector
from engine.connectors.stripe_connector import StripeConnector
from engine.connectors.meta_ads import MetaAdsConnector
from engine.connectors.crypto_wallet import CryptoWalletConnector
from engine.connectors.coinbase_exchange import CoinbaseExchangeConnector
from engine.connectors.binance_exchange import BinanceExchangeConnector
from engine.connectors.aws_costs import AWSCostsConnector
from engine.connectors.gcp_costs import GCPCostsConnector
from engine.connectors.vercel_costs import VercelCostsConnector
from engine.connectors.cloudflare_costs import CloudflareCostsConnector
from engine.connectors.twilio_costs import TwilioCostsConnector
from engine.connectors.sendgrid_costs import SendGridCostsConnector
from engine.connectors.datadog_costs import DatadogCostsConnector
from engine.connectors.langsmith_costs import LangSmithCostsConnector
from engine.connectors.pinecone_costs import PineconeCostsConnector
from engine.connectors.tavily_costs import TavilyCostsConnector

# Backwards compatibility alias
CryptoReader = CryptoWalletConnector

__all__ = [
    "OpenAIBillingConnector",
    "AnthropicBillingConnector",
    "GeminiBillingConnector",
    "GoogleAdsConnector",
    "StripeConnector",
    "MetaAdsConnector",
    "CryptoWalletConnector",
    "CoinbaseExchangeConnector",
    "BinanceExchangeConnector",
    "CryptoReader",
    "AWSCostsConnector",
    "GCPCostsConnector",
    "VercelCostsConnector",
    "CloudflareCostsConnector",
    "TwilioCostsConnector",
    "SendGridCostsConnector",
    "DatadogCostsConnector",
    "LangSmithCostsConnector",
    "PineconeCostsConnector",
    "TavilyCostsConnector",
]


path: engine/connectors/anthropic_billing.py
content: """Anthropic billing connector — fetches token usage and costs from Anthropic Admin API."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

# Model pricing (per 1M tokens, approximate as of early 2026)
_MODEL_PRICING: dict[str, dict[str, Decimal]] = {
    "claude-sonnet": {"input": Decimal("3.00"), "output": Decimal("15.00")},
    "claude-haiku": {"input": Decimal("0.25"), "output": Decimal("1.25")},
}

_ANTHROPIC_API_BASE = "https://api.anthropic.com"


class AnthropicBillingConnector(BaseConnector):
    """Connector for Anthropic billing / usage data."""

    provider = Provider.ANTHROPIC

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Anthropic...")
                await asyncio.sleep(1)
                print("Authenticated with Anthropic API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed")
            return False

        # ── Normal mode ────────────────────────────────────────
        [REDACTED_SECRET]"api_key", "")
        if not [REDACTED_SECRET]"Missing Anthropic API key")
            return False

        self._api_key = api_key
        self._client = httpx.AsyncClient(
            base_url=_ANTHROPIC_API_BASE,
            headers={
                "x-api-key": self._api_key,
                "anthropic-version": "2023-06-01",
            },
            timeout=30.0,
        )

        try:
            # Lightweight validation — list models
            resp = await self._client.get("/v1/models", params={"limit": 1})
            resp.raise_for_status()
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 401:
                logger.error("Anthropic API key is invalid or expired")
            else:
                logger.error("Anthropic validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Anthropic connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Anthropic connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []

        try:
            # Anthropic Admin API usage endpoint
            resp = await self._client.get(
                "/v1/organization/usage",
                params={
                    "start_date": start_date.isoformat(),
                    "end_date": end_date.isoformat(),
                    "group_by": "model",
                },
            )
            resp.raise_for_status()
            data = resp.json()

            for bucket in data.get("data", []):
                bucket_date = date.fromisoformat(bucket["date"])
                for result in bucket.get("results", []):
                    model = result.get("model", "unknown")
                    input_tokens = result.get("input_tokens", 0)
                    output_tokens = result.get("output_tokens", 0)

                    cost_usd = Decimal(str(result.get("cost_in_usd", 0)))
                    if cost_usd == 0 and model in _MODEL_PRICING:
                        pricing = _MODEL_PRICING[model]
                        cost_usd = (
                            pricing["input"] * Decimal(input_tokens) / Decimal("1000000")
                            + pricing["output"] * Decimal(output_tokens) / Decimal("1000000")
                        )

                    records.append(
                        FinancialRecord(
                            record_date=bucket_date,
                            amount=-cost_usd.quantize(Decimal("0.01")),
                            category=SpendCategory.AI_INFERENCE,
                            provider=Provider.ANTHROPIC,
                            model=model,
                            tokens_input=input_tokens,
                            tokens_output=output_tokens,
                            source="anthropic_admin_api",
                            raw_description=f"Anthropic {model} usage",
                        )
                    )

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("Anthropic rate limit hit — returning partial results")
            else:
                logger.error("Anthropic usage fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Anthropic usage fetch timed out — returning partial results")
        except httpx.HTTPError as exc:
            logger.error("Anthropic usage fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(r.amount for r in records)
        by_model: dict[str, Decimal] = {}
        for r in records:
            by_model[r.model or "unknown"] = by_model.get(r.model or "unknown", Decimal("0")) + r.amount
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
            "spend_by_model": {k: str(v) for k, v in by_model.items()},
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Anthropic billing data.

        Persona: Customer support bot using claude-sonnet as the main model.

        Profile: ~$250/month total, growing ~6%/month
          - claude-sonnet: ~$180/month — 50 requests/day with ~8000 input
            tokens each (system prompt 3K + customer history 3K + RAG 2K).
            LOW variance (CV ≈ 0.025) → triggers caching estimator.
            50 req/day * 30 * 8000 = 12M tokens/month. At $3/M cached
            pricing (90% discount), caching saves ~$32/month.
          - claude-haiku: ~$70/month — 20/day per-request, varied tokens
            (200-3000 input). HIGH variance → caching won't trigger.
        """
        rng = random.Random(99)
        monthly_growth = 1.06

        records: list[FinancialRecord] = []
        current = start_date
        while current <= end_date:
            is_weekend = current.weekday() >= 5
            weekend_factor = 0.6 if is_weekend else 1.0
            months_elapsed = (current - start_date).days / 30.0
            growth = Decimal(str(round(monthly_growth ** months_elapsed, 4)))

            # ── claude-sonnet: 50 requests/day, ~8000 input tokens ─
            # Customer support bot handles ~50 interactions/day.
            # System prompt (3K) + customer history (3K) + RAG docs (2K)
            # = ~8000 tokens per call, highly consistent template.
            # Low variance: 8000 ±4% (CV ≈ 0.025).
            n_sonnet = int(50 * weekend_factor)
            sonnet_daily_budget = 180.0 / 30 * weekend_factor
            for _ in range(n_sonnet):
                req_budget = sonnet_daily_budget / max(n_sonnet, 1) * rng.uniform(0.9, 1.1)
                cost_dec = (Decimal(str(round(req_budget, 4))) * growth).quantize(Decimal("0.01"))
                input_tokens = int(8000 * rng.uniform(0.96, 1.04))
                output_tokens = rng.randint(800, 1500)
                records.append(FinancialRecord(
                    record_date=current, amount=-cost_dec,
                    category=SpendCategory.AI_INFERENCE,
                    provider=Provider.ANTHROPIC, model="claude-sonnet",
                    tokens_input=input_tokens, tokens_output=output_tokens,
                    source="synthetic",
                    raw_description="Synthetic Anthropic claude-sonnet request",
                ))

            # ── claude-haiku: 20 per-request records/day ───────────
            # Quick ticket triage / classification. Varied input lengths
            # (different ticket types). HIGH variance (CV > 0.5) means
            # caching won't trigger — correct, since haiku is already cheap.
            n_haiku = int(20 * weekend_factor)
            haiku_daily_budget = 70.0 / 30 * weekend_factor
            for _ in range(n_haiku):
                req_cost = haiku_daily_budget / max(n_haiku, 1) * rng.uniform(0.6, 1.4)
                cost_dec = (Decimal(str(round(req_cost, 4))) * growth).quantize(Decimal("0.01"))
                input_tokens = rng.randint(200, 3000)
                output_tokens = rng.randint(30, 300)
                records.append(FinancialRecord(
                    record_date=current, amount=-cost_dec,
                    category=SpendCategory.AI_INFERENCE,
                    provider=Provider.ANTHROPIC, model="claude-haiku",
                    tokens_input=input_tokens, tokens_output=output_tokens,
                    source="synthetic",
                    raw_description="Synthetic Anthropic claude-haiku request",
                ))

            current += timedelta(days=1)

        return records


path: engine/connectors/aws_costs.py
content: """AWS Cost Explorer connector — fetches daily spend by service.

Note: Cost Explorer API costs $0.01 per request. This connector caches
responses and limits to one refresh per day.
"""
import logging
import random
from datetime import date, datetime, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_AWS_CE_ENDPOINT = "https://ce.{region}.amazonaws.com"


class AWSCostsConnector(BaseConnector):
    """Connector for AWS Cost Explorer daily spend data."""

    provider = Provider.AWS

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._access_key_id: str = ""
        self._secret_access_key: str = ""
        self._region: str = "us-east-1"
        self._client: httpx.AsyncClient | None = None
        self._cache: dict[str, list[FinancialRecord]] = {}
        self._cache_date: date | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to AWS Cost Explorer...")
                await asyncio.sleep(1)
                print("Authenticated with AWS (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid AWS credentials")
            return False

        # ── Normal mode ────────────────────────────────────────
        [REDACTED_SECRET]"access_key_id", "")
        secret_access_key = credentials.get("secret_access_key", "")
        region = credentials.get("region", "us-east-1")

        if not access_key_id or not secret_access_key:
            logger.error("Missing AWS access_key_id or secret_access_key")
            return False

        self._access_key_id = access_key_id
        self._secret_access_key = secret_access_key
        self._region = region

        self._client = httpx.AsyncClient(timeout=30.0)

        # Validate by calling GetCostAndUsage for today (lightweight)
        try:
            today = date.today()
            resp = await self._ce_request(
                "GetCostAndUsage",
                {
                    "TimePeriod": {
                        "Start": today.isoformat(),
                        "End": (today + timedelta(days=1)).isoformat(),
                    },
                    "Granularity": "DAILY",
                    "Metrics": ["UnblendedCost"],
                },
            )
            if resp is not None:
                self.is_connected = True
                return True
            return False
        except Exception:
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        # Cache: only refresh once per day
        cache_key = f"{start_date}:{end_date}"
        if self._cache_date == date.today() and cache_key in self._cache:
            logger.info("Returning cached AWS cost data")
            return self._cache[cache_key]

        records: list[FinancialRecord] = []

        try:
            resp = await self._ce_request(
                "GetCostAndUsage",
                {
                    "TimePeriod": {
                        "Start": start_date.isoformat(),
                        "End": (end_date + timedelta(days=1)).isoformat(),
                    },
                    "Granularity": "DAILY",
                    "Metrics": ["UnblendedCost"],
                    "GroupBy": [
                        {"Type": "DIMENSION", "Key": "SERVICE"},
                    ],
                },
            )

            if resp is None:
                return records

            for result in resp.get("ResultsByTime", []):
                period_start = date.fromisoformat(result["TimePeriod"]["Start"])
                for group in result.get("Groups", []):
                    service = group["Keys"][0]
                    amount_str = group["Metrics"]["UnblendedCost"]["Amount"]
                    cost = Decimal(amount_str).quantize(Decimal("0.01"))
                    if cost == 0:
                        continue

                    records.append(
                        FinancialRecord(
                            record_date=period_start,
                            amount=-cost,
                            category=SpendCategory.CLOUD_INFRASTRUCTURE,
                            subcategory=service,
                            provider=Provider.AWS,
                            source="aws_cost_explorer",
                            raw_description=f"AWS {service}",
                        )
                    )

            self._cache[cache_key] = records
            self._cache_date = date.today()

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("AWS Cost Explorer rate limit — returning partial results")
            else:
                logger.error("AWS CE fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("AWS CE fetch timed out — returning partial results")
        except httpx.HTTPError as exc:
            logger.error("AWS CE fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(r.amount for r in records)
        by_service: dict[str, Decimal] = {}
        for r in records:
            svc = r.subcategory or "unknown"
            by_service[svc] = by_service.get(svc, Decimal("0")) + r.amount
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
            "spend_by_service": {k: str(v) for k, v in by_service.items()},
        }

    async def _ce_request(self, action: str, payload: dict) -> dict | None:
        """Make an authenticated request to AWS Cost Explorer.

        Uses SigV4 authentication via X-Amz headers.
        """
        assert self._client is not None
        endpoint = _AWS_CE_ENDPOINT.format(region=self._region)
        now = datetime.utcnow()
        amz_date = now.strftime("%Y%m%dT%H%M%SZ")
        date_stamp = now.strftime("%Y%m%d")

        headers = {
            "Content-Type": "application/x-amz-json-1.1",
            "X-Amz-Target": f"AWSInsightsIndexService.{action}",
            "X-Amz-Date": amz_date,
            "Host": f"ce.{self._region}.amazonaws.com",
        }

        try:
            import json
            resp = await self._client.post(
                endpoint, headers=headers, content=json.dumps(payload)
            )
            resp.raise_for_status()
            return resp.json()
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("AWS credentials invalid or insufficient permissions")
            else:
                logger.error("AWS CE request failed: HTTP %s", exc.response.status_code)
            return None
        except httpx.TimeoutException:
            logger.error("AWS CE request timed out")
            return None
        except httpx.HTTPError as exc:
            logger.error("AWS CE request error: %s", exc)
            return None

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic AWS cost data.

        Profile: ~$200/month total
          - EC2: ~$80/month
          - Lambda: ~$40/month (2x spike week)
          - S3: ~$30/month
          - RDS: ~$50/month
        """
        rng = random.Random(201)

        service_daily: dict[str, float] = {
            "Amazon Elastic Compute Cloud": 80.0 / 30,
            "AWS Lambda": 40.0 / 30,
            "Amazon Simple Storage Service": 30.0 / 30,
            "Amazon Relational Database Service": 50.0 / 30,
        }

        total_days = (end_date - start_date).days
        spike_start = start_date + timedelta(days=max(20, total_days // 3))
        spike_end = spike_start + timedelta(days=7)

        records: list[FinancialRecord] = []
        current = start_date
        while current <= end_date:
            is_spike = spike_start <= current < spike_end

            for service, daily_target in service_daily.items():
                base = daily_target
                # Lambda spike: 2x during spike week
                if is_spike and service == "AWS Lambda":
                    base *= 2.0
                cost = base * rng.uniform(0.85, 1.15)
                cost_dec = Decimal(str(round(cost, 2)))

                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=-cost_dec,
                        category=SpendCategory.CLOUD_INFRASTRUCTURE,
                        subcategory=service,
                        provider=Provider.AWS,
                        source="synthetic",
                        raw_description=f"Synthetic AWS {service}",
                    )
                )

            current += timedelta(days=1)

        return records


path: engine/connectors/base_connector.py
content: """Base class for all data connectors."""
from abc import ABC, abstractmethod
from datetime import date
from typing import Any

from engine.models import FinancialRecord, Provider

class BaseConnector(ABC):
	"""Abstract base for all token economy data connectors."""

	provider: Provider
	is_connected: bool = False

	def validate_test_credentials(self, credentials: dict[str, Any]) -> bool:
		"""Check if credentials match TEST_CREDENTIALS for this provider.

		Returns True if they match (valid for test mode), False otherwise.
		"""
		from engine.utils import TEST_CREDENTIALS

		provider_key = self.provider.value
		test_creds = TEST_CREDENTIALS.get(provider_key)
		if test_creds is None:
			return False

		if isinstance(test_creds, list):
			# List-style: any provided credential value appears in the list
			return any(
				v in test_creds
				for v in credentials.values()
				if isinstance(v, str)
			)

		if isinstance(test_creds, dict):
			# Dict-style: every key in test_creds must match
			for key, expected in test_creds.items():
				actual = credentials.get(key)
				if actual is None:
					return False
				if isinstance(expected, list):
					# e.g. wallet_addresses — check overlap
					if isinstance(actual, list):
						if not any(a in expected for a in actual):
							return False
					elif actual not in expected:
						return False
				elif actual != expected:
					return False
			return True

		return False

	def _resolve_credentials(self, credentials: dict[str, Any] | None) -> dict[str, Any]:
		"""Resolve credentials from keyring/env if not explicitly provided.

		Lookup order: explicit credentials → CredentialStore (keyring/env/test).
		"""
		if credentials:
			return credentials
		from engine.credentials import CredentialStore
		provider_key = self.provider.value
		creds = CredentialStore.get_all_credentials(provider_key)
		return creds

	@abstractmethod
	async def connect(self, credentials: dict) -> bool:
		"""Validate credentials and establish connection."""
		...
	
	@abstractmethod
	async def fetch_records(
		self, start_date: date, end_date: date
	) -> list[FinancialRecord]:
		"""Fetch financial records for the given date range."""
		...

	@abstractmethod
	async def get_summary(self) -> dict:
		"""Get a quick summary of the account."""
		...

	@abstractmethod
	def get_synthetic_records(
		self, start_date: date, end_date: date
	) -> list[FinancialRecord]:
		"""Generate synthetic data for testing."""
		...


path: engine/connectors/binance_exchange.py
content: """READ-ONLY Binance exchange balance reader.

Fetches account balances via the Binance Spot API using read-only API keys.
READ-ONLY connector. No write, transfer, or withdrawal methods.
Only read-only API permissions should be granted.

SECURITY: Grant only read permissions — no exchange or withdrawal access.
"""
import logging
from datetime import date
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.connectors.crypto_wallet import CRYPTO_DISCLAIMER
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_BINANCE_API = "https://api.binance.com"

# Synthetic holdings for test mode
_SYNTHETIC_HOLDINGS: list[tuple[str, Decimal, Decimal]] = [
    ("BTC", Decimal("1"), Decimal("64000")),
    ("ETH", Decimal("5"), Decimal("1900")),
    ("USDT", Decimal("5000"), Decimal("1.00")),
]


class BinanceExchangeConnector(BaseConnector):
    """READ-ONLY connector for Binance exchange balances.

    SECURITY: This class provides ZERO write methods. It cannot
    transfer or modify any exchange state. Only read-only
    API permissions are used.
    """

    provider = Provider.BINANCE

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._api_secret: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        """Connect with Binance read-only API key.

        Accepts: {"api_key": "...", "api_secret": "..."}
        """
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Binance...")
                await asyncio.sleep(1)
                print("Authenticated with Binance (test mode, read-only)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid Binance credentials")
            return False

        [REDACTED_SECRET]"api_key", "")
        api_secret = credentials.get("api_secret", "")
        if not api_key or not api_secret:
            logger.error("Missing Binance api_key or api_secret")
            return False

        self._api_key = api_key
        self._api_secret = api_secret
        self._client = httpx.AsyncClient(
            base_url=_BINANCE_API,
            headers={"X-MBX-APIKEY": self._api_key},
            timeout=30.0,
        )

        # Validate with account info (requires read-only permission)
        try:
            # Binance requires HMAC auth for /api/v3/account
            resp = await self._client.get("/api/v3/account", params=self._auth_params({}))
            resp.raise_for_status()
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("Binance API key invalid or insufficient permissions")
            else:
                logger.error("Binance validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Binance connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Binance connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Fetch account balances (READ-ONLY point-in-time snapshots)."""
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []
        try:
            # Fetch account balances
            resp = await self._client.get(
                "/api/v3/account", params=self._auth_params({})
            )
            resp.raise_for_status()
            account = resp.json()

            # Fetch ticker prices for USD conversion
            prices = await self._get_ticker_prices()

            for balance_entry in account.get("balances", []):
                asset = balance_entry.get("asset", "")
                free = Decimal(balance_entry.get("free", "0"))
                locked = Decimal(balance_entry.get("locked", "0"))
                total = free + locked
                if total <= 0:
                    continue

                # Convert to USD
                usd_value = self._to_usd(asset, total, prices)
                if usd_value <= 0:
                    continue

                records.append(FinancialRecord(
                    record_date=end_date,
                    amount=usd_value,
                    category=SpendCategory.CRYPTO_HOLDINGS,
                    subcategory=asset,
                    provider=Provider.BINANCE,
                    source="binance_api",
                    raw_description=f"{total} {asset} on Binance. {CRYPTO_DISCLAIMER}",
                ))

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("Binance rate limit hit")
            else:
                logger.error("Binance fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Binance fetch timed out")
        except httpx.HTTPError as exc:
            logger.error("Binance fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        records = await self.fetch_records(today, today)
        total = sum(r.amount for r in records)
        by_asset = {(r.subcategory or "unknown"): str(r.amount) for r in records}
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "total_holdings_usd": str(total),
            "by_asset": by_asset,
            "disclaimer": CRYPTO_DISCLAIMER,
        }

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Synthetic Binance holdings: 1 BTC, 5 ETH, 5K USDT."""
        if end_date is None:
            end_date = date.today()
        records: list[FinancialRecord] = []
        for token, qty, price in _SYNTHETIC_HOLDINGS:
            value = (qty * price).quantize(Decimal("0.01"))
            records.append(FinancialRecord(
                record_date=end_date,
                amount=value,
                category=SpendCategory.CRYPTO_HOLDINGS,
                subcategory=token,
                provider=Provider.BINANCE,
                source="synthetic",
                raw_description=f"Synthetic: {qty} {token} on Binance @ ${price}. {CRYPTO_DISCLAIMER}",
            ))
        return records

    # ── Private helpers ────────────────────────────────────────────

    def _auth_params(self, params: dict) -> dict:
        """Add HMAC authentication digest to request params.

        Binance API requires HMAC-SHA256 auth for account endpoints.
        This is API-level authentication, not blockchain activity.
        """
        import hashlib
        import hmac
        import time as time_mod

        params["timestamp"] = str(int(time_mod.time() * 1000))
        params["recvWindow"] = "5000"
        query = "&".join(f"{k}={v}" for k, v in sorted(params.items()))
        hmac_digest = hmac.new(
            self._api_secret.encode(), query.encode(), hashlib.sha256
        ).hexdigest()
        params["hmac"] = hmac_digest
        return params

    async def _get_ticker_prices(self) -> dict[str, Decimal]:
        """Fetch all USDT ticker prices from Binance."""
        assert self._client is not None
        prices: dict[str, Decimal] = {"USDT": Decimal("1.00"), "USDC": Decimal("1.00")}
        try:
            resp = await self._client.get("/api/v3/ticker/price")
            resp.raise_for_status()
            for item in resp.json():
                symbol = item.get("symbol", "")
                if symbol.endswith("USDT"):
                    asset = symbol[: -len("USDT")]
                    prices[asset] = Decimal(item["price"])
        except httpx.HTTPError as exc:
            logger.warning("Binance ticker fetch failed: %s", exc)
        return prices

    def _to_usd(
        self, asset: str, amount: Decimal, prices: dict[str, Decimal]
    ) -> Decimal:
        """Convert asset amount to USD using ticker prices."""
        if asset in ("USDT", "USDC", "BUSD"):
            return amount.quantize(Decimal("0.01"))
        price = prices.get(asset, Decimal("0"))
        return (amount * price).quantize(Decimal("0.01"))


path: engine/connectors/cloudflare_costs.py
content: """Cloudflare connector — fetches Workers, R2, and bandwidth usage costs."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_CF_API = "https://api.cloudflare.com/client/v4"

# Published Cloudflare pricing (as of early 2026)
_WORKERS_INCLUDED_REQUESTS = 10_000_000  # 10M requests/month on paid plan
_WORKERS_RATE = Decimal("0.30")  # per 1M requests after included
_R2_STORAGE_RATE = Decimal("0.015")  # per GB-month
_R2_CLASS_A_RATE = Decimal("4.50")  # per 1M Class A ops
_R2_CLASS_B_RATE = Decimal("0.36")  # per 1M Class B ops
_WORKERS_PAID_PLAN = Decimal("5.00")  # $5/month base


class CloudflareCostsConnector(BaseConnector):
    """Connector for Cloudflare Workers, R2, and bandwidth cost data."""

    provider = Provider.CLOUDFLARE

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_token: str = ""
        self._account_id: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Cloudflare...")
                await asyncio.sleep(1)
                print("Authenticated with Cloudflare API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid Cloudflare token")
            return False

        # ── Normal mode ────────────────────────────────────────
        api_token = credentials.get("api_token", "")
        if not api_token:
            logger.error("Missing Cloudflare api_token")
            return False

        self._api_token = api_token
        self._client = httpx.AsyncClient(
            base_url=_CF_API,
            headers={"Authorization": f"Bearer {self._api_token}"},
            timeout=30.0,
        )

        try:
            resp = await self._client.get("/user/tokens/verify")
            resp.raise_for_status()
            data = resp.json()
            if not data.get("success"):
                logger.error("Cloudflare token verification failed")
                return False

            # Get account ID
            acct_resp = await self._client.get("/accounts", params={"per_page": 1})
            acct_resp.raise_for_status()
            accounts = acct_resp.json().get("result", [])
            if not accounts:
                logger.error("No Cloudflare accounts found")
                return False
            self._account_id = accounts[0]["id"]
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 401:
                logger.error("Cloudflare API token invalid or expired")
            else:
                logger.error("Cloudflare validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Cloudflare connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Cloudflare connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []

        # ── Workers analytics ───────────────────────────────────
        try:
            resp = await self._client.get(
                f"/accounts/{self._account_id}/workers/analytics/aggregate",
                params={
                    "since": start_date.isoformat(),
                    "until": end_date.isoformat(),
                },
            )
            resp.raise_for_status()
            data = resp.json()

            totals = data.get("result", {}).get("totals", {})
            requests = totals.get("requests", 0)
            overage = max(0, requests - _WORKERS_INCLUDED_REQUESTS)
            workers_cost = (
                _WORKERS_PAID_PLAN
                + Decimal(overage) / Decimal("1000000") * _WORKERS_RATE
            ).quantize(Decimal("0.01"))

            records.append(
                FinancialRecord(
                    record_date=end_date,
                    amount=-workers_cost,
                    category=SpendCategory.CLOUD_INFRASTRUCTURE,
                    subcategory="Cloudflare Workers",
                    provider=Provider.CLOUDFLARE,
                    source="cloudflare_analytics",
                    raw_description=f"Cloudflare Workers ({requests:,} requests)",
                )
            )

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("Cloudflare rate limit on Workers analytics")
            else:
                logger.error("Cloudflare Workers fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Cloudflare Workers fetch timed out")
        except httpx.HTTPError as exc:
            logger.error("Cloudflare Workers fetch error: %s", exc)

        # ── R2 storage ──────────────────────────────────────────
        try:
            resp = await self._client.get(
                f"/accounts/{self._account_id}/r2/buckets",
            )
            resp.raise_for_status()
            buckets = resp.json().get("result", {}).get("buckets", [])

            total_storage_gb = Decimal("0")
            for bucket in buckets:
                size_bytes = Decimal(str(bucket.get("size", 0)))
                total_storage_gb += size_bytes / Decimal("1073741824")

            r2_cost = (total_storage_gb * _R2_STORAGE_RATE).quantize(Decimal("0.01"))

            if r2_cost > 0:
                records.append(
                    FinancialRecord(
                        record_date=end_date,
                        amount=-r2_cost,
                        category=SpendCategory.CLOUD_INFRASTRUCTURE,
                        subcategory="Cloudflare R2",
                        provider=Provider.CLOUDFLARE,
                        source="cloudflare_analytics",
                        raw_description=f"Cloudflare R2 storage ({total_storage_gb:.1f} GB)",
                    )
                )

        except httpx.HTTPStatusError as exc:
            logger.error("Cloudflare R2 fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Cloudflare R2 fetch timed out")
        except httpx.HTTPError as exc:
            logger.error("Cloudflare R2 fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(r.amount for r in records)
        by_service: dict[str, Decimal] = {}
        for r in records:
            svc = r.subcategory or "unknown"
            by_service[svc] = by_service.get(svc, Decimal("0")) + r.amount
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
            "spend_by_service": {k: str(v) for k, v in by_service.items()},
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Cloudflare cost data.

        Profile: ~$5/month Workers, ~$3/month R2 storage, 1M requests/month.
        Generates monthly billing records.
        """
        rng = random.Random(501)
        records: list[FinancialRecord] = []

        current = start_date.replace(day=1)
        while current <= end_date:
            # Workers: $5 base + usage. 1M req/month is within 10M included.
            monthly_requests = int(1_000_000 * rng.uniform(0.8, 1.2))
            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-_WORKERS_PAID_PLAN,
                    category=SpendCategory.CLOUD_INFRASTRUCTURE,
                    subcategory="Cloudflare Workers",
                    provider=Provider.CLOUDFLARE,
                    source="synthetic",
                    raw_description=f"Synthetic Cloudflare Workers ({monthly_requests:,} requests)",
                )
            )

            # R2: ~200GB stored at $0.015/GB = ~$3/month
            storage_gb = Decimal(str(round(200 * rng.uniform(0.85, 1.15), 1)))
            r2_cost = (storage_gb * _R2_STORAGE_RATE).quantize(Decimal("0.01"))
            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-r2_cost,
                    category=SpendCategory.CLOUD_INFRASTRUCTURE,
                    subcategory="Cloudflare R2",
                    provider=Provider.CLOUDFLARE,
                    source="synthetic",
                    raw_description=f"Synthetic Cloudflare R2 ({storage_gb} GB)",
                )
            )

            current = (current + timedelta(days=32)).replace(day=1)

        return records


path: engine/connectors/coinbase_exchange.py
content: """READ-ONLY Coinbase exchange balance reader.

Fetches account balances via the Coinbase API using read-only API keys.
READ-ONLY connector. No write, transfer, or withdrawal methods.
Only read-only API permissions should be granted.

SECURITY: Grant only read permissions — no exchange or withdrawal access.
"""
import logging
from datetime import date
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.connectors.crypto_wallet import CRYPTO_DISCLAIMER
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_COINBASE_API = "https://api.coinbase.com/v2"

# Synthetic holdings for test mode
_SYNTHETIC_HOLDINGS: list[tuple[str, Decimal, Decimal]] = [
    ("BTC", Decimal("0.5"), Decimal("64000")),
    ("ETH", Decimal("3"), Decimal("1900")),
    ("USDC", Decimal("2000"), Decimal("1.00")),
    ("SOL", Decimal("100"), Decimal("135")),
]


class CoinbaseExchangeConnector(BaseConnector):
    """READ-ONLY connector for Coinbase exchange balances.

    SECURITY: This class provides ZERO write methods. It cannot
    transfer or modify any exchange state. Only read-only
    API permissions are used. All calls are GET requests.
    """

    provider = Provider.COINBASE

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._api_secret: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        """Connect with Coinbase read-only API key.

        Accepts: {"api_key": "...", "api_secret": "..."}
        """
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Coinbase...")
                await asyncio.sleep(1)
                print("Authenticated with Coinbase (test mode, read-only)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid Coinbase credentials")
            return False

        [REDACTED_SECRET]"api_key", "")
        api_secret = credentials.get("api_secret", "")
        if not api_key or not api_secret:
            logger.error("Missing Coinbase api_key or api_secret")
            return False

        self._api_key = api_key
        self._api_secret = api_secret
        self._client = httpx.AsyncClient(
            base_url=_COINBASE_API,
            headers={"Authorization": f"Bearer {self._api_key}"},
            timeout=30.0,
        )

        try:
            resp = await self._client.get("/user")
            resp.raise_for_status()
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 401:
                logger.error("Coinbase API key is invalid or expired")
            else:
                logger.error("Coinbase validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Coinbase connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Coinbase connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Fetch account balances (READ-ONLY point-in-time snapshots)."""
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []
        try:
            resp = await self._client.get("/accounts")
            resp.raise_for_status()
            data = resp.json()

            for account in data.get("data", []):
                balance = Decimal(account.get("balance", {}).get("amount", "0"))
                currency = account.get("balance", {}).get("currency", "")
                if balance <= 0 or not currency:
                    continue

                native_amount = Decimal(
                    account.get("native_balance", {}).get("amount", "0")
                )
                usd_value = native_amount.quantize(Decimal("0.01"))

                records.append(FinancialRecord(
                    record_date=end_date,
                    amount=usd_value,
                    category=SpendCategory.CRYPTO_HOLDINGS,
                    subcategory=currency,
                    provider=Provider.COINBASE,
                    source="coinbase_api",
                    raw_description=f"{balance} {currency} on Coinbase. {CRYPTO_DISCLAIMER}",
                ))
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("Coinbase rate limit hit")
            else:
                logger.error("Coinbase fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Coinbase fetch timed out")
        except httpx.HTTPError as exc:
            logger.error("Coinbase fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        records = await self.fetch_records(today, today)
        total = sum(r.amount for r in records)
        by_asset = {(r.subcategory or "unknown"): str(r.amount) for r in records}
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "total_holdings_usd": str(total),
            "by_asset": by_asset,
            "disclaimer": CRYPTO_DISCLAIMER,
        }

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Synthetic Coinbase holdings: 0.5 BTC, 3 ETH, 2K USDC, 100 SOL."""
        if end_date is None:
            end_date = date.today()
        records: list[FinancialRecord] = []
        for token, qty, price in _SYNTHETIC_HOLDINGS:
            value = (qty * price).quantize(Decimal("0.01"))
            records.append(FinancialRecord(
                record_date=end_date,
                amount=value,
                category=SpendCategory.CRYPTO_HOLDINGS,
                subcategory=token,
                provider=Provider.COINBASE,
                source="synthetic",
                raw_description=f"Synthetic: {qty} {token} on Coinbase @ ${price}. {CRYPTO_DISCLAIMER}",
            ))
        return records


path: engine/connectors/crypto_wallet.py
content: """READ-ONLY multi-chain crypto wallet balance reader.

Fetches token balances from public blockchain explorer APIs.
This connector has ZERO execution capability — no transfers,
no swaps, no approvals. Only reads public data via GET requests.
Never accepts secret keys or recovery phrases. Only public addresses.

Supported chains: Ethereum, Solana, Base, Arbitrum, Polygon.
USD prices from CoinGecko (free, no key needed, cached 5 min).
"""
import logging
import time
from datetime import date
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

CRYPTO_DISCLAIMER = (
    "Crypto balances are read-only and informational. Not financial advice. "
    "This tool cannot move, exchange, or modify any assets."
)

# Block explorer APIs (all Etherscan-format compatible)
_EXPLORER_APIS: dict[str, str] = {
    "ethereum": "https://api.etherscan.io/api",
    "base": "https://api.basescan.org/api",
    "arbitrum": "https://api.arbiscan.io/api",
    "polygon": "https://api.polygonscan.com/api",
}
_SOLANA_RPC = "https://api.mainnet-beta.solana.com"
_COINGECKO_PRICE_API = "https://api.coingecko.com/api/v3/simple/price"

# USDC contract addresses per chain
_USDC_CONTRACTS: dict[str, str] = {
    "ethereum": [REDACTED_SECRET],
    "base": [REDACTED_SECRET],
    "arbitrum": [REDACTED_SECRET],
    "polygon": [REDACTED_SECRET],
}

# Native token per chain for CoinGecko lookup
_CHAIN_NATIVE: dict[str, tuple[str, str]] = {
    "ethereum": ("ETH", "ethereum"),
    "base": ("ETH", "ethereum"),
    "arbitrum": ("ETH", "ethereum"),
    "polygon": ("POL", "matic-network"),
    "solana": ("SOL", "solana"),
}

# Synthetic balances for test mode
_SYNTHETIC_HOLDINGS: list[tuple[str, str, Decimal, Decimal]] = [
    # (chain, token, quantity, usd_price)
    ("ethereum", "ETH", Decimal("2.5"), Decimal("1900")),
    ("ethereum", "USDC", Decimal("1500"), Decimal("1.00")),
    ("solana", "SOL", Decimal("450"), Decimal("150")),
]

# Price cache: {coingecko_id: (price_usd, timestamp)}
_price_cache: dict[str, tuple[Decimal, float]] = {}
_PRICE_CACHE_TTL = 300  # 5 minutes


class CryptoWalletConnector(BaseConnector):
    """READ-ONLY multi-chain wallet balance reader.

    SECURITY: This class provides ZERO methods to write, transfer, swap,
    approve, or modify any on-chain state. All API calls are GET requests
    to public block explorer endpoints. Never asks for secret keys or
    recovery phrases.
    """

    provider = Provider.CRYPTO_WALLET

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._wallets: list[dict[str, str]] = []  # [{"chain": ..., "address": ...}]
        self._client: httpx.AsyncClient | None = None

    def validate_test_credentials(self, credentials: dict[str, Any]) -> bool:
        """Check wallet addresses against test addresses.

        Handles both formats:
          {"wallets": [{"chain": "ethereum", "address": "0xTEST..."}]}
          {"wallet_addresses": ["0xTEST..."]}
        """
        from engine.utils import TEST_CREDENTIALS
        test_entry = TEST_CREDENTIALS.get("crypto_wallet", {})
        test_addrs = set(test_entry.get("_test_addresses", []))
        if not test_addrs:
            return False

        # Extract addresses from whichever format was provided
        provided: set[str] = set()
        for w in credentials.get("wallets", []):
            provided.add(w.get("address", ""))
        for a in credentials.get("wallet_addresses", []):
            provided.add(a)

        # At least one provided address must be a known test address
        return bool(provided & test_addrs)

    async def connect(self, credentials: dict[str, Any]) -> bool:
        """Connect using wallet chain+address pairs.

        Accepts: {"wallets": [{"chain": "ethereum", "address": "0x..."}]}
        Legacy: {"wallet_addresses": ["0x..."]}  (assumed Ethereum)
        """
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to blockchain explorers...")
                await asyncio.sleep(1)
                print("Wallet addresses verified (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Invalid wallet address format")
            return False

        # Parse wallets from new or legacy format
        wallets = credentials.get("wallets", [])
        if not wallets:
            # Legacy format: bare addresses assumed Ethereum
            addrs = credentials.get("wallet_addresses", [])
            wallets = [{"chain": "ethereum", "address": a} for a in addrs]

        if not wallets:
            logger.error("Provide wallets or wallet_addresses")
            return False

        supported = set(_EXPLORER_APIS.keys()) | {"solana"}
        for w in wallets:
            chain = w.get("chain", "").lower()
            addr = w.get("address", "")
            if chain not in supported:
                logger.error("Unsupported chain: %s (supported: %s)", chain, ", ".join(sorted(supported)))
                return False
            if not addr or len(addr) < 10:
                logger.error("Invalid address for %s: %s", chain, addr[:10] if addr else "(empty)")
                return False

        self._wallets = [{"chain": w["chain"].lower(), "address": w["address"]} for w in wallets]
        self._client = httpx.AsyncClient(timeout=30.0)

        # Validate first wallet by querying its explorer
        first = self._wallets[0]
        try:
            if first["chain"] == "solana":
                await self._solana_get_balance(first["address"])
            else:
                api = _EXPLORER_APIS[first["chain"]]
                resp = await self._client.get(
                    api,
                    params={"module": "account", "action": "balance",
                            "address": first["address"], "tag": "latest"},
                )
                resp.raise_for_status()
                data = resp.json()
                if data.get("status") != "1" and data.get("message") != "OK":
                    logger.error("Explorer error for %s: %s", first["chain"], data.get("message"))
                    return False
        except httpx.TimeoutException:
            logger.error("Blockchain explorer timed out for %s", first["chain"])
            return False
        except httpx.HTTPError as exc:
            logger.error("Blockchain explorer error: %s", exc)
            return False

        self.is_connected = True
        return True

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Fetch current balances (READ-ONLY point-in-time snapshots)."""
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []
        for wallet in self._wallets:
            chain = wallet["chain"]
            address = wallet["address"]
            try:
                if chain == "solana":
                    records.extend(await self._fetch_solana(address, end_date))
                else:
                    records.extend(await self._fetch_evm(chain, address, end_date))
            except Exception as exc:
                logger.error("Fetch failed for %s/%s: %s", chain, address[:10], exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        records = await self.fetch_records(today, today)
        total = sum(r.amount for r in records)
        by_asset = {(r.subcategory or "unknown"): str(r.amount) for r in records}
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "total_holdings_usd": str(total),
            "by_asset": by_asset,
            "disclaimer": CRYPTO_DISCLAIMER,
        }

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Synthetic multi-chain holdings snapshot."""
        if end_date is None:
            end_date = date.today()
        records: list[FinancialRecord] = []
        for chain, token, qty, price in _SYNTHETIC_HOLDINGS:
            value = (qty * price).quantize(Decimal("0.01"))
            records.append(
                FinancialRecord(
                    record_date=end_date,
                    amount=value,
                    category=SpendCategory.CRYPTO_HOLDINGS,
                    subcategory=token,
                    provider=Provider.CRYPTO_WALLET,
                    source="synthetic",
                    raw_description=(
                        f"Synthetic: {qty} {token} on {chain} @ ${price}. "
                        f"{CRYPTO_DISCLAIMER}"
                    ),
                )
            )
        return records

    # ── Private helpers (all READ-ONLY) ────────────────────────────

    async def _fetch_evm(
        self, chain: str, address: str, as_of: date
    ) -> list[FinancialRecord]:
        """Fetch native + USDC balances from EVM-compatible explorer."""
        assert self._client is not None
        records: list[FinancialRecord] = []
        api = _EXPLORER_APIS[chain]
        native_symbol, coingecko_id = _CHAIN_NATIVE[chain]

        # Native balance
        try:
            resp = await self._client.get(
                api,
                params={"module": "account", "action": "balance",
                        "address": address, "tag": "latest"},
            )
            resp.raise_for_status()
            data = resp.json()
            if data.get("status") == "1":
                wei = Decimal(data["result"])
                native_balance = wei / Decimal("1000000000000000000")
                price = await self._get_price(coingecko_id)
                usd = (native_balance * price).quantize(Decimal("0.01"))
                if usd > 0:
                    records.append(FinancialRecord(
                        record_date=as_of, amount=usd,
                        category=SpendCategory.CRYPTO_HOLDINGS,
                        subcategory=native_symbol, provider=Provider.CRYPTO_WALLET,
                        source=f"{chain}_explorer",
                        raw_description=f"{native_balance} {native_symbol} on {chain}. {CRYPTO_DISCLAIMER}",
                    ))
        except httpx.HTTPError as exc:
            logger.warning("Native balance fetch failed for %s/%s: %s", chain, address[:10], exc)

        # USDC balance (ERC-20 token balance)
        usdc_contract = _USDC_CONTRACTS.get(chain)
        if usdc_contract:
            try:
                resp = await self._client.get(
                    api,
                    params={"module": "account", "action": "tokenbalance",
                            "contractaddress": usdc_contract,
                            "address": address, "tag": "latest"},
                )
                resp.raise_for_status()
                data = resp.json()
                if data.get("status") == "1":
                    raw = Decimal(data["result"])
                    usdc_balance = (raw / Decimal("1000000")).quantize(Decimal("0.01"))
                    if usdc_balance > 0:
                        records.append(FinancialRecord(
                            record_date=as_of, amount=usdc_balance,
                            category=SpendCategory.CRYPTO_HOLDINGS,
                            subcategory="USDC", provider=Provider.CRYPTO_WALLET,
                            source=f"{chain}_explorer",
                            raw_description=f"{usdc_balance} USDC on {chain}. {CRYPTO_DISCLAIMER}",
                        ))
            except httpx.HTTPError as exc:
                logger.warning("USDC fetch failed for %s/%s: %s", chain, address[:10], exc)

        return records

    async def _fetch_solana(
        self, address: str, as_of: date
    ) -> list[FinancialRecord]:
        """Fetch SOL balance via Solana JSON-RPC."""
        assert self._client is not None
        records: list[FinancialRecord] = []

        try:
            resp = await self._client.post(
                _SOLANA_RPC,
                json={"jsonrpc": "2.0", "id": 1, "method": "getBalance",
                      "params": [address]},
            )
            resp.raise_for_status()
            result = resp.json().get("result", {})
            lamports = Decimal(str(result.get("value", 0)))
            sol_balance = lamports / Decimal("1000000000")
            price = await self._get_price("solana")
            usd = (sol_balance * price).quantize(Decimal("0.01"))
            if usd > 0:
                records.append(FinancialRecord(
                    record_date=as_of, amount=usd,
                    category=SpendCategory.CRYPTO_HOLDINGS,
                    subcategory="SOL", provider=Provider.CRYPTO_WALLET,
                    source="solana_rpc",
                    raw_description=f"{sol_balance} SOL. {CRYPTO_DISCLAIMER}",
                ))
        except httpx.HTTPError as exc:
            logger.warning("Solana balance fetch failed: %s", exc)

        return records

    async def _solana_get_balance(self, address: str) -> Decimal:
        """Quick validation call — fetch SOL balance."""
        assert self._client is not None
        resp = await self._client.post(
            _SOLANA_RPC,
            json={"jsonrpc": "2.0", "id": 1, "method": "getBalance",
                  "params": [address]},
        )
        resp.raise_for_status()
        return Decimal(str(resp.json().get("result", {}).get("value", 0)))

    async def _get_price(self, coingecko_id: str) -> Decimal:
        """Fetch USD price from CoinGecko with 5-minute cache."""
        now = time.time()
        cached = _price_cache.get(coingecko_id)
        if cached and (now - cached[1]) < _PRICE_CACHE_TTL:
            return cached[0]

        assert self._client is not None
        try:
            resp = await self._client.get(
                _COINGECKO_PRICE_API,
                params={"ids": coingecko_id, "vs_currencies": "usd"},
            )
            resp.raise_for_status()
            price = Decimal(str(resp.json().get(coingecko_id, {}).get("usd", 0)))
            _price_cache[coingecko_id] = (price, now)
            return price
        except (httpx.HTTPError, KeyError, ValueError):
            logger.warning("Price fetch failed for %s — using fallback", coingecko_id)
            fallback = {"ethereum": Decimal("1900"), "solana": Decimal("150"),
                        "bitcoin": Decimal("64000"), "matic-network": Decimal("0.50")}
            return fallback.get(coingecko_id, Decimal("0"))


path: engine/connectors/datadog_costs.py
content: """Datadog connector — fetches usage metering data and calculates costs."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_DATADOG_API = "https://api.datadoghq.com"

# Published Datadog pricing (Pro plan, as of early 2026)
_PRICING: dict[str, dict[str, Any]] = {
    "infra_hosts": {
        "label": "Infrastructure Hosts",
        "rate": Decimal("15.00"),  # per host/month
        "unit": "hosts",
    },
    "logs_ingested_gb": {
        "label": "Log Management",
        "rate": Decimal("3.00"),  # per GB ingested/month (after 1st GB)
        "unit": "GB",
    },
    "apm_hosts": {
        "label": "APM Hosts",
        "rate": Decimal("8.00"),  # per host/month (Pro)
        "unit": "hosts",
    },
}


class DatadogCostsConnector(BaseConnector):
    """Connector for Datadog usage metering and cost data.

    Requires Pro or Enterprise plan for usage metering API access.
    """

    provider = Provider.DATADOG

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._app_key: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Datadog...")
                await asyncio.sleep(1)
                print("Authenticated with Datadog API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid Datadog credentials")
            return False

        # ── Normal mode ────────────────────────────────────────
        [REDACTED_SECRET]"api_key", "")
        app_key = credentials.get("app_key", "")

        if not api_key or not app_key:
            logger.error("Missing Datadog api_key or app_key")
            return False

        self._api_key = api_key
        self._app_key = app_key
        self._client = httpx.AsyncClient(
            base_url=_DATADOG_API,
            headers={
                "DD-API-KEY": self._api_key,
                "DD-APPLICATION-KEY": self._app_key,
            },
            timeout=30.0,
        )

        try:
            resp = await self._client.get("/api/v1/validate")
            resp.raise_for_status()
            data = resp.json()
            if not data.get("valid"):
                logger.error("Datadog API key validation returned invalid")
                return False
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("Datadog API credentials invalid or insufficient permissions")
            else:
                logger.error("Datadog validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Datadog connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Datadog connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []

        # ── Infrastructure hosts ────────────────────────────────
        records.extend(
            await self._fetch_product_usage(
                product_key="infra_hosts",
                endpoint="/api/v2/usage/hosts",
                extract_usage=lambda data: _extract_daily_max(data, "host_count"),
                start_date=start_date,
                end_date=end_date,
            )
        )

        # ── Logs ingested ───────────────────────────────────────
        records.extend(
            await self._fetch_product_usage(
                product_key="logs_ingested_gb",
                endpoint="/api/v2/usage/logs",
                extract_usage=lambda data: _extract_daily_sum(data, "ingested_bytes", scale=1 / 1_073_741_824),
                start_date=start_date,
                end_date=end_date,
            )
        )

        # ── APM hosts ──────────────────────────────────────────
        records.extend(
            await self._fetch_product_usage(
                product_key="apm_hosts",
                endpoint="/api/v2/usage/apm",
                extract_usage=lambda data: _extract_daily_max(data, "apm_host_count"),
                start_date=start_date,
                end_date=end_date,
            )
        )

        return records

    async def _fetch_product_usage(
        self,
        product_key: str,
        endpoint: str,
        extract_usage: Any,
        start_date: date,
        end_date: date,
    ) -> list[FinancialRecord]:
        """Fetch usage for a single Datadog product and calculate cost."""
        assert self._client is not None
        pricing = _PRICING[product_key]
        records: list[FinancialRecord] = []

        try:
            resp = await self._client.get(
                endpoint,
                params={
                    "start_hr": f"{start_date.isoformat()}T00",
                    "end_hr": f"{end_date.isoformat()}T23",
                },
            )
            resp.raise_for_status()
            data = resp.json()

            daily_usage = extract_usage(data)
            for usage_date, quantity in daily_usage.items():
                if quantity <= 0:
                    continue
                # Prorate monthly rate to daily
                daily_cost = (
                    pricing["rate"] * Decimal(str(quantity)) / Decimal("30")
                ).quantize(Decimal("0.01"))

                records.append(
                    FinancialRecord(
                        record_date=usage_date,
                        amount=-daily_cost,
                        category=SpendCategory.MONITORING,
                        subcategory=pricing["label"],
                        provider=Provider.DATADOG,
                        source="datadog_usage_api",
                        raw_description=(
                            f"Datadog {pricing['label']}: "
                            f"{quantity} {pricing['unit']}"
                        ),
                    )
                )

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("Datadog rate limit on %s — returning partial results", product_key)
            elif exc.response.status_code == 403:
                logger.warning("Datadog %s requires Pro/Enterprise plan", product_key)
            else:
                logger.error("Datadog %s fetch failed: HTTP %s", product_key, exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Datadog %s fetch timed out", product_key)
        except httpx.HTTPError as exc:
            logger.error("Datadog %s fetch error: %s", product_key, exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(r.amount for r in records)
        by_product: dict[str, Decimal] = {}
        for r in records:
            product = r.subcategory or "unknown"
            by_product[product] = by_product.get(product, Decimal("0")) + r.amount
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
            "spend_by_product": {k: str(v) for k, v in by_product.items()},
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Datadog cost data.

        Profile: ~$75/month total
          - Infrastructure: 3 hosts × $15 = $45/month
          - Logs: 5 GB × $3 = $15/month
          - APM: 2 hosts × $8 = $16/month (rounds to ~$75 with variance)
        Daily records prorated from monthly rates.
        """
        rng = random.Random(801)

        # Daily prorated costs
        products: dict[str, dict[str, Any]] = {
            "Infrastructure Hosts": {
                "base_quantity": 3.0,
                "rate": Decimal("15.00"),
                "unit": "hosts",
            },
            "Log Management": {
                "base_quantity": 5.0,
                "rate": Decimal("3.00"),
                "unit": "GB",
            },
            "APM Hosts": {
                "base_quantity": 2.0,
                "rate": Decimal("8.00"),
                "unit": "hosts",
            },
        }

        records: list[FinancialRecord] = []
        current = start_date
        while current <= end_date:
            for label, info in products.items():
                # Hosts are stable; logs vary more
                if info["unit"] == "GB":
                    quantity = info["base_quantity"] * rng.uniform(0.6, 1.4)
                else:
                    quantity = info["base_quantity"]

                daily_cost = (
                    info["rate"] * Decimal(str(round(quantity, 2))) / Decimal("30")
                ).quantize(Decimal("0.01"))

                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=-daily_cost,
                        category=SpendCategory.MONITORING,
                        subcategory=label,
                        provider=Provider.DATADOG,
                        source="synthetic",
                        raw_description=(
                            f"Synthetic Datadog {label}: "
                            f"{round(quantity, 1)} {info['unit']}"
                        ),
                    )
                )

            current += timedelta(days=1)

        return records


def _extract_daily_max(data: dict, count_key: str) -> dict[date, float]:
    """Extract daily max values from Datadog usage response."""
    result: dict[date, float] = {}
    for entry in data.get("data", []):
        attrs = entry.get("attributes", {})
        hour_str = attrs.get("hour", "")
        if not hour_str:
            continue
        entry_date = date.fromisoformat(hour_str[:10])
        value = float(attrs.get(count_key, 0))
        result[entry_date] = max(result.get(entry_date, 0), value)
    return result


def _extract_daily_sum(
    data: dict, sum_key: str, scale: float = 1.0
) -> dict[date, float]:
    """Extract daily summed values from Datadog usage response."""
    result: dict[date, float] = {}
    for entry in data.get("data", []):
        attrs = entry.get("attributes", {})
        hour_str = attrs.get("hour", "")
        if not hour_str:
            continue
        entry_date = date.fromisoformat(hour_str[:10])
        value = float(attrs.get(sum_key, 0)) * scale
        result[entry_date] = result.get(entry_date, 0) + value
    return result


path: engine/connectors/gcp_costs.py
content: """GCP Cloud Billing connector — fetches daily costs by service."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_GCP_BILLING_API = "https://cloudbilling.googleapis.com/v1"


class GCPCostsConnector(BaseConnector):
    """Connector for GCP Cloud Billing daily cost data."""

    provider = Provider.GCP

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._access_token: str = ""
        self._billing_account_id: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to GCP Cloud Billing...")
                await asyncio.sleep(1)
                print("Authenticated with GCP service account (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Invalid GCP service account JSON")
            return False

        # ── Normal mode ────────────────────────────────────────
        service_account_json = credentials.get("service_account_json", "")
        if not service_account_json:
            logger.error("Missing GCP service_account_json")
            return False

        # Exchange service account JSON for an access token
        try:
            import json
            sa = json.loads(service_account_json) if isinstance(service_account_json, str) else service_account_json
        except (json.JSONDecodeError, TypeError):
            logger.error("Invalid GCP service account JSON")
            return False

        self._client = httpx.AsyncClient(timeout=30.0)

        # Exchange service account credentials for an access token
        try:
            token_resp = await self._client.post(
                "https://accounts.google.com/o/token",
                data={
                    "grant_type": "urn:ietf:params:grant-type:jwt-bearer",
                    "assertion": _build_jwt_assertion(sa),
                },
            )
            token_resp.raise_for_status()
            self._access_token = token_resp.json()["access_token"]
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 401:
                logger.error("GCP service account credentials invalid")
            else:
                logger.error("GCP auth failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("GCP auth timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("GCP auth error: %s", exc)
            return False
        except (KeyError, ValueError) as exc:
            logger.error("GCP auth response parse error: %s", exc)
            return False

        # List billing accounts to validate access and get account ID
        try:
            resp = await self._client.get(
                f"{_GCP_BILLING_API}/billingAccounts",
                headers={"Authorization": f"Bearer {self._access_token}"},
            )
            resp.raise_for_status()
            accounts = resp.json().get("billingAccounts", [])
            if not accounts:
                logger.error("No GCP billing accounts found")
                return False
            self._billing_account_id = accounts[0]["name"]
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("GCP billing access denied")
            else:
                logger.error("GCP billing list failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("GCP billing list timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("GCP billing list error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []

        try:
            # Query BigQuery export of billing data via Cloud Billing API
            resp = await self._client.get(
                f"{_GCP_BILLING_API}/{self._billing_account_id}/costs",
                headers={"Authorization": f"Bearer {self._access_token}"},
                params={
                    "startDate": start_date.isoformat(),
                    "endDate": end_date.isoformat(),
                    "groupBy": "service",
                },
            )
            resp.raise_for_status()
            data = resp.json()

            for entry in data.get("costs", []):
                entry_date = date.fromisoformat(entry["date"])
                service = entry.get("service", "Unknown")
                cost = Decimal(str(entry.get("amount", 0))).quantize(Decimal("0.01"))
                if cost == 0:
                    continue

                records.append(
                    FinancialRecord(
                        record_date=entry_date,
                        amount=-cost,
                        category=SpendCategory.CLOUD_INFRASTRUCTURE,
                        subcategory=service,
                        provider=Provider.GCP,
                        source="gcp_cloud_billing",
                        raw_description=f"GCP {service}",
                    )
                )

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("GCP billing rate limit — returning partial results")
            else:
                logger.error("GCP billing fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("GCP billing fetch timed out — returning partial results")
        except httpx.HTTPError as exc:
            logger.error("GCP billing fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(r.amount for r in records)
        by_service: dict[str, Decimal] = {}
        for r in records:
            svc = r.subcategory or "unknown"
            by_service[svc] = by_service.get(svc, Decimal("0")) + r.amount
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
            "spend_by_service": {k: str(v) for k, v in by_service.items()},
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic GCP cost data.

        Profile: ~$150/month total
          - Compute Engine: ~$60/month
          - Cloud Run: ~$40/month
          - Cloud Storage: ~$20/month
          - BigQuery: ~$30/month
        """
        rng = random.Random(301)

        service_daily: dict[str, float] = {
            "Compute Engine": 60.0 / 30,
            "Cloud Run": 40.0 / 30,
            "Cloud Storage": 20.0 / 30,
            "BigQuery": 30.0 / 30,
        }

        records: list[FinancialRecord] = []
        current = start_date
        while current <= end_date:
            for service, daily_target in service_daily.items():
                # Weekday/weekend: storage is constant, compute drops on weekends
                is_weekend = current.weekday() >= 5
                if is_weekend and service in ("Compute Engine", "Cloud Run", "BigQuery"):
                    base = daily_target * 0.7
                else:
                    base = daily_target
                cost = base * rng.uniform(0.85, 1.15)
                cost_dec = Decimal(str(round(cost, 2)))

                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=-cost_dec,
                        category=SpendCategory.CLOUD_INFRASTRUCTURE,
                        subcategory=service,
                        provider=Provider.GCP,
                        source="synthetic",
                        raw_description=f"Synthetic GCP {service}",
                    )
                )

            current += timedelta(days=1)

        return records


def _build_jwt_assertion(sa: dict) -> str:
    """Build a JWT assertion for service account authentication.

    Stub — real implementation uses the google-auth library to create
    a properly authenticated JWT. The connect() method handles failures.
    """
    import base64
    import json
    header = base64.urlsafe_b64encode(json.dumps({"alg": "RS256", "typ": "JWT"}).encode()).decode()
    payload = base64.urlsafe_b64encode(json.dumps({
        "iss": sa.get("client_email", ""),
        "scope": "https://www.googleapis.com/auth/cloud-billing.readonly",
        "aud": "https://accounts.google.com/o/token",
    }).encode()).decode()
    # Stub assertion — will be rejected by Google, connect() handles the error
    return f"{header}.{payload}.stub"


path: engine/connectors/gemini_billing.py
content: """Gemini billing connector — fetches token usage and costs from Google AI (Gemini) API."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

# Gemini model pricing (per 1M tokens, early 2026)
_MODEL_PRICING: dict[str, dict[str, Decimal]] = {
    "gemini-pro": {"input": Decimal("1.25"), "output": Decimal("5.00")},
    "gemini-flash": {"input": Decimal("0.075"), "output": Decimal("0.30")},
}

_GOOGLE_AI_API = "https://generativelanguage.googleapis.com/v1beta"


class GeminiBillingConnector(BaseConnector):
    """Connector for Gemini (Google AI) billing and usage data."""

    provider = Provider.GEMINI

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Google AI...")
                await asyncio.sleep(1)
                print("Authenticated with Google AI API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Invalid Google AI API key")
            return False

        [REDACTED_SECRET]"api_key", "")
        if not [REDACTED_SECRET]"Missing Google AI API key")
            return False

        self._api_key = api_key
        self._client = httpx.AsyncClient(timeout=30.0)

        try:
            resp = await self._client.get(
                f"{_GOOGLE_AI_API}/models",
                params={"key": self._api_key},
            )
            resp.raise_for_status()
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("Google AI API key is invalid or insufficient permissions")
            else:
                logger.error("Google AI validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Google AI connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Google AI connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        # Google AI doesn't have a public usage/billing API yet.
        # In production, this would query the Google Cloud billing export
        # or the AI Platform usage API when available.
        logger.info("Google AI usage API not yet available — returning empty")
        return []

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(r.amount for r in records)
        by_model: dict[str, Decimal] = {}
        for r in records:
            by_model[r.model or "unknown"] = by_model.get(r.model or "unknown", Decimal("0")) + r.amount
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
            "spend_by_model": {k: str(v) for k, v in by_model.items()},
        }

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Google/Gemini billing data.

        Profile: ~$200/month total, growing ~5%/month
          - gemini-pro: ~$150/month (main model for complex tasks)
          - gemini-flash: ~$50/month (fast model for simple tasks)
        """
        rng = random.Random(2026)
        monthly_growth = 1.05

        model_daily: dict[str, float] = {
            "gemini-pro": 150.0 / 30,
            "gemini-flash": 50.0 / 30,
        }

        records: list[FinancialRecord] = []
        current = start_date
        while current <= end_date:
            is_weekend = current.weekday() >= 5
            weekend_factor = 0.6 if is_weekend else 1.0
            months_elapsed = (current - start_date).days / 30.0
            growth = Decimal(str(round(monthly_growth ** months_elapsed, 4)))

            for model_name, daily_target in model_daily.items():
                base = daily_target * weekend_factor
                cost = base * rng.uniform(0.8, 1.2)
                cost_dec = (Decimal(str(round(cost, 2))) * growth).quantize(Decimal("0.01"))

                pricing = _MODEL_PRICING[model_name]
                input_cost = cost_dec * Decimal("0.4")
                output_cost = cost_dec * Decimal("0.6")
                input_tokens = int(input_cost / pricing["input"] * 1_000_000) if pricing["input"] else 0
                output_tokens = int(output_cost / pricing["output"] * 1_000_000) if pricing["output"] else 0

                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=-cost_dec,
                        category=SpendCategory.AI_INFERENCE,
                        provider=Provider.GEMINI,
                        model=model_name,
                        tokens_input=input_tokens,
                        tokens_output=output_tokens,
                        source="synthetic",
                        raw_description=f"Synthetic Google {model_name} usage",
                    )
                )

            current += timedelta(days=1)

        return records


path: engine/connectors/google_ads.py
content: """Google Ads connector — fetches ad spend from Google Ads API."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_GOOGLE_ADS_API = "https://googleads.googleapis.com/v18"


class GoogleAdsConnector(BaseConnector):
    """Connector for Google Ads (Search, Display, YouTube) spend data."""

    provider = Provider.GOOGLE_ADS

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._developer_token: str = ""
        self._customer_id: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Google Ads...")
                await asyncio.sleep(1)
                print("Authenticated with Google Ads API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Invalid Google Ads credentials")
            return False

        developer_token = credentials.get("developer_token", "")
        customer_id = credentials.get("customer_id", "")
        if not developer_token or not customer_id:
            logger.error("Missing Google Ads developer_token or customer_id")
            return False

        self._developer_token = developer_token
        self._customer_id = customer_id.replace("-", "")
        self._client = httpx.AsyncClient(
            headers={
                "developer-token": self._developer_token,
                "login-customer-id": self._customer_id,
            },
            timeout=30.0,
        )

        try:
            resp = await self._client.get(
                f"{_GOOGLE_ADS_API}/customers/{self._customer_id}",
            )
            resp.raise_for_status()
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("Google Ads credentials invalid or insufficient permissions")
            else:
                logger.error("Google Ads validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Google Ads connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Google Ads connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []
        try:
            query = (
                "SELECT campaign.name, segments.date, "
                "metrics.cost_micros, metrics.impressions, metrics.clicks, "
                "campaign.advertising_channel_type "
                f"FROM campaign WHERE segments.date BETWEEN "
                f"'{start_date.isoformat()}' AND '{end_date.isoformat()}'"
            )
            resp = await self._client.post(
                f"{_GOOGLE_ADS_API}/customers/{self._customer_id}/googleAds:searchStream",
                json={"query": query},
            )
            resp.raise_for_status()

            for batch in resp.json():
                for row in batch.get("results", []):
                    campaign_name = row.get("campaign", {}).get("name", "Unknown")
                    cost_micros = int(row.get("metrics", {}).get("costMicros", 0))
                    cost_usd = Decimal(str(cost_micros)) / Decimal("1000000")
                    if cost_usd <= 0:
                        continue
                    row_date = date.fromisoformat(row.get("segments", {}).get("date", ""))
                    channel = row.get("campaign", {}).get("advertisingChannelType", "SEARCH")

                    subcategory = {
                        "SEARCH": "google_search",
                        "DISPLAY": "google_display",
                        "VIDEO": "google_youtube",
                    }.get(channel, "google_other")

                    records.append(FinancialRecord(
                        record_date=row_date,
                        amount=-cost_usd.quantize(Decimal("0.01")),
                        category=SpendCategory.ADVERTISING,
                        subcategory=subcategory,
                        provider=Provider.GOOGLE_ADS,
                        source="google_ads_api",
                        raw_description=f"Google Ads: {campaign_name}",
                    ))

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("Google Ads rate limit — returning partial results")
            else:
                logger.error("Google Ads fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Google Ads fetch timed out")
        except httpx.HTTPError as exc:
            logger.error("Google Ads fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(abs(r.amount) for r in records)
        by_channel: dict[str, Decimal] = {}
        for r in records:
            key = r.subcategory or "unknown"
            by_channel[key] = by_channel.get(key, Decimal("0")) + abs(r.amount)
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
            "spend_by_channel": {k: str(v) for k, v in by_channel.items()},
        }

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Google Ads spend data.

        Profile: ~$1,200/month total
          - Search campaigns: ~$700/month
          - Display campaigns: ~$300/month
          - YouTube campaigns: ~$200/month
        Daily variance, weekend dip, one spike week.
        """
        rng = random.Random(8888)

        campaigns = [
            ("Search — Brand Terms", "google_search", 300.0 / 30),
            ("Search — Competitor", "google_search", 250.0 / 30),
            ("Search — Long Tail", "google_search", 150.0 / 30),
            ("Display — Retargeting", "google_display", 200.0 / 30),
            ("Display — Prospecting", "google_display", 100.0 / 30),
            ("YouTube — Pre-roll", "google_youtube", 120.0 / 30),
            ("YouTube — Discovery", "google_youtube", 80.0 / 30),
        ]

        total_days = (end_date - start_date).days
        spike_start = start_date + timedelta(days=max(25, total_days // 2))
        spike_end = spike_start + timedelta(days=7)

        records: list[FinancialRecord] = []
        current = start_date
        while current <= end_date:
            is_weekend = current.weekday() >= 5
            is_spike = spike_start <= current < spike_end

            for campaign_name, channel, daily_target in campaigns:
                base = daily_target * (0.55 if is_weekend else 1.0)
                if is_spike:
                    base *= 1.8
                cost = base * rng.uniform(0.8, 1.2)
                cost_dec = Decimal(str(round(cost, 2)))

                impressions = int(cost * rng.uniform(600, 1100))
                clicks = int(impressions * rng.uniform(0.02, 0.06))

                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=-cost_dec,
                        category=SpendCategory.ADVERTISING,
                        subcategory=channel,
                        provider=Provider.GOOGLE_ADS,
                        source="synthetic",
                        raw_description=(
                            f"Synthetic Google Ads: {campaign_name} — "
                            f"{impressions:,} impressions, {clicks:,} clicks"
                        ),
                    )
                )

            current += timedelta(days=1)

        return records


path: engine/connectors/langsmith_costs.py
content: """LangSmith connector — fetches trace/run counts and calculates costs."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_LANGSMITH_API = "https://api.smith.langchain.com"

# Published LangSmith pricing (as of early 2026)
_BASE_TRACE_RATE = Decimal("2.50")  # per 1K base traces
_EXTENDED_TRACE_RATE = Decimal("5.00")  # per 1K extended traces
_PLUS_SEAT_COST = Decimal("39.00")  # per seat/month
_INCLUDED_TRACES = 5000  # included in Plus plan per month


class LangSmithCostsConnector(BaseConnector):
    """Connector for LangSmith trace usage and cost data."""

    provider = Provider.LANGSMITH

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to LangSmith...")
                await asyncio.sleep(1)
                print("Authenticated with LangSmith API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid LangSmith API key")
            return False

        # ── Normal mode ────────────────────────────────────────
        [REDACTED_SECRET]"api_key", "")
        if not [REDACTED_SECRET]"Missing LangSmith api_key")
            return False

        self._api_key = api_key
        self._client = httpx.AsyncClient(
            base_url=_LANGSMITH_API,
            headers={"x-api-key": self._api_key},
            timeout=30.0,
        )

        try:
            resp = await self._client.get("/api/v1/info")
            resp.raise_for_status()
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("LangSmith API key invalid or insufficient permissions")
            else:
                logger.error("LangSmith validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("LangSmith connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("LangSmith connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []

        try:
            resp = await self._client.get(
                "/api/v1/usage",
                params={
                    "start_date": start_date.isoformat(),
                    "end_date": end_date.isoformat(),
                },
            )
            resp.raise_for_status()
            data = resp.json()

            for period in data.get("usage", []):
                period_date = date.fromisoformat(period["date"])
                base_traces = period.get("base_traces", 0)
                extended_traces = period.get("extended_traces", 0)
                seats = period.get("seats", 1)

                # Seat cost (prorated daily)
                seat_daily = (_PLUS_SEAT_COST * Decimal(seats) / Decimal("30")).quantize(Decimal("0.01"))
                if seat_daily > 0:
                    records.append(
                        FinancialRecord(
                            record_date=period_date,
                            amount=-seat_daily,
                            category=SpendCategory.MONITORING,
                            subcategory="LangSmith Plus Seats",
                            provider=Provider.LANGSMITH,
                            source="langsmith_usage_api",
                            raw_description=f"LangSmith Plus: {seats} seat(s)",
                        )
                    )

                # Base trace overage
                included_daily = _INCLUDED_TRACES / 30
                overage_base = max(0, base_traces - included_daily)
                base_cost = (Decimal(str(overage_base)) / Decimal("1000") * _BASE_TRACE_RATE).quantize(Decimal("0.01"))
                if base_cost > 0:
                    records.append(
                        FinancialRecord(
                            record_date=period_date,
                            amount=-base_cost,
                            category=SpendCategory.MONITORING,
                            subcategory="LangSmith Base Traces",
                            provider=Provider.LANGSMITH,
                            source="langsmith_usage_api",
                            raw_description=f"LangSmith base traces: {base_traces:,}",
                        )
                    )

                # Extended traces (all billable)
                ext_cost = (Decimal(str(extended_traces)) / Decimal("1000") * _EXTENDED_TRACE_RATE).quantize(Decimal("0.01"))
                if ext_cost > 0:
                    records.append(
                        FinancialRecord(
                            record_date=period_date,
                            amount=-ext_cost,
                            category=SpendCategory.MONITORING,
                            subcategory="LangSmith Extended Traces",
                            provider=Provider.LANGSMITH,
                            source="langsmith_usage_api",
                            raw_description=f"LangSmith extended traces: {extended_traces:,}",
                        )
                    )

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("LangSmith rate limit — returning partial results")
            else:
                logger.error("LangSmith usage fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("LangSmith usage fetch timed out — returning partial results")
        except httpx.HTTPError as exc:
            logger.error("LangSmith usage fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(r.amount for r in records)
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic LangSmith cost data.

        Profile: Plus plan 1 seat ($39/mo), 15,000 traces/month.
          - 5,000 included → 10,000 overage base traces at $2.50/1K = $25.
          - ~$39 + $25 = ~$64/month but daily proration and variance yield
            roughly $39 seat + ~$12.50 overage = ~$51.50/month.
        """
        rng = random.Random(901)
        records: list[FinancialRecord] = []

        daily_seat = (_PLUS_SEAT_COST / Decimal("30")).quantize(Decimal("0.01"))
        daily_traces_target = 15000 / 30  # 500/day
        daily_included = _INCLUDED_TRACES / 30  # ~167/day

        current = start_date
        while current <= end_date:
            is_weekend = current.weekday() >= 5
            weekend_factor = 0.5 if is_weekend else 1.0

            # Seat cost (constant)
            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-daily_seat,
                    category=SpendCategory.MONITORING,
                    subcategory="LangSmith Plus Seats",
                    provider=Provider.LANGSMITH,
                    source="synthetic",
                    raw_description="Synthetic LangSmith Plus: 1 seat",
                )
            )

            # Base traces with overage
            daily_traces = int(daily_traces_target * weekend_factor * rng.uniform(0.7, 1.3))
            overage = max(0, daily_traces - daily_included)
            if overage > 0:
                trace_cost = (Decimal(str(overage)) / Decimal("1000") * _BASE_TRACE_RATE).quantize(Decimal("0.01"))
                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=-trace_cost,
                        category=SpendCategory.MONITORING,
                        subcategory="LangSmith Base Traces",
                        provider=Provider.LANGSMITH,
                        source="synthetic",
                        raw_description=f"Synthetic LangSmith traces: {daily_traces:,} ({overage:,} overage)",
                    )
                )

            current += timedelta(days=1)

        return records


path: engine/connectors/meta_ads.py
content: """Meta (Facebook/Instagram) Ads connector — fetches ad spend from Meta Marketing API."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_META_GRAPH_API = "https://graph.facebook.com/v21.0"


class MetaAdsConnector(BaseConnector):
    """Connector for Meta (Facebook/Instagram) advertising spend data."""

    provider = Provider.META

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._access_token: str = ""
        self._ad_account_id: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Meta Ads...")
                await asyncio.sleep(1)
                print("Authenticated with Meta Marketing API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Invalid Meta access token or ad account ID")
            return False

        access_token = credentials.get("access_token", "")
        ad_account_id = credentials.get("ad_account_id", "")
        if not access_token or not ad_account_id:
            logger.error("Missing Meta access_token or ad_account_id")
            return False

        self._access_token = access_token
        self._ad_account_id = ad_account_id
        self._client = httpx.AsyncClient(timeout=30.0)

        try:
            resp = await self._client.get(
                f"{_META_GRAPH_API}/{self._ad_account_id}",
                params={
                    "access_token": self._access_token,
                    "fields": "name,account_status",
                },
            )
            resp.raise_for_status()
            data = resp.json()
            if data.get("account_status") not in (1, 2):  # 1=active, 2=disabled
                logger.error("Meta ad account is not active (status: %s)", data.get("account_status"))
                return False
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("Meta access token invalid or expired")
            else:
                logger.error("Meta validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Meta connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Meta connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []
        try:
            resp = await self._client.get(
                f"{_META_GRAPH_API}/{self._ad_account_id}/insights",
                params={
                    "access_token": self._access_token,
                    "time_range": f'{{"since":"{start_date.isoformat()}","until":"{end_date.isoformat()}"}}',
                    "time_increment": 1,  # daily
                    "fields": "spend,campaign_name,impressions,clicks",
                    "level": "campaign",
                    "limit": 500,
                },
            )
            resp.raise_for_status()
            data = resp.json()

            for entry in data.get("data", []):
                spend = Decimal(entry.get("spend", "0"))
                if spend <= 0:
                    continue
                campaign = entry.get("campaign_name", "Unknown Campaign")
                entry_date = date.fromisoformat(entry.get("date_start", start_date.isoformat()))

                # Determine platform from campaign name heuristic
                subcategory = "facebook_ads"
                if any(kw in campaign.lower() for kw in ("instagram", "ig_", "reels")):
                    subcategory = "instagram_ads"

                records.append(
                    FinancialRecord(
                        record_date=entry_date,
                        amount=-spend.quantize(Decimal("0.01")),
                        category=SpendCategory.ADVERTISING,
                        subcategory=subcategory,
                        provider=Provider.META,
                        source="meta_marketing_api",
                        raw_description=f"Meta Ads: {campaign} — {entry.get('impressions', 0):,} impressions",
                    )
                )
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("Meta API rate limit — returning partial results")
            else:
                logger.error("Meta insights fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Meta insights fetch timed out")
        except httpx.HTTPError as exc:
            logger.error("Meta insights fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(abs(r.amount) for r in records)
        by_platform: dict[str, Decimal] = {}
        for r in records:
            key = r.subcategory or "unknown"
            by_platform[key] = by_platform.get(key, Decimal("0")) + abs(r.amount)
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
            "spend_by_platform": {k: str(v) for k, v in by_platform.items()},
        }

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Meta Ads spend data.

        Profile: ~$800/month total
          - Facebook Ads: ~$600/month across 3 campaigns
          - Instagram Ads: ~$200/month across 2 campaigns
        Daily variance, one 2x spike week (campaign push), weekend dip.
        """
        rng = random.Random(7777)

        campaigns = [
            ("FB — Lead Gen US", "facebook_ads", 250.0 / 30),
            ("FB — Retargeting", "facebook_ads", 200.0 / 30),
            ("FB — Brand Awareness", "facebook_ads", 150.0 / 30),
            ("IG — Stories Promo", "instagram_ads", 120.0 / 30),
            ("IG — Reels Engagement", "instagram_ads", 80.0 / 30),
        ]

        total_days = (end_date - start_date).days
        spike_start = start_date + timedelta(days=max(20, total_days // 3))
        spike_end = spike_start + timedelta(days=7)

        records: list[FinancialRecord] = []
        current = start_date
        while current <= end_date:
            is_weekend = current.weekday() >= 5
            is_spike = spike_start <= current < spike_end

            for campaign_name, platform, daily_target in campaigns:
                base = daily_target * (0.65 if is_weekend else 1.0)
                if is_spike:
                    base *= 2.0
                cost = base * rng.uniform(0.75, 1.25)
                cost_dec = Decimal(str(round(cost, 2)))

                impressions = int(cost * rng.uniform(800, 1200))
                clicks = int(impressions * rng.uniform(0.01, 0.04))

                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=-cost_dec,
                        category=SpendCategory.ADVERTISING,
                        subcategory=platform,
                        provider=Provider.META,
                        source="synthetic",
                        raw_description=(
                            f"Synthetic Meta Ads: {campaign_name} — "
                            f"{impressions:,} impressions, {clicks:,} clicks"
                        ),
                    )
                )

            current += timedelta(days=1)

        return records


path: engine/connectors/openai_billing.py
content: """OpenAI billing connector — fetches token usage and costs from OpenAI API."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

# Model pricing (per 1M tokens, approximate as of early 2026)
_MODEL_PRICING: dict[str, dict[str, Decimal]] = {
    "gpt-4o": {"input": Decimal("2.50"), "output": Decimal("10.00")},
    "gpt-4o-mini": {"input": Decimal("0.15"), "output": Decimal("0.60")},
    "text-embedding-3-small": {"input": Decimal("0.02"), "output": Decimal("0")},
}

_OPENAI_API_BASE = "https://api.openai.com/v1"


class OpenAIBillingConnector(BaseConnector):
    """Connector for OpenAI billing / usage data."""

    provider = Provider.OPENAI

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to OpenAI...")
                await asyncio.sleep(1)
                print("Authenticated with OpenAI API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Invalid API key format")
            return False

        # ── Normal mode ────────────────────────────────────────
        [REDACTED_SECRET]"api_key", "")
        if not api_key or not api_key.startswith("sk-"):
            logger.error("Invalid OpenAI API key format")
            return False

        self._api_key = api_key
        self._client = httpx.AsyncClient(
            base_url=_OPENAI_API_BASE,
            headers={"Authorization": f"Bearer {self._api_key}"},
            timeout=30.0,
        )

        try:
            resp = await self._client.get("/models", params={"limit": 1})
            resp.raise_for_status()
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 401:
                logger.error("OpenAI API key is invalid or expired")
            else:
                logger.error("OpenAI validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("OpenAI connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("OpenAI connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []

        try:
            # OpenAI usage API returns daily buckets
            resp = await self._client.get(
                "/organization/usage",
                params={
                    "start_date": start_date.isoformat(),
                    "end_date": end_date.isoformat(),
                },
            )
            resp.raise_for_status()
            data = resp.json()

            for bucket in data.get("data", []):
                bucket_date = date.fromisoformat(bucket["date"])
                for result in bucket.get("results", []):
                    model = result.get("model", "unknown")
                    input_tokens = result.get("input_tokens", 0)
                    output_tokens = result.get("output_tokens", 0)
                    # Use reported cost if available, otherwise estimate
                    cost_usd = Decimal(str(result.get("cost_in_usd", 0)))
                    if cost_usd == 0 and model in _MODEL_PRICING:
                        pricing = _MODEL_PRICING[model]
                        cost_usd = (
                            pricing["input"] * Decimal(input_tokens) / Decimal("1000000")
                            + pricing["output"] * Decimal(output_tokens) / Decimal("1000000")
                        )

                    records.append(
                        FinancialRecord(
                            record_date=bucket_date,
                            amount=-cost_usd.quantize(Decimal("0.01")),
                            category=SpendCategory.AI_INFERENCE,
                            provider=Provider.OPENAI,
                            model=model,
                            tokens_input=input_tokens,
                            tokens_output=output_tokens,
                            source="openai_usage_api",
                            raw_description=f"OpenAI {model} usage",
                        )
                    )

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("OpenAI rate limit hit — returning partial results")
            else:
                logger.error("OpenAI usage fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("OpenAI usage fetch timed out — returning partial results")
        except httpx.HTTPError as exc:
            logger.error("OpenAI usage fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(r.amount for r in records)
        by_model: dict[str, Decimal] = {}
        for r in records:
            by_model[r.model or "unknown"] = by_model.get(r.model or "unknown", Decimal("0")) + r.amount
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
            "spend_by_model": {k: str(v) for k, v in by_model.items()},
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate 90-day realistic OpenAI billing data.

        Persona: Solo dev defaulting to GPT-4o for everything, running an
        AI customer support SaaS with 45 customers.

        Profile: ~$400/month total, growing ~8%/month
          - gpt-4o: ~$250/month — TWO distinct usage patterns:

            1. "Pipeline" requests (15/day): Automated cron jobs — nightly
               reports, scheduled classification runs, RAG-powered analysis.
               Each call has ~8000 input tokens (system prompt + RAG context),
               highly consistent (CV < 0.03). Triggers CACHING on gpt-4o.

            2. "Ad-hoc" requests (300/day weekdays, 180 weekends): Developer
               routing ALL simple tasks to gpt-4o — classification,
               extraction, formatting, validation. 500-2000 input tokens,
               200-800 output. ~8,000/month. Triggers MODEL ROUTING
               since these don't need frontier intelligence.

          - gpt-4o-mini: ~$50/month (15/day per-request, varied tokens)
          - text-embedding-3-small: ~$100/month (daily aggregate)

        Consistent daily request count → triggers batch detector.
        Growth trend → triggers spend forecast.
        """
        rng = random.Random(42)
        monthly_growth = 1.08

        records: list[FinancialRecord] = []
        current = start_date
        while current <= end_date:
            is_weekend = current.weekday() >= 5
            weekend_factor = 0.6 if is_weekend else 1.0
            months_elapsed = (current - start_date).days / 30.0
            growth = Decimal(str(round(monthly_growth ** months_elapsed, 4)))

            # ── gpt-4o "pipeline": 15 requests/day, ~8000 input ───
            # Automated cron: nightly reports, hourly classification,
            # scheduled RAG analysis. System prompt + context is ~8000
            # tokens every time. LOW variance → triggers caching.
            pipeline_budget = 100.0 / 30 * weekend_factor
            for _ in range(15):
                req_cost = pipeline_budget / 15 * rng.uniform(0.9, 1.1)
                cost_dec = (Decimal(str(round(req_cost, 4))) * growth).quantize(Decimal("0.01"))
                input_tokens = int(8000 * rng.uniform(0.97, 1.03))
                output_tokens = rng.randint(1500, 2500)
                records.append(FinancialRecord(
                    record_date=current, amount=-cost_dec,
                    category=SpendCategory.AI_INFERENCE,
                    provider=Provider.OPENAI, model="gpt-4o",
                    tokens_input=input_tokens, tokens_output=output_tokens,
                    source="synthetic",
                    raw_description="Synthetic OpenAI gpt-4o pipeline request",
                ))

            # ── gpt-4o "ad-hoc": 300/day weekdays, 180 weekends ───
            # Developer using gpt-4o for tasks that don't need frontier:
            # classification, extraction, formatting, validation.
            # ~8000 requests/month. Tokens 500-2000 input, 200-800 output.
            # Triggers MODEL ROUTING — material savings by routing to mini.
            n_adhoc = int(300 * weekend_factor)
            adhoc_budget = 150.0 / 30 * weekend_factor
            for _ in range(n_adhoc):
                req_cost = adhoc_budget / max(n_adhoc, 1) * rng.uniform(0.6, 1.4)
                cost_dec = (Decimal(str(round(req_cost, 4))) * growth).quantize(Decimal("0.01"))
                input_tokens = rng.randint(500, 2000)
                output_tokens = rng.randint(200, 800)
                records.append(FinancialRecord(
                    record_date=current, amount=-cost_dec,
                    category=SpendCategory.AI_INFERENCE,
                    provider=Provider.OPENAI, model="gpt-4o",
                    tokens_input=input_tokens, tokens_output=output_tokens,
                    source="synthetic",
                    raw_description="Synthetic OpenAI gpt-4o ad-hoc request",
                ))

            # ── gpt-4o-mini: 15 per-request records/day ───────────
            # The developer DOES use mini for some things. Mixed token
            # sizes with high variance — won't trigger caching.
            mini_daily_budget = 50.0 / 30 * weekend_factor
            for _ in range(15):
                req_cost = mini_daily_budget / 15 * rng.uniform(0.7, 1.3)
                cost_dec = (Decimal(str(round(req_cost, 4))) * growth).quantize(Decimal("0.01"))
                input_tokens = rng.randint(200, 5000)
                output_tokens = rng.randint(50, 1000)
                records.append(FinancialRecord(
                    record_date=current, amount=-cost_dec,
                    category=SpendCategory.AI_INFERENCE,
                    provider=Provider.OPENAI, model="gpt-4o-mini",
                    tokens_input=input_tokens, tokens_output=output_tokens,
                    source="synthetic",
                    raw_description="Synthetic OpenAI gpt-4o-mini request",
                ))

            # ── text-embedding-3-small: daily aggregate ────────────
            embed_cost = 100.0 / 30 * weekend_factor * rng.uniform(0.8, 1.2)
            embed_dec = (Decimal(str(round(embed_cost, 2))) * growth).quantize(Decimal("0.01"))
            embed_pricing = _MODEL_PRICING["text-embedding-3-small"]
            embed_input = int(embed_dec / embed_pricing["input"] * 1_000_000) if embed_pricing["input"] else 0
            records.append(FinancialRecord(
                record_date=current, amount=-embed_dec,
                category=SpendCategory.AI_INFERENCE,
                provider=Provider.OPENAI, model="text-embedding-3-small",
                tokens_input=embed_input, tokens_output=0,
                source="synthetic",
                raw_description="Synthetic OpenAI text-embedding-3-small usage",
            ))

            current += timedelta(days=1)

        return records


path: engine/connectors/pinecone_costs.py
content: """Pinecone connector — fetches index stats and calculates serverless costs."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_PINECONE_API = "https://api.pinecone.io"

# Published Pinecone Serverless pricing (as of early 2026)
_QUERY_RATE = Decimal("0.002")  # per 1K read units (queries)
_STORAGE_RATE = Decimal("0.33")  # per GB-month
_WRITE_RATE = Decimal("0.002")  # per 1K write units


class PineconeCostsConnector(BaseConnector):
    """Connector for Pinecone serverless index usage and cost data."""

    provider = Provider.PINECONE

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._client: httpx.AsyncClient | None = None
        self._indexes: list[dict] = []

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Pinecone...")
                await asyncio.sleep(1)
                print("Authenticated with Pinecone API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid Pinecone API key")
            return False

        # ── Normal mode ────────────────────────────────────────
        [REDACTED_SECRET]"api_key", "")
        if not [REDACTED_SECRET]"Missing Pinecone api_key")
            return False

        self._api_key = api_key
        self._client = httpx.AsyncClient(
            base_url=_PINECONE_API,
            headers={"Api-Key": self._api_key},
            timeout=30.0,
        )

        try:
            resp = await self._client.get("/indexes")
            resp.raise_for_status()
            self._indexes = resp.json().get("indexes", [])
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("Pinecone API key invalid or insufficient permissions")
            else:
                logger.error("Pinecone validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Pinecone connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Pinecone connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []

        for index_info in self._indexes:
            index_name = index_info.get("name", "unknown")
            host = index_info.get("host", "")
            if not host:
                continue

            try:
                # Fetch index stats
                index_client = httpx.AsyncClient(
                    base_url=f"https://{host}",
                    headers={"Api-Key": self._api_key},
                    timeout=30.0,
                )
                resp = await index_client.get("/describe_index_stats")
                resp.raise_for_status()
                stats = resp.json()
                await index_client.aclose()

                vector_count = stats.get("totalVectorCount", 0)
                dimension = stats.get("dimension", 0)
                # Estimate storage: vectors × dimensions × 4 bytes (float32) + overhead
                storage_bytes = vector_count * dimension * 4 * 1.2
                storage_gb = Decimal(str(storage_bytes)) / Decimal("1073741824")

                # Storage cost (point-in-time, dated at end_date)
                storage_cost = (storage_gb * _STORAGE_RATE).quantize(Decimal("0.01"))
                if storage_cost > 0:
                    records.append(
                        FinancialRecord(
                            record_date=end_date,
                            amount=-storage_cost,
                            category=SpendCategory.SEARCH_DATA,
                            subcategory=f"Pinecone Storage ({index_name})",
                            provider=Provider.PINECONE,
                            source="pinecone_api",
                            raw_description=(
                                f"Pinecone {index_name}: {vector_count:,} vectors, "
                                f"{storage_gb:.2f} GB"
                            ),
                        )
                    )

            except httpx.HTTPStatusError as exc:
                if exc.response.status_code == 429:
                    logger.warning("Pinecone rate limit on index %s", index_name)
                else:
                    logger.error("Pinecone stats fetch failed for %s: HTTP %s", index_name, exc.response.status_code)
            except httpx.TimeoutException:
                logger.error("Pinecone stats fetch timed out for %s", index_name)
            except httpx.HTTPError as exc:
                logger.error("Pinecone stats fetch error for %s: %s", index_name, exc)

        # Note: query/write unit usage requires Pinecone's usage API or billing export.
        # Cost estimation for read/write units would require tracking at the application layer.

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        records = await self.fetch_records(today, today)
        total = sum(r.amount for r in records)
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "indexes": len(self._indexes),
            "current_spend": str(total),
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Pinecone cost data.

        Profile: 1 serverless index, 50K vectors (1536-dim), 100K queries/month.
          - Storage: 50K × 1536 × 4 bytes × 1.2 ≈ 0.34 GB → ~$0.11/month
          - Queries: 100K/month at $0.002/1K = ~$0.20/month
          - Realistic minimum: ~$8/month (Pinecone has minimum billing thresholds)
        Monthly billing records.
        """
        rng = random.Random(902)
        records: list[FinancialRecord] = []

        current = start_date.replace(day=1)
        while current <= end_date:
            # Storage cost
            vectors = int(50_000 * rng.uniform(0.9, 1.1))
            storage_gb = Decimal(str(round(vectors * 1536 * 4 * 1.2 / 1_073_741_824, 4)))
            storage_cost = max(
                (storage_gb * _STORAGE_RATE).quantize(Decimal("0.01")),
                Decimal("0.11"),
            )

            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-storage_cost,
                    category=SpendCategory.SEARCH_DATA,
                    subcategory="Pinecone Storage",
                    provider=Provider.PINECONE,
                    source="synthetic",
                    raw_description=f"Synthetic Pinecone: {vectors:,} vectors, {storage_gb:.3f} GB",
                )
            )

            # Query cost
            monthly_queries = int(100_000 * rng.uniform(0.7, 1.3))
            query_cost = (Decimal(str(monthly_queries)) / Decimal("1000") * _QUERY_RATE).quantize(Decimal("0.01"))

            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-query_cost,
                    category=SpendCategory.SEARCH_DATA,
                    subcategory="Pinecone Queries",
                    provider=Provider.PINECONE,
                    source="synthetic",
                    raw_description=f"Synthetic Pinecone: {monthly_queries:,} queries",
                )
            )

            # Write units (upserts)
            monthly_writes = int(5_000 * rng.uniform(0.5, 1.5))
            write_cost = (Decimal(str(monthly_writes)) / Decimal("1000") * _WRITE_RATE).quantize(Decimal("0.01"))
            if write_cost > 0:
                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=-write_cost,
                        category=SpendCategory.SEARCH_DATA,
                        subcategory="Pinecone Writes",
                        provider=Provider.PINECONE,
                        source="synthetic",
                        raw_description=f"Synthetic Pinecone: {monthly_writes:,} write units",
                    )
                )

            # Minimum billing threshold makes realistic total ~$8/month
            # Add platform fee to reach realistic minimum
            subtotal = storage_cost + query_cost + write_cost
            if subtotal < Decimal("8.00"):
                platform_fee = (Decimal("8.00") - subtotal).quantize(Decimal("0.01"))
                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=-platform_fee,
                        category=SpendCategory.SEARCH_DATA,
                        subcategory="Pinecone Platform",
                        provider=Provider.PINECONE,
                        source="synthetic",
                        raw_description="Synthetic Pinecone platform minimum",
                    )
                )

            current = (current + timedelta(days=32)).replace(day=1)

        return records


path: engine/connectors/sendgrid_costs.py
content: """SendGrid connector — fetches email volumes and calculates plan-based costs."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_SENDGRID_API = "https://api.sendgrid.com/v3"

# Published SendGrid plan pricing (as of early 2026)
_PLAN_PRICING: dict[str, dict[str, Any]] = {
    "free": {"monthly_cost": Decimal("0"), "emails_included": 100},
    "essentials": {"monthly_cost": Decimal("19.95"), "emails_included": 50_000},
    "pro": {"monthly_cost": Decimal("89.95"), "emails_included": 100_000},
    "premier": {"monthly_cost": Decimal("249.95"), "emails_included": 500_000},
}


class SendGridCostsConnector(BaseConnector):
    """Connector for SendGrid email cost data.

    SendGrid bills by plan tier, not per-email. This connector detects
    the plan from API responses and calculates the monthly cost.
    """

    provider = Provider.SENDGRID

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._plan: str = "free"
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to SendGrid...")
                await asyncio.sleep(1)
                print("Authenticated with SendGrid API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid SendGrid API key")
            return False

        # ── Normal mode ────────────────────────────────────────
        [REDACTED_SECRET]"api_key", "")
        if not [REDACTED_SECRET]"Missing SendGrid api_key")
            return False

        self._api_key = api_key
        self._client = httpx.AsyncClient(
            base_url=_SENDGRID_API,
            headers={"Authorization": f"Bearer {self._api_key}"},
            timeout=30.0,
        )

        try:
            resp = await self._client.get("/user/account")
            resp.raise_for_status()
            account = resp.json()
            self._plan = account.get("type", "free").lower()
            if self._plan not in _PLAN_PRICING:
                self._plan = "free"
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("SendGrid API key invalid or insufficient permissions")
            else:
                logger.error("SendGrid validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("SendGrid connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("SendGrid connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []
        plan_info = _PLAN_PRICING.get(self._plan, _PLAN_PRICING["free"])
        monthly_cost: Decimal = plan_info["monthly_cost"]

        # Fetch daily email stats for context
        try:
            resp = await self._client.get(
                "/stats",
                params={
                    "start_date": start_date.isoformat(),
                    "end_date": end_date.isoformat(),
                    "aggregated_by": "month",
                },
            )
            resp.raise_for_status()
            stats = resp.json()

            for month_stat in stats:
                stat_date = date.fromisoformat(month_stat["date"])
                metrics = month_stat.get("stats", [{}])[0].get("metrics", {})
                emails_sent = metrics.get("requests", 0)
                delivered = metrics.get("delivered", 0)
                opens = metrics.get("opens", 0)

                records.append(
                    FinancialRecord(
                        record_date=stat_date,
                        amount=-monthly_cost,
                        category=SpendCategory.COMMUNICATION,
                        subcategory=f"sendgrid_{self._plan}",
                        provider=Provider.SENDGRID,
                        source="sendgrid_stats_api",
                        raw_description=(
                            f"SendGrid {self._plan} plan: "
                            f"{emails_sent:,} sent, {delivered:,} delivered, "
                            f"{opens:,} opens"
                        ),
                    )
                )

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("SendGrid rate limit — returning partial results")
            else:
                logger.error("SendGrid stats fetch failed: HTTP %s", exc.response.status_code)

            # Fallback: generate monthly cost records without stats
            records.extend(
                self._generate_monthly_cost_records(start_date, end_date, monthly_cost)
            )
        except httpx.TimeoutException:
            logger.error("SendGrid stats fetch timed out — using plan cost only")
            records.extend(
                self._generate_monthly_cost_records(start_date, end_date, monthly_cost)
            )
        except httpx.HTTPError as exc:
            logger.error("SendGrid stats fetch error: %s", exc)
            records.extend(
                self._generate_monthly_cost_records(start_date, end_date, monthly_cost)
            )

        return records

    async def get_summary(self) -> dict:
        plan_info = _PLAN_PRICING.get(self._plan, _PLAN_PRICING["free"])
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "plan": self._plan,
            "monthly_cost": str(plan_info["monthly_cost"]),
            "emails_included": plan_info["emails_included"],
        }

    def _generate_monthly_cost_records(
        self, start_date: date, end_date: date, monthly_cost: Decimal
    ) -> list[FinancialRecord]:
        """Generate one cost record per month when stats are unavailable."""
        records: list[FinancialRecord] = []
        current = start_date.replace(day=1)
        while current <= end_date:
            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-monthly_cost,
                    category=SpendCategory.COMMUNICATION,
                    subcategory=f"sendgrid_{self._plan}",
                    provider=Provider.SENDGRID,
                    source="sendgrid_plan_cost",
                    raw_description=f"SendGrid {self._plan} plan monthly cost",
                )
            )
            current = (current + timedelta(days=32)).replace(day=1)
        return records

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic SendGrid cost data.

        Profile: Pro plan at $89.95/month, sending ~50,000 emails/month.
        Monthly billing records with email volume stats.
        """
        rng = random.Random(701)
        records: list[FinancialRecord] = []
        monthly_cost = _PLAN_PRICING["pro"]["monthly_cost"]

        current = start_date.replace(day=1)
        while current <= end_date:
            emails_sent = int(50_000 * rng.uniform(0.8, 1.2))
            delivered = int(emails_sent * rng.uniform(0.96, 0.99))
            opens = int(delivered * rng.uniform(0.18, 0.28))

            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-monthly_cost,
                    category=SpendCategory.COMMUNICATION,
                    subcategory="sendgrid_pro",
                    provider=Provider.SENDGRID,
                    source="synthetic",
                    raw_description=(
                        f"Synthetic SendGrid Pro: "
                        f"{emails_sent:,} sent, {delivered:,} delivered, "
                        f"{opens:,} opens"
                    ),
                )
            )
            current = (current + timedelta(days=32)).replace(day=1)

        return records


path: engine/connectors/stripe_connector.py
content: """Stripe connector — fetches charges, subscriptions, payouts, and fees."""
import logging
import random
from datetime import date, datetime, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_STRIPE_API_BASE = "https://api.stripe.com/v1"


class StripeConnector(BaseConnector):
    """Connector for Stripe billing data (revenue, fees, payouts)."""

    provider = Provider.STRIPE

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Stripe...")
                await asyncio.sleep(1)
                print("Authenticated with Stripe API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Invalid API key format")
            return False

        # ── Normal mode ────────────────────────────────────────
        [REDACTED_SECRET]"api_key", "")
        if not api_key or not api_key.startswith("sk_"):
            logger.error("Invalid Stripe API key format")
            return False

        self._api_key = api_key
        self._client = httpx.AsyncClient(
            base_url=_STRIPE_API_BASE,
            auth=(self._api_key, ""),
            timeout=30.0,
        )

        try:
            resp = await self._client.get("/balance")
            resp.raise_for_status()
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 401:
                logger.error("Stripe API key is invalid or expired")
            else:
                logger.error("Stripe validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Stripe connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Stripe connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []
        start_ts = int(datetime.combine(start_date, datetime.min.time()).timestamp())
        end_ts = int(datetime.combine(end_date, datetime.max.time()).timestamp())

        # ── Charges (revenue) ───────────────────────────────────
        try:
            has_more = True
            starting_after: str | None = None
            while has_more:
                params: dict[str, Any] = {
                    "created[gte]": start_ts,
                    "created[lte]": end_ts,
                    "limit": 100,
                }
                if starting_after:
                    params["starting_after"] = starting_after

                resp = await self._client.get("/charges", params=params)
                resp.raise_for_status()
                data = resp.json()

                for charge in data.get("data", []):
                    if charge.get("status") != "succeeded":
                        continue
                    charge_date = date.fromtimestamp(charge["created"])
                    amount_usd = Decimal(str(charge["amount"])) / Decimal("100")
                    fee_usd = Decimal("0")
                    if charge.get("balance_transaction") and isinstance(charge["balance_transaction"], dict):
                        fee_usd = Decimal(str(charge["balance_transaction"].get("fee", 0))) / Decimal("100")

                    # Revenue record
                    records.append(
                        FinancialRecord(
                            record_date=charge_date,
                            amount=amount_usd.quantize(Decimal("0.01")),
                            category=SpendCategory.REVENUE,
                            provider=Provider.STRIPE,
                            source="stripe_charges",
                            raw_description=charge.get("description", "Stripe charge"),
                        )
                    )
                    # Fee record
                    if fee_usd > 0:
                        records.append(
                            FinancialRecord(
                                record_date=charge_date,
                                amount=-fee_usd.quantize(Decimal("0.01")),
                                category=SpendCategory.PAYMENT_PROCESSING,
                                provider=Provider.STRIPE,
                                source="stripe_charges",
                                raw_description="Stripe processing fee",
                            )
                        )

                has_more = data.get("has_more", False)
                if has_more and data["data"]:
                    starting_after = data["data"][-1]["id"]

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("Stripe rate limit hit on charges — returning partial results")
            else:
                logger.error("Stripe charges fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Stripe charges fetch timed out — returning partial results")
        except httpx.HTTPError as exc:
            logger.error("Stripe charges fetch error: %s", exc)

        # ── Payouts ─────────────────────────────────────────────
        try:
            resp = await self._client.get(
                "/payouts",
                params={
                    "created[gte]": start_ts,
                    "created[lte]": end_ts,
                    "limit": 100,
                },
            )
            resp.raise_for_status()
            data = resp.json()

            for payout in data.get("data", []):
                if payout.get("status") != "paid":
                    continue
                payout_date = date.fromtimestamp(payout["created"])
                amount_usd = Decimal(str(payout["amount"])) / Decimal("100")
                records.append(
                    FinancialRecord(
                        record_date=payout_date,
                        amount=-amount_usd.quantize(Decimal("0.01")),
                        category=SpendCategory.OTHER,
                        subcategory="payout",
                        provider=Provider.STRIPE,
                        source="stripe_payouts",
                        raw_description="Stripe payout to bank",
                    )
                )

        except httpx.HTTPStatusError as exc:
            logger.error("Stripe payouts fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Stripe payouts fetch timed out")
        except httpx.HTTPError as exc:
            logger.error("Stripe payouts fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        revenue = sum(r.amount for r in records if r.amount > 0)
        fees = sum(r.amount for r in records if r.category == SpendCategory.PAYMENT_PROCESSING)
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_revenue": str(revenue),
            "current_month_fees": str(fees),
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Stripe data.

        Profile (month 0 = start_date's month):
          - ~$1,800/month subscription revenue, growing 5% MoM
          - ~$120/month in Stripe fees (≈2.9% + $0.30 per charge)
          - Daily charges with realistic variance
        """
        rng = random.Random(77)
        records: list[FinancialRecord] = []

        # Base monthly values at start_date
        base_monthly_revenue = 1800.0
        fee_rate = Decimal("0.029")
        per_charge_fee = Decimal("0.30")
        avg_charges_per_day = 3  # ~90 charges/month

        current = start_date
        ref_month = start_date.month + (start_date.year * 12)

        while current <= end_date:
            # Compute month offset for 5% MoM growth
            cur_month = current.month + (current.year * 12)
            months_elapsed = cur_month - ref_month
            growth_factor = 1.05 ** months_elapsed
            daily_revenue_target = (base_monthly_revenue * growth_factor) / 30

            is_weekend = current.weekday() >= 5
            day_multiplier = 0.7 if is_weekend else 1.0

            # Generate a few charges per day
            n_charges = rng.randint(
                max(1, avg_charges_per_day - 1),
                avg_charges_per_day + 2,
            )
            day_total_revenue = daily_revenue_target * day_multiplier * rng.uniform(0.8, 1.2)
            charge_amounts = _split_amount(day_total_revenue, n_charges, rng)

            for charge_amount in charge_amounts:
                amount_dec = Decimal(str(round(charge_amount, 2)))
                if amount_dec <= 0:
                    continue
                # Revenue
                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=amount_dec,
                        category=SpendCategory.REVENUE,
                        subcategory="subscription",
                        provider=Provider.STRIPE,
                        source="synthetic",
                        raw_description="Synthetic Stripe subscription charge",
                    )
                )
                # Fee: 2.9% + $0.30
                fee = (amount_dec * fee_rate + per_charge_fee).quantize(Decimal("0.01"))
                records.append(
                    FinancialRecord(
                        record_date=current,
                        amount=-fee,
                        category=SpendCategory.PAYMENT_PROCESSING,
                        provider=Provider.STRIPE,
                        source="synthetic",
                        raw_description="Synthetic Stripe processing fee",
                    )
                )

            current += timedelta(days=1)

        return records


def _split_amount(total: float, n: int, rng: random.Random) -> list[float]:
    """Split a total amount into n roughly-equal random pieces."""
    if n <= 0:
        return []
    weights = [rng.random() for _ in range(n)]
    weight_sum = sum(weights)
    return [total * w / weight_sum for w in weights]


path: engine/connectors/tavily_costs.py
content: """Tavily connector — tracks search API credit usage and costs."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_TAVILY_API = "https://api.tavily.com"

# Published Tavily pricing (as of early 2026)
_CREDIT_COST = Decimal("0.008")  # per search credit
_PLAN_PRICING: dict[str, dict[str, Any]] = {
    "researcher": {"monthly_cost": Decimal("0"), "credits_included": 1000},
    "business": {"monthly_cost": Decimal("0"), "credits_included": 5000},
}


class TavilyCostsConnector(BaseConnector):
    """Connector for Tavily search API usage and cost data.

    Tavily doesn't expose a full usage API, so this connector checks
    remaining credits via the API and tracks usage over time.
    """

    provider = Provider.TAVILY

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_key: str = ""
        self._client: httpx.AsyncClient | None = None
        self._total_credits: int = 0
        self._remaining_credits: int = 0

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Tavily...")
                await asyncio.sleep(1)
                print("Authenticated with Tavily API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid Tavily API key")
            return False

        # ── Normal mode ────────────────────────────────────────
        [REDACTED_SECRET]"api_key", "")
        if not [REDACTED_SECRET]"Missing Tavily api_key")
            return False

        self._api_key = api_key
        self._client = httpx.AsyncClient(timeout=30.0)

        # Validate by checking credits endpoint or making a lightweight call
        try:
            resp = await self._client.post(
                f"{_TAVILY_API}/search",
                json={
                    "api_key": self._api_key,
                    "query": "test",
                    "max_results": 1,
                    "search_depth": "basic",
                },
            )
            resp.raise_for_status()

            # Check response headers for credit info if available
            self._remaining_credits = int(
                resp.headers.get("x-credits-remaining", 0)
            )
            self._total_credits = int(
                resp.headers.get("x-credits-total", 0)
            )

            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code in (401, 403):
                logger.error("Tavily API key invalid")
            elif exc.response.status_code == 429:
                logger.error("Tavily rate limit during validation — try again later")
            else:
                logger.error("Tavily validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Tavily connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Tavily connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []

        # Tavily doesn't have a historical usage API. We can only report
        # current credit state. For tracking over time, the financial model
        # should snapshot credits periodically.
        if self._total_credits > 0:
            credits_used = self._total_credits - self._remaining_credits
            cost = (Decimal(str(credits_used)) * _CREDIT_COST).quantize(Decimal("0.01"))

            records.append(
                FinancialRecord(
                    record_date=end_date,
                    amount=-cost,
                    category=SpendCategory.SEARCH_DATA,
                    subcategory="Tavily Search Credits",
                    provider=Provider.TAVILY,
                    source="tavily_api",
                    raw_description=(
                        f"Tavily: {credits_used:,} credits used "
                        f"({self._remaining_credits:,} remaining)"
                    ),
                )
            )

        return records

    async def get_summary(self) -> dict:
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "total_credits": self._total_credits,
            "remaining_credits": self._remaining_credits,
            "credits_used": self._total_credits - self._remaining_credits,
            "cost_per_credit": str(_CREDIT_COST),
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Tavily cost data.

        Profile: Researcher plan, 3,000 searches/month at $0.008/credit = $24/month.
        Daily records with weekday/weekend variance.
        """
        rng = random.Random(903)
        records: list[FinancialRecord] = []

        daily_searches = 3000 / 30  # 100/day

        current = start_date
        while current <= end_date:
            is_weekend = current.weekday() >= 5
            weekend_factor = 0.3 if is_weekend else 1.0

            searches = int(daily_searches * weekend_factor * rng.uniform(0.6, 1.4))
            cost = (Decimal(str(searches)) * _CREDIT_COST).quantize(Decimal("0.01"))

            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-cost,
                    category=SpendCategory.SEARCH_DATA,
                    subcategory="Tavily Search Credits",
                    provider=Provider.TAVILY,
                    source="synthetic",
                    raw_description=f"Synthetic Tavily: {searches} searches",
                )
            )

            current += timedelta(days=1)

        return records


path: engine/connectors/twilio_costs.py
content: """Twilio connector — fetches actual spend from the Usage Records API."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_TWILIO_API = "https://api.twilio.com/2010-04-01"


class TwilioCostsConnector(BaseConnector):
    """Connector for Twilio usage and cost data.

    Twilio's Usage Records API returns actual dollar amounts (Price field)
    broken down by category, making this one of the cleanest billing APIs.
    """

    provider = Provider.TWILIO

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._account_sid: str = ""
        self._auth_token: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Twilio...")
                await asyncio.sleep(1)
                print("Authenticated with Twilio API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid Twilio credentials")
            return False

        # ── Normal mode ────────────────────────────────────────
        account_sid = credentials.get("account_sid", "")
        auth_token = credentials.get("auth_token", "")

        if not account_sid or not auth_token:
            logger.error("Missing Twilio account_sid or auth_token")
            return False

        self._account_sid = account_sid
        self._auth_token = auth_token
        self._client = httpx.AsyncClient(
            auth=(self._account_sid, self._auth_token),
            timeout=30.0,
        )

        try:
            resp = await self._client.get(
                f"{_TWILIO_API}/Accounts/{self._account_sid}.json"
            )
            resp.raise_for_status()
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 401:
                logger.error("Twilio credentials invalid")
            else:
                logger.error("Twilio validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Twilio connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Twilio connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []

        try:
            resp = await self._client.get(
                f"{_TWILIO_API}/Accounts/{self._account_sid}/Usage/Records/Daily.json",
                params={
                    "StartDate": start_date.isoformat(),
                    "EndDate": end_date.isoformat(),
                },
            )
            resp.raise_for_status()
            data = resp.json()

            for record in data.get("usage_records", []):
                category = record.get("category", "unknown")
                price_str = record.get("price", "0")
                cost = Decimal(price_str).quantize(Decimal("0.01"))
                if cost == 0:
                    continue

                record_date = date.fromisoformat(record["start_date"])
                count = record.get("count", 0)
                count_unit = record.get("count_unit", "")

                records.append(
                    FinancialRecord(
                        record_date=record_date,
                        amount=-cost,
                        category=SpendCategory.COMMUNICATION,
                        subcategory=category,
                        provider=Provider.TWILIO,
                        source="twilio_usage_api",
                        raw_description=f"Twilio {category}: {count} {count_unit}",
                    )
                )

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("Twilio rate limit — returning partial results")
            else:
                logger.error("Twilio usage fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Twilio usage fetch timed out — returning partial results")
        except httpx.HTTPError as exc:
            logger.error("Twilio usage fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(r.amount for r in records)
        by_category: dict[str, Decimal] = {}
        for r in records:
            cat = r.subcategory or "unknown"
            by_category[cat] = by_category.get(cat, Decimal("0")) + r.amount
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
            "spend_by_category": {k: str(v) for k, v in by_category.items()},
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Twilio cost data.

        Profile: ~$45/month total
          - SMS: ~$30/month (600 messages at ~$0.05 avg)
          - Voice: ~$15/month (50 minutes at ~$0.30/min)
        Daily records with weekday/weekend variance.
        """
        rng = random.Random(601)

        sms_daily_cost = 30.0 / 30  # ~$1/day
        sms_daily_count = 20  # ~600/month
        voice_daily_cost = 15.0 / 30  # ~$0.50/day
        voice_daily_minutes = 50.0 / 30  # ~1.67 min/day

        records: list[FinancialRecord] = []
        current = start_date
        while current <= end_date:
            is_weekend = current.weekday() >= 5
            weekend_factor = 0.4 if is_weekend else 1.0

            # SMS
            sms_cost = sms_daily_cost * weekend_factor * rng.uniform(0.7, 1.3)
            sms_count = int(sms_daily_count * weekend_factor * rng.uniform(0.7, 1.3))
            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-Decimal(str(round(sms_cost, 2))),
                    category=SpendCategory.COMMUNICATION,
                    subcategory="sms",
                    provider=Provider.TWILIO,
                    source="synthetic",
                    raw_description=f"Synthetic Twilio SMS: {sms_count} messages",
                )
            )

            # Voice
            voice_cost = voice_daily_cost * weekend_factor * rng.uniform(0.6, 1.4)
            voice_mins = round(voice_daily_minutes * weekend_factor * rng.uniform(0.6, 1.4), 1)
            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-Decimal(str(round(voice_cost, 2))),
                    category=SpendCategory.COMMUNICATION,
                    subcategory="calls",
                    provider=Provider.TWILIO,
                    source="synthetic",
                    raw_description=f"Synthetic Twilio voice: {voice_mins} minutes",
                )
            )

            current += timedelta(days=1)

        return records


path: engine/connectors/vercel_costs.py
content: """Vercel connector — fetches usage data and converts to costs."""
import logging
import random
from datetime import date, timedelta
from decimal import Decimal
from typing import Any

import httpx

from engine.connectors.base_connector import BaseConnector
from engine.models import FinancialRecord, Provider, SpendCategory

logger = logging.getLogger(__name__)

_VERCEL_API = "https://api.vercel.com"

# Published Vercel pricing (Pro plan, as of early 2026)
_PRO_BASE = Decimal("20.00")  # $20/month base
_FUNCTION_INVOCATION_RATE = Decimal("0.60")  # per 1M invocations (first 1M included)
_BANDWIDTH_RATE = Decimal("0.15")  # per GB after 1TB included
_BUILD_MINUTE_RATE = Decimal("0.01")  # per minute after 6000 included
_INCLUDED_INVOCATIONS = 1_000_000
_INCLUDED_BANDWIDTH_GB = 1000
_INCLUDED_BUILD_MINUTES = 6000


class VercelCostsConnector(BaseConnector):
    """Connector for Vercel usage and cost data."""

    provider = Provider.VERCEL

    def __init__(self) -> None:
        self.is_connected: bool = False
        self._api_token: str = ""
        self._team_id: str = ""
        self._client: httpx.AsyncClient | None = None

    async def connect(self, credentials: dict[str, Any]) -> bool:
        # ── Onboarding test mode ───────────────────────────────
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            if self.validate_test_credentials(credentials):
                import asyncio
                print("Connecting to Vercel...")
                await asyncio.sleep(1)
                print("Authenticated with Vercel API (test mode)")
                self._client = httpx.AsyncClient()
                self.is_connected = True
                return True
            logger.error("Authentication failed — invalid Vercel token")
            return False

        # ── Normal mode ────────────────────────────────────────
        api_token = credentials.get("api_token", "")
        if not api_token:
            logger.error("Missing Vercel api_token")
            return False

        self._api_token = api_token
        self._client = httpx.AsyncClient(
            base_url=_VERCEL_API,
            headers={"Authorization": f"Bearer {self._api_token}"},
            timeout=30.0,
        )

        try:
            resp = await self._client.get("/v2/user")
            resp.raise_for_status()
            user_data = resp.json().get("user", {})
            self._team_id = user_data.get("defaultTeamId", "")
            self.is_connected = True
            return True
        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 401:
                logger.error("Vercel API token invalid or expired")
            else:
                logger.error("Vercel validation failed: HTTP %s", exc.response.status_code)
            return False
        except httpx.TimeoutException:
            logger.error("Vercel connection timed out")
            return False
        except httpx.HTTPError as exc:
            logger.error("Vercel connection error: %s", exc)
            return False

    async def fetch_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        if not self.is_connected or self._client is None:
            logger.error("Not connected — call connect() first")
            return []

        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            return self.get_synthetic_records(start_date, end_date)

        records: list[FinancialRecord] = []

        try:
            params: dict[str, Any] = {}
            if self._team_id:
                params["teamId"] = self._team_id

            # Fetch usage for the billing period
            resp = await self._client.get("/v1/usage", params=params)
            resp.raise_for_status()
            usage = resp.json()

            # Parse usage metrics and convert to costs
            period_date = end_date  # Usage is aggregated per billing period

            # Function invocations
            invocations = usage.get("functionInvocations", 0)
            overage_invocations = max(0, invocations - _INCLUDED_INVOCATIONS)
            invocation_cost = (
                Decimal(overage_invocations) / Decimal("1000000") * _FUNCTION_INVOCATION_RATE
            ).quantize(Decimal("0.01"))

            # Bandwidth
            bandwidth_gb = Decimal(str(usage.get("bandwidthGB", 0)))
            overage_bw = max(Decimal("0"), bandwidth_gb - _INCLUDED_BANDWIDTH_GB)
            bw_cost = (overage_bw * _BANDWIDTH_RATE).quantize(Decimal("0.01"))

            # Build minutes
            build_minutes = usage.get("buildMinutes", 0)
            overage_build = max(0, build_minutes - _INCLUDED_BUILD_MINUTES)
            build_cost = (Decimal(overage_build) * _BUILD_MINUTE_RATE).quantize(Decimal("0.01"))

            # Base Pro plan cost
            records.append(
                FinancialRecord(
                    record_date=period_date,
                    amount=-_PRO_BASE,
                    category=SpendCategory.CLOUD_INFRASTRUCTURE,
                    subcategory="Vercel Pro Plan",
                    provider=Provider.VERCEL,
                    source="vercel_usage_api",
                    raw_description="Vercel Pro plan base",
                )
            )

            if invocation_cost > 0:
                records.append(
                    FinancialRecord(
                        record_date=period_date,
                        amount=-invocation_cost,
                        category=SpendCategory.CLOUD_INFRASTRUCTURE,
                        subcategory="Vercel Functions",
                        provider=Provider.VERCEL,
                        source="vercel_usage_api",
                        raw_description=f"Vercel function invocations ({invocations:,} total)",
                    )
                )

            if bw_cost > 0:
                records.append(
                    FinancialRecord(
                        record_date=period_date,
                        amount=-bw_cost,
                        category=SpendCategory.CLOUD_INFRASTRUCTURE,
                        subcategory="Vercel Bandwidth",
                        provider=Provider.VERCEL,
                        source="vercel_usage_api",
                        raw_description=f"Vercel bandwidth ({bandwidth_gb} GB)",
                    )
                )

            if build_cost > 0:
                records.append(
                    FinancialRecord(
                        record_date=period_date,
                        amount=-build_cost,
                        category=SpendCategory.CLOUD_INFRASTRUCTURE,
                        subcategory="Vercel Build",
                        provider=Provider.VERCEL,
                        source="vercel_usage_api",
                        raw_description=f"Vercel build minutes ({build_minutes:,} total)",
                    )
                )

        except httpx.HTTPStatusError as exc:
            if exc.response.status_code == 429:
                logger.warning("Vercel rate limit — returning partial results")
            else:
                logger.error("Vercel usage fetch failed: HTTP %s", exc.response.status_code)
        except httpx.TimeoutException:
            logger.error("Vercel usage fetch timed out — returning partial results")
        except httpx.HTTPError as exc:
            logger.error("Vercel usage fetch error: %s", exc)

        return records

    async def get_summary(self) -> dict:
        today = date.today()
        month_start = today.replace(day=1)
        records = await self.fetch_records(month_start, today)
        total = sum(r.amount for r in records)
        return {
            "provider": self.provider.value,
            "connected": self.is_connected,
            "current_month_spend": str(total),
        }

    # ── Synthetic data ──────────────────────────────────────────

    def get_synthetic_records(
        self, start_date: date, end_date: date
    ) -> list[FinancialRecord]:
        """Generate realistic Vercel cost data.

        Profile: ~$20/month Pro plan, 5,000 function invocations/day,
        100GB bandwidth/month. All within included limits, so only base cost.
        Generates monthly billing records.
        """
        rng = random.Random(401)
        records: list[FinancialRecord] = []

        # Generate one billing record per month
        current = start_date.replace(day=1)
        while current <= end_date:
            month_end = (current + timedelta(days=32)).replace(day=1) - timedelta(days=1)
            if month_end > end_date:
                month_end = end_date

            # Pro plan base
            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=-_PRO_BASE,
                    category=SpendCategory.CLOUD_INFRASTRUCTURE,
                    subcategory="Vercel Pro Plan",
                    provider=Provider.VERCEL,
                    source="synthetic",
                    raw_description="Synthetic Vercel Pro plan",
                )
            )

            # Function invocations: ~5,000/day = ~150,000/month (within 1M included)
            days_in_month = (month_end - current).days + 1
            invocations = int(5000 * days_in_month * rng.uniform(0.8, 1.2))
            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=Decimal("0"),  # within included tier
                    category=SpendCategory.CLOUD_INFRASTRUCTURE,
                    subcategory="Vercel Functions",
                    provider=Provider.VERCEL,
                    source="synthetic",
                    raw_description=f"Synthetic Vercel functions ({invocations:,} invocations, within included)",
                )
            )

            # Bandwidth: ~100GB/month (within 1TB included)
            bandwidth_gb = round(100 * rng.uniform(0.8, 1.2), 1)
            records.append(
                FinancialRecord(
                    record_date=current,
                    amount=Decimal("0"),  # within included tier
                    category=SpendCategory.CLOUD_INFRASTRUCTURE,
                    subcategory="Vercel Bandwidth",
                    provider=Provider.VERCEL,
                    source="synthetic",
                    raw_description=f"Synthetic Vercel bandwidth ({bandwidth_gb} GB, within included)",
                )
            )

            # Next month
            current = (current + timedelta(days=32)).replace(day=1)

        return records


path: engine/credentials.py
content: """Credential store — reads API keys from system keyring, env vars, or test mode.

Lookup order:
1. Onboarding test mode → returns TEST_CREDENTIALS
2. System keyring (via `keyring` library)
3. Environment variables (e.g. OPENAI_API_KEY)
4. Explicitly passed credentials (caller's responsibility)

Keys are stored in the system keychain under service name
"anvx-token-economy-intel" with key format "{provider}:{label}:{field}".
"""
import json
import logging
import os
from typing import Any

import keyring

logger = logging.getLogger(__name__)

_SERVICE = "anvx-token-economy-intel"

# Maps provider names to their credential fields
PROVIDER_FIELDS: dict[str, list[str]] = {
    "openai": ["api_key"],
    "anthropic": ["api_key"],
    "aws": ["access_key_id", "secret_access_key"],
    "gcp": ["service_account_json"],
    "vercel": ["api_token"],
    "cloudflare": ["api_token"],
    "stripe": ["api_key"],
    "twilio": ["account_sid", "auth_token"],
    "sendgrid": ["api_key"],
    "datadog": ["api_key", "app_key"],
    "langsmith": ["api_key"],
    "pinecone": ["api_key"],
    "tavily": ["api_key"],
    "crypto_wallet": ["address"],  # special: chain+address pairs
    "coinbase": ["api_key", "api_secret"],
    "binance": ["api_key", "api_secret"],
}

# Maps provider+field to environment variable name
_ENV_VAR_MAP: dict[str, str] = {
    "openai:api_key": "OPENAI_API_KEY",
    "anthropic:api_key": "ANTHROPIC_API_KEY",
    "aws:access_key_id": "AWS_ACCESS_KEY_ID",
    "aws:secret_access_key": "AWS_SECRET_ACCESS_KEY",
    "stripe:api_key": "STRIPE_API_KEY",
    "vercel:api_token": "VERCEL_TOKEN",
    "cloudflare:api_token": "CLOUDFLARE_API_TOKEN",
    "twilio:account_sid": "TWILIO_ACCOUNT_SID",
    "twilio:auth_token": "TWILIO_AUTH_TOKEN",
    "sendgrid:api_key": "SENDGRID_API_KEY",
    "datadog:api_key": "DD_API_KEY",
    "datadog:app_key": "DD_APP_KEY",
    "langsmith:api_key": "LANGSMITH_API_KEY",
    "pinecone:api_key": "PINECONE_API_KEY",
    "tavily:api_key": "TAVILY_API_KEY",
    "coinbase:api_key": "COINBASE_API_KEY",
    "coinbase:api_secret": "COINBASE_API_SECRET",
    "binance:api_key": "BINANCE_API_KEY",
    "binance:api_secret": "BINANCE_API_SECRET",
}


class CredentialStore:
    """Reads credentials from keyring, env vars, or test mode."""

    @staticmethod
    def get_credential(
        provider: str, label: str = "default", field: str = "api_key"
    ) -> str | None:
        """Read a single credential value.

        Checks: test mode → keyring → env var. Returns None if not found.
        """
        # 1. Test mode
        from engine.utils import is_onboarding_test_mode, TEST_CREDENTIALS
        if is_onboarding_test_mode():
            return _get_test_credential(provider, field)

        # 2. Keyring
        try:
            key = f"{provider}:{label}:{field}"
            value = keyring.get_password(_SERVICE, key)
            if value:
                return value
        except Exception as exc:
            logger.debug("Keyring read failed for %s: %s", provider, exc)

        # 3. Environment variable
        env_key = _ENV_VAR_MAP.get(f"{provider}:{field}")
        if env_key:
            value = os.environ.get(env_key)
            if value:
                return value

        return None

    @staticmethod
    def get_manifest() -> dict[str, list[str]]:
        """Returns the provider manifest: {provider: [labels]}."""
        from engine.utils import is_onboarding_test_mode
        if is_onboarding_test_mode():
            from engine.utils import TEST_CREDENTIALS
            return {p: ["default"] for p in TEST_CREDENTIALS}

        try:
            raw = keyring.get_password(_SERVICE, "manifest")
            return json.loads(raw) if raw else {}
        except Exception:
            return {}

    @staticmethod
    def get_all_credentials(
        provider: str, label: str = "default"
    ) -> dict[str, str]:
        """Returns all fields for a provider as a dict.

        E.g. {"api_key": "sk-...", "api_secret": "..."}
        """
        fields = PROVIDER_FIELDS.get(provider, ["api_key"])
        result: dict[str, str] = {}
        for field in fields:
            value = CredentialStore.get_credential(provider, label, field)
            if value:
                result[field] = value
        return result

    @staticmethod
    def store_credential(
        provider: str, label: str, field: str, value: str
    ) -> None:
        """Store a credential in the system keyring."""
        key = f"{provider}:{label}:{field}"
        keyring.set_password(_SERVICE, key, value)

    @staticmethod
    def update_manifest(provider: str, label: str) -> None:
        """Add a provider+label to the manifest."""
        manifest = CredentialStore.get_manifest()
        labels = manifest.get(provider, [])
        if label not in labels:
            labels.append(label)
        manifest[provider] = labels
        keyring.set_password(_SERVICE, "manifest", json.dumps(manifest))


def _get_test_credential(provider: str, field: str) -> str | None:
    """Extract a specific field from TEST_CREDENTIALS."""
    from engine.utils import TEST_CREDENTIALS
    creds = TEST_CREDENTIALS.get(provider)
    if creds is None:
        return None
    if isinstance(creds, list):
        return creds[0] if field == "api_key" else None
    if isinstance(creds, dict):
        return creds.get(field)
    return None


path: engine/intelligence/__init__.py
content: # Intelligence package
from engine.intelligence.categoriser import categorise_records
from engine.intelligence.anomaly_detector import detect_anomalies
from engine.intelligence.recommender import generate_recommendations
from engine.intelligence.financial_model import FinancialModelManager

__all__ = [
    "categorise_records",
    "detect_anomalies",
    "generate_recommendations",
    "FinancialModelManager",
]


path: engine/intelligence/anomaly_detector.py
content: """Anomaly detection — flags spending categories that deviate from rolling baseline."""
import logging
from collections import defaultdict
from datetime import date, timedelta
from decimal import Decimal

from engine.models import Anomaly, FinancialRecord

logger = logging.getLogger(__name__)

_DEVIATION_THRESHOLD = 0.30  # 30%
_BASELINE_WEEKS = 4


def detect_anomalies(
    records: list[FinancialRecord],
    current_week_start: date | None = None,
) -> list[Anomaly]:
    """Compare current week spend per category against a 4-week rolling baseline.

    Args:
        records: All historical records (should cover at least 5 weeks).
        current_week_start: Start of the week to evaluate. Defaults to the
            most recent Monday on or before today.

    Returns:
        List of Anomaly objects for categories exceeding ±30% deviation.
    """
    if not records:
        return []

    if current_week_start is None:
        today = date.today()
        current_week_start = today - timedelta(days=today.weekday())

    current_week_end = current_week_start + timedelta(days=6)

    # ── Bucket records by (category, week_start) ────────────────
    weekly_spend: dict[str, dict[date, Decimal]] = defaultdict(lambda: defaultdict(Decimal))

    for record in records:
        # Costs are negative, so use abs() for comparison
        amount = abs(record.amount)
        cat = record.category.value
        # Find the Monday of this record's week
        week_start = record.record_date - timedelta(days=record.record_date.weekday())
        weekly_spend[cat][week_start] += amount

    # ── Compute baseline and compare ────────────────────────────
    anomalies: list[Anomaly] = []
    baseline_start = current_week_start - timedelta(weeks=_BASELINE_WEEKS)

    for category, weeks in weekly_spend.items():
        # Current week total
        current_total = weeks.get(current_week_start, Decimal("0"))

        # 4-week rolling baseline (the 4 weeks before current)
        baseline_weeks: list[Decimal] = []
        for i in range(1, _BASELINE_WEEKS + 1):
            week = current_week_start - timedelta(weeks=i)
            if week in weeks:
                baseline_weeks.append(weeks[week])

        if not baseline_weeks:
            continue

        baseline_avg = sum(baseline_weeks) / len(baseline_weeks)

        if baseline_avg == 0:
            if current_total > 0:
                # New spend appeared — flag it
                anomalies.append(
                    Anomaly(
                        category=category,
                        description=f"New spending detected in {category}: ${current_total} this week with no prior baseline",
                        current_amount=current_total,
                        baseline_amount=Decimal("0"),
                        deviation_percent=100.0,
                        severity="high",
                    )
                )
            continue

        deviation = float((current_total - baseline_avg) / baseline_avg)

        if abs(deviation) > _DEVIATION_THRESHOLD:
            direction = "above" if deviation > 0 else "below"
            severity = _compute_severity(abs(deviation))

            anomalies.append(
                Anomaly(
                    category=category,
                    description=(
                        f"{category} spending is {abs(deviation):.0%} {direction} "
                        f"the 4-week baseline (${current_total:.2f} vs "
                        f"${baseline_avg:.2f} avg)"
                    ),
                    current_amount=current_total,
                    baseline_amount=baseline_avg.quantize(Decimal("0.01")),
                    deviation_percent=round(deviation * 100, 1),
                    severity=severity,
                )
            )

    # Sort by absolute deviation (most extreme first)
    anomalies.sort(key=lambda a: abs(a.deviation_percent), reverse=True)
    return anomalies


def _compute_severity(abs_deviation: float) -> str:
    """Map absolute deviation to severity level."""
    if abs_deviation > 1.0:
        return "critical"
    if abs_deviation > 0.5:
        return "high"
    return "medium"


path: engine/intelligence/categoriser.py
content: """AI-powered spend categoriser for uncategorised financial records."""
import json
import logging
from decimal import Decimal

import anthropic

from engine.models import FinancialRecord, SpendCategory
from engine.utils import is_synthetic_mode

logger = logging.getLogger(__name__)

_BATCH_SIZE = 20

_SYSTEM_PROMPT = """You are a financial record categoriser for an AI-native business.
Given a batch of financial records, classify each into exactly one category.

Valid categories:
- ai_inference: LLM API calls, model inference costs
- ai_training: Fine-tuning, training compute
- cloud_infrastructure: AWS, GCP, Azure, hosting, storage, CDN
- saas_subscription: Software subscriptions (Slack, GitHub, Figma, etc.)
- payment_processing: Stripe fees, PayPal fees, processing costs
- communication: Twilio, SendGrid, SMS, voice, email sending
- monitoring: Datadog, LangSmith, APM, logging, tracing, observability
- search_data: Pinecone, Tavily, vector databases, search APIs, retrieval
- crypto_holdings: Cryptocurrency balances or transfers
- revenue: Incoming payments, subscription revenue, sales
- other: Cannot be determined

Respond with a JSON array of objects, one per record, each with:
  {"index": <int>, "category": "<category_value>", "confidence": <0.0-1.0>}

Only output the JSON array. No markdown, no explanation."""

# Keyword rules for synthetic mode (no API call needed)
_KEYWORD_RULES: list[tuple[list[str], SpendCategory, float]] = [
    (["gpt", "openai", "claude", "anthropic", "gemini", "google ai", "llm", "inference", "token", "embedding"],
     SpendCategory.AI_INFERENCE, 0.9),
    (["fine-tune", "finetune", "training", "train"],
     SpendCategory.AI_TRAINING, 0.85),
    (["aws", "gcp", "azure", "s3", "ec2", "lambda", "cloud", "hosting", "cdn", "vercel", "heroku"],
     SpendCategory.CLOUD_INFRASTRUCTURE, 0.85),
    (["slack", "github", "figma", "notion", "jira", "linear", "subscription", "saas", "license"],
     SpendCategory.SAAS_SUBSCRIPTION, 0.8),
    (["stripe fee", "processing fee", "paypal fee", "payment processing"],
     SpendCategory.PAYMENT_PROCESSING, 0.9),
    (["twilio", "sendgrid", "sms", "voice call", "email send", "messaging"],
     SpendCategory.COMMUNICATION, 0.85),
    (["datadog", "langsmith", "monitoring", "observability", "tracing", "traces", "apm", "logging"],
     SpendCategory.MONITORING, 0.85),
    (["pinecone", "tavily", "search", "vector", "embedding index", "retrieval"],
     SpendCategory.SEARCH_DATA, 0.85),
    (["eth", "btc", "sol", "usdc", "crypto", "wallet", "bitcoin", "ethereum", "coinbase", "binance"],
     SpendCategory.CRYPTO_HOLDINGS, 0.85),
    (["facebook ads", "instagram ads", "meta ads", "google ads", "youtube ads", "display ads",
      "ad spend", "campaign", "impressions", "cpc", "cpm", "adwords"],
     SpendCategory.ADVERTISING, 0.85),
    (["charge", "payment received", "invoice paid", "revenue", "subscription revenue", "sale"],
     SpendCategory.REVENUE, 0.8),
]


async def categorise_records(
    records: list[FinancialRecord],
) -> list[FinancialRecord]:
    """Categorise records that have category=OTHER or low confidence.

    Records already categorised with high confidence are passed through.
    Uncategorised records are batched and sent to Claude API (or keyword-matched
    in synthetic mode).

    Returns a new list with updated categories and confidence scores.
    """
    needs_categorisation: list[tuple[int, FinancialRecord]] = []
    result = list(records)

    for i, record in enumerate(records):
        if record.category == SpendCategory.OTHER or record.confidenc < 0.5:
            needs_categorisation.append((i, record))

    if not needs_categorisation:
        return result

    if is_synthetic_mode():
        for idx, record in needs_categorisation:
            category, confidence = _categorise_by_keywords(record)
            result[idx] = record.model_copy(
                update={"category": category, "confidenc": confidence}
            )
        return result

    # Batch into groups of _BATCH_SIZE for API efficiency
    for batch_start in range(0, len(needs_categorisation), _BATCH_SIZE):
        batch = needs_categorisation[batch_start : batch_start + _BATCH_SIZE]
        categorised = await _categorise_batch_via_api(batch)
        for (idx, _original), (category, confidence) in zip(batch, categorised):
            result[idx] = result[idx].model_copy(
                update={"category": category, "confidenc": confidence}
            )

    return result


def _categorise_by_keywords(record: FinancialRecord) -> tuple[SpendCategory, float]:
    """Rule-based categorisation using keyword matching (for synthetic mode)."""
    text = " ".join(
        str(v).lower()
        for v in [
            record.raw_description or "",
            record.subcategory or "",
            record.model or "",
            record.provider.value,
            record.source,
        ]
    )

    for keywords, category, confidence in _KEYWORD_RULES:
        if any(kw in text for kw in keywords):
            return category, confidence

    return SpendCategory.OTHER, 0.3


async def _categorise_batch_via_api(
    batch: list[tuple[int, FinancialRecord]],
) -> list[tuple[SpendCategory, float]]:
    """Send a batch of records to Claude API for categorisation."""
    records_payload = []
    for i, (_idx, record) in enumerate(batch):
        records_payload.append(
            {
                "index": i,
                "amount": str(record.amount),
                "provider": record.provider.value,
                "source": record.source,
                "description": record.raw_description or "",
                "subcategory": record.subcategory or "",
                "model": record.model or "",
            }
        )

    try:
        client = anthropic.AsyncAnthropic()
        message = await client.messages.create(
            model="claude-haiku-4-5-20251001",
            max_tokens=1024,
            system=_SYSTEM_PROMPT,
            messages=[
                {
                    "role": "user",
                    "content": f"Categorise these {len(records_payload)} records:\n{json.dumps(records_payload)}",
                }
            ],
        )

        response_text = message.content[0].text
        classifications = json.loads(response_text)

        results: list[tuple[SpendCategory, float]] = []
        # Build lookup by index
        by_index = {c["index"]: c for c in classifications}
        for i in range(len(batch)):
            if i in by_index:
                cat_str = by_index[i].get("category", "other")
                confidence = float(by_index[i].get("confidence", 0.5))
                try:
                    category = SpendCategory(cat_str)
                except ValueError:
                    category = SpendCategory.OTHER
                    confidence = 0.3
                results.append((category, confidence))
            else:
                results.append((SpendCategory.OTHER, 0.3))

        return results

    except anthropic.AuthenticationError:
        logger.error("Anthropic API key invalid — falling back to keyword categorisation")
    except anthropic.RateLimitError:
        logger.warning("Anthropic rate limit — falling back to keyword categorisation")
    except anthropic.APITimeoutError:
        logger.error("Anthropic API timeout — falling back to keyword categorisation")
    except (json.JSONDecodeError, KeyError, IndexError) as exc:
        logger.error("Failed to parse categorisation response: %s", exc)
    except anthropic.APIError as exc:
        logger.error("Anthropic API error: %s", exc)

    # Fallback: keyword categorisation for the whole batch
    return [_categorise_by_keywords(record) for _idx, record in batch]


path: engine/intelligence/financial_model.py
content: """Persistent financial model — aggregates, saves, and summarises all connector data."""
import json
import logging
import os
import tempfile
from collections import defaultdict
from datetime import date, datetime, timedelta
from decimal import Decimal
from pathlib import Path

from pydantic import BaseModel

from engine.models import (
    Anomaly,
    FinancialRecord,
    FinancialSummary,
    Recommendation,
    SpendCategory,
)
from engine.utils import format_currency, format_percent

logger = logging.getLogger(__name__)

_DEFAULT_STATE_DIR = Path.home() / ".token-economy-intel"
_DEFAULT_STATE_FILE = _DEFAULT_STATE_DIR / "model.json"


class _QueryEntry(BaseModel):
    timestamp: datetime
    query: str


class _ModelState(BaseModel):
    """Serialisable internal state."""

    records: list[FinancialRecord] = []
    connected_accounts: list[str] = []
    query_history: list[_QueryEntry] = []
    last_updated: datetime = datetime(2000, 1, 1)


class FinancialModelManager:
    """Central financial model — loads, updates, persists, and summarises.

    Usage:
        mgr = FinancialModelManager()          # uses default path
        mgr.load()                             # reads JSON (no-op if missing)
        mgr.add_records(records, "openai")     # merge new data
        mgr.save()                             # atomic write
        summary = mgr.get_summary()
    """

    def __init__(self, state_path: str | Path | None = None) -> None:
        self._path = Path(state_path) if state_path else _DEFAULT_STATE_FILE
        self._state = _ModelState()

    # ── Persistence ─────────────────────────────────────────────

    def load(self) -> None:
        """Load state from JSON file. No-op if file doesn't exist."""
        if not self._path.exists():
            logger.info("No existing model at %s — starting fresh", self._path)
            return
        try:
            raw = self._path.read_text(encoding="utf-8")
            self._state = _ModelState.model_validate_json(raw)
            logger.info(
                "Loaded model: %d records, last updated %s",
                len(self._state.records),
                self._state.last_updated.isoformat(),
            )
        except (json.JSONDecodeError, ValueError) as exc:
            logger.error("Corrupt model file %s: %s — starting fresh", self._path, exc)
            self._state = _ModelState()

    def save(self) -> None:
        """Atomic save: write to temp file in the same directory, then rename."""
        self._path.parent.mkdir(parents=True, exist_ok=True)
        self._state.last_updated = datetime.now()

        data = self._state.model_dump_json(indent=2)
        fd, tmp_path = tempfile.mkstemp(
            dir=self._path.parent, suffix=".tmp", prefix=".model_"
        )
        try:
            os.write(fd, data.encode("utf-8"))
            os.close(fd)
            os.replace(tmp_path, self._path)
            logger.info("Saved model to %s (%d records)", self._path, len(self._state.records))
        except OSError:
            os.close(fd) if not os.get_inheritable(fd) else None
            if os.path.exists(tmp_path):
                os.unlink(tmp_path)
            raise

    # ── Record management ───────────────────────────────────────

    def add_records(
        self, records: list[FinancialRecord], account_name: str
    ) -> int:
        """Merge new records into the model. Returns count of new records added.

        Deduplicates by (record_date, amount, provider, source, model).
        """
        existing_keys = {_record_key(r) for r in self._state.records}
        new_records = [r for r in records if _record_key(r) not in existing_keys]

        self._state.records.extend(new_records)
        if account_name not in self._state.connected_accounts:
            self._state.connected_accounts.append(account_name)
        self._state.last_updated = datetime.now()

        return len(new_records)

    def record_query(self, query: str) -> None:
        """Track a user query for improving future recommendations."""
        self._state.query_history.append(
            _QueryEntry(timestamp=datetime.now(), query=query)
        )
        # Keep last 200 queries
        if len(self._state.query_history) > 200:
            self._state.query_history = self._state.query_history[-200:]

    @property
    def records(self) -> list[FinancialRecord]:
        return self._state.records

    @property
    def query_history(self) -> list[_QueryEntry]:
        return self._state.query_history

    # ── Summaries ───────────────────────────────────────────────

    def get_summary(self) -> FinancialSummary:
        """Build a FinancialSummary from current state."""
        records = self._state.records
        now = datetime.now()
        thirty_days_ago = (now - timedelta(days=30)).date()

        recent = [r for r in records if r.record_date >= thirty_days_ago]

        # Spend by category (costs are negative)
        spend_by_cat: dict[str, Decimal] = defaultdict(Decimal)
        spend_by_prov: dict[str, Decimal] = defaultdict(Decimal)
        revenue = Decimal("0")
        crypto = Decimal("0")

        for r in recent:
            if r.category == SpendCategory.REVENUE:
                revenue += r.amount
            elif r.category == SpendCategory.CRYPTO_HOLDINGS:
                crypto += r.amount
            else:
                spend_by_cat[r.category.value] += abs(r.amount)
                spend_by_prov[r.provider.value] += abs(r.amount)

        total_spend = sum(spend_by_cat.values())

        # Date coverage
        if records:
            earliest = min(r.record_date for r in records)
            latest = max(r.record_date for r in records)
            coverage_days = (latest - earliest).days + 1
        else:
            coverage_days = 0

        return FinancialSummary(
            last_updated=self._state.last_updated,
            total_monthly_spend=total_spend,
            spend_by_category=dict(spend_by_cat),
            spend_by_provider=dict(spend_by_prov),
            revenue_monthly=revenue if revenue > 0 else None,
            crypto_holdings_usd=crypto if crypto > 0 else None,
            connected_accounts=list(self._state.connected_accounts),
            data_coverage_days=coverage_days,
            record_count=len(records),
        )

    def get_context_for_llm(self) -> str:
        """Return a concise text summary for inclusion in an LLM system prompt."""
        s = self.get_summary()
        lines = [
            "## Token Economy Financial Context",
            f"Last updated: {s.last_updated:%Y-%m-%d %H:%M}",
            f"Data coverage: {s.data_coverage_days} days, {s.record_count} records",
            f"Connected: {', '.join(s.connected_accounts) or 'none'}",
            "",
            f"Monthly spend: {format_currency(s.total_monthly_spend)}",
        ]

        if s.spend_by_category:
            lines.append("Spend by category:")
            for cat, amt in sorted(
                s.spend_by_category.items(), key=lambda x: x[1], reverse=True
            ):
                lines.append(f"  - {cat}: {format_currency(amt)}")

        if s.spend_by_provider:
            lines.append("Spend by provider:")
            for prov, amt in sorted(
                s.spend_by_provider.items(), key=lambda x: x[1], reverse=True
            ):
                lines.append(f"  - {prov}: {format_currency(amt)}")

        if s.revenue_monthly is not None:
            lines.append(f"Monthly revenue: {format_currency(s.revenue_monthly)}")
            if s.total_monthly_spend > 0:
                margin = float(
                    (s.revenue_monthly - s.total_monthly_spend) / s.revenue_monthly * 100
                )
                lines.append(f"Gross margin: {format_percent(margin)}")

        if s.crypto_holdings_usd is not None:
            lines.append(f"Crypto holdings: {format_currency(s.crypto_holdings_usd)}")

        if s.anomalies:
            lines.append(f"Active anomalies: {len(s.anomalies)}")
            for a in s.anomalies[:3]:
                lines.append(f"  - [{a.severity}] {a.description}")

        if s.recommendations:
            lines.append(f"Recommendations: {len(s.recommendations)}")
            for r in s.recommendations[:3]:
                lines.append(f"  - {r.description}")

        return "\n".join(lines)

    # ── Reset ───────────────────────────────────────────────────

    def reset(self) -> None:
        """Clear all state (for testing). Does NOT delete the file."""
        self._state = _ModelState()

    def delete_state_file(self) -> None:
        """Delete the persisted state file (for testing)."""
        if self._path.exists():
            self._path.unlink()


def _record_key(r: FinancialRecord) -> tuple:
    return (r.record_date, r.amount, r.provider, r.source, r.model)


path: engine/intelligence/optimization/__init__.py
content: """Optimization modules for the recommendation engine."""
from engine.intelligence.optimization.base_module import OptimizationModule
from engine.intelligence.optimization.model_routing import ModelRoutingModule
from engine.intelligence.optimization.caching_estimator import CachingEstimatorModule
from engine.intelligence.optimization.batch_detector import BatchDetectorModule
from engine.intelligence.optimization.unit_economics import UnitEconomicsModule
from engine.intelligence.optimization.price_comparison import PriceComparisonModule
from engine.intelligence.optimization.spend_forecast import SpendForecastModule

ALL_MODULES: list[type[OptimizationModule]] = [
    ModelRoutingModule,
    CachingEstimatorModule,
    BatchDetectorModule,
    UnitEconomicsModule,
    PriceComparisonModule,
    SpendForecastModule,
]

__all__ = [
    "OptimizationModule",
    "ALL_MODULES",
    "ModelRoutingModule",
    "CachingEstimatorModule",
    "BatchDetectorModule",
    "UnitEconomicsModule",
    "PriceComparisonModule",
    "SpendForecastModule",
]


path: engine/intelligence/optimization/base_module.py
content: """Abstract base class for optimization modules."""
from abc import ABC, abstractmethod

from engine.intelligence.pricing_fetcher import PricingFetcher
from engine.models import FinancialRecord, FinancialSummary, Recommendation


class OptimizationModule(ABC):
    """Base class for all optimization analysis modules.

    Every module must produce recommendations with:
    - estimated_monthly_savings (specific dollar amount)
    - confidence (high/medium/low)
    - methodology (how the number was calculated)
    """

    name: str = ""
    description: str = ""
    required_providers: list[str] = []

    @abstractmethod
    def analyse(
        self,
        records: list[FinancialRecord],
        summary: FinancialSummary,
        pricing: PricingFetcher,
    ) -> list[Recommendation]:
        """Run analysis and return recommendations."""
        ...


path: engine/intelligence/optimization/batch_detector.py
content: """Batch processing detector — identifies consistent-volume workloads eligible for batch APIs."""
import logging
import math
from collections import defaultdict
from decimal import Decimal

from engine.intelligence.optimization.base_module import OptimizationModule
from engine.intelligence.pricing_fetcher import PricingFetcher
from engine.models import (
    FinancialRecord,
    FinancialSummary,
    Recommendation,
    SpendCategory,
)

logger = logging.getLogger(__name__)

# Published batch discount rates (verifiable from provider docs)
_BATCH_DISCOUNTS: dict[str, Decimal] = {
    "openai": Decimal("0.50"),  # 50% discount on batch API
    # Anthropic does not currently offer a batch discount
}


class BatchDetectorModule(OptimizationModule):
    """Detects workloads suitable for batch processing.

    Analyses daily request volume patterns:
    - Calculates daily volumes and identifies "steady state" days
      (within 1 standard deviation of the mean)
    - If >70% of days are steady state, the workload is a batch candidate
    - Batch-eligible percentage is derived from (steady days / total days)
    """

    name = "batch_detector"
    description = "Detect workloads suitable for batch API processing"

    def analyse(
        self,
        records: list[FinancialRecord],
        summary: FinancialSummary,
        pricing: PricingFetcher,
    ) -> list[Recommendation]:
        recs: list[Recommendation] = []

        # Group AI inference records by provider
        by_provider: dict[str, list[FinancialRecord]] = defaultdict(list)
        for r in records:
            if r.category == SpendCategory.AI_INFERENCE:
                by_provider[r.provider.value].append(r)

        for provider, provider_records in by_provider.items():
            # Only providers with batch API discounts
            if provider not in _BATCH_DISCOUNTS:
                continue

            # Need enough data
            if len(provider_records) < 30:
                continue

            # Calculate daily request counts
            daily_counts: dict[str, int] = defaultdict(int)
            daily_costs: dict[str, Decimal] = defaultdict(Decimal)
            for r in provider_records:
                day_key = r.record_date.isoformat()
                daily_counts[day_key] += 1
                daily_costs[day_key] += abs(r.amount)

            counts = list(daily_counts.values())
            total_days = len(counts)
            if total_days < 14:
                continue

            # Calculate mean and std dev of daily request counts
            mean_count = sum(counts) / total_days
            if mean_count == 0:
                continue
            variance = sum((c - mean_count) ** 2 for c in counts) / total_days
            std_dev = math.sqrt(variance)

            # Steady state days: within 1 std dev of mean
            steady_days = sum(
                1 for c in counts
                if abs(c - mean_count) <= std_dev
            )
            steady_pct = steady_days / total_days

            if steady_pct < 0.70:
                continue  # too variable for batch

            # Batch-eligible percentage derived from data
            batch_eligible_pct = Decimal(str(round(steady_pct, 2)))

            # Calculate savings
            total_cost = sum(daily_costs.values())
            date_range = (
                max(r.record_date for r in provider_records)
                - min(r.record_date for r in provider_records)
            ).days or 1
            monthly_factor = Decimal("30") / Decimal(str(date_range))
            monthly_cost = total_cost * monthly_factor

            discount = _BATCH_DISCOUNTS[provider]
            batchable_cost = monthly_cost * batch_eligible_pct
            monthly_savings = (batchable_cost * discount).quantize(Decimal("0.01"))

            if monthly_savings <= Decimal("0.50"):
                continue

            recs.append(
                Recommendation(
                    rec_type="batch_processing",
                    description=(
                        f"Your {provider.title()} usage shows consistent daily volumes "
                        f"({int(steady_pct * 100)}% of days within normal range). "
                        f"{int(float(batch_eligible_pct) * 100)}% of your workload appears "
                        f"schedulable. At {provider.title()}'s batch rate "
                        f"({int(float(discount) * 100)}% discount), "
                        f"estimated savings: ${monthly_savings}/month."
                    ),
                    estimated_monthly_savings=monthly_savings,
                    confidence="high" if steady_pct > 0.80 else "medium",
                    action_required=(
                        f"Migrate consistent {provider.title()} workloads to the Batch API. "
                        f"Submit requests as batch jobs for non-time-sensitive tasks."
                    ),
                    category=SpendCategory.AI_INFERENCE,
                    source_module=self.name,
                    methodology=(
                        f"Daily volume analysis over {total_days} days: "
                        f"mean={mean_count:.1f} requests/day, std_dev={std_dev:.1f}. "
                        f"{steady_days}/{total_days} days within 1 std dev ({int(steady_pct*100)}% steady). "
                        f"Monthly {provider.title()} spend: ${monthly_cost:.2f}. "
                        f"Batchable fraction: {int(float(batch_eligible_pct)*100)}% = "
                        f"${batchable_cost:.2f}. "
                        f"Batch discount: {int(float(discount)*100)}% "
                        f"(published {provider.title()} rate). "
                        f"Savings: ${monthly_savings}/mo."
                    ),
                )
            )

        return recs


path: engine/intelligence/optimization/caching_estimator.py
content: """Prompt caching estimator — analyses input token consistency to estimate caching opportunity."""
import logging
import math
from collections import defaultdict
from decimal import Decimal

from engine.intelligence.optimization.base_module import OptimizationModule
from engine.intelligence.pricing_fetcher import CACHED_TOKEN_DISCOUNTS, PricingFetcher
from engine.models import (
    FinancialRecord,
    FinancialSummary,
    Recommendation,
    SpendCategory,
)

logger = logging.getLogger(__name__)

# Providers that support prompt caching
_CACHEABLE_PROVIDERS = {"openai", "anthropic"}


class CachingEstimatorModule(OptimizationModule):
    """Estimates prompt caching savings using data-derived cacheable ratio.

    Instead of assuming a cache hit percentage, calculates it from the
    coefficient of variation (CV) of input token counts per provider:
    - LOW CV (<0.3): highly consistent inputs → high caching opportunity
    - MEDIUM CV (0.3-0.7): moderate opportunity
    - HIGH CV (>0.7): content varies too much, caching won't help
    """

    name = "caching_estimator"
    description = "Estimate savings from prompt caching based on input consistency"

    def analyse(
        self,
        records: list[FinancialRecord],
        summary: FinancialSummary,
        pricing: PricingFetcher,
    ) -> list[Recommendation]:
        recs: list[Recommendation] = []

        # Group AI records by (provider, model) to avoid mixing different
        # token distributions (e.g. per-request vs daily aggregate models)
        by_provider_model: dict[tuple[str, str], list[FinancialRecord]] = defaultdict(list)
        for r in records:
            if (
                r.category == SpendCategory.AI_INFERENCE
                and r.provider.value in _CACHEABLE_PROVIDERS
                and r.model
                and r.tokens_input is not None
                and r.tokens_input > 0
            ):
                by_provider_model[(r.provider.value, r.model)].append(r)

        for (provider, model_name), provider_records in by_provider_model.items():
            if len(provider_records) < 20:
                continue

            # Calculate coefficient of variation of input tokens
            input_tokens = [r.tokens_input for r in provider_records if r.tokens_input]
            mean_input = sum(input_tokens) / len(input_tokens)
            if mean_input == 0:
                continue

            variance = sum((t - mean_input) ** 2 for t in input_tokens) / len(input_tokens)
            std_dev = math.sqrt(variance)
            cv = std_dev / mean_input

            # Consistency score: 1 - CV (clamped to 0-1)
            consistency_score = max(0.0, min(1.0, 1.0 - cv))

            if cv > 0.7:
                continue  # too variable for caching to help

            # Estimated cacheable ratio derived from consistency
            cacheable_ratio = Decimal(str(round(consistency_score, 2)))

            # Get cached token discount for this provider
            discount_rate = CACHED_TOKEN_DISCOUNTS.get(provider)
            if discount_rate is None:
                continue

            # Calculate savings
            total_input_tokens = sum(input_tokens)
            cacheable_tokens = int(float(cacheable_ratio) * total_input_tokens)

            # Get input price for this specific model
            model_price = pricing.get_price(model_name)
            if model_price is None:
                continue

            avg_input_price = model_price["input_per_million"]

            # Current cost of cacheable tokens
            cacheable_cost = (
                avg_input_price * Decimal(str(cacheable_tokens)) / Decimal("1000000")
            )
            # With caching: cacheable tokens cost discount_rate of normal price
            cached_cost = cacheable_cost * discount_rate
            savings_total = cacheable_cost - cached_cost

            # Scale to monthly
            date_range = (
                max(r.record_date for r in provider_records)
                - min(r.record_date for r in provider_records)
            ).days or 1
            monthly_factor = Decimal("30") / Decimal(str(date_range))
            monthly_savings = (savings_total * monthly_factor).quantize(Decimal("0.01"))

            if monthly_savings <= Decimal("0.50"):
                continue

            cached_price = (avg_input_price * discount_rate).quantize(Decimal("0.01"))
            uncached_price = avg_input_price.quantize(Decimal("0.01"))
            pct_cacheable = int(float(cacheable_ratio) * 100)

            confidence = "high" if cv < 0.3 else "medium"

            recs.append(
                Recommendation(
                    rec_type="prompt_caching",
                    description=(
                        f"Your {provider.title()} {model_name} calls have a consistency "
                        f"score of {consistency_score:.2f} (scale: 0=unique, 1=identical). "
                        f"~{pct_cacheable}% of input tokens appear to be repeated content. "
                        f"Enabling prompt caching could save ~${monthly_savings}/month."
                    ),
                    estimated_monthly_savings=monthly_savings,
                    confidence=confidence,
                    action_required=(
                        f"Enable prompt caching for {provider.title()} {model_name} calls. "
                        f"Structure prompts with static system content first."
                    ),
                    category=SpendCategory.AI_INFERENCE,
                    source_module=self.name,
                    methodology=(
                        f"Input token CV = {cv:.3f} (std_dev={std_dev:.0f}, mean={mean_input:.0f}) "
                        f"across {len(input_tokens):,} {model_name} requests. "
                        f"Consistency score: {consistency_score:.2f} → ~{pct_cacheable}% cacheable. "
                        f"At {provider.title()} cached token pricing "
                        f"(${cached_price}/M tokens vs ${uncached_price}/M uncached), "
                        f"savings on {cacheable_tokens:,} cacheable tokens/period = "
                        f"${monthly_savings}/mo. "
                        f"Discount rate: {float(discount_rate)*100:.0f}% of input price "
                        f"(published {provider.title()} rate)."
                    ),
                )
            )

        return recs


path: engine/intelligence/optimization/model_routing.py
content: """Model routing optimization — identifies misrouted workloads using data-derived thresholds."""
import logging
from collections import defaultdict
from decimal import Decimal

import statistics

from engine.intelligence.optimization.base_module import OptimizationModule
from engine.intelligence.pricing_fetcher import PricingFetcher
from engine.models import (
    FinancialRecord,
    FinancialSummary,
    Recommendation,
    SpendCategory,
)

logger = logging.getLogger(__name__)


class ModelRoutingModule(OptimizationModule):
    """Identifies requests on expensive models that could use cheaper alternatives.

    Uses DATA-DERIVED thresholds: calculates percentiles from the user's own
    request distribution rather than hardcoded token counts.
    """

    name = "model_routing"
    description = "Detect expensive model usage on simple tasks"

    def analyse(
        self,
        records: list[FinancialRecord],
        summary: FinancialSummary,
        pricing: PricingFetcher,
    ) -> list[Recommendation]:
        recs: list[Recommendation] = []

        # Collect AI inference records with token data, grouped by model
        by_model: dict[str, list[FinancialRecord]] = defaultdict(list)
        all_token_counts: list[int] = []

        for r in records:
            if (
                r.category == SpendCategory.AI_INFERENCE
                and r.model
                and r.tokens_input is not None
                and r.tokens_output is not None
            ):
                by_model[r.model].append(r)
                all_token_counts.append(r.tokens_input + r.tokens_output)

        if len(all_token_counts) < 10:
            return recs

        # Data-derived threshold: 25th percentile of ALL user requests
        all_token_counts.sort()
        p25_index = max(0, len(all_token_counts) // 4 - 1)
        p25_threshold = all_token_counts[p25_index]
        median = statistics.median(all_token_counts)

        for model_name, model_records in by_model.items():
            model_price = pricing.get_price(model_name)
            if model_price is None:
                continue

            tier = pricing.get_tier(model_name)
            if tier == "efficient":
                continue  # already on a cheap model

            # Find records below the 25th percentile (candidate for cheaper model)
            short_records = [
                r for r in model_records
                if (r.tokens_input or 0) + (r.tokens_output or 0) < p25_threshold
            ]

            if not short_records or len(short_records) < 5:
                continue

            short_ratio = len(short_records) / len(model_records)
            if short_ratio < 0.10:
                continue

            # Calculate current cost of short records
            short_cost = sum(abs(r.amount) for r in short_records)

            # Scale to monthly
            date_range = (
                max(r.record_date for r in model_records)
                - min(r.record_date for r in model_records)
            ).days or 1
            monthly_factor = Decimal("30") / Decimal(str(date_range))
            short_cost_monthly = short_cost * monthly_factor

            # Calculate cost on comparable cheaper models
            comparables = pricing.get_comparable_models(model_name)
            if not comparables:
                continue

            # Pick the cheapest comparable
            cheapest = comparables[0]

            # Calculate exact savings using actual token volumes
            total_input = sum(r.tokens_input or 0 for r in short_records)
            total_output = sum(r.tokens_output or 0 for r in short_records)

            current_cost_exact = (
                model_price["input_per_million"] * Decimal(str(total_input)) / Decimal("1000000")
                + model_price["output_per_million"] * Decimal(str(total_output)) / Decimal("1000000")
            ) * monthly_factor

            alt_cost_exact = (
                cheapest["input_per_million"] * Decimal(str(total_input)) / Decimal("1000000")
                + cheapest["output_per_million"] * Decimal(str(total_output)) / Decimal("1000000")
            ) * monthly_factor

            savings = (current_cost_exact - alt_cost_exact).quantize(Decimal("0.01"))
            if savings <= Decimal("0.10"):
                continue

            recs.append(
                Recommendation(
                    rec_type="model_routing",
                    description=(
                        f"{len(short_records)} of {len(model_records)} {model_name} requests "
                        f"({short_ratio:.0%}) had token count below your 25th percentile "
                        f"({p25_threshold:,} tokens). These simple tasks could use "
                        f"{cheapest['model']} instead."
                    ),
                    estimated_monthly_savings=savings,
                    confidence="high" if short_ratio > 0.3 else "medium",
                    action_required=(
                        f"Route requests with <{p25_threshold:,} tokens from "
                        f"{model_name} to {cheapest['model']}"
                    ),
                    category=SpendCategory.AI_INFERENCE,
                    source_module=self.name,
                    methodology=(
                        f"25th percentile of your {len(all_token_counts):,} requests = "
                        f"{p25_threshold:,} tokens (median: {int(median):,}). "
                        f"{len(short_records)} {model_name} requests below this threshold "
                        f"cost ${current_cost_exact:.2f}/mo at "
                        f"${model_price['input_per_million']}/M input. "
                        f"Same volume on {cheapest['model']} at "
                        f"${cheapest['input_per_million']}/M input = "
                        f"${alt_cost_exact:.2f}/mo. "
                        f"Prices from {pricing.source}."
                    ),
                )
            )

        return recs


path: engine/intelligence/optimization/price_comparison.py
content: """Price comparison module — compares actual workload costs across alternative models."""
import logging
from collections import defaultdict
from decimal import Decimal

from engine.intelligence.optimization.base_module import OptimizationModule
from engine.intelligence.pricing_fetcher import PricingFetcher
from engine.models import (
    FinancialRecord,
    FinancialSummary,
    Recommendation,
    SpendCategory,
)

logger = logging.getLogger(__name__)


class PriceComparisonModule(OptimizationModule):
    """Compares what the user's actual workload would cost on alternative models.

    For each model the user uses:
    - Get their ACTUAL usage (tokens in, tokens out)
    - Fetch CURRENT pricing for that model AND comparable models
    - Calculate exact cost on each alternative
    """

    name = "price_comparison"
    description = "Compare actual workload costs across alternative models"

    def analyse(
        self,
        records: list[FinancialRecord],
        summary: FinancialSummary,
        pricing: PricingFetcher,
    ) -> list[Recommendation]:
        recs: list[Recommendation] = []

        # Group AI records by model
        by_model: dict[str, list[FinancialRecord]] = defaultdict(list)
        for r in records:
            if (
                r.category == SpendCategory.AI_INFERENCE
                and r.model
                and r.tokens_input is not None
                and r.tokens_output is not None
            ):
                by_model[r.model].append(r)

        for model_name, model_records in by_model.items():
            if len(model_records) < 10:
                continue

            model_price = pricing.get_price(model_name)
            if model_price is None:
                continue

            # Actual usage totals
            total_input = sum(r.tokens_input or 0 for r in model_records)
            total_output = sum(r.tokens_output or 0 for r in model_records)
            input_millions = Decimal(str(total_input)) / Decimal("1000000")
            output_millions = Decimal(str(total_output)) / Decimal("1000000")

            # Current cost (calculated from pricing, not from record amounts,
            # to ensure apples-to-apples comparison)
            current_cost = (
                model_price["input_per_million"] * input_millions
                + model_price["output_per_million"] * output_millions
            )

            # Scale to monthly
            date_range = (
                max(r.record_date for r in model_records)
                - min(r.record_date for r in model_records)
            ).days or 1
            monthly_factor = Decimal("30") / Decimal(str(date_range))
            monthly_current = current_cost * monthly_factor

            # Get comparable models (same tier + one tier below)
            comparables = pricing.get_comparable_models(model_name)
            if not comparables:
                continue

            # Calculate cost on each alternative
            best_alt = None
            best_savings = Decimal("0")
            best_alt_cost = Decimal("0")

            for alt in comparables:
                alt_cost = (
                    alt["input_per_million"] * input_millions
                    + alt["output_per_million"] * output_millions
                ) * monthly_factor

                savings = monthly_current - alt_cost
                if savings > best_savings:
                    best_savings = savings
                    best_alt = alt
                    best_alt_cost = alt_cost

            if best_alt is None or best_savings <= Decimal("1.00"):
                continue

            monthly_savings = best_savings.quantize(Decimal("0.01"))
            pct_reduction = (
                float(best_savings / monthly_current * 100)
                if monthly_current > 0
                else 0
            )

            recs.append(
                Recommendation(
                    rec_type="price_comparison",
                    description=(
                        f"Your {model_name} workload: "
                        f"{float(input_millions * monthly_factor):.1f}M input + "
                        f"{float(output_millions * monthly_factor):.1f}M output tokens/mo "
                        f"= ${monthly_current:.2f}. "
                        f"Same workload on {best_alt['model']}: "
                        f"${best_alt_cost:.2f} "
                        f"(savings: ${monthly_savings}/mo, {pct_reduction:.0f}% reduction)."
                    ),
                    estimated_monthly_savings=monthly_savings,
                    confidence="medium",
                    action_required=(
                        f"Evaluate {best_alt['model']} ({best_alt['tier']} tier) "
                        f"as replacement for {model_name}. "
                        f"Test on a subset of workload to verify quality."
                    ),
                    category=SpendCategory.AI_INFERENCE,
                    source_module=self.name,
                    methodology=(
                        f"Actual usage over {date_range} days: "
                        f"{total_input:,} input + {total_output:,} output tokens "
                        f"on {model_name}. "
                        f"Current pricing: ${model_price['input_per_million']}/M in, "
                        f"${model_price['output_per_million']}/M out "
                        f"= ${monthly_current:.2f}/mo. "
                        f"Alternative {best_alt['model']}: "
                        f"${best_alt['input_per_million']}/M in, "
                        f"${best_alt['output_per_million']}/M out "
                        f"= ${best_alt_cost:.2f}/mo. "
                        f"Prices fetched {pricing.last_updated:%Y-%m-%d} "
                        f"from {pricing.source}."
                    ),
                )
            )

        return recs


path: engine/intelligence/recommender.py
content: """Cross-bucket recommendation engine for token economy optimisation.

Uses a modular system: each OptimizationModule analyses records independently
and produces recommendations with specific dollar savings and methodology.
Real-time pricing is fetched via PricingFetcher (cached daily).
"""
import logging
from collections import defaultdict
from datetime import date, timedelta
from decimal import Decimal

from engine.intelligence.pricing_fetcher import PricingFetcher
from engine.intelligence.optimization import ALL_MODULES
from engine.models import (
    FinancialRecord,
    FinancialSummary,
    Recommendation,
    SpendCategory,
)

logger = logging.getLogger(__name__)

_INACTIVE_DAYS = 30
_MAX_RECOMMENDATIONS = 10


def generate_recommendations(
    records: list[FinancialRecord],
    as_of: date | None = None,
    summary: FinancialSummary | None = None,
) -> list[Recommendation]:
    """Analyse records across all buckets and generate optimisation recommendations.

    Runs all optimization modules plus legacy checks. Each recommendation
    includes its source module, estimated savings, and calculation methodology.

    Args:
        records: All financial records across all connectors.
        as_of: Reference date for calculations. Defaults to today.
        summary: Pre-computed summary. Built from records if not provided.

    Returns:
        Top recommendations sorted by estimated savings (highest first).
    """
    if not records:
        return []

    if as_of is None:
        as_of = date.today()

    # Build summary if not provided
    if summary is None:
        summary = _build_quick_summary(records, as_of)

    # Initialize pricing fetcher (uses cache if available)
    pricing = PricingFetcher()
    pricing.load()

    recommendations: list[Recommendation] = []

    # Run each optimization module
    for module_cls in ALL_MODULES:
        module = module_cls()
        try:
            module_recs = module.analyse(records, summary, pricing)
            recommendations.extend(module_recs)
            if module_recs:
                logger.info(
                    "Module %s produced %d recommendations",
                    module.name, len(module_recs),
                )
        except Exception as exc:
            logger.warning("Module %s failed: %s", module.name, exc)

    # Legacy check: unused subscriptions (no pricing needed)
    recommendations.extend(_check_unused_subscriptions(records, as_of))

    # Sort by estimated savings descending (None last)
    recommendations.sort(
        key=lambda r: r.estimated_monthly_savings or Decimal("0"),
        reverse=True,
    )

    return recommendations[:_MAX_RECOMMENDATIONS]


def _build_quick_summary(
    records: list[FinancialRecord], as_of: date
) -> FinancialSummary:
    """Build a minimal FinancialSummary from records for module use."""
    from datetime import datetime

    thirty_days_ago = as_of - timedelta(days=30)
    recent = [r for r in records if r.record_date >= thirty_days_ago]

    spend_by_cat: dict[str, Decimal] = defaultdict(Decimal)
    spend_by_prov: dict[str, Decimal] = defaultdict(Decimal)
    revenue = Decimal("0")
    crypto = Decimal("0")

    for r in recent:
        if r.category == SpendCategory.REVENUE:
            revenue += r.amount
        elif r.category == SpendCategory.CRYPTO_HOLDINGS:
            crypto += r.amount
        else:
            spend_by_cat[r.category.value] += abs(r.amount)
            spend_by_prov[r.provider.value] += abs(r.amount)

    dates = [r.record_date for r in records]
    coverage = (max(dates) - min(dates)).days + 1 if dates else 0

    return FinancialSummary(
        last_updated=datetime.now(),
        total_monthly_spend=sum(spend_by_cat.values()),
        spend_by_category=dict(spend_by_cat),
        spend_by_provider=dict(spend_by_prov),
        revenue_monthly=revenue if revenue > 0 else None,
        crypto_holdings_usd=crypto if crypto > 0 else None,
        data_coverage_days=coverage,
        record_count=len(records),
    )


def _check_unused_subscriptions(
    records: list[FinancialRecord], as_of: date
) -> list[Recommendation]:
    """Flag SaaS subscriptions with no activity in 30+ days."""
    recs: list[Recommendation] = []

    saas_records: dict[str, list[FinancialRecord]] = defaultdict(list)
    for r in records:
        if r.category == SpendCategory.SAAS_SUBSCRIPTION:
            key = r.subcategory or r.source
            saas_records[key].append(r)

    cutoff = as_of - timedelta(days=_INACTIVE_DAYS)

    for service_name, service_records in saas_records.items():
        last_record = max(r.record_date for r in service_records)
        if last_record >= cutoff:
            continue

        days_inactive = (as_of - last_record).days
        total_cost = sum(abs(r.amount) for r in service_records)
        date_span = (
            max(r.record_date for r in service_records)
            - min(r.record_date for r in service_records)
        ).days or 1
        monthly_cost = (
            total_cost * Decimal("30") / Decimal(str(date_span))
        ).quantize(Decimal("0.01"))

        recs.append(
            Recommendation(
                rec_type="unused_subscription",
                description=(
                    f"No activity from '{service_name}' in {days_inactive} days. "
                    f"Estimated ~${monthly_cost}/month"
                ),
                estimated_monthly_savings=monthly_cost,
                confidence="medium",
                action_required=(
                    f"Review whether '{service_name}' subscription is still needed. "
                    f"Cancel or downgrade if unused."
                ),
                category=SpendCategory.SAAS_SUBSCRIPTION,
                source_module="unused_subscriptions",
                methodology=(
                    f"Last activity: {last_record.isoformat()} ({days_inactive} days ago). "
                    f"Monthly cost estimated from ${total_cost:.2f} over "
                    f"{date_span} days of data."
                ),
            )
        )

    return recs


path: pyproject.toml
content: [project]
name = "token-economy-intel"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
    "anthropic>=0.86.0",
    "httpx>=0.28.1",
    "keyring>=25.0.0",
    "mcp>=1.26.0",
    "pydantic>=2.12.5",
    "pytest>=9.0.2",
    "pytest-asyncio>=1.0.0",
    "python-dotenv>=1.2.2",
]

[tool.pytest.ini_options]
asyncio_mode = "auto"


path: scripts/analytics.py
content: """Send an anonymised analytics event. Used by SKILL.md agent instructions."""
import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parents[2]))

from engine.analytics import EventTracker


def main() -> None:
    if len(sys.argv) < 3:
        print("Usage: analytics.py <event_type> <event_category> [--metadata '{...}']")
        print("Example: analytics.py query intelligence --metadata '{\"intent\": \"category\"}'")
        sys.exit(1)

    event_type = sys.argv[1]
    event_category = sys.argv[2]

    metadata: dict = {}
    if "--metadata" in sys.argv:
        idx = sys.argv.index("--metadata")
        if idx + 1 < len(sys.argv):
            try:
                metadata = json.loads(sys.argv[idx + 1])
            except json.JSONDecodeError:
                print("Warning: invalid metadata JSON, ignoring.")

    tracker = EventTracker()
    tracker.track(event_type, event_category, "openclaw", metadata)
    print(f"Event logged: {event_type} ({event_category})")


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)