Skill: daveglaser0823/ach-volume-estimator (version 1.0.0)

--- name: ach-volume-estimator description: Estimate Dwolla's end-of-month ACH transaction volume from daily KPI emails. Use when processing ACH KPI emails, when Dave asks about monthly volume projections, or when the daily ACH cron needs a monthly estimate. Extracts transactions and business days from email data, calculates per-BD rate, and projects monthly total using US Federal Reserve bank holiday calendar. --- # ACH Monthly Volume Estimator ## Workflow 1. **Fetch the ACH KPI email:** ```bash gog gmail messages search "subject:ACH KPIs" --account [REDACTED_SECRET] --max 1 --json ``` 2. **Download the PDF attachment** (contains all charts and data): ```bash # Get message details with attachment IDs gog gmail get <MESSAGE_ID> --account [REDACTED_SECRET] --json # Find the attachment named "ACH KPIs.pdf" (mimeType: application/pdf) # Download it: gog gmail attachment <MESSAGE_ID> <ATTACHMENT_ID> --account [REDACTED_SECRET] --out ~/clawd/work/ach-kpis-latest.pdf ``` 3. **Extract data from the PDF.** The PDF contains Tableau dashboard exports with: - ACH Transactions chart: MoM bar chart with monthly totals, per-BD averages, 5/20/60-day comparisons - Client Growth charts (10K and 5K thresholds) - Top 60 Clients tables (60-day and 20-day comparisons) Key numbers to extract: - Current month transaction total (partial, from MoM bar chart) - Business days elapsed (derive from date range in "Prev. 20 Days" row or count from month start to report date) - YTD avg transactions per business day - SPLY monthly total for comparison 4. **Run the estimator:** ```bash python3 ~/clawd/skills/ach-volume-estimator/scripts/estimate.py \ --transactions <MTD_TOTAL> --bds-elapsed <BDS_SO_FAR> [--month YYYY-MM] ``` Or if you already have per-BD rate: ```bash python3 ~/clawd/skills/ach-volume-estimator/scripts/estimate.py \ --per-bd <RATE> [--month YYYY-MM] ``` Add `--json` for structured output. 5. **Compare to benchmarks:** - Jan 2026 actual: 6.12M (20 BDs, 305K/BD) - Dec 2025 actual: 6.41M (22 BDs, 291K/BD) - SPLY from the PDF MoM chart 6. **Generate the visual dashboard:** ```bash ~/clawd/scripts/ach-dashboard-gen ``` This reads `~/clawd/work/ach-reports/latest-ach-data.json` (or falls back to the latest markdown report) and writes `~/clawd/work/ach-reports/dashboard.html`. View at: http://192.168.1.60:3013/html/work/ach-reports/dashboard.html TV mode: http://192.168.1.60:3013/html/work/ach-reports/dashboard.html?tv=1 7. **Deliver:** Include estimate in the ACH KPI summary sent to Dave. Include the dashboard URL. ## Output Format One-liner: `2026-02 estimate: 5.81M txns | (19 BDs, 12 elapsed, 7 remaining) | 306,000/BD | (3,672,000 so far)` ## Business Day Calendar Script uses US Federal Reserve bank holiday calendar (2025-2027 hardcoded). Excludes weekends and all Fed holidays (New Year's, MLK, Presidents' Day, Memorial Day, Juneteenth, Independence Day, Labor Day, Columbus Day, Veterans Day, Thanksgiving, Christmas). ## Integration Wire into the existing "Daily ACH KPIs summary to Dave" cron (9:30 AM ET weekdays). After extracting email data, run the estimator and append the monthly projection to Dave's summary.

path: scripts/estimate.py
content: #!/usr/bin/env python3
"""
ACH Volume Forecaster v3.1

Sophisticated forecasting with:
- Seasonal decomposition (monthly indices from historical data)
- Linear trend extraction (growth trajectory)
- Weighted recency bias (recent months matter more)
- Confidence intervals (80% band)
- Business day normalization using Fed holiday calendar
- Current month run-rate blending with seasonal/trend forecast
- Payroll spillover detection and BD1 adjustment

The more historical data in MONTHLY_ACTUALS, the better the forecasts.
Add new months as they close.

Usage:
    estimate.py --transactions N --bds-elapsed N [--month YYYY-MM] [--json]
    estimate.py --per-bd N [--month YYYY-MM] [--json]
    estimate.py --forecast-only [--month YYYY-MM] [--json]   # no live data, pure model
"""

import argparse
import json
import math
import sys
from datetime import date, timedelta
from collections import defaultdict

# ---------------------------------------------------------------------------
# US Federal Reserve bank holidays (observed dates 2023-2027)
# Source: https://www.federalreserve.gov/aboutthefed/k8.htm
# ---------------------------------------------------------------------------
FED_HOLIDAYS = {
    # 2023
    date(2023, 1, 2),   # New Year's (observed)
    date(2023, 1, 16),  # MLK Jr Day
    date(2023, 2, 20),  # Presidents' Day
    date(2023, 5, 29),  # Memorial Day
    date(2023, 6, 19),  # Juneteenth
    date(2023, 7, 4),   # Independence Day
    date(2023, 9, 4),   # Labor Day
    date(2023, 10, 9),  # Columbus Day
    date(2023, 11, 10), # Veterans Day (observed)
    date(2023, 11, 23), # Thanksgiving
    date(2023, 12, 25), # Christmas

    # 2024
    date(2024, 1, 1),   # New Year's Day
    date(2024, 1, 15),  # MLK Jr Day
    date(2024, 2, 19),  # Presidents' Day
    date(2024, 5, 27),  # Memorial Day
    date(2024, 6, 19),  # Juneteenth
    date(2024, 7, 4),   # Independence Day
    date(2024, 9, 2),   # Labor Day
    date(2024, 10, 14), # Columbus Day
    date(2024, 11, 11), # Veterans Day
    date(2024, 11, 28), # Thanksgiving
    date(2024, 12, 25), # Christmas

    # 2025
    date(2025, 1, 1),   # New Year's Day
    date(2025, 1, 20),  # MLK Jr Day
    date(2025, 2, 17),  # Presidents' Day
    date(2025, 5, 26),  # Memorial Day
    date(2025, 6, 19),  # Juneteenth
    date(2025, 7, 4),   # Independence Day
    date(2025, 9, 1),   # Labor Day
    date(2025, 10, 13), # Columbus Day
    date(2025, 11, 11), # Veterans Day
    date(2025, 11, 27), # Thanksgiving
    date(2025, 12, 25), # Christmas

    # 2026
    date(2026, 1, 1),   # New Year's Day
    date(2026, 1, 19),  # MLK Jr Day
    date(2026, 2, 16),  # Presidents' Day
    date(2026, 5, 25),  # Memorial Day
    date(2026, 6, 19),  # Juneteenth
    date(2026, 7, 3),   # Independence Day (observed)
    date(2026, 9, 7),   # Labor Day
    date(2026, 10, 12), # Columbus Day
    date(2026, 11, 11), # Veterans Day
    date(2026, 11, 26), # Thanksgiving
    date(2026, 12, 25), # Christmas

    # 2027
    date(2027, 1, 1),   # New Year's Day
    date(2027, 1, 18),  # MLK Jr Day
    date(2027, 2, 15),  # Presidents' Day
    date(2027, 5, 31),  # Memorial Day
    date(2027, 6, 18),  # Juneteenth (observed)
    date(2027, 7, 5),   # Independence Day (observed)
    date(2027, 9, 6),   # Labor Day
    date(2027, 10, 11), # Columbus Day
    date(2027, 11, 11), # Veterans Day
    date(2027, 11, 25), # Thanksgiving
    date(2027, 12, 24), # Christmas (observed)
}

# ---------------------------------------------------------------------------
# Historical monthly actuals (transactions)
# Source: ACH KPI emails. Add new months as they close.
# The more data here, the better the seasonal model.
# ---------------------------------------------------------------------------
MONTHLY_ACTUALS = {
    # 2025 actuals (Jan-Feb estimated; Mar-Dec confirmed from Tableau ACH KPIs charts)
    "2025-01": 5_633_673,    # confirmed from Tableau MoM/YoY 2026-03-19
    "2025-02": 5_220_612,    # confirmed from Tableau MoM/YoY 2026-03-19
    "2025-03": 5_700_654,    # confirmed from Tableau 2026-03-04
    "2025-04": 5_820_172,    # confirmed from Tableau 2026-03-04
    "2025-05": 5_779_929,    # confirmed from Tableau 2026-03-04
    "2025-06": 5_638_789,    # confirmed from Tableau 2026-03-04
    "2025-07": 5_482_190,    # confirmed from Tableau 2026-03-04
    "2025-08": 5_352_917,    # confirmed from Tableau 2026-03-04
    "2025-09": 5_858_619,    # confirmed from Tableau 2026-03-04
    "2025-10": 5_958_010,    # confirmed from Tableau 2026-03-04
    "2025-11": 5_283_677,    # confirmed from Tableau 2026-03-04
    "2025-12": 6_406_067,    # confirmed from Tableau 2026-03-04
    # 2026 actuals
    "2026-01": 6_121_127,    # confirmed from Tableau 2026-03-04
    "2026-02": 5_681_773,    # confirmed from Tableau 2026-03-04 (month closed)
}


# ---------------------------------------------------------------------------
# Calendar utilities
# ---------------------------------------------------------------------------

def business_days_in_month(year: int, month: int) -> int:
    """Count business days (weekdays minus Fed holidays) in a month."""
    first = date(year, month, 1)
    if month == 12:
        last = date(year + 1, 1, 1) - timedelta(days=1)
    else:
        last = date(year, month + 1, 1) - timedelta(days=1)
    count = 0
    d = first
    while d <= last:
        if d.weekday() < 5 and d not in FED_HOLIDAYS:
            count += 1
        d += timedelta(days=1)
    return count


def months_in_quarter(year: int, month: int):
    q_start = ((month - 1) // 3) * 3 + 1
    return [(year, q_start + i) for i in range(3)]


def months_in_year(year: int):
    return [(year, m) for m in range(1, 13)]


def detect_spillover(year: int, month: int) -> dict:
    """
    Check if payroll transactions from the prior month likely spilled into
    this month's first business day.

    Spillover occurs when the last 1-2 calendar days of the prior month were
    non-business days (weekends or Fed holidays), causing payroll that would
    have processed on those days to roll to this month's BD1.

    Returns dict with:
      detected (bool): True if spillover risk exists
      reason (str): Human-readable explanation
      non_business_days (int): Count of trailing non-BDs in prior month
    """
    # Prior month
    if month == 1:
        prior_year, prior_month = year - 1, 12
    else:
        prior_year, prior_month = year, month - 1

    # Last calendar day of prior month
    if prior_month == 12:
        last_day = date(prior_year + 1, 1, 1) - timedelta(days=1)
    else:
        last_day = date(prior_year, prior_month + 1, 1) - timedelta(days=1)

    # Count trailing non-business days (check last 2 calendar days)
    day_names = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
    trailing = []
    for offset in range(2):
        d = last_day - timedelta(days=offset)
        if d.weekday() >= 5 or d in FED_HOLIDAYS:
            trailing.append(f"{d} ({day_names[d.weekday()]})")
        else:
            break  # stop at first business day going backward

    if trailing:
        reason = (
            f"Prior month ({prior_year}-{prior_month:02d}) ended on non-business "
            f"day(s): {', '.join(trailing)}. Payroll likely spilled into BD1."
        )
        return {"detected": True, "reason": reason, "non_business_days": len(trailing)}

    return {
        "detected": False,
        "reason": f"No spillover: {prior_year}-{prior_month:02d} ended on a business day.",
        "non_business_days": 0,
    }


# ---------------------------------------------------------------------------
# Forecasting engine
# ---------------------------------------------------------------------------

def get_sorted_actuals():
    """Return actuals sorted chronologically as list of (key, year, month, value)."""
    items = []
    for key, val in MONTHLY_ACTUALS.items():
        y, m = int(key[:4]), int(key[5:7])
        items.append((key, y, m, val))
    items.sort(key=lambda x: (x[1], x[2]))
    return items


def compute_per_bd_series():
    """Normalize actuals to per-BD rates for apples-to-apples comparison."""
    series = {}
    for key, val in MONTHLY_ACTUALS.items():
        y, m = int(key[:4]), int(key[5:7])
        bds = business_days_in_month(y, m)
        series[key] = val / bds if bds > 0 else 0
    return series


def compute_seasonal_indices():
    """
    Calculate monthly seasonal indices from per-BD normalized data.
    Index > 1.0 means that month runs hotter than average.
    Uses ratio-to-moving-average when enough data, otherwise ratio-to-mean.
    """
    per_bd = compute_per_bd_series()
    if len(per_bd) < 6:
        return {m: 1.0 for m in range(1, 13)}

    # Group by calendar month
    by_month = defaultdict(list)
    for key, rate in per_bd.items():
        m = int(key[5:7])
        by_month[m].append(rate)

    # Overall mean per-BD rate
    all_rates = list(per_bd.values())
    overall_mean = sum(all_rates) / len(all_rates)

    if overall_mean == 0:
        return {m: 1.0 for m in range(1, 13)}

    # Seasonal index = avg per-BD for that month / overall avg per-BD
    indices = {}
    for m in range(1, 13):
        if by_month[m]:
            # Weight recent years more heavily (exponential decay)
            vals = by_month[m]
            if len(vals) == 1:
                month_avg = vals[0]
            else:
                # Most recent observation gets weight 1.0, prior gets 0.6, etc.
                weights = [0.6 ** i for i in range(len(vals) - 1, -1, -1)]
                month_avg = sum(v * w for v, w in zip(vals, weights)) / sum(weights)
            indices[m] = month_avg / overall_mean
        else:
            indices[m] = 1.0

    # Normalize indices so they average to 1.0
    avg_idx = sum(indices.values()) / 12
    if avg_idx > 0:
        indices = {m: v / avg_idx for m, v in indices.items()}

    return indices


def compute_trend():
    """
    Extract linear trend from per-BD series using weighted least squares.
    Recent months weighted more heavily.
    Returns (slope_per_month, intercept) in per-BD rate space.
    """
    per_bd = compute_per_bd_series()
    items = get_sorted_actuals()

    if len(items) < 3:
        return 0.0, sum(per_bd.values()) / max(len(per_bd), 1)

    # Assign sequential index (0, 1, 2, ...)
    n = len(items)
    xs = list(range(n))
    ys = [per_bd[item[0]] for item in items]

    # Exponential weights: most recent = 1.0, decaying backward
    decay = 0.85
    weights = [decay ** (n - 1 - i) for i in range(n)]

    # Weighted linear regression
    sw = sum(weights)
    sx = sum(w * x for w, x in zip(weights, xs))
    sy = sum(w * y for w, y in zip(weights, ys))
    sxx = sum(w * x * x for w, x in zip(weights, xs))
    sxy = sum(w * x * y for w, x, y in zip(weights, xs, ys))

    denom = sw * sxx - sx * sx
    if abs(denom) < 1e-10:
        return 0.0, sy / sw

    slope = (sw * sxy - sx * sy) / denom
    intercept = (sy - slope * sx) / sw

    return slope, intercept


def compute_forecast_error():
    """
    Calculate historical forecast error (MAPE) for confidence intervals.
    Uses leave-one-out on the last 6 months.
    """
    items = get_sorted_actuals()
    if len(items) < 8:
        return 0.05  # default 5% error if not enough data

    per_bd = compute_per_bd_series()
    errors = []

    # Test on last 6 months
    for i in range(max(len(items) - 6, 3), len(items)):
        # Build model from data before month i
        subset_keys = [items[j][0] for j in range(i)]
        subset_vals = [per_bd[k] for k in subset_keys]
        if len(subset_vals) < 3:
            continue

        # Simple trend extrapolation
        n = len(subset_vals)
        weights = [0.85 ** (n - 1 - j) for j in range(n)]
        sw = sum(weights)
        xs = list(range(n))
        sx = sum(w * x for w, x in zip(weights, xs))
        sy = sum(w * y for w, y in zip(weights, subset_vals))
        sxx = sum(w * x * x for w, x in zip(weights, xs))
        sxy = sum(w * x * y for w, x, y in zip(weights, xs, subset_vals))
        denom = sw * sxx - sx * sx
        if abs(denom) < 1e-10:
            predicted = sy / sw
        else:
            sl = (sw * sxy - sx * sy) / denom
            intc = (sy - sl * sx) / sw
            predicted = intc + sl * n  # extrapolate one step

        actual = per_bd[items[i][0]]
        if actual > 0:
            errors.append(abs(predicted - actual) / actual)

    if not errors:
        return 0.05
    return sum(errors) / len(errors)


def forecast_month_model(year: int, month: int) -> dict:
    """
    Forecast a future month using trend + seasonality.
    Returns per-BD rate forecast and confidence band.
    """
    items = get_sorted_actuals()
    if not items:
        return {"per_bd": 0, "confidence_low": 0, "confidence_high": 0, "method": "no_data"}

    seasonal = compute_seasonal_indices()
    slope, intercept = compute_trend()
    mape = compute_forecast_error()

    # How many months ahead from the last actual?
    last_y, last_m = items[-1][1], items[-1][2]
    last_idx = len(items) - 1
    months_ahead = (year - last_y) * 12 + (month - last_m)
    forecast_idx = last_idx + months_ahead

    # Base forecast: trend line
    trend_per_bd = intercept + slope * forecast_idx

    # Apply seasonal index
    seasonal_per_bd = trend_per_bd * seasonal.get(month, 1.0)

    # Confidence widens with distance
    distance_factor = 1.0 + 0.15 * max(0, months_ahead - 1)
    error_margin = mape * distance_factor * 1.28  # 80% CI ~ 1.28 * std

    bds = business_days_in_month(year, month)
    est = round(seasonal_per_bd * bds)
    low = round(seasonal_per_bd * (1 - error_margin) * bds)
    high = round(seasonal_per_bd * (1 + error_margin) * bds)

    return {
        "per_bd": round(seasonal_per_bd),
        "estimated_total": est,
        "confidence_low": low,
        "confidence_high": high,
        "seasonal_index": round(seasonal.get(month, 1.0), 3),
        "trend_per_bd": round(trend_per_bd),
        "method": "trend_seasonal",
    }


# ---------------------------------------------------------------------------
# Month estimation (combines live data with model)
# ---------------------------------------------------------------------------

def estimate_month(year: int, month: int, current_per_bd: float = None,
                   current_txns: int = None, current_bds_elapsed: int = None) -> dict:
    """Estimate a single month's volume."""
    key = f"{year}-{month:02d}"
    total_bds = business_days_in_month(year, month)

    # If we have actuals, use them
    if key in MONTHLY_ACTUALS:
        actual = MONTHLY_ACTUALS[key]
        return {
            "month": key,
            "total_business_days": total_bds,
            "estimated_total": actual,
            "estimated_total_M": round(actual / 1_000_000, 2),
            "method": "actual",
            "per_bd": round(actual / total_bds) if total_bds > 0 else 0,
        }

    # Current month with live data: blend run-rate with model
    today = date.today()
    if year == today.year and month == today.month and current_per_bd:
        model = forecast_month_model(year, month)

        # Blending: as more BDs elapse, trust run-rate more, model less
        # At 0 BDs elapsed, 100% model. At all BDs elapsed, 100% run-rate.
        if current_bds_elapsed and total_bds > 0:
            run_rate_weight = current_bds_elapsed / total_bds
        else:
            run_rate_weight = 0.5

        model_weight = 1.0 - run_rate_weight

        blended_per_bd = (current_per_bd * run_rate_weight +
                          model.get("per_bd", current_per_bd) * model_weight)

        est = round(blended_per_bd * total_bds)

        # Run-rate only estimate for comparison
        run_rate_est = round(current_per_bd * total_bds)

        return {
            "month": key,
            "total_business_days": total_bds,
            "bds_elapsed": current_bds_elapsed,
            "bds_remaining": total_bds - (current_bds_elapsed or 0),
            "transactions_so_far": current_txns,
            "estimated_total": est,
            "estimated_total_M": round(est / 1_000_000, 2),
            "run_rate_estimate_M": round(run_rate_est / 1_000_000, 2),
            "model_estimate_M": round(model.get("estimated_total", 0) / 1_000_000, 2),
            "blend_weights": {
                "run_rate": round(run_rate_weight, 2),
                "model": round(model_weight, 2),
            },
            "per_bd": round(blended_per_bd),
            "run_rate_per_bd": round(current_per_bd),
            "seasonal_index": model.get("seasonal_index"),
            "confidence_low_M": round(model.get("confidence_low", 0) / 1_000_000, 2),
            "confidence_high_M": round(model.get("confidence_high", 0) / 1_000_000, 2),
            "method": f"blended_{int(run_rate_weight*100)}pct_runrate",
        }

    # Future month: pure model forecast
    model = forecast_month_model(year, month)
    return {
        "month": key,
        "total_business_days": total_bds,
        "estimated_total": model["estimated_total"],
        "estimated_total_M": round(model["estimated_total"] / 1_000_000, 2),
        "per_bd": model["per_bd"],
        "seasonal_index": model.get("seasonal_index"),
        "confidence_low_M": round(model.get("confidence_low", 0) / 1_000_000, 2),
        "confidence_high_M": round(model.get("confidence_high", 0) / 1_000_000, 2),
        "method": model["method"],
    }


# ---------------------------------------------------------------------------
# Full forecast
# ---------------------------------------------------------------------------

def forecast(transactions: int = None, bds_elapsed: int = None,
             per_bd: float = None, month_str: str = None,
             forecast_only: bool = False) -> dict:
    """Full forecast: current month, quarter, year, with model diagnostics."""
    today = date.today()
    if month_str:
        parts = month_str.split("-")
        year, month = int(parts[0]), int(parts[1])
    else:
        year, month = today.year, today.month

    # Detect payroll spillover from prior month
    spillover = {"detected": False, "reason": "", "non_business_days": 0}
    if transactions is not None and bds_elapsed is not None:
        spillover = detect_spillover(year, month)

    # Calculate per-BD rate with optional spillover adjustment
    txn_per_bd = None
    effective_bds = bds_elapsed
    if per_bd is not None:
        txn_per_bd = per_bd
    elif transactions is not None and bds_elapsed is not None and bds_elapsed > 0:
        if spillover["detected"] and bds_elapsed < 5:
            # BD1 is inflated by prior-month spillover. Adjust by treating it as
            # ~1.5 effective BDs. The extra weight fades as more BDs elapse.
            # At bds_elapsed=1: +0.50, 2: +0.35, 3: +0.20, 4: +0.05, 5+: 0
            extra = max(0.0, 0.5 - (bds_elapsed - 1) * 0.15)
            effective_bds = bds_elapsed + extra
        txn_per_bd = transactions / effective_bds

    if not forecast_only and txn_per_bd is None:
        return {"error": "Provide --transactions + --bds-elapsed, or --per-bd, or --forecast-only"}

    # Current month
    current_month = estimate_month(year, month, txn_per_bd, transactions, bds_elapsed)

    # Annotate current_month with spillover info
    if spillover["detected"] and bds_elapsed is not None:
        current_month["spillover_detected"] = True
        current_month["spillover_note"] = spillover["reason"]
        if effective_bds != bds_elapsed:
            current_month["spillover_effective_bds"] = round(effective_bds, 2)
    else:
        current_month["spillover_detected"] = False

    # Quarter
    q_months = months_in_quarter(year, month)
    q_estimates = []
    q_total = 0
    q_low = 0
    q_high = 0
    for y, m in q_months:
        est = estimate_month(y, m, txn_per_bd, transactions, bds_elapsed)
        q_estimates.append(est)
        q_total += est["estimated_total"]
        q_low += est.get("confidence_low_M", est["estimated_total_M"]) * 1_000_000
        q_high += est.get("confidence_high_M", est["estimated_total_M"]) * 1_000_000

    quarter_num = ((month - 1) // 3) + 1

    # Year
    y_estimates = []
    y_total = 0
    y_low = 0
    y_high = 0
    for y, m in months_in_year(year):
        est = estimate_month(y, m, txn_per_bd, transactions, bds_elapsed)
        y_estimates.append(est)
        y_total += est["estimated_total"]
        y_low += est.get("confidence_low_M", est["estimated_total_M"]) * 1_000_000
        y_high += est.get("confidence_high_M", est["estimated_total_M"]) * 1_000_000

    # Prior year total
    prior_year_total = sum(
        MONTHLY_ACTUALS.get(f"{year-1}-{m:02d}", 0) for m in range(1, 13)
    )

    # Model diagnostics
    seasonal = compute_seasonal_indices()
    slope, intercept = compute_trend()
    mape = compute_forecast_error()

    result = {
        "current_month": current_month,
        "quarter": {
            "label": f"Q{quarter_num} {year}",
            "months": q_estimates,
            "total": q_total,
            "total_M": round(q_total / 1_000_000, 2),
            "confidence_low_M": round(q_low / 1_000_000, 2),
            "confidence_high_M": round(q_high / 1_000_000, 2),
        },
        "year": {
            "label": str(year),
            "months": y_estimates,
            "total": y_total,
            "total_M": round(y_total / 1_000_000, 2),
            "confidence_low_M": round(y_low / 1_000_000, 2),
            "confidence_high_M": round(y_high / 1_000_000, 2),
            "prior_year_total": prior_year_total,
            "prior_year_total_M": round(prior_year_total / 1_000_000, 2),
            "yoy_growth_pct": round((y_total / prior_year_total - 1) * 100, 1) if prior_year_total > 0 else None,
        },
        "model": {
            "data_points": len(MONTHLY_ACTUALS),
            "trend_slope_per_month": round(slope),
            "seasonal_indices": {f"{m:02d}": round(v, 3) for m, v in seasonal.items()},
            "mape_pct": round(mape * 100, 1),
            "confidence_level": "80%",
            "notes": [
                f"Model uses {len(MONTHLY_ACTUALS)} months of history.",
                "Seasonal indices > 1.0 = hotter than avg, < 1.0 = cooler.",
                f"Trend: +{round(slope):,}/BD per month." if slope > 0 else f"Trend: {round(slope):,}/BD per month.",
                f"Forecast error (MAPE): {round(mape*100, 1)}%. Confidence bands widen for months further out.",
                "Current month blends live run-rate with model (weight shifts to run-rate as more BDs elapse).",
            ],
        },
    }

    return result


# ---------------------------------------------------------------------------
# Output formatting
# ---------------------------------------------------------------------------

def format_summary(r: dict) -> str:
    """Multi-line executive summary."""
    if "error" in r:
        return f"Error: {r['error']}"

    cm = r["current_month"]
    q = r["quarter"]
    y = r["year"]
    model = r["model"]

    lines = []

    # Current month
    method_note = f" [{cm['method']}]" if cm.get("method") else ""
    lines.append(f"MONTH: {cm['month']} estimate: {cm['estimated_total_M']}M txns "
                 f"({cm['total_business_days']} BDs, {cm.get('bds_elapsed', '-')} elapsed, "
                 f"{cm.get('bds_remaining', '-')} remaining) | {cm['per_bd']:,}/BD{method_note}")
    if cm.get("transactions_so_far"):
        lines.append(f"  MTD: {cm['transactions_so_far']:,}")
    if cm.get("run_rate_estimate_M") and cm.get("model_estimate_M"):
        lines.append(f"  Run-rate only: {cm['run_rate_estimate_M']}M | Model only: {cm['model_estimate_M']}M "
                     f"| Blend: {cm.get('blend_weights', {}).get('run_rate', '?')}/"
                     f"{cm.get('blend_weights', {}).get('model', '?')} (run-rate/model)")
    if cm.get("confidence_low_M") and cm.get("confidence_high_M"):
        lines.append(f"  80% CI: {cm['confidence_low_M']}M - {cm['confidence_high_M']}M")
    if cm.get("seasonal_index"):
        idx = cm["seasonal_index"]
        label = "hotter" if idx > 1.02 else "cooler" if idx < 0.98 else "avg"
        lines.append(f"  Seasonal: {idx:.3f} ({label} than avg)")
    if cm.get("spillover_detected"):
        eff = cm.get("spillover_effective_bds")
        adj_note = f" (BD1 adjusted: effective BDs used = {eff})" if eff else ""
        lines.append(f"  ** SPILLOVER: {cm['spillover_note']}{adj_note}")

    lines.append("")

    # Quarter
    lines.append(f"QUARTER: {q['label']} forecast: {q['total_M']}M txns "
                 f"(80% CI: {q['confidence_low_M']}M - {q['confidence_high_M']}M)")
    for m in q["months"]:
        ci = ""
        if m.get("confidence_low_M") and m.get("confidence_high_M") and m["method"] != "actual":
            ci = f" [CI: {m['confidence_low_M']}-{m['confidence_high_M']}M]"
        si = f" SI:{m['seasonal_index']:.2f}" if m.get("seasonal_index") else ""
        lines.append(f"  {m['month']}: {m['estimated_total_M']}M "
                     f"({m['total_business_days']} BDs, {m.get('per_bd', 0):,}/BD) "
                     f"[{m['method']}]{si}{ci}")

    lines.append("")

    # Year
    yoy_str = f" | YoY: {'+' if y.get('yoy_growth_pct', 0) >= 0 else ''}{y['yoy_growth_pct']}%" if y.get("yoy_growth_pct") is not None else ""
    lines.append(f"YEAR: {y['label']} forecast: {y['total_M']}M txns{yoy_str}")
    lines.append(f"  80% CI: {y['confidence_low_M']}M - {y['confidence_high_M']}M")
    if y.get("prior_year_total_M") and y["prior_year_total_M"] > 0:
        lines.append(f"  vs {int(y['label'])-1} actual: {y['prior_year_total_M']}M")

    lines.append("")

    # Model diagnostics
    lines.append(f"MODEL: {model['data_points']} months history | "
                 f"MAPE: {model['mape_pct']}% | "
                 f"Trend: {'+' if model['trend_slope_per_month'] >= 0 else ''}"
                 f"{model['trend_slope_per_month']:,}/BD per month")

    # Seasonal highlights
    si = model["seasonal_indices"]
    hottest = max(si, key=si.get)
    coolest = min(si, key=si.get)
    lines.append(f"  Hottest month: {hottest} (SI: {si[hottest]}) | "
                 f"Coolest: {coolest} (SI: {si[coolest]})")

    return "\n".join(lines)


def main():
    parser = argparse.ArgumentParser(description="ACH Volume Forecaster v3.1")
    parser.add_argument("--transactions", type=int, help="Transactions so far this month")
    parser.add_argument("--bds-elapsed", type=int, help="Business days elapsed")
    parser.add_argument("--per-bd", type=float, help="Known transactions per business day")
    parser.add_argument("--month", type=str, help="Month context (YYYY-MM, default: current)")
    parser.add_argument("--forecast-only", action="store_true", help="Pure model forecast, no live data")
    parser.add_argument("--json", action="store_true", help="Output JSON")
    parser.add_argument("--with-revenue", action="store_true", help="Include revenue forecast")
    args = parser.parse_args()

    result = forecast(
        transactions=args.transactions,
        bds_elapsed=args.bds_elapsed,
        per_bd=args.per_bd,
        month_str=args.month,
        forecast_only=args.forecast_only,
    )

    if "error" in result:
        print(f"Error: {result['error']}", file=sys.stderr)
        sys.exit(1)

    # Revenue forecast integration
    if args.with_revenue:
        try:
            from revenue_forecast import revenue_forecast, format_revenue_report
            rev = revenue_forecast(
                ach_projected_month=result["current_month"]["estimated_total"],
                ach_projected_quarter=result["quarter"]["total"],
                ach_projected_year=result["year"]["total"],
                ach_mtd=args.transactions,
                month_str=args.month,
            )
            result["revenue"] = rev
        except Exception as e:
            result["revenue"] = {"error": str(e)}

    if args.json:
        print(json.dumps(result, indent=2))
    else:
        output = format_summary(result)
        if args.with_revenue and "revenue" in result and "error" not in result.get("revenue", {}):
            output += "\n" + format_revenue_report(result["revenue"])
        print(output)


if __name__ == "__main__":
    main()


path: scripts/monthly_calibration.py
content: #!/usr/bin/env python3
"""
Monthly Revenue Calibration Check (v1.0)

Runs on BD 5 of each month. Checks if a new closed month needs calibration.
If the latest Tableau Annual Revenue Estimator email has data for a month
not yet in the calibration file, flags it for manual update.

For now: reports calibration status and drift. Future: auto-extract from
Tableau subscription PNG attachment.
"""

import json
import sys
from datetime import date, datetime
from pathlib import Path

CALIBRATION_PATH = Path.home() / "clawd" / "work" / "bionics" / "revenue_calibration.json"


def check_calibration():
    """Check calibration freshness and report status."""
    if not CALIBRATION_PATH.exists():
        return {"status": "error", "message": "Calibration file not found"}

    with open(CALIBRATION_PATH) as f:
        cal = json.load(f)

    series = sorted(cal["calibration_data"], key=lambda x: x["month"])
    last_month = series[-1]["month"]
    last_cal_date = cal.get("last_calibration_date", "unknown")

    today = date.today()
    # What month SHOULD be calibrated? (previous month if we're past BD 5)
    if today.month == 1:
        expected_month = f"{today.year - 1}-12"
    else:
        expected_month = f"{today.year}-{today.month - 1:02d}"

    is_current = last_month >= expected_month
    
    # Calculate days since last calibration
    try:
        last_cal = datetime.strptime(last_cal_date, "%Y-%m-%d").date()
        days_stale = (today - last_cal).days
    except (ValueError, TypeError):
        days_stale = -1

    # Check for drift in recent months
    drift_warning = False
    if len(series) >= 5:
        # Simple check: is the rev/txn rate volatile?
        recent_rates = [s["txn_revenue"] / s["transactions"] for s in series[-3:] if s["transactions"] > 0]
        if recent_rates:
            avg_rate = sum(recent_rates) / len(recent_rates)
            max_deviation = max(abs(r - avg_rate) / avg_rate for r in recent_rates)
            drift_warning = max_deviation > 0.15  # 15% deviation from 3-month avg

    result = {
        "status": "current" if is_current else "stale",
        "last_calibrated_month": last_month,
        "expected_month": expected_month,
        "last_calibration_date": last_cal_date,
        "days_since_calibration": days_stale,
        "data_points": len(series),
        "drift_warning": drift_warning,
        "needs_update": not is_current,
    }

    if not is_current:
        result["action_needed"] = (
            f"Missing calibration for {expected_month}. "
            f"Need closed-month product revenue (txn + FBO split) "
            f"and total core transaction count."
        )

    return result


def main():
    result = check_calibration()
    
    if "--json" in sys.argv:
        print(json.dumps(result, indent=2))
    else:
        status = result["status"].upper()
        print(f"Revenue Calibration Status: {status}")
        print(f"  Last calibrated month: {result['last_calibrated_month']}")
        print(f"  Expected: {result['expected_month']}")
        print(f"  Days since calibration: {result['days_since_calibration']}")
        print(f"  Data points: {result['data_points']}")
        if result.get("drift_warning"):
            print("  WARNING: Rate volatility detected in recent months")
        if result.get("needs_update"):
            print(f"  ACTION: {result['action_needed']}")
        else:
            print("  OK: Calibration is current")


if __name__ == "__main__":
    main()


path: scripts/revenue_forecast.py
content: #!/usr/bin/env python3
"""
Revenue Forecaster v2.0

Extends the ACH volume forecasting pipeline with revenue projections.
Separates transaction revenue from FBO incentive income:
  - Transaction revenue: trends with volume (per-txn rate model)
  - FBO revenue: SOFR-linked, treated as flat monthly line (~$433K/mo)

Core formula:
  Txn Rate = Transaction Revenue / Total Core Transactions
  Projected Total Txns = Projected ACH Txns / ACH Share Ratio
  Projected Txn Revenue = Projected Total Txns * Trending Txn Rate
  Projected FBO Revenue = Flat monthly amount ($433K default)
  Projected Total Revenue = Txn Revenue + FBO Revenue

Usage:
  revenue_forecast.py --ach-projected N [--month YYYY-MM] [--json]
  revenue_forecast.py --standalone [--json]   # pure model, no live ACH data
"""

import argparse
import json
import math
import os
import sys
import urllib.request
from datetime import date, datetime, timedelta
from pathlib import Path

# ---------------------------------------------------------------------------
# Paths
# ---------------------------------------------------------------------------
SCRIPT_DIR = Path(__file__).parent
CALIBRATION_PATH = Path.home() / "clawd" / "work" / "bionics" / "revenue_calibration.json"

# ACH actuals from estimate.py (duplicated here for ACH share ratio calculation)
# These are ACH-only transaction counts from KPI emails.
ACH_MONTHLY_ACTUALS = {
    "2025-01": 5_633_673,
    "2025-02": 5_220_612,
    "2025-03": 5_700_654,
    "2025-04": 5_820_172,
    "2025-05": 5_779_929,
    "2025-06": 5_638_789,
    "2025-07": 5_482_190,
    "2025-08": 5_352_917,
    "2025-09": 5_858_619,
    "2025-10": 5_958_010,
    "2025-11": 5_283_677,
    "2025-12": 6_406_067,
    "2026-01": 6_121_127,
    "2026-02": 5_681_773,
}


# ---------------------------------------------------------------------------
# Calibration data loading
# ---------------------------------------------------------------------------

def load_calibration() -> dict:
    """Load calibration dataset from JSON file."""
    if not CALIBRATION_PATH.exists():
        print(f"ERROR: Calibration file not found: {CALIBRATION_PATH}", file=sys.stderr)
        sys.exit(1)
    with open(CALIBRATION_PATH) as f:
        return json.load(f)


def get_calibration_series(cal_data: dict) -> list[dict]:
    """Return sorted calibration data series with txn-only rates."""
    series = sorted(cal_data["calibration_data"], key=lambda x: x["month"])
    # Compute txn-only rev_per_txn (v2.0: excludes FBO)
    for entry in series:
        txn_rev = entry.get("txn_revenue", entry["product_revenue"])
        entry["txn_rev_per_txn"] = txn_rev / entry["transactions"] if entry["transactions"] > 0 else 0
        # Keep legacy blended rate for backward compat
        entry["rev_per_txn"] = entry["product_revenue"] / entry["transactions"] if entry["transactions"] > 0 else 0
    return series


def get_fbo_monthly(cal_data: dict) -> int:
    """Get flat monthly FBO amount from calibration config."""
    fbo_model = cal_data.get("fbo_model", {})
    return fbo_model.get("monthly_amount", 433000)


# ---------------------------------------------------------------------------
# ACH Share Ratio
# ---------------------------------------------------------------------------

def compute_ach_share_ratio(cal_data: dict) -> dict:
    """
    Compute ACH share ratio by cross-referencing ACH-only actuals
    against total core transactions from the calibration dataset.
    Returns trailing 3-month average from the most recent overlapping months.
    """
    series = get_calibration_series(cal_data)
    ratios = []

    for entry in series:
        month = entry["month"]
        total_core = entry["transactions"]
        ach_only = ACH_MONTHLY_ACTUALS.get(month)
        if ach_only and total_core > 0:
            ratio = ach_only / total_core
            ratios.append({"month": month, "ach": ach_only, "total_core": total_core, "ratio": ratio})

    if not ratios:
        # Fallback to Dave's estimate
        return {"ratio": 0.85, "method": "dave_estimate", "months_used": 0, "detail": []}

    # Use trailing 3 months for the average
    recent = ratios[-3:]
    avg_ratio = sum(r["ratio"] for r in recent) / len(recent)

    return {
        "ratio": round(avg_ratio, 4),
        "method": f"trailing_{len(recent)}_month",
        "months_used": len(recent),
        "detail": [{"month": r["month"], "ratio": round(r["ratio"], 4)} for r in recent],
        "all_overlapping": [{"month": r["month"], "ratio": round(r["ratio"], 4)} for r in ratios],
    }


# ---------------------------------------------------------------------------
# Trend models
# ---------------------------------------------------------------------------

def linear_regression(series: list[dict], weight_jan_2026: bool = True, rate_key: str = "txn_rev_per_txn") -> dict:
    """
    Fit weighted linear regression to per-txn rate series.
    v2.1: Adds outlier detection to downweight anomalous months.
    Jan 2026 gets reduced weight (measurement artifact).
    Returns slope, intercept, and projected rate function.
    """
    n = len(series)
    if n < 3:
        avg = sum(e[rate_key] for e in series) / n
        return {"slope": 0.0, "intercept": avg, "method": "insufficient_data"}

    xs = list(range(n))
    ys = [e[rate_key] for e in series]

    # Weights: exponential decay (recent = higher weight)
    decay = 0.90
    weights = [decay ** (n - 1 - i) for i in range(n)]

    # Downweight Jan 2026 if present (measurement artifact)
    if weight_jan_2026:
        for i, entry in enumerate(series):
            if entry["month"] == "2026-01":
                weights[i] *= 0.5

    # v2.1: Outlier detection - downweight months whose rate deviates
    # significantly from the rolling median. This prevents measurement
    # artifacts (e.g., different revenue sources) from distorting the trend.
    if n >= 6:
        import statistics
        median_rate = statistics.median(ys)
        mad = statistics.median([abs(y - median_rate) for y in ys])
        mad_scale = mad * 2.5 if mad > 0 else median_rate * 0.15
        outliers_flagged = []
        for i, y in enumerate(ys):
            deviation = abs(y - median_rate)
            if deviation > mad_scale:
                # Downweight proportionally to how far it deviates
                dampen = max(0.1, 1.0 - (deviation - mad_scale) / median_rate)
                weights[i] *= dampen
                outliers_flagged.append(series[i]["month"])

    sw = sum(weights)
    sx = sum(w * x for w, x in zip(weights, xs))
    sy = sum(w * y for w, y in zip(weights, ys))
    sxx = sum(w * x * x for w, x in zip(weights, xs))
    sxy = sum(w * x * y for w, x, y in zip(weights, xs, ys))

    denom = sw * sxx - sx * sx
    if abs(denom) < 1e-15:
        return {"slope": 0.0, "intercept": sy / sw, "method": "degenerate"}

    slope = (sw * sxy - sx * sy) / denom
    intercept = (sy - slope * sx) / sw

    # Backtest: calculate prediction errors for known months
    errors = []
    for i in range(max(n - 6, 0), n):
        predicted = intercept + slope * xs[i]
        actual = ys[i]
        if actual > 0:
            errors.append(abs(predicted - actual) / actual)

    mape = sum(errors) / len(errors) if errors else 0.0

    return {
        "slope": slope,
        "intercept": intercept,
        "n_points": n,
        "mape_pct": round(mape * 100, 2),
        "method": "linear_regression",
    }


def weighted_moving_average(series: list[dict], window: int = 6, rate_key: str = "txn_rev_per_txn") -> dict:
    """
    Weighted moving average on per-txn rate series.
    v2.0: Defaults to txn_rev_per_txn (FBO excluded).
    More recent months get higher weight.
    """
    n = len(series)
    if n < 3:
        avg = sum(e[rate_key] for e in series) / n
        return {"rate": avg, "method": "insufficient_data"}

    recent = series[-window:] if n >= window else series
    k = len(recent)
    weights = [(i + 1) for i in range(k)]  # 1, 2, 3, ...
    sw = sum(weights)

    wma_rate = sum(w * e[rate_key] for w, e in zip(weights, recent)) / sw

    # Backtest: predict each month using WMA of prior months
    errors = []
    for i in range(max(n - 6, window), n):
        window_data = series[max(0, i - window):i]
        if len(window_data) < 3:
            continue
        wk = len(window_data)
        wts = [(j + 1) for j in range(wk)]
        pred = sum(w * e[rate_key] for w, e in zip(wts, window_data)) / sum(wts)
        actual = series[i][rate_key]
        if actual > 0:
            errors.append(abs(pred - actual) / actual)

    mape = sum(errors) / len(errors) if errors else 0.0

    return {
        "rate": wma_rate,
        "window": k,
        "mape_pct": round(mape * 100, 2),
        "method": "weighted_moving_average",
    }


# ---------------------------------------------------------------------------
# Plan-anchored rate ceiling (v2.1)
# ---------------------------------------------------------------------------

# 2026 plan: $19.4M txn revenue on ~79M ACH (~88.5M total core @ 89.4% ACH share)
# Plan-implied txn rate = $19.4M / 88.5M = ~$0.2192/txn
# But plan was set conservatively. Use a ceiling at 130% of plan rate to allow
# upside while preventing runaway extrapolation.
PLAN_TXN_REVENUE_2026 = 19_400_000
PLAN_TOTAL_CORE_TXNS_2026 = 88_500_000  # approximate
PLAN_IMPLIED_RATE = PLAN_TXN_REVENUE_2026 / PLAN_TOTAL_CORE_TXNS_2026  # ~0.2192
RATE_CEILING_MULTIPLIER = 1.30  # allow 30% above plan-implied rate
RATE_CEILING = round(PLAN_IMPLIED_RATE * RATE_CEILING_MULTIPLIER, 4)  # ~0.285


def project_rate(series: list[dict], months_ahead: int, model: str = "auto",
                 apply_ceiling: bool = True) -> dict:
    """
    Project rev/txn rate for a future month.
    If model="auto", picks the method with lower backtest MAPE.
    v2.1: Applies plan-anchored rate ceiling to prevent runaway extrapolation.
    The ceiling blends the raw projected rate with the plan-implied rate
    when the projection exceeds RATE_CEILING.
    """
    lr = linear_regression(series)
    wma = weighted_moving_average(series)

    # Auto-select based on backtest accuracy
    if model == "auto":
        if lr["mape_pct"] <= wma["mape_pct"]:
            model = "linear_regression"
        else:
            model = "wma"

    n = len(series)
    ceiling_applied = False
    raw_rate = None

    if model == "linear_regression":
        rate = lr["intercept"] + lr["slope"] * (n - 1 + months_ahead)
    else:
        # WMA doesn't trend-project well, so add simple momentum
        rate = wma["rate"]

    raw_rate = round(rate, 4)

    # v2.1: Apply rate ceiling - soft cap that blends toward plan rate
    if apply_ceiling and rate > RATE_CEILING:
        # Blend: 70% ceiling, 30% raw rate (allows some upside signal)
        rate = RATE_CEILING * 0.7 + rate * 0.3
        ceiling_applied = True

    result = {
        "projected_rate": round(rate, 4),
        "method": model if model != "wma" else "weighted_moving_average",
        "lr_mape": lr["mape_pct"],
        "wma_mape": wma["mape_pct"],
    }
    if model == "linear_regression":
        result["lr_slope_per_month"] = round(lr["slope"], 5)
    if ceiling_applied:
        result["ceiling_applied"] = True
        result["raw_rate"] = raw_rate
        result["ceiling"] = RATE_CEILING
        result["plan_implied_rate"] = round(PLAN_IMPLIED_RATE, 4)

    return result


def flat_rate(series: list[dict], trailing: int = 3, rate_key: str = "txn_rev_per_txn") -> float:
    """Trailing N-month average rate (flat, no trend). v2.0: Uses txn-only rate."""
    recent = series[-trailing:]
    return sum(e[rate_key] for e in recent) / len(recent)


# ---------------------------------------------------------------------------
# SOFR
# ---------------------------------------------------------------------------

def fetch_sofr() -> dict:
    """
    Fetch current SOFR rate from FRED API (public, no auth needed).
    Returns rate, date, and direction vs prior month.
    """
    try:
        url = "https://fred.stlouisfed.org/graph/fredgraph.csv?id=SOFR&cosd=2026-01-01"
        req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
        with urllib.request.urlopen(req, timeout=3) as resp:
            lines = resp.read().decode().strip().split("\n")

        # Parse CSV: DATE,SOFR
        rates = []
        for line in lines[1:]:  # skip header
            parts = line.split(",")
            if len(parts) == 2 and parts[1] not in (".", ""):
                try:
                    rates.append({"date": parts[0], "rate": float(parts[1])})
                except ValueError:
                    continue

        if not rates:
            return {"available": False, "error": "No SOFR data found"}

        current = rates[-1]

        # Find rate from ~30 days ago for direction
        prior = None
        for r in reversed(rates):
            d = datetime.strptime(r["date"], "%Y-%m-%d").date()
            curr_d = datetime.strptime(current["date"], "%Y-%m-%d").date()
            if (curr_d - d).days >= 25:
                prior = r
                break

        direction = "FLAT"
        bps_change = 0
        if prior:
            bps_change = round((current["rate"] - prior["rate"]) * 100)
            if bps_change > 0:
                direction = "UP"
            elif bps_change < 0:
                direction = "DOWN"

        return {
            "available": True,
            "rate": current["rate"],
            "date": current["date"],
            "direction": direction,
            "bps_change": bps_change,
            "prior_rate": prior["rate"] if prior else None,
            "prior_date": prior["date"] if prior else None,
        }
    except Exception as e:
        return {"available": False, "error": str(e)}


# ---------------------------------------------------------------------------
# Drift detection
# ---------------------------------------------------------------------------

def check_drift(cal_data: dict, series: list[dict]) -> dict:
    """
    Compare actual closed-month revenue vs model prediction for the most
    recent 2 months. Flag if error > 10% for 2 consecutive months.
    """
    if len(series) < 5:
        return {"drift_detected": False, "reason": "insufficient_data"}

    consecutive_drift = 0
    drift_months = []

    for i in range(max(len(series) - 3, 2), len(series)):
        # Predict month i using data up to i-1
        prior_series = series[:i]
        lr = linear_regression(prior_series)
        predicted_rate = lr["intercept"] + lr["slope"] * i
        actual_rate = series[i]["rev_per_txn"]
        actual_rev = series[i]["product_revenue"]
        predicted_rev = series[i]["transactions"] * predicted_rate

        if actual_rev > 0:
            error_pct = abs(predicted_rev - actual_rev) / actual_rev * 100
        else:
            error_pct = 0

        if error_pct > 10:
            consecutive_drift += 1
            drift_months.append({
                "month": series[i]["month"],
                "predicted_rev": round(predicted_rev),
                "actual_rev": actual_rev,
                "error_pct": round(error_pct, 1),
            })
        else:
            consecutive_drift = 0
            drift_months = []

    return {
        "drift_detected": consecutive_drift >= 2,
        "consecutive_months": consecutive_drift,
        "drift_months": drift_months,
    }


# ---------------------------------------------------------------------------
# Dust analysis revenue extension
# ---------------------------------------------------------------------------

def dust_revenue_impact(volume_delta: int, blended_rate: float) -> dict:
    """
    When dust analysis flags client volume changes, multiply volume delta
    by system-wide blended rate to estimate revenue impact.
    """
    revenue_delta = volume_delta * blended_rate
    return {
        "volume_delta": volume_delta,
        "blended_rate": round(blended_rate, 4),
        "estimated_revenue_impact": round(revenue_delta),
        "estimated_revenue_impact_K": round(revenue_delta / 1000, 1),
    }


# ---------------------------------------------------------------------------
# Main forecast
# ---------------------------------------------------------------------------

def revenue_forecast(
    ach_projected_month: int = None,
    ach_projected_quarter: int = None,
    ach_projected_year: int = None,
    ach_mtd: int = None,
    month_str: str = None,
    dust_deltas: list = None,
) -> dict:
    """
    Produce revenue forecast from ACH volume projections.

    Args:
        ach_projected_month: Projected ACH txns for the current month
        ach_projected_quarter: Projected ACH txns for the quarter
        ach_projected_year: Projected ACH txns for the year
        ach_mtd: ACH transactions month-to-date
        month_str: Target month (YYYY-MM, default: current)
        dust_deltas: List of {"client": str, "volume_delta": int} for dust analysis
    """
    today = date.today()
    if month_str:
        parts = month_str.split("-")
        year, month = int(parts[0]), int(parts[1])
    else:
        year, month = today.year, today.month

    cal_data = load_calibration()
    series = get_calibration_series(cal_data)
    fbo_monthly = get_fbo_monthly(cal_data)

    # ACH share ratio
    ach_share = compute_ach_share_ratio(cal_data)
    ach_ratio = ach_share["ratio"]

    # Rate models (v2.0: txn-only rates, FBO excluded)
    lr = linear_regression(series)
    wma = weighted_moving_average(series)

    # Determine months ahead from last calibration point
    last_cal = series[-1]
    last_y, last_m = int(last_cal["month"][:4]), int(last_cal["month"][5:7])
    months_ahead = (year - last_y) * 12 + (month - last_m)
    if months_ahead < 0:
        months_ahead = 0

    # Trending rate (auto-select best model)
    rate_info = project_rate(series, months_ahead)
    trending_rate = rate_info["projected_rate"]

    # Flat rate (trailing 3-month average)
    flat_r = flat_rate(series, trailing=3)

    # SOFR
    sofr = fetch_sofr()

    # Drift check
    drift = check_drift(cal_data, series)

    # Base volume scenarios (from ACH projections)
    base_vol_month = ach_projected_month or 0
    base_vol_quarter = ach_projected_quarter or 0
    base_vol_year = ach_projected_year or 0

    # Conservative volume: ~5% lower (73-75M annual vs 79M base)
    conservative_factor = 0.95

    # Scale ACH to total core transactions
    def ach_to_total(ach_txns):
        return int(ach_txns / ach_ratio) if ach_ratio > 0 else ach_txns

    # Revenue calculation helper
    def calc_revenue(total_txns, rate):
        return total_txns * rate

    # FBO annual amount
    fbo_annual = fbo_monthly * 12

    # MTD revenue estimate
    mtd_revenue = None
    if ach_mtd and ach_mtd > 0:
        mtd_total = ach_to_total(ach_mtd)
        # Use current month's actual txn-only rate (latest available) for MTD
        current_txn_rate = series[-1]["txn_rev_per_txn"]
        # Prorate FBO for days elapsed in month
        days_in_month = 30  # approximation
        days_elapsed = min(today.day, days_in_month)
        mtd_fbo = int(fbo_monthly * days_elapsed / days_in_month)
        mtd_txn_rev = round(mtd_total * current_txn_rate)
        mtd_revenue = {
            "mtd_ach_txns": ach_mtd,
            "mtd_total_txns": mtd_total,
            "mtd_txn_rate_used": round(current_txn_rate, 4),
            "mtd_txn_revenue": mtd_txn_rev,
            "mtd_fbo_revenue": mtd_fbo,
            "mtd_total_revenue": mtd_txn_rev + mtd_fbo,
            "mtd_revenue_estimate_M": round((mtd_txn_rev + mtd_fbo) / 1_000_000, 2),
        }

    # Monthly projection (txn revenue + FBO)
    month_total = ach_to_total(base_vol_month) if base_vol_month else 0
    month_txn_trending = calc_revenue(month_total, trending_rate)
    month_txn_flat = calc_revenue(month_total, flat_r)
    month_rev_trending = month_txn_trending + fbo_monthly
    month_rev_flat = month_txn_flat + fbo_monthly

    # Quarter projection
    quarter_total = ach_to_total(base_vol_quarter) if base_vol_quarter else 0
    q_rate_info = project_rate(series, months_ahead + 1)
    quarter_txn_trending = calc_revenue(quarter_total, q_rate_info["projected_rate"])
    quarter_txn_flat = calc_revenue(quarter_total, flat_r)
    # Estimate months remaining in quarter for FBO
    quarter_month = ((month - 1) % 3)  # 0, 1, or 2
    months_in_quarter = 3
    quarter_fbo = fbo_monthly * months_in_quarter
    quarter_rev_trending = quarter_txn_trending + quarter_fbo
    quarter_rev_flat = quarter_txn_flat + quarter_fbo

    # Year projection - use average rate across remaining months
    year_total = ach_to_total(base_vol_year) if base_vol_year else 0
    yr_rate_info = project_rate(series, months_ahead + 6)
    annual_trending_rate = (trending_rate + yr_rate_info["projected_rate"]) / 2
    year_txn_trending = calc_revenue(year_total, annual_trending_rate)
    year_txn_flat = calc_revenue(year_total, flat_r)
    year_rev_trending = year_txn_trending + fbo_annual
    year_rev_flat = year_txn_flat + fbo_annual

    # 2x2 Scenario matrix (v2.0: txn revenue varies, FBO stays flat)
    scenarios = {
        "primary": {
            "label": "Trending Rate + Base Volume + FBO",
            "txn_revenue": round(year_txn_trending),
            "fbo_revenue": fbo_annual,
            "annual_revenue": round(year_rev_trending),
            "annual_revenue_M": round(year_rev_trending / 1_000_000, 1),
        },
        "volume_downside": {
            "label": "Trending Rate + Conservative Volume + FBO",
            "txn_revenue": round(year_txn_trending * conservative_factor),
            "fbo_revenue": fbo_annual,
            "annual_revenue": round(year_txn_trending * conservative_factor + fbo_annual),
            "annual_revenue_M": round((year_txn_trending * conservative_factor + fbo_annual) / 1_000_000, 1),
        },
        "rate_stall": {
            "label": "Flat Rate + Base Volume + FBO",
            "txn_revenue": round(year_txn_flat),
            "fbo_revenue": fbo_annual,
            "annual_revenue": round(year_rev_flat),
            "annual_revenue_M": round(year_rev_flat / 1_000_000, 1),
        },
        "full_downside": {
            "label": "Flat Rate + Conservative Volume + FBO",
            "txn_revenue": round(year_txn_flat * conservative_factor),
            "fbo_revenue": fbo_annual,
            "annual_revenue": round(year_txn_flat * conservative_factor + fbo_annual),
            "annual_revenue_M": round((year_txn_flat * conservative_factor + fbo_annual) / 1_000_000, 1),
        },
    }

    # Rate trend direction (v2.0: based on txn-only rate)
    if len(series) >= 2:
        prior_rate = series[-2]["txn_rev_per_txn"]
        current_rate_val = series[-1]["txn_rev_per_txn"]
        if current_rate_val > prior_rate * 1.02:
            rate_direction = "UP"
        elif current_rate_val < prior_rate * 0.98:
            rate_direction = "DOWN"
        else:
            rate_direction = "FLAT"
    else:
        rate_direction = "UNKNOWN"

    # Last month prediction accuracy (v2.0: txn revenue only)
    if len(series) >= 2:
        last_actual = series[-1]
        prior_series_for_test = series[:-1]
        lr_test = linear_regression(prior_series_for_test)
        pred_rate = lr_test["intercept"] + lr_test["slope"] * (len(prior_series_for_test))
        pred_txn_rev = last_actual["transactions"] * pred_rate
        actual_txn_rev = last_actual.get("txn_revenue", last_actual["product_revenue"])
        last_month_error = round(abs(pred_txn_rev - actual_txn_rev) / actual_txn_rev * 100, 1) if actual_txn_rev > 0 else None
    else:
        last_month_error = None

    # Dust analysis revenue impact
    dust_impacts = []
    if dust_deltas:
        for d in dust_deltas:
            impact = dust_revenue_impact(d["volume_delta"], trending_rate)
            impact["client"] = d.get("client", "Unknown")
            dust_impacts.append(impact)

    result = {
        "forecast_date": today.isoformat(),
        "target_month": f"{year}-{month:02d}",
        "mtd": mtd_revenue,
        "month_end": {
            "projected_ach_txns": base_vol_month,
            "projected_total_txns": month_total,
            "txn_trending_rate": trending_rate,
            "txn_flat_rate": round(flat_r, 4),
            "projected_txn_revenue_trending": round(month_txn_trending),
            "projected_fbo_revenue": fbo_monthly,
            "projected_total_revenue_trending": round(month_rev_trending),
            [REDACTED_SECRET]: round(month_rev_trending / 1_000_000, 2),
            "projected_total_revenue_flat": round(month_rev_flat),
            "projected_total_revenue_flat_M": round(month_rev_flat / 1_000_000, 2),
        },
        "quarter": {
            "projected_ach_txns": base_vol_quarter,
            "projected_total_txns": quarter_total,
            "projected_txn_revenue_M": round(quarter_txn_trending / 1_000_000, 2),
            "projected_fbo_revenue_M": round(quarter_fbo / 1_000_000, 2),
            "projected_total_revenue_M": round(quarter_rev_trending / 1_000_000, 2),
        },
        "year": {
            "projected_ach_txns": base_vol_year,
            "projected_total_txns": year_total,
            "annual_txn_trending_rate": round(annual_trending_rate, 4),
            "projected_txn_revenue_M": round(year_txn_trending / 1_000_000, 1),
            "projected_fbo_revenue_M": round(fbo_annual / 1_000_000, 1),
            "projected_total_revenue_M": round(year_rev_trending / 1_000_000, 1),
        },
        "fbo": {
            "monthly_amount": fbo_monthly,
            "annual_amount": fbo_annual,
            "method": "flat",
            "note": "SOFR-linked, treated as flat line per v2.0 model",
        },
        "scenarios": scenarios,
        "rate_analysis": {
            "current_txn_rate": round(series[-1]["txn_rev_per_txn"], 4),
            "current_blended_rate": round(series[-1]["rev_per_txn"], 4),
            "trending_projected_txn_rate": trending_rate,
            "flat_txn_rate_t3": round(flat_r, 4),
            "rate_direction": rate_direction,
            "lr_slope_per_month": round(lr["slope"], 5) if "slope" in lr else 0,
            "lr_mape_pct": lr.get("mape_pct", 0),
            "wma_mape_pct": wma.get("mape_pct", 0),
            "selected_model": rate_info["method"],
            "last_month_prediction_error_pct": last_month_error,
            "model_version": "2.0",
            "model_note": "Rates are txn-only (FBO excluded). FBO added as flat line.",
        },
        "ach_share": ach_share,
        "sofr": sofr,
        "drift": drift,
        "dust_revenue_impacts": dust_impacts if dust_impacts else None,
        "calibration": {
            "data_points": len(series),
            "date_range": f"{series[0]['month']} to {series[-1]['month']}",
            "q4_2025_gap": "filled_v2",
            "last_calibration": cal_data.get("last_calibration_date"),
        },
    }

    return result


# ---------------------------------------------------------------------------
# Report formatting
# ---------------------------------------------------------------------------

def format_revenue_report(r: dict) -> str:
    """Format revenue forecast as markdown section for the daily ACH report. v2.0: Shows txn + FBO split."""
    lines = []
    lines.append("")
    lines.append("## REVENUE FORECAST (v2.0)")
    lines.append("")

    me = r["month_end"]
    yr = r["year"]
    ra = r["rate_analysis"]
    ach = r["ach_share"]
    sofr = r["sofr"]
    fbo = r.get("fbo", {})
    scenarios = r["scenarios"]

    # MTD
    if r.get("mtd"):
        mtd = r["mtd"]
        lines.append(f"**MTD Revenue Estimate:** ${mtd['mtd_revenue_estimate_M']}M "
                     f"(Txn: ${mtd['mtd_txn_revenue']:,} + FBO: ${mtd['mtd_fbo_revenue']:,})")
    else:
        lines.append("**MTD Revenue Estimate:** N/A (no MTD transaction data)")

    # Month-end
    lines.append(f"**Projected Month-End:** ${me['projected_total_revenue_trending_M']}M "
                 f"(Txn: ${me['projected_txn_revenue_trending']:,} + FBO: ${me['projected_fbo_revenue']:,})")

    # Quarter
    q = r["quarter"]
    lines.append(f"**Projected Q{((int(r['target_month'][5:7]) - 1) // 3) + 1} "
                 f"{r['target_month'][:4]}:** ${q['projected_total_revenue_M']}M "
                 f"(Txn: ${q['projected_txn_revenue_M']}M + FBO: ${q['projected_fbo_revenue_M']}M)")

    # Year
    lines.append(f"**Projected FY{r['target_month'][:4]}:** ${yr['projected_total_revenue_M']}M "
                 f"(Txn: ${yr['projected_txn_revenue_M']}M + FBO: ${yr['projected_fbo_revenue_M']}M)")
    lines.append("")

    # Scenario matrix
    lines.append("| Scenario | Txn Revenue | FBO | Total FY{} |".format(r["target_month"][:4]))
    lines.append("|----------|------------|-----|------------|")
    for key in ["primary", "volume_downside", "rate_stall", "full_downside"]:
        s = scenarios[key]
        txn_m = round(s["txn_revenue"] / 1_000_000, 1)
        fbo_m = round(s["fbo_revenue"] / 1_000_000, 1)
        lines.append(f"| {s['label']} | ${txn_m}M | ${fbo_m}M | ${s['annual_revenue_M']}M |")
    lines.append("")

    # Key metrics
    lines.append(f"**Current Txn Rate:** ${ra['current_txn_rate']:.4f}/txn "
                 f"(trending {ra['rate_direction']} vs prior month)")
    lines.append(f"**Legacy Blended Rate:** ${ra['current_blended_rate']:.4f}/txn (includes FBO, for reference only)")
    lines.append(f"**FBO Monthly:** ${fbo.get('monthly_amount', 433000):,} (flat, SOFR-linked)")
    lines.append(f"**ACH Share Ratio:** {ach['ratio'] * 100:.1f}% ({ach['method']})")

    if sofr.get("available"):
        lines.append(f"**SOFR:** {sofr['rate']:.2f}% (as of {sofr['date']}, "
                     f"{sofr['direction']} {abs(sofr['bps_change'])}bps from prior month)")
    else:
        lines.append(f"**SOFR:** Unavailable ({sofr.get('error', 'fetch failed')})")

    if ra["last_month_prediction_error_pct"] is not None:
        lines.append(f"**Model Accuracy:** Last month txn prediction error: {ra['last_month_prediction_error_pct']}%")

    # Drift warning
    drift = r.get("drift", {})
    if drift.get("drift_detected"):
        lines.append("")
        lines.append("**WARNING: MODEL DRIFT DETECTED**")
        for dm in drift.get("drift_months", []):
            lines.append(f"  {dm['month']}: predicted ${dm['predicted_rev']:,} vs actual ${dm['actual_rev']:,} "
                         f"(error: {dm['error_pct']}%)")

    # Dust revenue impacts
    if r.get("dust_revenue_impacts"):
        lines.append("")
        lines.append("**Client Volume Impact (Revenue Estimate):**")
        for d in r["dust_revenue_impacts"]:
            sign = "+" if d["volume_delta"] > 0 else ""
            lines.append(f"  {d['client']}: {sign}{d['volume_delta']:,} txns = "
                         f"{sign}${d['estimated_revenue_impact_K']}K revenue")

    return "\n".join(lines)


# ---------------------------------------------------------------------------
# Standalone CLI
# ---------------------------------------------------------------------------

def main():
    parser = argparse.ArgumentParser(description="Revenue Forecaster v1.0")
    parser.add_argument("--ach-projected", type=int, help="Projected ACH txns for current month")
    parser.add_argument("--ach-quarter", type=int, help="Projected ACH txns for quarter")
    parser.add_argument("--ach-year", type=int, help="Projected ACH txns for year")
    parser.add_argument("--ach-mtd", type=int, help="ACH transactions month-to-date")
    parser.add_argument("--month", type=str, help="Target month (YYYY-MM, default: current)")
    parser.add_argument("--standalone", action="store_true", help="Run standalone with model defaults")
    parser.add_argument("--json", action="store_true", help="Output JSON")
    args = parser.parse_args()

    if args.standalone and not args.ach_projected:
        # Use estimate.py's model for volume projections
        # Import dynamically to avoid circular dependency
        sys.path.insert(0, str(SCRIPT_DIR))
        from estimate import forecast as volume_forecast
        vol = volume_forecast(forecast_only=True, month_str=args.month)
        args.ach_projected = vol["current_month"]["estimated_total"]
        args.ach_quarter = vol["quarter"]["total"]
        args.ach_year = vol["year"]["total"]

    result = revenue_forecast(
        ach_projected_month=args.ach_projected,
        ach_projected_quarter=args.ach_quarter,
        ach_projected_year=args.ach_year,
        ach_mtd=args.ach_mtd,
        month_str=args.month,
    )

    if args.json:
        print(json.dumps(result, indent=2))
    else:
        print(format_revenue_report(result))


if __name__ == "__main__":
    main()


path: scripts/update_dashboard.py
content: #!/usr/bin/env python3
"""
Dashboard Updater v1.0

Regenerates the embedded DATA block in dashboard.html with fresh forecast data.
Reads the latest ACH KPI email data, runs the forecast pipeline, and patches
the dashboard HTML in-place.

Usage:
  update_dashboard.py --transactions N --bds-elapsed N [--month YYYY-MM]
  update_dashboard.py --auto   # reads from latest-ach-data.json
"""

import argparse
import json
import re
import sys
from datetime import date
from pathlib import Path

SCRIPT_DIR = Path(__file__).parent
WORK_DIR = Path.home() / "clawd" / "work" / "ach-reports"
DASHBOARD_PATH = WORK_DIR / "dashboard.html"
LATEST_DATA_PATH = WORK_DIR / "latest-ach-data.json"


def build_dashboard_data(transactions: int, bds_elapsed: int, month_str: str = None,
                          daily_volumes: list = None) -> dict:
    """Run forecast pipeline and build the full dashboard data object."""
    sys.path.insert(0, str(SCRIPT_DIR))
    from estimate import forecast
    
    result = forecast(
        transactions=transactions,
        bds_elapsed=bds_elapsed,
        month_str=month_str,
    )
    
    if "error" in result:
        print(f"Forecast error: {result['error']}", file=sys.stderr)
        sys.exit(1)

    # Run revenue forecast
    try:
        from revenue_forecast import revenue_forecast
        rev = revenue_forecast(
            ach_projected_month=result["current_month"]["estimated_total"],
            ach_projected_quarter=result["quarter"]["total"],
            ach_projected_year=result["year"]["total"],
            ach_mtd=transactions,
            month_str=month_str,
        )
        result["revenue"] = rev
    except Exception as e:
        result["revenue"] = {"error": str(e)}

    # Build dashboard-shaped data, mapping forecast output to dashboard JS expectations
    today = date.today()
    cm = result["current_month"]
    q = result["quarter"]
    yr = result["year"]
    
    # Map current_month fields to dashboard expected names
    cm_dashboard = dict(cm)  # start with all existing fields
    cm_dashboard["estimate"] = cm.get("estimated_total", 0)
    cm_dashboard["business_days_elapsed"] = cm.get("bds_elapsed", 0)
    cm_dashboard["business_days_total"] = cm.get("total_business_days", 0)
    cm_dashboard["ci_low"] = int(cm.get("confidence_low_M", 0) * 1_000_000)
    cm_dashboard["ci_high"] = int(cm.get("confidence_high_M", 0) * 1_000_000)
    cm_dashboard["blend_description"] = cm.get("method", "")
    if daily_volumes:
        cm_dashboard["daily_volumes"] = daily_volumes
    
    # SPLY and prior month data (from the old data format)
    # estimate.py doesn't directly give these, but we can derive
    sply_actual = yr.get("prior_year_total", 0)
    # approximate: prior year same month = prior year total / 12
    # Better: use the actuals from estimate.py
    from estimate import get_sorted_actuals, MONTHLY_ACTUALS
    target_month = cm.get("month", f"{today.year}-{today.month:02d}")
    target_year = int(target_month[:4])
    target_m = int(target_month[5:7])
    sply_key = f"{target_year - 1}-{target_m:02d}"
    sply_val = MONTHLY_ACTUALS.get(sply_key, 0)
    cm_dashboard["sply_actual"] = sply_val
    prior_key = f"{target_year}-{target_m - 1:02d}" if target_m > 1 else f"{target_year - 1}-12"
    cm_dashboard["prior_month_actual"] = MONTHLY_ACTUALS.get(prior_key, 0)
    
    # Prior year daily volumes (uniform approximation for the chart)
    sply_bds = cm_dashboard.get("business_days_total", 22)
    if sply_val > 0:
        sply_daily = int(sply_val / sply_bds)
        cm_dashboard["prior_year_daily"] = [sply_daily] * sply_bds
    cm_dashboard["prior_year_label"] = f"{target_month[:4].replace(str(target_year), str(target_year-1))}-{target_m:02d}"
    cm_dashboard["sply_bds"] = sply_bds
    
    # YTD avg per BD
    from estimate import business_days_in_month
    ytd_txns = 0
    ytd_bds = 0
    for m_num in range(1, target_m):
        k = f"{target_year}-{m_num:02d}"
        if k in MONTHLY_ACTUALS:
            ytd_txns += MONTHLY_ACTUALS[k]
            ytd_bds += business_days_in_month(target_year, m_num)
    # Add current month
    ytd_txns += cm.get("transactions_so_far", 0)
    ytd_bds += cm.get("bds_elapsed", 0)
    if ytd_bds > 0:
        cm_dashboard["ytd_avg_per_bd"] = int(ytd_txns / ytd_bds)
    else:
        cm_dashboard["ytd_avg_per_bd"] = cm.get("per_bd", 0)
    
    # Momentum: approximate 20-day trailing % change
    # Compare last 5 BD avg to prior 5 BD avg if we have daily data
    if daily_volumes and len(daily_volumes) >= 10:
        recent5 = sum(daily_volumes[-5:]) / 5
        prior5 = sum(daily_volumes[-10:-5]) / 5
        if prior5 > 0:
            cm_dashboard["momentum_20day_pct"] = round((recent5 - prior5) / prior5 * 100, 1)
    if "momentum_20day_pct" not in cm_dashboard:
        cm_dashboard["momentum_20day_pct"] = 0.0
    
    # Per BD rate
    cm_dashboard["per_bd_rate"] = cm.get("run_rate_per_bd", cm.get("per_bd", 0))
    
    # Map quarter fields
    q_dashboard = dict(q)
    q_estimate = q.get("total", 0)
    q_dashboard["q1_estimate"] = q_estimate
    # Ensure months have 'value' and 'status' fields
    if "months" in q_dashboard:
        for m in q_dashboard["months"]:
            if "value" not in m:
                m["value"] = m.get("estimated_total", 0)
            if "status" not in m:
                m["status"] = m.get("method", "estimate")
    
    # Map full_year fields
    fy_dashboard = {
        "estimate": yr.get("total", 0),
        "prior_year_actual": yr.get("prior_year_total", 0),
        "yoy_pct": yr.get("yoy_growth_pct", 0),
        "confidence_low": int(yr.get("confidence_low_M", 0) * 1_000_000),
        "confidence_high": int(yr.get("confidence_high_M", 0) * 1_000_000),
    }
    
    dashboard = {
        "report_date": today.isoformat(),
        "data_through": today.isoformat(),
        "current_month": cm_dashboard,
        "quarter": q_dashboard,
        "full_year": fy_dashboard,
        "model": result.get("model", {}),
        "model_notes": result.get("model", {}),
        "client_movers": {},
        "alerts": [],
    }
    
    # Add revenue section
    if "revenue" in result and "error" not in result["revenue"]:
        rev = result["revenue"]
        dashboard["revenue"] = {
            "mtd": rev.get("mtd"),
            "month_end": rev.get("month_end"),
            "quarter": rev.get("quarter"),
            "year": rev.get("year"),
            "scenarios": rev.get("scenarios"),
            "rate_analysis": rev.get("rate_analysis"),
            "calibration": rev.get("calibration"),
        }
    
    return dashboard


def patch_dashboard_html(data: dict) -> bool:
    """Replace the embedded DATA block in dashboard.html."""
    if not DASHBOARD_PATH.exists():
        print(f"Dashboard not found: {DASHBOARD_PATH}", file=sys.stderr)
        return False
    
    html = DASHBOARD_PATH.read_text()
    
    # Find and replace the const DATA = {...}; block
    # The data block starts with "const DATA = {" and ends with "};"
    # We need to match the full JSON object including nested braces
    pattern = r'const DATA = \{[^;]*\};'
    
    # More robust: find start, then count braces to find end
    start_marker = 'const DATA = '
    start_idx = html.find(start_marker)
    if start_idx == -1:
        print("Could not find 'const DATA = ' in dashboard.html", file=sys.stderr)
        return False
    
    json_start = start_idx + len(start_marker)
    # Count braces to find the matching close
    depth = 0
    i = json_start
    found_end = -1
    while i < len(html):
        if html[i] == '{':
            depth += 1
        elif html[i] == '}':
            depth -= 1
            if depth == 0:
                found_end = i + 1
                break
        i += 1
    
    if found_end == -1:
        print("Could not find end of DATA block", file=sys.stderr)
        return False
    
    # Skip any trailing semicolon
    if found_end < len(html) and html[found_end] == ';':
        found_end += 1
    
    # Build new DATA block
    new_json = json.dumps(data, indent=2)
    new_block = f'const DATA = {new_json};'
    
    new_html = html[:start_idx] + new_block + html[found_end:]
    DASHBOARD_PATH.write_text(new_html)
    
    # Also save latest data JSON
    with open(LATEST_DATA_PATH, 'w') as f:
        json.dump(data, f, indent=2)
    
    return True


def main():
    parser = argparse.ArgumentParser(description="Dashboard Updater v1.0")
    parser.add_argument("--transactions", type=int, help="Transactions so far this month")
    parser.add_argument("--bds-elapsed", type=int, help="Business days elapsed")
    parser.add_argument("--month", type=str, help="Month (YYYY-MM)")
    parser.add_argument("--auto", action="store_true", help="Read from latest-ach-data.json")
    parser.add_argument("--json", action="store_true", help="Output JSON only (don't patch HTML)")
    parser.add_argument("--daily-volumes", type=str, help="Comma-separated daily volumes")
    args = parser.parse_args()
    
    if args.auto:
        if not LATEST_DATA_PATH.exists():
            print(f"No latest data at {LATEST_DATA_PATH}", file=sys.stderr)
            sys.exit(1)
        with open(LATEST_DATA_PATH) as f:
            existing = json.load(f)
        transactions = existing["current_month"].get("transactions_so_far", 0)
        bds = existing["current_month"].get("bds_elapsed", 0)
        month_str = existing["current_month"].get("month")
        daily_volumes = existing["current_month"].get("daily_volumes")
    elif args.transactions and args.bds_elapsed:
        transactions = args.transactions
        bds = args.bds_elapsed
        month_str = args.month
        daily_volumes = [int(x) for x in args.daily_volumes.split(",")] if args.daily_volumes else None
    else:
        print("Provide --transactions + --bds-elapsed, or --auto", file=sys.stderr)
        sys.exit(1)
    
    data = build_dashboard_data(transactions, bds, month_str, daily_volumes)
    
    if args.json:
        print(json.dumps(data, indent=2))
    else:
        ok = patch_dashboard_html(data)
        if ok:
            print(f"Dashboard updated: {DASHBOARD_PATH}")
            print(f"Data saved: {LATEST_DATA_PATH}")
            print(f"Report date: {data['report_date']}")
            cm = data['current_month']
            print(f"MTD: {cm.get('transactions_so_far',0):,} txns, {cm.get('bds_elapsed',0)} BDs")
            print(f"EOM estimate: {cm.get('estimated_total_M', '?')}M")
        else:
            print("Failed to update dashboard", file=sys.stderr)
            sys.exit(1)


if __name__ == "__main__":
    main()


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)