npm package: @new-code/spec-forecasting
Description: BMAD module for rigorous time series forecasting specification - prevents MAPE disasters and enforces forecasting discipline through AI agents
Author: Philippe Béliveau
Latest version: 1.0.0

---
last-redoc-date: 2025-10-08
---

# Time Series Forecasting Specification Framework

[![NPM Version](https://img.shields.io/npm/v/@new-code/spec-forecasting?style=flat-square&color=blue)](https://www.npmjs.com/package/@new-code/spec-forecasting)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg?style=flat-square)](https://opensource.org/licenses/MIT)
[![GitHub Stars](https://img.shields.io/github/stars/philbeliveau/ts-forecast-spec?style=flat-square&color=gold)](https://github.com/philbeliveau/ts-forecast-spec)
[![GitHub Issues](https://img.shields.io/github/issues/philbeliveau/ts-forecast-spec?style=flat-square)](https://github.com/philbeliveau/ts-forecast-spec/issues)
[![BMAD Compatible](https://img.shields.io/badge/BMAD-V6%20Compatible-green?style=flat-square)](https://github.com/bmad-code-org/BMAD-METHOD)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](https://github.com/philbeliveau/ts-forecast-spec/pulls)

**BMAD Module Code:** `ts-forecast-spec`
**Version:** 1.0.0-alpha.1
**Status:** 🎉 **Phase 1 Complete** - Production Ready
**Author:** Philippe Béliveau ([Newcode](https://newcode.ai))
**License:** MIT

---

## 📥 Installation & Quick Start

### NPM Installation (Recommended)

```bash
# Install globally
npm install -g @new-code/spec-forecasting

# Or use with npx (no installation)
npx @new-code/spec-forecasting

# Or install in your BMAD project
cd your-bmad-project
npm install @new-code/spec-forecasting
```

### Manual Installation (BMAD Integration)

**👉 [Installation Guide](INSTALL.md)** - Full installation instructions
**👉 [Quick Start Guide](QUICKSTART.md)** - Get started in 5 minutes
**👉 [Roadmap](ROADMAP.md)** - See what's built and what's coming

**TL;DR Installation:**
1. Add `ts-forecast-spec` to `bmad/_cfg/manifest.yaml`
2. Add 3 agents to `bmad/_cfg/agent-manifest.csv`
3. Run: `bmad business-context-architect`

See [`_module-installer/MANUAL_INSTALL.md`](_module-installer/MANUAL_INSTALL.md) for step-by-step instructions.

---

## 🎯 Mission

**Prevent forecasting failures by forcing rigorous problem specification BEFORE any modeling begins.**

Most forecasting projects fail not because of bad models, but because teams skip fundamental diagnostic questions and specification steps. This BMAD module provides a framework of specialized AI agents that act as "blocking gatekeepers" — refusing to let you proceed until critical diagnostics are addressed.

**Core Philosophy:**
> "Most forecasting failures stem from specification problems, not model problems."
> — Inspired by Valeriy Manokhin's "5% models, 95% everything else"

---

## 💥 The Story: The MAPE Disaster

**Real Corporate Failure (Valeriy Manokhin's Case Study):**

Two forecasting projects at a large corporation optimized for MAPE (Mean Absolute Percentage Error) without understanding its destructiveness for their data. The result:

- Teams had a "contract" to deliver the best possible MAPE
- Expensive models performed well on MAPE... but failed in production
- Customer service collapsed
- **Billion-dollar clients walked away**

**The Problem:** MAPE explodes when actual values are near zero (division by small numbers). The models optimized to avoid zeros, not to make good business decisions.

**This module prevents this exact disaster** — and many others like it.

---

## 🏗️ Module Architecture

### ✅ Phase 1: Built & Ready (3 Agents, 2 Workflows, 4 Tasks)

**Phase 1 is complete and production-ready.** The following components are fully implemented and tested:

#### 🤖 Built Agents (Phase 1)

#### 1. **Business Context Architect** ✅ (Elena 🎯)
- **Role:** Frame forecasting problem in business terms FIRST
- **Personality:** Strategic, pragmatic, asks "so what?" relentlessly
- **Blocks Until:**
  - You articulate what decision this forecast drives
  - You define cost of over-forecasting vs under-forecasting
  - You identify stakeholders and their uncertainty tolerance
- **File:** `agents/business-context-architect.md`
- **Usage:** `bmad business-context-architect`

#### 2. **Metrics Guardian** ✅ (Marcus 🛡️)
- **Role:** Prevent metric disasters and align evaluation with business reality
- **Personality:** Battle-scarred, skeptical, references real failures (MAPE disaster)
- **Blocks Until:**
  - MAPE challenged if near-zero data detected
  - Metric reflects business cost function
  - Asymmetric costs addressed if relevant
  - Point forecasts vs prediction intervals decided
- **Famous Reference:** The billion-dollar MAPE disaster
- **File:** `agents/metrics-guardian.md`
- **Usage:** `bmad metrics-guardian`

#### 3. **Data Diagnostician** ✅ (Dr. Sarah Chen 🔬)
- **Role:** Force rigorous statistical analysis of data characteristics
- **Personality:** Obsessed with fundamentals, refuses shortcuts, speaks in tests (ADF, KPSS, ACF)
- **Blocks Until:**
  - Stationarity checked with evidence (ADF, KPSS, PP tests)
  - Trend, seasonality, cyclical components identified
  - Autocorrelation structure assessed
- **Will NOT discuss models until data characteristics understood**
- **File:** `agents/data-diagnostician.md`
- **Usage:** `bmad data-diagnostician`

---

#### 📋 Built Workflows (Phase 1)

**1. Problem Specification** ✅
*Lead: Business Context Architect (Elena)*
Output: 790-line Problem Specification Document with business context, stakeholders, cost asymmetry, viability
**Path:** `workflows/problem-specification/`

**2. Metric Selection** ✅
*Lead: Metrics Guardian (Marcus)*
Output: Metric Justification Document with primary metric, supporting metrics, avoided metrics
**Path:** `workflows/metric-selection/`

---

#### 🔧 Built Tasks (Phase 1)

All 4 core tasks are fully implemented:

1. **Conformal Prediction Validator** ✅ - `tasks/conformal-prediction-validator.xml`
2. **Calibration Monitor** ✅ - `tasks/calibration-monitor.xml`
3. **Uncertainty Diagnostics** ✅ - `tasks/uncertainty-diagnostics.xml`
4. **Antipattern Detector** ✅ - `tasks/antipattern-detector.xml`

---

### 🔮 Phase 2+: Coming Soon (4 Agents, 4 Workflows)

The following components are planned for future releases (see [ROADMAP.md](ROADMAP.md)):

#### Agents (Phase 2+)
- **Signal Architect** - Feature engineering and signal design
- **Missing Data & Anomaly Specialist** - MCAR/MAR/MNAR analysis
- **Method Advisor** - Model selection with NIXTLA ecosystem
- **Validation & Deployment Strategist** - Production readiness

#### Workflows (Phase 2+)
- **Data Diagnostics & Quality** - Full data diagnostic workflow
- **Signal Design & Feature Engineering**
- **Model Selection & Approach**
- **Validation & Deployment Strategy**

---

## 🔧 Core Tasks (From Valeriy Manokhin's Methodology)

Tasks are reusable diagnostic utilities that agents invoke. All 4 tasks are fully implemented.

#### 1. **Conformal Prediction Validator** ✅
- **Purpose:** Validates CP implementation, checks coverage guarantees
- **Validates:**
  - Prediction intervals present (not just point estimates)
  - Empirical coverage matches stated confidence (95% intervals contain 95% actuals)
  - Calibration set exists and sized appropriately
  - CP method appropriate for time series (EnbPI, adaptive methods)
- **Blocks If:** No intervals, coverage <85% for 95% PI, no calibration set
- **Reference:** Valeriy Batch 1 & 3 (Conformal Prediction for Time Series)

#### 2. **Calibration Monitor** ✅
- **Purpose:** Monitors calibration of probabilistic forecasts
- **Calculates:**
  - **ECE (Expected Calibration Error):** Predicted confidence vs empirical accuracy
  - **Log Loss:** Penalizes confident wrong predictions
  - **Brier Score:** Decomposed into calibration + refinement loss
- **Detects:** Calibration drift over time (distribution shift warning)
- **Blocks If:** ECE >0.15, Brier >0.30, severe drift detected
- **Reference:** Valeriy Batch 4 (Advanced Topics - Calibration)

#### 3. **Uncertainty Diagnostics** ✅
- **Purpose:** Distinguish aleatoric (irreducible) vs epistemic (reducible) uncertainty
- **Decomposes:** Total uncertainty into components
- **Strategic Guidance:**
  - **Aleatoric-dominated:** Accept and manage (robust planning)
  - **Epistemic-dominated:** Reduce through better data/models
- **Blocks If:** Epistemic >80% at high-stakes + long horizons
- **Reference:** Valeriy Batch 1 (Foundation - Uncertainty Types)

#### 4. **Antipattern Detector** ✅
- **Purpose:** Detect forecasting antipatterns that cause real-world disasters
- **Detects:**
  - **MAPE on near-zero data** (the billion-dollar disaster)
  - **SMOTE + Conformal Prediction** (destroys coverage guarantees)
  - **Metric-business misalignment** (symmetric metrics, asymmetric costs)
  - **Ignored stationarity** (ARIMA without testing)
  - **Point forecasts only** (no uncertainty quantification)
  - **Temporal leakage** (random splits on time series)
- **Blocks If:** Critical antipatterns detected (MAPE+zeros, SMOTE+CP, random splits)
- **Reference:** Valeriy MAPE case study + Batch 4 (SMOTE antipattern)

---

## 🚀 Quick Start

**See [QUICKSTART.md](QUICKSTART.md) for a complete 5-minute getting started guide.**

### Installation (2 minutes)

**Full Instructions:** [INSTALL.md](INSTALL.md)

**Quick Install:**
1. Edit `bmad/_cfg/manifest.yaml` - add `ts-forecast-spec` to modules list
2. Edit `bmad/_cfg/agent-manifest.csv` - add 3 agent entries (see [_module-installer/MANUAL_INSTALL.md](_module-installer/MANUAL_INSTALL.md))
3. Test: `bmad business-context-architect`

### Your First Workflow (3 minutes)

**Scenario:** You need to forecast daily sales for inventory planning

```bash
# Step 1: Activate Business Context Architect
bmad business-context-architect

# Step 2: Select Problem Specification Workflow
> 1  # or type: problem-spec

# Step 3: Answer Elena's diagnostic questions
Elena will guide you through:
- What decision does this forecast drive?
- Who makes the decision?
- What happens if forecast is wrong?
- Cost asymmetry analysis (over vs under-forecast)
- Stakeholder mapping
- Viability checks

# Output: Problem Specification Document (790 lines)
# Saved to: output/forecasting/problems/problem-specification-{date}.md
```

### Challenge Your Metrics (2 minutes)

```bash
# Activate Metrics Guardian
bmad metrics-guardian

# Quick MAPE disaster check
> 2  # or type: mape-check

# Marcus will ask about near-zero values
# If detected: 🚨 CRITICAL WARNING - MAPE is BLOCKED
# Recommendation: Switch to MAE or sMAPE

# Full metric selection workflow
> 1  # or type: metric-selection

# Output: Metric Justification Document
# Includes: Primary metric, supporting metrics, avoided metrics
```

### Diagnose Your Data (2 minutes)

```bash
# Activate Data Diagnostician
bmad data-diagnostician

# Check stationarity
> 1  # or type: stationarity

# Dr. Sarah Chen will guide you through:
# - ADF and KPSS test interpretation
# - Differencing recommendations
# - Implications for model selection

# Other options:
> 2  # Decomposition (trend, seasonality)
> 3  # ACF/PACF analysis
> 4  # Missing data classification
> 5  # Signal-to-noise analysis
```

### Recommended Workflow Sequence

1. **Problem Specification** (Business Context Architect) → Outputs cost asymmetry
2. **Data Diagnostics** (Data Diagnostician - consulting mode) → Outputs data characteristics
3. **Metric Selection** (Metrics Guardian) → Uses outputs from #1 and #2
4. **(Phase 2) Model Selection** → Coming soon

---

## 📊 Real-World Failure Patterns Prevented

### 1. The MAPE Disaster
**Symptom:** Optimizing MAPE with near-zero values
**Prevention:** Antipattern Detector + Metrics Guardian block MAPE usage with zeros
**Outcome:** Billion-dollar client loss avoided

### 2. The False Precision Trap
**Symptom:** Point forecasts without uncertainty for high-stakes decisions
**Prevention:** Metrics Guardian mandates prediction intervals
**Outcome:** Risk-aware decision making, contingency planning enabled

### 3. The Data Leakage Disaster
**Symptom:** Random train/test split on time series (future trains the model)
**Prevention:** Antipattern Detector catches temporal violations
**Outcome:** Valid performance metrics, production success

### 4. The SMOTE Coverage Violation
**Symptom:** Synthetic oversampling destroys conformal prediction guarantees
**Prevention:** Antipattern Detector blocks SMOTE + CP combination
**Outcome:** Mathematically valid prediction intervals

### 5. The Ignored Stationarity Sin
**Symptom:** ARIMA on trending data without differencing (spurious regression)
**Prevention:** Data Diagnostician mandates stationarity tests
**Outcome:** Reliable statistical forecasts

### 6. The Metric Misalignment Failure
**Symptom:** Symmetric metrics (RMSE) with asymmetric costs (stockout ≠ overstock)
**Prevention:** Metrics Guardian forces business cost function alignment
**Outcome:** Optimized for real business decisions, not academic metrics

---

## 🧠 Methodology Foundation

### Valeriy Manokhin's Principles

This module is built on the rigorous forecasting methodology of **Valeriy Manokhin** (PhD Machine Learning, "Black Belt 🥋 in Time Series"):

**Key Insights:**
- **The 5% Rule:** Only 5% of success is about models. 95% is data, metrics, stakeholder alignment, and specification.
- **Conformal Prediction Advocacy:** Distribution-free uncertainty estimates that work with data "as it really is"
- **Failure Analysis:** Learn from real disasters (MAPE case study)
- **Academic Rigor:** PhD-level discipline applied to real-world problems

**Referenced Works:**
- "Mastering Modern Time Series Forecasting"
- "Applied Conformal Prediction"
- "Mastering Forecasting Metrics & Accuracy"
- LinkedIn posts on forecasting methodology (300+ likes, engaged audience)

### Philippe's Experience Integration

**Real-World Context from:**
- **Videns Analytics:** Built 3 cash flow forecasting approaches (statistical + MDP) for corporate treasurers at daily/weekly horizons. Key challenge: distinguishing signal from noise at granular time scales.
- **CroesusLab:** Developed GANs for synthetic relational databases, focusing on preserving temporal correlation structures.
- **Groupe Azur:** Led user discovery with corporate treasurers, translating business needs to data science teams.

**Newcode Philosophy:**
> "If you learn to ask the right question, then you can build anything with AI agents."

---

## 📁 Module Structure

```
bmad/ts-forecast-spec/
├── agents/                          # ✅ 3/7 agents (Phase 1 complete)
│   ├── business-context-architect.md     # ✅ Elena 🎯
│   ├── metrics-guardian.md               # ✅ Marcus 🛡️
│   ├── data-diagnostician.md             # ✅ Dr. Sarah Chen 🔬
│   ├── signal-architect.md               # 📋 Phase 2
│   ├── missing-data-specialist.md        # 📋 Phase 2
│   ├── method-advisor.md                 # 📋 Phase 2
│   └── validation-strategist.md          # 📋 Phase 2
│
├── workflows/                       # ✅ 2/6 workflows (Phase 1 complete)
│   ├── problem-specification/            # ✅ Full workflow
│   │   ├── workflow.yaml
│   │   ├── instructions.md
│   │   ├── template.md
│   │   └── checklist.md
│   ├── metric-selection/                 # ✅ Full workflow
│   │   ├── workflow.yaml
│   │   ├── instructions.md
│   │   ├── template.md
│   │   └── checklist.md
│   ├── data-diagnostics/                 # 📋 Phase 2
│   ├── signal-design/                    # 📋 Phase 2
│   ├── model-selection/                  # 📋 Phase 2
│   └── validation-deployment/            # 📋 Phase 2
│
├── tasks/                           # ✅ 4/4 core tasks (ALL IMPLEMENTED)
│   ├── conformal-prediction-validator.xml
│   ├── calibration-monitor.xml
│   ├── uncertainty-diagnostics.xml
│   └── antipattern-detector.xml
│
├── _module-installer/               # ✅ Installation infrastructure
│   ├── install.py                        # Python installer
│   ├── install.sh                        # Shell installer
│   ├── MANUAL_INSTALL.md                 # Manual installation guide
│   └── README.md
│
├── config.yaml                      # ✅ Module configuration
├── README.md                        # ✅ This file
├── INSTALL.md                       # ✅ Full installation guide
├── QUICKSTART.md                    # ✅ 5-minute getting started
└── ROADMAP.md                       # ✅ Development roadmap
```

---

## 🛠️ Development Status

### ✅ Phase 1 Complete (Production Ready)

**Delivered:**
- **Agents (3/3 Phase 1):** ✅ ALL BUILT
  - Business Context Architect (Elena 🎯)
  - Metrics Guardian (Marcus 🛡️)
  - Data Diagnostician (Dr. Sarah Chen 🔬)
- **Workflows (2/2 Phase 1):** ✅ ALL BUILT
  - Problem Specification (9 steps, 790-line output)
  - Metric Selection (8 steps, comprehensive justification)
- **Core tasks (4/4):** ✅ ALL IMPLEMENTED
  - Conformal Prediction Validator
  - Calibration Monitor
  - Uncertainty Diagnostics
  - Antipattern Detector
- **Module infrastructure:** ✅ COMPLETE
  - Configuration system
  - Installation system (manual + automated)
  - Complete documentation

**Total Delivered:** ~7,500 lines of code and documentation

### 📋 Phase 2+ Planned

See [ROADMAP.md](ROADMAP.md) for detailed plans:

- **Phase 2:** Diagnostic Infrastructure (2-3 weeks)
  - Signal Architect, Missing Data Specialist
  - Data Diagnostics, Signal Design workflows

- **Phase 3:** Complete Framework (2-4 weeks)
  - Method Advisor, Validation Strategist
  - Model Selection, Validation & Deployment workflows

---

## 📚 Documentation

### Module Documentation

| Document | Description |
|----------|-------------|
| **[README.md](README.md)** | This file - Module overview and architecture |
| **[INSTALL.md](INSTALL.md)** | Complete installation guide with troubleshooting |
| **[QUICKSTART.md](QUICKSTART.md)** | 5-minute getting started guide with examples |
| **[ROADMAP.md](ROADMAP.md)** | Development roadmap and phase planning |
| **[config.yaml](config.yaml)** | Module configuration file |

### Installation Documentation

| Document | Description |
|----------|-------------|
| **[_module-installer/MANUAL_INSTALL.md](_module-installer/MANUAL_INSTALL.md)** | Step-by-step manual installation (recommended) |
| **[_module-installer/README.md](_module-installer/README.md)** | Installer directory overview |
| **[_module-installer/install.py](_module-installer/install.py)** | Automated Python installer |

### Agent Documentation

| Agent | File | Description |
|-------|------|-------------|
| **Elena 🎯** | [agents/business-context-architect.md](agents/business-context-architect.md) | Problem framing specialist |
| **Marcus 🛡️** | [agents/metrics-guardian.md](agents/metrics-guardian.md) | Metric selection expert |
| **Dr. Sarah Chen 🔬** | [agents/data-diagnostician.md](agents/data-diagnostician.md) | Data diagnostics specialist |

### Workflow Documentation

| Workflow | Path | Description |
|----------|------|-------------|
| **Problem Specification** | [workflows/problem-specification/](workflows/problem-specification/) | Business context and viability |
| **Metric Selection** | [workflows/metric-selection/](workflows/metric-selection/) | Metric justification and selection |

Each workflow directory contains:
- `workflow.yaml` - Configuration
- `instructions.md` - Execution logic
- `template.md` - Output template
- `checklist.md` - Validation checklist

### Task Documentation

| Task | File | Purpose |
|------|------|---------|
| **Antipattern Detector** | [tasks/antipattern-detector.xml](tasks/antipattern-detector.xml) | Detect forecasting antipatterns |
| **Calibration Monitor** | [tasks/calibration-monitor.xml](tasks/calibration-monitor.xml) | Monitor probabilistic calibration |
| **Conformal Prediction Validator** | [tasks/conformal-prediction-validator.xml](tasks/conformal-prediction-validator.xml) | Validate CP coverage |
| **Uncertainty Diagnostics** | [tasks/uncertainty-diagnostics.xml](tasks/uncertainty-diagnostics.xml) | Decompose uncertainty types |

---

## 🎓 Learning Resources

### For Understanding This Module

1. **Start Here:** [QUICKSTART.md](QUICKSTART.md)
   Get up and running in 5 minutes with example scenarios

2. **Deep Dive:** Explore agent files
   - [business-context-architect.md](agents/business-context-architect.md) - See Elena's personality and philosophy
   - [metrics-guardian.md](agents/metrics-guardian.md) - Learn from Marcus's MAPE disaster story
   - [data-diagnostician.md](agents/data-diagnostician.md) - Understand Dr. Sarah's diagnostic toolkit

3. **Workflow Details:** Read workflow instructions
   - [workflows/problem-specification/instructions.md](workflows/problem-specification/instructions.md)
   - [workflows/metric-selection/instructions.md](workflows/metric-selection/instructions.md)

### For Forecasting Fundamentals

**Valeriy Manokhin's Books:**
- "Mastering Modern Time Series Forecasting"
- "Applied Conformal Prediction"
- "Mastering Forecasting Metrics & Accuracy"

**LinkedIn:** Follow Valeriy Manokhin for case studies and methodology insights

**Classic Textbook:**
- "Forecasting: Principles and Practice" (Hyndman & Athanasopoulos)

---

## 🤝 Contributing

**Current Status:** Module in active development by Philippe Béliveau.

**Future Collaboration:**
- Planned outreach to Valeriy Manokhin with working demo
- Community contributions welcome after Phase 1 complete
- Issues and feedback: [GitHub repository](https://github.com/philbeliveau/ts-forecast-spec)

---

## 📜 License

MIT License

Copyright (c) 2025 Philippe Béliveau / Newcode

---

## 🙏 Acknowledgments

- **Valeriy Manokhin:** Methodology foundation, conformal prediction expertise, MAPE disaster case study
- **BMAD Community:** Framework and module builder tools
- **Nixtla Team:** Comprehensive forecasting library ecosystem
- **Corporate Treasurers (Videns Analytics):** Real-world problem framing insights

---

## 📞 Contact

**Philippe Béliveau**
Co-Founder, Newcode
Montréal, Quebec, Canada

**Newcode Philosophy:**
*"If you learn to ask the right question, then you can build anything with AI agents"*

---

**Built with BMAD V6 Module Builder**
*Making AI agents work together to solve hard problems*