Initial release: sf-ml-baseline v0.1 (LightGBM + XGBoost + CatBoost ensemble)

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+weights/cat_v1_t1_seed137.cbm filter=lfs diff=lfs merge=lfs -text
+weights/cat_v1_t1_seed2026.cbm filter=lfs diff=lfs merge=lfs -text
+weights/cat_v1_t1_seed42.cbm filter=lfs diff=lfs merge=lfs -text
+weights/cat_v2_t4_seed137.cbm filter=lfs diff=lfs merge=lfs -text
+weights/cat_v2_t4_seed2026.cbm filter=lfs diff=lfs merge=lfs -text
+weights/cat_v2_t4_seed42.cbm filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,52 @@
+Model Weights License — sf-ml-baseline v0.1
+
+Copyright (c) 2026 SimpleFunctions (simplefunctions.dev)
+
+Licensed under the Creative Commons Attribution 4.0 International License
+(CC-BY-4.0), the full text of which is at:
+    https://creativecommons.org/licenses/by/4.0/legalcode
+
+In addition to the CC-BY-4.0 requirements, the following SimpleFunctions
+Attribution Addendum applies:
+
+1. Attribution.  You MUST retain a readable credit to SimpleFunctions
+   whenever you distribute, serve, or display predictions generated by
+   these weights, in any form.
+
+   Acceptable credit formats include, but are not limited to:
+     (a) "Powered by sf-ml-baseline (SimpleFunctions, simplefunctions.dev)"
+     (b) "Forecast by sf-ml-baseline — https://simplefunctions.dev/opensource/sf-ml-baseline"
+     (c) "SimpleFunctions sf-ml-baseline v0.1" in a documentation footnote,
+         `X-Attribution` HTTP header, or equivalent machine-readable field.
+
+2. Derived weights.  If you fine-tune, distill, or otherwise derive new
+   weights from these, you MUST:
+     (a) make the same CC-BY-4.0 license apply to the derivative,
+     (b) prefix the derivative model name with "sf-ml-baseline-" (e.g.,
+         "sf-ml-baseline-v0.2-crypto-specialist"), and
+     (c) document in the derivative's README the training data,
+         procedure, and how it differs from upstream.
+
+3. Attribution does not grant endorsement.  Your use of the name
+   "SimpleFunctions" or "sf-ml-baseline" does not imply endorsement of
+   your product, service, or paper by SimpleFunctions.
+
+4. Non-warranty. THE WEIGHTS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF
+   ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
+   WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+   NONINFRINGEMENT. SIMPLEFUNCTIONS DISCLAIMS ALL LIABILITY FOR ANY
+   PREDICTION, DECISION, OR OUTCOME DERIVED FROM USE OF THESE WEIGHTS,
+   INCLUDING BUT NOT LIMITED TO FINANCIAL LOSS FROM TRADING DECISIONS.
+
+5. Training data. These weights were trained on SimpleFunctions'
+   public-domain prediction-market indicator dataset, itself released
+   under CC-BY-4.0 per the SimpleFunctions Terms of Service §13.
+   See https://simplefunctions.dev/data-license for details.
+
+6. Responsible use. These weights are intended for academic research,
+   software integration, and personal analysis. Do NOT use them to
+   provide unlicensed financial advice, market manipulation, or
+   deceptive trading signals. You are responsible for all use.
+
+Contact: patrick@simplefunctions.dev
+License questions: see https://simplefunctions.dev/terms

README.md

@@ -0,0 +1,169 @@
+---
+license: other
+license_name: cc-by-4-0-with-sf-attribution
+license_link: LICENSE
+language:
+- en
+library_name: lightgbm
+tags:
+- prediction-markets
+- forecasting
+- calibration
+- brier
+- tabular-classification
+- gradient-boosting
+- lightgbm
+- xgboost
+- catboost
+- time-series
+- kalshi
+- polymarket
+- simplefunctions
+pipeline_tag: tabular-classification
+model-index:
+- name: sf-ml-baseline-v0.1
+  results:
+  - task:
+      type: tabular-classification
+      name: 24h direction forecast (V1 x T1)
+    metrics:
+    - type: brier
+      value: 0.2294
+      name: Brier score
+    - type: brier_improvement
+      value: 0.0206
+      name: Improvement vs coinflip baseline (0.2500)
+  - task:
+      type: tabular-classification
+      name: 24h resolution forecast (V2 x T4)
+    metrics:
+    - type: brier
+      value: 0.1681
+      name: Brier score (XGBoost)
+    - type: brier_improvement
+      value: 0.0086
+      name: Improvement vs market-price/100 baseline (0.1767)
+---
+
+# sf-ml-baseline v0.1
+
+**What it is**: gradient-boosted tree ensembles that predict prediction-market outcomes from engineered microstructure features.
+
+**Published**: 2026-04-19 (initial, time-capsule — see "Retrain plan" below).
+**Trained on**: 11 days of SimpleFunctions (`market_indicator_history` + `marketwide_resolutions`) — 2026-04-08 → 2026-04-19.
+**License**: CC-BY-4.0 with SimpleFunctions attribution — see `LICENSE`.
+**Author**: SimpleFunctions — https://simplefunctions.dev
+**Model repo**: https://huggingface.co/SimpleFunctions/sf-ml-baseline *(pending upload)*
+
+## Why release this
+
+Nobody has published a calibrated feature-based baseline for prediction-market forecasting. All prior art (Halawi 2024, Schoenegger 2024, AIA 2025) uses LLM + news retrieval. We release this as the **feature-based reference** that LLM systems should ensemble with.
+
+**Brier scores (vs market-implied baseline, 95% CI):**
+
+| Task | Model | Brier | CI | Δ vs baseline |
+|------|-------|------:|---|---:|
+| V1 × T1: direction 24h | LGBM 3-seed | 0.2295 | [0.2290, 0.2299] | **−0.0205** (vs coinflip 0.2500) |
+| V1 × T1: direction 24h | XGBoost 3-seed | 0.2296 | [0.2292, 0.2301] | −0.0204 |
+| V1 × T1: direction 24h | CatBoost 3-seed | 0.2295 | [0.2290, 0.2299] | −0.0205 |
+| **V1 × T1: direction 24h** | **Ensemble (3-model × 3-seed = 9)** | **0.2294** | [0.2289, 0.2299] | **−0.0206** |
+| V2 × T4: resolution 24h | XGBoost 3-seed | 0.1681 | [0.1605, 0.1759] | −0.0086 (vs price/100 = 0.1767) |
+
+Statistically significant (non-overlapping 95% CI) on V1 × T1 at 246,862 test samples.
+
+## Install
+
+```bash
+pip install lightgbm xgboost catboost numpy pandas
+```
+
+## Use
+
+```python
+from pathlib import Path
+from sf_ml_baseline import SFBaseline
+
+model = SFBaseline(weights_dir='sf-ml-baseline/weights')
+
+# Direction forecast: probability that the 24h-forward price will be HIGHER than now.
+# Features: current price (cents, 0-100), 24h price delta (cents),
+#           implicit yield (%), calibration ratio index (unitless), calibration variability ratio.
+p_up = model.predict_direction(price_cents=55, delta_cents=3, iy=12.5, cri=0.6, cvr=0.8)
+print(f'P(price rises in next 24h) = {p_up:.3f}')
+
+# Or batch-predict from a DataFrame:
+import pandas as pd
+df = pd.DataFrame([
+    {'price_cents': 55, 'delta_cents': 3, 'iy': 12.5, 'cri': 0.6, 'cvr': 0.8},
+    {'price_cents': 82, 'delta_cents': -1, 'iy': 4.5, 'cri': 0.3, 'cvr': 0.9},
+])
+probas = model.predict_direction_batch(df)
+```
+
+See `predict.py` for the full inference code.
+
+## Architecture
+
+- **Features (V1)**: `price_cents`, `delta_cents` (24h price change), `iy` (implicit yield), `cri` (calibration ratio index), `cvr` (calibration variability ratio). Spec: SimpleFunctions [indicator documentation](https://simplefunctions.dev/concepts).
+- **Models**: 3 LightGBM + 3 XGBoost + 3 CatBoost, each trained with different seeds. Ensemble predictions by simple mean.
+- **Split**: temporal — 80% train / 24h embargo / 20% test. 90/10 inner train/val split for early stopping.
+- **Target T1**: binary `sign(price(t+24h) - price(t))`, excludes no-move rows (delta==0 at t+24h).
+- **Target T4**: `resolved_outcome` ∈ {0, 1} for markets that resolved in 22-26h after the feature capture time.
+
+## Known limitations
+
+1. **Only 11 days of training data.** The full feature history table (`market_indicator_history`) was introduced to SF's data pipeline on 2026-04-08. A proper 30d+ / 180d+ re-train is scheduled; see "Retrain plan" below.
+2. **5 base features only.** `market_indicator_history` holds a compact subset of the full indicator stack. The live `market_indicators` table has ~20 features (`iyYes`, `iyNo`, `ee`, `las`, `vr`, `iar`, `rv`, `adjIy`, `cvrDelta`, `overround`, etc.) but only for the current snapshot. Future versions will store history for the full feature set.
+3. **`cvr` has 0% feature importance in the direction model.** Investigate whether the window/computation needs tuning.
+4. **V2 × T4 (rolling features + resolution label) is below the 0.01 Brier gate globally** — works well on Crypto (Δ=−0.041) and Commodities (Δ=−0.036) but not Sports (Δ=−0.006) or Financials (+0.004, model worse).
+5. **Do not use for live trading without backtesting against your own execution model.** This is a calibration baseline, not a PnL strategy.
+
+## Retrain plan
+
+**v0.1 is a time-capsule.** Planned retrains:
+
+| Version | Trigger | When | What changes |
+|---------|---------|------|------|
+| v0.2 | R2 dump archive ≥ 30d of indicator history | ~2026-05-20 | Same architecture, more data; per-category specialist models for Crypto/Commodities/Sports |
+| v0.3 | Full indicator feature set stored in history (schema change) | TBD | V1 grows from 5 → 20 features; re-run Phase A.3 full grid |
+| v1.0 | ≥ 6 months of R2 data | ~2026-10 | FT-Transformer + TabPFN + ensemble; formal paper submission (ICLR 2027 FinAI Workshop) |
+
+## Reproduce
+
+```bash
+git clone https://github.com/spfunctions/simplefunctions-landing  # (private — OSS mirror pending)
+cd simplefunctions-landing
+source scripts/ml/.venv/bin/activate
+
+# Data pull (uses DIRECT_DATABASE_URL in .env.local)
+python scripts/ml/phase-a/01-pull-training-data.py
+
+# Train
+python scripts/ml/phase-a/02-train-lgbm.py
+python scripts/ml/phase-a/04-bakeoff.py
+
+# Evaluate
+python scripts/ml/phase-a/03-evaluate.py
+```
+
+All hyperparameters are documented inline. 3-seed ensembling uses {42, 137, 2026}.
+
+## Citation
+
+```bibtex
+@software{sf_ml_baseline_v0_1,
+  author       = {SimpleFunctions},
+  title        = {{sf-ml-baseline}: A feature-based prediction-market forecaster},
+  year         = {2026},
+  version      = {0.1},
+  publisher    = {SimpleFunctions},
+  url          = {https://simplefunctions.dev/opensource/sf-ml-baseline},
+  license      = {CC-BY-4.0 with SimpleFunctions attribution}
+}
+```
+
+## See also
+
+- `docs/ml/phase-a-investigation.md` — 10 open-question investigation (SPEC-19 §13)
+- `docs/ml/phase-a-results.md` — gate decision + per-category breakdown
+- `.claude/specs/SPEC-19-model-deep-investigation.md` — full 6-phase research plan

sf_ml_baseline.py

@@ -0,0 +1,142 @@
+"""
+sf-ml-baseline v0.1 — inference helpers.
+
+Load pre-trained LightGBM + XGBoost + CatBoost ensembles and predict
+24h prediction-market outcomes from 5 engineered indicator features.
+
+Model weights are in `./weights/`. Licensed CC-BY-4.0 with attribution
+— see LICENSE.
+
+Example:
+    from sf_ml_baseline import SFBaseline
+    model = SFBaseline()
+    p_up = model.predict_direction(price_cents=55, delta_cents=3,
+                                   iy=12.5, cri=0.6, cvr=0.8)
+"""
+
+from __future__ import annotations
+from pathlib import Path
+from typing import Iterable, Sequence
+import numpy as np
+import pandas as pd
+import lightgbm as lgb
+import xgboost as xgb
+from catboost import CatBoostClassifier
+
+FEATURE_COLS_V1 = ('price_cents', 'delta_cents', 'iy', 'cri', 'cvr')
+SEEDS = (42, 137, 2026)
+
+class SFBaseline:
+    """9-model ensemble (3 architectures × 3 seeds) for 24h direction forecasting."""
+
+    def __init__(self, weights_dir: str | Path | None = None):
+        if weights_dir is None:
+            weights_dir = Path(__file__).parent / 'weights'
+        self.weights_dir = Path(weights_dir)
+        if not self.weights_dir.is_dir():
+            raise FileNotFoundError(f'weights dir not found: {self.weights_dir}')
+        self._load_t1_models()
+        self._load_t4_models()
+
+    def _load_t1_models(self):
+        """V1 × T1 — direction forecast."""
+        self.lgb_t1 = [lgb.Booster(model_file=str(self.weights_dir / f'lgbm_v1_t1_seed{s}.txt'))
+                       for s in SEEDS]
+        self.xgb_t1 = []
+        for s in SEEDS:
+            m = xgb.XGBClassifier()
+            m.load_model(str(self.weights_dir / f'xgb_v1_t1_seed{s}.json'))
+            self.xgb_t1.append(m)
+        self.cat_t1 = []
+        for s in SEEDS:
+            m = CatBoostClassifier()
+            m.load_model(str(self.weights_dir / f'cat_v1_t1_seed{s}.cbm'))
+            self.cat_t1.append(m)
+
+    def _load_t4_models(self):
+        """V2 × T4 — resolution forecast. Requires 35 features (price + rolling stats) —
+        see docs/ml/phase-a-results.md. This helper loads them but the user must
+        supply the full 35-feature vector (via the training-data pipeline)."""
+        try:
+            self.lgb_t4 = [lgb.Booster(model_file=str(self.weights_dir / f'lgbm_v2_t4_seed{s}.txt'))
+                           for s in SEEDS]
+            self.xgb_t4 = []
+            for s in SEEDS:
+                m = xgb.XGBClassifier()
+                m.load_model(str(self.weights_dir / f'xgb_v2_t4_seed{s}.json'))
+                self.xgb_t4.append(m)
+            self.cat_t4 = []
+            for s in SEEDS:
+                m = CatBoostClassifier()
+                m.load_model(str(self.weights_dir / f'cat_v2_t4_seed{s}.cbm'))
+                self.cat_t4.append(m)
+            self.t4_features = list(self.lgb_t4[0].feature_name())
+        except Exception:
+            self.lgb_t4 = self.xgb_t4 = self.cat_t4 = None
+            self.t4_features = None
+
+    # --- direction forecast (T1) ---
+
+    def predict_direction_batch(self, df: pd.DataFrame) -> np.ndarray:
+        """Return P(up-move in 24h) for each row in df.
+        df must have columns: price_cents, delta_cents, iy, cri, cvr."""
+        missing = [c for c in FEATURE_COLS_V1 if c not in df.columns]
+        if missing:
+            raise ValueError(f'missing feature columns: {missing}')
+        X = df[list(FEATURE_COLS_V1)].astype('float32').values
+        return self._predict_t1(X)
+
+    def predict_direction(
+        self,
+        price_cents: float,
+        delta_cents: float,
+        iy: float,
+        cri: float,
+        cvr: float,
+    ) -> float:
+        """Single-row prediction. Returns scalar probability."""
+        X = np.array([[price_cents, delta_cents, iy, cri, cvr]], dtype='float32')
+        return float(self._predict_t1(X)[0])
+
+    def _predict_t1(self, X: np.ndarray) -> np.ndarray:
+        """9-model ensemble: 3 LGBM + 3 XGB + 3 Cat, equal weight."""
+        preds = []
+        for m in self.lgb_t1:
+            preds.append(m.predict(X))
+        for m in self.xgb_t1:
+            preds.append(m.predict_proba(X)[:, 1])
+        for m in self.cat_t1:
+            preds.append(m.predict_proba(X)[:, 1])
+        return np.mean(preds, axis=0)
+
+    # --- resolution forecast (T4) ---
+
+    def predict_resolution_batch(self, df: pd.DataFrame) -> np.ndarray:
+        """Return P(YES resolution in 24h) for each row in df.
+        df must have all 35 V2 features (base 5 + rolling stats). See self.t4_features."""
+        if self.lgb_t4 is None:
+            raise RuntimeError('T4 weights not loaded')
+        missing = [c for c in self.t4_features if c not in df.columns]
+        if missing:
+            raise ValueError(f'missing V2 feature columns: {missing[:5]}... ({len(missing)} total)')
+        X = df[self.t4_features].astype('float32').values
+        preds = []
+        for m in self.lgb_t4:
+            preds.append(m.predict(X))
+        for m in self.xgb_t4:
+            preds.append(m.predict_proba(X)[:, 1])
+        for m in self.cat_t4:
+            preds.append(m.predict_proba(X)[:, 1])
+        return np.mean(preds, axis=0)
+
+
+if __name__ == '__main__':
+    # smoke test
+    model = SFBaseline()
+    p = model.predict_direction(price_cents=55, delta_cents=3, iy=12.5, cri=0.6, cvr=0.8)
+    print(f'P(price rises in 24h | market at 55c, +3c delta, iy=12.5%, cri=0.6, cvr=0.8) = {p:.3f}')
+    p_batch = model.predict_direction_batch(pd.DataFrame([
+        {'price_cents': 55, 'delta_cents': 3, 'iy': 12.5, 'cri': 0.6, 'cvr': 0.8},
+        {'price_cents': 82, 'delta_cents': -1, 'iy': 4.5, 'cri': 0.3, 'cvr': 0.9},
+    ]))
+    print(f'Batch predictions: {p_batch}')

weights/cat_v1_t1_seed137.cbm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51a0aa6b9fcc8e5ea6ec594e3eb396bde99b1966089cec9a07db5582a482cccc
+size 1394536

weights/cat_v1_t1_seed2026.cbm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03e6ce1ad122784e88b42409f404a22c11b9bf8ae40f740e3503cf70bd4d324e
+size 2134656

weights/cat_v1_t1_seed42.cbm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0f7d0074f0ff73e648148a1d43834abbe8884852717fb623db8d6048c574505
+size 1074240

weights/cat_v2_t4_seed137.cbm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0501566f668270f0e17a38934c422f0ef6e71cfbd24babb4de287d4527127a52
+size 418272

weights/cat_v2_t4_seed2026.cbm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73aef991e22dececf5bba619aef7f7304f8e641124e3616fcc34049c93c35291
+size 348656

weights/cat_v2_t4_seed42.cbm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6e168b8a1baa3ee018a20e1c4f1b07c493caccc019750b3705af57cfc480564
+size 454152