# c-trader ML Systems

Three production-grade ML subsystems for the c-trader algorithmic trading platform, integrating the **Aurea architecture** (orthogonal risk-integrated alpha) with passivity-preserving cross-impact theory.

## Architecture

```
Market Data ──┐
              ▼
┌─────────────────────────┐
│   1. FORECASTING LAYER  │  TimesFM 2.0 + Chronos-2 + Kronos-base
│   Regime-Gated Router   │  with regime detection & fallback chains
└──────────┬──────────────┘
           ▼
┌─────────────────────────┐
│   2. UNCERTAINTY PIPELINE│  ACI conformal prediction + ensemble
│   Calibrated σ̂          │  disagreement + quantile calibration
└──────────┬──────────────┘
           ▼
┌─────────────────────────┐
│   3. CROSS-IMPACT MODEL │  Stieltjes kernel calibration (SDP)
│   Passivity-Preserving  │  + execution optimizer (QP)
└──────────┬──────────────┘
           ▼
┌─────────────────────────┐
│   AUREA ALLOCATOR       │  edge - κ_u·σ̂ - friction
│   Risk Box + Execution  │  + overlap/turnover penalties + cash option
└─────────────────────────┘
```

## System 1: Forecasting Layer

**Module**: `ctrader_ml.forecasting`

Unified multi-model forecaster combining three time-series foundation models with intelligent regime-gated routing:

| Model | HF ID | When Used | Key Strength |
|---|---|---|---|
| **Kronos-base** | `NeoQuasar/Kronos-base` (102M) | Trending / High Volatility | Financial-native OHLCV, +93% RankIC vs generic TSFMs |
| **Chronos-2** | `amazon/chronos-2` (120M) | Regime Transitions / Covariates | SOTA zero-shot (90.7% win rate), 8192 context, multivariate |
| **TimesFM 2.0** | `google/timesfm-2.0-500m-pytorch` (500M) | General / Fallback | Fast point forecasts, 2048 context, PyTorch-native |
| **LightGBM/XGBoost** | Your existing models | Mean-reverting / Low Vol | Proven stronger than generic TSFMs on financial returns |

### Routing Logic (literature-backed)
- **arxiv:2511.18578** — generic TSFMs underperform tree ensembles on financial returns → keep LightGBM/XGBoost for mean-reverting regimes
- **arxiv:2508.02739** — Kronos: +93% RankIC on financial data → primary model for trending/volatile markets
- **arxiv:2510.15821** — Chronos-2: 90.7% win rate, group attention → best for multi-asset / regime transitions
- **arxiv:2508.02686** — MoE volatility-sensitive routing → gated ensemble architecture

### Regime Detector
Uses three orthogonal signals:
1. **Realized volatility** (rolling window) — classifies vol regime
2. **Hurst exponent** (R/S method) — trending (H>0.6) vs mean-reverting (H<0.4)
3. **Directional Movement Index** — trend strength and direction

## System 2: Calibrated Uncertainty Pipeline

**Module**: `ctrader_ml.uncertainty`

Produces σ̂ (the uncertainty penalty) for the Aurea allocator's scoring function:

```
net_utility_i = α̂_i − κ_u · σ̂_i − ĉ_i
```

### Three-component uncertainty:

1. **Ensemble Disagreement** (weight=0.30)
   - Normalized std across all model forecasts (TimesFM, Chronos-2, Kronos, LightGBM)
   - Inverse-MSE weighted by recent model performance

2. **Quantile Width** (weight=0.35)
   - Width of calibrated prediction intervals from Chronos-2 / Kronos MC samples
   - Online isotonic calibration corrects systematic quantile biases

3. **Conformal Prediction** (weight=0.35)
   - **Adaptive Conformal Inference (ACI)** from Gibbs & Candes (2021)
   - Tracks (1−α̂) quantile of rolling residual window
   - α̂ adapted online: drives empirical coverage to target (e.g., 90%)
   - Handles distribution shifts (regime changes) automatically

## System 3: Stieltjes Kernel + Execution Optimizer

**Module**: `ctrader_ml.cross_impact` + `ctrader_ml.execution`

### Cross-Impact Kernel Calibration

Fits the reduced passive cross-impact model:

```
G_r(t) = Σ_{k=1}^{K} A_k · exp(−ρ_k · t)
```

where each `A_k ≽ 0` (PSD) — **admissibility by construction** (no profitable round-trips).

**Calibration pipeline:**
1. Empirical kernel estimation from trades/quotes (lagged regression)
2. Scalar NNLS warmstart for decay rate initialization
3. Matrix SDP fitting with PSD constraints (cvxpy + SCS)
4. Alternating optimization: poles (L-BFGS-B) + residues (SDP)

### Passivity Test Suite
5 tests that MUST pass before deployment:
- PSD residue eigenvalues ≥ 0
- Kernel PSD at all time points
- Round-trip cost ≥ 0 (1000 random programs)
- Energy dissipation identity (state-space simulation)
- L¹ approximation error tracking

### Execution Optimizer
Multi-asset Almgren-Chriss QP with the passive kernel:
- **Block-QP formulation**: stacked trades V' Q V (DCP-compliant)
- Q is the Toeplitz block-kernel matrix — PSD by Stieltjes theory
- Constraints: participation limits, completion, spread guards
- Feeds friction estimates (ĉ_i) back to the Aurea allocator

## Integration: AureaBridge

**Module**: `ctrader_ml.integration.aurea_bridge`

Orchestrates the full cycle:

```python
from ctrader_ml.integration.aurea_bridge import AureaBridge

bridge = AureaBridge(
    device="cuda",
    prediction_horizon=24,
    n_assets=5,
    ensemble_mode=True,
    external_models={  # plug in existing c-trader models
        "lightgbm": your_lgbm_predict_fn,
        "xgboost": your_xgb_predict_fn,
    },
)

# One-time: calibrate cross-impact kernel
bridge.calibrate_kernel(signed_flow, price_changes, Sigma)

# Each trading cycle:
result = bridge.run_cycle(data_dict, Sigma, market_state)

# Access outputs:
result.forecasts["AAPL"]          # ForecastResult
result.uncertainties["AAPL"]       # UncertaintyEstimate (σ̂)
result.friction_estimates["AAPL"]  # ĉ
result.sleeve_scores["AAPL"]       # edge - κ_u·σ̂ - ĉ
result.execution_schedules["AAPL"] # optimal trade schedule

# Feedback loop (after prices realized):
bridge.update_with_realized("AAPL", y_true=152.5, y_predicted=153.0)
```

## Installation

```bash
# Core dependencies (always needed)
pip install numpy pandas scipy cvxpy[scs,osqp]

# For TimesFM 2.0
pip install timesfm[torch]

# For Chronos-2
pip install "chronos-forecasting>=2.0"

# For Kronos
git clone https://github.com/shiyu-coder/Kronos && cd Kronos && pip install -r requirements.txt
```

## Tests

```bash
cd /path/to/project
python ctrader_ml/tests/test_all_systems.py
# Expected: 31 passed, 0 failed, 0 skipped
```

## File Structure

```
ctrader_ml/
├── forecasting/
│   ├── regime_detector.py       # Hurst + ADX + vol regime classification
│   ├── model_registry.py        # Lazy-loading model management
│   └── unified_forecaster.py    # Regime-gated multi-model router
├── uncertainty/
│   └── calibrated_uncertainty.py  # ACI + ensemble + quantile calibration
├── cross_impact/
│   └── stieltjes_kernel.py      # SDP calibration + passivity tests
├── execution/
│   └── cross_impact_executor.py # Block-QP optimizer + friction estimator
├── integration/
│   └── aurea_bridge.py          # Full-cycle orchestrator + allocator scorer
├── utils/
│   └── types.py                 # Shared data types and enums
└── tests/
    └── test_all_systems.py      # 31-test comprehensive suite
```

## Key References

| Paper | What It Provides |
|---|---|
| [arxiv:2511.18578](https://arxiv.org/abs/2511.18578) | Generic TSFMs underperform tree ensembles on financial returns |
| [arxiv:2508.02739](https://arxiv.org/abs/2508.02739) | Kronos: financial-native OHLCV foundation model |
| [arxiv:2510.15821](https://arxiv.org/abs/2510.15821) | Chronos-2: SOTA zero-shot with group attention |
| [arxiv:2310.10688](https://arxiv.org/abs/2310.10688) | TimesFM: decoder-only time-series foundation model |
| [arxiv:2508.02686](https://arxiv.org/abs/2508.02686) | MoE volatility-sensitive routing |
| Gibbs & Candes (2021) | Adaptive Conformal Inference (ACI) |
| Almgren & Chriss (2000) | Optimal execution with market impact |
| Uploaded study | Passivity-preserving cross-impact reduction |
| Aurea transcript | Orthogonal Risk Integrated Alpha architecture |