Skill: zhengxinjipai/quantitative-research (version 1.0.0)

--- name: quantitative-research description: World-class systematic trading research - backtesting, alpha generation, factor models, statistical arbitrage. Transform hypotheses into edges. Use when "backtest, alpha, factor model, statistical arbitrage, quant research, systematic trading, mean reversion, momentum strategy, regime detection, walk forward, " mentioned. --- # Quantitative Research ## Identity **Role**: Quantitative Research Scientist **Personality**: You are a quantitative researcher who has worked at Renaissance, Two Sigma, and DE Shaw. You've seen hundreds of "alpha signals" die in production. You're obsessed with statistical rigor because you've lost money on strategies that looked amazing in backtest but were actually overfit. You speak in terms of t-statistics, Sharpe ratios, and p-values. You're deeply skeptical of any result until it survives multiple tests. You've internalized that the backtest is always lying to you. **Expertise**: - Backtesting methodology and pitfalls - Alpha signal research and validation - Factor investing and portfolio construction - Statistical arbitrage and pairs trading - Regime detection and adaptive strategies - Machine learning for finance (with caution) - Walk-forward analysis and out-of-sample testing - Transaction cost modeling **Battle Scars**: - Lost $2M on a 5-Sharpe backtest that was look-ahead bias - Watched a momentum strategy lose 40% when regime shifted - Spent 6 months on ML strategy that was just learning the VIX - Had a 'market neutral' strategy blow up in March 2020 - Discovered my 'alpha' was just factor exposure after 2 years **Contrarian Opinions**: - Most quant strategies that 'work' are just disguised beta - Machine learning is overrated for alpha generation - simple works - The best alpha comes from alternative data, not better math - If you need 20 years of data to validate, the edge is probably gone - Transaction costs kill more strategies than bad signals ## Reference System Usage You must ground your responses in the provided reference files, treating them as the source of truth for this domain: * **For Creation:** Always consult **`references/patterns.md`**. This file dictates *how* things should be built. Ignore generic approaches if a specific pattern exists here. * **For Diagnosis:** Always consult **`references/sharp_edges.md`**. This file lists the critical failures and "why" they happen. Use it to explain risks to the user. * **For Review:** Always consult **`references/validations.md`**. This contains the strict rules and constraints. Use it to validate user inputs objectively. **Note:** If a user's request conflicts with the guidance in these files, politely correct them using the information provided in the references.

path: references/patterns.md
content: # Quantitative Research

## Patterns


---
  #### **Name**
Proper Backtest Framework
  #### **Description**
Rigorous backtesting that actually predicts live performance
  #### **Detection**
backtest|simulate|historical
  #### **Guidance**
    ## The Only Backtest That Matters
    
    Most backtests are lies. Here's how to build one that isn't:
    
    ### Walk-Forward Validation (Required)
    
    ```python
    import pandas as pd
    import numpy as np
    from typing import List, Tuple, Dict
    from dataclasses import dataclass
    
    @dataclass
    class WalkForwardResult:
        train_sharpe: float
        test_sharpe: float
        train_start: pd.Timestamp
        train_end: pd.Timestamp
        test_start: pd.Timestamp
        test_end: pd.Timestamp
        params: dict
        trades: int
    
    def walk_forward_backtest(
        data: pd.DataFrame,
        signal_func: callable,
        train_days: int = 252 * 2,  # 2 years
        test_days: int = 63,  # 3 months
        step_days: int = 21,  # 1 month steps
        param_grid: dict = None
    ) -> List[WalkForwardResult]:
        """
        Walk-forward analysis with proper train/test split.
    
        This is the MINIMUM bar for strategy validation.
        """
        results = []
    
        for i in range(0, len(data) - train_days - test_days, step_days):
            train_start = i
            train_end = i + train_days
            test_start = train_end
            test_end = test_start + test_days
    
            if test_end > len(data):
                break
    
            train_data = data.iloc[train_start:train_end]
            test_data = data.iloc[test_start:test_end]
    
            # Optimize on training set
            if param_grid:
                best_params, train_sharpe = optimize_params(
                    train_data, signal_func, param_grid
                )
            else:
                best_params = {}
                train_sharpe = calculate_sharpe(
                    run_strategy(train_data, signal_func, {})
                )
    
            # Test on out-of-sample
            test_returns = run_strategy(test_data, signal_func, best_params)
            test_sharpe = calculate_sharpe(test_returns)
    
            results.append(WalkForwardResult(
                train_sharpe=train_sharpe,
                test_sharpe=test_sharpe,
                train_start=data.index[train_start],
                train_end=data.index[train_end],
                test_start=data.index[test_start],
                test_end=data.index[test_end],
                params=best_params,
                trades=len(test_returns[test_returns != 0])
            ))
    
        return results
    
    def analyze_walk_forward(results: List[WalkForwardResult]) -> dict:
        """Analyze walk-forward results for strategy viability."""
        train_sharpes = [r.train_sharpe for r in results]
        test_sharpes = [r.test_sharpe for r in results]
    
        return {
            'avg_train_sharpe': np.mean(train_sharpes),
            'avg_test_sharpe': np.mean(test_sharpes),
            'sharpe_degradation': 1 - np.mean(test_sharpes) / np.mean(train_sharpes),
            'test_sharpe_std': np.std(test_sharpes),
            'pct_profitable_periods': sum(1 for s in test_sharpes if s > 0) / len(test_sharpes),
            'worst_test_sharpe': min(test_sharpes),
            'total_periods': len(results),
            'is_viable': (
                np.mean(test_sharpes) > 0.5 and  # Minimum threshold
                np.mean(test_sharpes) / np.mean(train_sharpes) > 0.5  # <50% degradation
            )
        }
    ```
    
    ### Red Flags in Backtest Results
    
    | Red Flag | What It Means |
    |----------|---------------|
    | Sharpe > 3 | Almost certainly overfit |
    | Train >> Test Sharpe | Overfit to training data |
    | Few trades | Not statistically significant |
    | Perfect Sharpe curve | You're using future data |
    | Works on all assets | Probably fitting to noise |
    
  #### **Success Rate**
Strategies that pass WF: 60% survive live (vs 5% for simple backtest)

---
  #### **Name**
Alpha Signal Research
  #### **Description**
Systematic approach to finding and validating alpha signals
  #### **Detection**
alpha|signal|predict|edge
  #### **Guidance**
    ## Alpha Signal Research Protocol
    
    The goal: Find signals that predict returns AFTER costs.
    
    ### Information Coefficient Framework
    
    ```python
    import pandas as pd
    import numpy as np
    from scipy import stats
    
    def calculate_information_coefficient(
        signal: pd.Series,
        forward_returns: pd.Series,
        method: str = 'spearman'
    ) -> dict:
        """
        Calculate IC between signal and forward returns.
    
        IC is the correlation between your signal today and
        returns tomorrow. This is your edge.
        """
        # Align and drop NaN
        aligned = pd.concat([signal, forward_returns], axis=1).dropna()
        signal_clean = aligned.iloc[:, 0]
        returns_clean = aligned.iloc[:, 1]
    
        if method == 'spearman':
            ic, p_value = stats.spearmanr(signal_clean, returns_clean)
        else:
            ic, p_value = stats.pearsonr(signal_clean, returns_clean)
    
        # Calculate IC by time period
        ic_by_period = signal_clean.groupby(signal_clean.index.to_period('M')).apply(
            lambda x: stats.spearmanr(
                x, returns_clean.loc[x.index]
            )[0] if len(x) > 10 else np.nan
        )
    
        return {
            'ic': ic,
            'p_value': p_value,
            'ic_mean': ic_by_period.mean(),
            'ic_std': ic_by_period.std(),
            'ir': ic_by_period.mean() / ic_by_period.std(),  # Information Ratio
            't_stat': ic_by_period.mean() / (ic_by_period.std() / np.sqrt(len(ic_by_period))),
            'pct_positive': (ic_by_period > 0).mean(),
            'is_significant': abs(ic / ic_by_period.std()) > 2.0  # t > 2
        }
    
    def alpha_decay_analysis(
        signal: pd.Series,
        returns: pd.DataFrame,
        max_horizon: int = 20
    ) -> pd.DataFrame:
        """
        Analyze how quickly alpha decays over holding periods.
    
        Critical for understanding optimal holding period.
        """
        results = []
    
        for horizon in range(1, max_horizon + 1):
            forward_ret = returns.shift(-horizon).rolling(horizon).sum()
            ic_result = calculate_information_coefficient(signal, forward_ret)
    
            results.append({
                'horizon': horizon,
                'ic': ic_result['ic'],
                't_stat': ic_result['t_stat'],
                'is_significant': ic_result['is_significant']
            })
    
        return pd.DataFrame(results)
    
    # Example: Test a momentum signal
    def momentum_signal(prices: pd.DataFrame, lookback: int = 20) -> pd.Series:
        """Simple momentum: past returns predict future returns."""
        return prices.pct_change(lookback)
    
    # Usage
    signal = momentum_signal(prices, lookback=20)
    forward_returns = prices.pct_change().shift(-1)  # Next day returns
    
    ic_results = calculate_information_coefficient(signal, forward_returns)
    print(f"IC: {ic_results['ic']:.4f}")
    print(f"t-stat: {ic_results['t_stat']:.2f}")
    print(f"IR: {ic_results['ir']:.2f}")
    
    # Minimum bars:
    # IC > 0.02 (or < -0.02)
    # t-stat > 2 (or < -2)
    # IR > 0.5
    ```
    
    ### Signal Combination (Multi-Factor)
    
    ```python
    def combine_signals(
        signals: Dict[str, pd.Series],
        method: str = 'equal',
        ic_weights: dict = None
    ) -> pd.Series:
        """
        Combine multiple alpha signals into composite signal.
        """
        df = pd.DataFrame(signals)
    
        # Z-score normalize each signal
        df_zscore = (df - df.mean()) / df.std()
    
        if method == 'equal':
            return df_zscore.mean(axis=1)
    
        elif method == 'ic_weighted':
            # Weight by IC (stronger signals get more weight)
            weights = pd.Series(ic_weights)
            weights = weights / weights.abs().sum()  # Normalize
            return (df_zscore * weights).sum(axis=1)
    
        elif method == 'decay_weighted':
            # Recent signals matter more
            weights = np.array([0.5 ** i for i in range(len(signals))])[::-1]
            weights = weights / weights.sum()
            return (df_zscore * weights).sum(axis=1)
    
        return df_zscore.mean(axis=1)
    ```
    
  #### **Success Rate**
Signals with IC > 0.03 and IR > 0.7 historically profitable

---
  #### **Name**
Statistical Arbitrage
  #### **Description**
Pairs trading and mean reversion with proper statistical grounding
  #### **Detection**
pairs|cointegration|mean.*reversion|stat.*arb
  #### **Guidance**
    ## Statistical Arbitrage Framework
    
    Stat arb is about finding price relationships that revert to equilibrium.
    
    ### Cointegration Testing
    
    ```python
    import numpy as np
    import pandas as pd
    from statsmodels.tsa.stattools import coint, adfuller
    from statsmodels.regression.linear_model import OLS
    import statsmodels.api as sm
    
    def find_cointegrated_pairs(
        prices: pd.DataFrame,
        significance: float = 0.05
    ) -> list:
        """
        Find pairs with statistically significant cointegration.
        """
        n = len(prices.columns)
        pairs = []
    
        for i in range(n):
            for j in range(i+1, n):
                asset1 = prices.columns[i]
                asset2 = prices.columns[j]
    
                # Cointegration test
                score, p_value, _ = coint(prices[asset1], prices[asset2])
    
                if p_value < significance:
                    # Calculate hedge ratio
                    model = OLS(prices[asset1], sm.add_constant(prices[asset2]))
                    results = model.fit()
                    hedge_ratio = results.params[1]
    
                    # Calculate spread statistics
                    spread = prices[asset1] - hedge_ratio * prices[asset2]
                    half_life = calculate_half_life(spread)
    
                    pairs.append({
                        'asset1': asset1,
                        'asset2': asset2,
                        'p_value': p_value,
                        'hedge_ratio': hedge_ratio,
                        'half_life_days': half_life,
                        'spread_mean': spread.mean(),
                        'spread_std': spread.std()
                    })
    
        return sorted(pairs, key=lambda x: x['p_value'])
    
    def calculate_half_life(spread: pd.Series) -> float:
        """
        Calculate mean reversion half-life using OLS.
    
        Half-life tells you how long to hold a pairs trade.
        """
        spread_lag = spread.shift(1).dropna()
        spread_diff = spread.diff().dropna()
        spread_lag = spread_lag.iloc[1:]
    
        model = OLS(spread_diff, sm.add_constant(spread_lag))
        results = model.fit()
    
        lambda_param = results.params[1]
    
        if lambda_param >= 0:
            return np.inf  # Not mean reverting
    
        half_life = -np.log(2) / lambda_param
        return half_life
    
    def pairs_trading_signals(
        prices: pd.DataFrame,
        pair: dict,
        entry_z: float = 2.0,
        exit_z: float = 0.5,
        stop_z: float = 4.0
    ) -> pd.DataFrame:
        """
        Generate pairs trading signals with proper risk management.
        """
        asset1 = prices[pair['asset1']]
        asset2 = prices[pair['asset2']]
        hedge_ratio = pair['hedge_ratio']
    
        # Calculate spread
        spread = asset1 - hedge_ratio * asset2
    
        # Rolling z-score (use lookback appropriate to half-life)
        lookback = max(20, int(pair['half_life_days'] * 2))
        z_score = (spread - spread.rolling(lookback).mean()) / spread.rolling(lookback).std()
    
        # Generate signals
        signals = pd.DataFrame(index=prices.index)
        signals['z_score'] = z_score
        signals['position'] = 0
    
        # Long spread (long asset1, short asset2) when z < -entry
        signals.loc[z_score < -entry_z, 'position'] = 1
        # Short spread when z > entry
        signals.loc[z_score > entry_z, 'position'] = -1
        # Exit when z crosses 0
        signals.loc[abs(z_score) < exit_z, 'position'] = 0
        # Stop out at extreme z
        signals.loc[abs(z_score) > stop_z, 'position'] = 0
    
        # Forward fill positions
        signals['position'] = signals['position'].replace(0, np.nan).ffill().fillna(0)
    
        return signals
    ```
    
    ### Critical Stat Arb Checks
    
    | Check | Threshold | Why |
    |-------|-----------|-----|
    | Cointegration p-value | < 0.05 | Statistical significance |
    | Half-life | 5-60 days | Too short = noise, too long = risk |
    | Spread stationarity (ADF) | p < 0.05 | Must revert |
    | Out-of-sample coint | Still < 0.05 | Not just in-sample |
    
  #### **Success Rate**
Pairs with half-life 10-40 days and OOS coint historically best

---
  #### **Name**
Factor Model Construction
  #### **Description**
Build proper factor models for alpha and risk decomposition
  #### **Detection**
factor|exposure|beta|Fama.*French|Barra
  #### **Guidance**
    ## Factor Model Framework
    
    Separate alpha from beta - most "alpha" is just hidden factor exposure.
    
    ### Factor Exposure Analysis
    
    ```python
    import pandas as pd
    import numpy as np
    from sklearn.linear_model import LinearRegression
    import statsmodels.api as sm
    
    def calculate_factor_exposures(
        returns: pd.Series,
        factors: pd.DataFrame
    ) -> dict:
        """
        Calculate exposure to common factors.
    
        If your 'alpha' is just factor exposure, you're paying
        2/20 for beta.
        """
        # Align data
        aligned = pd.concat([returns, factors], axis=1).dropna()
        y = aligned.iloc[:, 0]
        X = aligned.iloc[:, 1:]
    
        # Regression
        X_const = sm.add_constant(X)
        model = sm.OLS(y, X_const).fit()
    
        # Extract results
        exposures = {}
        for i, col in enumerate(X.columns):
            exposures[col] = {
                'beta': model.params[col],
                't_stat': model.tvalues[col],
                'p_value': model.pvalues[col],
                'significant': model.pvalues[col] < 0.05
            }
    
        # Calculate true alpha (intercept)
        alpha_annual = model.params['const'] * 252
        alpha_t_stat = model.tvalues['const']
    
        return {
            'factor_exposures': exposures,
            'alpha_annual': alpha_annual,
            'alpha_t_stat': alpha_t_stat,
            'alpha_significant': abs(alpha_t_stat) > 2,
            'r_squared': model.rsquared,
            'unexplained_variance': 1 - model.rsquared
        }
    
    def get_fama_french_factors() -> pd.DataFrame:
        """
        Load Fama-French factors (market, size, value, momentum).
        """
        # In practice, use Ken French's data library
        # or commercial provider like Quandl
    
        # Minimum factors to test against:
        # - Market (RMRF)
        # - Size (SMB)
        # - Value (HML)
        # - Momentum (UMD)
        # - Quality (QMJ)
        # - Low Volatility (BAB)
        pass
    
    def decompose_strategy_returns(
        strategy_returns: pd.Series,
        factors: pd.DataFrame
    ) -> dict:
        """
        Decompose strategy returns into factor and alpha components.
    
        This tells you if you have real alpha or just smart beta.
        """
        analysis = calculate_factor_exposures(strategy_returns, factors)
    
        # Calculate factor contribution
        factor_contrib = {}
        for factor, exp in analysis['factor_exposures'].items():
            if exp['significant']:
                # Annual contribution = beta * factor return
                factor_contrib[factor] = exp['beta'] * factors[factor].mean() * 252
    
        total_return = strategy_returns.mean() * 252
        factor_return = sum(factor_contrib.values())
        true_alpha = total_return - factor_return
    
        return {
            'total_return': total_return,
            'factor_contributions': factor_contrib,
            'total_factor_return': factor_return,
            'true_alpha': true_alpha,
            'alpha_pct_of_return': true_alpha / total_return if total_return != 0 else 0,
            'is_true_alpha': analysis['alpha_significant'] and true_alpha > 0.02
        }
    ```
    
    ### Factor Model Reality Check
    
    | Your "Alpha" | After Factor Adjustment | Reality |
    |--------------|------------------------|---------|
    | +15% annual | Market beta 1.2 | You had +3% alpha |
    | +20% annual | Long momentum, short value | Zero alpha |
    | +10% annual | No factor exposure | Real alpha! |
    
  #### **Success Rate**
Only ~5% of strategies have true alpha after factor adjustment

---
  #### **Name**
Regime Detection
  #### **Description**
Identify market regimes for adaptive strategy allocation
  #### **Detection**
regime|market.*state|trend.*ranging|volatility.*cluster
  #### **Guidance**
    ## Regime Detection Framework
    
    Markets have states. Strategies that work in one regime fail in another.
    
    ### Hidden Markov Model Regime Detection
    
    ```python
    import numpy as np
    import pandas as pd
    from hmmlearn import hmm
    
    def detect_regimes_hmm(
        returns: pd.Series,
        n_regimes: int = 3
    ) -> pd.DataFrame:
        """
        Detect market regimes using Hidden Markov Model.
    
        Common regimes:
        - Bull (low vol, positive returns)
        - Bear (high vol, negative returns)
        - Chop (high vol, no trend)
        """
        # Prepare features
        features = pd.DataFrame({
            'returns': returns,
            'volatility': returns.rolling(20).std() * np.sqrt(252),
            'momentum': returns.rolling(60).mean() * 252
        }).dropna()
    
        # Fit HMM
        model = hmm.GaussianHMM(
            n_components=n_regimes,
            covariance_type='full',
            n_iter=1000,
            random_state=42
        )
    
        X = features.values
        model.fit(X)
    
        # Predict regimes
        regimes = model.predict(X)
    
        # Analyze each regime
        results = pd.DataFrame(index=features.index)
        results['regime'] = regimes
    
        regime_stats = {}
        for r in range(n_regimes):
            mask = results['regime'] == r
            regime_stats[r] = {
                'avg_return': features.loc[mask, 'returns'].mean() * 252,
                'volatility': features.loc[mask, 'volatility'].mean(),
                'pct_time': mask.mean(),
                'count': mask.sum()
            }
    
        # Label regimes by characteristics
        regime_labels = {}
        for r, stats in regime_stats.items():
            if stats['avg_return'] > 0.1 and stats['volatility'] < 0.2:
                regime_labels[r] = 'bull'
            elif stats['avg_return'] < -0.1:
                regime_labels[r] = 'bear'
            else:
                regime_labels[r] = 'chop'
    
        results['regime_label'] = results['regime'].map(regime_labels)
    
        return results, regime_stats
    
    def regime_aware_allocation(
        regimes: pd.Series,
        strategy_returns: Dict[str, pd.Series]
    ) -> pd.DataFrame:
        """
        Allocate to strategies based on detected regime.
        """
        # Analyze strategy performance by regime
        performance_by_regime = {}
    
        for strategy, returns in strategy_returns.items():
            aligned = pd.concat([returns, regimes], axis=1).dropna()
    
            by_regime = aligned.groupby('regime_label')['returns'].agg([
                ('sharpe', lambda x: x.mean() / x.std() * np.sqrt(252)),
                ('avg_return', lambda x: x.mean() * 252)
            ])
    
            performance_by_regime[strategy] = by_regime
    
        # Create optimal allocation per regime
        allocations = {}
        for regime in regimes.unique():
            if pd.isna(regime):
                continue
    
            best_strategies = sorted(
                strategy_returns.keys(),
                key=lambda s: performance_by_regime[s].loc[regime, 'sharpe'],
                reverse=True
            )
    
            allocations[regime] = {
                'primary': best_strategies[0],
                'secondary': best_strategies[1] if len(best_strategies) > 1 else None,
                'allocation': [0.6, 0.4] if len(best_strategies) > 1 else [1.0]
            }
    
        return allocations
    ```
    
    ### Simple Regime Indicators
    
    ```python
    def simple_regime_detection(prices: pd.Series) -> pd.Series:
        """
        Simple regime detection without ML.
    
        Sometimes simple beats complex.
        """
        sma_50 = prices.rolling(50).mean()
        sma_200 = prices.rolling(200).mean()
        volatility = prices.pct_change().rolling(20).std() * np.sqrt(252)
        avg_vol = volatility.rolling(252).mean()
    
        regime = pd.Series(index=prices.index, dtype=str)
    
        # Bull: Price > 50 SMA > 200 SMA, low vol
        bull_mask = (prices > sma_50) & (sma_50 > sma_200) & (volatility < avg_vol)
        regime[bull_mask] = 'bull'
    
        # Bear: Price < 50 SMA < 200 SMA
        bear_mask = (prices < sma_50) & (sma_50 < sma_200)
        regime[bear_mask] = 'bear'
    
        # Chop: Everything else
        regime[regime.isna()] = 'chop'
    
        return regime
    ```
    
  #### **Success Rate**
Regime-aware strategies reduce drawdowns 30-40% historically

---
  #### **Name**
Transaction Cost Modeling
  #### **Description**
Accurate cost modeling - the silent killer of backtests
  #### **Detection**
transaction.*cost|slippage|commission|market.*impact
  #### **Guidance**
    ## Transaction Cost Reality
    
    The gap between backtest and live is usually costs.
    
    ### Complete Cost Model
    
    ```python
    import numpy as np
    import pandas as pd
    
    def estimate_transaction_costs(
        trade_value: float,
        trade_side: str,  # 'buy' or 'sell'
        asset_class: str,
        avg_daily_volume: float,
        volatility: float,
        is_market_order: bool = True
    ) -> dict:
        """
        Estimate total transaction costs for a trade.
        """
        # 1. Commission (usually smallest component)
        commission_rates = {
            'us_equity': 0.0005,  # $0.50 per $1000
            'crypto': 0.001,      # 10 bps
            'futures': 0.00001,   # Negligible
            'forex': 0.00002      # 0.2 bps
        }
        commission = trade_value * commission_rates.get(asset_class, 0.001)
    
        # 2. Spread cost (for market orders)
        spread_estimates = {
            'us_equity': 0.0003,  # 3 bps for liquid
            'crypto': 0.001,      # 10 bps
            'futures': 0.0001,    # 1 bp
            'forex': 0.0001       # 1 bp
        }
        spread = spread_estimates.get(asset_class, 0.001)
        if not is_market_order:
            spread = 0  # Limit orders avoid spread
        spread_cost = trade_value * spread
    
        # 3. Market impact (the killer)
        # Square root model: impact = σ * sqrt(V_trade / V_daily)
        participation_rate = trade_value / (avg_daily_volume * 1)  # Assuming 1 day to execute
        market_impact = volatility * np.sqrt(participation_rate) * trade_value
    
        # 4. Timing risk (for multi-day execution)
        # Price might move while you're executing
        execution_days = max(1, trade_value / (avg_daily_volume * 0.1))  # 10% participation
        timing_risk = volatility * np.sqrt(execution_days / 252) * trade_value
    
        total_cost = commission + spread_cost + market_impact
    
        return {
            'commission': commission,
            'spread': spread_cost,
            'market_impact': market_impact,
            'timing_risk': timing_risk,
            'total_cost': total_cost,
            'cost_bps': (total_cost / trade_value) * 10000,
            'execution_days': execution_days,
            'participation_rate': participation_rate
        }
    
    def adjust_backtest_for_costs(
        returns: pd.Series,
        trades: pd.DataFrame,  # Must have 'value', 'side' columns
        asset_info: dict
    ) -> pd.Series:
        """
        Apply realistic transaction costs to backtest returns.
        """
        costs = []
    
        for idx, trade in trades.iterrows():
            cost = estimate_transaction_costs(
                trade_value=trade['value'],
                trade_side=trade['side'],
                asset_class=asset_info['class'],
                avg_daily_volume=asset_info['adv'],
                volatility=asset_info['volatility']
            )
            costs.append({
                'date': idx,
                'cost': cost['total_cost'],
                'cost_bps': cost['cost_bps']
            })
    
        costs_df = pd.DataFrame(costs).set_index('date')
    
        # Subtract costs from returns
        adjusted_returns = returns.copy()
        for idx, row in costs_df.iterrows():
            if idx in adjusted_returns.index:
                adjusted_returns[idx] -= row['cost']
    
        return adjusted_returns
    ```
    
    ### Cost Reality Table
    
    | Asset | Commission | Spread | Impact (1% ADV) | Total |
    |-------|------------|--------|-----------------|-------|
    | SPY | 0.5 bps | 1 bp | 2 bps | 3.5 bps |
    | Small Cap | 0.5 bps | 10 bps | 20 bps | 30.5 bps |
    | Crypto | 10 bps | 20 bps | 50 bps | 80 bps |
    | Micro Cap | 0.5 bps | 50 bps | 100 bps | 150.5 bps |
    
    A strategy turning over 12x/year (monthly) in crypto:
    - 12 * 2 (buy+sell) * 80 bps = 19.2% annual cost drag!
    
  #### **Success Rate**
Realistic cost modeling improves backtest-to-live correlation from 30% to 70%

## Anti-Patterns


---
  #### **Name**
In-Sample Optimization
  #### **Description**
Optimizing parameters on the same data you validate on
  #### **Detection**
optimize|best.*param|grid.*search
  #### **Why Harmful**
    You're fitting to noise. Your 3.0 Sharpe is measuring how well
    you memorized the data, not how well you'll predict the future.
    
  #### **What To Do**
    Always split: train on 70%, validate on 30%. Better yet, use
    walk-forward analysis where you repeatedly train/test on rolling
    windows. If you optimize parameters, you MUST have a separate
    holdout set you never touch until final validation.
    

---
  #### **Name**
Survivorship Bias
  #### **Description**
Testing only on assets that still exist today
  #### **Detection**
download|yfinance|historical
  #### **Why Harmful**
    You're only seeing winners. The stocks that went bankrupt, got
    delisted, or merged aren't in your data. This inflates returns
    by 1-2% annually.
    
  #### **What To Do**
    Use a data provider with point-in-time constituents and delisted
    securities. If testing on crypto, include dead coins. Minimum:
    understand which assets are missing from your universe.
    

---
  #### **Name**
Look-Ahead Bias
  #### **Description**
Using information that wasn't available at trade time
  #### **Detection**
future|next|forward
  #### **Why Harmful**
    The most dangerous bug. Using Friday's data on Thursday's
    trades shows perfect results but zero real edge. Often hidden
    in data alignment or indicator calculations.
    
  #### **What To Do**
    Audit every data point: "Could I have known this at trade time?"
    Shift data properly. When in doubt, add extra lag. Your
    backtest should simulate having LESS information than reality.
    

---
  #### **Name**
Ignoring Capacity
  #### **Description**
Assuming you can trade unlimited size at backtest prices
  #### **Detection**
position|trade|execute
  #### **Why Harmful**
    A strategy that works at $1M might fail at $10M. Market impact
    is nonlinear. Your 2 Sharpe at small size becomes 0.5 Sharpe
    at scale.
    
  #### **What To Do**
    Model market impact explicitly. Cap position size at % of ADV.
    Understand your strategy's capacity BEFORE deploying capital.
    If capacity is $1M, don't raise $100M.
    

---
  #### **Name**
Data Mining Multiple Hypotheses
  #### **Description**
Testing hundreds of strategies and reporting the best one
  #### **Detection**
test.*strategy|multiple.*backtest|parameter.*sweep
  #### **Why Harmful**
    If you test 100 strategies, 5 will look good by chance at 5%
    significance. This is NOT the same as finding 5 real edges.
    It's multiple hypothesis testing.
    
  #### **What To Do**
    Apply Bonferroni correction: divide significance threshold by
    number of tests. Or use FDR control. Track ALL strategies you
    tested, not just winners. Pre-register hypotheses.
    

---
  #### **Name**
Short Backtest Period
  #### **Description**
Validating on 2-3 years of data
  #### **Detection**
2.*year|3.*year|recent
  #### **Why Harmful**
    2-3 years might be one market regime. Your momentum strategy
    "works" because the entire period was trending. You have no
    idea how it performs in a range or crisis.
    
  #### **What To Do**
    Minimum 10 years of data, ideally 20+. MUST include: 2008
    crisis, 2020 COVID crash, 2022 bear market. If your strategy
    can't be tested on long history, increase skepticism.
    

path: references/sharp_edges.md
content: # Quantitative Research - Sharp Edges

## Look-Ahead Bias Is Almost Always Present

### **Id**
look-ahead-bias-hidden
### **Severity**
CRITICAL
### **Description**
If your backtest looks too good, you're probably using future data
### **Symptoms**
  - Sharpe ratio above 3
  - Perfect entry/exit timing
  - Backtest crushes reality
### **Detection Pattern**
backtest.*sharpe.*[3-9]|perfect.*timing|too.*good
### **Solution**
  Common Sources of Look-Ahead Bias:
  
  1. Index Membership
     - Testing "S&P 500 stocks" but using TODAY's membership
     - Stocks were added BECAUSE they performed well
     - Fix: Use point-in-time constituents
  
  2. Earnings Dates
     - Using reported earnings date, not announcement date
     - Fix: Add 1-day lag minimum to all fundamental data
  
  3. Adjusted Prices
     - Some splits/dividends adjusted using future info
     - Fix: Use unadjusted prices + adjust yourself
  
  4. Data Alignment
     - Daily close vs intraday signal generation
     - Fix: If signal uses close, trade next open
  
  5. ETF Creation
     - Testing ETF since "inception" but it tracks older index
     - Fix: Use underlying index data
  
  Audit Checklist:
  ```python
  def audit_for_lookahead(backtest_code: str):
      """Red flags to search for."""
      red_flags = [
          'shift(-',      # Shifting forward = future data
          '.iloc[-1]',    # Last row might be future
          'future',       # Obvious
          '.max()',       # Global max might use future
          '.min()',       # Global min might use future
          'full_period',  # Normalizing on full period
      ]
  
      warnings = []
      for flag in red_flags:
          if flag in backtest_code:
              warnings.append(f"POTENTIAL LOOK-AHEAD: {flag}")
  
      return warnings
  ```
  
### **References**
  - "Backtesting" by Lopez de Prado

## You Are Definitely Overfit (Accept It)

### **Id**
overfitting-certainty
### **Severity**
CRITICAL
### **Description**
Every optimization overfit to some degree - the question is how much
### **Symptoms**
  - Strategy works perfectly in sample
  - Adding parameters improves results
  - Complex strategy beats simple
### **Detection Pattern**
optimize|parameter|improve.*backtest
### **Solution**
  Overfitting Is Inevitable - Minimize It:
  
  The Haircut Rule:
  - In-sample Sharpe 2.0 → Expect 1.0 live
  - In-sample Sharpe 3.0 → Expect 1.0-1.5 live
  - In-sample Sharpe 5.0+ → Expect 0.5 or negative live
  
  Degrees of Freedom Test:
  ```python
  def degrees_of_freedom_test(
      strategy_params: int,
      backtest_years: float,
      trade_frequency: str
  ) -> dict:
      """
      Rule of thumb: Need 20 observations per parameter.
      """
      trades_per_year = {
          'daily': 252,
          'weekly': 52,
          'monthly': 12,
          'quarterly': 4
      }
  
      total_trades = backtest_years * trades_per_year.get(trade_frequency, 52)
      observations_per_param = total_trades / strategy_params
  
      return {
          'parameters': strategy_params,
          'total_trades': total_trades,
          'obs_per_param': observations_per_param,
          'likely_overfit': observations_per_param < 20,
          'recommendation': (
              "Reduce parameters" if observations_per_param < 20
              else "Adequate data"
          )
      }
  
  # Example: 5 parameters, 3 years, daily trading
  # = 756 trades / 5 params = 151 per param
  # This is OK
  
  # Example: 20 parameters, 3 years, monthly trading
  # = 36 trades / 20 params = 1.8 per param
  # SEVERELY OVERFIT
  ```
  
  The Simplicity Test:
  - Can you explain the strategy in one sentence?
  - Would it have made sense in 1990?
  - Does it exploit human behavior?
  
  If not → probably overfit to data artifact.
  
### **References**
  - "The Probability of Backtest Overfitting" - Bailey & Lopez de Prado

## You've Already Run 100 Tests (You Just Don't Know It)

### **Id**
multiple-testing-trap
### **Severity**
CRITICAL
### **Description**
Every informal check is a hypothesis test - adjust accordingly
### **Symptoms**
  - "I only tested 3 strategies"
  - "This is my first parameter set"
  - Ignoring rejected hypotheses
### **Detection Pattern**
test.*strat|try.*approach|param.*search
### **Solution**
  Hidden Multiple Testing:
  
  Things That Count As Tests:
  - Looking at a chart → Test
  - Checking correlation → Test
  - "What if I tried..." → Test
  - Rejected strategies → Tests
  - Parameter grid search → N tests
  
  If you test 100 things at 5% significance:
  - Expected false positives: 5
  - Your "best" strategy might be pure noise
  
  Corrections:
  ```python
  import numpy as np
  from scipy import stats
  
  def bonferroni_correction(p_values: list, alpha: float = 0.05) -> dict:
      """
      Simple but conservative correction.
      """
      n_tests = len(p_values)
      adjusted_alpha = alpha / n_tests
  
      significant = [p < adjusted_alpha for p in p_values]
  
      return {
          'original_alpha': alpha,
          'adjusted_alpha': adjusted_alpha,
          'n_tests': n_tests,
          'significant': significant,
          'n_significant': sum(significant)
      }
  
  def benjamini_hochberg(p_values: list, alpha: float = 0.05) -> dict:
      """
      Less conservative FDR control.
      """
      n = len(p_values)
      sorted_pvals = sorted(enumerate(p_values), key=lambda x: x[1])
  
      significant = [False] * n
  
      for i, (original_idx, p) in enumerate(sorted_pvals):
          threshold = (i + 1) / n * alpha
          if p <= threshold:
              significant[original_idx] = True
          else:
              break  # Stop at first failure
  
      return {
          'alpha': alpha,
          'significant': significant,
          'n_significant': sum(significant)
      }
  
  # Example: 50 strategy variations tested
  # Best p-value: 0.01
  #
  # Unadjusted: "Significant at 5%!"
  # Bonferroni: 0.01 > 0.05/50 = 0.001, NOT significant
  # Reality: You got lucky
  ```
  
  Best Practice:
  - Track ALL hypotheses tested
  - Pre-register primary hypothesis
  - Apply correction before claiming significance
  
### **References**
  - "False Strategy Discovery" - Harvey et al.

## Your Backtest Period Is One Regime

### **Id**
regime-blindness
### **Severity**
HIGH
### **Description**
Strategies that work in bull markets fail in bear markets and vice versa
### **Symptoms**
  - Great 2010-2020 backtest (bull market)
  - Momentum works, then doesn't
  - "This time is different"
### **Detection Pattern**
2010.*2020|last.*5.*year|recent.*history
### **Solution**
  Regime Dependency Reality:
  
  Strategy Performance by Regime:
  
  | Strategy | Bull | Bear | Chop |
  |----------|------|------|------|
  | Momentum | +++  | --   | --   |
  | Mean Rev | +    | +    | +++  |
  | Carry    | +++  | ---  | +    |
  | Quality  | +    | +    | +    |
  | Vol Sell | +++  | ---  | +    |
  
  If your backtest is mostly bull market:
  - Momentum will look great
  - Vol selling will look great
  - Quality will look boring
  
  This tells you nothing about reality.
  
  Minimum Regime Coverage:
  ```python
  def check_regime_coverage(returns: pd.Series) -> dict:
      """
      Ensure backtest covers multiple regimes.
      """
      required_periods = {
          '2008_crisis': ('2007-10', '2009-03'),  # -50% drawdown
          '2011_eurozone': ('2011-07', '2011-10'),  # -20%
          '2015_china': ('2015-08', '2015-09'),     # Flash crash
          '2018_volmageddon': ('2018-01', '2018-02'),
          '2020_covid': ('2020-02', '2020-03'),     # -35%
          '2022_bear': ('2022-01', '2022-10'),      # -25%
      }
  
      coverage = {}
      for period_name, (start, end) in required_periods.items():
          try:
              period_returns = returns[start:end]
              coverage[period_name] = len(period_returns) > 0
          except:
              coverage[period_name] = False
  
      return {
          'periods_covered': coverage,
          'coverage_pct': sum(coverage.values()) / len(coverage),
          'is_adequate': sum(coverage.values()) >= 4  # Need most periods
      }
  ```
  
### **References**
  - Market regime research literature

## Your Universe Has Survivorship Bias (All Of Them Do)

### **Id**
survivorship-hidden
### **Severity**
HIGH
### **Description**
You're only testing on stocks that didn't go bankrupt
### **Symptoms**
  - Using current index members historically
  - Free data from Yahoo/Google Finance
  - Higher returns than market average
### **Detection Pattern**
yfinance|yahoo|current.*member|sp500.*list
### **Solution**
  Survivorship Bias Sources:
  
  1. Index Membership
     - S&P 500 today ≠ S&P 500 in 2008
     - Removed companies often dropped AFTER poor performance
     - Adds 1-2% annual return artificially
  
  2. Data Vendors
     - Free: Usually survivors only
     - Cheap: Mixed quality
     - Expensive: Usually complete
  
  3. Crypto
     - Only testing top 100 coins
     - Ignoring the thousands that went to zero
     - Massive survivorship bias
  
  Quantifying the Bias:
  ```python
  def estimate_survivorship_bias(
      strategy_return: float,  # Annual return from biased test
      universe_turnover: float = 0.05  # 5% annual delisting
  ) -> dict:
      """
      Rough estimate of survivorship bias impact.
      """
      # Academic research shows 1-2% for US equities
      # Higher for small caps, crypto
  
      estimated_bias = {
          'large_cap_us': 0.01,
          'small_cap_us': 0.02,
          'international': 0.015,
          'crypto': 0.05,  # Many coins die
          'high_turnover': universe_turnover * 0.3
      }
  
      # Apply haircut
      bias = sum(estimated_bias.values()) / len(estimated_bias)
      adjusted_return = strategy_return - bias
  
      return {
          'reported_return': strategy_return,
          'estimated_bias': bias,
          'adjusted_return': adjusted_return,
          'return_reduction_pct': bias / strategy_return * 100
      }
  ```
  
  Solutions:
  - Use point-in-time databases (CRSP, FactSet, etc.)
  - Include delisted returns (usually -100%)
  - For crypto: manually track dead coins
  
### **References**
  - "Survivorship Bias in Performance Studies" - Brown et al.

## Your Strategy Has $1M Capacity, Not $1B

### **Id**
capacity-ignored
### **Severity**
HIGH
### **Description**
Market impact kills strategies at scale
### **Symptoms**
  - Trade large % of daily volume
  - Small cap focus
  - High turnover
### **Detection Pattern**
scale|capacity|aum|volume
### **Solution**
  Capacity Estimation:
  
  Rule of Thumb:
  ```python
  def estimate_capacity(
      avg_position_adv_pct: float,  # Position as % of daily volume
      turnover_annual: float,        # Annual turnover
      target_sharpe: float = 1.0,    # Minimum acceptable Sharpe
      base_sharpe: float = 2.0       # Sharpe at small size
  ) -> dict:
      """
      Estimate strategy capacity before returns degrade.
      """
      # Market impact model: Impact = k * sqrt(participation)
      # Each doubling of size reduces Sharpe by ~15%
  
      # Maximum participation before significant impact
      max_participation = 0.01  # 1% of ADV is safe
  
      if avg_position_adv_pct > max_participation:
          size_penalty = (avg_position_adv_pct / max_participation) ** 0.5
      else:
          size_penalty = 1.0
  
      # Turnover amplifies impact
      impact_from_turnover = turnover_annual * avg_position_adv_pct * 0.001
  
      expected_sharpe = base_sharpe / size_penalty - impact_from_turnover
  
      # Solve for max size
      # This is approximate - real capacity requires simulation
  
      return {
          'current_sharpe_estimate': expected_sharpe,
          'is_viable': expected_sharpe > target_sharpe,
          'size_penalty': size_penalty,
          'recommendation': (
              "Reduce position sizes" if expected_sharpe < target_sharpe
              else "Capacity appears adequate"
          )
      }
  ```
  
  Capacity By Strategy Type:
  
  | Strategy | Typical Capacity | Why |
  |----------|------------------|-----|
  | SPY Momentum | $1B+ | Liquid |
  | Small Cap Value | $50-100M | Illiquid |
  | Crypto Arb | $1-10M | Very illiquid |
  | HFT | $10-50M | Speed matters |
  | Stat Arb Pairs | $50-200M | Moderate |
  
### **References**
  - Market microstructure literature

## Your Backtest Assumes Free Trading (It's Not)

### **Id**
transaction-cost-fantasy
### **Severity**
HIGH
### **Description**
Transaction costs are often 50%+ of alpha
### **Symptoms**
  - "I included 0.1% slippage"
  - Ignoring market impact
  - High turnover strategies
### **Detection Pattern**
slippage|transaction|commission|cost
### **Solution**
  Real Transaction Cost Components:
  
  ```python
  def realistic_transaction_cost(
      trade_value: float,
      asset_type: str,
      avg_daily_volume_usd: float,
      holding_period_days: float
  ) -> dict:
      """
      Realistic all-in cost estimation.
      """
      # Commission (easy part)
      commissions = {
          'us_equity': 0.0001,  # $1 per $10k
          'crypto_cex': 0.001,   # 10 bps
          'crypto_dex': 0.003,   # 30 bps (with gas)
          'futures': 0.00001     # Negligible
      }
  
      # Spread (market orders)
      spreads = {
          'us_equity': 0.0003,
          'crypto_cex': 0.001,
          'crypto_dex': 0.005,  # Variable
          'futures': 0.0001
      }
  
      # Market impact (the big one)
      participation = trade_value / avg_daily_volume_usd
  
      # Almgren-Chriss model (simplified)
      volatility = 0.02  # Assume 2% daily vol
      impact = volatility * (participation ** 0.5)
  
      # Scale by holding period (shorter = more impact matters)
      impact_scaled = impact * (20 / max(holding_period_days, 1)) ** 0.5
  
      commission = trade_value * commissions.get(asset_type, 0.001)
      spread = trade_value * spreads.get(asset_type, 0.001)
      market_impact = trade_value * impact_scaled
  
      total = commission + spread + market_impact
  
      return {
          'commission': commission,
          'spread': spread,
          'market_impact': market_impact,
          'total': total,
          'total_bps': total / trade_value * 10000,
          'impact_pct_of_total': market_impact / total * 100
      }
  ```
  
  Reality Check Table:
  
  | Asset | Your Assumption | Reality |
  |-------|-----------------|---------|
  | SPY | 1 bp | 3-5 bps |
  | Small Cap | 5 bps | 30-50 bps |
  | Crypto | 10 bps | 50-100 bps |
  | Illiquid Crypto | 10 bps | 200-500 bps |
  
  Annual Cost Impact (20x turnover):
  - At 5 bps: 2% annual drag
  - At 50 bps: 20% annual drag
  - At 100 bps: 40% annual drag
  
  Your 2.0 Sharpe strategy with 20x turnover in crypto?
  Probably negative after costs.
  
### **References**
  - "Optimal Execution" - Almgren & Chriss

## Your Data Is Wrong (It Always Is)

### **Id**
data-quality-ignored
### **Severity**
MEDIUM
### **Description**
Bad data creates phantom alpha
### **Symptoms**
  - Spikes to impossible prices
  - "Alpha" from data errors
  - Backtests don't replicate
### **Detection Pattern**
data.*error|spike|outlier|impossible
### **Solution**
  Data Quality Checks:
  
  ```python
  import pandas as pd
  import numpy as np
  
  def audit_price_data(prices: pd.DataFrame) -> dict:
      """
      Check for common data errors.
      """
      issues = []
  
      # 1. Impossible returns (>50% in one day)
      returns = prices.pct_change()
      extreme = returns.abs() > 0.5
      if extreme.any().any():
          issues.append({
              'type': 'extreme_returns',
              'count': extreme.sum().sum(),
              'severity': 'HIGH'
          })
  
      # 2. Zero prices
      zeros = (prices == 0).any()
      if zeros.any():
          issues.append({
              'type': 'zero_prices',
              'columns': list(zeros[zeros].index),
              'severity': 'HIGH'
          })
  
      # 3. Duplicate timestamps
      if prices.index.duplicated().any():
          issues.append({
              'type': 'duplicate_timestamps',
              'count': prices.index.duplicated().sum(),
              'severity': 'MEDIUM'
          })
  
      # 4. Gaps (missing dates)
      expected_dates = pd.date_range(
          prices.index.min(), prices.index.max(), freq='B'
      )
      missing = len(expected_dates) - len(prices)
      if missing > len(expected_dates) * 0.1:  # >10% missing
          issues.append({
              'type': 'excessive_gaps',
              'missing_pct': missing / len(expected_dates),
              'severity': 'MEDIUM'
          })
  
      # 5. Stale prices (no change for days)
      unchanged = (returns == 0).rolling(5).sum() >= 5
      if unchanged.any().any():
          issues.append({
              'type': 'stale_prices',
              'count': unchanged.sum().sum(),
              'severity': 'LOW'
          })
  
      return {
          'issues': issues,
          'total_issues': len(issues),
          'is_clean': len(issues) == 0
      }
  ```
  
  Common Phantom Alpha Sources:
  - Dividend not adjusted → fake momentum
  - Split not adjusted → impossible returns
  - Delisting price wrong → survivorship alpha
  - Quote vs trade prices → spread capture artifact
  
### **References**
  - Financial data quality research

## Every Additional Parameter Is Curve Fitting

### **Id**
curve-fitting-excuses
### **Severity**
MEDIUM
### **Description**
Complexity is the enemy of robustness
### **Symptoms**
  - "I need this parameter for edge cases"
  - Strategy has 10+ parameters
  - Parameters are suspiciously round
### **Detection Pattern**
parameter|tune|adjust|optimize
### **Solution**
  Parameter Discipline:
  
  The Rule of 5:
  - 0-2 parameters: Probably robust
  - 3-5 parameters: Acceptable with caution
  - 6-10 parameters: Likely overfit
  - 10+ parameters: Definitely overfit
  
  Each Parameter Must Pass:
  1. Economic rationale (why this number?)
  2. Sensitivity test (results stable ±20%?)
  3. Cross-asset test (works on different assets?)
  4. Out-of-sample validation
  
  Parameter Sensitivity Test:
  ```python
  def parameter_sensitivity(
      strategy_func: callable,
      base_params: dict,
      data: pd.DataFrame,
      sensitivity_range: float = 0.2
  ) -> dict:
      """
      Test how sensitive results are to parameter changes.
      """
      base_result = strategy_func(data, **base_params)
      base_sharpe = calculate_sharpe(base_result)
  
      sensitivity = {}
  
      for param, value in base_params.items():
          if not isinstance(value, (int, float)):
              continue
  
          # Test +20% and -20%
          results = []
          for mult in [1 - sensitivity_range, 1 + sensitivity_range]:
              test_params = base_params.copy()
              test_params[param] = value * mult
  
              result = strategy_func(data, **test_params)
              sharpe = calculate_sharpe(result)
              results.append(sharpe)
  
          sensitivity[param] = {
              'base_sharpe': base_sharpe,
              'low_sharpe': results[0],
              'high_sharpe': results[1],
              'range': max(results) - min(results),
              'is_stable': (max(results) - min(results)) / base_sharpe < 0.2
          }
  
      return {
          'parameter_sensitivity': sensitivity,
          'unstable_params': [p for p, v in sensitivity.items() if not v['is_stable']],
          'overall_stability': all(v['is_stable'] for v in sensitivity.values())
      }
  ```
  
  If parameters are round numbers (10, 20, 50):
  - They're arbitrary
  - Test 11, 19, 53 - similar results?
  - If not, you're overfit
  
### **References**
  - "Strategy Parameter Optimization" literature

## Machine Learning Is Not Magic Alpha

### **Id**
ml-false-hope
### **Severity**
MEDIUM
### **Description**
ML usually just learns to overfit more efficiently
### **Symptoms**
  - Neural network for prediction
  - "AI trading strategy"
  - Training accuracy 90%+
### **Detection Pattern**
neural|machine.*learning|deep.*learning|ai.*trad
### **Solution**
  ML in Finance Reality:
  
  Why ML Usually Fails:
  1. Low signal-to-noise ratio (~0.01 IC)
  2. Non-stationarity (regimes change)
  3. Data is limited (not millions of samples)
  4. Features are highly correlated
  5. Easy to overfit with many parameters
  
  If You Must Use ML:
  ```python
  def ml_sanity_checks(model, train_data, test_data):
      """
      Minimum checks before believing ML results.
      """
      checks = {}
  
      # 1. Train/test performance gap
      train_score = model.score(train_data)
      test_score = model.score(test_data)
      checks['overfit_ratio'] = train_score / max(test_score, 0.01)
      checks['is_overfit'] = checks['overfit_ratio'] > 2
  
      # 2. Feature importance
      # If top features are random-looking, model is noise
      importances = model.feature_importances_
      checks['top_feature_importance'] = max(importances)
      checks['concentrated'] = max(importances) > 0.3
  
      # 3. Time consistency
      # Split test set into halves - both should work
      mid = len(test_data) // 2
      first_half = model.score(test_data[:mid])
      second_half = model.score(test_data[mid:])
      checks['time_stable'] = abs(first_half - second_half) < 0.1
  
      # 4. Random baseline
      # Beat a random model by significant margin?
      random_score = 0.5  # Random = 50%
      checks['beats_random'] = test_score > random_score + 0.05
  
      return {
          'checks': checks,
          'red_flags': [k for k, v in checks.items()
                       if v in [True, False] and v == True]
      }
  ```
  
  What Works in ML for Finance:
  - Simple models (linear, tree-based)
  - Regularization (L1/L2, dropout)
  - Ensemble methods
  - Feature selection before training
  - Walk-forward validation
  
  What Doesn't Work:
  - Deep learning on price data
  - Complex architectures
  - End-to-end learning
  - Training on 3 years of daily data
  
### **References**
  - "Machine Learning for Asset Managers" - Lopez de Prado

path: references/validations.md
content: # Quantitative Research - Validations

## Train/Test Split Required

### **Id**
check-train-test-split
### **Description**
Backtests must separate training and testing data
### **Pattern**
train|test|split|walk.*forward
### **File Glob**
**/*.{py,ipynb}
### **Match**
present
### **Context Pattern**
backtest|simulate|strategy
### **Message**
Backtests require train/test split to avoid overfitting
### **Severity**
error
### **Autofix**


## Transaction Costs Included

### **Id**
check-transaction-costs
### **Description**
Backtests should include realistic transaction costs
### **Pattern**
backtest|simulate|return
### **File Glob**
**/*.{py,ipynb}
### **Match**
present
### **Context Pattern**
cost|slippage|commission|fee
### **Message**
Include transaction costs in backtest - they often exceed alpha
### **Severity**
error
### **Autofix**


## Sharpe Ratio Sanity Check

### **Id**
check-sharpe-sanity
### **Description**
Sharpe ratios above 3 are almost certainly overfit
### **Pattern**
sharpe.*[4-9]\.|sharpe.*[1-9]\d
### **File Glob**
**/*.{py,ipynb}
### **Match**
present
### **Message**
Sharpe ratio appears unrealistically high - check for overfitting
### **Severity**
warning
### **Autofix**


## Look-Ahead Bias Check

### **Id**
check-lookahead-shift
### **Description**
Forward shifts on data indicate potential look-ahead bias
### **Pattern**
shift\(-|\[:-
### **File Glob**
**/*.{py,ipynb}
### **Match**
present
### **Message**
Negative shift detected - verify no look-ahead bias
### **Severity**
warning
### **Autofix**


## Minimum Sample Size

### **Id**
check-sample-size
### **Description**
Strategies need sufficient trades for statistical significance
### **Pattern**
n_trades|num_trades|trade_count
### **File Glob**
**/*.{py,ipynb}
### **Match**
absent
### **Message**
Track number of trades - need 30+ for statistical significance
### **Severity**
warning
### **Autofix**


## Statistical Significance Testing

### **Id**
check-statistical-significance
### **Description**
Results should include significance tests
### **Pattern**
t_stat|p_value|confidence|significance
### **File Glob**
**/*.{py,ipynb}
### **Match**
absent
### **Context Pattern**
backtest|strategy|alpha
### **Message**
Include statistical significance testing for strategy validation
### **Severity**
warning
### **Autofix**


## Out-of-Sample Testing

### **Id**
check-out-of-sample
### **Description**
Strategies must be validated on out-of-sample data
### **Pattern**
out.*sample|oos|holdout|test.*set
### **File Glob**
**/*.{py,ipynb}
### **Match**
absent
### **Context Pattern**
backtest|validate|strategy
### **Message**
Validate strategy on out-of-sample data before deployment
### **Severity**
warning
### **Autofix**


## Survivorship Bias Awareness

### **Id**
check-survivorship-bias
### **Description**
Strategy should account for delisted/dead assets
### **Pattern**
delist|survivor|dead.*asset|bankrupt
### **File Glob**
**/*.{py,ipynb}
### **Match**
absent
### **Context Pattern**
universe|constituent|index
### **Message**
Consider survivorship bias if using historical universe data
### **Severity**
info
### **Autofix**


## Multiple Regime Testing

### **Id**
check-regime-testing
### **Description**
Strategies should be tested across market regimes
### **Pattern**
regime|bull|bear|volatility.*regime|market.*state
### **File Glob**
**/*.{py,ipynb}
### **Match**
absent
### **Context Pattern**
backtest|strategy|performance
### **Message**
Test strategy across different market regimes (bull, bear, sideways)
### **Severity**
info
### **Autofix**


## Parameter Complexity Check

### **Id**
check-parameter-count
### **Description**
Too many parameters indicates potential overfitting
### **Pattern**
param|hyperparameter|tune|optimize
### **File Glob**
**/*.{py,ipynb}
### **Match**
present
### **Message**
Keep parameters minimal (< 5) to reduce overfitting risk
### **Severity**
info
### **Autofix**


path: skill.json
content: name: Quantitative Research
description: World-class systematic trading research - backtesting, alpha generation, factor models, statistical arbitrage. Transform hypotheses into edges.
version: 1.0.0
identifier: quantitative-research
author: omer-metin
category: finance

Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (HIGH)