Skill: obekt/numerai-tournament (version 1.0.0)

--- name: numerai-tournament description: Autonomous Numerai tournament participation — train models, submit predictions, and earn NMR cryptocurrency. tags: - finance - machine-learning - cryptocurrency - trading - numerai - lightgbm - data-science metadata: clawdbot: requires: env: - NUMERAI_PUBLIC_ID - NUMERAI_SECRET_KEY bins: - python3 - pip primaryEnv: NUMERAI_SECRET_KEY --- # Numerai Tournament Participate autonomously in the [Numerai](https://numer.ai) data science tournament. Numerai is a hedge fund that crowdsources stock market predictions from data scientists. You submit predictions on obfuscated financial data and earn (or lose) NMR cryptocurrency based on performance. ## Overview - **What**: Predict stock market returns using obfuscated tabular features - **How**: Download data, train a model, submit predictions each round - **Reward**: Stake NMR on predictions; earn or lose based on correlation with targets - **Frequency**: New rounds open Tue–Sat at 13:00 UTC; scores resolve ~31 days later ## Setup ### 1. Create a Numerai Account ```bash # Visit https://numer.ai to sign up # Then create API keys at https://numer.ai/account # Store credentials: mkdir -p ~/.numerai cat > ~/.numerai/credentials.json << 'CREDS' { "public_id": "YOUR_PUBLIC_ID", "secret_key": "YOUR_SECRET_KEY" } CREDS chmod 600 ~/.numerai/credentials.json ``` Alternatively, set environment variables: ```bash export NUMERAI_PUBLIC_ID="YOUR_PUBLIC_ID" export NUMERAI_SECRET_KEY="YOUR_SECRET_KEY" ``` ### 2. Install Dependencies ```bash python3 -m venv venv && source venv/bin/activate pip install numerapi lightgbm pandas numpy cloudpickle scikit-learn ``` On macOS ARM (Apple Silicon), LightGBM also requires: ```bash brew install libomp ``` ### 3. Download Tournament Data ```python from numerapi import NumerAPI from pathlib import Path napi = NumerAPI() # No auth needed for data download data_dir = Path("data") data_dir.mkdir(exist_ok=True) # Current dataset version is v5.2 napi.download_dataset("v5.2/train.parquet", dest_path=str(data_dir / "train.parquet")) napi.download_dataset("v5.2/validation.parquet", dest_path=str(data_dir / "validation.parquet")) napi.download_dataset("v5.2/live.parquet", dest_path=str(data_dir / "live.parquet")) napi.download_dataset("v5.2/features.json", dest_path=str(data_dir / "features.json")) napi.download_dataset("v5.2/live_benchmark_models.parquet", dest_path=str(data_dir / "live_benchmark_models.parquet")) ``` **Note**: Training data is ~8GB. Only `live.parquet` and `features.json` are needed for prediction. ## Training a Model The recommended approach is a LightGBM ensemble trained on multiple targets. This provides strong and stable performance. ### Feature Selection ```python import json with open("data/features.json") as f: feature_metadata = json.load(f) # Three feature set sizes: # "small" — ~42 features (fast iteration) # "medium" — ~780 features (good tradeoff) # "all" — ~2748 features (maximum signal, slow) features = feature_metadata["feature_sets"]["medium"] ``` ### Target Selection The main target is `target`. Additional targets improve ensemble diversity: | Target | Description | |--------|-------------| | `target` | Primary tournament target | | `target_teager2b_20` | Current payout-correlated target | | `target_cyrusd_20` | Complementary target for ensemble diversity | ### LightGBM Training ```python import lightgbm as lgb import pandas as pd import pickle train = pd.read_parquet("data/train.parquet", columns=["era"] + features + targets) lgbm_params = { "n_estimators": 5000, # Use 20000 for production quality "learning_rate": 0.005, "max_depth": 6, "num_leaves": 64, "min_child_samples": 5000, "colsample_bytree": 0.1, "subsample": 0.8, "subsample_freq": 1, "reg_alpha": 0.1, "reg_lambda": 1.0, "verbose": -1, "n_jobs": -1, } models = {} for target in targets: X = train[features] y = train[target] mask = y.notna() model = lgb.LGBMRegressor(**lgbm_params) model.fit(X[mask], y[mask]) models[target] = model with open("models/ensemble_models.pkl", "wb") as f: pickle.dump(models, f) ``` ### Validation Evaluate per-era correlation and Sharpe ratio: ```python val = pd.read_parquet("data/validation.parquet", columns=["era"] + features + targets) predictions = pd.DataFrame(index=val.index) for target, model in models.items(): raw = model.predict(val[features]) predictions[target] = pd.Series(raw, index=val.index).rank(pct=True) ensemble = predictions.mean(axis=1).rank(pct=True) corrs = [] for era in val["era"].unique(): m = val["era"] == era pred_era = ensemble[m] tgt = val.loc[m, "target"] if tgt.notna().sum() >= 10: corrs.append(pred_era.corr(tgt)) corrs = pd.Series(corrs) print(f"Mean Corr: {corrs.mean():.4f}") print(f"Sharpe: {corrs.mean() / corrs.std():.2f}") print(f"% Positive: {(corrs > 0).mean() * 100:.1f}%") ``` Target validation performance: Mean Corr > 0.02, Sharpe > 1.0, >90% positive eras. ## Submitting Predictions ### Option A: Upload a Predictions CSV (Manual) ```python import json from numerapi import NumerAPI with open("~/.numerai/credentials.json") as f: creds = json.load(f) napi = NumerAPI(creds["public_id"], creds["secret_key"]) # Check round status current_round = napi.get_current_round() is_open = napi.check_round_open() print(f"Round {current_round}, Open: {is_open}") if is_open: # Download live data napi.download_dataset("v5.2/live.parquet", dest_path="data/live.parquet") live = pd.read_parquet("data/live.parquet") # Generate predictions (same ensemble logic as validation) predictions = pd.DataFrame(index=live.index) for target, model in models.items(): raw = model.predict(live[features]) predictions[target] = pd.Series(raw, index=live.index).rank(pct=True) ensemble = predictions.mean(axis=1).rank(pct=True) # Save and submit submission = ensemble.to_frame("prediction") submission.to_csv("predictions.csv") napi.upload_predictions("predictions.csv", model_id="YOUR_MODEL_ID") ``` ### Option B: Upload a Model Pickle (Zero-Maintenance) Upload a pickled function and Numerai runs it daily — no cron, no server. **Critical constraints for model upload:** - Must be a **pickled function** (not a class), loaded via `pd.read_pickle()` - Must use **Python 3.12** (Numerai's max supported version) - Must match Numerai runtime packages: `lightgbm==4.5.0`, `numpy==2.1.3`, `pandas==2.3.1` - Runtime limits: 1 CPU, 4GB RAM, 10 minute timeout - Use **native LightGBM Boosters** (not sklearn wrappers) to avoid dependency issues ```python # Build the upload pickle (run with Python 3.12!) import cloudpickle import lightgbm as lgb import pandas as pd import pickle # Load trained sklearn models and extract native boosters with open("models/ensemble_models.pkl", "rb") as f: sklearn_models = pickle.load(f) boosters = {} for name, model in sklearn_models.items(): bstr = model.booster_.model_to_string() boosters[name] = lgb.Booster(model_str=bstr) feature_cols = features # medium feature set list models = boosters def predict(live_features: pd.DataFrame, live_benchmark_models: pd.DataFrame = None) -> pd.DataFrame: predictions = pd.DataFrame(index=live_features.index) for target, booster in models.items(): raw = booster.predict(live_features[feature_cols]) predictions[target] = pd.Series(raw, index=live_features.index).rank(pct=True) ensemble = predictions.mean(axis=1).rank(pct=True) return ensemble.to_frame("prediction") with open("models/model_upload.pkl", "wb") as f: cloudpickle.dump(predict, f) ``` Then upload via the Numerai web UI at https://numer.ai or via API: ```python napi.upload_model("models/model_upload.pkl", model_id="YOUR_MODEL_ID") ``` ## Checking Performance ```python from numerapi import NumerAPI napi = NumerAPI(public_id, secret_key) # Round status print(f"Current round: {napi.get_current_round()}") # Get model performance (scores resolve after ~31 days) # Check via https://numer.ai/models/YOUR_USERNAME ``` ## Tournament Rules & Key Facts - **Dataset**: v5.2 — obfuscated financial features, ~2748 total features - **Rounds**: Open Tue–Sat at 13:00 UTC. Weekday windows: ~1hr. Saturday: ~49hrs. - **Scoring**: 20D2L framework, ~31 day resolution - **Payout formula**: `stake * clip(payout_factor * (0.75*CORR + 2.25*MMC), -0.05, +0.05)` - CORR = correlation of predictions with target - MMC = meta-model contribution (originality bonus) - **Staking**: Optional — stake NMR to earn/lose based on performance. Start with 0 stake until the model proves consistent. - **Current payout target**: Resembles `target_teager2b_20` ## Tips for Strong Performance 1. **Ensemble multiple targets** — reduces variance, improves Sharpe 2. **Rank-normalize predictions** — use `.rank(pct=True)` before averaging and after 3. **Use early stopping** — prevent overfitting with `lgb.early_stopping(300)` 4. **Feature neutralization** — improves MMC by decorrelating from common factors 5. **Era-aware validation** — always evaluate per-era, never row-level metrics 6. **Don't overfit to validation** — Numerai data is non-stationary; keep models simple ## External Endpoints This skill interacts with the following external services: - `api.numer.ai` — Numerai GraphQL API for round status, submissions, and scores - `numer.ai` — Data downloads (tournament datasets) ## Security & Privacy - Your `NUMERAI_PUBLIC_ID` and `NUMERAI_SECRET_KEY` are sent to `api.numer.ai` for authentication - Predictions (stock return rankings) are uploaded to Numerai's servers - No other data leaves your machine - Store credentials in `~/.numerai/credentials.json` with `chmod 600` permissions


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)