DOI: 10.5281/zenodo.17241640
title: IBTrACS + ERA5 Tropical Cyclone Benchmark (North Atlantic, 2000–2024) — Multi-Horizon Track, Intensity, RI, & Lifecycle

description: 🚀 Release v2 — IBTrACS + ERA5 Tropical Cyclone Benchmark (North Atlantic, 2000–2024)



Scope: North Atlantic basin • Years: 2000–2024 • Cadence: 6-hourly (00/06/12/18 UTC)
Includes: ML-ready master table, task subsets (intensity, track, RI-24h, lifecycle), splits, and dictionaries.



✨ Highlights (v2)



Unified master (2000–2024) at 6-hour cadence with harmonized IBTrACS tracks and collocated ERA5 fields.

Forecast targets at +6 / +12 / +24 h for wind and position (e.g., wmo_wind_next6, lat_next24, lon_next24).

Ready-made tasks:


Intensity regression (evaluate with MAE/RMSE)

Track regression (evaluate with great-circle error)

Rapid Intensification (24 h) classification

Lifecycle classification (dissipation / ET within 48 h)



Leakage-safe storm-level splits (train / val / test) and sample weights for imbalance.

Schemas in human & machine-readable form (DATA_DICTIONARY.md, *_dictionary.csv).

Reproducible build with two scripts: scripts/build_dataset.py → scripts/add_targets.py.




📦 Release Assets



outputs/model_ready_2000_2024_v2.parquet — Master benchmark table (6-hour). IBTrACS + ERA5 features + multi-horizon targets.

outputs/merged_ibtracs_era5_2000_2024_clean.parquet — Clean merged intermediate (pre-target engineering) for custom target design.

outputs/tasks/*.parquet — Slim task-specific subsets (intensity, track, RI-24h, lifecycle).

outputs/DATA_DICTIONARY.md, outputs/model_ready_2000_2024_v2_dictionary.csv — Variable names, units, descriptions.




[!TIP]
Use the task subsets for baselines/benchmarks; use the master for feature engineering & R&D.




🧪 Tasks & Targets (at a glance)



Intensity (regression): wmo_wind_next{6,12,24} (kt) → MAE / RMSE

Track (regression): lat_next{6,12,24}, lon_next{6,12,24} (deg) → Haversine km

Rapid Intensification (24 h): ri_next24 (Δwind ≥ 30 kt) → AUROC / AUPRC / F1

Lifecycle (48 h horizon): dissipates_within48, et_within48 → F1 / AUROC


Anti-leakage: use the provided storm-level split; never mix fixes from the same storm across sets.



⚙️ ERA5 Region & Variables (Benchmark Cut)

Bounds: N 50°, S 0°, W 100°, E 10° • Cadence: 6-hourly • Format: NetCDF4 (source)

Variables: u10, v10, wind10, t2m_c, sst_c, msl_hpa, tp
(+ IBTrACS: wmo_wind, wmo_pres, basin, metadata)



[!IMPORTANT]
ERA5 raw data are not included (size/licensing). Fetch from Copernicus CDS and rebuild with the scripts if needed.




🏃 Quickstart

The following examples show how to load the dataset, apply the provided splits, and run simple baselines for intensity and track forecasting. These baselines help verify correctness before building advanced ML models.

Load the master dataset

import pandas as pd, numpy as np
df = pd.read_parquet("outputs/model_ready_2000_2024_v2.parquet")
print(df.shape, df.columns[:12])


Splits

train = df[df["split"] == "train"]
val   = df[df["split"] == "val"]
test  = df[df["split"] == "test"]


Intensity baseline: persistence @ +6 h

tmp  = df[["storm_id","iso_time","wmo_wind","wmo_wind_next6"]].dropna()
y    = tmp["wmo_wind_next6"].to_numpy()
yhat = tmp["wmo_wind"].to_numpy()
mae  = np.mean(np.abs(y - yhat))
print(f"Persistence MAE (+6h): {mae:.2f} kt")


Track error (Haversine, +12 h):

import numpy as np
R = 6371.0  # km
def haversine(lat1, lon1, lat2, lon2):
    p = np.pi/180
    dlat, dlon = (lat2-lat1)*p, (lon2-lon1)*p
    a = np.sin(dlat/2)**2 + np.cos(lat1*p)*np.cos(lat2*p)*np.sin(dlon/2)**2
    return 2*R*np.arcsin(np.sqrt(a))
tt = df[["lat","lon","lat_next12","lon_next12"]].dropna()
err_km = haversine(tt["lat"], tt["lon"], tt["lat_next12"], tt["lon_next12"])
print(f"Median great-circle error (+12h): {np.median(err_km):.1f} km")




🔁 Build Pipeline



Raw ingestion — IBTrACS v04r01 (global tracks) + ERA5 (regional cut), 6-hour alignment.

Cleaning & normalization — keys: storm_id, iso_time (UTC), lat, lon; units; duplicates.

Feature & target engineering — multi-horizon shifts {6,12,24} for wind/position; masks has_next{6,12,24,48}; ri_next24; lifecycle flags; splits; sample_weight.

Export — model_ready_2000_2024_v2.parquet, merged_*_clean.parquet, tasks/*.parquet.


Recreate:

python scripts/build_dataset.py
python scripts/add_targets.py




📊 Versioning & Changelog

Version: v2



Added: unified 2000–2024 master; refreshed task subsets; improved data dictionary.

Improved: leakage-safe storm-level splits and sample weighting.

Clarified: licensing and ERA5 acquisition notes.


v1: initial merge & targets.



📜 Licensing



Code: MIT License

Processed Data: CC-BY 4.0

Raw ERA5: acquire from Copernicus CDS per C3S/ECMWF terms.




🧾 Cite This Dataset

DOI: https://doi.org/10.5281/zenodo.17239540

@dataset{hasan_ibtracs_era5_tc_benchmark_2000_2024_v2,
  author    = {Saifur Rahman Mehedi},
  title     = {IBTrACS + ERA5 Tropical Cyclone Benchmark (2000–2024), v2},
  year      = {2025},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.17239540},
  url       = {https://doi.org/10.5281/zenodo.17239540}
}




⚠️ Intended Use & Ethics

This dataset is for research and education in climate science and ML.
Do not use for real-time hazard guidance — rely on official sources (e.g., NOAA/JTWC).
Always disclose uncertainties and limitations when publishing results.



👤 Maintainer

Saifur Rahman Mehedi — Issues and questions welcome via the repository's Issue Tracker.
LinkedIn