OneScience-Group/CausalModelEvaluation

CausalModelEvaluation

Model Introduction

CausalModelEvaluation (CME) addresses the limitations of climate-model evaluations that rely mainly on mean-state errors and do not determine whether key process relationships are represented correctly. It is primarily intended to construct causal climate fingerprints, identify dependencies among models with shared development backgrounds, evaluate precipitation simulation skill, and use historical process skill to constrain uncertainty in future precipitation change.

Paper: Causal networks for climate model evaluation and constrained projections
https://doi.org/10.1038/s41467-020-15195-y

Model Description

Causal Model Evaluation was proposed by research teams from Imperial College London, the German Aerospace Center, the University of Bremen, and the University of East Anglia. The paper evaluates CMIP5 sea-level-pressure and precipitation simulations with NCEP-NCAR and ERA-Interim reanalyses and CRU TS v4.02 precipitation observations. The model supports causal climate-fingerprint reconstruction, process-oriented climate-model evaluation, and constrained precipitation-change projection.

Use Cases

Use Case	Description
Causal fingerprinting	Estimate a directed, signed, lagged network from seasonal 50-node time series.
Climate model evaluation	Compute reference-oriented asymmetric F1 with direction, sign, and lag tolerance.
Precipitation skill	Compute latitude-area-weighted Taylor S-score and pattern correlation.
Constrained projection	Fit an RBF plus white-noise GP between CME F1 and precipitation change.
ModelScope/OneCode	Validate data, fitting, checkpoint, inference, evaluation, and task-figure workflows.

Usage Instructions

1.OneCode

Experience intelligent, one-click AI4S programming through the OneCode online environment:

Try intelligent, one-click AI4S programming

2. Download and Installation

hf download OneScience-Group/CausalModelEvaluation --local-dir ./CausalModelEvaluation
cd CausalModelEvaluation

Environment Dependencies

Hardware Requirements

• A GPU or DCU is recommended.
• A CPU can be used for connectivity validation with the default small-sample configuration.
• DCU users must install DTK in advance. DTK 25.04.2 or later, or the OneScience-recommended version matching the current cluster, is recommended.

DCU Environment

# Activate DTK and Conda first
conda create -n onescience311 python=3.11 -y
conda activate onescience311
pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai

GPU Environment

# Activate Conda first
conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
conda activate onescience311
pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai

Training Data

python scripts/fake_data.py

This repository uses a small number of structured synthetic samples to validate the engineering workflow. The data contains seasonal sea-level-pressure mode time series, distinct lagged causal relationships across models, and precipitation fields and changes related to model process skill. It preserves the paper's node, time, lag, and spatial dimensions and validates causal-network construction, model comparison, and constrained projection only; it does not represent the real climate-data distribution or paper performance.

Training

For single-device training, use:

python scripts/train.py

For multi-GPU training, use:

torchrun --nproc_per_node=8 --nnodes=1 --rdzv_id=1000 --rdzv_backend=c10d --max_restarts=0 --master_addr="localhost" --master_port=29500 scripts/train.py

Training results include reference and model causal networks, model-comparison scores, and the precipitation-constraint relationship, with a checkpoint and training metrics saved for subsequent inference.

result/checkpoints/causalmodelevaluation.pt
result/training/metrics.json

Trained Weights

No weights are bundled under weight/. The paper method produces no conventional neural-network weights; the engineering checkpoint stores statistical networks and GP state and is not claimed to be compatible with external weights.

Inference

python scripts/inference.py

Inference reloads the checkpoint and exports complete model and four-season edges/pvalues/mci, complete reference networks, reference/model precipitation fields, CME F1, precipitation changes, GP means and 95% intervals, and network/projection metadata to result/output/inference.npz.

Evaluation and Visualization

python scripts/result.py

Evaluation generates structured results for climate-model causal-network comparison, precipitation simulation skill, and constrained projection, and saves them to result/evaluation/metrics.json. It also creates the model-skill and precipitation-change comparison figure result/evaluation/cme_task.png. Synthetic-data results are only for engineering-workflow validation and do not represent paper performance on the real test set.

Official OneScience Information

Platform	OneScience Main Repository	Skills Repository
Gitee	https://gitee.com/onescience-ai/onescience	https://gitee.com/onescience-ai/oneskills
GitHub	https://github.com/onescience-ai/OneScience	https://github.com/onescience-ai/oneskills

Citation and License

This repository is an independent engineering reproduction of the public Causal Model Evaluation paper specifications.

Use of this repository's code, official model weights, and data remains subject to the licenses and terms of their respective projects.