NeCast

NeCast estimates the size history of a population from the unphased genotypes of 32 diploid individuals in one window of 30 Mb of accessible sequence, together with the recombination and mutation maps estimated for that window. It returns log10 Ne on 64 time intervals, evenly spaced in log time between 50 and 500,000 generations ago, and analyses a window in seconds on a CPU. This repository holds the network used in the NeCast paper, its training record, and the other networks the paper compares it with.

Files

path	contents	size
final/checkpoint.pt	weights of the network used in the paper	4.2 MB
final/training_state.pt	the state saved at the end of training	16.7 MB
final/PLAN.json	the full training configuration, input checksums and the lists of training, validation and test windows	30.6 MB
final/DEVELOPMENT.json	training and validation error at every epoch	small
final/test_predictions.npz	predictions and truth for the 500 test windows	0.6 MB
final/report.json, DONE.json, run_DONE.json	run summaries with checksums	small
final/INTAKE_VERIFY.json, compshare_sha256.txt	checksums of the copied files against the training machine	small
final/training_log/	the training command and logs	0.8 MB
final/snapshots/epoch06/, epoch09/	states saved after epochs 6 and 9	47 MB each
training_contrasts/	7 networks trained on 8,000 or 88,000 windows	0.36 GB
ablations/	5 networks trained on 12,500 windows	0.25 GB
SHA256SUMS	a checksum for every file except this card	small

The network

Five streams read the same sites. A site encoder reads the genotypes at every segregating site with each site's position on the recombination and mutation maps, so that distances count expected crossovers and mutations. Three summaries come from the same sites: the folded site-frequency spectrum, linkage disequilibrium at seven lags measured in segregating sites, and the spectra of identity-by-state tracts over the 496 pairs of individuals. The fifth stream is the posterior of PSMC+ (Cousins et al. 2024), decoded for each individual with both maps and pooled over the 32. The network outputs a correction for each of the 64 intervals, added to a baseline curve: the window's Watterson estimate of log10 Ne plus a fixed offset per interval.

Training

• Data. Larrivhan/NeCast-simulations: 89,996 training windows; 500 further training windows to choose the epoch; 500 development windows as the test set.
• Settings. 12 epochs on four RTX 4090 GPUs (50.3 h, 201.2 GPU-hours), width 128, seed
  20260728. Loss: mean squared error of log10 Ne over the 64 intervals. AdamW with learning rate 3 × 10⁻⁴, cosine decay to 5% of it over the 12 epochs, weight decay 10⁻⁴, gradient-norm clipping at 1, one update per 32 windows. The last epoch had the lowest validation error, and its checkpoint is final/checkpoint.pt.
• Test error. 0.287: the root mean squared error of log10 Ne over all 64 intervals of the 500 test windows taken together, from final/test_predictions.npz.

Running it

The network reads inputs built by the NeCast code (summary statistics, PSMC+ posteriors and the site encoder's input), and its checkpoint is loaded by that code. The code is being prepared for release; its link will be added here. final/checkpoint.pt is a plain PyTorch file: torch.load(path, map_location="cpu", weights_only=True) returns a dictionary whose state_dict holds the weights. The inputs the network read for every simulated and real window in the paper are in Larrivhan/NeCast-simulations, Larrivhan/NeCast-benchmark and Larrivhan/NeCast-1000G.

Training contrasts (training_contrasts/)

Each network was trained for 12 epochs with the same settings and scored on the same 500 test windows. results/ has the paired comparisons (paired_20260925.json), every prediction (eval_all_20260925.npz) and a summary (RESULTS.md).

folder	training windows	trained on	inputs
full_mix_8k	8,000	the full training mix	all five streams
no_errors_8k	8,000	windows without errors, constant maps	all five streams
errors_constant_maps_8k	8,000	windows with errors, constant maps	all five streams
no_templates_8k	8,000	the full mix without published human histories	all five streams
site_encoder_only_8k	8,000	the full training mix	site encoder only
summaries_only_8k	8,000	the full training mix	the three summaries only
summaries_only_88k	88,000	the full training mix	the three summaries only

no_errors_8k against errors_constant_maps_8k gives the effect of training with errors (Fig. 3d of the paper); errors_constant_maps_8k against full_mix_8k that of training with varying maps (Fig. 4a).

Ablations (ablations/)

Each network was trained once on the same 12,500 windows of the full mix for 12 epochs and scored on the 500 test windows. results/ablation_paired.json has the paired comparisons behind Extended Data Fig. 1 (ablation_paired.py recomputes them).

folder	inputs
full_12500	all five streams
no_posterior_12500	without the PSMC+ posterior
no_summaries_12500	without the three summaries
summaries_only_12500	the three summaries alone
converged_posterior_12500	all five streams, with a PSMC+ posterior whose prior was refitted five times instead of once

Inside a run folder, PLAN.json holds the configuration, DEVELOPMENT.json the per-epoch errors, <inputs>.final.pt the final state, and <inputs>/checkpoint.pt the weights with mechanism_evaluation.npz (the test predictions).

Licence

The weights and files here are released under CC BY 4.0.

Citation

The NeCast paper is in preparation; its reference will be added here.