mrvictoru/energydecision-dt-grpo

AEMO Decision Transformer — GRPO Post-Trained

This is a GRPO online-RL fine-tuned version of the v2 pretrained DT for battery energy storage trading in Australia's NEM market.

see https://github.com/mrvictoru/energydecision

Performance Summary

Evaluated on Q4 2024 held-out SA1 data with dispatch-matched battery (Dalrymple North: 8 MWh / 30 MW, 3.75 C, full_fcas action mode):

Metric	Value
Profit/episode	+$8,242
Profit/MWh (normalised)	+$1,030
FCAS revenue/ep	$7,686
Degradation cost/ep	$760
vs v2 pretrained baseline	+285% profit
vs PPO (SB3)	+6% profit

Beats both the pretrained baseline and a trained PPO reference on the same dispatch-matched battery.

GRPO Configuration (Phase 1)

Parameter	Value
Iterations	5
Learning rate	1e‑5
KL coefficient	0.02
RTG count	4
RTG spread	2.0
RTG distribution	Gaussian
Group size	8
Degradation penalty weight	1.5
Reference sync every	5 iterations
Adaptive RTG	Yes (EWMA α=0.1)
Training regions	NSW1, SA1, QLD1, VIC1, TAS1
Training date range	2024-01-01 → 2024-09-30
Training step duration	5 minutes
Optimal eval RTG	0.5

Architecture

Parameter	Value
state_dim	18
act_dim	9
n_block	8
h_dim	384
n_heads	8
context_len	180
drop_p	0.15
Action mode	full_fcas (energy + 8 FCAS services)

Training Data

The base model was pretrained on the v2 FCAS-rich dataset (2,401 episodes, 77 M rows, 4 realistic battery configurations). GRPO post-training used online rollouts from the AEMO simulation environment on 2024 data (Jan–Sep), sampled across all 5 NEM regions.

Recommended RTG

For evaluation, use rtg_value=0.5 (not 0.0). The RTG prompt significantly affects performance. See the calibration sweep:

RTG	Profit/ep	FCAS/ep	Deg/ep
0.0	$5,451	$7,962	$2,769
0.5	$8,242	$7,637	$1,323
1.0	$7,901	$7,781	$1,207
2.0	$6,477	$7,219	$963

Standalone Inference

from decision_transformer import DecisionTransformer
from huggingface_hub import hf_hub_download

model_kwargs = {
    "state_dim": 18, "act_dim": 9, "n_block": 8,
    "h_dim": 384, "context_len": 180, "n_heads": 8,
    "drop_p": 0.15, "max_timestep": 100000,
}
checkpoint = hf_hub_download("mrvictoru/energydecision-dt", "aemo_dt_grpo_model.pt")
model = DecisionTransformer(**model_kwargs)
model.load_from_checkpoint(checkpoint)
model.eval()