meter-load-forecasting-transformer

Small Transformer (1.46M params) trained from scratch to forecast household electricity load 24 hours ahead from 168 hours (1 week) of history.

• Task: 24h-ahead electricity load forecasting
• Input: last 168 hourly readings of Global_active_power (kW)
• Output: next 24 hourly readings (kW)
• Dataset: UCI Individual Household Electric Power Consumption, resampled to hourly means
• Architecture: Transformer encoder (2 layers, d_model=64, 4 heads, ffn=128) + linear regression head
• Params: 1,457,304
• Framework: PyTorch (no external ML framework dependency)

Results

split	metric	value
val (final epoch)	MSE (normalized)	0.503
val (final epoch)	MAE (normalized)	0.530
test	MSE (normalized)	0.427
test	MAE (kW)	0.440

Trained 8 epochs, ~104s on Apple Silicon (MPS).

Full training log and config: run_info.json, config.json. Code: https://github.com/shivam2003-dev/meter-load-forecasting-transformer

Usage

import json
import torch
from huggingface_hub import hf_hub_download

repo_id = "Shivam3002/meter-load-forecasting-transformer"
cfg = json.load(open(hf_hub_download(repo_id, "config.json")))
weights_path = hf_hub_download(repo_id, "pytorch_model.bin")

# rebuild TinyForecastTransformer class from train.py in the GitHub repo, then:
# model = TinyForecastTransformer(d_model=cfg["d_model"], nhead=cfg["n_head"],
#                                  num_layers=cfg["n_layers"], dim_ff=cfg["dim_ff"],
#                                  input_len=cfg["input_hours"], pred_len=cfg["pred_hours"])
# model.load_state_dict(torch.load(weights_path))
# model.eval()

# normalize input window with cfg["normalization"]["mean"] / ["std"] before inference,
# and de-normalize the output the same way.

Limitations

Trained on a single household's meter data — not tuned for other households, regions, or seasonal patterns beyond what the dataset (2006-2010, France) covers. Small model and short training run; intended as a lightweight demo/baseline, not a production forecaster.