kitfunso/suiren-1

suiren 1: model card

suiren 1, card version 2, 2026-10-14. suiren 1 is the first public release. Inside the project it is v3.1, the name the tables and result files below use. Each later release gets its own number and repository (kitfunso/suiren-2 and on), and suiren 1 stays as published here. The verdict rows come from eval/results/v31_reads.json and the files it reads, and from eval/results/lane_o_stepC.json, eval/results/backtest_2025_tso_stepC.json, eval/results/backtest_2025.json and eval/results/backtest_2025_winter.json, all in this repository.

See inside it: suiren-anatomy.pages.dev is an interactive 3D tour of every part of the model, with its shape and parameter count.

In one paragraph. On a 2025-26 backtest, suiren 1 (long-memory v3.1) is level with Chronos-2 when both get the same weather and calendar inputs (2.98 against 3.11 MAPE, inside the 0.17 seed floor), which is the headline bar. Against Chronos-2 given only the load history (3.62), it is lower in 33 of 33 zones; suiren itself always reads the weather and calendar. It meets the pre-registered bar against the grid operators' own day-ahead forecasts, and fed alongside the operator's forecast it improves it. Both operator reads also pass a strict re-run that gives suiren only weather issued before the operator's deadline. v3, trained on the same recipe with prices in its corpus, reads the same on every rule. Its weak spot is public holidays.

Every figure on this card is a backtest, on a year that had been read before the recipe was chosen. The clean forward window, 2026-09-15 to 2026-12-01, is shut until 2026-12-08 and nothing from it appears here.

What suiren is

A decoder-only transformer with 4,216,576 parameters, written by hand in PyTorch and pretrained from scratch on eleven years of European electricity data. It forecasts the next day's hourly electricity demand for each of 33 European bidding zones. Its working name was fifty, after the 50 Hz grid, and the code, the protocol and the result files still use it.

It exists to answer two questions. Does a small specialist beat a general time-series foundation model, Chronos-2, on its own ground? And can it add anything to the day-ahead forecast each grid operator already publishes for its own zone?

Two configurations are reported. Short-memory reads the last 512 hours, about three weeks. Long-memory reads the last 2,048 hours, about twelve weeks. Long-memory is the verdict configuration. Short-memory rows are labelled second rows.

Two trainings of long-memory are reported. v3 read every Energy-Charts stream, day-ahead prices included, so its weights cannot ship (see Data). v3.1 is the same recipe without price and with the holiday table back to 2015; its weights ship here as suiren 1. v3 stays the model the forward window will judge, and its rows stay on this card.

Model

Architecture	decoder-only transformer, pre-norm RMSNorm, rotary position embeddings, output head tied to the token embedding
Size	4 blocks, d_model 256, 4 heads of 64, 4,216,576 parameters
Tokens	each hour's value, divided by the mean absolute value of its window, then binned into 4,096 uniform bins on [-15, 15]; two special ids, one for a gap and one for end of series
Context	2,048 hours (long-memory) or 512 hours (short-memory)
Output	a distribution over the next hour's bin; 100 sampled trajectories per origin give the median and the deciles
Inputs beside the target's own history	eight covariates, each as a value and a presence flag: hour, weekday, month, holiday flag, and the forecast issued the day before of 2 m temperature, 10 m wind speed, shortwave radiation and cloud cover at one point per zone; plus a learned identity for each of 60 areas in the corpus
Loss	next-token cross-entropy over bins, teacher forced
Dependencies	torch, numpy, duckdb, polars, requests, matplotlib. No pretrained component anywhere in the model, the training loop or the data pipeline

Code: model/transformer.py, model/tokenizer.py, model/covariates.py, and eval/fifty.py, which turns a checkpoint into quantile forecasts. The two baselines that suiren is measured against are kept apart from the model, and nothing in it imports them.

Training

Recipe	30,000 steps, batch 64, AdamW (betas 0.9, 0.95), peak learning rate 3e-4, 200 warm-up steps then cosine decay to a floor of 3e-5, bf16 autocast, gradient clipping at 1.0
Data cut, backtest checkpoints	every hour before 2025-09-01 00:00 UTC; the run reads no later row, in the SQL or the feature build
Data cut, forward checkpoints	every hour before 2026-09-01 00:00 UTC; scored once, on the fortnight 2026-09-01 to 2026-09-15 that was read early (D59), in eval/results/forward_sep.json
Windows, backtest cut	v3.1: 41,025,533 training windows over 522 series, loss monitor 52,528 windows. v3: 43,186,870 over 547, monitor 55,328. The monitor is the last 90 days before the cut
What it samples	every stream in the corpus, load and generation alike, uniformly across zones and series; v3.1 leaves out day-ahead price, and each v3.1 training log reads ec.price series in training: 0
Seeds	0, 1 and 2 for every configuration; every reported number is the mean of three
Hardware	one RTX 5080, 16 GB; a long-memory seed trains in two hours on a free card

Data

Source	What	Span	Terms
Energy-Charts, Fraunhofer ISE	load, residual load, generation by fuel, day-ahead price, 15-minute resolution resampled to hourly; the target in 32 of the 33 zones	load from 2015-01-04 in most zones; later starts ba 2017-02, md 2020-01, xk 2021-06, ge 2021-12	CC BY 4.0; price outside the SMARD zones is internal use only, so v3, which read it, does not ship
Elexon BMRS	GB demand outturn, half-hourly; the target in gb, which Energy-Charts does not serve	2016-02 onward	BMRS data licence, attribution required
ENTSO-E File Library	6.1.B, each operator's day-ahead load forecast, with the row's own publish time as its vintage	first stored vintage 2025-11-11	scores only, no values published
Open-Meteo	day-ahead weather forecasts as issued the day before, one point per zone	2024 onward	CC BY 4.0; the free tier is non-commercial

Corpus: the v3.1 backtest runs train on 41,025,533 windows over 522 series and the v3 runs on 43,186,870 over 547 (data/logs/v31_backtest_stepC_seed0.log, data/logs/v3_backtest_stepC_seed0.log). The last full point count (43,164,891 hourly points, 474 streams, 1.07% gaps) was taken on 2026-09-01, before seven zones were added. Every value is stored with the time it was retrieved, and training reads only what was visible at the time. The operator's forecast is never a model input. It is used as a bar (lane T) and as the starting point of lane O, both below.

The target is ec.load, which matches the ENTSO-E published actual, 6.1.A, to the digit, and Elexon's demand outturn in gb. Thirty-three zones:

at ba be bg ch cz de dk ee es fi fr gb ge gr hr hu ie
it lt lu lv md nl no pl pt ro rs se si sk xk

In ba, md and nl the published actual is net of something at midday from some date. suiren forecasts the published series there and claims no operator comparison.

How it is scored

Origin 00:00 UTC every day, 24 hours ahead, a 2,048-hour warm-up for every model, 100 samples, equal weight per zone. The metrics were pinned before any result existed and one scoring module scores every model including the baselines.

• WQL: weighted quantile loss over the nine deciles, the GluonTS convention Chronos reports.
• MASE: mean absolute error over the mean absolute 168-hour seasonal step, with the denominator taken on the context, never on the scored window.
• MAPE: on the median forecast, as a percent.
• Coverage: share of actuals inside the 5 to 95 band, nominal 90%.

Windows on this card, all read at store vintage 2026-09-13 00:00 UTC:

• Primary: 2025-09-01 to 2026-06-01, 273 origins per zone, the whole winter inside it.
• Winter: 2025-12-01 to 2026-03-01, 90 origins. A diagnostic; it cannot flip a verdict.
• Operator rows: honest days run from mid-November 2025 to June 2026, because the first stored operator vintage is 2025-11-11.

June to August 2026 is left out of every window because it chose the long-memory configuration.

Results

Against Chronos-2 at equal context

The rule, written before any number: on the primary window, mean of three seeds, suiren beats Chronos-2 when the MAPE gap is over 0.17 points (the measured seed floor), suiren is lower in at least 22 of 33 zones, and WQL agrees. All three must hold. A gap inside 0.17 points is level. Chronos-2 is read twice: on the load history alone, and handed the same eight covariates suiren reads. The covariate read is the headline bar.

Window	Model	MAPE %	WQL	MASE	Zones lower on MAPE	Read
Primary	suiren v3.1 long-memory, mean of 3, the shipped recipe	2.98	0.0197	0.546	24 of 33 against Chronos-2 at 2,048 with covariates; 33 of 33 against load history only	level with covariates (gap 0.14, inside 0.17); beats Chronos-2 given load history only
Primary	suiren v3 long-memory, mean of 3	3.02	0.0200	0.555	22 of 33 against Chronos-2 at 2,048 with covariates; 33 of 33 against load history only	level with covariates (gap 0.10, inside 0.17); beats Chronos-2 given load history only
Primary	Chronos-2 at 2,048, with the eight covariates	3.11	0.0207	0.572		the long-memory bar
Primary	Chronos-2 at 2,048, load history only	3.62	0.0242	0.667
Primary	suiren short-memory, mean of 3	3.22	0.0214	0.598	26 of 33 against Chronos-2 at 512 with covariates; 33 of 33 against load history only	beats, on both reads
Primary	Chronos-2 at 512, with the eight covariates	3.45	0.0230	0.633
Primary	Chronos-2 at 512, load history only	3.81	0.0257	0.701
Primary	Seasonal naive, t minus 168 h	7.21	0.0508	1.401
Winter	suiren long-memory, mean of 3	2.53	0.0177	0.516	21 of 33 with covariates; 31 of 33 load history only	diagnostic
Winter	Chronos-2 at 2,048, with the eight covariates	2.61	0.0182	0.529
Winter	Chronos-2 at 2,048, load history only	3.06	0.0213	0.620
Winter	suiren short-memory, mean of 3	2.78	0.0195	0.569		diagnostic
Winter	Chronos-2 at 512, with the eight covariates	2.89	0.0202	0.588
Winter	Chronos-2 at 512, load history only	3.25	0.0227	0.662

v3.1 seeds on the primary window: 3.00, 2.98 and 2.94 MAPE, coverage 0.88; Chronos-2 with covariates is as good or better in bg, dk, fi, gb, ge, md, no, si, xk. v3 long-memory seeds: 2.98, 3.05 and 3.02 MAPE. The zones where Chronos-2 with covariates is as good or better than v3: bg, dk, ee, fi, gb, ge, lt, md, no, si, xk. The other rows in the table are v3. Short-memory seeds on the primary window: 3.23, 3.18 and 3.26 MAPE. Giving Chronos-2 the weather and the calendar moved it from 3.81 to 3.45 at 512 hours and from 3.62 to 3.11 at 2,048; the short-memory margin over it is 0.23 points on the covariate read against 0.58 on the plain one.

Against each operator's own forecast, blind

The bar, fixed in writing before any v3 number: on the 26 zones where the operator's midday error is under 10%, mean of three seeds, at the operator's own timing, suiren is lower on MAPE in a majority of zones (14), lower on MASE in a majority, and the median zone ratio suiren / operator is under 1.0 on both. All three must hold. The script computes the verdict; nobody judges it by eye.

Model	Zones won, MAPE	Zones won, MASE	Median ratio, MAPE	Median ratio, MASE	Bar
suiren v3.1 long-memory, mean of 3	15 of 26	15 of 26	0.931	0.931	met, plain row
suiren v3.1 long-memory, strict weather timing	15 of 26	15 of 26	0.929	0.929	met (Limit 8)
suiren v3 long-memory, mean of 3	15 of 26	15 of 26	0.948	0.951	met, plain row
suiren v3 long-memory, strict weather timing	15 of 26	15 of 26	0.943	0.948	met (Limit 8)
suiren short-memory, mean of 3	11 of 26	12 of 26	1.079	1.090	not met

Equal-weight means, which decide nothing: operator 3.76 MAPE, v3.1 3.02, long-memory v3 3.06, short-memory suiren 3.44, naive 6.99. Every v3 long-memory seed passes on its own. The margin is one zone, and the three thinnest zones (be, dk, gr) all count as wins. Short-memory has the lower mean and still fails the bar, because the mean is carried by a few zones where the operator's forecast is poor. It wins where the operator is weak (at, ch, de, dk, xk) and loses where the operator is sharp (cz, es, fr, pt, ro).

The operator's forecast as an input

Every published day-ahead pipeline the project could find starts from the operator's forecast and corrects it, rather than trying to beat it on its own. So a third read asks whether suiren adds information to that forecast. Three nested rows per zone: the operator alone; the operator plus a walk-forward model of its own past errors; and that plus suiren's disagreement with the operator. Read 1 is row 3 against row 1. Read 2 is row 3 against row 2, the honest bar, and needs the shape above plus a pooled daily gap with t at or above 2.0 under Newey-West with Bartlett lag 7. Twenty-three zones enter; be, dk and gr fall under the 30-day floor.

Row	Read 1, against the operator	Read 2, against the operator plus its error model	t	Result
suiren v3.1 long-memory	21 of 23 on MAPE and MASE, medians 0.848 and 0.854	20 of 23 on MAPE and MASE, medians 0.943 and 0.945	10.89	both pass, plain row
suiren v3.1 long-memory, strict weather timing	20 of 23 on MAPE and MASE, medians 0.847 and 0.853	20 of 23 on MAPE and MASE, medians 0.931 and 0.935	11.20	both pass
suiren v3 long-memory, the verdict row	21 of 23 on MAPE, 20 on MASE, medians 0.869 and 0.877	18 of 23 on MAPE, 19 on MASE, medians 0.937 and 0.928	11.76	both pass, plain row
suiren v3 long-memory, strict weather timing	20 of 23 on MAPE, 19 on MASE, medians 0.865 and 0.873	20 of 23 on MAPE, 18 on MASE, medians 0.951 and 0.951	11.30	both pass
suiren short-memory, interim row	19 of 23 on MAPE and MASE, medians 0.878 and 0.893	15 of 23 on MAPE and MASE, medians 0.972 and 0.972	6.93	both pass

The sentence the protocol allows for the short-memory row: on this spent backtest window, suiren adds information to the operator's forecast, and the combination beats the operator's forecast. This is not the bar. It does not say suiren beats the operator blind, and short-memory does not. Where suiren adds most: at, lv, si, de, rs, sk, hr, bg. Where it adds nothing: cz, es, fr, pt, ro, the zones where the operator is already sharp.

Earlier versions, on the frozen 2025-09 to 2026-09 year

Version	suiren MAPE %	Chronos-2	LightGBM, same inputs	Operator
v1, load history only, 512 hours	4.27	4.06
v2, with the eight covariates, 512 hours	3.66	4.06	7.47	6.90, on different hours

v1 passed its pinned gate, WQL 0.0265 against 0.0271, and lost every other metric; the whole margin came from one gappy zone, md. v2 beat LightGBM in 33 of 33 zones and Chronos-2 in 32 of 33, losing only dk, and did not beat the operator (10 of 26 zones). v3 is the v2 recipe plus the zone identity, and long-memory adds the 2,048-hour context.

Limits

1. This is a backtest. The year was read for v1, v2 and the first operator comparison before the v3 ideas existed. The forward window after 2026-09-15 is the only untouched test and it is not on this card.
2. Chronos-2's real pretraining data (its report, arXiv 2510.15821, Table 6) is older public sets with no European zone load, so it does not reach the window (D68).
3. The 26 operator-valid zones were screened on this same year by the operator's own midday error. The screen drops the zones where the operator looks worst, so it leans against suiren.
4. The operator rows cover mid-November to June only. No autumn.
5. One weather point per zone. Operators forecast from weather over their whole area.
6. Zones share weather, so a count of zones won overstates the evidence.
7. Eight verdict lanes and twenty-two thresholded reads share one window, v3.1's eight reads included; at 0.05 that is 1.1 expected false passes across the set. v3.1's reads repeat v3's on a near-identical model, so they are far from independent.
8. Weather timing. On the plain rows the day-before weather values were issued after the operator's forecast deadline (D63). The strict rows use weather issued at least 6 hours before it, and every lead-1 hour re-fetched matched the stored copy. The 6-hour margin is a stated guess, not a measured publication delay.
9. The published v1 and v2 rows stand as published and were not re-scored.
10. Holidays. On public holidays suiren's error is 1.7 times its normal-day error, and the operator is better in most zones. Clock-change days are too few to read (D68). Part of the cause was a data bug: the holiday table started in 2022, so older holidays trained as working days. A retrain with the table back to 2015 cut holiday-day error by a median 7.6%, lower in 18 of 24 zones, and left ordinary days unchanged (D69). The v3 rows are from the model trained before the fix. v3.1 trained with the fixed table, and its holiday-day error was not measured on its own.
11. Bands run slightly narrow. The 5% to 95% band holds about 88% of actuals, against Chronos-2's 86%; narrowest in de, fi, ge, sk (D68).

Intended use

Research. suiren is not an operational forecast and does not replace an operator's day-ahead forecast. The weather inputs come from Open-Meteo's non-commercial free tier, and anything built on this code inherits that limit until the weather source changes.

What ships

• Weights: six files in weights/, all shipped, none picked by score (D71).
  • v31_backtest_stepC_seed{0,1,2}.pt trained on hours before 2025-09-01. The v3.1 rows on this card score these three.
  • v31_final_stepC_seed{0,1,2}.pt trained on hours before 2026-09-01. Use these for new forecasts.
• The code that runs them: model/ and eval/fifty.py. It needs only torch and numpy, pinned in requirements.txt to the versions the published rows ran on.
• GUIDE.md: how to run suiren every day, and how an energy supplier or trader would use it. python -m eval.forecast --zone de fetches a country's inputs from the sources the model trained on and prints the forecast. anatomy.html is a 3D tour of the model's parts, live at suiren-anatomy.pages.dev.
• This card, the result files in eval/results/, and weather_points.json, the one weather point per zone that the training weather came from.
• The training code, the data pipeline and the scoring harness are private. D-numbers refer to the project's decision log, also private.
• No v3.1 file read a price series in training. Each training log shows ec.price series in training: 0, and the packager refuses a file without that line. Earlier versions trained on Energy-Charts prices whose terms bar derived use, so their weights never ship.
• Each file holds the model weights, the model config, the zone vocabulary and the training config. The optimizer state is gone. torch.load(path, weights_only=True) reads every file, and Fifty.from_checkpoint loads it that way.

Making a forecast

suiren reads a zone's own hourly load and eight covariate columns per hour. The covariate rows cover the context hours and the 24 hours to forecast, oldest first, with NaN for any value you do not have:

Column	Value
0 hour	0 to 23, UTC
1 weekday	0 is Monday, 6 is Sunday, of the UTC date
2 month	1 to 12
3 holiday	1 on a nationwide public holiday of the zone's country (training used Nager.Date's Public days flagged global), else 0; the flag follows the UTC date
4 to 7 weather	temperature_2m, wind_speed_10m, shortwave_radiation, cloud_cover

The model was trained on Open-Meteo's forecast issued the day before each hour, at the zone's point in weather_points.json; for a live forecast of tomorrow, that is today's forecast. Each column is standardised over the window before the model sees it, so the units do not matter, but one source and one unit per column should hold for the whole window.

import numpy as np

from eval.fifty import Fifty

model = Fifty.from_checkpoint("weights/v31_final_stepC_seed0.pt")

start = np.datetime64("2026-10-12T00", "h")        # first hour to forecast, UTC
hours = np.arange(start - 2048, start + 24)        # 2,048 context hours, then 24 ahead
history = ...  # 2,048 hourly loads in MW, oldest first, NaN for a missing hour
weather = ...  # 2,072 rows, the four weather columns in the order above

calendar = np.column_stack([
    hours.astype(int) % 24,
    (hours.astype("datetime64[D]").astype(int) + 3) % 7,   # 1970-01-01 was a Thursday
    hours.astype("datetime64[M]").astype(int) % 12 + 1,
    np.full(hours.size, np.nan),                           # no holiday table to hand
])
bands = model.predict(history, horizon=24, levels=(0.05, 0.5, 0.95),
                      covariates=np.column_stack([calendar, weather]), zone="de")

bands[0.5] is the median forecast for the 24 hours, in MW. bands[0.05] and bands[0.95] give the 5% to 95% band. model.zones lists the 60 zone ids the model knows; the 33 lower-case ids on this card are the ones it is scored on. The 25 upper-case ids (DE-LU, DK1 and so on) are price zones; v3.1 read no price, so it never trained them. A shorter history runs, but every score on this card read the full 2,048 hours. Fifty uses the GPU when torch finds one; pass device="cpu" to from_checkpoint to stay off it. On a desktop CPU one forecast of 100 sampled paths took about two minutes.

Attribution and licence

• Weather data by Open-Meteo.com, CC BY 4.0.
• Load and generation data from Energy-Charts.info, Fraunhofer ISE, CC BY 4.0.
• Operator forecasts from the ENTSO-E Transparency Platform. Only accuracy statistics are published here, never an operator's values.
• Contains BMRS data © Elexon Limited copyright and database right 2026, under the BMRS data licence.
• Weights and code: CC BY-NC 4.0 (LICENSE), because the training weather came from Open-Meteo's non-commercial tier. Commercial use needs a separate licence from the author; ask on this repository's Community tab.
• This card and the result files: CC BY 4.0.