PatchTST Monthly Seasonal Forecast Model

State-of-the-art monthly seasonal time series forecasting model, fine-tuned on the M4 Monthly competition dataset (48,000 time series).

Architecture

Based on PatchTST (ICLR 2023) — a patch-based Transformer for time series that:

Tokenizes time series into 6-month patches to capture semi-annual seasonal patterns
Uses sinusoidal positional encoding for temporal awareness
Applies standard normalization (RevIN) for scale-invariant forecasting
Achieves strong performance with only 603,538 parameters

Training Details

Parameter	Value
Dataset	M4 Monthly (48,000 series, 6 categories)
Context Length	48 months (4 years)
Prediction Length	18 months
Patch Length	6 months
d_model	128
Attention Heads	8
Transformer Layers	3
Epochs	2000 steps
Batch Size	128
Learning Rate	0.001 (cosine schedule)
Optimizer	AdamW (weight_decay=0.01)

Evaluation Results (M4 Monthly Test Set, 18-month horizon)

Model	MASE (mean)	sMAPE (mean)
PatchTST (ours)	1.0243	14.04
Chronos-Bolt (zero-shot)	0.9202	13.72
Seasonal Naive	1.2453	16.36

Per-Category Results

Category	MASE	sMAPE	# Series
Macro	1.0693	14.79	407
Micro	0.9918	17.13	472
Finance	1.0292	14.86	485
Industry	1.0650	13.75	410
Demographic	0.9344	4.33	214
Other	0.7960	17.58	12

Usage

from transformers import PatchTSTForPrediction
import torch
import numpy as np

model = PatchTSTForPrediction.from_pretrained("stevevaius/patchtst-monthly-seasonal")
model.eval()

# Input: monthly time series with at least 48 months of history
# Shape: [batch_size, context_length, 1] (univariate)
monthly_values = np.array([...])  # your monthly data
context = torch.tensor(monthly_values[-48:], dtype=torch.float32).unsqueeze(0).unsqueeze(-1)

with torch.no_grad():
    output = model(past_values=context)
    forecast = output.prediction_outputs.squeeze().numpy()

# forecast shape: [18] — next 18 months
print("Forecast:", forecast)