Tabby

A probabilistic univariate time series foundation model. One forward pass returns 99 quantiles of the future.

Tabby is the post-trained model: a Tabby-Pretrain backbone that was pretrained for 165,000 steps and then adapted on GIFT-Eval train set by a small prompt module, with the backbone frozen throughout. This repository ships both halves merged into one checkpoint, so it runs on its own — you do not need to fetch a separate backbone. The frozen backbone alone is published as Tabby-Pretrain.

Resources

Quick start

pip install torch transformers einops numpy

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained("<namespace>/Tabby", trust_remote_code=True).eval()

history = torch.randn(4, 8096).cumsum(-1)      # [B, Tc] — any length; NaN = unobserved
q = model.predict(history, prediction_length=96)   # [4, 99, 96]

median = q[:, 49]                               # the 0.50 level
lo, hi = q[:, 9], q[:, 89]                      # the 0.10 / 0.90 band

model.quantile_levels gives the level of each of the 99 rows, 0.01 … 0.99. Index 49 is the median.

Inputs are forecast one series at a time: pass a 1-D sequence, a 2-D [B, Tc] batch, or anything torch.as_tensor accepts. Series may be shorter than the window and may contain NaN, which is read as unobserved rather than as a value. Nothing needs to be scaled or normalised beforehand — normalisation is internal and the output comes back in the input's own units.

Results

Seasonal-Naive normalised geometric means at history_length=8096. GIFT-Eval is 97 configs; TIME is 98 tasks that are disjoint from the post-training pool.

	GIFT-Eval MASE	GIFT-Eval CRPS	TIME MASE	TIME CRPS
Tabby-Pretrain, zero-shot	0.7192	0.4922	0.6915	0.5715
Tabby	0.6966 (−3.14 %)	0.4806 (−2.36 %)	0.6770 (−2.10 %)	0.5644 (−1.24 %)

Architecture

Total parameters	146.70 M
— frozen backbone	145.84 M, 20 layers, d_model 768, 12 heads, MLP 3072
— prompt module	0.85 M stored, of which 0.73 M trainable (0.50 % of the backbone)
Patch size	16
Backbone window	8192 steps (512 patches)
History used	8096 steps, horizon 96 (the published protocol)
Quantiles	99, levels 0.01 … 0.99

The prompt is prepended to the patch-token sequence as M = 160 extra tokens that carry no positional embedding:

P(x) = P_shared + sigma(g) * P_adaptive(x)

P_shared is 160 learned vectors. P_adaptive(x) is generated from summary statistics of the input window through a rank-4 map, so the prompt depends on the series rather than being one global prefix. The history is split into 16 adaptive segments and a cross-attention refiner lets each prompt slot read them, which is what makes the prompt sensitive to where in the history the information sits. g is a learned scalar gate, initialised at sigma(+5) ≈ 0.993.

Files

File	What it is
model.safetensors	merged backbone + prompt weights, 587 MB, 278 tensors
config.json	full config for both halves, with auto_map for trust_remote_code
modeling_tabby.py	TabbyForForecasting: the inference model
configuration_tabby.py	TabbyConfig
patchtstfm_adapter.py, modeling_patchtst_fm.py, basic.py, normalization.py, configuration_patchtst_fm.py, tools.py	the backbone, vendored from the pretraining repository
prompt_blocks.py	statistics extractors, segment refiner, prompt generator
input_preprocessing.py	the visible-history-only asinh normalisation

The backbone weights are bit-for-bit those of Tabby-Pretrain, and the prompt weights are bit-for-bit the released seed-4 prompt checkpoint. Merging changed no value; it only renamed keys onto one module tree.

Things that change the numbers

1. prediction_length. The published results are at 96. Other horizons work — pass any 1 ≤ H ≤ 8176 — but they are not what the table above measures.
2. History length. The results use 8096 steps. Feeding less is allowed and degrades gracefully, but the segment split anchors to 8096, so a much shorter history shifts the prompt's view of the series. History longer than 8192 − H is truncated to the most recent values.
3. Batch size changes the last bits. Different batch sizes select different GEMM kernels, so forecasts move by about 1e-7 relative. This is ordinary floating-point behaviour, not a bug, and it is why a batched run and a one-at-a-time run are not bitwise identical.
4. Multivariate data. Tabby is univariate. Multivariate series are forecast one variate at a time with no cross-variate information.

Training

The backbone was pretrained for 165,000 steps in bf16 on a mix of real series (BLAST, GIFT-Eval Pretraining pool) and synthetic series (Chronos KernelSynth, CauKer-V2), with a masked-patch objective and a 99-quantile pinball loss. See Tabby-Pretrain for the full recipe.

Post-training adapted the prompt only, on 48 GIFT-Eval tasks: prompt length 160, history 8096, horizon 96, segmented context-aware prompts (S=16, adaptive_horizon, min_seg_len 48, season_period 48), rank-4 context map, both gates initialised at logit 5.0. AdamW at lr 1e-3, weight decay 1e-4, cosine anneal over 20 epochs to 10 % of peak, gradient clip 1.0, effective batch 48, seed 4, stopped at epoch 20 (val_loss 1.9750). Roughly 3.5 hours on one H200. For the 26 short-horizon tasks the training target is truncated to the horizon each task is actually evaluated at.

Reference environment: Python 3.12.3, PyTorch 2.6.0+cu124.

License

CC BY-NC 4.0 — non-commercial use only.