EventGPT

EventGPT is an open generative pretrained transformer for banking event sequences. It reads a customer's history of transactions and other events, predicts the next event token by token, and gives one embedding per customer and point in time. It was pretrained on the public Multimodal Banking Dataset (MBD-mini: 100,224 clients, 38.8 million transactions and 66.3 million geolocation pings).

• Paper: EventGPT: Next-Event Pretraining on Banking Customer Histories (Burakov, 2026), https://arxiv.org/abs/2611.03302
• Code: https://github.com/deburky/EventGPT

Model

EventGPT is hierarchical. An event encoder turns the tokens of one event (name, type, amount bin, time gap and other fields) into a vector; a causal history decoder reads these vectors with rotary positions taken from elapsed time since the client's first event; a token decoder predicts the next event's tokens from the history state. The models here have 23 million parameters (width 384) and were trained on 655 million tokens.

Variants

Each variant is a subfolder with model.safetensors, config.json and vocab.json (the token table and amount-bin edges the model was trained on; histories must be tokenized with it).

variant	events	training windows	official mean ROC-AUC
aligned-s42, aligned-s43	transactions and geolocation	aligned, query-key normalization	0.786, 0.788
aligned-profile-s42, aligned-profile-s43	as above, with early fusion of the profile fields	aligned, query-key normalization	0.803, 0.807 with late fusion of the profile fields
trx-s42, trx-s43	transactions only	random offsets	0.794, 0.791
trx-oot-s42, trx-oot-s43	transactions only, pretrained on events before February 2022	random offsets	0.790, 0.788

The official mean is the mean ROC-AUC over MBD's four product targets on test fold 0, with a logistic head on the frozen embedding. On the same rows, CatBoost on engineered features reaches 0.816 and on token counts over the full history 0.803. A rank-averaged ensemble of EventGPT (with late fusion of the profile fields and token counts) and the engineered features reaches 0.830.

The paper repeats this comparison with each of MBD's five folds as the test fold, pretraining new models on the other four. Averaged over the five folds, the embedding scores 0.787 against 0.808 for the engineered features, 0.804 with the profile fields and token counts, and the ensemble 0.824. The ensemble is above the engineered features on every fold and seed.

Without labels, the next-event distribution scores five "first transaction of a kind next month" targets at ROC-AUC 0.78 to 0.91 (aligned variants, mean of two seeds).

Use

from eventgpt.hub import from_pretrained

model, vocab = from_pretrained("deburky/EventGPT", "aligned-profile-s42")

Install the code from the GitHub repository. Tokenizing MBD-mini (egpt prepare), embedding histories (egpt probe) and zero-shot scoring (egpt zero-shot) are commands of the egpt CLI.

Limitations

The vocabulary is MBD's event schema. Another institution's events need their own tokenization and their own pretraining; these weights serve to reproduce and extend the paper's results on MBD. The models were pretrained on the pretraining clients' events from 2021 and 2022, which include the months in which the test clients' labels fall (never the test clients themselves); pretraining only on transactions before February 2022 (trx-oot) lowers the official mean by 0.004 to 0.006. The weights here are pretrained with fold 0 held out; the paper's models for the other four test folds are not released.

License and citation

The weights are released under CC BY 4.0, the license of MBD-mini, on which they were trained. Please cite the paper and MBD (Mollaev et al., 2024).

@misc{burakov2026eventgpt,
  title  = {EventGPT: Next-Event Pretraining on Banking Customer Histories},
  author = {Burakov, Denis},
  year   = {2026},
  eprint = {2611.03302},
  archivePrefix = {arXiv},
  primaryClass = {stat.ML},
  url    = {https://arxiv.org/abs/2611.03302}
}