Polifemus/prefix-next-activity

Next-Activity Prediction on BPI Event Logs — GNN and Fine-Tuned Transformer Models

This repository contains the models from a controlled study of next-activity prediction in predictive process monitoring on seven BPI Challenge event logs. Two model families were trained and compared with simple baselines:

1. PrefixGNN — a graph neural network over the directly-follows graph of each case prefix, trained from scratch per log.
2. EventTransformer — a ~27M-parameter transformer foundation model pretrained with masked event modeling on a multi-log corpus (12→18 logs), then fine-tuned per log ("FT" checkpoints).

All comparisons use the same honest protocol: temporal, case-level splits; model/ensemble selection on the validation set only; the first-order argmax baseline included. Headline finding: the fine-tuned transformer beats the GNN on the medium/large logs and on a held-out log (pure transfer), and a validation-selected ensemble of both is never worse than the GNN alone.

Models in this repository

folder	model	log	test accuracy	macro-F1	notes
gnn-bpi2012/	PrefixGNN	BPIC 2012	0.7742	0.6372	best single model on this log
gnn-bpi2017/	PrefixGNN	BPIC 2017	0.8151	0.6365	kept for reference; superseded by ft-bpi2017
ft-bpi2017/	EventTransformer (FT)	BPIC 2017	0.8210	0.6632	+0.6pp acc / +2.7pp F1 over GNN
ft-bpi2018/	EventTransformer (FT)	BPIC 2018	0.8718	0.5347	+0.6pp acc / +3.3pp F1 over GNN
ft-bpi2015_1/	EventTransformer (FT)	BPIC 2015 (municipality 1)	0.4277	0.1863	held-out log: pure transfer, +1.1pp acc over GNN

Note on the HF pipeline tag — this repository spans two model families: the gnn-* checkpoints are graph neural networks over directly-follows graphs; the ft-* checkpoints are standard sequence transformers. graph-ml reflects the graph-based half and the project's process-mining roots — check which checkpoint you download (see the table above).

Two inference scripts are provided: inference.py (GNN, requires torch-geometric) and fm_inference.py (transformer, requires only torch + numpy).

Quickstart

pip install -r requirements.txt

# graph models (BPIC 2012 / 2017 demos)
python inference.py

# fine-tuned transformer models (BPIC 2017 / 2018 / 2015-1 demos)
python fm_inference.py

# GNN
from inference import load_model, predict
model, vocab = load_model("gnn-bpi2012")
events = [
    {"activity": "A_SUBMITTED",       "resource": "112", "timestamp": "2011-09-30 22:38:44"},
    {"activity": "A_PARTLYSUBMITTED", "resource": "112", "timestamp": "2011-09-30 22:38:44"},
    {"activity": "A_PREACCEPTED",     "resource": "112", "timestamp": "2011-09-30 22:39:37"},
]
predict(model, vocab, events, amount=20000.0, top_k=3)

# transformer
from fm_inference import load_model, predict
model, vocab = load_model("ft-bpi2017")
predict(model, vocab, events=events, top_k=3)

events is the running case prefix (activity / resource / timestamp per event, monotonic timestamps); amount is the optional case-level requested amount. Prefixes longer than 100 events are truncated to the most recent 100; unknown activities/resources map to UNK.

Protocol (same for every model and log)

• One event per activity (COMPLETE lifecycle events where applicable); cases sorted by timestamp.
• Temporal, case-level split 70/15/15 by case start date; every prefix of every case is an example (95%+ of the logs' prefixes), truncated to the last 100 events.
• Model selection and ensemble weights chosen on validation only; test sets used once for final reporting. (The literature reports example leakage in many NAO papers when prefixes of the same case span train/test — the split here is per-case, so this cannot happen.)
• Baselines included everywhere: majority class, MLP over the last 10 activities, first-order argmax P(a | last activity) (a strong count-based baseline in recent work).
• Ensembles are convex combinations of per-model probabilities with weights searched on validation (grid, step 0.05).

Model architectures

PrefixGNN (from scratch, per log): GINEConv × 3 message passing over the bidirectional directly-follows chain of the prefix; activity embedding 32d + resource 16d + 7 numeric features (elapsed time since previous event / since case start, hour sin/cos, weekday, log-amount, position); readout = MLP(concat(mean-pool, max-pool, last-event embedding)); ~160k parameters.

EventTransformer (pretrained once, fine-tuned per log): transformer encoder, d_model 512, 8 layers, 8 heads, len 256, 27M parameters; token = activity embedding 64d + resource 32d + log-id embedding 16d + 6 numeric features (same times/amount as above). Pretraining = masked next-event modeling over windows (256/64) sampled from a 12→18-log corpus, 40 epochs (20M trainable events). Fine-tuning per log = new local classification head initialized from the global head, full bidirectional attention, 30 epochs, early stopping on validation accuracy (patience 5), lr 3e-4 cosine.

Results — all evaluated logs (test sets)

Accuracy (macro-F1) · best single model and best validation-selected ensemble in bold:

log	cases / events / classes	majority	argmax-1	MLP-last10	GNN	FM fine-tuned	best ensemble (val-selected)
BPIC 2012	13.1k / 164k / 23	0.1303	0.5726	0.7636	0.7742 (0.637)	0.7619 (0.611)	0.7768 (gnn+distill+argmax)
BPIC 2014	46.6k / 467k / 39	0.1839	0.3637	0.4728	0.5368 (0.351)	0.5272 (0.356)	0.5414 (gnn+ft+zs)
BPIC 2015-1 (held out)	1.2k / 52k / 398	0.0234	0.1524	0.3504	0.4169 (0.179)	0.4277 (0.186)	0.4571 (gnn+ft+zs)
BPIC 2017	31.5k / 475k / 24	0.1035	0.7554	0.8121	0.8151 (0.637)	0.8210 (0.663)	0.8210 (ft)
BPIC 2018	43.8k / 2.51M / 41	0.1931	0.6614	0.8419	0.8655 (0.502)	0.8718 (0.535)	0.8794 (gnn+ft+argmax)
BPIC 2019	251.7k / 1.60M / 42	0.2831	0.4931	0.7562	0.7906 (0.467)	0.7757 (0.477)	0.8032 (gnn+ft+zs)
Traffic fines	150.4k / 561k / 11	0.3091	0.8347	0.8834	0.8837 (0.697)	0.8835 (0.691)	0.8836 (tie with GNN)

What replicates across logs

• The ensemble never loses: a validation-selected mix of GNN + fine-tuned transformer (see ft-* and gnn-* checkpoints) matches or beats the GNN alone on all seven logs, by up to +4.0pp (BPIC 2015-1) and +1.3pp (BPIC 2019).
• The fine-tuned transformer wins outright on BPIC 2017 (+0.6pp acc / +2.7pp F1), BPIC 2018 (+0.6pp / +3.3pp) and on the held-out BPIC 2015-1 (+1.1pp / +0.8pp — the only log of the seven never seen during pretraining: this is the cleanest evidence that the multi-log pretraining transfers).
• Where the GNN still wins: small/short-case logs (BPIC 2012) and very short-case, single-domain logs (BPIC 2014, traffic fines — where the transformer ties). Raw log size does not predict which family wins; case structure does.
• The argmax baseline is remarkably strong on short/skewed cases (0.83 on traffic fines, 0.66 on BPIC 2018) and helps the ensemble only on BPIC 2012 and 2018 — on the other logs the learned models (and their ensembles) stay clearly ahead (e.g. +30pp on BPIC 2019).
• Auxiliary findings: early-stopping patience 5 was verified against full-30-epoch runs (no hidden gains); warm restarts degrade (the fine-tuned models are already at their optimum); the causally-pretrained transformer is a better zero-shot model with causal attention but fine-tunes better with full bidirectional attention.

Selective prediction (GNN checkpoints)

By abstaining on low-confidence predictions, accuracy on the predicted subset approaches 1:

confidence threshold	BPIC 2012: coverage / accuracy	BPIC 2017: coverage / accuracy
0.80	42.8% / 96.99%	63.5% / 96.18%
0.90	35.1% / 99.46%	48.4% / 99.23%
0.95	29.6% / 99.91%	41.5% / 99.86%

Files

path	content
inference.py / fm_inference.py	standalone inference (GNN / transformer), with demos
gnn-bpi2012/, gnn-bpi2017/	model.pt + config.json + vocab.json (+ selection.json)
ft-bpi2017/, ft-bpi2018/, ft-bpi2015_1/	model.pt (fp32, ~107MB) + config.json + vocab.json (local vocabulary + global embedding maps)

The transformer checkpoints are the best-validation fine-tuned weights; the packaged predictions were verified to be identical to the training-time evaluation outputs (100% argmax agreement on 447k test prefixes).

Limitations

• Checkpoints are tied to their log's vocabulary and domain; for a new log, re-run the fine-tuning (the code paths are log-parametric, the vocabulary is not).
• Temporal drift exists inside these logs; expect future-period accuracy to be closer to the temporal test results reported here than to random-split numbers seen in the literature.
• Single training seed per fine-tuned model (the GNN results were additionally verified across 3–4 seeds; see macro-F1 spread in the BPIC 2017 study).
• Probabilities are not formally calibrated.
• The pretraining corpus included the cases of the logs used here except BPIC 2015-1 and traffic fines (which are the held-out transfer tests). "Zero-shot" in the tables means "no task-specific fine-tuning", not "unseen log" unless explicitly marked.

Data & licensing

Only trained artifacts are published in this repository (model weights, the small vocabularies of anonymized activity/resource labels each model needs, and metrics). The raw event logs are not redistributed; they remain under their own licenses and must be cited when reused:

dataset	used for	license (verified 2026-10-14)
BPIC 2012 (gnn-bpi2012)	training/report	4TU General Terms of Use
BPIC 2017 (gnn-bpi2017, ft-bpi2017)	training/report	4TU General Terms of Use
BPIC 2015, municipality 1 (ft-bpi2015_1, held out)	transfer test	4TU General Terms of Use
BPIC 2018 (ft-bpi2018)	training/report	CC0
BPIC 2014, BPIC 2019, traffic fines, BPIC 2013/2020, sepsis, helpdesk (pretraining corpus and report tables)	pretraining/report	4TU General Terms of Use (2019: CC BY 4.0)

The "4TU General Terms of Use" permit download and use of the datasets, which must be properly acknowledged (author, year, title, DOI), and do not transfer IP rights; the BPIC logs are anonymized by their publishers. Model weights released here are our own trained artifacts (derived works), published under MIT. If you use these models or numbers, please cite both this repository and the underlying datasets (see References).

🇮🇹 Sintesi in italiano

Questo repository pubblica i modelli di uno studio controllato di next-activity prediction (process mining predittivo) su sette log BPI. Due famiglie a confronto — una GNN (message passing sulla catena directly-follows del prefisso) e un transformer da ~27M parametri pretrainato su un corpus multi-log con masked event modeling e poi fine-tunato per log — più baseline (majority, MLP, primo-ordine argmax) e ensemble a pesi scelti solo su validation.

Risultati chiave (tutti su test, protocollo temporale per caso):

• Il transformer fine-tunato batte la GNN su BPIC 2017 (+0.6pp), BPIC 2018 (+0.6pp acc, +3.3pp F1) e sul log tenuto fuori BPIC 2015-1 (+1.1pp: trasferimento puro).
• L'ensemble validation-selected non è mai peggiore della GNN su nessuno dei 7 log (fino a +4.0pp).
• La GNN resta migliore su log a casi corti (BPIC 2012, 2014, multe). La dimensione grezza non decide chi vince: conta la struttura dei casi, non la scala.
• L'argmax di primo ordine è forte sui log a casi corti/sbilanciati ma aggiunge all'ensemble solo su BPIC 2012 e 2018.
• Controlli metodologici: split per caso (niente leakage da prefissi dello stesso caso), selezione su validation, patience 5 verificata contro run a 30 epoche intere, warm restart inutile.

Modelli: gnn-bpi2012, gnn-bpi2017 (GNN), ft-bpi2017, ft-bpi2018, ft-bpi2015_1 (transformer). Demo: python inference.py e python fm_inference.py.

Citation

@misc{prefix-next-activity,
  title  = {Next-Activity Prediction on BPI Event Logs: GNN and Fine-Tuned Transformer Models},
  author = {Polidori, Filippo},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Polifemus/prefix-next-activity}}
}

Author

Filippo Polidori (Polifemus, github.com/Polifemu).

References

• BPIC 2012: van Dongen, B.F. BPI Challenge 2012. 4TU.ResearchData, dataset. https://data.4tu.nl/datasets/533f66a4-8911-4ac7-8612-1235d65d1f37
• BPIC 2017: van Dongen, B.F. BPI Challenge 2017. 4TU.ResearchData, dataset. https://data.4tu.nl/datasets/34c3f44b-3101-4ea9-8281-e38905c68b8d
• BPIC 2018: van Dongen, B.F. BPI Challenge 2018. 4TU.ResearchData, dataset. https://data.4tu.nl/datasets/443451fd-d38a-4464-88b4-0fc641552632
• Traffic fines: de Leoni, M.; Mannhardt, F. (2015). Road Traffic Fine Management Process. 4TU.ResearchData, dataset. https://doi.org/10.4121/uuid:270fd440-1057-4fb9-89a9-b699b47990f5
• BPIC 2015 municipalities: van Dongen, B.F. BPI Challenge 2015. 4TU.ResearchData, dataset (municipalities 1–5).
• Lischka, Rauch, Stritzel. Directly Follows Graphs Go Predictive Process Monitoring With Graph Neural Networks. arXiv:2503.03197, 2025.
• Revisiting PPM in the Age of Foundation Models. arXiv:2607.27797, 2026.
• David vs Goliath: Argmax vs LSTM, Transformer, LLM. arXiv:2606.15868, 2026.