EBES: Easy Benchmarking for Event Sequences
Authors: name: Dmitry Osin; name: Igor Udovichenko; name: Viktor Moskvoretskii; name: Egor Shvetsov; name: Evgeny Burnaev
Published: 2024-10-04 13:03:43

Abstract: Event sequences, characterized by irregular sampling intervals and a mix of
categorical and numerical features, are common data structures in various
real-world domains such as healthcare, finance, and user interaction logs.
Despite advances in temporal data modeling techniques, there is no standardized
benchmarks for evaluating their performance on event sequences. This
complicates result comparison across different papers due to varying evaluation
protocols, potentially misleading progress in this field. We introduce EBES, a
comprehensive benchmarking tool with standardized evaluation scenarios and
protocols, focusing on regression and classification problems with
sequence-level targets. Our library simplifies benchmarking, dataset addition,
and method integration through a unified interface. It includes a novel
synthetic dataset and provides preprocessed real-world datasets, including the
largest publicly available banking dataset. Our results provide an in-depth
analysis of datasets, identifying some as unsuitable for model comparison. We
investigate the importance of modeling temporal and sequential components, as
well as the robustness and scaling properties of the models. These findings
highlight potential directions for future research. Our benchmark aim is to
facilitate reproducible research, expediting progress and increasing real-world
impacts.