CoLES: Contrastive learning for event sequences with self-supervision
Authors: Dmitrii Babaev, Nikita Ovsov, Ivan A Kireev, Gleb Gusev, Maria Ivanova, Alexander Tuzhilin
Keywords: representation learning, contrastive learning, neural networks, event sequiences
Abstract: We address the problem of self-supervised learning on discrete event sequences generated by real-world users. Self-supervised learning incorporates complex information from the raw data in low-dimensional fixed-length vector representations that could be easily applied in various downstream machine learning tasks. In this paper, we propose a new method CoLES, which adopts contrastive learning, previously used for audio and computer vision domains, to the discrete event sequences domain in a self-supervised setting. Unlike most previous studies, we theoretically justify under mild conditions that the augmentation method underlying CoLES provides representative samples of discrete event sequences. We evaluated CoLES on several public datasets and showed that CoLES representations consistently outperform other methods on different downstream tasks.