## 1 Introduction
Network analytics has been used to capture complex interactions between customers [23]. [PERSON] et al. [23] introduced tcc2 vec, an approach to representation  
Figure 1: The number of articles by year obtained using the CatBoost method. This result was obtained from the Web Of Science platform with an advanced search query, “TS = (CatBoost) OR AK = (CatBoost)”. learning on enriched call networks, integrating interaction and structural information to capture the dynamic nature of customer interactions. While these studies advance churn prediction in various ways, few fully address how model performance changes when the data used for training and testing are obtained at different times--a key factor in real deployments, where concept drift can emerge. To address these challenges, this article proposes a direct evaluation approach, simulating the real-world application of churn prediction models. We use a sliding window approach for additional (realistic) test data, allowing us to simulate various delays between the training and testing phases. This method provides a detailed analysis of how different delays affect model performance, offering insights that are useful for practitioners. Thus, in this article we investigate the effect of concept drift in the most straightforward way--we compare the performance of the same model when using different time windows. This means that we do not address the problem of concept drift itself but are able to measure it precisely in context of our used data and models. Our case study focuses on churn prediction for a telecommunications provider, utilizing a dataset of call detail records (CDRs) and other aggregated daily user data, covering over 12,000 users for 720 days. The performance of six different classification methods for these data was analyzed in [24], with the Gradient Boosting Classifier achieving the best performance, with an accuracy rate of 0.832 and an F1 score of 0.646. Thus, this was the chosen method of classification in our investigation. More specifically, the CatBoost classifier was used--this classifier offers the widest range of possible ways to tune the method. As noted in [25], CatBoost handles overfitting well, showing good results on testing data. This method was the best of all the methods tested in the aforementioned research. Machine learning models often aim to predict outcomes through either classification or regression [26]. However, important questions are often overlooked, leading to discrepancies between a model's performance during research and its performance in real-world applications [7]. For instance, existing datasets typically do not account for concept drift or the delays between a model's training and deployment; for example, in [6] 11 datasets were analyzed by synthetically splitting existing data into two equally sized learning and testing parts. [PERSON] and [PERSON] [27] emphasized the need to align predictive models with business objectives, proposing an AI-driven personalized pricing model to retain customers. In this article, we address some the following key issues:  
* Behavioral Changes Over Time: The behavior patterns of observed objects can change over time, and delays between the prediction and training intervals can affect the accuracy of the results. While these delays cannot be avoided, they must be taken into account when optimizing the practical outcome of the technology, as demonstrated in [24]. * Limitations of Traditional Data Splitting: The classic approach of artificially splitting a dataset into training, validation, and testing parts sections does not allow for the precise estimation of a model's performance in a real-world scenario. Therefore, an additional set of data must be included, which we refer to as the \"true testing data.\" These data must be qualitatively different from the training/validation data and should consist of information from the future that can simulate and evaluate the model's application with a delay.