## 1 Introduction
* Impact of Delays in Different Time Windows: The effect of delays in training and prediction can vary depending on the specific time window. Observing different time windows is important to understand how model performance changes over time--we demonstrate the informative outcome of such an approach. A churner is defined as a user who has stopped using a specific service, such as telecommunication services from a specific operator. While the most common definition of a churner in telecommunications is a client who has not generated any revenue for three months, alternative definitions may lead to better prediction or retention results, as noted in [24]. This study was dedicated to the specific question of labeling rule selection, and different definitions were compared. In this article, we will follow on from our findings in the previous work and stick to the partial labeling definition, which is defined as a client with an absence of activity for at least 40 days, since this was noted as a good compromise, obtaining much better results than a period of absence lasting 30 days and without significant improvements being shown for periods above 40 days. However, we will also provide data about the most recognized churner-defining activity absence period, which is 90 days. Other authors [4] adopted a 90-day inactivity threshold to define churn. This has two potential drawbacks. First, waiting 90 days to confirm churn delays any intervention and increases the time between model training and actual deployment, exacerbating concept drift. Second, such a strict definition may omit customers who disengage earlier. Consequently, we previously proposed using shorter inactivity periods (40 days) to capture churn more promptly, although these periods can include users who are not definitively \"lost\". In this paper, we strike a balance by using a 40-day threshold (partial churn) for training but demonstrating that this same model can effectively address the 90-day (full churn) scenario with a simple threshold adjustment. This approach will potentially reduce drift while still covering the longer inactivity definition when needed. Existing datasets often do not use qualitatively different data for testing purposes; instead, they typically split a single dataset into two parts [7]. There are studies [4] where authors use different time windows for the testing period; however, the aggregation of data from multiple days into a short time period can lead to overlap between data from different days, similar to oversampling. This practice can lead to overfitting that cannot be detected via testing. Our research aims to provide recommendations for the construction of a dataset that would lead to better model performance in real application scenarios. By simulating real-world delays and observing model performance over different time windows, we demonstrate the importance of accounting for temporal dynamics and concept drift in churn prediction models. Therefore, the main contributions of this research are as follows:  
* We propose a methodology that simulates real-world application delays using a sliding window approach, allowing for the assessment of model performance across different temporal gaps. This addresses the need to handle evolving customer behavior over time (behavioral changes) and exposes concept drift more accurately. * We highlight the limitations of traditional machine learning approaches in the context of changing customer behavior, demonstrating the necessity of using temporally distinct testing data. By comparing future-based \"true tests\" with artificial splits, we show how ignoring realistic data delays can affect the performance metrics. * We provide insights into the impact of concept drift on churn prediction models, showing the need for models that remain robust despite delays in data availability. Our sliding-window experiments capture how shifts in time windows affect both partial and full churn definitions.