## 3 Results
### Analysis of the Results  
In Figures 4-6, the main metrics are provided for different cases, as discussed in Section 3.3. Note that the accuracy metric, in the context of our problem with imbalanced data, complements the main F1 score metric. Moreover, as can be seen in the figures, the overall patterns were similar in terms of F1 score and accuracy, although the F1 score shows more pronounced differences between partial and full churn scenarios. For a brief evaluation of the mode, we can refer to the \"Full Train-True Test-Partial Churn\" scenario as the main realistic practical scenario. The mean performance metrics across all time windows were 0.798 for accuracy, 0.704 for F1 score, and 0.8868 for AUC. Thus, we will omit further analysis of the accuracy metric. As we can see from the metrics shown in Figures 4-6, the full train and partial train graphs differ only slightly, suggesting that the reduction in 20% of the data used to label the artificial test data was not critical. Still, we were forced to compare them, because our main goal here was to compare true test and artificial test scenarios; thus, we maintained the same training sample size. In this section, we present the results, focusing on the F1 score, area under the curve (AUC), recall, and precision over time. We studied these metrics to determine how the model performs under different training and testing situations, showing how concept drift affects the model and how the partial and full churn definitions are related. Looking at the F1 results (Figure 4) for the \"Partial Train-True Test-Partial Churn\" and \"Full Train-True Test-Partial Churn\" cases shows that their curves look almost the same for most modeling periods, indicating that the smaller training set performs almost as well as the full dataset. Practically, this means that the same level of partial churn prediction accuracy can be maintained with fewer training examples, saving resources while preserving performance. Also, it means that we were able to compare true test and artificial test (when a part of the potential training data are reserved for testing purposes) scenarios using the same training sample size. The AUC values (Figure 5) are always high for all settings, varying between 0.87 and 0.93. The AUC results for the \"Full Train-True Test-Partial Churn\" and \"Partial Train-True Test-Partial Churn\" hover at around 0.87-0.90. This range indicates that robust classification can be carried out using the partial churn definitions, though the results are slightly lower than the maximum AUC values seen in other scenarios. Nevertheless, the AUC differences between the full and partial churn scenarios were the smallest of all. Figure 7 illustrates the recall for partial and full churn scenarios. The \"Full Train-True Test-Full Churn\" setup achieved recall values of around 0.9, indicating that the model effectively identifies full churners. The recall for partial churn scenarios is lower, even though full churners form a smaller subset of the partial churners. Note that the full churn problem obtained worse F1 performance (Figure 4) results compared to the partial churn problem despite the fact that the recall was better, i.e., using partial churn labels in training data for full churn prediction leads to more true positives being predicted at the cost of more false positives, which is reflected by the significantly lower precision score (Figure 8). Partial churner labels are formed from softer churn requirements compared to full churner  
Figure 5: AUC (Area under curve). Figure 6: Accuracy. labels, meaning that the signal indicating churn is stronger and true positives increase; this inevitably increases the recall. However, Figure 8 shows that the full churn precision is lower, likely because the inclusion of borderline cases (users who are somewhat inactive but not definitively) leads to more false positives. A higher partial churn precision is a natural consequence of a model being trained on partially labeled data.