# Predicting Player Churn of a Free-to-Play Mobile Video Game Using Supervised Machine Learning
## 5 Results
### Churn Prediction for Varying Periods  
Further research was conducted using variable period duration and an adjusted feature set, as well as additional metrics. Average session count, duration, and event count were retained from the first attempt, while average money spent per session was replaced with the lifetime value parameter. Several new features were introduced: chapter start, completion and drop counts, story unlock, and completion counts, as well as level of engagement and the engagement vector. While the count features are straightforward, the final two require further explanation. Level of engagement is a feature that models playtime unevenness throughout the feature extraction period. It is calculated as the difference in event count between the first and last 30% of the period. A negative value indicates that the user played more toward the end of the observed time interval and is thus less likely to churn, as opposed to a player whose level of engagement is positive. An engagement vector offers a more detailed view of player activity distribution. It consists of 10 elements, each representing 10% of the feature extraction period's total running time. An element's value is the percentage of events in the entire period that were logged in the corresponding interval. For example, the third element's value is equal to 0.25 if 25% of recorded events happened between 20% and 30% of the feature extraction period. These new features proved highly valuable for increasing the performance of obtained models. Various combinations of feature extraction and churn period duration were extensively tested, coupled with several machine learning algorithms. For feature extraction, intervals of 1, 2, 3, and 7 days were considered, while churn was predicted over a period lasting between 1 and 7 days, for a total of 28 combinations. The resulting profiles were used as input for logistic regression, random forest, and K-nearest neighbors. Models constructed in this manner were evaluated using previously described metrics. Judging solely based on precision, the RF algorithm's 72% would prevail, while K-nearest neighbors ranks the lowest with 69%. However, in terms of F1 score, logistic regression achieved the highest result of 0.78. Taking AUC (Figure 8) and AP (Figure 9) into account, the RF algorithm exhibits the best performance for the given feature set, as shown in Table 2. It should be noted that the listed figures present the performance of models in which the feature extraction period is one day, while churn period is also one day. When compared to initial results, the accuracy is significantly higher, which can be attributed to new features. As expected, the accuracy and recall increase with longer periods of feature extraction, and decrease slightly the longer the churn period is (as certain people will not show on the first day and will be labeled as churners, but will show on the second day, as showcased by Figure 2). Overall, the accuracy of the model prediction varies from 66% to 95%. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Churn Period**} & \multicolumn{4}{c}{**Feature Extraction Period**} \\ \cline{2-5}  & **1 Day** & **2 Days** & **3 Days** & **7 Days** \\ \hline
1 day & 0.72, 0.73 & 0.87, 0.89 & 0.91, 0.92 & 0.95, 0.96 \\
2 days & 0.69, 0.69 & 0.85, 0. The ROC curve for this model is shown in Figure 8 and is used to calculate the AUC metric. The AP (average precision) metric is computed as average precision over recall. The precision-recall curve is presented by Figure 9.  
Additionally, precision, recall, and F1 score by feature extraction period duration are presented in Figure 10, with the churn period being 1 day, while the same curves calculated by changing the churn period length can be seen in Figure 11, with the feature extraction period corresponding to one day. It is interesting to note that, based on an only 1-day (feature extraction period), the precision, recall, and F1 metrics can be calculated for a 7-day churn period with values around 65%.