# Graph-based Method for App Usage Prediction with Attributed Heterogeneous Network Embedding
## 4 Experimental Evaluation
### Performance Analysis
Which scale to choose for prediction depends on your focus. **Performance of comparison of our framework AHNEAP and baselines. ** Figure 7 shows the comparison of prediction accuracy for AHNEAP and baselines: MFU, MRU, Bayes, Graph Embedding (GE), and our network embedding based framework (AHNEAP) for the selected five users at two temporal scales separately. For baselines and our framework AHNEAP, the prediction accuracy varies with different users. However, the experimental results show that our framework AHNEAP outperforms other strategies significantly almost in any situation at the two temporal scales. It achieves highest about 80% for user B02 at _interval of day_ temporal scale and 74% for user B04 at _interval of day_ temporal scale. However, at _hour of day_ scale, the prediction accuracy is not higher than all the four baselines with user B02. Above all, we analyze Figure 8, which describes application usage record statistics about B02. There are 14,565 application usage records of B02 and 56 applications installed. Seen from the first sub-graph, there are four Apps most frequently used. In addition, records about the top 10 most frequently used applications occupy about 91% of all records. Regardless of MFU or Bayes, the prediction accuracy is calculated based on the probability of the application usage in a certain context. About 91% of records are related to the top 10 applications, so that it is possible that MFU or Bayes can produce a higher rate than our framework AHNEAP. However, benchmarks are not friendly for newly installed apps while AHNEAP can increase possibility through the similarity  
Figure 6: The comparison of two types of time patterns with prediction _Accuracy@4_ of five users: interval of day, hour of day. Figure 7: The comparison of prediction _Accuracy@4_ for the five models: MFU, MRU, traditional Bayes (Bayes), Graph Embedding (GE), and our network embedding based framework(AHNEAP) for the five users, respectively, at _interval of day_ and _hour of day_ temporal scale. between applications, which is calculated by their features. Therefore, our framework AHNEAP is very adaptable in various situations. Even though, in the case of B02, AHNEAP is only slightly worse than the baselines. **Details of Accuracy@3-5:** There is more detailed information about our experiments. Table 1 shows the _Accuracy_ and \(F1\) scores for each user under the two temporal scales when predicting three, four, and five candidates, leading to three observations. To sum up, firstly the prediction accuracy of the method is on the increase with the \(k\) value increasing. This finding is easy to understand that, with the \(k\) value increasing, the denominator of the ratio stays the same. However, the numerator (hit counts) can only keep the same or increasing. Second, the prediction accuracy and F1 score have no inevitable correlation with the input training size according to each user's statistic in Table 2. Thus, even if it is increased when the amount of training data exceeds a certain limit, the accuracy will not be greatly improved. Third, performance under _interval of day_ temporal scale is usually better than that of _hour of day_ temporal scale. The results for user A12 are the only exception. However, from the aspect of effectiveness for a given period, the _interval of day_ scale is not as effective as the _hour of day_.