# CNN for User Activity Detection Using Encrypted In-App Mobile Data
## 4 Experimental Results and Discussion
### Unknown In-App Data (Noise) Detection  
In noise detection tests we use leave-one-out approach where two data sets were created, namely training and noise data sets. From the eight apps considered in the experiments, each time an app was singled out and used to create the noise data set. The remaining seven apps were used to create the train data set. While the training data set was used to train the model, the noise data set (with in-app activities unknown to the trained model) was input to the trained model to determine its ability to detect unknown data. The result of this experiment is shown in Figure 5.  
In Figure 5, F, I, G, M, S, V, W, and Y denote Facebook, Instagram, Gmail, Messenger, Skype, Viber, WhatsApp, and YouTube, respectively. On each vertical bar in Figure 5, the apps used to train the model at each instance are denoted. The app labelled on the X axis is the app in the noise data set. Let's us explain how to interpret Figure 5. The first vertical bar in Figure 5 shows that a CNN model is trained with the in-app activity data from the following 7 apps: I, G, M, S, V, W and Y. The in-app activity data from F is kept away from the training process. During the testing, the in-app activity data from F is used to determine the robustness of trained CNN model. As shown in the first bar in Figure 5, the trained CNN model successfully identified more than 90% of the in-app data from F as unknown data. As shown in Figure 5, the proposed method achieved 75% or more in detecting noise with average accuracy of 88%. Data from Gmail and Viber applications were detected with 94% accuracy. However, the proposed model couldn't distinguish data from Skype with the trained data only 75% of the Skype traffic got correctly detected as unknown traffic. The remining 25% was misclassified as in-app activities that belong to the training data set. To understand the nature of the misclassified traffic, further analysis was performed on the apps to which the unknown traffic got classified. The results are presented in Figure 6  
Figure 5: Noise detection rates of unknown apps. in percentage values (%). For example, only 8% of the data from F is classified as known data (see first bar in Figure 5). The distribution of this 8% of the misclassified data from F is shown in the first row in Figure 6. Majority of this data (35%) are assigned to S (Skype). Looking at Figure 6, the majority of the misclassified data was assigned to Facebook. We observed that there was a high correlation among all the considered apps with Facebook, which contributed to the misclassification. For Instagram, Gmail, Messenger, Viber, and WhatsApp, the second and third highest misclassifications came from Skype and YouTube, respectively. This can also be seen by plotting the correlation between these apps (see Figure 7). Figure 7a shows the correlation plots of Facebook with four different apps. Figure 7b shows the correlation plots of Skype and YouTube with two different apps. In Figure 7a,b the  
Figure 6: Confusion matrix (% values). Figure 7: Correlations among different apps. (**a**) correlation of Facebook with Instagram, Messenger, Skype and YouTube. (**b**) correlation of Skype vs YouTube and Instagram with YouTube and Skype. (**c**) correlation of WhatsApp with Viber and Gmail. points are closely packed, which means the strength of the correlation was high. Apps that are highly correlated with each other have similar in-app activities with similar behavior. Looking at the W column in Figure 5, the least amount of misclassification resulted from WhatsApp, which was 1% for all the test apps. This was due to the least correlation between W and other apps as shown in Figure 7c. It can be observed that the points are distributed loosely which means the considered apps are only slightly correlated to each other. The behavior of Facebook, Instagram, Gmail, Messenger, Skype, Viber, and YouTube in-app activities was not similar to WhatsApp's in-app activities, which makes them less correlated to each other.