# CNN for User Activity Detection Using Encrypted In-App Mobile Data
## 4 Experimental Results and Discussion
### Performance Comparison  
Even though in this work a CNN model was used to perform the traffic classifications, other types of neural networks could also be employed for this purpose, such as a Deep Neural Network (DNN). This section reports on the performance comparison between the DNN and CNN models when they are used in our tests.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**F** & **I** & **G** & **M** & **S** & **V** & **W** & **Y** \\ \hline Uttuv—27 & Uttuv—21 & Uttuv—21 & Uttuv—21 & Uttuv—26 & Uttuv—19 & Uttuv—19 & Fblive—20 \\ \hline Skvc—13 & Fblive—15 & Fblive—16 & Fblive—17 & Fblive—22 & Fblive—15 & Fblive—14 & Skvc—11 \\ \hline Msgac—9 & Skvc—9 & Skvc—8 & Skac—9 & Msgac—8 & Skvc—7 & Skvc—8 & Msgac—8 \\ \hline Skv—9 & Skac—8 & Skac—7 & Skvc—7 & Fbpv—6 & Skac—7 & Skac—7 & Skac—8 \\ \hline Skac—8 & Msgac—7 & Msgac—6 & Fbpv—6 & Indvc—6 & Msgac—6 & Msgac—6 & Skv—8 \\ \hline Indvc—6 & Skv—7 & Fbpv—6 & Skv—5 & Uttuv—4 & Skv—6 & Skv—5 & Fbpv—7 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Misclassified in-app activities with misclassification percentage (%) values.  
In both models, the output types were in-app activities. But the format of the input was different: in CNN, images were provided as the input whereas in DNN, statistical features related to the traffic flows constituted the input. Although the same data set was used for both DNN and CNN models to perform tests, we employed the network architecture that resulted in the highest accuracy for each model instead of using the same network architecture across the board for a fair comparison. This was needed as the input format was different in both cases, which had a direct influence on accuracy performance. For CNN, the network architecture proposed in Section 4.1 was used. For the DNN model, four hidden layers were used with 1024, 512, 256 and 128 nodes in each layer. Tanh activation function was used for all layers expect for the output layer which used the SoftMax layer. The 48 statistical features used in [4] were utilized as the input of DNN. Figure 8 shows the training and validation accuracies obtained at 0.5 s and 0.2 s time window sizes when both models were used to perform in-app activity classification.  
From Figure 8, it is noted that the DNN model has recorded the highest accuracy values compared to the CNN model in all categories. But when looked closely at the comparison at each time window size, the difference in accuracy values is maximum 5%. Significantly, all the accuracies of the CNN model are at 88% or above. Therefore, it can be concluded that both models can accurately detect previously trained in-app activities.  
To compare the detection of unknown in-app data, it can be observed from Figure 9 that the CNN model has outperformed the DNN model at all the instances when different noise test traffic data sets (Test app) were applied. In both models, Gmail and Viber reported the highest noise detection rates. Compared to the DNN model, when the CNN model is employed, there is an increment of 19%, 18%, and 12% noise detection rates when YouTube, Messenger, and Skype are input as test app, respectively.  
Even though the ability to detect previously trained in-app activities correctly by the CNN model is slightly weaker than that of DNN, its ability to detect previously untrained/unknown in-app activities as noise data is much stronger than that of DNN. Therefore, when the overall performance is considered, the proposed CNN model outperforms DNN. This is because when traffic flows are converted to images, the model can apply image processing techniques to reveal interesting properties of the traffic. As such, applying the proposed CNN model on the input traffic images allows for extracting and selecting salient features that enable the model to learn to differentiate trained and unknown traffic.  
Figure 8: Accuracy comparison of the CNN and DNN models.