# CNN for User Activity Detection Using Encrypted In-App Mobile Data
## 4 Experimental Results and Discussion
### Model Analysis  
The proposed CNN architecture comprises of seven layers excluding the input and output layers (as depicted in Figure 3). The three convolutional layers had 256, 128, and 64 numbers of filters in each layer, respectively. The fully connected layer had 128 nodes. Tanh activation function is applied to the output of every convolutional and fully connected layer. To reduce overfitting, the dropout technique is used to prevent complex co-adaptations on the training data [7]. In this architecture, the Adam optimizer and categorical cross entropy loss function were used.  
The model performance varied with different image sizes as inputs according to our tests. When the image size was too small, there was a sign of learning degradation. When the image size was too large, then the extraction phase took much longer time. Thus, we selected \(28\times 28\) pixel image size, which helped to reduce the run-time and memory consumption while improving detection rate in all our experiments.  
In addition to grayscale images, RGB images were also created from the collected network traffic data set. This was done to observe the performance of in-app activity classification when in-app data is converted to 3D images. Instead of considering the entire flow of an activity, we used the segment-based approach to perform the classification. Four different time windows were used to divide the traffic flows into segments: 1 s, 0.5 s, 0.2 s, and 0.1 s. Table 2 presents the training and validation accuracies, and time needed to train and test the models.  
From the experimental results, it can be observed that the accuracy values obtained for the grayscale models are higher than those recorded for the RGB models for all time windows tested. This is because RGB models suffer from overfitting due to the presence of large number of features resulting from the three color channels. Therefore, grayscale images are selected as inputs to avoid false classification and complexities for the rest of the experiments.  
Validation accuracy was highest when the time window size was 0.2 s. The reason is when traffic traces are segmented into a smaller window size, we were able to obtain plenty of samples. Training the model with large of samples contributed positively towards the classification accuracy. However, this was not true when the window size was further reduced. Even though the sample size increased when traffic traces were segmented  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{8}{c}{**Window Sizes**} \\ \cline{2-9}  & \multicolumn{2}{c}{**1 s**} & \multicolumn{2}{c}{**0.5 s**} & \multicolumn{2}{c}{**0.2 s**} & \multicolumn{2}{c}{**0.1 s**} \\ \cline{2-9}  & **Gray** & **RGB** & **Gray** & **RGB** & **Gray** & **RGB** & **Gray** & **RGB** \\ \hline Training accuracy (\%) & 97 & 84 & 98 & 84 & 96 & 83 & 95 & 81 \\ \hline Validation accuracy (\%) & 83 & 71 & 88 & 76 & 92 & 80 & 86 & 74 \\ \hline Training and testing time & 8 min & 15 min & 19 min & 27 min & 1 h & 1 h & 2 h & 2 h \\  & & & & & & 9 min & 20 min & 30 min & 48 min \\ \hline \hline \end{tabular}
\end{table}
Table 2: Classification performance of grayscale and RGB models.  
into a smaller window size such as 0.1 s, the classification accuracy decreased. This was because the number of frames contained in such smaller window segments is less. When frames were considered individually or when the number of frames was insignificant, they contained very little information to perform the classification.  
With the decrease in window size, leading to an increase in the number of samples, the results present that long times are needed to pre-process, train, and test the models. From the Table 2 data, it can be observed that all grayscale models achieved an average accuracy of 87%. This shows that the proposed model can identify fine-grained in-app activities even by observing only a small subset of an activity's traffic.