## 1 Introduction  
With the rise in Internet usage, proper management of the network traffic in modern ISPs has become a challenge of paramount importance. Categorizing each network packet on the fly is key to proper decision-making. For many applications, such as the quality of service policing or traffic shaping, routing might be more efficient if the type of data consumed is known. For example, in Internet telephony, latency is key and dropped packets are acceptable, whereas in a batch download latency is acceptable, but there must be guarantees that all information is received at least once. This packet classification cannot be performed offline, i.e., after the network flow has already been completed, but rather online, i.e., while the first packets in the flow are passing in real time through the network equipment. However, as the Internet has become a fundamental part of our society, privacy issues have arisen, causing the advent of the encryption of network traffic. Encryption poses yet another challenge to traffic classification, as classical methods such as those mentioned below do not make sense any longer. Several methods have been tested for real-time network packet classification over the years: initially, we saw the rudimentary use of port-based classification [1] (with a basis on TCP and UDP well-known port numbers); later, we saw the use of Deep Packet Inspection (DPI) using pattern matching and detecting protocol anomalies [2]; and, most recently, algorithms based on statistical features, machine learning-based techniques such as Support Vector Machine (SVM) [3], and deep neural networks such as convolutional neural networks (CNNs) [4; 5]. In particular, CNNs are at this moment a well-known solution for traffic classification, because their ability to find spatial correlations between earlier and later parts of the data is useful in network traces, where the data might be correlated spatially to a high degree, due to the relation between contiguous bytes. In fact, CNNs are becoming increasingly relevant, due to their reportedly high accuracy values, but they have a major drawback: they generally behave as black boxes, because the statistical features detected by the neural network are hard to be explained outside the context of the predictor itself. Thus, we decided to study the capabilities of CNNs in depth using a public network traffic dataset, focusing on whether it is feasible to extract usable data from the model. By analyzing state-of-the-art models and their methodologies for training and testing, we were able to identify common practices, keys to success, and possible malpractices, such as data leakage. Compared to previous works, our methodology includes the following main contributions: firstly, we needed to make sure that the separation between packets in the train and validation sets prevented data leakage. That is, we ensured that we did not train and test with different packets in the same flows, in order to prevent overfitting and avoid bias in the model. While this is an obvious practice in AI, it is not as simple as it sounds when conducting network packet processing, because packets in the same connection can cause data leakage if they are incorrectly split. Secondly, we wanted to gain a deeper understanding of what CNNs are actually watching. The lack of explainability or interpretability is a major concern for their potential feasibility and deployment in real-world scenarios. We expect to hold models to the same standards of privacy, unbiasedness, and reliability as other standards in the industry. Once we obtained the right information, we built a reference CNN model for network traffic classification. From the aforementioned dataset, we filtered the headers of the packets in order to manipulate just the encrypted payload data and avoid using IP addresses and ports. After training, we analyzed the performance and the explainability. Regarding explainability, we needed to study how to extract data directly from a deep neural network.