# A Real-Time Network Traffic Classifier for Online Applications Using Machine Learning
## 3 System Design
### Dataset
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Class \#** & **Application Name** & **Number of Samples** \\ \hline
1 & GOOGLE & 959,110 \\ \hline
2 & HTTP & 683,734 \\ \hline
3 & HTTP\_PROXY & 623,210 \\ \hline
4 & SSL & 404,883 \\ \hline
5 & HTTP\_CONNECT & 317,526 \\ \hline
6 & YOUTUBE & 170,781 \\ \hline
7 & AMAZON & 86,875 \\ \hline
8 & MICROSOFT & 54,710 \\ \hline
9 & GMAIL & 40,260 \\ \hline
10 & WINDOWS\_UPDATE & 34,471 \\ \hline
11 & SKYPE & 30,657 \\ \hline
12 & FACEBOOK & 29,033 \\ \hline
13 & DROPBOX & 25,102 \\ \hline
14 & YAHOO & 21,268 \\ \hline
15 & TWITTER & 18,259 \\ \hline
16 & CLOUDFLARE & 14,737 \\ \hline
17 & MSN & 14,478 \\ \hline
18 & CONTENT\_FLASH & 8589 \\ \hline
19 & APPLE & 7615 \\ \hline
20 & OFFICE\_365 & 5941 \\ \hline
21 & WHATSAPP & 4593 \\ \hline
22 & INSTAGRAM & 2415 \\ \hline
23 & WIKIPEDIA & 2025 \\ \hline
24 & MS\_ONE\_DRIVE & 1748 \\ \hline
25 & DNS & 1695 \\ \hline
26 & IP\_ICMP & 1631 \\ \hline
27 & NETFLIX & 1560 \\ \hline
28 & APPLE\_ITUNES & 1287 \\ \hline
29 & SPOTIFY & 1269 \\ \hline
30 & APPLE\_ICLOUD & 1200 \\ \hline
31 & EBAY & 1192 \\ \hline
32 & SSL\_NO\_CERT & 856 \\ \hline
33 & GOOCLE\_MAPS & 807 \\ \hline
34 & EASYTAXI & 705 \\ \hline
35 & TEAMVIEWER & 527 \\ \hline
36 & HTTP\_DOWNLOAD & 516 \\ \hline
37 & MQTT & 302 \\ \hline
38 & TOR & 276 \\ \hline
39 & FTP\_DATA & 251 \\ \hline
40 & UBUNTUONE & 249 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The number of samples used in the training phase across the online application classes. We also altered the Ethernet header for some packets in our dataset because the transport layer segments in the TCP and UDP protocols vary in header length. For instance, the TCP and UDP packets' header lengths are 20 and 8 bytes, respectively. Therefore, we inserted zeros to the end of the UDP protocol's packet headers to equalize all packets' length in our dataset. These preprocessing and feature extraction steps are summarized in Algorithm 1. ```
1:procedurePreprocessTrainingDataset
2:Input:RawDataset (\(RD\))
3:Output1:PreprocessedDataset (\(PD\))
4:Output2:FeatureAttributes (\(FT\))
5:/* generate the PCAP files */
6:PCAP-List = generatePCAP(\(RD\));
7:/* purify the PCAP files */
8:for each PCAP file in PCAP-List do
9:/* remove noise and incomplete traffic packets */
10:CSV-File = purifyDataset(file);
11:/* Standardize the length of packets' headers */
12:packetList = alterPacketHeader(CSV-File);
13:\(PD\).add(packetList);
14:/* extract the feature vector from each packet */
15:\(v=\)getFeatureVector(packetList);
16:\(FT\).add(\(v\));
17:endfor
18:return\(PD\) and\(FT\);
```  
**Algorithm 1** Preprocessing and Feature Extraction Algorithm  
The preprocessing operation was customized for each ML model based on its hyperparameters and structure. Furthermore, the size of the training set and the number of feature attributes were reduced from 16,545,768 to 3,577,296 packet flows and from 86 to 78, respectively. We eliminated the following feature attributes from the dataset because they could be derived from other exciting attributes: URG Flag Count, CWE Flag Count, Flow Bytes S, Subflow Fwd Bytes, Subflow Bwd Bytes, Bytes Bulk, Bwd Avg Bytes Bulk, Init Win bytes forward, Init Win bytes backward. Figure 2 gives a general overview of the number of samples in our dataset across the \begin{table}
\begin{tabular}{c c c} \hline \hline
**Class \#** & **Application Name** & **Number of Samples** \\ \hline
41 & NTP & 135 \\ \hline
42 & SSH & 102 \\ \hline
43 & EDONKEY & 95 \\ \hline
44 & WAZE & 79 \\ \hline
45 & DEEZER & 74 \\ \hline
46 & UNENCRYPED\_JABBER & 45 \\ \hline
47 & CITRIX\_ONLINE & 38 \\ \hline
48 & TIMMEU & 34 \\ \hline
49 & UPNP & 34 \\ \hline
50 & TELEGRAM & 33 \\ \hline
51 & FTP\_CONTROL & 25 \\ \hline
52 & TWITCH & 24 \\ \hline
53 & H323 & 21 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: _Cont._We had to normalize the range of values of the feature attributes in the dataset before training the ML models.