# A Real-Time Network Traffic Classifier for Online Applications Using Machine Learning
## 3 System Design
### ML Models  
#### 3.3.1 ANN Structure  
We trained a feed-forward ANN model with 2 hidden layers, one input layer and one output layer. \(I=[i_{1},i_{2},\ldots,i_{r}]\) and \(O=[o_{1},o_{2},\ldots,o_{h}]\) represent the input and output vectors, respectively, where \(r\) represents the number of elements in the input feature set and \(h\) is the number of classes. The main objective of the network is to learn a compressed representation of the dataset. In other words, it tries to approximately learns the identity function \(F\), which is defined as:  
\[F_{W,B}(I)\simeq I \tag{2}\]  
where \(W\) and \(B\) are the whole network weights and biases vectors.  
A log sigmoid function is selected as the activation function \(f\) in the hidden and output neurons. The log sigmoid function \(s\) is a special case of the logistic function in the \(t\) space, which is defined by the following formula:  
\[s(t)=\frac{1}{1+e^{-t}} \tag{3}\]  
Figure 2: The number of samples in our dataset after the preprocessing stage.  
The weights of the ANN network create the decision boundaries in the feature space, and the resulting discriminating surfaces can classify complex boundaries. During the training process, these weights are adapted for each new training image. In general, feeding the ANN model with more samples can recognize the online applications more accurately. We used the back-propagation algorithm, which has a linear time computational complexity, for training the ANN model.  
The input value \(\Theta\) going into a node \(i\) in the network is calculated by the weighted sum of outputs from all nodes connected to it, as follows:  
\[\Theta_{i}=\sum(\omega_{i,j}*\mathrm{Y}_{j})+\mu_{i} \tag{4}\]  
where \(\omega_{i,j}\) is the weight on the connections between neuron \(j\) to \(i\); \(\mathrm{Y}_{j}\) is the output value of neuron \(j\); \(\mu_{i}\) is a threshold value for neuron \(i\), which represents a baseline input to neuron \(i\) in the absence of any other inputs. If the value of \(\omega_{i,j}\) is negative, it is tagged as inhibitory value and excluded because it decreases net input.  
The training algorithm involves two phases: forward and backward phases. During the forward phase, the network's weights are kept fixed, and the input data is propagated through the network layer by layer. The forward phase is concluded when the error signal \(e_{i}\) computations converge as follows:  
\[e_{i}=(d_{i}-o_{i}) \tag{5}\]  
where \(d_{i}\) and \(o_{i}\) are the desired (target) and actual outputs of \(i\)th training image, respectively.  
In the backward phase, the error signal \(e_{i}\) is propagated through the network in the backward direction. During this phase, error adjustments are applied to the ANN network's weights for minimizing \(e_{i}\).  
We used the gradient descent first-order iterative optimization algorithm to calculate the change of each neuron weight \(\Delta\omega_{i,j}\), which is defined as follows:  
\[\Delta\omega_{i,j}=-\eta\frac{\delta\varepsilon(n)}{\delta\varepsilon_{j}(n)}y _{i}(n) \tag{6}\]  
where \(y_{i}(n)\) is the intermediate output of the previous neuron \(n\), \(\eta\) is the learning rate, and \(\varepsilon(n)\) is the error signal in the entire output. \(\varepsilon(n)\) is calculated as follows:  
\[\varepsilon(n)=\frac{1}{2}\sum_{j}e_{j}^{2}(n) \tag{7}\]  
As shown in Figure 3, the ANN model generated more than 81 k parameters during the training phase. The Adam optimization algorithm is used to update the network weights iteratively based on training data. We used the categorical cross-entropy as a loss function, \(\Gamma\), which is defined as follows:  
\[\Gamma(W,B)=\|I-F_{W,B}(I)\|^{2} \tag{8}\]