# Malware Classification Based on Shallow Neural Network
## 3 Classification Methodology
### The Shallow Neutral Network
#### 3.4.3 Output Layer  
Our model uses a hierarchical softmax layer to complete the final step in the family classification. Softmax is the most commonly used output function in multi-classification tasks. In a standard softmax regression, for each input, the probability of belonging to each class is calculated separately, and then the class with the highest probability is chosen as the output class label. As a result, when there are many class labels or the hidden layer output vector dimension is large, the calculation will be massive. Hierarchical softmax, based on the Huffman tree, transforms the multi-classification problem into multiple bi-classification problems. In the case where the family class number is k and the hidden  
Figure 4: The hidden layer projects each \(m\times N\) matrix into a \(1\times N\) vector.  
layer output file feature vector dimension is d, the hierarchical softmax can reduce the computational complexity from O(\(kd\)) when using the standard softmax to O(\(d\log_{2}k\)) [32].  
As shown in Figure 5, in the Huffman tree constructed by the output layer, the leaf nodes denote malware family labels, and the other nodes are called hidden nodes. When constructing a tree, the weight of each leaf node is the number of family samples in the training set, and the weight of the non-leaf node is the sum of the weights of all its child nodes. There is only one path from the root node to each leaf node that denotes a malware family. The output layer calculates the probability that a sample belongs to a certain family, according to this unique path. The probability of going to the right or left on every passing node will be multiplied. The formulas are shown below.  
The probability of going left at node \(n\), \(P(n,left)\), is calculated as Equation (3) shows:  
\[P(n,left)=\sigma(\theta_{n}^{T}\cdot h) \tag{3}\]  
According to the nature of the activation function sigmoid, we can get the probability of going right at node n as in Equation (4):  
\[P(n,right)=1-\sigma(\theta_{n}^{T}\cdot h)=\sigma(-\theta_{n}^{T}\cdot h) \tag{4}\]  
Then, if we want to calculate the probability that the sample file expressed by the feature vector \(h\) belongs to the malware family \(c_{1}\) in Figure 3, the calculation is as follows:  
\[\begin{split} P(c_{1}|h)&=P(root,left)\cdot P(n(c_ {1},1),left)\cdot P(n(c_{1},2),right)\\ &=\sigma(\theta_{root}^{T}\cdot h)\cdot\sigma(\theta_{n(c_{1},1 )}^{T}\cdot h)\cdot\sigma(-\theta_{n(c_{1},2)}^{T}\cdot h)\end{split} \tag{5}\]  
Extending to the general case, the probability of a malware sample expressed by vector \(h\) belonging to family \(c\) is calculated by Equation (6):  
\[P(clh)=\prod_{j=1}^{L(c)-1}\sigma(sign(c,j)\cdot\theta_{n(c,j)}^{T}h) \tag{6}\]  
where \(L(c)\) is the unique path length from the root node to the leaf node of the class \(c\), \(\theta_{n(c,j)}\) is the coefficient vector of the hidden node, whose transpose is \(\theta_{n(c,j)}^{T}\) and \(\sigma\) is the activation function sigmoid.  
Each hidden node vector is randomly initialized and updated with training feedback. Since the probability of the child node must be less than its parent node, when classifying a sample, we searched  
Figure 5: Hierarchical tree structure of the output layer.  
down along the branches having higher probabilities until the leaf node was reached, and then the corresponding label was obtained.