# SAViP: Semantic-Aware Vulnerability Prediction for Binary Programs with Neural Networks
## 3 Vulnerability Prediction Model
### Structural Features
#### 3.4.1 Model Structure  
Next, we will describe the composition of the neural network in detail. Figure 6 shows the overall flow of our improved GNN. \(F\) is the feature matrix, composed of all the basic block feature vectors in ACFG; \(A\) is the adjacency matrix of the ACFG plus a _self-loop_. The increase of the _self-loop_ ensures that each basic block always maintains self-attention while collecting information from its neighbors. Assuming that the ACFG has \(p\) nodes and the \begin{table}
\begin{tabular}{c c c} \hline \hline
**Type** & **Content** & **Num** \\ \hline Instructions & Num of each instruction in Table 2 & 43 \\ \hline \multirow{4}{*}{Oprands} & Num of void operands & \multirow{4}{*}{8} \\  & Num of general registers & \multirow{4}{*}{8} \\  & Num of direct memory references & \multirow{4}{*}{8} \\  & Num of memory references using register contents & \multirow{4}{*}{8} \\  & Num of memory references using register contents with displacement & \\  & Num of immediate operands & \multirow{4}{*}{8} \\  & Num of operands accessing immediate far addresses & \multirow{4}{*}{8} \\  & Num of operands accessing immediate near addresses & \multirow{4}{*}{8} \\ \hline Strings & Num of strings “malloc” & \multirow{4}{*}{5} \\  & Num of strings “callo” & \multirow{4}{*}{5} \\  & Num of strings “free” & \multirow{4}{*}{5} \\  & Num of strings “memcpy” & \multirow{4}{*}{7} \\  & Num of strings “memset” & \\ \hline
**Total** & & **56** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: List of statistical features. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Type** & **Detail** & **Num** \\ \hline Data Transfer & \multirow{2}{*}{mov push pop} & \multirow{2}{*}{3} \\ Instructions & & \\ \hline Binary Arithmetic & \multirow{2}{*}{adcx adox add adc sub sbb imul mul} & \multirow{2}{*}{14} \\ Instructions & & \\ \hline Logical Instructions & \multirow{2}{*}{and or xor not} & \multirow{2}{*}{4} \\ \hline Control Transfer & \multirow{2}{*}{jmp je jz jne jnz ja jnbe jz jnb j jnae jbe} & \multirow{2}{*}{22} \\ Instructions & & \\ \hline Total & & 43 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Instruction List. dimension of the block embedding is \(b*1\), then \(F\) is a \(p*b\) matrix, and A is a \(p*p\) matrix. In this GNN, we establish a matrix \(\varphi\) of dimension \(p*e\) to extract the information of each node, where \(e\) is the dimension of the final graph embedding vector. \(\varphi\) is initialized to a matrix of _zeros_, that is, \(\varphi^{0}\). We update the node information by continuously calculating a new \(\varphi^{t}\) as follows:  
\[\varphi^{t+1}=\xi(FW_{1}+\sigma A\varphi^{0}) \tag{1}\]  
where \(W_{1}\) is a \(b*e\) matrix, and \(\sigma\) is a neural network connected by d-layer fully connected layers. In actual implementation, after each fully connected layer, a nonlinear layer needs to be connected to improve the expressive ability of the model. \(\xi\) performs normalization and rectification (via a ReLU) on the result of the summation. Note that the operations \(\xi\) here cannot be omitted, as they can effectively prevent gradient dispersion and improve the training effect. After \(T\) iterations, we obtain the latest graph feature matrix \(\varphi^{T}\), the dimensions of which are still \(p*e\), and then we can obtain the embedding of the ACFG graph through transformation:  
\[\varphi_{g}=W_{2}[\rho(\varphi^{T})]^{tr} \tag{2}\]  
\(\rho\) is a mean pooling layer, and the dimension of the embedding after pooling becomes \(1*e\). \(W_{2}\) is an \(e*e\) matrix. Therefore, the final dimension of the graph embedding \(\varphi_{g}\) is \(e*1\). Afterward, to obtain the security and vulnerability scores, we transform \(\varphi_{g}\) to obtain a two-dimensional vector \(Z=\{z0,z1\}\)  
\[Z=W_{3}\varphi_{g}^{tr} \tag{3}\]  
where \(W_{3}\) is a \(2*e\) matrix. At this time, there is no constraint between the two values of \(Z\).