# Malware Collusion Attack against SVM: Issues and Countermeasures
## 5 Evaluation
### Q1&Q3: Effectiveness and Performance of ColluDroid
#### 5.1.2 Estimate the Performance of ColluDroid against Collusion and Evasion Attack  
In the previous experiment, we have demonstrated that our ColluDroid is effective in detection collusion apps. However, as we have mentioned in Section 3, malware developers may take both Collusion Attack and Evasion Attack to evade detection. In this subsection, we perform two other experiments to evaluate the ability of ColluDroid against these attacks.  
Experimental SetupThe dataset in the following two experiments is the same as the previous one in Section 5.1.1. The parameters of the Linear SVM and Sec-SVM are also the same. In our second experiment, we plan on evaluating the effect of Collusion Attack on these four methods. Collusion attack is performed according to Section 3.3.2, which simulate the collusion by splitting the malicious app into two apps. In our third experiment, we try to evaluate the performance of these four methods when collusion attack and Evasion Attack coexist. At first, we split the malware apps according to Section 3.3.2 and then perform evasion attacks according to Section 3.3.3 to simulate the simultaneous presence of Collusion and Evasion Attacks. Both experiments are tested in the case of Perfect Knowledge (PK) and Limited Knowledge (LK). PK assumes that attackers have all the training set and parameters; thus, they have the final discriminant function \(f(x)\). In the LK case, attackers  
\begin{table}
\begin{tabular}{l l l l l} \hline \hline
**App Name** & **Malicious Payload** & **Payload of Split App1** & **Payload of Split App2** \\ \hline \multirow{3}{*}{\(App_{A}\)} & Short Message & \multirow{3}{*}{SMS \(\rightarrow\) Intent} & \multirow{3}{*}{Intent \(\rightarrow\) Remote Server} \\  & Service(SMS)\(\rightarrow\) & & \\  & Remote Server & & \\ \cline{1-1}  & Read SMS \(\rightarrow\) Send & \multirow{2}{*}{SMS \(\rightarrow\) Intent} & \multirow{2}{*}{Intent \(\rightarrow\) Send SMS} \\ \cline{1-1}  & SMS & & \\ \cline{1-1}  & Remote Command \(\rightarrow\) & Remote Command \(\rightarrow\) SMS & \multirow{2}{*}{Intent \(\rightarrow\) Send SMS} \\ \cline{1-1}  & SMS \(\rightarrow\) Send SMS & \(\rightarrow\) Intent & \\ \cline{1-1}  & Location \(\rightarrow\) Remote & Location \(\rightarrow\) & \\ \cline{1-1}  & Server & Preference & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Manually made apps in our experiment.  
just know limited training data, they construct evasion points by attacking a surrogate classifier \(\hat{f}(x)\), and these points are then used to attack the true classifier \(f(x)\) and evaluate the performance. In our experiment, we learn the surrogate classifier using a smaller training set, consisting of only 20% of all the samples.  
_Experimental Results._ The results of Collusion Attack against these four methods are shown in the second column of Table 2, from which we can figure out that collusion attack is quite effective against Linear SVM. Collusion attack against Linear-SVM with PK has already been shown in Figure 2. Here, we show the result of the Collusion Attack against Sec-SVM with PK in Figure 5. Nearly 90% of malware samples can evade detection by just splitting into two apps. However, with our ColluDroid framework, the evasion rate will drop significantly to less than 10%. When attackers perform both collusion and evasion attack, the situation will become more serious. The average evasion rates of all scenarios are shown in the third column of Table 2. In addition, for each \(c\in\{5,10,20,40,60,80,100,150,200\}\), the evasion rate is shown in Figure 6, from which we can figure out that the evasion rate increases dramatically as the evasion cost \(c\) increases. When \(c=100\), almost all the samples can evade the detection of three other methods except ColluDroid-Sec-SVM. Thus, we can conclude that ColluDroid-Sec-SVM outperforms other methods in both PK and LK attack scenarios.