# Memory and Cache Contention Denial-of-Service Attack in Mobile Edge Devices
## 5 Evaluation
### Performance  
Figure 9 shows the results of performance comparison for nine cases. When we compare _spec_ and _spec_+_mal_t1_, the performance of _spec_+_mal_t1_ is degraded by 47.7%, on average, as compared to _spec_. In the case of _xalancbmk_, the performance degradation is up to 78.0%, which means that the malicious memory and cache contention incurs huge performance losses of the normal programs running at the same time. This is because of the shared resource (L2 cache and main memory) contention between the normal and malicious program. In the cases of the other types of the malicious codes, the overall trends are also similar to the results of _spec_+_mal_t1_. The performance degradations of the normal programs due to malcode type-2, type-3, and type-4 are 47.7%, 47.8%, and 47.3%, respectively, as shown in Figure 9.  
On the contrary, if our detection and mitigation mechanisms are employed, the performance of the normal programs is improved, as they restrict the shared resource usage of the malicious program. On average, with our detection and mitigation mechanisms (i.e., in the cases of _spec_+_mal_tX_fl_) performances of the normal programs are improved by 84.9%, 85.4%, 86.1%, and 83.6% as compared to the cases of _spec_+_mal_t1_, _spec_+_mal_t2_, _spec_+_mal_t3_, and _spec_+_mal_t4_, respectively. Table 3 shows the performance recovery rates. The performance recovery rates (\(\mathbf{R}_{\textit{perf\_recovery}}\)) are calculated, as follows:  
\[\mathbf{R}_{\textit{perf\_recovery}}=1-\frac{\mathbf{\text{IPC}}_{\textit{SPEC}}-\mathbf{ \text{IPC}}_{\textit{SPEC}}+\textit{mal\_tX}\_fl}{\mathbf{\text{IPC}}_{\textit{ SPEC}}-\mathbf{\text{IPC}}_{\textit{SPEC}}+\textit{mal\_tX}} \tag{2}\]  
Figure 9: Performance comparison across three broad cases (total nine cases): _spec_, _spec_+_mal_tX_, and _spec_+_mal_tX_fl_. The results are normalized to _spec_ for each workload. where \(\text{IPC}_{\text{SPEC}}\), \(\text{IPC}_{\text{SPEC}+\text{mal\_t}\_tX}\), and \(\text{IPC}_{\text{SPEC}+\text{mal\_t}\_tX\_ft}\) represents IPC in the cases of _spec_, _spec_+_mal_tX_, and _spec_+_mal_tX_ft_, respectively. In the case of _spec_+_mal_t1_ft_, a performance recovery rate is 91.7% on average. One of the main reasons why we cannot obtain 100% performance recovery rate is that there must be a profiling phase to detect the malicious code before the throttling phase. If we would increase the duration of each throttling phase (i.e., more than 500 million cycles), then we could further improve performance recovery rates though it may also lead to performance losses of the normal programs in case of misdetection (i.e., false alarm). For the other types of the malicious codes, our detection and mitigation mechanisms also result in very high \(\text{R}_{\text{perf\_recovery}}\)s (93.1%, 93.4%, and 91.5% in the cases of _spec_+_mal_t2_ft_, _spec_+_mal_t3_ft_, and _spec_+_mal_t4_ft_, respectively). With our detection mechanism, the RCounter pattern that is shown in Figure 6 should be consecutively seen by \(\text{{Thres}}_{\text{acc}}\) (i.e., 500 in this paper) times to be regarded as the malicious code. As a result, during our simulation, we did not see any false alarm case (i.e., falsely detect and actually throttle the normal program) for 13 SPEC2006 programs. When considering that our throttling phase is 500 million cycles while we simulate total one billion instructions, even with only one false alarm and throttling case, a serious performance loss in the SPEC2006 programs will occur. However, we can see a performance recovery rate of 92.4%, on average, which is attributed to zero false alarm case in our simulations, as shown in our recovery rate results. This is because we set the threshold values very conservatively to prevent false alarms. It also implies that our proposed mechanism accurately differentiates normal and malicious programs.