## 6 Experimental Evaluation
### Experimental Results for The Malware Detector  
**Evaluating the communication between the TApp and the MDetector.** We have tested four different cases to evaluate the communication between TApp and MDetector: (1) no extra delay is added; (2) 2-s delay is added; (3) 3-s delay is added; (4) 5-s delay is added. We have collected 20 data points and the results are shown in Figure 9. We have the following observations: (1) Without additional delay, 5 s are needed to execute a command with normal return. The overhead mainly includes: encoding a secret command into the -stepDATA (in the TrustZone secure world), writing the stepDATA to a flash page (in the normal world), extracting the secret command from the stepDATA (in the FTL), and reading the stepDATA from the flash page (in the normal world). A major time consumption comes from the Raspberry Pi since _it significantly slows down when reaching 75 \({}^{\circ}\)C_. (2) With the additional delay, TApp can successfully differentiate the responses from the normal return. This justifies the effectiveness of mobiDOM in conveying a response from the MDetector back to the TApp stealthily by adding extra delays.  
**Evaluating the effectiveness of malware detection in the FTL.** To understand the effectiveness of detecting malware in the FTL, we have collected 96 malware samples (mainly from VirusTotal [28]) and 36 benign software samples (including compression/encryption/deletion software, etc., which will cause I/Os to the external storage). For each sample, we manually ran it in a computer, which was connected to the LPC-H3131 via the USB, and collected the I/O trace records in the FTL into a trace file. Note that after running each malware sample, we need to restore the entire system to the initial clean state. Each I/O trace record consists of a flag (e.g., 0 indicates a read operation while 1 indicates a write operation), starting logical page address for this I/O, length of consecutive pages being read/written in this I/O. A snapshot of the trace records is shown in Figure 10, and each record is formatted as \(<flag,startingaddress,length>\). All the trace files--each stores the I/O trace records captured for each software/malware sample in our experiment--are publicly available in [29]. We used kNN to process the I/O traces. Our training set includes I/O traces of 80 malware samples and 30 benign software samples, and our test set includes I/O traces of 16 malware samples and 6 benign software samples. For each software/malware sample, we selected the first 30 I/O trace records from the corresponding trace file, and used this sequence of 30 I/Os as the feature of each sample. The dynamic time warping algorithm was used to calculate the distance between the two sequences (Section 4.1.2).  
To detect the malware in real-time, the detection should be as fast as possible. A small \(k\) with a good accuracy could achieve a good tradeoff for our application scenario. When choosing \(k\) as 1, we can achieve some good accuracy and the corresponding detection results, as shown in Table 1. The detection accuracy is 91%. This confirms the effectiveness of our FTL-based malware detector. In addition, the false positive rate and the false negative rate are 33% and 0%, respectively. The low false negative rate indicates that our malware detector does not miss any malware if present. The false positive rate seems a little high. The reason is that, we have deliberately chosen those benign software samples which  
Figure 9: Time for executing a command (START, QUERY, STOP) with/ without adding delays. The CPU temperature is around 75 \({}^{\circ}\)C.  
exhibit some similar patterns to the collected malware in the experiments. However, in practice, most of the benign software may not exhibit such similar patterns.