# Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis
## 3 Related Work  
In [12], [PERSON] et al. contribute to the detection of rootkits by integrating dynamic DL-based methods and heuristic approaches within a malware detection framework. By analyzing the behavior patterns and employing advanced detection techniques, their model achieved effective identification and classification of rootkits, as well as enhanced the overall capabilities of the malware detection system. In [13] [PERSON] et al. conducted a study on the effectiveness of extracting memory-based images to detect malware. They created a binary memory-based dataset available on GitHub [14] and employed classification algorithms such as SVM, RF, k-NN, Naive Bayes (NB), and DT. Their model achieved an accuracy of 98.5% using the Volatility v2.6 framework. [PERSON] et al. [15] utilized computer vision and ML techniques to detect and classify malware by analyzing memory dumps as RGB images. Their approach improved the detection of unknown malware by up to 20.78% across multiple ML algorithms. They employed RF, SVM, and XGBoost algorithms and demonstrated the practicality of computer vision-based schemes for protection against malicious applications. Another study by [16] emphasized the significance of memory analysis in capturing malware footprints and extracting hidden code from obfuscated malware. The authors developed a Python-based plugin called VolMemLyzer for Volatility v2.6 that is capable of extracting 36 features and converting the results into CSV format. The plugin showed high accuracy in malware classification, and a dataset of approximately 1,900 instances was created using the tool. Addressing the challenges of detecting obfuscated and hidden malware, [PERSON] et al. [17] transformed the VolMemLyzer framework to extract 26 new memory features, enhancing its efficiency [17]. The plugin was employed to detect ransomware, Trojan Horses, and spyware, and achieved an accuracy of 99% and an F1-Score of 99.02%. An extended dataset was created, contributing positively to research in this field. In [18], the Trusted Kernel Rootkit Detection (TKRD) system combined memory forensic analysis with bio-inspired ML techniques to detect kernel rootkits. It achieved very high accuracy. In [19], a hardware-assisted Virtualization-based Kernel-level Rootkit Detection (VXRD) system was introduced. It employed ML techniques and dynamic analysis to intercept and isolate the operations of kernel modules, albeit with performance overhead. [PERSON] et al. [20] addressed the challenge of detecting rootkits in embedded IoT devices by utilizing dynamic analysis within Trusted Execution Environments (TEE) available in popular IoT platforms. These studies highlight the significance of memory analysis and various detection techniques, including ML, computer vision, and dynamic analysis, in combating the threat of malware and rootkits. Collectively, these studies reveal that a multi-dimensional approach, combining memory analysis and a range of detection methodologies, holds great promise for addressing the evolving landscape of cyber threats. Table 1 presents a summary of the research discussed in this section. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Study** & **Learning Algorithm** & **Performance Metrics** & **Dataset** & **Strengths** & **Limitations** \\ \hline \multirow{4}{*}{[PERSON] et al. [12]} & DNN, CNN, RF, & Accuracy, precision, recall, F1-score & CICAndMal2017 & Utilizes dynamic deep learning and heuristic. & Lack of in-depth analysis on potential false positives. Limited exploration of feature engineering. \\  & & & & features extracted & \\ \cline{1-1}  & & & & from memory & \\ \cline{1-1}  & & & & images. & Potential \\ \cline{1-1}  & & & & over-fitting due to \\ \cline{1-1}  & & & & feature & high accuracy rate \\ \cline{1-1}  & & & engineering and & on training data. \\ \cline{1-1}  & & & & binary vectors for & \\ \cline{1-1}  & & & & training and & \\ \cline{1-1}  & & & & testing. & \\ \hline \hline \end{tabular}
\end{table}