# Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis
## 1 Introduction  
A rootkit is a malicious program with a highly deceptive nature. It operates by concealing its presence in the system and enabling unauthorized root access to attackers, allowing then to gain complete control over the compromised system. Rootkits are designed to evade detection and this makes it challenging for anti-malware tools to detect their infiltration. Once installed, a system may exhibit unusual behaviour, indicating remote access by an attacker. Rootkits are particularly hazardous and can lead to significant data loss and damage.  
The term \"rootkit\" is derived from the combination of two words, \"root\" and \"kit\". In UNIX and Linux environments, \"root\" refers to the system administrator who possesses the highest level of access. \"Kit\" denotes a collection of tools and techniques. Consequently, a rootkit is defined as a set of tools or techniques enabling unauthorized individuals to gain and sustain administrator-level access during an attack while evading detection by authorized users and administrators [1].  
Rootkits have been a significant concern for security engineers since the late 1980s. Initially, they were primarily used to hide log files and application binaries. The early generation of these malicious software targeted user-level programs, making them relatively easy to detect using simple checksum methods. However, the threat landscape has evolved, and modern rootkits pose more and more danger, as operating systems are now widely employed in various devices such as smartphones, IoT nodes, computers,and embedded devices [2; 3]. Consequently, researchers are actively working on developing efficient methods to detect these sophisticated rootkits that can evade traditional detection techniques.  
The frequency of rootkit attacks is also on the rise. According to reports from Avast, a leading cybersecurity company, the number of users affected by rootkits increased from 10,000 in 2020 to 100,000 in 2021. Therefore, the main objective of this research is to devise a reliable and efficient detection technique specifically tailored for rootkits that are challenging to detect.  
Rootkits are designed to conceal themselves from system administrators and users, allowing them to operate covertly and pose significant security threats. They pose a threat not only to the host systems, but also to the virtual machines [4]. Therefore, there is a crucial requirement for a standardized mechanism that can efficiently detect rootkits. This research aims to investigate the effectiveness of memory analysis techniques for rootkit detection. Additionally, we aim to develop an automated tool utilizing Machine Learning (ML) and Deep Learning (DL) for the quick and accurate identification of rootkits within a system.  
To overcome the limitations of previous research, this paper sets out to achieve the following objectives:  
* Introduce an effective technique that leverages memory analysis to detect concealed rootkits;
* Develop models and tools to automate the analysis of memory dumps for efficient rootkit detection;
* Investigate and explore a novel set of features extracted from memory images, including DLLs, handles, privileges, network connections, modules, injections, and services.  
By fulfilling these objectives, this research aims to contribute to the advancement of rootkit detection methods and overcome the existing limitations in the field.  
The rest of the paper is organized as follows: Section 2 provides a background on basics of ML and DL. Section 3 presents the related studies on malware and rootkit detection. The proposed rootkit detection approach and the memory-based dataset are described in Section 4. Section 5 explains the experimental results and discuss them and compares them with the existing literature. Finally, the conclusion and future works are presented in Section 6.