# Automated Vulnerability Exploitation Using Deep Reinforcement Learning
## 6 Conclusions and Future Work  
In this paper, we utilized RL to perform vulnerability exploitation. We built two RL agents: the first one uses Q-Table and the second one uses DRL. The state of the RL agent is identified by the OS and vulnerability under investigation. The actions of the RL agent are identified by the Metasploit payloads that can be used to exploit the given vulnerabilities. An exploitation is considered successful if the payload was able to create a reverse shell with the attacker. Both RL agents (i.e., the one using Q-Table and the one using DRL) were trained and tested on two use cases. The first use case has one vulnerability, while the second use case has four vulnerabilities. Our results demonstrate that using DRL outperformed Q-Table in the two use cases. In the first use case, DRL had a maximum success rate of 96.6%, while Q-Table had 88.4%. For the second use case, DRL had a maximum success rate of 73.6%, while Q-Table had 71.2%. The DRL agent's higher success rate stems from its ability to continuously learn and adapt during deployment, unlike the static Q-Table approach. This research introduces a new way to automate tasks during pentesting, making them faster and less expensive. Our method uses an AI agent trained with RL to take advantage of the Metasploit framework. By automating the exploitation process, our approach frees up security specialists to focus on other important tasks. This can significantly reduce the time and resources needed to identify and address vulnerabilities in computer systems. For future work, we plan to build a general agent that can exploit new vulnerabilities on which it was not trained. To achieve this, we are considering adding more information to the agent's decision-making process like the Common Weakness Enumeration (CWE). This information could be used to make better decisions and exploit new scenarios and vulnerabilities. Figure 8: Rewards’ moving average for GV using DRL in the training phase for 5 scenarios where the values of \(\epsilon,\alpha,\gamma\), rate of decrease, and success rate are shown, respectively, under each figure. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Trial** & **Epsilon** & **Alpha** & **Gamma** & **Decay Rate** & **SR \%** \\ \hline
1 & \(7\times 10^{-1}\) & \(1\times 10^{-1}\) & \(8\times 10^{-1}\) & \(1\times 10^{-2}\) & 72.1 \\
2 & \(1\times 10^{-1}\) & \(3\times 10^{-1}\) & \(4\times 10^{-1}\) & \(1\times 10^{-2}\) & 73.6 \\
3 & \(6\times 10^{-1}\) & \(3\times 10^{-1}\)  
**Author Contributions:** Conceptualization, A.A., L.A.-A., Z.S., K.M.A.Y. and B.J.M. ; methodology, A.-A., L.A.-A., Z.S., K.M.A.Y. and B.J.M. ; software, L.A.-A., Z.S. and A.A.S. ; validation, L.A.-A., Z.S. and A.A.S. ; formal analysis, A.A., L.A.-A., K.M.A.Y., B.J.M. and Z.S. ; investigation, A.A., L.A.-A., K.M.A.Y., B.J.M. and Z.S. ; data curation, A.A., L.A.-A., K.M.A.Y., B.J.M. and Z.S. ; writing--original draft preparation, A.A., L.A.-A., K.M.A.Y., B.J.M. and Z.S. ; writing--review and editing, A.A., K.M.A.Y. and B.J.M. ; visualization, A.A., L.A.-A., K.M.A.Y. and B.J.M. ; supervision, A.A.; project administration, A.A.; funding acquisition, A.A. All authors have read and agreed to the published version of the manuscript. The work in this paper was supported, in part, by the Open Access Program from the American University of Sharjah. This paper represents the opinions of the author(s) and does not mean to represent the position or opinions of the American University of Sharjah. Not applicable. The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author. The authors declare no conflicts of interest. The following abbreviations are used in this manuscript: