# Optimizing Internet of Things Honeypots with Machine Learning: A Review
## 5 Findings
### Detection Architecture
Then, actions (like allow, block, or substitute) are taken on states represented by bash commands, such as wget and sudo, replicating real-world IoT actions. HoneyIoT models its interactions with attackers as an MDP and uses differential content mutation to update response fields, enhancing deception by changing select response fields during runtime. HoneyIoT's reward structure motivates the agent to prolong interactions until it captures malware uploads or vulnerabilities, encouraging interaction patterns that lead to higher attacker engagement [31]. More generally, the honeypot receives rewards for successfully responding to attacks, reinforcing effective responses. Over time, this process converges to an optimal policy, ensuring the honeypot consistently applies the best response to each scenario. Reinforcement learning fosters an environment where honeypots can autonomously evolve, integrating new interaction strategies as they accumulate more attack data. This process enables a faster response to emerging threats, improving the honeypot's ability to handle complex, automated attack sequences. Recently, various adaptative honeypots have emerged. One of them is the AIIPot architecture. AIIPot introduces a sophisticated structure for adaptive, IoT-focused honeypots. The honey-Chatbot is a conversational interface that mimics IoT device responses, increasing realism by interacting with attackers in a seemingly natural way. The request/response database stores expected requests and responses typical to IoT devices. Unknown requests (OOV) are flagged and passed to the evaluator, gradually enriching the honeypot's database. The evaluator module determines if requests are trustworthy, dictating response paths. Trusted requests are broadcast locally, while suspicious ones are redirected, further strengthening the illusion of an authentic IoT environment [49]. One other is the HoneyShell ecosystem. HoneyShell's development highlights adaptivity through phases. Initially, it collects data and refines its responses to hide its honeypot nature. Later, it restricts accepted login credentials based on usage patterns and emulates file commands to align more closely with attacker expectations. These adaptations allow for more realistic interactions and better data collection on attacker behavior. The honeypot's ecosystem consists of three main components: server farms (both on-premises and cloud-based) hosting honeypot instances to engage with attackers and collect data; a vetting system to obscure the honeypot's true nature from attackers; and an analytics infrastructure to analyze data for insights into attacker behavior. Notably, \"HoneyShell\" was developed as a honeypot version of shell services, addressing a significant portion of the observed attack surface [61]. Figure 2: Feedback connection of automation and environment from the literature [60]. HoneyIoT specifically employs reinforcement learning to adaptively interact with attackers. Interactions are framed as a Markov Decision Process (MDP), and the agent selects optimal responses to keep attackers engaged and increase the likelihood of observing a full attack sequence. Differential content mutation further adapts response fields at runtime to better align with attacker expectations [35]. Testing the effectiveness of adaptive honeypots is crucial. To select the most effective algorithm for HoneyIoT, the collected attack data were modeled into an MDP (Markov Decision Process) framework and tested using OpenAI Gym and Stable Baselines3, as [PERSON] [35] shows, with a comparison of different learning algorithms in the adopted graph. For comparison, a baseline approach without reinforcement learning was implemented, where responses were chosen randomly with equal probability. Rewards were assigned based on attacker actions: positive rewards (+5 for malware uploads, +1 for vulnerability exploitation) and negative rewards (\(-1\) for other actions).