###### Abstract  
A large number of IoT devices access the Internet. While enriching our lives, IoT devices bring potential security risks. Device identification is one effective way to mitigate security risks and manage IoT assets. Typical identification algorithms generally separate data capture and target identification into two parts. As a result, it is inefficient and coarse-grained to evaluate the results only once the identification process is complete and then adjust the data capture strategy afterward. To solve this problem, we propose a fine-grained probe-scheduling approach based on information feedback. First, we model the probe surface as three layers for IoT devices and define their relationships. Then, we improve the policy gradient algorithm to optimize the probe policy and generate the optimal probe sequence for the target device. We implement a prototype system and evaluate it on 53,000 IoT devices across various categories to show its wide applicability. The results indicate that our approach can achieve success rates of 96.89%, 93.43%, and 83.71% for device brand, model, and firmware version, respectively, and reduce the identification time by 55.96%. Internet of Things; fine-grained fingerprints; device identification; reinforcement learning +
Footnote †: journal: Applied Sciences  
time (typically microseconds). In the standard reinforcement learning (RL) setting, the agent receives feedback from the environment at every step and chooses an action based on that feedback [8]. Interactions between the agent and the environment allow determining whether the information received is sufficient to complete the identification task and adjust the data collection policy accordingly. In this way, data capture and target identification can be performed simultaneously, avoiding sending additional probe requests. Therefore, a probe-scheduling approach with information feedback can balance the success rate and the communication overhead of device identification. In this paper, we propose a fine-grained probe-scheduling approach based on information feedback to identify large-scale IoT devices. We aim to improve identification success rates and efficiency for large-scale IoT devices. First, we model the probe surface as three layers for IoT devices and define their relationships. Then, we improve the policy gradient (PG) algorithm to optimize the probe policy and generate the optimal probe sequence for the target device. We implement a prototype system and evaluate it through real-world experiments to validate our approach. We use the Shodan API [3] to collect response data (open ports, protocol response data, and web feature information) from 53,000 real IoT devices. The dataset covers a wide range of device categories. Thus, our approach has wide applicability, i.e., different types of IoT devices can achieve high identification efficiency and success rates using our approach. Overall, our contributions are summarized as follows:  
* We model the probe surface as three layers by analyzing the characteristics of IoT devices and define their sequential relationships. * We propose a fine-grained probe-scheduling approach based on information feedback to achieve high identification efficiency and success rates. Using the improved RL algorithm, we update the identification state dynamically and select the next action with the greatest benefit. * We implement a prototype system and evaluate it on 53,000 IoT devices across various categories. The results show that our approach can achieves success rates of 96.89%, 93.43%, and 83.71% for device brand, model, and firmware version, respectively. Furthermore, our approach reduces the identification time by 55.96% compared with that of the protocol-popularity method.