# Indoor Positioning by Double Deep Q-Network in VLC-Based Empty Office Environment
###### Abstract  
Recently, artificial intelligence (AI) has been applied in various industries. One such application is indoor user positioning using Big Data. The traditional method for positioning is the global positioning system (GPS). However, the performance of GPS is limited indoors due to propagation loss. Hence, radio frequency (RF)-based communication methods such as WiFi and Bluetooth have been proposed as indoor positioning solutions. However, positioning performance inaccuracies arise due to signal interference caused by RF band saturation. Therefore, this study proposes indoor user positioning based on visible light communication (VLC). The proposed method involves the sequential application of fingerprinting and double deep Q-Network. Fingerprinting is utilized to define the action and state of the double deep Q-Network agent. The agent is designed to learn and locate the reference point (RP) closest to the user's position in a shorter search time. The core idea of the proposed system is to converge a Cell-ID scheme and fingerprinting. Through this, the initial state of the double deep Q-Network agent can be limited. A limited initial state can increase the positioning speed. Simulation results show that the proposed scheme attains a positioning resolution of less than 13 cm and achieves a processing time of less than 0.03 s to obtain the final position in VLC-based office environments.  
indoor positioning; artificial intelligence (AI); visible light communication (VLC); received signal strength (RSS); deep reinforcement learning (DRL) +
Footnote †: journal: Medical  
Existing indoor positioning algorithms include time of arrival (TOA), angle of arrival (AOA), time difference of arrival (TDOA), and received signal strength (RSS) [12]. Among these, the RSS-based positioning algorithm is widely adopted due to its simplicity of implementation and lack of additional hardware requirements. Although RSS-based positioning has relatively low accuracy, which is attributed to the issue of signal variability, the accuracy can be improved when combined with the fingerprinting technique [13].  
Recently, there has been active research on applying AI technology to positioning methods based on RSS data [14]. AI technology enables real-time positioning due to its fast processing speed and can achieve a high-precision positioning performance when provided with sufficient training data [15].  
In this study, we propose a technique for user positioning in an indoor VLC environment using a double deep Q-Network. Our focus is on achieving a precise positioning performance while minimizing processing time. To achieve this, we construct a fingerprinting database and train a double deep Q-Network model. Traditional fingerprinting database-based matching techniques suffer from increased processing time as the fingerprint database grows larger. However, our proposed double deep Q-Network model significantly reduces processing time while improving positioning accuracy. Furthermore, in this study, we take into account the impact of multipath reflections on the positioning performance. The main contributions of this paper are described as follows:  
1. The core technology of this study is the application of the double deep Q-Network to significantly improve the processing speed of the fingerprinting technique.
2. The Cell-ID technique can be used to speed up convergence by selecting the initial location of the reinforcement learning agent.
3. Through simulation results, it can be seen that a fast processing time is achieved when the initial location of the agent is selected.  
The paper is structured as follows. Section 2 analyzes related work for performing indoor positioning. In Section 3, the system model used in this study is described. Section 4 presents the proposed indoor positioning method. In Section 5, the simulation parameters and results are presented. In Section 6, the conclusion is summarized, and future work directions are explained in view of improving indoor positioning.