# Indoor Positioning by Double Deep Q-Network in VLC-Based Empty Office Environment
## 2 Related Work  
In this section, we analyze related work that performed positioning based on AI in an indoor VLC environment. AI can be broadly divided into supervised learning, unsupervised learning, and reinforcement learning. Therefore, in this section, we analyze the positioning using each learning method. First, we analyze the related literature that performed supervised learning for positioning. In [16], the authors introduced a method for multiple fingerprint positioning based on artificial neural networks (ANNs). They achieved a positioning performance of 2.68 m while simultaneously positioning 15 targets, demonstrating high accuracy even in environments with radio interference. However, they encountered an issue when a significant error of 2.29 m occurred even with a single target. In [17], the authors propose a method to estimate the location of a moving user in an indoor VLC environment. The authors applied the time division multiple access (TDMA) to eliminate inter-signal interference. The proposed positioning method uses the received power from three transmitters as the input for the ANN and the distance between the transmitter and receiver as the output. Simulation results showed that the neural network estimation approach provides more accurate positioning than the trilateration estimation with 94% accuracy when the receiver's normal has a tilt angle with the transmitter's normal. Second, we analyze the related literature that performed unsupervised-learning-based positioning. In [18], the authors proposed a method that utilizes the weighted k-nearest neighbor (kNN), an unsupervised learning method within the field of AI technologies. Their approach achieved a more precise positioning performance compared to conventional triangulation methods. In [19], the authors combined the unsupervised algorithm kNNand traditional machine learning random forest. In the results, a precise positioning performance of 0.069 m at the center, 0.157 m at the edge, and 0.166 m at the corner was found using their proposed method. In [20], the authors propose the adaptive residual weighted k-nearest neighbor (ARWKNN) technique to solve the k-value optimization problem to obtain the minimum positioning error in WKNN. The proposed method adaptively selects the k-value by matching the residual between the measured and calculated RSSI values. In other words, the k-value corresponding to the smallest residual sum of squares of the RSSI is obtained. Through simulation, the proposed ARWKNN significantly reduced the average position error while maintaining a similar level of complexity compared to the existing WKNN and KNN. Third, we analyze the related literature that performed reinforcement-learning-based positioning. In [21], the authors demonstrated a three-dimensional indoor VLC positioning system using multiple photodiodes and reinforcement learning to enhance positioning accuracy. To achieve this goal, the authors created a VLC positioning system using multiple photodiodes and enhanced performance through the application of a reinforcement learning algorithm. The experimental results demonstrate that VLC using multiple photodiodes can effectively determine location in a three-dimensional space. The results also show improved accuracy and performance through reinforcement learning. In [22], the primary objective of the research is to achieve precise positioning using visible light. To accomplish this, point-wise reinforcement learning is used iteratively. This involves maximizing performance through iterative improvement in the process of learning and applying strategies to enhance positioning accuracy. Simulation results confirm that the proposed IPWRL achieves a lower position error compared to the existing RSS algorithm and PWRL. Therefore, in this work, we try to use a fingerprinting scheme [23], Cell-ID scheme [24], and reinforcement learning [20; 21] to improve the positioning accuracy. The authors of [23] proposed a localization technique using fingerprinting and KNN.