# Application of Deep Convolutional Neural Networks and Smartphone Sensors for Indoor Localization
## 2 Related Work  
A large body of work has been presented which uses deep learning with using Wi-Fi, magnetic field, video, and scene recognition to perform indoor localization [1; 39; 40]. However, we will limit our discussion to only the most recent and relevant contributions which use deep learning and scene recognition for localization. Authors in [41] utilize smartphone camera to locate a person in narrow corridors. The proposed technique first builds the image database and then matches the user taken images to the database. Speeded-up Robust Features (SURF) are used for the matching process. Later, the actual location is calculated with the help of epipolar geometry. The localization results are compared with wireless local area localization results to evaluate the accuracy of the proposed method. While the proposed method achieves slightly better accuracy than that of wireless local area results, it lacks robustness. Time involved in feature extraction from captured images and their matching increases the latency. Additionally, the localization accuracy can be improved by a fusion of multiple localization technologies. For example, authors in [42] present an approach called Wi-Vi to improve the localization accuracy. The proposed approach is based on the use of Wi-Fi and visual images fingerprinting. First step involves a coarse location estimation using the Wi-Fi fingerprint database. During the second step, the calculated location is refined with the help of captured images. The images are matched using ORB which is the combination of oFAST (FAST with orientation), and rBRIEF (rotated BRIEF). The ORB feature of the captured image is matched against the ORB feature of the fingerprint image using Hamming distance as the metric of similarity. The approach shows good accuracy and average localization error is under 1 m at 96%. The major limitation of the approach is that the localization is performed only at special places called landmarks, which include all places with EXIT signs indoors. It means that any change in the location of such landmarks requires potential changes in the images fingerprint databases. A variety of feature extraction techniques have been proposed as well which are useful for scene recognition. For example, research [43; 44] works with dense depth images from depth camera. The features of spatial/temporal continuity, constraints of human motion information and centroids of each activity are used. Research [45] uses eigenfaces vector reduction by texture and shape vector phenomenon to remove complexity and density matching score with face boundary fixation extracted the most likelihood characteristics. Authors in [46] use multiple cameras to find the centroids of the blobs to calculate the 3D position of cues. Research [47] uses local descriptors for image description to classify images. The problems of low resolution, occlusion and pose and illumination change are tackled. Research [48] employs vertex-modeling structure for feature detection. Authors in [49] devise an approach which utilizes the received signal strength (RSS) of wireless signals and images to locate a person. Initial estimation of location search space shrinking is achieved using Wi-Fi signals. Results show that image assisted technique achieves an average error of 1 m at 50 percent. The localization error is increased due to random noise, path loss, multipath interference, etc., in Wi-Fi techniques. Authors in [50] work to overcome such limitations using the similarity of AP sets during offline and online phases. Similarly, channel state information (CSI) based fingerprinting can reduce these effects [51]. Recently, research [52] has used DNNs on RSS of Wi-Fi APs to perform floor detection and coarse localization. The fingerprinting process involves a substantial amount of time and one potential alternative to overcome this limitation is crowdsourcing. So, authors in [53] investigate the feasibility of using crowdsourced image data and develop a system called iMoon.