# Indoor-Guided Navigation for People Who Are Blind: Crowdsourcing for Route Mapping and Assistance
## 3 Indoor Navigation: Towards a Crowdsourced Approach
### Indoor Navigation Approach
GSM communication was implemented as a stand-alone device or smartphone that works as an intermediate processing device. Passive sensors passively collect environmental data, whereas active sensors, such as the 3D ToF IR camera, emit IR light to estimate distances to objects (see the principal scheme in Figure 3). Multi-sensory data are used to (i) find needed objects, (ii) locate obstacles, and (iii) infer users' locations in an indoor environment in order to help navigate. The devices and sensors observe the environment in real-time and send data via the controller to be machine learning processed, where feature extraction, object recognition, and data storage occur in the web cloud database server. Our approach integrates devices and interfaces using modern technology and methods from the machine learning and computational vision domain. We propose an algorithm for the automated pairing of camera images with motion classes extracted from raw wearable IMU data. This algorithm allows us to automatically label training sets for the training of imitation-learning controllers, whose outputs correspond to three movement classes (\"forward\", \"left\", \"right\") and a prediction reliability estimate, which is important for our application. In this way, raw IMU time series of movement classes (\"forward\", \"left\", \"right\") are collected by separately executing the corresponding movement. A convolutional LSTM classifier with four outputs (the first three outputs correspond to the classes \"forward\", \"left\", and \"right\", and the fourth is the prediction reliability estimate) is used. Softmax activation is used for the first three outputs, and sigmoid activation is used for prediction reliability estimates. The model is trained using modified cross-entropy (\(MCE\)) loss:  
\[MCE(x,y)=-\sum_{c=1}^{3}y_{c}logy_{c}(x)+\ \lambda(-\sum_{c=1}^{3}y_{c}logy_{c}(x)-y_ {4}(x))^{2} \tag{1}\]  
where \(y_{c}\) is the ground truth label, and \(y_{c}(x)\) is the corresponding class prediction (probability). Another example concerns an algorithm that allows transformation between the camera image and tactile display coordinates. This transformation is required to represent rectangles in camera frames in tactile display vibro motor activations. We assume that the tactile display's coordinate frame is located at the top-left element, and its orientation is the same as the orientation of the RGB camera image pixel coordinate frame. Since coordinate transformation between the tactile display coordinate frame and both the RGB camera coordinate frame and the depth camera coordinate frame is static, data can be stored in the configuration file or even omitted due to the insignificant differences between  
Figure 3: The principal scheme, where rectangular boxes indicate devices. The dotted line delineates the first modality (sighted volunteers mark indoor landscapes); the dashed line delineates the second modality (people who are BSVI obtain navigational instructions on the spot in real-time); the third modality includes the functionality of the second modality supplemented with the real-time web-crowd assistance. coordinate origins. Therefore, the camera's rectangle (\(x_{p}\), \(y_{p}\), \(w_{p}\), \(h_{p}\)) (top lowest point coordinates, width, and height) pixel space can be linearly mapped to a rectangle in the Vibro display matrix  
\[\big{(}x_{p},y_{p},\omega_{p},h_{p}\big{)}\rightarrow\bigg{(}\bigg{[}\frac{x_{ p}W_{m}}{W_{p}}\bigg{]},\bigg{[}\frac{y_{p}H_{m}}{H_{p}}\bigg{]},\bigg{[}\frac{ \omega_{p}W_{m}}{W_{p}}\bigg{]},\bigg{[}\frac{h_{p}H_{m}}{H_{p}}\bigg{]}\bigg{)}, \tag{2}\]  
where \(W_{p}\) and \(H_{p}\) are the image width and height, respectively, and \(W_{m}\) and \(H_{m}\) are the numbers of vibromotors in the columns and rows of the tactile display. Results are rounded to the nearest integer operator.