# Indoor-Guided Navigation for People Who Are Blind: Crowdsourcing for Route Mapping and Assistance
## 3 Indoor Navigation: Towards a Crowdsourced Approach
### Indoor Navigation Approach
In general, from the point of view of the end-user, the presented approach is distinguished from other related wearable indoor navigational ETA novelties in the sense of (a) having intelligent user interface integrity based on its unique tactile display and audio instructions; (b) having a hands-free intuitive control interface that uses EMG (or alternatively mobile app and panel); (c) having a comfortable user-orientated headband design; (d) providing machine-learning-based real-time guidance and object recognition; and (e) using web-crowd assistance to map indoor navigational routes and solve problematic situations. For efficient indoor navigation performance, the presented ETA system is used in three consequently interconnected modalities (see Figure 3):  
1. [label=()]
2. Web-crowd assistance when volunteers go through buildings and gather step-by-step indoor route information that is processed in the web cloud server and stored in the online DB;
3. BSVI usage of indoor web cloud DB routes when guided navigational assistance is needed;
4. in complex indoor situations (such as being lost, encountering unexpected obstacles and situations), the BSVI ETA system's multisensory data stream can be used in real-time to obtain voice-guided help from volunteers who are familiar with the particular route or building. Each modality is composed of the same set of eight modes that are active or work in the background. The user (volunteer or person who is BSVI) can activate these modes via a control interface (see Figure 4):  
1. [label=0., ref=0]
2. Object detector mode is based on the Faster RCNN neural network. Any other object detection architecture (e.g., CenterNet) can also be used. It accepts color image input from an RGB camera and detects a set of trained object classes, which are essential for people who are BSVI (e.g., corridor, door, elevator, stairs, etc.). Each detection consists of rectangles in the camera image, paired with corresponding class labels and reliability scores. CNN (convolutional neural network) object detection is trained by a standard gradient descend method and a custom training data augmentation algorithm. Our application object class detector can be used to detect physical objects and regions in an image with specific properties (e.g., a traversable/non-traversable area). 3. Specific object detector mode also accepts color image input from the user's camera and detects a set of pre-trained objects. The main difference is that this module not only relies on a CNN object detector but also uses template matching, which allows it to learn new objects instantaneously. Because new object learning is performed as corresponding images are included in the object model's database, new objects can be included by the user or its assistants. This module is based on commercial software. 4. Scene description mode accepts color image input from the user's camera and provides a textual description of the depicted scene. This module utilizes CNN to extract features from the input image and LSTM RNN to map it to a textual representation. Afterwards, the textual description is transformed into the speech for people who are BSVI to use. 4. Face recognition mode accepts color images and detects and recognizes faces within them. The list of faces that can be recognized can be managed by the user. The module is based on commercial software. 5. Optical character recognition mode relies on the CNN object detector module and commercial OCR API composition. The CNN object detector is used to detect the user's hand gestures, which potentially contain useful text. Afterwards, this corresponding region is processed via OCR to extract text. 6. Obstacle recognition mode relies on information from the depth camera. It detects obstacles that would be hard to detect with a white cane (e.g., obstacles in the upper body region). Standard point cloud segmentation methods detect obstacles. 7. Navigation mode relies on imitation-learning deep neural networks and object detection components.