Augmented Indoor Hybrid Maps using Catadioptric Vision

- Current research on appearance based mapping goes towards richer semantic models of the environment, which may allow the robots to perform higher level tasks and provide better human-robot interaction. This work presents a new omnidirectional vision based approach for augmented topological mapping. Omnidirectional vision systems are of particular interest for vision based mapping because they allow to have more compact and eﬃcient representation of the environment. Our proposal includes some novel ideas in order to augment the semantic information of a typical indoor topological map: we pay special attention to the semantic labels of the diﬀerent types of transitions between places, and propose a simple way to include this semantic information as part of the criteria to segment the environment. This work is built on eﬃcient catadioptric image representation based on the global Gist descriptor, which is used to classify the acquired views into types of indoor regions. The considered basic types of indoor regions are Places and Transitions , further divided into more speciﬁc subclasses, e.g., Transitions into doors, stairs and elevator. Besides using the result of this labeling, the proposed mapping approach includes a probabilistic model to account for temporal consistency. All the proposed ideas have been demonstrated and evaluated in a new indoor dataset acquired with our wearable catadioptric vision system 2 with promising results in a realistic prototype. This works presents a novel indoor topological mapping method augmented with diﬀerent labels of the basic indoor scenes. The mapping method uses catadioptric images and the adaptation of the Gist global descriptor to this kind of images. The general idea proposed is to simultaneously run the topological map building and a classiﬁer to label the diﬀerent types of indoor scenes considered. We have described a simple approach to label diﬀerent types of Places and Transitions . The result of our mapping method is a semantic-topological model, where the nodes are Places and the edges are Transitions between Places , including information about diﬀerent types of Places (Big, Medium or Small room, Corridors) and Transitions (Door, Jamb, Stairs, Elevator). A detailed semantic analysis of the types of transitions is not common although could provide important information for diﬀerent use of the map. Our topological mapping method is based on this semantic classiﬁcation of the images, using a previously built model of the environment, integrated with a Hidden Markov Model framework to add spatio-temporal consistency. We performed the experimental validation of our approach using the new Wearable OmniCam dataset acquired for this work. A second group of experiments was run to evaluate qualitatively the approach. They demonstrate the advantages of including the spatio-temporal framework and show the type of indoor topological models that can be obtained. Despite the simple and eﬃcient image representation proposed and the diﬃculty of the dataset, acquired from a camera on a helmet while the person walks normally, the map obtained is quite close to the ground truth manually generated. For future work, it is necessary to evaluate if this representation is valid not only for class labeling but also for loop detection, to provide a more consistent model of the environment when revisit to a certain place occurs. The step from our proposal that needs most improvements is the subclass classiﬁcation. Using only the gist based representation seems not enough sometimes, e.g., when trying to distinguish small rooms from other rooms or jambs from doors. Therefore, future work should include additional image features that allow to distinguish among indoor scenes with similar structure but small details that may provide important diﬀerences in the semantic label.