Topological Localization Based on Salient Landmarks

This paper presents a new topological localization system for mobile robot based on salient visual landmarks. The method computes opponency of color and texture among multi-scale image spaces and selects salient regions as natural landmarks according to the saliency map. Using hidden Markov model to construct each vertex, the topological map of environment is created online and robot localizes itself concurrently on it. Experiments show that salient landmarks are stable to large changes in scale, 2D rotation and viewpoint. It also shows that our localization system is effective and can guarantee mobile robot navigation safely under unknown environment. 1.0 INTRODUCTION AND RELATED WORKS Localization is a fundamental problem and one of challenges in the domain of mobile robot navigation. Current localization methods based on camera can be classified mostly as being geometric and topological. Topological approaches use a graph but exact coordinate to represent the environment, which makes them dominant in localization based on monocular vision. These methods are based on either landmarks detection or map matching typically. Landmarks (artificial landmark or corner or door etc.) are detected for localization under known environments. But in unknown environments it is not certain of what landmarks can be selected and how to detect them. And it is very difficult to get stable landmarks that are invariant to scale, rotation, illumination and viewpoint changes, which are required by mobile robot. Most segmentation algorithms are inefficient for natural environments. David Lowe presents a SIFT method which uses a similarity invariant descriptors formed by characteristic scale and orientation at interest points to obtain the features which are invariant to image scaling, translation, rotation and partially invariant to illumination changes . But SIFT can generate 1000 or more interest points which means difficult to satisfy real-time requirement of robot navigation. Robert Sim extracts local maxima from edge density map of the image to create a landmark prototype, then represents environment with a set of landmark prototypes built from learning. Unlike Lowe’s interest points Sim uses thumbnail image as landmark, that is, landmark region. Though the amount of landmarks decreases, they are not invariant to scale or viewpoint changes. Another method bypassing segmentation is appearance based environment representation. Iwan Ulrich uses omni-camera to sense environments and implements a place recognition system based on image histogram matching . The method is invariant to rotation but sensitive to illumination and viewpoint changes. Especially its computation cost is expensive. Chao Zhou uses ordinary CCD camera and multi-dimensional histogram to describe global appearance features . This method is simple but sensitive to scale, illumination and 2D rotation. Besides these, PCA method is also adopted , which is not appropriate for working online. To deal with the uncertainty of localization, probabilistic technologies are adopted such as Kalman filter and Markov localization . For Kalman filer, good statistical models of the sensors and their uncertainties must be provided and a large covariance matrix must be maintained. Markov localization needs propagate the probability density function. They all need expensive computational cost. The various topological approaches differ in what constitutes a node and how a node may be distinguished from others and the effect of sensor uncertainty etc. In our approach, we use single camera sense environment and extract natural landmarks, then create topological map based on hidden Markov model. So localization or environment recognition can be transform to the evaluation problem of HMM, which makes recognition robust and decreases the computational cost of traditional HMM based localization methods. 2.0 SALIENCY BASED NATURAL LANDMARK DETECTION Research on biological vision system indicate that organism (like drosophila) often pay attention to certain special regions in the scene for their behavioral relevance or local image cues when observing surroundings. These regions can be taken as natural landmarks to distinguish and represent effectively different environments. Enlightened by those and feature integrating theory, we create multi-scale images where the opponency of color and texture are computed to achieve the saliency map, which can be used to point out where the salient regions are. 2.1 Create multi-scale images Firstly multi-scale images from 1:1(scale 1) to 1:64(scale 4) are created. Let the inputted image G0, multi-scale images G1-G4, then G1 = w*G0 Gn+1=Subsampled Gn n∈[1,3] Here, w is a Gaussion low-pass filter, “*” denotes convolution operation. Let Center= {scale1, scale2} and Surrounding={sacle3, scale4}. The definition of opponency among scales is the feature difference of centers and surroundings denoted by “ ”, which means that the surroundings are interpolated then subtract the centers pixel by pixel. 2.2 Sub-maps of color opponency According to the property of human eyes, we transfer RGB space to RGBY space to emphasis the color opponencies. The space is calculated by R=r-(g+b)/2 G=g-(r+b)/2 B=b-(r+g)/2 Y=(r+g)/2-|r-g|/2-b. Then Sub-maps of color opponency are computed by RG(c,s)=|(R(c)-G(c)) (G(s)-R(s))| BY(c,s)=|(B(c)-Y(c)) (Y(s)-B(s))| Here, c∈Center,s∈Surrounding. RG(c,s) denotes the opponency between red and green ; BY(c,s) denotes the opponency between blue and yellow. So, 8 sub-maps are created. 2.3 Sub-maps of texture opponency Gabor filter is adopted to analyze texture for its ability of acquiring local optimum either in the time domain or in the frequency domain. Gabor filter is defined by h(x,y)=g(x,y)e. Because Gabor filter is polar symmetry in the frequency domain, the orientation 0-πcan cover the whole domain. For computing easily, only 4 orientations 0,45,90,135 are considered. The texture of 4 orientations are computed by Tθ(x,y)=| Gn (x,y)*hθ(x,y)| Here hθ(x,y) denotes Gabor filter function including 4 orientations. Now we compute the sub-maps of texture opponency by T(c,s,θ)=|Tcθ(x,y) Tsθ(x,y)| Here, c∈Center,s∈Surround,θ∈{0,45,90,135}. So 16 maps are obtained. 2.4 Combine all sub-maps to create saliency map (1) (2)