— Training a next-best-view (NBV) planner for ac- tive cross-domain self-localization is an important and chal- lenging problem. Unlike typical in-domain settings, the planner can no longer assume the environment state being constant, but must treat it as a high-dimensional component of the state variable. This study is motivated by the ability of recent visual place recognition (VPR) techniques to recognize such a high-dimensional environment state in the presence of domain- shifts. Thus, we wish to transfer the state recognition ability from VPR to NBV . However, such a VPR-to-NBV knowledge transfer is a non-trivial issue for which no known solution exists. Here, we propose to use a reciprocal rank feature, derived from the ﬁeld of transfer learning, as the dark knowledge to transfer. Speciﬁcally, our approach is based on the following two observations: (1) The environment state can be compactly represented by a local map descriptor, which is compatible with typical input formats (e.g., image, point cloud, graph) of VPR systems, and (2) An arbitrary VPR system (e.g., Bayes ﬁlter, image retrieval, deep neural network) can be modeled as a ranking function. Experiments with nearest neighbor Q- learning (NNQL) show that our approach can obtain a practical NBV planner even under severe domain-shifts. I. I NTRODUCTION In this paper, we aim to train a next-best-view (NBV) planner for active cross-domain self-localization. Given a landmark map built in a past domain (e.g., weather, season, times of the day), the goal of self-localization is to localize the robot-itself, using relative measurements from on-board landmark sensor and odometory [1]–[3]. This cross-domain self-localization problem becomes a challenging one due to appearance/removal of landmarks as well as perceptual aliasing. Thus far, most of previous works on cross-domain self-localization suppose a passive observer (i.e., robot) and do not take into account the viewpoint planning or controlling the observer [4]–[6]. However, such a passive self-localization setting can be ill-posed, due to spatial sparseness of domain-invariant landmarks (e.g., pole-like landmarks [7]), salient viewpoints, and sufﬁcient landmark views (Fig. 1). Therefore, we consider active self-localization with an active observer that can adapt its viewpoint trajectory, avoiding non-salient scenes that may not provide sufﬁcient landmark view, or moving efﬁciently towards places which are most informative, in the sense of reducing the sensing and computation costs. Our work has been supported in part by JSPS KAKENHI Grant-in-Aid for Scientiﬁc Research (C) 17K00361, and (C) 20K12008. K. Tanaka is with Faculty of Engineering, University of Fukui, Japan. tnkknj@u-fukui.ac.jp We would like to express our sincere gratitude to Kanya Kurauchi for development of deep learning architecture, and initial investigation on deep reinforcement learning on the dataset, which helped us to focus on our VPR-to-NBV project. Fig. 1. Motivation of the proposed work. Domain-invariant landmarks are spatially sparsely distributed, which makes the cross-domain self- localization an ill-posed problem for a passive observer, and leads to the necessity of an active self-localization framework. Shown in the ﬁgure are domain-invariant pole-like landmarks, which are extracted by state-of-the- art deep holistic landmark detection in [8]. Training an NBV planner for active cross-domain self- localization is computationally very challenging, and most likely the reason why there is little solution to this problem. Unlike in-domain self-localization, the planner can no longer assume the environment state being constant, but must treat it as a high-dimensional component of the state variable. This signiﬁcantlly increases the dimensionality and size of the state space, which makes a direct implementation of standard training procedure such as reinforcement learning computationally intractable. This study is motivated by the ability of recent visual place recognition (VPR) techniques to recognize such a high- dimensional environment state in the presense of domain- shifts [9]–[11]. Thus, we wish to transfer the state recognition ability from VPR to NBV. However, such a VPR-to-NBV knowledge transfer is a non trivial issue for which no known solution exists. Here, we propose a novel VPR-to-NBV knowledge trans- fer framework that uses reciprocal rank feature [12] as the dark knowledge. Speciﬁcally, our approach is based on the following two observations: (1) The environment state can be compactly represented by a local map descriptor, which is compatibile with typical input formats (e.g., image, point cloud, graph) of VPR systems [13], and (2) An arbitrary VPR system (e.g., Bayes ﬁlter [1], image retrieval [2], deep neural network [12]) can be modeled as a ranking function. The proposed approach is inspired by the recent development of transfer learning, where rank matching is used as the dark knowledge to transfer from a teacher to a student model (e.g., Fig. 2. Algorithm pipeline. In this paper, we focus on the module for knowledge transfer from VPR to NBV. classiﬁer) [14]. Such a ranking -based knowledge transfer has been further explored in our previous studies on knowledge distillation [15], rank fusion [16], image change detection [17], and graph convolutional neural network [12]. While these existing studies focus on the passive self-localization setting, the current study explores the issue of active self- localization. The proposed framework has the following advantages. (1) Compact: While a local map can grow in unbounded fashion as the robot explores a large area, it can be compressed into a compact bit pattern by using the hash-map technique [18]. (2) Discriminative: The reciprocal rank feature can serve as a discriminative dark knowledge to differentiate between different states [12]. (3) Drift-free: Viewpoint-drifts between local maps can be suppressed by using the local map descriptor [13]. Experiments with nearest neighbor Q-learning (NNQL) [19] show that our approach can obtain a practical NBV planner even under severe domain-shifts. II. A PPROACH Figure 2 depicts the architecture of the proposed frame- work. Without losing generality, we consider a typical 2D robot navigation scenario [20] (Fig. 3), which is character- sized by 2D point landmarks (e.g., pole-like landmarks) in a 2D bird’s-eye-view environment, and by 2D robot poses (x;y;q)and actions (forward, rotate) on the moving plane. A 2D omni-directional range ﬁnder (ORF) equipped on the robot provides both positive and negative observations for each direction a(jaj p)(Fig. 4). Positive observations 460 480 500 520 540 460 480 500 520 540t=1 460 480 500 520 540 460 480 500 520 540 t=4 460 480 500 520 540 460 480 500 520 540 t=6 460 480 500 520 540 460 480 500 520 540 t=9 460 480 500 520 540 460 480 500 520 540 t=19 460 480 500 520 540 460 480 500 520 540 t=26 460 480 500 520 540 460 480 500 520 540 t=28 460 480 500 520 540 460 480 500 520 540 t=29 460 480 500 520 540 460 480 500 520 540 t=31 Fig. 3. Examples of landmark detection and self-localization results. ’ ◦’: detected landmark. ’-’: hypothesized viewpoint location and orientation, for highly-ranked hypotheses that receive score values higher than 50%. are the 2D end-points of ORF readings fd(a)gthat are associated to obstacles or landmarks. Negative observations are the areas swept by ORF readings before reaching the end-points that are associated to free-space. The local map descriptor incorporates both positive and negative observations. These positive and negative maps are represented respectively with an array of 2D relative landmark locations and with a local 2D regular grid map where each grid cell can take either one of the states: unknown, free, or occupied (Fig. 4). The training engine for the NBV planner adapts the recently developed framework of NNQL [19]. NNQL has the following desirable properties. (1) Unlike many exisiting Q- learning frameworks, NNQL provides a good approximation of the Q-values for even unexperienced states and/or actions which the robot has not been explored yet. (2) Therefore, NNQL is expected to be robust against domain gaps be- tween training and test states. (3) Unlike many Q-learning algorihtms, the convergence of NNQL is guaranteed. (4) Its convergence speed could be further boosted by employing an external NN retrieval engine. A VPR system is generally modeled as a ranking function, which can work with arbitrary VPR systems (e.g., Bayes ﬁlter [1], image retrieval [6], deep neural network [21]). It evaluates the likelihood of the robot being located at each predeﬁned place class, given a query scene. In this study, we address a speciﬁc scenario, where a scene is represented by the local map descriptor, and thus each place class is predeﬁned as a cluster of descriptors. For the clustering, we use the standard clustering algorithm of k-means on 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8(a) Positive and negative maps 0 0.2 0.4 0.6 0.8 1 0 5 10 15 20 25 30 (b) Binary code 0 0.2 0.4 0.6 0.8 1 0 2000 4000 6000 8000 10000 (c) Class-speciﬁc ranks 0 0.2 0.4 0.6 0.8 1 0 2000 4000 6000 8000 10000 (d) Reciprocal rank feature Fig. 4. A visualization of intermadiate results. (a) An example of the positive and negative maps in the local map descriptor. The ’ □’ and ’ ◦’ indicate the locations of positive and negative grid cells. (b) Binary code vector. (c,d) Feature vectors for the class-speciﬁc rank and reciprocal rank features. It can be seen that the latter vector is much discriminative than the former one. a training set, although the problem of place clustering to optimize pureness of place clusters is an important topic of on-going researches [22]. We observe that domain-invariant landmarks are often spa- tially very sparse. For example, domain-invariant pole-like landmarks in [7] (Fig. 1) are signiﬁcantlly more spatially- sparse than general purpose landmarks such as local keypoint descriptors [23]. Such a characteristics will be also taken into consideration in the proposed approach. A. Active Self-localization Problem The active self-localization task takes a motion observation history stat a time step tand determines the NBV action at, which consists of a 2D translation ( x[m], y[m]) and a rotation q[deg]. The action space is deﬁned as the grid of f(x;y)j jxj+jyj>0,x;y2 f0;10;20gg. The state space is deﬁned as the space of high-dimensional local map descriptor. B. Reinforcement Learning Reinforcement learning (RL) [24] is used to train the NBV planner. In the active self-localization application, a model-based planner such as probabilistic road map [25] is not applicable, because of the unavailability of world-centric map information. Instead, a model-free planner is preferred, to map the available ego-centric map information to the world-centric NBV action plan. RL is a standard approach to such a model-free planner. In RL, the reward function rewards a given state-action in terms of the total self-localization performance. A naive strategy is to deﬁne an estimate of the self-localizationperformance (i.e., mAP) directly as a reward function. Un- fortunately, such an estimation is computationally expensive, and the training cost per episode grows in proportion to the trajectory length. To address this issue, in our approach, the number of detected landmarks is viewed as a light-weight proxy for such an expensive reward function. This strategy is based on our observation that the number of detected landmarks has strong positive correlation with the self-localization performance (Fig. 1) [8]. Thanks to such a lightweight proxy, the overhead of the reward function against the total training cost becomes negligively small. C. Local Map Descriptor Viewpoint drift between training and test scenes is a major source of difﬁculty in viewpoint planning [26]. It is inher- ently caused by accumulative pose tracking errors during long-term navigation, as well as robot kidnapped problems. A straightforward way to address this issue is aligning every pair of training and test scenes (e.g., via scan/map -matching [27]) in online. Unfortunately, this naive strategy requires a signiﬁcant increase in time cost per episode. Here, we propose to pre-align every training/test scene’s coordinate with a domain-invariant coordinate system (ICS), to avoid the need of online alignment. While the problem of ﬁnding such an ICS (i.e., an origin and axes) for a given scene is in general ill-posed [13], fortunately in our appli- cation domain the location of a domain-invariant landmark can be used as the ICS’s origin (Fig. 1) [28], and moreover, the spatial distribution of domain-invariant landmarks can be used as a cue to determine the ICS’s orientation (Fig. 3). The detailed procedure for determining ICS is as follows. First, the ICS’s axes are determined so that the Entropy of landmark locations along the x-axis becomes maximum [29]. Then, the ICS’s origin is ﬁne-adjusted to the location of the landmark with the shortest distance to the robot’s viewpoint. This ﬁne-adjustment is triggered only when the shortest distance does not exceed a predeﬁned threshold Td=1[m]. It should be noted that ICS is effective both for VPR and NBV, as demonstrated in our previous studies [28] and [8]. In VPR, the ICS provides a viewpoint-invariant discriminative scene descriptor [28]. In NBV, the ICS suppresses the effects of viewpoint-drifts between training and test scenes [8]. D. Hash Map The hash-map technique in [16] is employed to compress the variable-size local map into a constant-length binary code. In general, a local map can grow in unbounded fasion as the robot explores a large area. With a few exceptions (e.g., [30]), most existing VPR algortihms can not address such a variable-size input format. To address this issue, we map the variable size local map into a constant-length Z-dim representation z(Z=32). For this, the hash-map is used with a random projection Y=PX, followed by a mod operation z[j] =åj2SZy[i]where SZ=fjj(imod Z) =jg. More speciﬁcally, the positive and negative information in the local map are ﬁrst translated to two separete grid maps, named positive and negative maps, either of which is represented by a 8 8 grid with spatial resolution of 1[m]. Then, either map is compressed via the above hash-map into aZ-dim representation. Then, the results are concatenated and binarized to a (2Z)-dim bit-code. It should be noted that the projection matrix Pcan be recovered on-the-ﬂy as an array of pseudo random numbers given a predeﬁned random seed. Thus, the space cost for the projection matrix is constant and negligibly low. E. Reciprocal Rank Features The reciprocal rank feature [12] is used as the dark knowledge for the VPR-to-NBV knowledge transfer. This strategy is based on the observation that any off-the-shelf VPR system can be modeled as a ranking function that ranks a set of predeﬁned place classes based on their similarity to a given query scene. In this study, a query scene is representated by a local map descriptor, and the place classes are deﬁned in ofﬂine by clustering training descriptors into 10,000 place classes via k-means clustering. The proposed approach has several desirable properties. First, using rank values as the dark knowledge is theoretically supported by the rank matching loss, which is used in recent transfer learning techniques such as knowledge distillation [14]. Second, the reciprocal rank is additive feature [31]. Third, the reciprocal rank values are successfully used in multi-modal information retrieval [32] and in VPR [16]. F . Nearest Neighbor Q-Learning (NNQL) A key difference of NNQL [19] from the standard QL is that the Q-value of an input state-action-pair (s;a)is approximated by a collection of Q-values that are associated with its nearest neighbors N(s;a): Q(s;a) =1 jN(s;a)jå (s′;a′)2N(s;a)Q(s′;a′) (1) We build a set of independent jAjNN engines for individual jAjaction candidates. Thus, the Q-function is approximated by: Q(s;a) =argmax a1 jN(sja)jå (s′;a)2N(sja)Q(s′;a); (2) where N(sja)is the nearest neighbors of (s;a)conditioned on a given action a. In our implementation, the set N(sja) is deﬁned as f(s′;a′)j js′sj 2;a′=ag, where j  jis L1- norm. Such an action-speciﬁc NNQL also can be viewed as an instance of RL [24]. III. E XPERIMENTAL RESULTS The goal in this experiment is to test the effectiveness of the proposed framework, whose scene and VPR models respectively are based on the local map descriptor (LMD) and reciprocal rank transfer (RRT). We generated a collec- tion of 3,300 different settings, each of which consists of the robot initial viewpoint and a conﬁguration of domain- invariant landmarks. To make active self-localization a non- trivial problem, we focus on challenging environments,which consist of many repetitive/symmetric structures, and a few discriminative structures. Speciﬁcally, we consider three different types of landmark conﬁguration: “CIRCU- LAR”, “ROAD”, and “RECTANGULAR” (Fig. 5). The “CIRCULAR” conﬁguration is intended for an application in which poles lined up along the perimeter of a circular park are used as landmarks. The “ROAD” conﬁguration envisions an application in which pole landmarks are lined up at random intervals in [0, 1.0] [m] along a pole-lined road. The “RECTANGULAR” conﬁguration envisions an application where pole landmarks are lined up along the perimeter of a square parking lot. The training and test environments are created in three steps. (1) First, prototypes of the “CIRCULAR”, “ROAD” and “RECTANGULAR” landmark conﬁgurations are created in the following manner. For “CIRCULAR” conﬁguration, each i-th landmark loca- tion (xi;yi)is determined by: [x0+Rcos(∆(q)r2p=N),y0+ Rsin(∆(q)r2p=N)].ris a sample from a uniform distribution in [0, 1]. For “ROAD” conﬁguration, the ﬁrst landmark’s location is determined (x0;y0) = ( 0;500)and then each i- th landmark’s location (i>1)is incrementallly generated at the location (xi;500) = ( xi1+r∆L;500), until xi>1000 . For “RECTANGULAR” conﬁguration, landmarks are generated in the similar incremental manner as in “ROAD”, but along the perimeter of a square [400;600][400;600]instead of along the single line segment. ∆L=1:0[m]. N=100. R=40m. (2) Then, a set of 1,000 training environments are created by randomly modifying each prototype landmark conﬁguration. (3) Then, a set of 100 test environments are created by randomly modifying each of 100 randomly-selected training environments for Mtimes, where Mis set to 10% of the number of landmarks in the environment of interest. For the random modiﬁcation of an environment, we use two kinds of operations: appearance and removal of landmarks. The appearance operation adds a new landmark at a random location. The removal operation removes a randomly-chosen one of the existing landmarks. It should be noted that the above procedure yields a challenging set of training/test environments that consist of near-duplicate, repetitive and symmetric structures, which suffers from perceptual aliasing. The on-board ORF has the azimuth resolution of 1.0 deg and its maximum range is 10.0 m. In the training and test stages, the number of actions per episode is set to 100. Difﬁculty of the landmark-detection and self-localization tasks strongly depends on the distance from the initial robot viewpoint to the closest landmark. If the distance is larger than the ORF’s maximum range, a robot’s exploring behavior such as random walk would be required to encounter the ﬁrst landmark. To avoid dependency of the initial setting, each training/test episode starts with such an initial viewpoint at which at least one landmark exists in the ﬁeld-of-view. Likewise, action candidates are restricted to those by which the robot moves to a viewpoint at which at least one already- detected-landmark exists within the ﬁeld-of-view. During a navigation task, the robot repeats the three steps: observation, plan, and action, as in Section II. In the test stage, we perform a highly accurate map- 400 450 500 550 600 400 450 500 550 600Environment map Target environment Dummy environment #1 Dummy environment #2 400 450 500 550 600 400 450 500 550 600Environment map Target environment Dummy environment #1 Dummy environment #2 400 450 500 550 600 400 450 500 550 600Environment map Target environment Dummy environment #1 Dummy environment #2 0 10 20 30 40 50 60 70 80 0 10 20 30 40 50 60 70 80 90 100#detected-landmarks [%] time stepRANDOM LMD+RRT w/o adjust LMD+RRT w/ adjust axis LMD+RRT w/ adjust axis+origin 0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100#detected-landmarks [%] time stepRANDOM LMD+RRT w/o adjust LMD+RRT w/ adjust axis LMD+RRT w/ adjust axis+origin 0 5 10 15 20 25 30 35 40 45 0 10 20 30 40 50 60 70 80 90 100#detected-landmarks [%] time stepRANDOM LMD+RRT w/o adjust LMD+RRT w/ adjust axis LMD+RRT w/ adjust axis+origin 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100localization accuracy [%] time stepRANDOM LMD+RRT w/o adjust LMD+RRT w/ adjust axis LMD+RRT w/ adjust axis+originCIRCULAR 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100localization accuracy [%] time stepRANDOM LMD+RRT w/o adjust LMD+RRT w/ adjust axis LMD+RRT w/ adjust axis+origin ROAD 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 100localization accuracy [%] time stepRANDOM LMD+RRT w/o adjust LMD+RRT w/ adjust axis LMD+RRT w/ adjust axis+origin RECTANGULAR Fig. 5. Experimental results. Each column from left to right shows “CIRCULAR”, “ROAD”, and “RECTANGULAR”. For the sake of clarity, only a portion of the environment is shown for the “ROAD” type conﬁguration. Top: Examples of the traininig and test environments. For the test environments, the target environment and two dummy environments are shown. Middle: Landmark detection performance versus time step. Bottom: Self-localization performance versus time step. matching -based self-localization, at each time-step during the robot navigation, because there is no severe restriction on the computational cost per episode as in the training stage. For the map-matching -based self-localization, we employ RANSAC map-matching between the latest ego- centric local map and the a-priori given world-centric map. More speciﬁcally, a 2-point RANSAC algorithm is used to hypothesize and score a set of 3-dof viewpoint hypothe- ses. For performance evaluation, the success of the self- localizatiojn is deteremined by whether the error with respect to the ground-truth is less than 1m.For performance comparsion, we developed four differ- ent NBV methods: “RANDOM”, “LMD+RRT w/o adjust”, “LMD+RRT w/ adjust axis”, and “LMD+RRT w/ adjust origin+axis”. “RANDOM” is a naive strategy that randomly selects an action from the executable action candidates at each time step. “LMD+RRT w/ adjust origin+axis” is the proposed strategy, which uses LMD for environment state represenation, RRT for VPR-to-NBV knowledge transfer, and ICS for aligning origin/axes of the ego-centric local map. The methods “LMD+RRT w/ adjust axis” and “LMD+RRT w/o adjust” are ablations of the proposed method. The former does not use the alignment of the origin. The latter does not use both the alignment of the origin and axes. Figure 3 demonstrates typical viewpoint trajectories for the proposed NBV planner at the test stage. As can be seen, the viewpoint hypotheses are gradually reﬁned as the robot moves and detects new landmarks. Figure 5 shows the basic performance of methods. It can be seen that the proposed method, which combines LMD and RRT with adjusting the axes and origin, outperforms all the other methods in both landmark detection and active self-localization tasks. IV. C ONCLUSIONS In this study, we explored the novel task of active cross- domain self-localization from the perspective of VPR-to- NBV knowledge transfer. Speciﬁcally, our approach is based on the following two observations: (1) The environment state can be compactly represented by a local map descriptor, which is compatibile with typical input formats (e.g., image, point cloud, graph) of VPR systems, and (2) An arbitrary VPR system (e.g., Bayes ﬁlter, image retrieval, deep neural network) can be modeled as a ranking function. In our contri- butions, we proposed to use a reciprocal rank feature, derived from the ﬁeld of transfer learning, as the dark knowledge to transfer. Experiments showed that our approach can obtain a practical NBV planner even under severe domain-shifts.