—This paper addresses the problem of active visual place recognition (VPR) from a novel perspective of long-term autonomy. In our approach, a next-best-view (NBV) planner plans an optimal action-observation-sequence to maximize the expected cost-performance for a visual route classiﬁcation task. A difﬁculty arises from the fact that the NBV planner is trained and tested in different domains (times of day, weather conditions, and seasons). Existing NBV methods may be confused and deteriorated by the domain-shifts, and require signiﬁcant efforts for adapting them to a new domain. We address this issue by a novel deep convolutional neural network (DNN) -based NBV planner that does not require the adaptation. Our main contributions in this paper are summarized as follows: (1) We present a novel domain-invariant NBV planner that is speciﬁcally tailored for DNN-based VPR. (2) We formulate the active VPR as a POMDP problem and present a feasible solution to address the inherent intractability. Speciﬁcally, the probability distribution vector (PDV) output by the available DNN is used as a domain- invariant observation model without the need to retrain it. (3) We verify efﬁcacy of the proposed approach through challenging cross-season VPR experiments, where it is conﬁrmed that the proposed approach clearly outperforms the previous single-view- based or multi-view-based VPR in terms of VPR accuracy and/or action-observation-cost. I. I NTRODUCTION Long-term visual place recognition (VPR) is central to mo- bile robot and intelligent vehicle applications. The task of VPR is to classify egocentric view images into predeﬁned place classes [1]. Long-term VPR is a challenging scenario, in which the VPR system is trained and tested in different domains (e.g., times of day, weather conditions, and seasons) [2]. A key difﬁculty comes from appearance changes caused by domain shift, such as changes in viewpoints, illumination, weather conditions, and seasons. One of the most successful solutions to address this challenge is to ﬁne-tune a deep convolutional neural network (DNN) as a visual place classiﬁer (VPC) by using action-observation experiences in a past domain as the training data [3]–[5]. Most of the previous works on VPR have focused on a passive setting, in which the robot’s action is determined by a predeﬁned control rule, such as a constant speed motion rule. However, such a passive VPR system has several limitations. Our work has been supported in part by JSPS KAKENHI Grant-in-Aid for Scientiﬁc Research (C) 26330297, and (C) 17K00361. Fig. 1. The robot plans an optimal action-observation-sequence to maximize the expected cost-performance for a visual route classiﬁcation task. In the ﬁgure, the ground-truth route class is c=2. The action set consists of forward (FW) and backward (BW) moves. There is signiﬁcant difference in appearance between test (blue) and train (red) images due to domain shift. First, the viewpoints are not necessarily optimized for the VPR task, yielding less informative or non-discriminative images that may lead to suboptimal VPR performance. Second, such a simple control rule may produce an unnecessarily large number of redundant observations, which may result in a signiﬁcant decrease in the cost performance of VPR. In this work, we consider an active VPR setting called visual route classiﬁcation (Fig. 1). The task of active VPR is to classify a view sequence into predeﬁned route classes. To maximize the cost performance of VPR, we pretrain a state-to- action mapping function called next-best-view (NBV) planner in a past domain and use it to select an NBV at each time step during the active VPR task. However, a difﬁculty comes from the domain shift. Typical planners, such as those based on low-level image features (e.g., color, texture, and shape) [6] or middle-level image features (e.g., objects, landmarks, and GIST) [7], may be confused and deteriorated by domain shifts [8]. Adapting these planners to a new domain requires signiﬁcant efforts to collect training data and to retrain [9]. To address these issues, we propose a novel domain- invariant NBV planner that does not require the adaptation. Our study was motivated by recent ﬁndings in the VPR community that show that the probability distribution vector (PDV) output by the ﬁnal layer of a DNN provides domain- invariant semantic information for the input scene [10]. Specif- ically, we were inspired by our recent study on long-term knowledge distillation (KD) [11], in which the PDVs of DNNs (“teacher”) provide abundant information that can be used even for training other DNNs (“student”) in a different domain. Our idea in this study is to exploit such PDV information for an observation model to develop a novel domain-invariant NBV planner. Moreover, we formulate the NBV planning task as a partially observable Markov decision process (POMDP) [12] and show that PDVs can actually be used as useful information that realizes a feasible solution. Our main contributions in this paper are summarized as fol- lows: (1) We address the problem of active VPR from a novel perspective of long-term VPR, in which the effect of domain shift is explicitly addressed by introducing a domain-invariant NBV planner. (2) We formulate the active VPR task as a POMDP problem and provide a feasible solution to address the inherent intractability. Speciﬁcally, our domain-invariant planner extracts and exploits useful semantic information from the existing DNN component without the need to retrain it. (3) We verify the efﬁcacy of the proposed approach through chal- lenging cross-season VPR experiments, focusing on outdoor scenes with no distinctive landmark objects, using the publicly available NCLT dataset (Fig. 2). Through the experiments, it was conﬁrmed that the proposed approach clearly outperforms the previous single view-based or multiview-based VPR in terms of VPR accuracy and/or action-observation-cost. II. A PPROACH The goal of an NBV planner is to plan an optimal action- observation-sequence that is expected to maximally improve the cost performance of VPR. Our work is based on the following three assumptions. (1) We assume the availability of a DNN-based VPR that is pretrained in a self-supervised manner. Prior to the active VPR task, the robot workspace is partitioned into Kdisjoint routes, or place classes, by using a place partitioning technique such as that used in [13]. Then, the viewpoint trajectory in the training data is precisely reconstructed by using techniques such as the SfM [14] or the SLAM [15] technique. Then, a DNN is ﬁne-tuned as a K- class VPC by using the route-speciﬁc viewpoint-image pairs as the training data. (2) We also assume that, during an active VPR task, the robot is located in one of the routes or places. A robot’s action is deﬁned as a move between viewpoints along the route. It should be noted that even though the robot is located in the same route, there remains a non-negligible difference in appearance between the train and the test scenes. (3) Finally, we assume the availability of a scoring function, which takes as input an image sequence produced by an action- observation sequence, applies each image to the DNN, and aggregates the PDV sequence into a ﬁnal decision in the form of a K-dimensional score vector. In the implementation, we use a simple element-wise multiplication for the aggregation: score[c]= ptPDV[c]. This section provides a description of the active VPR framework. First, the active VPR task is formulated as a POMDP problem (Section II-A). Then, its main components, namely, the action set (Section II-B), reward function (SectionII-C), and observation function (Section II-D), are introduced. Finally, a multiple-hypothesis approach is presented for real- world VPR applications (Section II-E). A. Active VPR Formulation Following the deﬁnition by Kaelbling [12], we deﬁne POMDP as a six-tuple (S;A;T;R;W;O), where Sis a ﬁnite set of states, Ais a ﬁnite set of actions, T:SAS![0;1]is the transition function deﬁning the probability of state change caused by an action, R:SA!Ris the reward function that represents the reward granted after having reached the new state with the given action, Wis a ﬁnite set of observations, andO:SAW![0;1]is the probability distribution of the observations according to the states and the actions. At each time step, given a current state s2S, the agent receives an observation o2Wwith the probability O(s;o) = Pr(ojs). Depending on this observation and the current state, the agent takes an action a2A, which causes a transition to state s′with the probability T(s;a;s′) =Pr(s′js;a). Finally, the agent receives a reward from the reward function R(s;a). The above process repeats until the “terminate” action (see Section II-B) is selected by the planner. We deﬁne the state as a pairing of class cand viewpoint x: s= (x;c). The class cis the hidden class of the place or route. The viewpoint xis represented as a travel distance along the route from the start location on the route (Fig. 1). For a while, we assume that the viewpoint xis measurable. Obviously, in practice, this assumption does not hold in the application of VPR. Therefore, we relax this assumption by introducing a multiple-hypothesis-tracking approach (Section II-E). B. Action Set The set of actions involves moving between viewpoints along the class-speciﬁc route. In practice, such an action could be implemented by using a vision-based route-following algorithm such as that in [16]. In our experimental system, the action set Aconsists of seven different possible actions. This includes three forward moves (FWs, i.e., “a1,” “a3,” and “a5”), three backward moves (BWs, i.e., “a2,” “a4,” and “a6”), and an additional special action (TN, i.e., “a7”) that terminates the active VPR and outputs the most likely place class as the ﬁnal answer. These actions are deﬁned as follows: a1 : x x+L=2, a2 : x xL=2, a3 : x x+L=4, a4 : x xL=4, a5 : x x+L=8, a6 : x xL=8, a7 : TN, where Lis the length of the route in terms of the total number of image frames. The transition function deﬁnes the probability of state change given an action. The state transition function can be written as follows: T(s;a;s′) =Pr(s′js;a). In this study, we assumed that a transition causes no motion noise. In practice, this assumption could be held by introducing a precise visual odometry system [17]. C. Reward Function The reward function R(s;a)represents the reward for reach- ing a new state swith a given action a. We deﬁne rewards for several different events. When the robot takes an action, e.g., moves forward or backward for a distance i(L=8) (i2f1;2;4g) (i.e., “a1”-“a6”), the reward is iCl, which is proportional to the travel distance. To ensure that the robot should move within a predeﬁned ﬁxed area [0, L] on the class-speciﬁc route, we grant an inﬁnity negative reward - ¥when it attempts to move outside of the area. When the robot takes the special action “a7,” a large positive (Cc)or negative (Cf)reward is granted depending on whether the answer is correct or not. Every time the robot observes to take a new image, a ﬁxed predeﬁned reward of Cbis granted. D. Observation Function The observation set is represented as Ω. As mentioned previously, we are motivated to use the PDV output by a DNN as a domain-invariant observation model. A PDV is a K-dimensional real-valued vector, and its elements sum up to 1. For the DNN, we use a convolutional neural network with the VGG16 architecture [18] that has been ﬁne-tuned on our K-class VPR task in the training domain. The input images are resized to 256256 before being inputted into the VGG16 network. We expect that a PDV output by the ﬁnal layer of a DNN contains a high-dimensional rich semantic information that is invariant against domain shift. Recent ﬁndings from the DNN community (e.g., KD [11]) have proven that such a PDV can tell us not only the domain-invariant information about the most likely place class but also how much conﬁdence the DNN has in the prediction. Therefore, a PDV-driven planner is expected to terminate the active VPR task (i.e., the action “a7”) as early as possible to avoid unnecessary action-observation costs. Moreover, the PDVs also provide the relative strength of conﬁdence between different place classes. Therefore, a PDV- driven planner is expected to learn Kdifferent behaviors, each of which is tailored to one of the Kdifferent routes. E. Multiple Hypothesis Tracking Now, we relax the unrealistic assumption that the viewpoint xis measurable and consider multiple hypotheses for x. Under the assumption of no-motion noise (Section II-B), this is simply realized by hypothesizing the initial viewpoint. For- mally, at the initial time step, a set of Hindependent POMDP planners are initialized for Hdifferent initial viewpoints, and they are run in parallel. At each time step t, the Hdifferent actions outputted by the Hplanners are aggregated into a single action plan, which is the ﬁnal decision that is executed by the real robot controller. As a result, a single actual observation result is obtained and the observation is used to Fig. 2. Illustration of the workspaces. The colored curves, i.e., pink, light blue, and yellow, correspond to workspaces #1, #2, and #3, respectively. TABLE I PERFORMANCE RESULTS . (a) success rate [%] WS1 WS2 WS3 single 86.7 64.5 77.1 random 84.5 64.5 77.2 proposed 87.8 66.7 82.8 (b) average number of observations WS1 WS2 WS3 single 1.00 1.00 1.00 random 2.57 2.67 2.66 proposed 2.14 1.99 2.51 (c) average travel distance WS1 WS2 WS3 single 0.0 0.0 0.0 random 79.2 89.3 78.1 proposed 34.3 34.4 40.0 trigger the next step run of the planners. In this way, the above steps are iterated until the special action “a7” is outputted as the ﬁnal decision. For the action aggregation, we consider a voting scheme in which every POMDP planner votes for the action plan that it selected. Subsequently, the action plan that received the highest votes is outputted as the ﬁnal decision. When multiple action plans receive the same number of votes, the plan with the lowest cost (deﬁned in Section II-C) is chosen as the ﬁnal decision. When there is a combination of FW and BW actions receiving the same lowest cost (e.g., “a3” + “a4”), the FW one is chosen. III. E XPERIMENTS We veriﬁed the efﬁcacy of the proposed method through active VPR experiments. For the dataset, we considered the NCLT dataset [19]. This dataset is one of the largest datasets for long-term autonomy and contains sequences of image data from the robot’s onboard imagery collected during a multi- session exploration across several different seasons. We used three datasets in NCLT: “2012/5/26” for training the VGG16 classiﬁer, “2012/5/11” for training a POMDP planner, and “2012/3/31” for testing the active VPR system, respectively. We considered three independent workspaces (WSs), as shown in Fig. 2, using the outdoor parts of the datasets that contain no distinctive landmark objects. All the three WSs were almost of the same length in terms of travel distance, and each WS was divided into four equal-length subsequences, each of which deﬁnes a route or a place class. In this way, we considered three independent sets of four-class classiﬁcation problems. The image sequence in each route was downsampled to have the same number of L=225 image frames; thus, all the classes summed up to 900 frames in total. Considering the dependency of the initial viewpoints, we executed 45 independent tests for 45 different initial viewpoints that were spaced by 5 image frames on each train route. The total performance was measured as the average of these 45 4 = 180 independent tests. For the POMDP solver, we employed SARSOP [20] and a script that automatically produces POMDPX data before using them as the input to the solver. For the NCLT dataset, the original image size is 1232 1616, and we resized every image before it was inputted to the VGG16 classiﬁer. The parameters were set Cr=100, Cf=100, Cb=5, and Cl=15. For performance comparison, we considered three compar- ison methods. The ﬁrst method (i.e., “single”) is the con- ventional passive single-view VPR, which uses just a single image at the initial viewpoint as the sole query input. The second method (i.e., “random”) is a multi-view VPR, which randomly selects the next action among the several possible actions, namely, “a1,” , “a7,” deﬁned in Section II-B. The last one (i.e., “proposed”) is the proposed multi-view VPR method, which is based on the pretrained POMDP model and the PDV-based observation model. Table I shows the performance results. It is evident that the proposed method clearly outperformed the other two methods in terms of VPR accuracy for all the WSs. The increase in performance was signiﬁcant, ranging from 2.2 to 5.6 percentage points. Especially, the performance gain was highest in the case of WS #3. Table I summarizes the costs in terms of average number of observations and motion costs. Evidently, the “single” method produced the lowest average cost. Comparing the “random” and “proposed” methods, we can see that the proposed method resulted in a signiﬁcant cost reduction in terms of both the number of observations and the motion costs. IV. C ONCLUSIONS We have presented a DNN-based NBV planner that incor- porates the output PDVs as a domain-invariant observation model. An algorithm is developed for planning an optimal action-observation-sequence to maximize the expected cost- performance of a visual route classiﬁcation task. We also formulate the active VPR as a POMDP problem and present a feasible solution to address the inherent intractability. Our algorithm outperformed the existing results in a challenging dataset that does not contain any distinctive landmark object. Although this paper was focused on solving a cross-seasonvisual route classiﬁcation task, this idea could be extended to any cross-domain DNN-based VPR in general.