Original: CAtNIPP: Context-Aware Attention-based Network for Informative Path Planning
Revised: CAtNIPP: Context-Aware Attention-based Network for Informative Path Planning

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract:
Revised: Abstract:

Original: Informative path planning (IPP) is an NP-hard problem, which aims at planninga path allowing an agent to build an accurate belief about a quantity of interestthroughout a given search domain, within constraints on resource budget (e.g.,path length for robots with limited battery life).
Revised: Informative path planning (IPP) is an NP-hard problem, which aims at planning a path allowing an agent to build an accurate belief about a quantity of interest throughout a given search domain, within constraints on resource budget (e.g., path length for robots with limited battery life).

Original: IPP requires frequent online replanning as this belief is updated with every new measurement (i.e., adaptive IPP),while balancing short-term exploitation and longer-term exploration to avoid suboptimal, myopic behaviors.
Revised: IPP requires frequent online replanning as this belief is updated with every new measurement (i.e., adaptive IPP), while balancing short-term exploitation and longer-term exploration to avoid suboptimal, myopic behaviors.

Original: Encouraged by the recent developments in deep reinforcement learning, we introduce CAtNIPP, a fully reactive, neural approachto the adaptive IPP problem.
Revised: Encouraged by the recent developments in deep reinforcement learning, we introduce CAtNIPP, a fully reactive, neural approach to the adaptive IPP problem.

Original: CAtNIPP relies on self-attention for its powerfulability to capture dependencies in data at multiple spatial scales.
Revised: CAtNIPP relies on self-attention for its powerful ability to capture dependencies in data at multiple spatial scales.

Original: Specifically, ouragent learns to form a context of its belief over the entire domain, which it usesto sequence local movement decision that optimize short- and longer-term searchobjectives.
Revised: Speciﬁcally, our agent learns to form a context of its belief over the entire domain, which it uses to sequence local movement decisions that optimize short- and longer-term search objectives.

Original: We experimentally demonstrate that CAtNIPP significantly outperforms state-of-the-art non-learning IPP solvers in terms of solution quality andcomputing time, and present experimental results on hardware.
Revised: We experimentally demonstrate that CAtNIPP signiﬁcantly outperforms state-of-the-art non-learning IPP solvers in terms of solution quality and computing time once trained, and present experimental results on hardware.

Original:  Introduction
Revised:  Introduction

Original: In many real-life robotic deployments that involve data acquisition, such as mapping/explorationof unknown areas for inspection or search-and-rescue applications, environmental monitoring andsurface inspection/reconstruction [1, 2, 3], an autonomous robot needs to plan a path to visit a givendomain and obtain measurements about a scalar field of interest, without a priori knowledge of thetrue underlying distribution of this information.
Revised: In many real-life robotic deployments that involve data acquisition, such as mapping/exploration of unknown areas for inspection or search-and-rescue applications, environmental monitoring and surface inspection/reconstruction [1, 2, 3], an autonomous robot needs to plan a path to visit a given domain and obtain measurements about a scalar ﬁeld of interest, without a priori knowledge of the true underlying distribution of this information.

Original: That is, starting from a uniform distribution withhigh uncertainty, the agent must construct a belief over the distribution of interest throughout thedomain (e g., target likelihood, temperature, surface roughness) based on successive measurementsalong its path.
Revised: That is, starting from a uniform distribution with high uncertainty, the agent must construct a belief over the distribution of interest throughout the domain (e.g., target likelihood, temperature, surface roughness) based on successive measurements along its path.

Original: This problem is known as the informative path planning (IPP) problem.
Revised: This problem is known as the informative path planning (IPP) problem.

Original: Specifically,
Revised: Speciﬁcally,

Original: IPP aims to plan a path that maximizes information gain , while satisfying a budget constraint (e.g.,path length for robots with limited battery life).
Revised: IPP aims to plan a path that maximizes information gain , while satisfying a budget constraint (e.g., path length for robots with limited battery life).

Original: IPP problems can be further classified as eithernon-adaptive or adaptive .
Revised: IPP problems can be further classiﬁed as either non-adaptive or adaptive .

Original: Non-adaptive solvers pre-plan a complete path offline and execute thispre-determined path, without any replanning upon obtaining new measurements online [4, 5, 6].
Revised: Non-adaptive solvers pre-plan a complete path ofﬂine and execute this pre-determined path, without any replanning upon obtaining new measurements online [4, 5, 6].

Original: Onthe other hand, adaptive solvers replan the search path frequently as the agent’s belief is updatedbased on new measurements [2, 7, 1].
Revised: On the other hand, adaptive solvers replan the search path frequently as the agent’s belief is updated based on new measurements [1, 2, 7].

Original: While our approach can also be used for non-adaptive IPP, wefocus on the more general adaptive IPP problem for its wider applicability to real-life robotic tasks
Revised: While our approach can also be used for non-adaptive IPP, we focus on the more general adaptive IPP problem for its wider applicability to real-life robotic tasks.

Original: Differently from general path planning problems, where the agent is often assigned a goal position,
Revised: Differently from general path planning problems, where the agent is often assigned a goal position,

Original: IPP requires the agent to identify and visit all potential interesting areas throughout the environment
Revised: IPP requires the agent to identify and visit all potential interesting areas throughout the environment.

Original: Therefore, efficient IPP solvers must reason about the entire agent’s belief to make non-myopic decisions [8], which balance short-term exploitation of known interesting areas with longer-term exploration of unknown areas in the domain.
Revised: Therefore, efﬁcient IPP solvers must reason about the entire agent’s belief to make non-myopic decisions [8], which balance short-term exploitation of known interesting areas with longer-term exploration of unknown areas in the domain.

Original: Many IPP solvers rely on computationally expensive means to optimize long-horizon trajectories [4, 7, 5].
Revised: Many IPP solvers rely on computationally expensive means to optimize long-horizon trajectories [4, 5, 7].

Original: Trading off solution quality in favor of lower computing times, more recent approaches have embraced sampling-based planning [9, 7, 10]
Revised: Trading off solution quality in favor of lower computing times, more recent approaches have embraced sampling-based planning [7, 9, 10].

Original: [Figure]
Revised: [Figure]

Original: To further improve computingtimes and solution quality, we introduce CAtNIPP, a deep reinforcement learning (dRL) basedframework for 2D adaptive IPP.
Revised: To further improve computing time and solution quality, we introduce CAtNIPP, a deep reinforcement learning (dRL) based framework for 2D adaptive IPP.

Original: We first decrease the complexityof our continuous-space searchdomain by generating a probabilistic roadmap [11], i.e., a random sparse graph that covers thedomain.
Revised: We ﬁrst decrease the complexity of our continuous-space search domain by generating a probabilistic roadmap [11], i.e., a random sparse graph that covers the domain.

Original: We then associate thisroadmap with the agent’s belief,and formulate adaptive IPP as asequential decision-making problem on this graph.
Revised: We then associate this roadmap with the agent’s belief, and formulate adaptive IPP as a sequential decision-making problem on this graph.

Original: We propose and train an attention-based neuralnetwork that outputs a policy to select which neighboring node to visit next, thus allowing us toiteratively plan an (adaptive) informative path.
Revised: We propose and train an attention-based neural network that outputs a policy to select which neighboring node to visit next, thus allowing us to iteratively plan an (adaptive) informative path.

Original: There, self-attention over the graph nodes allows theagent to construct a global context , by embedding its entire belief into local decision features whileidentifying dependencies between nodes/areas at different spatial scales.
Revised: There, self-attention over the graph nodes allows the agent to construct a global context , by embedding its entire belief into local decision features while identifying dependencies between nodes/areas at different spatial scales.

Original: In doing so, the neuralbased, reactive nature of our approach drastically improves planning times, compared to plannersthat optimize full trajectories, while context-awareness helps improve solution quality by allowingour agent to identify and sequence non-myopic decisions that near-optimally balance short- andlong-term objectives.
Revised: In doing so, the neural-based, reactive nature of our approach drastically improves planning time, compared to planners that optimize full trajectories, while context-awareness helps improve solution quality by allowing our agent to identify and sequence non-myopic decisions that nearoptimally balance short- and long-term objectives.

Original: The main contributions of this work are:
Revised: The main contributions of this work are:

Original: • We propose a new fully reactive, policy-based dRL framework for adaptive IPP, which significantly outperforms state-of-the-art IPP solvers in terms of both solution quality and computingtime.
Revised: • We propose a new fully reactive, policy-based dRL framework for adaptive IPP, which signiﬁcantly outperforms state-of-the-art IPP solvers in terms of both solution quality and computing time.

Original: In CAtNIPP, the agent learns a subtle global representation of its entire belief over the domain, that allows it to sequence non-myopic decisions that can achieve longer-term objectives.
Revised: In CAtNIPP, the agent learns a subtle global representation of its entire belief over the domain, that allows it to sequence non-myopic decisions that can achieve longer-term objectives.

Original: • We further propose to rely on receding-horizon, sampling-based trajectory optimization (Fig. 1) to output higher-quality, longer-horizon trajectories by leveraging the nature of our stochastic policies, while remaining tractable and usable in real time.
Revised: • We further propose to rely on receding-horizon, sampling-based trajectory optimization (Fig. 1) to output higher-quality, longer-horizon trajectories by leveraging the nature of our stochastic policies, while remaining tractable and usable in real time.

Original: We demonstrate these variants of CAtNIPP in simulation as well as on hardware on a light-intensity-based IPP task.
Revised: We demonstrate these variants of CAtNIPP in simulation as well as on hardware on a light-intensity-based IPP task

Original:  Related Work
Revised:  Related Work

Original: To reduce the computing time and improve solution quality, most recent IPP methods have relied onrandomized, sampling-based methods.
Revised: To reduce the computing time and improve solution quality, most recent IPP methods have relied on randomized, sampling-based methods.

Original: For non-adaptive IPP solvers, Karaman et al.
Revised: For non-adaptive IPP solvers, Hollinger et al.

Original: [9] introducedthe RIG-tree algorithm, which utilizes RRT-star (a Rapidly-exploring Random Tree variant [12]) torandomly build a tree structure used to explore the environment and maximize information gain.
Revised: [9] introduced the RIG-tree algorithm, which utilizes RRT-star (a Rapidly-exploring Random Tree variant [12]) to randomly build a tree structure used to explore the environment and maximize information gain.

Original: Hitz et al.
Revised: Arora et al.

Original: [5] combined Constraint Satisfaction and Travelling Salesman Problems to introduce
Revised: [5] combined Constraint Satisfaction and Travelling Salesman Problems to introduce

Original: Randomized Anytime Orienteering (RAOr).
Revised: Randomized Anytime Orienteering (RAOr).

Original: RAOr iteratively samples the search space and solvesa TSP instance to visit all sampled locations within the given budget constraints.
Revised: RAOr iteratively samples the search space and solves a TSP instance to visit all sampled locations within the given budget constraints.

Original: According to
Revised: According to

Original: Popovi´c et al.
Revised: Popovi´c et al.

Original: [1], despite RIG-tree and RAOr not having been initially designed for adaptive IPP,they only require seconds to plan, which allows them to be generalized to adaptive online replanningscheme in a traditional receding-horizon manner.
Revised: [1], despite RIG-tree and RAOr not having been initially designed for adaptive IPP, they only require seconds to plan, which allows them to be generalized to adaptive online replanning scheme in a traditional receding-horizon manner.

Original: Regarding solvers initially designed for adaptive
Revised: Regarding solvers initially designed for adaptive

Original: IPP, Hitz et al.
Revised: IPP, Hitz et al.

Original: [7] proposed an evolutionary strategy to achieve state-of-the-art performance.
Revised: [7] proposed an evolutionary strategy to achieve state-of-the-art performance.

Original: Theyapplied CMA-ES to generate candidate solutions from a multi-variate Gaussian distribution, wherethe mean and covariance matrices are adaptively updated according to the evaluation of the candidates.
Revised: They applied CMA-ES to generate candidate solutions from a multi-variate Gaussian distribution, where the mean and covariance matrices are adaptively updated according to the evaluation of the candidates.

Original: Apart from these conventional solvers, there has been a couple of recent, value-based RLsolvers for IPP.
Revised: Apart from these conventional solvers, there has been a couple of recent, value-based RL solvers for IPP.

Original: Wei et al.
Revised: Wei et al.

Original: [6] proposed an RNN-based solver, which reasons about the positionsof measurement (for their predicted reduction in uncertainty) but without considering measurementvalues, thus limiting its use to non-adaptive IPP problems.
Revised: [6] proposed an RNN-based solver, which reasons about the positions of measurement (for their predicted reduction in uncertainty) but without considering measurement values, thus limiting its use to non-adaptive IPP problems.

Original: Ruckin et al.
Revised: Ruckin et al.

Original: [13] proposed a specialized
Revised: [13] proposed a specialized

Original: CNN-based solver for 3D IPP, developed for a specific problem statement that assumes image-likemeasurements and relies on Kalman Filtering for belief update.
Revised: CNN-based solver for 3D IPP, developed for a speciﬁc problem statement that assumes image-like measurements and relies on Kalman Filtering for belief update.

Original:  Background
Revised:  Background

Original: In this section, we first introduce Gaussian Processes (GPs), which are used to model the agent’sbelief (i.e., predicted interest map).
Revised: In this section, we ﬁrst introduce Gaussian Processes (GPs), which are used to model the agent’s belief (i.e., predicted interest map).

Original: We then formulate the general definition of our IPP problembased on such a GP.
Revised: We then formulate the general deﬁnition of our IPP problem based on such a GP.

Original: Finally, we describe the adaptive replanning requirement for adaptive IPP.
Revised: Finally, we describe the adaptive replanning requirement for adaptive IPP.

Original: Gaussian Process In IPP, interest (e.g., target distribution, temperature, or radiation level), is associated with the 2D environment E ⊂ R 2 and modeled as a continuous function ζ : E −→ R .
Revised: Gaussian Process In IPP, interest (e.g., target distribution, temperature, or radiation level), is associated with the 2D environment E ⊂ R 2 and modeled as a continuous function ζ : E −→ R .

Original: Gaussian
Revised: Gaussian

Original: Processes have been widely used to represent such a continuous interest distribution, by providinga natural means to interpolate between discrete measurements [7, 1, 6], so that ζ ≈ GP ( µ, P ) .
Revised: Processes have been widely used to represent such a continuous interest distribution, by providing a natural means to interpolate between discrete measurements [1, 6, 7], so that ζ ≈ GP ( µ, P ) .

Original: Specifically, given a set of n ′ locations X ∗ ⊂ E at which interest is to be inferred, a set of nobserved locations X ⊂ E and the corresponding measurements set Y , the mean and covarianceof the GP are regressed as: µ = µ ( X ∗ )
Revised: Speciﬁcally, given a set of n (cid:48) locations X ∗ ⊂ E at which interest is to be inferred, a set of n observed locations X ⊂ E and the corresponding measurements set Y , the mean and covariance of the GP are regressed as: µ = µ ( X ∗ )

Original: + K ( X ∗ , X )
Revised: + K ( X ∗ , X )

Original: [ K ( X , X ) + σ 2 n I ] − 1 ( Y − µ ( X )) , P =
Revised: [ K ( X , X ) + σ 2 n I ] − 1 ( Y − µ ( X )) , P =

Original: K ( X ∗ , X ∗ ) − K
Revised: K ( X ∗ , X ∗ ) − K

Original: ( X ∗ , X )
Revised: ( X ∗ , X )

Original: [ K ( X , X ) + σ 2 n I ] − 1 × K
Revised: [ K ( X , X ) + σ 2 n I ] − 1 × K

Original: ( X ∗ , X ) T , where K ( · ) is a pre-trained/selectedkernel function, σ 2 n is a hyperparameter describing the measurement noise, and I is the n × n identitymatrix.
Revised: ( X ∗ , X ) T , where K ( · ) is a pre-trained/selected kernel function, σ 2 n is a hyperparameter describing the measurement noise, and I is the n × n identity matrix.

Original: In this work, following [1, 10], we use the Mat´ern 3 / 2 kernel function.
Revised: In this work, following [1, 10], we use the Mat´ern 3 / 2 kernel function.

Original: Informative Path Planning
Revised: Informative Path Planning

Original: The general IPP problem aims to find an optimal trajectory ψ ∗ in thespace of all available trajectories Ψ for maximum gain in some information-theoretic measures:
Revised: The general IPP problem aims to ﬁnd an optimal trajectory ψ ∗ in the space of all available trajectories Ψ for maximum gain in some information-theoretic measures:

Original: [Equation]
Revised: [Equation]

Original:  where I : ψ −→ R + is the information gained from the measurements obtained along the trajectoryψ , C : ψ −→ R + maps a trajectory ψ to its associated execution cost, and B ∈ R + is the givenpath-length budget.
Revised: (1) where I : ψ −→ R + is the information gained from the measurements obtained along the trajectory ψ , C : ψ −→ R + maps a trajectory ψ to its associated execution cost, and B ∈ R + is the given path-length budget.

Original: Following [5, 6, 7], the trajectory ψ is given a start and a destination but wenote that our method can be easily extended to remove the need for a destination.
Revised: Following [5, 6, 7], the trajectory ψ is given a start and a destination but we note that our method can be easily extended to remove the need for a destination.

Original: To evaluate theinformation gained from measurements, following [4, 1], we use the variance reduction of the GP torepresent information gain: I( ψ ) =
Revised: To evaluate the information gained from measurements, following [1, 4], we use the variance reduction of the GP to represent information gain: I( ψ ) =

Original: Tr( P − ) − Tr( P + ) , where Tr( · ) denotes the trace of a matrix,
Revised: Tr( P − ) − Tr( P + ) , where Tr( · ) denotes the trace of a matrix,

Original: P − and P + are the prior and posterior covariances, which are obtained before and after takingmeasurements along the trajectory ψ .
Revised: P − and P + are the prior and posterior covariances, which are obtained before and after taking measurements along the trajectory ψ .

Original: In this work, to model the data collection of common sensors,we let the agent take a measurement every time it has traveled a fixed distance from the previousmeasurement, thus the number of measurement is only determined by the path length budget B .
Revised: In this work, to model the data collection of common sensors, we let the agent take a measurement every time it has traveled a ﬁxed distance from the previous measurement, thus the number of measurement is only determined by the path length budget B .

Original: Adaptive Replanning
Revised: Adaptive Replanning

Original: If the information gain only depends on  P , i.e., the location of measurements, the objective is considered non-adaptive since the trajectory could be entirely planned offlineahead of time, based on the agent’s initial (often uniform) belief.
Revised: If the information gain only depends on the covariance P , i.e., the location of measurements, the objective is considered non-adaptive since the trajectory could be entirely planned ofﬂine ahead of time, based on the agent’s initial (often uniform) belief.

Original: However, in real-world applications such as search-and-rescue, we usually aim to discover regions of high interest and furthercover (exploit) them.
Revised: However, in realworld applications such as search-and-rescue, we usually aim to discover regions of high interest and further cover (exploit) them.

Original: To this end, following [7, 1], we rely on the upper confidence bound to definehigh-interest areas
Revised: To this end, following [1, 7], we rely on the upper conﬁdence bound to deﬁne high-interest areas

Original: X I : X I = { x i ∈
Revised: X I : X I = { x i ∈

Original: X ∗ | µ − i + βP − i,i ≥ µ th } , where µ − i and P − i,i are the meanand variance of the GP at the measurement location x i .
Revised: X ∗ | µ − i + βP − i,i ≥ µ th } , where µ − i and P − i,i are the prior mean and variance of the GP at the measurement location x i .

Original: µ th , while β ∈ R + is used to control thethreshold and confidence interval respectively ( µ th = 0 . 4 , β = 1 , in practice).
Revised: µ th and β ∈ R + are used to control the threshold and conﬁdence interval respectively ( µ th = 0 . 4 , β = 1 , in practice).

Original: By replacing X ∗ with X I in the covariance calculation, we restrict the information gain in the objective function Eq.
Revised: By replacing X ∗ with X I in the covariance calculation, we restrict the information gain in the objective function Eq.

Original: (1)to the high-value areas predicted by the GP.
Revised: (1) to the high-value areas predicted by the GP.

Original: This formulation makes the IPP objective dependenton the measurement values in addition to their location, making the problem truly adaptive .
Revised: This formulation makes the IPP objective dependent on the measurement values in addition to their location, making the problem truly adaptive .

Original: Therefore, frequent online replanning of the trajectory is required to minimize uncertainty in the (nowdynamically defined) high-interest areas X I .
Revised: Therefore, frequent online replanning of the trajectory is required to minimize uncertainty in the (now dynamically deﬁned) high-interest areas X I .

Original:  Method
Revised:  Method

Original: In this section, we cast adaptive IPP as an RL problem and detail our attention-based neural network,as well as our long-horizon planning strategy to further boost the performance of a learned policy.
Revised: In this section, we cast adaptive IPP as an RL problem and detail our attention-based neural network, as well as our long-horizon planning strategy to further boost the performance of a learned policy.

Original: 4.1 IPP as a RL Problem
Revised: 4.1 IPP as a RL Problem

Original: Sequential Decision-making Problem First, to avoid the complexity associated with a continuoussearch domain, we rely on probabilistic roadmaps (PRM)
Revised: Sequential Decision-making Problem First, to avoid the complexity associated with a continuous search domain, we rely on probabilistic roadmaps (PRM)

Original: [11] to build a route graph G =
Revised: [11] to build a route graph G =

Original: ( V, E ) , with V a set of uniformly-random-sampled nodes over the domain, and E a set of edges.
Revised: ( V, E ) , with V a set of uniformly-random-sampled nodes over the domain, and E a set of edges.

Original: Each nodev i =
Revised: Each node v i =

Original: ( x i , y i ) ∈
Revised: ( x i , y i ) ∈

Original: V is connected to its k nearest neighboring nodes and v 0 is the destination.
Revised: V is connected to its k nearest neighboring nodes and v 0 is the destination.

Original: Then,to solve the adaptive IPP using RL, we formulate it as a sequential decision-making problem on thisgraph.
Revised: Then, to solve the adaptive IPP using RL, we formulate it as a sequential decision-making problem on this graph.

Original: That is, we let agent interact with the environment by choosing which node to move to fromamongst the neighbors of its current node.
Revised: That is, we let agent interact with the environment by choosing which node to move to from amongst the neighbors of its current node.

Original: Movement between nodes happens as a straight line.
Revised: Movement between nodes happens as a straight line.

Original: As aresult, the agent’s trajectory ψ can be represented as an ordered set ( ψ s , ψ 1 , ..., ψ d ) , ∀ ψ i ∈ V , whereψ s and ψ d denote the start and destination nodes respectively.
Revised: As a result, the agent’s trajectory ψ can be represented as an ordered set ( ψ s , ψ 1 , ..., ψ d ) , ∀ ψ i ∈ V , where ψ s and ψ d denote the start and destination nodes respectively.

Original: As a result, the trajectory is adaptivelyplanned, since it is constructed from sequential movements, each depending on the agent’s globalbelief, which gets updated online based on new measurements.
Revised: As a result, the trajectory is adaptively planned, since it is constructed from sequential movements, each depending on the agent’s global belief, which gets updated online based on new measurements.

Original: Observation The observation s t = { G ′ , v c , B c , ψ s,c , M } of our IPP agent consists of three parts:the augmented graph, the planning state, and the budget mask.
Revised: Observation The observation s t = { G (cid:48) , v c , B c , ψ s,c , M } of our IPP agent consists of three parts: the augmented graph, the planning state, and the budget mask.

Original: The augmented graph G ′ =
Revised: The augmented graph G (cid:48) =

Original: ( V ′ , E ) is used to describe the environment modeled by the GP.
Revised: ( V (cid:48) , E ) is used to describe the environment modeled by the GP.

Original: Itis a combination of the route graph G =
Revised: It is a combination of the route graph G =

Original: ( V, E ) and the GP GP ( µ, P ) , where each node v ′
Revised: ( V, E ) and the GP GP ( µ, P ) , where each node v (cid:48)

Original: i =( v i , µ ( v i ) , P ( v i ))
Revised: i = ( v i , µ ( v i ) , P ( v i ))

Original: ∈ V ′ .
Revised: ∈ V (cid:48) .

Original: This augmented graph stores the information about the agent’s globalbelief and determines the agent’s local action space.
Revised: This augmented graph stores the information about the agent’s global belief and determines the agent’s local action space.

Original: The planning state is defined by { v c , B c , ψ s,c } ,where v c ∈ V is the current position of the agent, B c = B − C( ψ s,c )  the remaining budget,and ψ s,c = ( ψ s , ψ 1 , ..., v c )  the executed trajectory so far.
Revised: The planning state is deﬁned by { v c , B c , ψ s,c } , where v c ∈ V is the current position of the agent, B c = B − C( ψ s,c ) is the remaining budget, and ψ s,c = ( ψ s , ψ 1 , ..., v c ) is the executed trajectory so far.

Original: The budget mask M is a binary vectorcontaining one element for each node in the route graph, stating whether selecting this node at thecurrent step would result in violating the budget constraint.
Revised: The budget mask M is a binary vector containing one element for each node in the route graph, stating whether selecting this node at the current step would result in violating the budget constraint.

Original: To obtain this mask, we pre-solve theshortest path problem using Dijkstra [14] to compute the minimal cost d i to the destination from eachnode v i .
Revised: To obtain this mask, we pre-solve the shortest path problem using Dijkstra [14] to compute the minimal cost to the current node and to the destination from each node v i .

Original: We then compute a virtual budget B ∗ i
Revised: We then compute a virtual budget B ∗ i

Original: = B c −  C( v c , v i ) − d i  for each node based on theplanning state.
Revised: = B c − min C( v c , v i ) − min C( v i , ψ d ) for each node based on the planning state.

Original: Finally, according to B ∗ , we compute each entry of M as M i = (cid:26) 1 if B ∗ i <0 otherwise .
Revised: Finally, according to B ∗ , we compute each entry of M as M i = (cid:26) 1 if B ∗ i <0 otherwise .

Revised: That is, actions (i.e., neighboring nodes) that would inevitably result in budget overruns are speciﬁcally ﬁltered out by the budget mask M i , which iteratively guarantees the completeness of CAtNIPP by forcing the agent towards the destination via the shortest possible path when the budget is about to run out.

Original: Action Each time the agent reaches a node, the GP is updated based on all new measurements,and the agent immediately selects its next action.
Revised: Action Each time the agent reaches a node, the GP is updated based on all measurements obtained so far, and the agent immediately selects its next action.

Original: Specifically, at each such decision step t , giventhe agent’s observation, our attention-based neural network outputs a stochastic policy to select thenext node to visit out of all neighboring nodes.
Revised: Speciﬁcally, at each such decision step t , given the agent’s observation, our attention-based neural network outputs a stochastic policy to select the next node to visit out of all neighboring nodes.

Original: The policy is parameterized by the set of weights θ :π θ ( ψ t = v i , ( v c , v i ) ∈
Revised: The policy is parameterized by the set of weights θ : π θ ( ψ t = v i , ( v c , v i ) ∈

Original: E | s t ) , where E is the edge set of the underlying graph.
Revised: E | s t ) , where E is the edge set of the underlying graph.

Original: Reward At each decision step, to maximize information gain, the agent is given a positive rewardbased on the reduction in uncertainty associated with its most recent action: r t =
Revised: At each decision step, to maximize information gain, the agent is given a positive reward based on the reduction in uncertainty associated with its most recent action: r t =

Original: Tr( P t − 1 ) −
Revised: Tr( P t − 1 ) −

Original: Tr( P t )) /
Revised: Tr( P t )) /

Original: Tr( P t − 1 ) , where we experimentally found that scaling the reward by Tr( P t − 1 ) helpedstabilize training by keeping the rewards consistent in magnitude.
Revised: Tr( P t − 1 ) , where we experimentally found that scaling the reward by Tr( P t − 1 ) helped stabilize training by keeping the rewards consistent in magnitude.

Original: However, this normalizationintroduces a deviation between the training objective and the IPP objective.
Revised: However, this normalization introduces a deviation between the training objective and the IPP objective.

Original: Therefore, at the lastdecision step of each episode, we introduce a negative correction reward r d = − α ·
Revised: Therefore, at the last decision step of each episode, we introduce a negative correction reward r d = − α ·

Original: Tr( P d ) , where
Revised: Tr( P d ) , where

Original: P d is the covariance after executing the whole trajectory ψ , and α is a scaling factor ( 1 in practice).
Revised: P d is the covariance after executing the whole trajectory ψ , and α is a scaling factor ( 1 in practice).

Revised: We empirically observed that this correction reward helps mitigate the bias introduced by the (dynamic) normalization used in our dense rewards, towards the true IPP reward.

Original: 4.2 Neural Network Structure
Revised: 4.2 Neural Network Structure

Original: The proposed attention-based neural network consists of an encoder and a decoder modules (see
Revised: The proposed attention-based neural network consists of an encoder and a decoder modules (see

Original: Fig. 2).
Revised: Fig. 2).

Original: We use the encoder to model the observed environment by learning the dependencies between nodes in the augmented graph G ′ , i.e., the context .
Revised: We use the encoder to model the observed environment by learning the dependencies between nodes in the augmented graph G (cid:48) , i.e., the context .

Original: Based on the features extracted by theencoder, the planning state { v c , B c , ψ s,c } , and the budget mask M , the decoder then outputs thepolicy over which neighboring node to visit next.
Revised: Based on the features extracted by the encoder, the planning state { v c , B c , ψ s,c } , and the budget mask M , the decoder then outputs the policy over which neighboring node to visit next.

Original: To handle graphs with arbitrary topologies, ourencoder uses a standard Transformer attention layer with graph Positional Encoding (PE) based onthe graph Laplacian’s eigenvector [15], thus providing the neural network with the ability to reasonabout node connectivity.
Revised: To handle graphs with arbitrary topologies, our encoder uses a standard Transformer attention layer with graph Positional Encoding (PE) based on the graph Laplacian’s eigenvector [15], thus providing the neural network with the ability to reason about node connectivity.

Original: While general policy-based RL agents have a fixed action space, our decoder is inspired by the Pointer Network [16] to allow the dimension of the final policy to dependon the number of neighboring nodes, allowing our network to generalize to arbitrary graphs.
Revised: While general policy-based RL agents have a ﬁxed action space, our decoder is inspired by the Pointer Network [16] to allow the dimension of the ﬁnal policy to depend on the number of neighboring nodes, allowing our network to generalize to arbitrary graphs.

Original: Attention Layer The Transformer attention layer [17] is used as the fundamental building blockin our model.
Revised: Attention Layer The Transformer attention layer [17] is used as the fundamental building block in our model.

Original: The input of such an attention layer consists of the query source h q and the keyand-value source h k,v .
Revised: The input of such an attention layer consists of the query source h q and the keyand-value source h k,v .

Original: The attention layer updates the query source using the weighted sum ofthe value vector, where the attention weight depends on the similarity between query and key.
Revised: The attention layer updates the query source using the weighted sum of the value vector, where the attention weight depends on the similarity between key and query.

Original: We[Figure] compute the updated feature h ′ i as: q i = W Q h qi , k i =
Revised: We compute the updated feature h (cid:48) i as: q i = W Q h qi , k i =

Original: W k h k,vi , v i 
Revised: W K h k,vi , v i =

Original: = W v h k,vi , u ij = qTi · k j √ d , a ij =
Revised: W V h k,vi , u ij = q

Original:  e uij (cid:80) nj =1 e uij , h ′ i =
Revised: Ti · k j √ d , a ij = e uij (cid:80) nj =1 e uij , h (cid:48)

Original:  (cid:80) nj =1 a ij v j , where W Q , W K , W V areall learnable matrices with size d × d .
Revised: i = (cid:80) nj =1 a ij v j , where W Q , W K , W V are d × d learnable matrices.

Original: Theupdated features are then passed through the contains linear layers  ReLU activation as in [
Revised: The updated fea- [Figure] tures are then passed through the feed forward sublayer, composed of two linear layers and a ReLU activation.

Revised: As in [17], we use layer normalization and residual connections in these two sublayers.

Original: Encoder The encoder is used to model the observed environment by learning dependencies between nodes in the augmented graph G ′ .
Revised: Encoder The encoder is used to model the observed environment by learning dependencies between nodes in the augmented graph G (cid:48) .

Original: We first embed the node inputs V′into d -dimensional node features h ni and add Laplacian positional embeddings: h ni
Revised: We ﬁrst embed the node inputs V

Original: [Equation]
Revised: [Figure]

Original: 192 (cid:26) [Equation] i [Equation]
Revised: into d -dimensional node features h ni and add Laplacian positional embeddings:

Original: [Equation]
Revised: [Equation]

Original: PE i = [-
-] where λ i is the pre-computed k -dimensional Laplacianeigenvector, and W L ,
Revised: i = [-
-]0 where λ i is the pre-computed k -dimensional Laplacian eigenvector, and W L ,

Original: W D ∈ R
Revised: W D ∈ R

Original: d × 4 ,
Revised: d × 4 ,

Original: W PE ∈ R
Revised: W PE ∈ R

Original: d
Revised: d

Original: × k , b L , b D , b PE ∈ R d are learnable parameters.
Revised: × k , b L , b D , b PE ∈ R d are learnable parameters.

Original: Note that the destination node V ′ 0 is embedded by another linear layer.
Revised: Note that the destination node v (cid:48) 0 is embedded by another linear layer.

Original: The node features are thenpassed to an attention layer, where h q = h k,v = h n , as is commonly done in self-attention mechanisms.
Revised: The node features are then passed to an attention layer, where h q = h k,v = h n , as is commonly done in self-attention mechanisms.

Original: We term the output of the encoder, h en , the context-aware node features , since each of theseupdated node features h eni contains the dependencies of v 
Revised: We term the output of the encoder, h en , the context-aware node features , since each of these updated node features h eni contains the dependencies of v (cid:48)

Original: i with all other nodes.
Revised: i with all other nodes.

Original: Decoder The decoder is used to output a policy based on the context-aware node features, theplanning state { v c , B c , ψ s,c } , and the budget mask M .
Revised: Decoder The decoder is used to output a policy based on the context-aware node features, the planning state { v c , B c , ψ s,c } , and the budget mask M .

Original: We first merge the information about thebudget and the high-interest areas to the context-aware node features: ˆ h eni =
Revised: We ﬁrst merge the information about the budget and the high-interest areas to the context-aware node features: ˆ h eni =

Original: W B [ h eni , B ∗ i , µ th ] +b B , where W B ∈ R d × ( d +2) and
Revised: W B [ h eni , B ∗ i , µ th ] + b B , where W B ∈ R d × ( d +2) and

Original: b B ∈ R
Revised: b B ∈ R

Original: d are learnable parameters.
Revised: d are learnable parameters.

Original: Then, according to the currentposition v c and the edge set E , we select the current node feature h c , the neighboring featuresh n from ˆ h en , and the neighbor mask M n from M .
Revised: Then, according to the current position v c and the edge set E , we select the current node feature h enc , the neighboring features h enn from ˆ h en , and the neighbor mask M n from M .

Original: After that, the current node feature h c are passedto a LSTM block, where the hidden state and cell state are input from previous current node featurealong the executed trajectory ψ s,c .
Revised: After that, the current node feature h enc are passed to a LSTM block, where the hidden state and cell state are input from previous current node feature along the executed trajectory ψ s,c .

Original: The LSTM output ˆ h c is merged with the destination featureˆ h en 0 to compute the enhanced current node feature h ec : h ec =
Revised: The LSTM output ˆ h enc is merged with the destination feature ˆ h en 0 to compute the enhanced current node feature h ec : h ec =

Original: W C
Revised: W C

Original: [ˆ h c , ˆ h en 0 ] + b C , where W C ∈ 
Revised: [ˆ h enc , ˆ h en 0 ] + b C , where W C ∈ R

Original: R d × 2 d and b C ∈ R
Revised: d × 2 d and b C ∈ R

Original: d are learnable parameters.
Revised: d are learnable parameters.

Original: We feed the enhanced node current feature andthe neighboring features to an attention layer, where h q = h ec and h k,v = h n .
Revised: We feed the enhanced node current feature and the neighboring features to an attention layer, where h q = h ec and h k,v = h n .

Original: We denote theoutput of this attention layer ˆ h ec , which is simultaneously passed to a linear layer to output the statevalue V ( s t ) , and to the final attention layer with the neighboring features, where h q = ˆ h ec andh k,v = h n .
Revised: We denote the output of this attention layer ˆ h ec , which is simultaneously passed to a linear layer to output the state value V ( s t ) , and to the ﬁnal attention layer with the neighboring features, where h q = ˆ h ec and h k,v = h n .

Original: For this final attention layer, we directly treat the attention weights a i as the final policyu i for the 
Revised: For this ﬁnal attention layer, we directly treat the attention weights a i as the ﬁnal policy u i for the IPP agent, where invalid nodes are explicitly masked using M n . The masked u i is ﬁnally normalized to yield the probability distribution π for the next node to visit: π i = π θ ( ψ t = v i | s t ) = e u i / (cid:80) ni

Revised: =1 e u i .

Original: 4.3 Training
Revised: 4.3 Training

Original: Our model is trained using PPO [18].
Revised: Our model is trained using PPO [18].

Original: At the beginning of each training episode, we average 8 to 12random 2-dimensional Gaussian distributions in the unit square [0 , 1] 2 , to construct the true interestmap.
Revised: At the beginning of each training episode, we averageto 12 random 2-dimensional Gaussian distributions in the unit square [0 , 1] 2 , to construct the true interest map.

Original: The robot’s belief starts as a uniform distribution GP (0 , 1) .{+P 0 ) , P 
+}
Revised: The robot’s belief starts as a uniform distribution GP (0 , {+P 0 ) , P 
+}

Original:  The start and destination positionsare randomly generated in [0 , 1] 2 .
Revised: i,i = 1 . The start and destination positions are randomly generated in [0 , 1] 2 .

Revised: During training, the number of nodes for our

Original: During training, the number of nodes for our graph is randomizedwithin [200 , 400] for each episode, the number of neighboring nodes is fixed to k = 20 , and thebudget is randomized within [6 , 8] .
Revised: [Table] graph is randomized within [200 , 400] for each episode, the number of neighboring nodes is ﬁxed to k = [Table] and the budget is randomized within [6 , 8] .

Original: A measurement is obtained every time the agent has traveled0 .
Revised: A measurement is obtained every time the agent has traveled 0 .

Original: 2 from the previous measurement.
Revised: 2 from the previous measurement.

Original: We set the max episode length to 256 time steps, and the batchsize to 1024 .
Revised: We set the max episode length to 256 time steps, and the batch size to 1024 .

Original: We use the Adam optimizer with learning rate 10 − 4 , which decays every 32 stepsby a factor of 0 .
Revised: We use the Adam optimizer with learning rate 10 − 4 , which decays every 32 steps by a factor of 0 .

Original: For each training episode, PPO runs 8 iterations.
Revised: For each training episode, PPO runs 8 iterations.

Original: Our model is trained on aworkstation equipped with a i9-10980XE CPU and four NVIDIAGeForce RTX 3090 GPUs.
Revised: Our model is trained on a workstation equipped with a i9-10980XE CPU and four NVIDIA RTX 3090 GPUs.

Original: Wetrain our model utilizing Ray, a distributed framework for machine learning [19].
Revised: We train our model utilizing Ray, a distributed framework for machine learning [19].

Original: We run 32 IPPinstances in parallel to accelerate the data collection and training, and need around 24h to converge.
Revised: We run 32 IPP instances in parallel to accelerate the data collection and training, and need around 24h to converge.

Original: 4.4 Trajectory Sampling
Revised: 4.4 Trajectory Sampling

Original: Until now, we discussed solving the IPP in the standard RL manner, i.e., iteratively selecting thenext node to visit each time the agent reaches a node.
Revised: Until now, we discussed solving the IPP in the standard RL manner, i.e., iteratively selecting the next node to visit each time the agent reaches a node.

Original: Inspired by conventional non-learning IPPsolvers, we further propose a receding-horizon strategy for our RL agent, where an m -step trajectoryis output at each (re)planning step but only a portion of it is executed before the next replanning step(see Fig. 1).
Revised: Inspired by conventional non-learning IPP solvers, we further propose a receding-horizon strategy for our RL agent, where an m -step trajectory is output at each (re)planning step but only a portion of it is executed before the next replanning step (see Fig. 1).

Original: We utilize the learned policy for further optimization by sampling , which has beenshown to be a reliable optimization strategy for learning-based routing planner [20, 21].
Revised: We utilize the learned policy for further optimization by sampling , which has been shown to be a reliable optimization strategy for learning-based routing planner [20, 21].

Original: That is, ateach planning step, based on the learned policy, our trajectory sampling method parallely plans anumber s of m -step trajectories, and then selects the trajectory that maximizes the information gainas the final trajectory ψ ∗ .
Revised: That is, at each planning step, based on the learned policy, our trajectory sampling method parallely plans a number s of m -step trajectories, and then selects the trajectory that maximizes the information gain as the ﬁnal trajectory ψ ∗ .

Original: During the m -step planning process, only the covariance of the GP can bepredicted, since no measurement is actually taken before executing the trajectory.
Revised: During the m -step planning process, only the covariance of the GP can be predicted, since no measurement is actually taken before executing the trajectory.

Original:  Experiments
Revised:  Experiments

Original: In this section, we compare CAtNIPP with state-of-the-art (SOTA) baselines IPP solvers on a fixedset of randomly generated environments with identical randomized conditions.
Revised: In this section, we compare CAtNIPP with state-of-the-art (SOTA) baselines IPP solvers on a ﬁxed set of randomly generated environments with identical randomized conditions.

Original: We also presentnumerical and experimental validation of CAtNIPP on an light-intensity-based adaptive IPP task.
Revised: We also present numerical and experimental validation of CAtNIPP on an light-intensity-based adaptive IPP task.

Original: Inour supplemental material, we also tested a number of variants of our model and its generalizability.
Revised: In our supplemental material, we also tested a number of variants of our model and its generalizability.

Original: 5.1 Comparison Results
Revised: 5.1 Comparison Results

Original: We compare CAtNIPP against a number of state-of-the-art IPP solvers: (a) CMA-ES [7] (we uselinear B-spline for the CMA-ES solver for a fair comparison), (b) RAOr [5], and (c) RIG-tree [9].
Revised: We compare CAtNIPP against a number of state-of-the-art IPP solvers: (a) CMA-ES [7] (we use linear B-spline for the CMA-ES solver for a fair comparison), (b) RAOr [5], and (c) RIG-tree [9].

Original: Following [1], we implement RAOr and RIG-tree in a receding-horizon manner to make them adaptive.
Revised: Following [1], we implement RAOr and RIG-tree in a receding-horizon manner to make them adaptive.

Original: All considered solvers (except our fully reactive, greedy variants) replan paths after executing0 .
Revised: All considered solvers (except our fully reactive, greedy variants) replan paths after executing 0 .

Original: 4 of their previously planned trajectory.
Revised: 4 of their previously planned trajectory.

Original: Starting from hyperparameters suggested by their originalpapers, we tuned these solvers to output highest-quality solutions, while keeping the total planningtime similar to our trajectory sampling variants.
Revised: Starting from hyperparameters suggested by their original papers, we tuned these solvers to output highest-quality solutions, while keeping the total planning time similar to our trajectory sampling variants.

Original: This enables us to offer a fair comparison betweenour methods and these baselines.
Revised: This enables us to offer a fair comparison between our methods and these baselines.

Original: CAtNIPP’s greedy and trajectory sampling variant are denotedby g. ( n ) and ts.
Revised: CAtNIPP’s greedy and trajectory sampling variant are denoted by g. ( n ) and ts.

Original: ( m ) respectively, where n is the number of nodes for the route graph and m is thenumber of trajectories sampled at each planning step ( n is fixed to 400 for ts.).
Revised: ( m ) respectively, where n is the number of nodes for the route graph and m is the number of trajectories sampled at each planning step ( n is ﬁxed to 400 for ts.).

Original: Greedy variantswork in the standard RL manner, i.e., the agent always selects the action with highest activation inits policy.
Revised: Greedy variants work in the standard RL manner, i.e., the agent always selects the action with highest activation in its policy.

Original: Trajectory sampling variants plan a 15 -step trajectory and execute the first 3 steps beforereplanning (in practice, ∼ 0 .
Revised: Trajectory sampling variants plan a 15 -step trajectory and execute the ﬁrst 3 steps before replanning (in practice, ∼ 0 .

Original: 4 of traveled distance), following the receding-horizon setup in [7].
Revised: 4 of traveled distance), following the receding-horizon setup in [7].

Original: Note that trajectory sampling is accelerated by multithreading using up to 8 threads.
Revised: [Figure]

Original: We report the uncertainty remaining (covariance matrix trace
Revised: We report the uncertainty remaining (covariance matrix trace

Original: Tr( P )  , lower is better) after finishing the mission, as well as the total planning time in Table 1.
Revised: Tr( P ) in the high-interest areas X I , lower is better) after ﬁnishing the mission, as well as the total planning time in Table 1.

Original: The evolution of thesetwo metrics during path planning, uncertaintyremaining, and root mean square error (RMSE) [Table] [Figure] [Table]
Revised: The evolution of the remaining uncertainty and root mean square error compared to the ground truth || µ − ζ

Original: [Figure]

Original: [Table]

Original: [Figure] [Table] [Figure]
Revised: || 2 are plotted in Fig.

Original: for a fixed budget of 10 (10 trials on 30 instances).

Original: Our two CAtNIPP variants significantly outperform all solvers in reducing uncertainty

Original: ) as well as root mean square error (

Original: ).|| µ − ζ || 2 compared to the ground truth are plotted in Fig.

Original: Upon closer inspection, we note that RAOr tends to plan trajectories that exploit nearby high-interest areas, thus reducing the uncertaintyslowly in the early stageof a mission and more rapidlyin later stages.
Revised: Upon closer inspection, we note that RAOr tends to plan trajectories that exploit nearby high-interest areas, thus reducing the uncertainty slowly in the early stage and more rapidly later on.

Original: RIG-tree, on the other hand,has a strong tendency for exploration, which leads to a fast uncertainty reduction earlier on, buta slower one in the later stages of an episode.
Revised: RIG-tree, on the other hand, has a strong tendency for exploration, which leads to a fast uncertainty reduction earlier on, but a slower one in the later stages of an episode.

Original: As a meta-heuristic solver, CMA-ES finds a goodtrade-off between exploration and exploitation, resulting in the best overall performance amongnon-learning solvers (even better than CAtNIPP with budget 6 ).
Revised: As a metaheuristic solver, CMA-ES ﬁnds a good trade-off between exploration and exploitation, resulting in the best overall performance among non-learning solvers (even better than CAtNIPP with budget 6 ).

Original: Fig.
Revised: Fig.

Original: 3 shows that CAtNIPP maintains best overall performance with respect to all metrics throughoutthe whole budget span.
Revised: 3 shows that CAtNIPP maintains best overall performance with respect to all metrics throughout the whole budget span.

Revised: We performed paired t-tests between the covariance reduction results of our variants and each of the baselines (6 tests in total), which all yielded p-values lower than 1 . 06 · 10 − 4 .

Revised: Using a Bonferroni correction for these multiple tests, we ﬁnd the ﬁnal signiﬁcance threshold p = 1 .

Original: Regarding the CAtNIPP variants, we note that trajectory sampling variantsexhibit improved solution quality over  greedy variants ( 8% better in average).
Revised: 67 · 10 − 3 (for a standard, original p = 0 . 01 threshold), indicating that CAtNIPP (most likely) signiﬁcantly outperforms all other baselines in terms of covariance reduction. We further note that our ts. variants exhibit improved solution quality over our greedy variants ( 8% better in average) by further reﬁning these solutions, e.g., in some extreme cases where greedy variants do particularly poorly.

Original: However, greedyvariants plan up to  100 x faster than trajectory sampling variants and  SOTA IPP solvers.
Revised: However, greedy variants plan up to around 100 × faster than ts. variants and other SOTA IPP solvers, making it the most time-efﬁcient CAtNIPP version.

Original: 5.2 Attention Visualization: Learning to Be Context-aware
Revised: 5.2 Generalization Results

Original: [Table]
Revised: In our supplemental material, we include detailed testings of CAtNIPP’s generalization capabilities. We demonstrate that our model, trained on randomly generated ground truths, can generalize to completely different environments never seen during training, such as different Gaussian mixture models or even handcrafted, non-Gaussian distributions, while still outperforming baselines. Our results suggest that CAtNIPP endows the agent with a general strategy for IPP, rather than overﬁtting to a particular class of ground truths. 5.3 Attention Visualization: Learning to Be Context-aware

Original: We believe that the superior solutionquality of CAtNIPP mainly comes fromits ability to be context-aware , and thusto avoid the type of short-sightednessusually associated with local, reactive IPP planners.
Revised: We believe that the superior solution quality of CAtNIPP mainly comes from its ability to be contextaware , and thus to avoid the type of short-sightedness usually associated with local, reactive IPP planners.

Original: We investigated the learned attention mechanism at the core of CAtNIPP by visualizing learned attention weights (larger dots means higher weight) at representative time steps.
Revised: We investigated the learned attention mechanism at the core of CAtNIPP by visualizing learned attention weights (larger dots means higher weight) at representative time steps.

Original: Inparticular, Fig.
Revised: In particular, Fig.

Original: 4 shows that the left attention head and the right three attention heads of the encoder have learned to focus on the longer- and shorter-term high-interest regions respectively.
Revised: 4 shows that the left attention head and the right three attention heads of the encoder have learned to focus on the longer- and shorter-term high-interest regions respectively.

Original: Giventhese context-aware node features , the agent finally learns to make local decisions that can optimizeobjectives at the different scales identified by the encoder.
Revised: Given these context-aware node features, the agent ﬁnally learns to make local decisions that can optimize objectives at the different scales identiﬁed by the encoder.

Original: Our ablation results (Table 2) furtherconfirm that the presence of the encoder is critical: without it, performance is drastically degraded,as the agent mostly sequences locally greedy decisions into suboptimal search paths. Nevertheless,we note that relying on receding-horizon optimization, trajectory sampling drastically improves thesolution quality of the model without encoder.
Revised: Our ablation results in the supplemental material further conﬁrm that the presence of the encoder is critical.

Original: 5.3 Numerical and Experimental Validation
Revised: 5.4 Numerical and Experimental Validation

Original: We carried out experiments to validate CAtNIPP’s performance on a TurtleBot3 robot, over a printed[Figure] grayscale image of 2 .
Revised: We carried out experiments to validate CAtNIPP’s performance on a TurtleBot3 robot, over a printed grayscale image of 2 .

Original: 38 × 2 .
Revised: 38 × 2 .

Original: 38 m 2representing the ground truth interest map (see Fig. 5).
Revised: 38m 2 representing the ground truth interest map (see Fig. 5).

Original: Therobot is equipped with an onboard camera used to measure theground light (grayscale) intensity(as an example of a simple onboard intensity-level sensor, e.g.,temperature/radioactivity/gas levels), while its position is obtainedby a downward-facing, overheadcamera.
Revised: The robot is equipped with an on-board camera used to measure the ground light (grayscale) intensity (as an example of a simple onboard intensity-level sensor, e.g., temperature/radioactivity/gas levels), while its position is obtained by a downward-facing, overhead camera. [Figure]

Original: In this experiment, a trained CAtNIPP model adaptively outputs the next node locationto visit based on the agent’s current belief, using a 400 -nodes graph, and the agent only takes measurements when reaching a node.
Revised: In this experiment, a trained CAtNIPP model adaptively outputs the next node location to visit based on the agent’s current belief, using a 400 -nodes graph, and the agent only takes measurements when reaching a node.

Original: This experiment confirms that CAtNIPP is easily deployable onrobot for online, reactive planning, and highlights its low computational cost ( ≤ 0 .
Revised: This experiment conﬁrms that CAtNIPP is easily deployable on robot for online, reactive planning, and highlights its low computational cost ( ≤ 0 .

Original: 1 s per decisionon CPU).
Revised: 1 s per decision on CPU).

Original: Our supplemental material includes simulation videos in environments of up to 8 × 8 m 2 .
Revised: Our supplemental material includes simulation videos in environments of up to 8 × 8m 2 .

Original:  Limitations
Revised:  Limitations

Original: We believe that the limitations of CAtNIPP lie at three different levels: generalizability, area discretization, and implementation readiness:
Revised: We believe that the limitations of CAtNIPP lie at three different levels: generalizability, area discretization, and implementation readiness:

Original: • As a model-free approach, CAtNIPP learns to implicitly comprehend the upper confidence bound used to define high-interest areas, as well as the  kernel function of the GP used to represent the agent’s sensor model.
Revised: • Our proposed method is sensitive to changes in the hyperparameters of the GP’s kernel function and the upper bound conﬁdence that deﬁnes high-interest areas.

Original: Therefore, the model requires retraining if any of these parameters change
Revised: Currently, CAtNIPP would require retraining if any of these parameters were changed signiﬁcantly.

Original: (e.g., change in sensor used, or in task specifications), thus limiting its generalizability.
Revised: In practice, the hyperparameters of the GP’s kernel are decided by key characteristics of the actual sensor equipped on the robot (e g., range or resolution of a camera).

Original: Future
Revised: Nevertheless, based on our experience, for a given type of real-world task (e.g., search and rescue, ﬁre detection, agriculture monitoring), the deﬁnition of “high-interest” is usually rather static.

Original: work will look at model-based RL approaches, which may allow the agent to explicitly reason about these mission parameters and adapt to new settings without the need for retraining.
Revised: Thus we would expect that, after an initial training based on these parameters, our learned model will likely not need any further training in practice, since retraining would only be needed if the task or onboard sensor are drastically changed.

Original: • We currently assume uniform sampling of the route graph, which may prevent the agent fromreaching an interesting area due to insufficient graph coverage.
Revised: • We currently assume uniform sampling of the route graph, which may prevent the agent from reaching an interesting area due to insufﬁcient graph coverage.

Original: However, CAtNIPP is already able to handle arbitrary graphs.
Revised: However, CAtNIPP is already able to handle arbitrary graphs.

Original: To address this issue, especially in later stages of the planning, we will further investigate online re-sampling of graph nodes, e.g., according to the current belief.
Revised: To address this issue, especially in later stages of the planning, we will further investigate online re-sampling of graph nodes, e.g., according to the current belief.

Original: • We currently plan paths in a simplified graph, where paths between nodes are straight lines, ignor- ing most real-life robot motion constraints (i
Revised: • We currently plan paths in a simpliﬁed graph, where paths between nodes are straight lines, ignoring most real-life robot motion constraints (i

Original: e., holonomic robot assumption).
Revised: e., holonomic robot assumption).

Original: Future work will explicitly consider the robot’s motion model, e
Revised: Future work will explicitly consider the robot’s motion model, e

Original: g., in the state representation (velocity, heading,kinematic/dynamics constraints), to better trade-off robot-specificity with ease of implementation.
Revised: g., in the state representation (velocity, heading, kinematic/dynamics constraints), to better trade-off robot-speciﬁcity with ease of implementation.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we introduce CAtNIPP, a policy-gradient-based dRL method for adaptive IPP that relies on self-attention to endow the agent with the ability to sequence local decisions, informed by itsglobal context over the search domain to avoid short-sightenedness.
Revised: In this paper, we introduce CAtNIPP, a policy-gradient-based dRL method for adaptive IPP that relies on self-attention to endow the agent with the ability to sequence local decisions, informed by its global context over the search domain to avoid short-sightenedness.

Original: In addition to solving adaptive
Revised: In addition to solving adaptive

Original: IPP by simply greedily exploiting our learned policy, which can be done at very low computationalcost ( ∼ 0 .
Revised: IPP by simply greedily exploiting our learned policy, which can be done at very low computational cost ( ∼ 0 .

Original: 1 s per decision), we propose a sampling-based strategy that utilizes the learned policymore efficiently to output higher-quality solutions, while keeping the computing time on par withexisting IPP solvers.
Revised: 1 s per decision once trained), we propose a sampling-based strategy that utilizes the learned policy more efﬁciently to output higher-quality solutions, while keeping the computing time on par with existing IPP solvers.

Original: We experimentally demonstrate that both variants of CAtNIPP significantlyoutperform state-of-the-art IPP solvers in terms of solution quality.
Revised: We experimentally demonstrate that both variants of CAtNIPP signiﬁcantly outperform state-of-the-art IPP solvers in terms of solution quality and planning time, with strong generalization to classes of distributions never seen during training.

Original: Finally, we present experimental results on physical and simulated robots in a representative online, adaptive IPP task, showingpromises for robotic deployments in real-life monitoring, inspection, or mapping scenarios.
Revised: Finally, we present experimental results on physical and simulated robots in a representative online, adaptive IPP task, showing promises for robotic deployments in real-life monitoring, inspection, or mapping scenarios.

Original: Future work will mainly focus on extending our model to multi-agent IPP, where robots need to reason about each other to cooperatively plan informative paths, by leveraging synergies and avoidingredundant work.
Revised: Future work will mainly focus on extending our model to multi-agent IPP, where robots need to reason about each other to cooperatively plan informative paths, by leveraging synergies and avoiding redundant work.

Original: We also plan to investigate the use of CAtNIPP for robot exploration tasks, wheremore real-world object such as obstacles and sensors need to be considered in the planning process.
Revised: We also plan to investigate the use of CAtNIPP for robot exploration tasks, where more real-world object such as obstacles and sensors need to be Acknowledgments

Revised: This work was supported by Temasek Laboratories (TL@NUS) under grant TL/SRP/21/19.