BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation
Authors: Zibo Zhou
Yue Hu
Lingkai Zhang
Zonglin Li
Siheng Chen
Venue: NeurIPS 2025 poster ()
Keywords: zero shot object Navigation
belief map
mobile agent
Primary area: applications

Abstract: Zero-shot object navigation (ZSON) allows robots to find target objects in unfamiliar environments using natural language instructions, without relying on pre-built maps or task-specific training. Recent general-purpose models, such as large language models (LLMs) and vision-language models (VLMs), equip agents with semantic reasoning abilities to estimate target object locations in a zero-shot manner. However, these models often greedily select the next goal without maintaining a global understanding of the environment and are fundamentally limited in the spatial reasoning necessary for effective navigation. To overcome these limitations, we propose a novel 3D voxel-based belief map that estimates the target’s prior presence distribution within a voxelized 3D space. This approach enables agents to integrate semantic priors from LLMs and visual embeddings with hierarchical spatial structure, alongside real-time observations, to build a comprehensive 3D global posterior belief of the target’s location. Building on this 3D voxel map, we introduce BeliefMapNav, an efficient navigation system with two key advantages: i) grounding LLM semantic reasoning within the 3D hierarchical semantics voxel space for precise target position estimation, and ii) integrating sequential path planning to enable efficient global navigation decisions. Experiments on HM3D and HSSD benchmarks show that BeliefMapNav achieves state-of-the-art (SOTA) Success Rate (SR) and Success weighted by Path Length (SPL), with a notable 9.7 SPL improvement over the previous best SR method, validating its effectiveness and efficiency.

BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation
Zibo Zhou', Yue Hu?, Lingkai Zhang', Zonglin Li', Siheng Chen
Shanghai Jiao Tong University
University of Michigan
{zb zho_, zhanglingkai, channeler_xfyye sihengch@sjtu. edu.
muyu@uiich.o edu
Abstract
Zero-shot object navigation (ZSON) allows robots to find target objects in un- familiar environments using natural language instructions, without relying on pre-built maps or task-specific training. Recent general-purpose models, such as large language models (LLMMs) and vision-language models (VLMss,, equip agents with semantic reasoning abilities to estimate target object locations in zero-shot manner. However, these models often greedily select the next goal with- out maintaining global understanding of the environment and are fundamentally limited in the spatial reasoning necessary for effective navigation. To overcome these limitations, propose novel 3D voxel-basee belief map that estimates the target's 's prior presence distribution within a voxelized 3D space. This approach enables agents integrate semantic priors from LLMs and visual embeddings with hierarchical spatial structure, alongside real-time observations, to build a comprehensive 3D global posterior belief of the target's location. Building on this 3D voxel map, we introduce MeliefMapNNav, an efficient navigation system with two key advantages: i) grounding LLM semantic reasoning within the 3D hierarchical semantics voxel space for precise target position estimation, and ii) integrating sequential path planning enable efficient global navigation decisions. Experiments on HM3D and HSSD benchmarks show that BeliefMapNav achieves Lfateof-the-aa (SOTA) Success Rate (SR) and Success weighted by Path Length (SPL), with a notable 9.7 SPL improvement over the previous best SR method, validating its effectiveness and efficiency. The source code is publicly available at:
Introduction
Zero-shot object navigation (ZSON) enables robots to locate targets in novel environments through natural language instructions (e.g., "find the red sofa"), eliminating reliance on pre-mapped scenes or object-specific training 2
3 5. In domestic settings, ZSON supports assistive tasks such as
retrieving user-specified objects 6 industrial inspection, ZSON enables autonomous localization of malfunctioning components (e.g., detecting a leaking pipe) within complex facilities. In warehouse operations, ZSON enhances robotic picking and inventory management by allowing flexible retrieval of objects without bre-built maps. These real-world applications highlight the necessity of robust zero-shot navigation for scalable, adaptable robot deployment across diverse domains.
To enable ZSON, prior works have progressed along two main directions. The first approach constructs bird's s-eye view (BEV) value maps 7
9 by leveraging pixel-level semantic cues to
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Shap
Next Navagation
Best Observatio Direction
leret pelee wn
Hea Padi
larget Bobee a FGV
Row Troe Park Plesing
Targer Be
Rasl liw hark Farney
Time
Max
Siog
Aayer
Ouuch
Min
3D Voxel-Based Belief Map
Time Step
Time Stup
Step
Frontlers
Best Observation Direction
Teeget FMM
Beaf Tiee Phunney
Serget Balied
Planinn
Targen Balert POW
Placag
The search process: BeliefMapNav plans frontier paths by minimizing the expected search distance based on the 3D voxel-based belief map, ensuring efficient and stable exploration.
Figure 1:
provide dense estimations of target locations in the BEV space. While dense BEV representations provide object position estimation at a fine-grained level, the actual numerical distinctions between positions remain ambiguous with limited discriminative semantics. This insufficient differentiation in positional values, combined with the lack of high-level semantic reasoning, ultimately leads to compromised accuracy in the target location prediction. More recently, second approach has emerged that employs large language models (LLMs) or vision-language models (VLMs) to reason about the target locations m [12 13 14]. However, both LLMs and VLMs face limitations in spatial understanding and reasoning (15). which significantly affect target location prediction accuracy, Additionally, VLMs are limited by their inability effectively extract the most relevant and fine- grained semantic information from image observations, as well as by the lack of spatial information, resulting in imprecise target position predictions [IO 12 13 [4] For LLMs, while leveraging spatial information, reliance on semantic content from environment language descriptions leads to significant loss of information and reduced prediction precision [66 12 [8]. Consequently, LLMs and VLMs both struggle with reliable and accurate target location inference. Furthermore, existing methods generally rely on greedy navigation strategies [12 8. which cause frequent back- and-forth movements, significantly hindering search efficiency. Together, in existing works. the lack of semantic cues and spatial reasoning leads to inaccurate and imprecise target object position estimation. Meanwhile, the absence of efficiency optimization in the search behavior hinders robust searching for diverse, open-set targets in unbounded real-world 3D spaces.
To enable more precise and accurate predictions of the target object's 's location within 3D space, we propose a novel 3D voxel-based belief map that considers rich hierarchical spatial semantic cues and LLM-generated target-adaptive semantic cues to reason about prior belief of target presence in dense 3D voxel space. This structured representation enables spatially fine-grained estimation across the 3D voxel space, facilitating more precise and generalizable localization of target objects in complex, unbounded environments. Moreover, this tine-grained and accurate representation enables efficient guidance for gigh-degree-offfreedod mobile agents, facilitating precise, task-relevant, language- driven search within localized areas and enhancing the robustness of manipulation tasks.
To further enhance search efficiency, we introduce BeliefMapNav, an efficient zero-shot object navigation system based on path sequence optimization over the belief map. The system is composed of three tightly integrated modules and builds upon the belief map framework to enable efficient, goal-directed exploration. 1) The 3D voxel-based belief mapping module encodes prior beliefs of object presence in 3D space by integrating hierarchical spatial semantics with commonsense knowledge from an LLM. 2) The frontier observation belief estimation module combines the belief map with visibility map, which encodes real-time observation feedback likelihood, to produce posterior beliefs of the target object's position. Then, the module estimates the posterior observation belief of detecting the target in each frontier's field of view (FOV). 3) The observation belief-based planning module selects the next navigation goal by minimizing the expected distance cost based on the posterior observation beliefs to find the target. By explicitly modeling uncertainty and updating

---

spatial beliefs dynamically, BeliefMapNay enables more efficient, goal-directed exploration than methods using static priors or reactive policies.
The contributions of our method are mainly summarized as follows: 1) We propose BeliefMapNav, an efficient zero-shot object navigation system that accurately predicts target location through fine-grained belief' estimation in a 3D voxel-based belief map and real-time feedback, enabling belief-driven sequential planning for efficient navigation. At the core of our system is 3D voxel- based belief map that integrates hierarchical spatial-visual features with LLM-derived commonsense, enabling accurate and fine- geeigraaned prior estimation of the target's location. Based on the prior estimation, we design a planner that optimizes navigation by minimizing the expected path distance cost for efficient navigation. 3) The proposed system achieves SOTA performance on the HM3D [19] and HSSD [20] benchmarks, surpassing all the previous zero-shot methods by 3.4% in SR and 9.7 in SPL. on HM3D, 14.2% SR and 7.2 in SPL. on HSSD, thereby demonstrating the overall effectiveness and efficiency of our approach.
2 Related Works
Object Navigation Object navigation refers to the task of guiding a robot to search given target objects in an unknown environment. It can be divided into two categories: i) training-required methods, such as reinforcement learning (RL) and imitation learning, which require extensive training
on task-specific data 21 [2] [3 24 25], and ii) zero-shot methods, which leverage pre-trained
models, such as VLMs or LLMs, to perform navigation without additional training [14]
3 g g [2].
Training- methods typically require large amounts of data and have difficulty generalizing due to
limited environmental diversity [19 26), while zero-shot methods offer flexibility and adaptability to novel environments, but are constrained by the spatial reasoning limitations of L.LMs and VLMS [15] Our method follows zero-shot approach. We leverage LLMs and rich hierarchical spatial semantics to provide accurate and fine-grained estimations of the target's location, while employing probability- based optimization algorithms to ensure the efficiency of the search path.
Semantic Mapping for Object Navigation. Semantic mapping important in object navigation as it provides spatially structured representations that guide the robot in locating targets. Traditional
methods, such as category- based approaches [4 27 17 and scene graph- based methods [28 16 18
29 30 31]. often rely on predefined categories or topological graphs, leading to semantic information
loss and mapping errors due to detection failures [32 33
Although value map- based methods [8] 7 7
aim for con-vocabulary representations, existing methods struggle to align spatial and semantic scales, resulting in incomplete or misaligned spatial semantics under varying scene extents. As a result, the generated maps lack the precision needed to accurately localize target objects. In contrast, our method constructs a multi-level, spatially-aligned semantic map that supports accurate target object localization estimation.
Method
In this section, we first define the object navigation task and then present the BeliefMapNav system, which consists of three main modules: 3D voxel-based belief mapping, frontier observation belief estimation, and observation belief-based planning, shown in Fig. 2
3.1 Task definition
We define the ZSON task, where agent is required locate specified target object in an unknown environment without task-specific training, pre-built maps, or a fixed vocabulary. The target category e is specified in free-form text. At each timestep 1,, the agent receives RGB-I) observations
I (1rsb, I Illpphh), where Irab E R #x and rdepth E IR. #W and its pose s; (x), with Ie E R and rt E SO(3). from odometry. The action space A includes: MOVE FORWARD (0.25 m), TURN LEFT/RIGHT (30"), LOOK UP/DOWN (30"), and STOP. The task successful if the agent issues STOP within 0.1 m of the target object within 500 steps. At each timestep, the system takes as input the current RGB-D observation 7, the agent's pose 8t, and the text-specified target c, and outputs a navigation action a, E A from the discrete action set.
3

---

Voxel-bosed Belief Mapping Module
Frontier Observation Belief Estimation Module
Nop
Illenchiod Lardiroris Geseraler
201 Selef Agangatios
bypt
Scacces
Stepping
Bklle
Serrantic Nappting
Comtratee
C Noud banel Bop
Frwcliss Map
Semsors
Initalizee
Vonel-based Nalief Mapping Module
Frontier
Froutions Belif
Observation Based
Based
Module
Planning Module
Exploration
Theari
Nettane
Loodl Planner
Goal Navigation
Apral Mollie
Ototadio Mappingg
Actinn
Preettine
Figure 2: BelieiMapNav pipeline: The agent initializes with 360" rotation. During exploration, 3D voxel-based belief mapping module fuses sensor input, the 3D hierarchical semantic map, and landmarks create belief map. The frontier observation belief estimation module computes frontier observation belief from the belief, frontiers, and visibility maps via FOV- roV-based aggregation. The observation belief-based planning module selects the next goal based this belief and outputs navigation actions. Upon detecting the target, the agent navigates to it.
3.2 System overview
BeliefMapNav is novel 3D voxel-based zero-shot open-vocabulary object navigation system; see the overview Fig. 2 BeliefMapNNv has three key modules: 1) 3D voxel-based belief mapping module in Sec. 3.3 which encodes a prior belief of the target's presence by combining hierarchical
spatial semantics with LLM commonsense a belief map; 2) frontier observation belief estimation module in Sec. 3.4] which fuses the prior belief map with real-time observation likelihood to estimate the posterior belief of detecting target in each frontier's FOV; 3) observation belief-based planning which selects the next navigation point by optimizing expected distance cost to
module in Sec. 3.5
detect the object.
3.3 3D voxel-based belief mapping
3D voxel-based belief mapping aims to represent the spatial prior belief of the target object's presence in 3D voxel grid. The key intuition is that maintaining fine-grained representation enables more spatially detailed and accurate prediction of the target's location. To achieve this, we construct a 3D) voxel map where each voxel stores the belief of the target existing within its spatial region:
B = {(u, bw) uuu Z²}
where B denotes the belief map, u E 3 represents the discrete spatial coordinate of voxel in 3D space and b, E R. represents the estimated prior belief that the target object exists within voxel Unlike previous methods 7 g the 3D voxel- belief map leverages hierarchical language and spatial semantics to provide more precise and nuanced estimation of the target belief in each voxel of 3D space. It involves three steps as shown Fig. 3 1) constructing 3D hierarchical semantic voxel map based on visual observations in Sec. 3.3. 2) using LLM to infer hierarchical landmarks with
corresponding relevance scores in Sec. 3.3.2 and 3) mapping the inferred landmarks and relevance
scores into the hierarchical semantic voxel map form the belief map in Sec. 3.3.3
3.3.1
3D Hierarchical semantic mapping
The 3D hierarchical semantic voxel map Me represents the environment across three levels, L, {scene, region, object]. with progressively finer semantics. This structure enables reasoning across spatial scales, from coarse layouts fine object details, improving the accuracy of target object position estimation. Formally: Mc {M,, (0% 1 Z 1, E L,}. Here, for each level
la Lg, voxel u stores image CLIP 34 feature vector oyz IRd and feature confidence score

---

Hierarchical Landmarks Generation
Hiccorchiedl lendmarks
Target Offjoet
Tixl
Prompt
LIM
Indevsten
inlevssee
Aare Busiion
Senaurs
Cip Danade
30 Hisrarchical Semantic Vinaal Map
Conntootin
BejefMap
Cipy
SAM
Illenerr heal
Vonel based Bellet Map
sagga
Secar Lond Fatax Soore
Mrli leaage
Regins Leve Suote
Lkcurtinn
Dropo
Odjelctem Puttandone
Figure 3: The pipeline begins by passing the target and prompt to the LLM to generate hicrarchical landmarks with relevance scores. Mcanwhile, RGB and depth images cropped into multi-scale patches and processed via SAM and point cloud back- projecroet..oo.. Multi-scale CLIP image features extracted, and the top features selected by hierarchical feature scorers update the hierarchical 3D semantic map. Finally, landmarks encoded by text CLIP and the semantic map are combined via the belief map construction to update the 3D voxel-baan -aaee belief map.
8/ E IR, which quantifies the reliability of the semantic feature at each level and guides the selection of the most informative features for belief map updating,
The module operates in three stages: 1) Multi-scale feature extraction: Extract image CLIP features from multi-scale RGB images and spatial information from the depth images. 2) Hierarchical feature scoring: Assign confidence scores to features at different image scales, reflecting their relevance to specific semantic levels L., These scores enable adaptive scale selection for each semantic level. 3) Adaptive hierarchical feature selection: At each hierarchical level, features from the most confident scale are selected and back-projected to update the 3D voxel map M
Multi-scale feature extraction: To better capture both global context and local details, the observed RGB image g E = x3 is divided into equal-sized patches at multiple scales. At scale k. the image is partitioned into 2(k-1) x 2(k-1) patches, with each patch denoted as Pi We use CLIP 34 to extract visual features Uhiw for each patch and each patch pk is processed by the Segment Anything Model (SAM) 35 to estimate the number of semantic instances n parallel, the corresponding depth image is divided the same way. We back-project depth values of patches into 3D space to form a point cloud and compute two geometric properties: the volume h,w and point density pi... The detailed extraction process is in Appendix A.I
Hierarchical feature scoring: To select features for different spatial semantic levels, we design hierarchical feature scorers for L., which assign confidence scores to each image patch. A higher score indicates better alignment of the feature with the corresponding semantic level. The hierarchical feature scorers are defined as: (1) Scene Scorer: This scorer favors patches covering larger spatial extents with more semantic instances, score defined as 5scene.k = wn V + W2 n/ (2) Region Scorer: This scorer favors patches with more densely packed instances and concentrated point clouds,
score defined as A,,w0
W3
+ W1 Ph. (3) Object Scorer: This scorer favors patches
N.W
with a high average point density per instance, score defined as S1.a
gubject, Ph...
Adaptive hierarchical feature selection: Each image pixel position has i scale candidate features, and in each pixel position, we select the image CLIP features for hierarchical semantic levels from multi-scale patches that achieve the highest score under the corresponding semantic-level feature scorer. After scoring, pixels are back-projected into the 3D semantic map, where for each semantic level in every voxel, only the feature with the highest confidence score retained in the map. The detailed selection method is provided in the Appendix A.2
5

---

3.3.2 Hierarchical landmarks generation
Landmarks are semantic cues inferred by language model from the target object description, indicating where the object is likely to appear. In our method, we focus on three levels of landmarks L, = {room, region, object) to assist in locating the target object. To extract these landmarks, we prompt an LLM (GPT-40 (36] with the target object description, asking it to generate two outputs: (I) a set of landmark strings R = [R Reegium Ruubjeus, where R [r] r2 ...... rly denotes the landmarks at level 4 E Le; and (2) the corresponding relevance scores [o d object where at [a ab ..... a'i. Where nı, is the number of landmarks in level 4. Each score of indicates the likelihood that the target object appears near the corresponding landmark r/c Details about the prompt design and generation process are provided in the Appendix A.3
3.3.3 Belief map construction
After obtaining hierarchical textual landmarks with associated relevance scores, we project both the landmarks and the target object name into the 3D hierarchical semantic voxel map to generate a 3D voxel-based belief map, which represents a prior belief over the target object's presence in space. Each landmark r/ E R4 and the target object name Ttarget are encoded using the CLIP text encoder, resulting in embeddings E for landmarks and Eurger for the target object. For each of these textual inputs, we compute the maximum cosine similarity scores with stored spatial semantic features at corresponding levels: pt: max cosine (E 6%/)) and Patuurge = max cosine (Eurges, 6/2 ). Each i,
i,
similarity score is weighted by its associated relevance score af for landmarks and Charget for the target object. The final belief score at voxel u is computed as: b, - Ei, Ec, Ein af -P Pu,tagt The values b, in the 3D voxel-based belief map represent the prior belief of the target object's presence in each voxel.
3.4 Frontier observation belief estimation
The 3D voxel-based belief map serves as the prior, through which the frontier observation belief module dynamically integrates visibility map likelihoods to calculate the belief of detecting the target object within each frontier's FOV.
3.4.1 Visibility map
To capture the impact of real-time detection feedback on the target belief distribution, we in- troduce visibility map. The visibility map is inspired by (37) and is defined as p {(u, u E 23, D E [0] P. is the likelihood of detecting the target at voxel u based on visual observations during the search. The key intuition is that if a voxel has high detection confi- dence in the FOV but no detection, the belief that the target object exists in that region is very low (P" + 0). By estimating the object absence likelihood, the visibility map can refine prior beliefs to prevent revisiting areas that have been well observed. This correction improves navigation efficiency. The detection confidence is calculated in a way that it decreases near image boundaries and longer distances from the camera pose. Details of the construction of the visibility map are provided in the Appendix A.4
3.4.2 FOV-based belief aggregation
After constructing the visibility map p and belief map B, we fuse them obtain the posterior u E Z3, 8 = P% b.) This fusion enables more dynamic and accurate estimation of the belief of detecting the target from each frontier's FOV by combining spatial priors belief map with the observation feedback visibility map. It improves search efficiency and reduces exploration of well-observed regions. For each candidate frontier position Ty,, we evaluate four viewing directions 0 E f0p, 90" 180 270%). For each 8 perform ray casting from Fy to identify voxels within the FOV, separating them into Og (x), (voxels in the belief map) and On (x (zz,) (voxels out of belief map). The observation belief for direction 0 is computed
belief map B3osst = {(, kg (0.68g))
as: obs f.)))) = Encon ipoat + 10 (1 )l Wunobserved- Where 1 (xx))) is the number of

---

voxels not in map and Wunebserved is a constant weight. The final observation belief at s, is defined as: fz = max, ( Aggregating over the FOV allows us to account for obscure vision and the agent's limited FOV, leading to more accurate estimation of the likelihood of observing the target in each frontier's FOV.
3.5 Observation belief-based planning module
Using the estimated observation belief each frontier based the posterior belief map, pri- oritize frontiers to minimize expected search distance for a more stable and efficient target search. Unlike previous greedy approaches that myopically prioritize immediate gain, our distance- based optimization ensures smoother early-stage movement under uncertain belief distributions and gradu- ally converges the optimal path the belief becomes exploration. The optimal exploration strategy seeks a permutation of frontier visiting sequence [m fag..... fx,] that minimizes the expected search cost, where fx: E (Ifi...,, denotes the i-th frontier position. The objective is formulated as:
reliable, resulting in efficient
Σ
Σ da (on (ss
argmin SES,
where Sn denotes the permutation group over frontier points, Sr, represents the k-th visited frontier permutation ı (with the initial point S.o = xo defined as the agent's current position), d (J l'xz) denotes the path distance between adjacent frontiers computed via the A* (38 algorithm, Σi= (-)(-) calculates the cumulative path cost to the i-th frontier, and (abe(fr.)) the observation belief of frontier fr..
The proposed objective improves search efficiency by minimizing exploration cost with A o-ttiiiieed paths and prioritizing high-belief frontiers via Doservation-weighte costs. Combining geometric path costs and belief weights, it reduces noise impact on navigation stability. Integrating shortest- path guarantees with probabilistic reasoning, it enables dynamic, real-time replanning, adapting to evolving beliefs for flexible, efficient navigation. At the same time, the precise and detailed belief map provides solid foundation for effective optimization. We solve this optimization via GPU-accelerated simulated annealing [39] The detailed optimization process Before each action, the agent selects the first frontier in the optimized sequence x as the next navigation target and replans at every step with the updated belief map. We adopt the local point navigation planner from VLFM [7] to generate actions toward the given goal. An open-vocabulaty detector [40] and GPT-4o (36 verify detected objects; if confirmed, the system localizes the target using the bounding box, SAM 35,, and depth, then sets it as the final navigation goal.
the Appendix AA.5
4 Experimental Results
In this section, we outline datasets and key implementation details, then compare BelieMMapNav's performance against SOTA baselines on HM3D [9), MP3D [26]. and HSSD (20] Ablation studies assess each component's 's contribution. Qualitative analysis visualizes maps indicating target presence
probability. Search process visualizations highlighting our approach are in Appendix A.9
Baseline
summaries and HM3D failure analyses appear in Appendix (orendix/A.66lan and A.7 respectively.
4.1 Benchmarks and Implementation details
Dataset: We evaluate our method on three standard benchmarks: HM3D 19). MP3D 26 and HSSD 20). HM3D, the official dataset of the Habitat 2022 ObjectNav Challenge, includes 2,000 validation episodes across 20 environments and 6 object categories. MP3D, a large-scale indoor 3D scene dataset, is commonly used in Habitat-based ObjectNav evaluations. We conduct experiments on its validation set, consisting of 11 environments, 21 object categories, and 2,195 object-goot navigation episodes. HSSD, a synthetic dataset with scenes based on real house layouts, contains 40 validation scenes, 1.248 navigation episodes, and 6 object categories.
Evaluation Metrics: We use two standard metrics: Success Rate (SR) and Success weighted by Path Length (SPL). SR measures the proportion of episodes where the agent reaches the target within
7

---

Table Zero-shot object navigation results on MP3D, HM3D and HSSD. We compare the SR and SPL of state-of-the-an methods in different settings.
HM3D
MP3D
HSSD
Method
Unsupervised
Zero-shot
SRT
SPL
SRT
SPLT
SRT
SPL
Habitat Web
41.5
16.0
31,6
8.5
OVRL 42
X
28.6
7.4
ProcTHOR
43
X
54.4
31.8
SGM
X
60.2
30,8
37.7
14.7
ZSON 24
X
25.5
12.6
15.3
4.8
PSL 45
42.4
19.2
18.9
6.4
PixNav
X
37.9
20.5
VLFM
52.5
30.4
36.4
17.5
ESC [3
39.2
22.3
28.7
14.2
38.1
22.2
CoWs 33
9.2
4.9
L3MVN 14
50.4
23.1
34.9
14.5
41.2
22.5
ImagineNay
46
53.0
23.8
51.0
24.9
VoroNav 28
42.0
26.0
41.0
23.2
GAMap
8
53.1
26.0
OpenFMNav
47
52.5
24.1
37.2
15.7
SG-Nav [12
54.0
24.9
40.2
16.0
UniGoal
48
54.5
25.1
41.0
16.4
InstructNav
58.0
20.9
BeliefMapNav
61.4
30.6
37.3
17.6
65.2
32.1
preset distance. SPL evaluates path efficiency by considering both success and trajectory optimality: otherwise SPL 0. Higher values indicate better performance.
if successful, SPL Optimal path length
path length
Implementation details: We limit navigation to 500 steps, defining success stopping within 0.1 m of the target. Each step moves the agent 0.25 m forward or rotates it by 30 The RGB-D camera, mounted 0.88 high, captures 640 x 480 images. The 3D voxel map has 45,000 voxels at 0.25 m resolution. We set Wpnobserved = 0.01 (Sec. 3.4.2). CLIP-ViT-B-32 encodes visual/text features with image crop scale ks 3. GPT-4o generates three landmarks per level (nine total). Hierarchical scorer weights are Wi = 0.05, w2 0.1, W3 = Wy = 0.01. The system runs on a single RTX 4090 (24 GB VRAM) and uses approximately 13 GB of VRAM. Local planner parameters vary slightly by dataset, while the exploration module remains unchanged.
4.2 Comparison with SOTA methods
In this section, we compare our proposed BeliefMapNay with SOTA object navigation approaches in different settings, including unsupervised, supervised, and zero-shot methods on the MP3D 22).. HM3D [19, and HSSD (20 benchmarks. As shown in Table [ our method outperforms all existing zero-shot baselines except the SR of UniGoal and SG-Nav, achieving significant improvements across multiple benchmarks. Specifically,
HM3D, we observe gain of +3.4% in SR and +0.2 in SPL. On MP3D, we achieve +0.1 SPL. Finally, on HSSD, our method delivers remarkable gains of +14.2% in SR and +7.2 in SPI.. These results highlight the effectiveness of our approach in enhancing both SR and SPL across diverse datasets. While our SR on the MP3D dataset is marginally lower than that of UniGoal and SG-Nav, this is primarily due to their utilization of object verification mechanism. This feature specifically targets the reduction detector false positives (FP) caused by poor mesh quality inherent in MP3D. Crucially, while such object verification is important technique, remains orthogonal to our main contribution, which focuses squarely on efficient target-oriented exploration. This distinction is further evidenced by performance the high-quality HM3D dataset, where our method significantly outperforms UniGoal both SR (+6.9%) and SPL (+5.5).
On the HM3D dataset, our method improves SPL by 9.7 compared to the zero-shot method Instruct- Nav (9, which achieves the highest SR. While InstructNav prioritizes with dense search strategy, approach maintains high success rates and boosts search efficiency by generating more accurate

---

larget position estimates and optimizing the search path with distance cost-aware planner. On the MP3D benchmark, improvements are less pronounced due two factors; first, the lower data quality of MP3D, which makes target recognition more challenging. Second, there are lot of mesh "holes" in MP3D, which allow the agent to see through obstacles, causing mistakenly prioritize these holes as targets, leading to navigation failures when it gets stuck near non-traversable areas. However, on the HSSD dataset, performance significantly improves because the synthetic scenes avoid the issues present in MP3D and HM3D. Across all datasets, the performance limitations of the local planner in 7 lead to significant degradation, especially in narrow areas.
4.3 Ablation study
To evaluate the effectiveness of each module our system, we conduct an ablation study on 400 randomly sampled episodes from the HM3D validation set, using fixed random seed. The ablation study of the effectiveness of LLMs and image scale are in Appendix A.8
Table 2: Impact of the planner and visibility map.
Table 3: Impact of vision- encoders.
Method
SR
SPL1
Encoder
SR
SPL
BeliciMapNav w/o Planner
56.0
29.3
BLIP 49
59.3
31.0
BeliefMapNav w/o Visibility Map
57.2
28.0
BLIP2 [50]
62.0
31.1
BeliefMapNav
62.5
31.6
CLIP 34
62.5
31.6
Effectiveness of visibility map and belief-based planning: In Table compare the effectiveness of the Visibility Map and Belief-based planning against basic exploration. Without the Visibility Map, relying on spatial priors alone leads a 5.3% & drop in SR and 3.6 4 in SPL, as the agent revisits previously observed regions. Without the planner, navigation is based solely on the highest posterior belief, resulting in 6.5% 4 drop in SR and 2.3 4 in SPI. due to frequent navigation goal switching and inefficient hack-and-fortt movement.
Effectiveness of different levels of hierarchical 3D semantics; As shown in Table we evaluate the impact of different levels of the Hierarchical 3D Semantic Map on performance, comparing four settings: no semantics (random walk), scene-level only, scene region levels, and the full hierarchy with object-level semantics. Results indicate that incorporating more semantic levels generally improves SR. However, omitting object-level semantics enhances efficiency (32.0), as fine-grate fine-grained searches with object-level cues increase success rates but often result in slower, localized exploration, leading to longer paths and slightly reduced efficiency.
Table 4: Impact of different levels of landmarks.
Table 5: Impact of different semantic levels.
Landmarks w/o
SR
SPL
Semantics
SR1
SPL
60.0
30.9
Random Walking
21.5
10.8
Room
61.0
31.1
Scene
59.0
30.4
Room+Region
61.5
31.2
Scene Region
61.5
32.0
Room + Region Object
62.5
31.6
Scene Region + Object
62.5
31.6
Effectiveness of different vision-language encoders: shown in Table 3 CLIP- and BLIP-2-based
systems achieve comparable performance (SR: 62.5 vs. 62.0; SPL.: 31.6 vs. 31.1), both outperforming BLIP. Prior work [51] similarly shows that BIIP-2 slightly surpasses CLIP in zero-shot text-to-image -i-aggee retricval accuracy, while both are significantly better than BLIP. However, CLIP demonstrates stronger generalization to ofitt---iistriuution data and supports efficient inference via independent encoders and pre-computed features. These properties make CLIP a more robust and practical encoder choice for retrieval-based navigation tasks.
Effectiveness of hierarchical landmarks: As shown in Table without landmarks, we retrieve directly using the object name in the hierarchical 3D semantic map. With incremental landmark introduction, we observe a gradual improvement in SR (60 to 62.5) and SPL (30.9 to 31.6). However, this improvement is less significant than the gain from incorporating spatial semantics at different hierarchical levels in Table as object names already show inherent relevance to scene, region, and object semantics. In this context, hierarchical landmarks mainly reinforce these semantic associations.

---

30 Voxel-Bosed Belief Map
Visillity
Posterior Delief
PAO
Thrgut Ciljeets Coads
Apent
Puth
Figure 4: Visualization of the prior belief map, visibility map, and the posterior belief map, with an enlarged section highlighting the target object.
4.4 Qualitative Analysis
Fig. 4 shows the 3D voxel-based belief map, visibility map, and posterior belief map, and the complete visualization is Fig. o2 Appendix. The posterior belief map assigns high belief to the target object's s (couch) location. While the visibility map indicates low likelihood, the prior belief map, guided by vision-and-lan laiin-and-languuge cues, strongly suggests the target's presence, effectively guiding the agent's s search (Fig. [. Additional visualizations are provided in Appendix A.9
5 Conclusion
In this paper, we have proposed BeliefMapNav, a zero-shot object navigation system that integrates hierarchical spatial semantics, commonsense reasoning from LLMs, and real-time feedback through a 3D voxel-based belief representation. Compared to prior approaches that rely on planar value maps local greedy goal selection, BeliefMapNav maintains a global 3D belief posterior, performs usiiiliity-aaan updates, and plans over informative frontiers, improving robustness to environmental complexity, reducing backtracking, and enabling more efficient exploration. Experiments three benchmarks show that it outperforms previous methods. Ablation studies highlight the effectiveness of our belief map and belief-based planning for efficient exploration, emphasizing the system's 's effectiveness and efficiency.
Limitation and future work. We validate the effectiveness of the 3D voxel-based belief map solely on object navigation tasks. This high-resolution 3D map can be further extended to enable robot interaction for mobile manipulation tasks, with future work focusing on real-world implementation.
Acknowledgements
The authors gratefully acknowledge the computational support provided by the CMIC of Shanghai Jiao Tong University.
References
[1] Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhut- dinov. Learning explore using active neural slam. arXiv preprint 05x5v:2004.05/55, 2020.
[2] Devendra Singh Chaplot, Dhiraj Prakashchand Gandhi, Abhinav Gupta, and Russ R Salakhut- dinov. Object goal navigation using goal-oriented semantic exploration. Advances in Neural Information Processing Systems, 33:4247 2020.
[3] Kaiwen Zhou, Kaizhi Zheng, Connor Pryor, Yilin Shen, Hongxia Jin, Lise Getoor, and Xin Eric Wang. Esc: Exploration with soft commonsense constraints for zero-shot object navigation. In International Conference Machine Learning, pages 42829 42842. PMLR, 2023.
[4] Gengze Zhou, Yicong Hong, and Qi Wu. Navgpt: Explicit reasoning in vision-and-lange langugge navigation with large language models. Proceedings of the AAAÍ Conference on Artificial Intelligence, volume 38, pages 7641-7649, 2024.
[5] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023. URL https://arxiv. org/abs/2210.03629 2023.
[6] Peiqi Liu, Yaswanth Orr,, Jay Vakil, Chris Paxton, Nur Muhammad Mahi Shafiullah, and Lerrel Pinto. Ok-robot: What really matters integrating open-knowledge models for roboties. arXiv preprint 242.:22011..202.. 2024.
10

---

[7] Naoki Yokoyama, Schoon Ha, Dhruv Batra, Jiuguang Wang, and Beraaettte Bucher. Vifm: Vision-language frontier maps for zero-shot semantic navigation. In 2024 IEEE International Conference Robolics and Automation (ICRA), pages 42-48. IEFE, 2024.
[8] Hao Huang, Yu Hao, Congcong Wen, Anthony Tzes, Yi Fang, et al. Gamap: Zero-shot object goal navigation with multi-scale geometric-affordance guidance. Advances in Neural Information Processing Systems, 37:39386-3399 -9448,, 2024.
[9] Yuxing Long, Wenzhe Cai, Hongcheng Wang. Guanqi Zhan, and Hao Dong, Instructnav: Zero-shot system for generic instruction navigation in unexplored environment. arXiv preprint arXiv:2406. 04882, 2024.
[10] Dylan Goetting, Himanshu Gaurav Singh, and Antonio Loquercio. End-to-end navigation with vision language models: Transforming spatial reasoning into anestion-answeringg arXiv preprint arXiv:2411.0577 2024.
[11] Wenzhe Cai, Siyuan Huang, Guangran Cheng, Yuxing Long, Peng Gao, Changyin Sun, and Hao Dong. Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 5228-5234. IEEE, 2024.
[12] Hang Yin, Xiuwei Xu, Zhenyu Wu, Jie Zhou, and Jiwen Lu. Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation. Advances in Neural Information Processing Systems, 37:5285-5307, 2024.
[13] Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, and Shuran Song. Cows pasture: Baselines and benchmarks for language-driven zero-shot object navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23171-23181, 2023.
[14] Bangguo Yu, Hamidreza Kasaci, and Ming Cao. L3mvn: Leveraging large language models for visual target navigation. In 2023 TEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3554-3560. IEEE, 2023.
[15] Jihan Yang, Shusheng Yang. Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. arXiv preprint arXiv:2412. 14171, 2024.
[16] Kunal Pratap Singh, Jordi Salvador, Luca Weihs, and Aniruddha Kembhavi. Scene graph contrastive learning for embodied navigation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 10884-10894, 2023.
[17] Junting Chen, Guohao Li, Suryansh Kumar, Bernard Ghanem, and Fisher Yu. How to not train your dragon: Training-fres embodied object goal navigation with semantic frontiers. arXiv preprint arXiv:2305. 169255 2023.
[18) Fengda Zhu, Xiwen Liang, Zhu, Qizhi Yu, Xiaojun Chang, and Xiaodan Liang. Soon: Scenario oriented object navigation with graph-based exploration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12689-12699, 2021.
[19] Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg. John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv preprint (1t.::20000002238 2021.
[20] Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X Chang, and Manolis Savva. Habitat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal navigation. Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 16384-16393, 2024.

---

[21] Erik Wijmans, Abhishek Kadian, Ari Morcos. Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. Dd-ppo: Learning near- pear-perfect pointgoal navigators from 2.5 billion frames. arXiv preprint arXiv: 1911v:1191100033030 2019.
[22] Arsalan Mousavian, Alexander Toshev, Marek Fišer, Jana Košecká, Ayzaan Wahid, and James Davidson. Visual representations for semantic target driven navigation. In 2019 International Conference Robotics and Automation (ICRA), pages 8846-88855 IEEE. 2019.
[23] Wei Yang, Xiaolong Wang, Ali Farhadi, Abhinav Gupta, and Roozbeh Mottaghi. Visual semantic navigation using scene priors. arXiv preprint arXiv:1800.000
2018.
[24] Arjun Majumdar, Gunjan Aggarwal, Bhavika Devnani, Judy Hoffmann, and Dhruv Batra. Zson: Zero-shot object-goal navigation using multimodal goal embeddings. Advances in Neural Information Processing Systems, 35:32340-32352, 2022.
[25] Oleksandr Maksymets, Vincent Cartillier, Aaron Gokaslan, Erik Wijmans, Wojeiech Galuba, Stefan Lee, and Dhruv Batra. Thda: Treasure hunt data augmentation for semantic navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15374- 15383, 2021.
[26] Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv: 077.006990008..8 2017.
[27] Qianfan Zhao, Lu Zhang, Bin He, Hong Qiao, and Zhiyong Liu. Zero-shot object goal visual navigation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 2025-2031. IEEE, 2023.
[28] Pengying Wu, Yao Mu, Bingxian Wu, Yi Hou, Ji Ma, Shanghang Zhang, and Chang Liu. Voronav: Voronoi-based zero-shot object navigation with large language model. arXiv preprint vrXiv:2401.02694 2024.
[29] Rui Liu, Xiaohan Wang. Wenguan Wang, and Yi Yang. Bird's-cye-viee scene graph for vision- language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10968-10980, 2023.
[30] Zachary Ravichandran, Lisa Peng. Nathan Hughes, J Daniel Griffith, and Luca Carlone. Hi- erarchical representations and explicit memory: Learning effective navigation policies on 3d scene graphs using graph neural networks. In 2022 International Conference on Robotics and Automation (ICRA), pages 9272-9273 IEEE, 2022.
[31] Hayato Suzuki, Kota Shimomura, Tsubasa Hirakawa, Takayoshi Yamashita, Hironobu Fujiyoshi, Shota Okubo, Nanri Takuya, and Wang Siyuan. Human-like guidance by generating navigation using spatial-temporal scene graph. In 2024 IEEE Intelligent Vehicles Symposium (IV), pages 1995. IEBE, 2024.
[32] Chenguang Huang, Oier Mees, Andy Zeng, and Wolfram Burgard. Visual language maps for robot navigation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 10608-10615. IEEE, 2023.
[33] Hao Huang, Shuaihang Yuan, CongCong Wen, Yu Hao, and Yi Fang. Noisy few-shot 3d point cloud scene segmentation. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 11070-11077. IEEE, 2024.
[34] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages $748-8763. PmLR, 2021.
[35] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan- Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4025-4026 2023.
12

---

[36] Aaron Hurst. Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
[37] Muhammad Fadhil Ginting, Sung- Kyun Kim, David D Fan, Matteo Palieri, Mykel J Kochender- fer, and Ali-akbar Agha-Mohammadi. Seek: Semantic reasoning for object goal navigation in real world inspection tasks. arXiv preprint N.Kiv:2405.09822 2024.
[38] Peter E Hart, Nils J Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics, 4(2):100-107, 1968.
[39] Dimitris Bertsimas and John Tsitsiklis. Simulated annealing. Statistical science, 8(1):10-15, 1993.
[40] Shilong Liu, Zhaoyang Zeng. Tianhe Ren, Feng Li, Hao Zhang, Jie Yang. Qing Jiang, Chunyuan Li. Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, pages 38-55. Springer, 2024.
[41] Ram Ramrakhya, Eric Undersander, Dhruv Batra, and Abhishek Das. Habitat- Learning embodied object-search strategies from human demonstrations at scale. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 5173-5188 2022.
[42] Karmesh Yadav, Ram Ramrakhya, Arjun Majumdar, Vincent-Pierre Berges, Sachit Kuhar, Dhruv Batra, Alexei Baevski, and Oleksandr Maksymets. Offline visual representation learning for embodied navigation. In Workshop Reincarnating Reinforcement Learning at ICLR 2023, 2023.
[43] Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. procthor: Large-scale embodied ai using procedural generation. Advances in Neural Information Processing Systems, 35:5982-5994, 2022.
[44] Sixian Zhang, Xinyao Yu, Xinhang Song, Xiaohan Wang, and Shuqiang Jiang. Imagine before go: Self-supervised generative map for object goal navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16414-16425, 2024.
[45] Xinyu Sun, Lizhao Liu, Hongyan Zhi, Ronghe Qiu, and Junwei Liang. Prioritized semantic learning for zero-shot instance navigation. In European Conference on Computer Vision, pages 161-178. Springer, 2024.
[46] Xinxin Zhao, Wenzhe Cai. Likun Tang, and Teng Wang. Imaginenav: Prompting vision-language models as embodied navigator through scene imagination. arXiv preprint arXiv:2410.098% 190.07944. 2024.
[47] Yuxuan Kuang, Hai Lin, and Meng Jiang. Openfmnav: Towards open-set zero-shot object navigation via dision-languaal foundation models. arXiv preprint arXiv:2402.1065 2024.
[48] Hang Yin, Xiuwei Xu, Lingqing Zhao, Ziwei Wang, Jie Zhou, and Jiwen Lu, Unigoal: Towards universal shot gral-oriented navigation. arXiv preprint 2rXiv:2503. 2025.
[49] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pages 22888-12900 PMLR, 2022.
[50] Junnan Li. Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730-19747 PMLR, 2023.
[51] Pavan Kalyan Tankala, Piyush Pasi, Sahil Dharod, Azeem Motiwala, Preethi Jyothi, Aditi Chaudhary, and Krishna Srinivasan. Wikido: A new benchmark evaluating cross-modal retrieval for language models. Advances in Neural Information Processing Systems, 17:140812- 140827, 2024.
13

---

[52] ROYUD Nishino and Shohei Hido Crissman Loomis. Cupy: A numpy-compatile library for nvidia gpu calculations. 31st confernce on neural information processing systems, 151(7). 2017.
[53] An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng. Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang. Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. arXiv preprint 200.022505.09388 2025.
[54] Gheorghe Comanici, Eric Bieber, Mike Schaekermann. Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.0622 2025.
[55] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Ilama 3 herd of models. arXiv e-prints, pages arXiv-2407, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly outline the main contributions of the paper, including the introduction of BeliefMapNav, an efficient zero-shot object navigation system. And all the modules in the system, including a 3D-voxel based belief map, a frontiers observation belief map, and an optimization planner.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation of our work in Sec. 5
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
We do not have any assumptions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We have some implementation details in Sec. 4.1 The results in our paper can be reproduced by the readers.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We have open- sourced our code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ "pippscc/puullc//uuu httoe//uuuddeeoooouuuubmmiiiia ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We follow the standard testing details which are stated the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We have some case studies in Appendix A.7
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
17

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Our system can run on a RTX 4090 GPU, as shown in Sec. 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We reviewed the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification: is discussed Sec.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited all the code and dataset that use.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification:
We have discussed the LLM usage in landmarks generation in Sec. 3.3.2
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https://neure) //teerurpp...c//oooffeerre/////s for what should or should not be described.
21

---

A Appendix
A.1 Multi-scale feature extraction
To better capture both global context and local details, the observed images are divided into equal- sized patches at different scales. Specifically, given an RGB observation image r of height // and width W. at image scale k the image is divided into ((h-1) x 2(*-1) equally sized patches, resulting W
H 2 X 214-17 and we denote in total of 4(k-1) patches. Each patch at level & has image size:
S h S 2(k-1), I w S 2(k-1)), where Pi,,. represents the patch located at the h-th row and w-th column of the partitioned image scale k. We employ the CLIP model [4 to compute visual features for each patch. Specifically, for given patch h.,. the corresponding visual feature oft... This process yields & candidate features for each pixel, and then each patch Ph, E 11 is independently processed using the Segment Anything Model (SAM) [35] to estimate the number semantic instances contains n/ where ni,, denotes the number of instances detected within the patch P h,w: at scale k.
the set of patches level k as: T/ {Ph,,
In parallel, the depth image is divided into patches using the same scheme as the RGB image. For each depth patch 1 that corresponds to the position of RGB patch Pk h. We back-project the depth values into 3D space to generate point cloud: Ch,,.. We then compute two geometric properties of each point cloud: the volume h,w of the point cloud, density ph defined as the number of points per unit volume: ph,u-
Ich where (C), denotes the total number 3D points in the patch.
R.W.
A.2 Adaptive hierarchical feature selection
For each image pixel p at hierarchical spatial level l, we select the CLIP feature from all candidate patches across scales that achieves the highest score under the corresponding scorer: v/ uh - x/7 = kw.""' and (k*, hh, w²) = arg max ki sh..k Where, denotes the image CLIP feature of pixel at level 1g. J..k denotes the score assigned to patch Ptt at level 1.. $,,0
and the constraint p - ensures that only p in patches considered. After scoring, each pixel is back-projjetedd into 3D and mapped to voxel. For each semantic level, we keep only the feature with the highest confidence score in each voxel.
After scoring, each image pixel p is back- projeeroj into 3D space global position Tp using the depth image and mapped to the corresponding voxel in the spatial map:
BackProject(p, 1.MPthh (p), st) E R Up = Tp
Z
where r denotes the voxel resolution, and indicates the element- floor operation used to discretize the 3D coordinate into voxel space. For each semantic level, if the voxel does not contain an existing feature, directly store the current feature and its associated confidence score. If feature already exists, we compare the new score with the stored score and retain the feature with the higher confidence. The voxel map is then updated according to the following rule:
if up Mc or stz Sii, otherwise
s gla uop
if Up & Me s/y 8t.
8// up
s/e
G
otherwise
A.3 Prompting
In the hicrarchical landmarks generation, the complete prompt is as follows:
22

---

System: You are helpful robot to find an object in unknown environment. User:
Now that we need to find a/an [target), please provide information about how it might be found in private house. This information will be embedded with CLIP and must be useful for the robot to recognize the object.
Provide which room it is likely to be found in.
Provide where is likely to be located within specific rooms. type in front of the location.
Please add the room
Provide what other items it is likely to be near.
What is the probability of finding the (target) respectively around these landmarks? The sum of the probabilities each level is equal to one.
At the same time, the output must meet the following requirements:
Output three related strings for each level. Each string should contain only one piece of information and avoid using "or" constructions.
2. Each piece of information should consist of only the most relevant phrases, not complete sentences. Keep the phrases simple and common; avoid uncertain words like "maybe".
3. Output two two-dimensional lists. For the landmarks string: The first dimension represents the information level, and the second dimension contains the three related strings for each level. For the Probability: The first dimension represents the information level, and the second dimension contains the three related probabilities for each level.
4. Output only the string and Probability, do not include any additional text.
In the generation of parameters dmin and dmax in the visibility map, the complete prompt is as follows:
System:
You are helpful robot to find an object in unknown environment.
User:
When we use YOLOv7 and GroundingDINO to detect a/an [target] a simulated mesh private house environment, the quality is poor. What is the best distance from the camera pose to the (target) to detect the [target]? The input images are 640 480 resolution.
1. Please output the distance meters.
2. Please only output the distance range as list: (distance_min, distance_man _max], without any other text.
A.4 Visibility map
As shown in Fig. 5 detection confidence depends the pixel locations of the voxels in the image and the distance to the camera. Pixels near the image center and at moderate distances to the camera yield higher confidence, while peripheral or extreme-distance regions tend to have lower confidence due to reduced detectability. For each pixel in the RGBD image, we compute three components: the horizontal oegle-based confidence Chorizonall the vertical angle-based confidence Cvertiall and the distance-based confidence Ca. The detection confidence of pixel position is defined as:
8p
Chorizoutal = cos2
Ontov (T
Óp Ovton
Overtical = 2
(1, Ca eexp (-a min ((dp d (dp dmax)) )dma)))))
if dmin I dp S dmax otherwise
Cp=
5==Ca Ca Chorizmtal Cvertical
Here, Op and db denote the horizontal and vertical angles pixel p. and Ontow &vtov are the horizontal and vertical FOV angles in radians. Ca is computed from the pixel depth d, based on the optimal
23

---

(a) Input depth image
(b) Confidence value
Figure 5: (a) is the input depth image. (b) shows the confidence computed from the depth image. Bluer regions indicate higher confidence, meaning the likelihood an object being present is low if not detected there. Redder regions indicate lower confidence, implying that even if no object is detected in those areas, the probability of object presence remains relatively high.
range [dmin: dmax given by A.3 The final confidence score Cp means the confidence detect the object at pixel position
Each pixel back-projected into 3D space obtain the corresponding voxel Up 2 The visibility map is then updated as:
1 Cp. Dūg.
if Up P or Cp <
Pup
otherwise
A.5 Path optimization
The optimization objective of the path planning problem is to find a frontier visiting order [f,, f......, that minimizes the expected search distance:
π = argmin
Σ
d dd-
(fm fre))
Pobalfa.)
EES
Therefore the cost function W(T) to evaluate the quality of path x can be defined as:
W(+) Σ
Σ da (Jr-11
Pots((
1=1
The simulated annealing algorithm is probabilistic optimization algorithm that be used find an approximate solution to the path planning problem. The algorithm is inspired by the annealing process in metallurgy, where material is heated and then cooled to remove defects and improve its properties. The algorithm works by iteratively exploring the solution space and accepting or rejecting new solutions based their cost and temperature parameter. Our implementation is based on the following steps:
Initialization: Set the initial and terminal temperature Tp and Ty, the cooling rate a, and the number of samples to simulate N. When the current temperature T is greater than the terminal temperature Ty, the algorithm will continue to run.
2. Iterative Process: While the termination criterion is not met, for each sample, the algorithm will perform the following steps:
(a) Generating Neighbor Solution: Generate neighbor solution n' from the current
solution π by applying three kinds of operations: swap, shift, or reverse.
(1) swap: swap two points in the path. (2) shift: move a segment of the path different position. (3) reverse: reverse segment of the path.

---

The repetition times of the operations are controlled by the temperature T, which decreases with time. Due to different operations having different degrees of impact on the probability of selecting each operation is different. And because the first point in the path the starting point, it does not participate in this transformation.
(b) Evaluation and Acceptance: Evaluate the cost of the new path W(*') and apply the Metropolis Criterion: Compare it with the cost of the current path (()). If W(T') < (x).. accept the new path. Otherwise, accept with a probability of W ((x)-W((r)
exp
(c) Cooling: Update the temperature T by multiplying it with the cooling rate
3. Termination: The algorithm ends when the temperature T' is less than the terminal tempera- ture Ty. The final output the sample with the lowest cost.
The algorithm is implemented Python and uses the CuPy [52] library to accelerate the process. All the parameters tuned to achieve balance between the quality of the solution and the time taken to find it, for the path planning problem. On a laptop with Intel Core i5-12500H CPU, 16GB RAM, and NVIDIA GeForce RTX 2050 Laptop GPU, for task of 10 frontiers, the algorithm takes about 0.2 seconds to solve the problem. A visualized example problem and the solution are shown in Fig.
jjnt=7.329, Finalls,, 49, alpha-0.9U5
spartecli=2000 alphamC.08 065
0.40
37.5
0-
33.0
10.5
owwogg
0.20
23.0
0.3
0.55
200
Figure 6: Error analysis of the algorithm for parameter tuning. All the results are based on 10 frontiers, over 50 different scenes, and each scene for 100 times. We take the solution obtained when the algorithm converges as the optimal solution. A solution considered as an error the cost is greater than 100% of which of the optimal solution. Between the performance of the algorithm and the time taken to find the solution, we take a balanced approach.
600
0.00035
560
0.00030
Best cose
0.00025
540
0.00020
520
0.00015
500
0.00010
0.00005
460
7.0
5.5
Temperature
Figure 7: Result demo of the algorithm. The chart on the left shows the relationship between the cost and the temperature under a run. The chart on the right shows the path generated by the algorithm. The number on the lines indicates the order of visiting the frontiers. The arrow at each frontier is the orientation. The color of the background represents the distribution of the occurrence probability. The observation probability of a frontier is the sum of the weights of all points within sector in front of the frontier, representing points within the FOV of the robot.
25

---

4.0%% 2.0%%
Donection
Nevir Sow Target (Iffaasibe)
12.02%
Detection
Oithers
44.30%
96.78%
Figure 8: The proportion of different causes of failure in the HM3D dataset.
A.6
Baselines
We evaluate our approach in comparison with range of ZSON baselines, including several SOTA methods. ZSON [24] incorporates object category cues to enable object-aware navigation. CoWs [3] leverages CLIP features to extract semantic object information and directs exploration toward nearby frontiers. ESC 3 combines a semantic scene representation with commonsense reasoning to guide object search. L3MVN [14] utilizes large language models infer exploration goals from semantic map constructed by pretrained detector. VL.MM 7 adopts BLIP-2 (50) for vision-language align- ment, using target object descriptions to prioritize exploration frontiers. VoroNav 28 introduces a navigation method based on Voronoi partitioning. InstructNav (9) supports agent navigation by converting the output instructions from the VLM into various value maps. However, this method directly relies on the VLM to reason in spatial contexts, which can lead to hallucinations and reduced navigation efficiency. GAMap 8 guides navigation by leveraging object parts and affordance at- tributes through image-based multi-scale scoring approach that effectively captures both geometric components and functional affordances. SG-Nav [12] guides navigation by leveraging an online 3D scene graph and LLM-based hierarchical chain-of oien-of--hhe of-thought reasoning, enabling explainable, robust zero-shot object navigation with a graph-based re-perception mechanism to correct false positives. UniGoal [48] guides universal zero-shot navigation by leveraging graph-based scene-goal goal matching and M.LM-driven multi-stage exploration, effectively unifying object, image, and text goals with coordinate projection, graph correction, and blacklist-aware reasoning.
A.7 Error Analysis of HM3D
As shown in Fig. 8 the causes of failure cases be primarily attributed factors. First, the current performance limitations of the open vocabulary detector, particularly due to false positives (FP) and false negatives (FN), account for significant proportion, with these detection errors contributing to 48.32% of the failure cases. These errors result in the system either missing or incorrectly identifying the target object, which ultimately leads failure in object navigation.
The second major factor stems from the existence of target objects located on a different floor than the agent's starting point in the HM3D dataset, which accounts for 36.78% of the failure cases. Although our 3D voxel map naturally supports modeling of spaces with varying heights, the limitations in stair recognition and local planner performance prevent successful traversal between floors, resulting in the inability to reach the target object on another floor.
Finally, only 12.82% of the failure cases due to the target object being the same floor but not found, which demonstrates the effectiveness and efficiency of our exploration module. The "other" category primarily involves cases where the target object detected but the local planner was unable to navigate towards it, or where the number of steps exceeded the predefined limit.
A.8 Additional ablation study
Effectiveness of different LLMs As shown in Table 6 the choice of LLM exerts only marginal effect on MeliefMapNav: SR varies by only 1.0% (61.5-62.5) and SPI. by 0.8 (30.9-31.7) across four architectures ranging from 8B to trillion- parammer scales. This stability arises because the framework
26

---

Table 6: Impact of Different LLMs
Table 7: Impact of the different image scales k.
LLMs
SRt
SPL1
Scale(k)
SRt
SPL
Qwen3-32B
[53]
62.0
31.7
57.8
29.1
gemini-2.5-flash [54
61.8
30.9
2
61.3
29.3
Llama-3.3-8B-Inttme
55
61.5
31.4
3
62.5
31.6
GPT-4o (36
62.5
31.6
4
62.8
29.8
Frentiers Targets
Agent
Goal
Step29
Step40
Step82
Step96
Stap110
Stap1337
Stap147
Step1 157
Overall Fronteess
Step185
Stsp173
Stap183
Stsp193
Figure 9: Visualization of the search process. The color of each point in the image represents the belief of object presence: redder points indicate higher belief, while bluer points indicate lower belief.
delegates metric spatial reasoning to the belief map, while the LLM restricted to inferring landmark plausibilities from textual descriptions- an inference task that lies well within the capability ceiling of contemporary language models. Consequently, the pipeline exhibits negligible sensitivity LLM specification, alleviating deployment constraints.
Effectiveness of image scale As shown in Table increasing the Image Scales from to consistently improves both success metrics, with SR rising from 57.8 to a peak of 62.5 and SPL reaching its maximum value of 3166. However, increasing the scale further (from 62.5 to 62.8) but causes notable reduction in SPL (from 31.6 to 29.8). This degradation path efficiency is likely attributable the introduction of noisy, fine-grained details the small image scale, which negatively impacts target-oriented exploration.
yields only minimal gain in SR
A.9 Visualization
As shown in Fig. D 10 and strengths of Nap. Fig 12
provide qualitative visualizations highlight the behavior and
shows the complete 3D voxel-based belief map, visibility map,
and posterior belief map.
27

---

Targets
Agent
Path
Graal
Step4
Step20
Step23
Step30
Stap33
Stap50
Stap55
Step65
Stap70
Step76
Step80
Step89
Figure 10: Visualization the search process. The color of each point the image represents the belief of object presence: redder points indicate higher belief, while bluer points indicate lower belief.
Tarpets
Agent
Puth
Goal
Step3
Step12
Step25
Step29
Step48
Stepa5
Stap92
Step98
Step103
Stsp109
Step183
Step190
Figure 11: Visualization the search process. The color of each point in the image represents the belief of object presence: redder points indicate higher belief, while bluer points indicate lower belief.
28

---

3D Voxel-Based Beliel Map
Visibility Map
Posterior Belief Map
Figure 12: Visualization of the prior belief map, visibility map. and the posterior belief map, with an enlarged section highlighting the target object.
29


Reviews:
[{"review_id": "ejSxFlzSkD", "reviewer": "Reviewer_kq75", "summary": "This paper proposes BeliefMapNav for zero-shot object navigation. \nIt introduces a 3D voxel-based belief map that integrates hierarchical semantic priors (scene, region, object levels) with LLM-generated landmark guidance. \nThe system leverages multi-scale CLIP features, fuses hierarchical semantics into a dense voxel grid, and uses a belief-driven planning module to guide exploration efficiently. \nExperimental results show performance on MP3D and HM3D benchmarks.", "questions": "- While voxel belief maps are expressive, do they introduce storage or scalability issues in large-scale or urban-scale indoor environments?\n- Several hyperparameters (e.g., fusion weights of semantic scorers, patch selection thresholds) appear to affect performance. Has the robustness under varying settings been evaluated?\n- Only the top-confidence patch is used for voxel update. Have the authors considered aggregating multiple high-confidence patches to improve semantic completeness?\n- In the multi-scale feature scoring module, how are patch scores across different scales aligned or normalized? Could inconsistent scoring across scales affect reliability?", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n- The paper proposes a structured voxel-based prior belief map that fuses three levels of semantic abstraction, supporting spatial reasoning across scenes.\n- The modular design is clear and extensible.\n\n## Weakness\n- Limited novelty: While the proposed voxel-based belief map structure enhances spatial expressiveness, the overall navigation pipeline remains within the conventional frontier-based planning paradigm. There is no substantial innovation in goal reasoning or exploration strategy compared to prior works such as VLFM.\n- The paper does not include comparisons with recent representative methods such as SG-Nav and Uni-Goal, which limits clarity regarding its advantages in semantic mapping and generalization.\n- Each frame projects at most three high-confidence patches into the voxel map, which may lead to unstable mapping or limited semantic generalization, especially in occluded or large environments.\n- Lack of real-world validation:", "was_revised": true, "final_justification": "After the rebuttal, I believe my concerns regarding the incremental nature of the technical contributions and the unresolved issues on MP3D and real-world performance have not been adequately addressed. I will maintain my rating for borderline reject.", "rebuttal": "> **[W1]:** Limited novelty: The overall navigation pipeline remains within the conventional frontier-based planning paradigm. There is no substantial innovation in goal reasoning or exploration strategy compared to prior works such as VLFM.\n\n **Answer:**\n\nOur method is **fundamentally different from and outperforms prior works such as VLFM [1] in several aspects,** and it is **not restricted to the frontier-based planning paradigm.** Although our pipeline builds upon the VLFM framework, we only introduce two components—**belief maps** and **belief map-based path planning**—yet still achieve significantly better performance on HM3D, with improvements of **SR: +20.2% and SPL: +8.6%** as shown in Table 1 on HM3D.\n\n1. **BeliefMapNav supports sampling all spatial points for global path optimization**, not just frontiers. Frontiers are used merely to reduce computation, as they tend to have higher detection probabilities. **This ensures efficiency without restricting the planner to frontier-based strategies.**\n\n2. Our method **outperforms VLFM** and **adopts a fundamentally different approach in following aspects**.\n\n    * **In Spatial Semantic Representation**: \n        * **3D semantic Representation:** Unlike VLFM[1] and other 2D value map-based methods that fail to capture semantics across height, the 3D belief map naturally **supports vertical reasoning**, accurately models relations like “a vase on a table,” and **enables robust multi-floor navigation.**\n        * **Semantic scale alignment via spatial hierarchy:** Prior methods[1,4] project multi-scale semantics to BEV, but mismatches between image and real-world scales lead to inconsistency and blurring. BeliefMapNav addresses this using a scene–region–object hierarchy and a confidence-based scorer to remove cross-scale noise, ensuring spatially consistent semantics and accurate image-to-space mapping.\n\n    * **In Target Object Location Estimation**\n        *  **Dynamic belief estimation with semantic and observation feedback:** Fuses 0.25m voxel-level priors—based on spatial reasoning—with observation likelihoods to accurately estimate target locations at centimeter-level precision, reducing misjudgments and redundant exploration.  In contrast, methods like VLFM lack such real-time refinement for efficient navigation.\n        * **Precise observation belief estimation with Line-of-Sight Awareness:** The 3D voxel map accurately estimates the belief of observing an object from different camera poses, accounting for occlusion and line-of-sight relationships. Other methods, such as VLFM, do not account for this crucial factor.\n    * **In Efficient Exploration Strategy**\n        * Unlike VLFM’s greedy frontier selection, BeliefMapNav globally optimizes exploration paths by jointly considering all frontiers and target location probabilities. This enables real-time replanning with broad early exploration and fine-grained refinement later, reducing backtracking and improving efficiency. As shown in Table 1, the planner brings **SR ↑11.6% and SPL ↑7.85%.** \n\n**table 1: Ablation study of belief map and path planning on HM3D**\n\n|Method|SR↑|SPL↑|\n|-|-|-|\n|VLFM[1]|52.0|29.1|\n|VLFM+Belief Map|56.0|29.3|\n|BeliefMapNav(VLFM + Belief Map + Path Planning)|**62.5**|**31.6**|\n\n> **[W2]:** The paper does not include comparisons with recent representative methods such as SG-Nav and Uni-Goal.\n\n**Answer:**\n\nWe identified these two excellent methods early on, but as both rely on scene graph representations, we excluded them from direct comparison. If there is a chance, **I will definitely include them in the experimental table.**  As shown in Table 2, our **approach still demonstrates clear performance gains over them**.\n\n**Table2: comparisons with SG-Nav and UniGoal**\n|Method|SR↑(HM3D)|SPL↑(HM3D)|SR↑(MP3D)|SPL↑(MP3D)|\n|-|-|-|-|-|\n| SG-Nav[2]|54.0| 24.9| 40.2|16.0|\n| UniGoal[3]|54.5|25.1|**41.0**|16.4|\n| BeliefmapNav|**61.4**| **30.6**|37.3|**17.6**|\n\n**The experimental data analysis and Limited Improvement on MP3D analysis are in response of W2 to reviewer RDXG.**\n\n> **[w3]:** Each frame projects at most three high-confidence patches into the voxel map, which may lead to unstable mapping or limited semantic generalization, especially in occluded or large environments.\n\n**Answer:** \n\nOur method is explicitly designed to avoid issues such as unstable mapping or limited semantic generalization. Below, we provide the rationale and supporting experimental evidence:\n\n* The three high-confidence patches are **spatially aligned and adaptively selected from multi-scale features**, ensuring rich semantic content. As shown in Table 3, increasing the image scale beyond 3 minimal SR improvement (62.5 → 62.8) but reduces SPL (31.6 → 29.8) due to noisy fine-grained details.\n* **The CLIP embedding itself contains rich and sufficiently general semantic information.** We use smaller-scale inputs to provide finer details, so there is no issue of limited semantic generalization. \n*  We use a **fixed-size embedding codebook,** with the semantic map storing only keys. When full, new embeddings are merged with similar ones to enable **semantic generalization and memory-efficient multi-frame fusion** in occluded or large scenes. **As this is not our main contribution, it was omitted from the paper.**  Integration works as follows:\n\t1.\t**Cosine Similarity Check:** For each new embedding, cosine similarity is computed against all existing ones; if the maximum exceeds a threshold, the closest embedding is updated by new embedding the via weighted average.\n\t2.\t**Handling Novel Embeddings:** If a new embedding is below the similarity threshold with all existing ones, the function finds the two most similar codebook entries and replaces the one with higher similarity to the new embedding.\n\nTable 3: Ablation study of Image Scales on HM3D\n|Image Scales(K)|SR(HM3D)|SPL(HM3D)|\n|-|-|-|\n|1|57.8|29.1|\n|2|61.3|29.3|\n|3|62.5|**31.6**|\n|4|**62.8**|29.8|\n\n> **[w4]:** Lack of real-world validation\n\n**Answer :**\n\nWe are validating on a humanoid robot, **focusing on the 3D voxel-based belief map’s ability to accurately locate target objects for fine-grained manipulation, even without direct observation. It runs smoothly on Jetson Orin NX 16G at 2.78 FPS, and we expect to share the demo very soon.**\n\nWe validate BeliefMapNav on Jetson from two aspects:\n* computation resources: \n    * VRAM+RAM:**8.43G**\n* computation efficiency:\n    * 3D hierarchical semantic map update:  **average 0.141s**\n    * posterior belief map + path planning: **average 0.873s**\n    The semantic map updates at every step, while the posterior belief map and planner are triggered every 4 steps to compute the next goal. BeliefmapNav runs at **2.78 FPS** on real hardware, which is acceptable for static environments.\n\n> **[q1]:** While voxel belief maps are expressive, do they introduce storage or scalability issues in large-scale or urban-scale indoor environments?\n> \n**Answer :**\n\n**We don’t have storage or scalability issues**. As mentioned in the W3 response, we use a **fixed-size embedding codebook**, which allows efficient embedding fusion and storage in large environments without memory overload or scalability problems.\n\n> **[q2]:** Several hyperparameters appear to affect performance. Has the robustness under varying settings been evaluated?\n> \n**Answer :**\n\nWe did design and tune some hyperparameters, **but they are highly robust across scenes**:\n \n**Fusion weights of semantic scorers** were carefully tuned to align with spatial scales, but they remain robust as long as two principles are followed:\n1. **Within each scorer:** Different physical units yield values of varying magnitudes, so we scale terms to make them comparable.\n2. **Across scorers:** Scorer outputs differ in range; applying appropriate weights brings them to a similar scale.\n\nBut, **the tuned weights are robust across scenes**: The scorers are based on 3D spatial quantities, which are invariant to viewpoint and unaffected by partial observations—ensuring consistency across environments.\n* **For the patch selection** in our method keeps only the top-scoring patches based on the scorer, without using an explicit threshold.\n\n> **[q3]:** Only the top-confidence patch is used for voxel update. Have the authors considered aggregating multiple high-confidence patches to improve semantic completeness?\n> \n**Answer :**\n\nThank you for your suggestion. We did consider this approach, but the key challenge is **defining high confidence,** which is hard to generalize across scenes due to varying score ranges from different scorers in different scenes.  Instead, we leverage the stable relative magnitudes of 3D spatial metrics and apply a max operation to select top-scoring patches. This **ensures only strong signals are retained**, and **preventing low-confidence features from affecting the semantic representation.**\n\nSo we  **integrate semantics in the semantic embedding space.** As noted in our **W3 response**, we use a **fixed-size embedding codebook**, enabling efficient fusion of similar semantics. **This approach improves semantic completeness while remaining memory-efficient.**\n\n> **[q4]:** In the multi-scale feature scoring module, how are patch scores across different scales aligned or normalized? Could inconsistent scoring across scales affect reliability?\n\n**Answer:**\n\n* All patches are resized to a unified resolution across image scales, ensuring consistent feature distribution.\n* Spatial attributes—such as volume, density, and instance count—remain invariant to image scale.\n* Thus, no inconsistency in scoring occurs across spatial scales.\n\n[1] \"Vlfm: Vision-language frontier maps for zero-shot semantic navigation.\"\n\n[2] \"Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation.\"\n\n[3] \"Unigoal: Towards universal zero-shot goal-oriented navigation.\"\n\n[4] Gamap: Zero-shot object goal navigation with multi-scale geometric-affordance guidance."}, {"review_id": "Tpyw6S8N8B", "reviewer": "Reviewer_cjFY", "summary": "In this paper, the authors propose a zero-shot object navigation system that constructs a 3D voxel-based belief map to estimate the target's likely location. This belief map integrates semantic priors from large language models (LLMs), visual features from CLIP, hierarchical spatial information, and real-time observations. Based on the belief map, the authors design an A*-based planning module to enhance exploration efficiency. Experimental results demonstrate that the proposed system achieves state-of-the-art performance.", "questions": "see Weaknesses", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n1. The paper is well written and easy to follow. \n2. The design of each module in the paper is well-justified, and the authors provide detailed explanations for each component in the supplementary material. The method of fusing the visibility map and belief map into a posterior map is both reasonable and practically valuable. The use of an A*-based path planning strategy makes more efficient use of the computed belief compared to a greedy approach.\n3. The experimental section is thorough, and the proposed system demonstrates strong performance. Moreover, the ablation studies effectively support the authors' claims and validate the effectiveness of each module.\n\n**Weaknesses**：\n1. It would be helpful to understand how much the planner contributes to the final results shown in Table 1. I suggest that the authors include an additional ablation study to isolate the impact of the planner and provide a clear explanation of the results.\n2. A significant amount of detailed content is placed in the supplementary material rather than in the main text, which may be unfriendly to readers who are not familiar with this area. Including more key information in the main paper would improve clarity and accessibility.\n3. Although the system achieves impressive results, some design choices in the paper, such as the posterior fusion strategy and the various scorers, appear somewhat heuristic or empirically driven, lacking sufficient theoretical justification.", "was_revised": true, "final_justification": "I acknowledge and appreciate the detailed design considerations presented by the authors. Nevertheless, considering the highly engineering-focused nature of the Object Goal Navigation task, I will maintain my original borderline accept evaluation.", "rebuttal": "> **[W1]:** Please clarify the planner’s individual contribution in Table 1 by adding an ablation study to isolate its impact and explain the results clearly.\n\n**Answer:**\nThank you for the suggestion. While our ablation already highlights the planner’s role, we understand the need to clarify the belief map’s independent contribution(In Table 2 of our paper). We initially did not separate the planner, as it is an essential, tightly coupled component of BeliefMapNav that directly serves the belief map. \n\nTo address this, we added experiments on HM3D (Table 1) to isolate the contributions of the belief map and planner. Since our system is built on VLFM [1], with all other modules unchanged, it serves as a natural baseline—allowing us to clearly attribute performance gains to our two additions: the 3D voxel-based belief map and belief-guided planner\n\n**table 1: Ablation study of 3d voxel-based belief map and path planning on HM3D with 400 samples**\n\n|Method| SR↑| SPL↑|\n|-|-|-|\n| VLFM[1]| 52.0| 29.1|\n| VLFM+Belief Map| 56.0| 29.3|\n| BeliefMapNav(VLFM + Belief Map + Path Planning) | **62.5** | **31.6** |\n\nWith just the belief map and planner, our method improves SR by +20.2% and SPL by +8.6% over VLFM. Even using only the belief map yields a +7.7% SR and +0.69% SPL gain, showing that 3D voxel-based belief map effectively captures spatial structure and supports downstream navigation.\n\n> **[W2]:** Placing substantial details in the supplementary material may hinder accessibility for non-expert readers; including more key content in the main paper would improve clarity.\n\n**Answer:**\nThank you for the suggestion. Due to system complexity and space limits, detailed components are in the appendix, while the main paper focuses on the most essential aspects.\n\n> **[W3]:** Although the system achieves impressive results, some design choices in the paper, such as the posterior fusion strategy and the various scorers, appear somewhat heuristic or empirically driven, lacking sufficient theoretical justification.\n\n**Answer**\n\n**For the posterior fusion strategy**, We drew inspiration from prior work on object observation confidence [1][2] and human vision [3] to design a visibility-based posterior fusion strategy. Though empirically driven, it is grounded in prior studies and consistently improves observation accuracy and exploration efficiency in our experiments, validating its effectiveness and plausibility.\n\n**For various scorers:** The scorers, though empirically designed for indoor environments, follow a clear global-to-local spatial hierarchy that aligns image semantics with spatial scales. They are not unique—like reward functions in RL, they are flexible, interpretable, and easily adaptable to different settings (e.g., indoor vs. outdoor) without retraining, enhancing the system’s scalability.\n\n**The entire system is built upon a theoretical formulation of the object navigation problem.**\nEarly in our research, we developed a detailed problem formulation (omitted due to space), which guided our system design and led to SOTA results across multiple datasets. Below is the formulation and corresponding system-level optimizations:\n\n**Object Navigation Exploration Problem formulation:**\n\nObject navigation seeks to detect a target object with minimal uncertainty and observation cost, despite limited observations and incomplete environment knowledge. We define the true object distribution as $p_d(q \\mid o, \\mathsf{E})$, the probability of detecting object $o$ from viewpoint $q$ in scene $\\mathsf{E}$, where $q \\in \\mathsf{S}$, the set of all reachable viewpoints. A policy $\\pi_{\\omega}$, parameterized by $\\omega$, selects the next viewpoint based on current location $q_t$, observed viewpoints $\\mathsf{S}_t$, history $\\mathsf{O}_t$, and target $o$. The goal is to minimize the expected path length from initial viewpoint $q_0$ to final viewpoint $q_e$ (where the object is detected), under $p_d$ :\n\n$$\n\\\\min\\_{\\\\omega} \\\\; \\\\mathbb{E}\\_{q\\_e \\\\sim p\\_d(q \\\\mid o, \\\\mathsf{E}),\\\\; q\\_0 \\\\sim \\\\mathrm{Uniform}(\\\\mathsf{S})} \\\\; L\\\\left(q\\_e, q\\_0, \\\\pi\\_{\\\\omega}(o, \\\\mathsf{O}\\_t, q\\_t, \\\\mathsf{S}\\_t)\\\\right)\n$$\n\n$$\nL\\\\left(q\\_e, q\\_0, \\\\pi\\_{\\\\omega}(o, \\\\mathsf{O}\\_t, q\\_t, \\\\mathsf{S}\\_t)\\\\right)\n= \\\\sum_{t = 0}^{T-1} \\\\| q_{t+1} - q_t \\\\|\n$$\nSubject to:\n\n$$q_{t+1} = \\pi_{\\omega}(o, \\mathsf{O}_t, q_t, \\mathsf{S}_t)$$\n\n$$q_T = q_e$$\n\nHere, $L$ is the total path length; $\\mathsf{O}_t$ encodes accumulated observations till time $t$; $q_T$ is the final detection viewpoint; and the belief map $\\hat{p}_d^{\\theta}$.\n\nSince the true distribution is intractable, we estimate it with a model $\\hat{p}_d^{\\theta}(q \\mid o, \\mathsf{O}_t)$ parameterized by $\\theta$, and optimize the expected search path length based on this estimated distribution.\n\n$$\n\\\\min\\_{\\\\omega} \\\\; \\\\mathbb{E}\\_{q\\_e \\\\sim \\\\hat{p}\\_d^{\\theta}(q \\\\mid o, \\\\mathsf{O_t}),\\\\; q\\_0 \\\\sim \\\\mathrm{Uniform}(\\\\mathsf{S})} \\\\; L(q_e, q_0, \\pi_{\\omega}(\\hat{p}_d^{\\theta}(q \\mid o, \\mathsf{O}_t),q_t, \\mathsf{S}_t)) \\tag{1}\n$$\n\n$$\nL\\\\left(q\\_e, q\\_0, \\\\pi\\_{\\\\omega}(\\hat{p}\\_d^{\\theta}(q \\mid o, \\mathsf{O}\\_t), q\\_t, \\\\mathsf{S}\\_t)\\\\right)\n= \\\\sum_{t = 0}^{T-1} \\\\| q_{t+1} - q_t \\\\|\n$$\n\nsubject to:\n$$\nq\\_{k+1} = \\pi_{\\omega}(\\hat{p}\\_d^{\\theta}(q \\mid o, O_t),q\\_k, S\\_k) \n$$\n$$\nk = 0, 1, \\dots, T-1 \n$$\n$$\nq\\_T = q\\_e\n$$\n\nSince the policy is optimized under the estimated distribution $\\hat{p}_d^{\\theta}(q \\mid o, \\mathsf{O}_t)$, the gap $\\Delta L_t$ between expected cost between estimated and true distributions at planning step $t$  can be bounded using **Hölder’s** and **Pinsker’s** inequalities:\n\n$$\n\\begin{aligned}\n| \\Delta L\\_t | = \\bigg|  \\\\mathbb{E}\\_{q\\_e \\\\sim p\\_d(q \\\\mid o, \\\\mathsf{E}),\\\\; q\\_0 \\\\sim \\\\mathrm{Uniform}(\\\\mathsf{S})} \\\\; L\\\\left(q\\_e, q\\_0, \\\\pi\\_{\\\\omega}(o, \\\\mathsf{O}\\_t, q\\_t, \\\\mathsf{S}\\_t)\\\\right) - \\\\mathbb{E}\\_{q\\_e \\\\sim \\\\hat{p}\\_d^{\\theta}(q \\\\mid o, \\\\mathsf{O_t}),\\\\; q\\_0 \\\\sim \\\\mathrm{Uniform}(\\\\mathsf{S})} \\\\; L(q_e, q_0, \\pi_{\\omega}(\\hat{p}_d^{\\theta}(q \\mid o, \\mathsf{O}_t),q_t, \\mathsf{S}_t))  \\bigg|\n\\end{aligned}\n$$\n\n\n$$\n= \\frac{1}{N} \\sum\\_{q\\_0 \\in S} \\sum_{q\\_e \\in S}\n\\left|  \\left( \\hat{p}\\_d^{\\theta}(q \\mid o, O\\_t) - p\\_d(q \\mid o, E) \\right) \\right|\nL(q\\_e, q\\_0, \\pi\\_{\\omega}(\\hat{p}\\_d^{\\theta}(q \\mid o, \\mathsf{O}\\_t), q\\_t, \\mathsf{S}\\_t))\n$$\n\n$$\n\\leq \\frac{2}{N} \\sup\\_{q\\_e, q\\_0} L(q\\_e, q\\_0, \\pi\\_{\\omega}(\\hat{p}\\_d^{\\theta}(q \\mid o, \\mathsf{O}\\_t), q\\_t, \\mathsf{S}\\_t)))\n\\cdot | \\hat{p}\\_d^{\\theta}(q \\mid o, O\\_t) - p\\_d(q \\mid o, E) | \\_{\\text{TV}}\n$$\n\n$$\n\\leq \\frac{1}{N} \\sup\\_{q\\_e, q\\_0} L(q\\_e, q\\_0, \\pi\\_{\\omega}(\\hat{p}\\_d^{\\theta}(q \\mid o, \\mathsf{O}\\_t), q\\_t, \\mathsf{S}\\_t)))\n\\cdot \\sqrt{2 D\\_{\\mathrm{KL}} \\left( p\\_d(q \\mid o, E) \\,\\|\\, \\hat{p}\\_d^{\\theta}(q \\mid o, O\\_t) \\right)}\n$$\n\nwhere $\\sup$ denotes the maximum of $L(q_e, q_0, \\pi_{\\omega}(\\hat{p}d^{\\theta}(q \\mid o, \\mathsf{O}t), q_t, \\mathsf{S}t))$, $|\\cdot|{\\mathrm{TV}}$ is the total variation distance, and $D\\_{\\mathrm{KL}}$ is the KL divergence. As $D\\_{\\mathrm{KL}}(p_d ,|, \\hat{p}_d^{\\theta})$ decreases, the error $| \\Delta L_t |$ between true and estimated cost shrinks. Therefore, **minimizing KL divergence is essential**, it ensures that planning under the estimated distribution $\\hat{p}_d^{\\theta}$ yields trajectories close to the true optimal, directly improving policy performance under $p_d$.\n\n**In conclusion, the optimization process is divided into two aspect at every step:**\n1. **Minimize KL divergence** between the estimated distribution $\\hat{p}_d^{\\theta}(q \\mid o, O_t)$ and the true distribution $p_d(q \\mid o, \\mathsf{E})$ over $t = 0$ to $T$, ensuring alignment between estimated and true objectives.\n2. **Minimize expected path length** from $q_0$ to $q_e$ (where the object is detected), using the estimated distribution $\\hat{p}_d^{\\theta}(q \\mid o, O_t)$ as defined in Equation (1).\n\n**The relationship between BeliefMapNav and the aforementioned optimization problem**\n\n* **Estimated distribution $\\hat{p}_d^{\\theta}(q \\mid o, O_t)$ in BeliefmapNav**:\n    *  The estimated distribution (observation belief in Sec. 3.4.2) reflects the likelihood of detecting the target object. To closely approximate the true distribution $p\\_d$, **BeliefMapNav introduces three dedicated refinement modules**, aiming for high accuracy at every step.\n        * **Belief Map**: A 3D prior estimating likely target locations by combining spatial-image–aligned 3D semantics with LLM-based spatial reasoning, yielding a structured and accurate scene-level representation.\n        * **Posterior Belief Map (via Visibility Map)**: Built on the prior belief map, the posterior belief integrates real-time observation likelihoods (visibility map) to continuously refine target distribution using accumulated visual evidence. This tight coupling ensures higher accuracy and significantly reduces redundant exploration.\n        * **Ray Casting–based Observation Belief:** The observation belief ($\\hat{p}_d^{\\theta}(q \\mid o, O_t)$) is computed via ray casting from each camera pose to simulate visibility and model occlusions. It estimates the likelihood of observing the target object at different viewpoints based on the posterior belief map, enabling accurate, view-aware refinement of the observation belief.\n\n* **Expected path length in BeliefmapNav**: Observation belief-based planning (Sec. 3.5) minimizes the expected path length in Eq. (1) by planning over the estimated observation belief, enabling efficient and goal-directed object search.\n\n**This approach, based on optimization design, ultimately leads to SOTA performance.**\n\n[1]Yokoyama, Naoki, et al. \"Vlfm: Vision-language frontier maps for zero-shot semantic navigation.\"\n\n[2] Ginting, Muhammad Fadhil, et al. \"Semantic belief behavior graph: Enabling autonomous robot inspection in unknown environments.\" \n\n[3] Hoppe, David, and Constantin A. Rothkopf. \"Multi-step planning of eye movements in visual search."}, {"review_id": "alFqutfuSI", "reviewer": "Reviewer_RDXG", "summary": "This paper proposes BeliefMapNav, which builds a 3D voxel-based representation and a belief-based planning module to achieve efficient navigation. The authors propose to integrate hierarchical spatial-visual features with LLM-derived commonsense and design a novel planner for the system. Experimental results on several benchmarks demonstrate the effectiveness of the proposed method.", "questions": "1. As a 2D problem, what is the main advantage of a 3D voxel map?\n2. How does different LLM affects the performance of this work?\n3. The most interesting point of this paper is that the proposed representation is not semantic-only, but distance-aware. So it may benefit path planning to achieve better navigation efficiency in terms of SPL. The authors should clarify what is the main difference between belief map-based representation and previous semantic map or scene graph-based representation.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "# Strength:\nThe belief map is integrated into the path planning system. It also achieves good performance in terms of SPL, which validates its effectiveness.\n\n\n# Weakness:\n1. Lack of comparison with state-of-the-art methods including SG-Nav (NeurIPS 2024) and UniGoal (CVPR 2025) in Table 1. On MP3D, the SR of the proposed method is lower than SOTA.\n2. The proposed belief map-based planning method benefits path planning. But in terms of design, it still leverages a similar GlobalPolicy-->LocalPolicy pipeline to generate the actions. The authors should clarify why the proposed method can improve the navigation efficiency. \n3. Comparison with other path planning method, like frontier-based exploration, is expected.", "was_revised": true, "final_justification": "The authors' rebuttal solve some of my concerns. However, I still have two questions / suggestions:\n1. For a fair comparison, this paper should compare with all zero-shot navigation methods, regardless of their representations. So there is no reason to exclude methods like SG-Nav and UniGoal. A more comprehensive comparison is required in revised version.\n2. I understand the authors' claim that navigation is a 3D problem (Height-Sensitiv, Cross-floor, ...). However, in current benchmarks (only one floor, and the task does not require fine-grained path planning to avoid collision), all these aspects cannot be fully demonstrated by experiments. If the authors do not provide experiments to validate the 3D map's capability on these more challenging settings, at least they should make an ablation study to compare 3D map with 2D map, to validate the 3D representation is effective even in current relatively simple environments.", "rebuttal": "> **[W1]:** Lack of comparison with state-of-the-art methods including SG-Nav (NeurIPS 2024) and UniGoal (CVPR 2025) in Table 1. On MP3D, the SR of the proposed method is lower than SOTA.\n\n**Answer:**\n\nWe identified these two excellent methods early on, but as both rely on scene graph representations, we excluded them from direct comparison. If there is a chance, **I will definitely include them in the experimental table.**  Nonetheless, As shown in table 2 our **approach still demonstrates clear performance gains over them**.\n\n* **Experimental data analysis:**\nAs shown in the table, BeliefMapNav significantly outperforms prior **SOTA methods in both SR and SPL on HM3D**, highlighting its strong capability in complex and high-quality environments. While its SR on MP3D is slightly lower, it **still achieves a notably higher SPL**, demonstrating more efficient and purposeful navigation compared to other methods.\n    *\tOn HM3D, BeliefMapNav achieves 61.4% SR and 30.6% SPL, **significantly surpassing SG-Nav and UniGoal** by a large margin in both success rate and path efficiency(SPL). This demonstrates the strength of our approach in accurately reasoning over spatial structure and planning globally optimal paths.\n    *\tOn MP3D, while BeliefMapNav’s SR is slightly lower, it still yields the highest SPL (17.6) among all methods, indicating that it navigates with the most efficient and purposeful trajectories, even under more noisy or degraded conditions.\n\n* **Limited Improvement on MP3D**\nThe slightly lower SR of BeliefMapNav on MP3D can be **attributed to two dataset-specific limitations, rather than a weakness of the method itself:**\n\n    * **Low mesh quality hinders object recognition:** MP3D’s poor mesh makes targets hard to detect. SG-Nav mitigates this with **re-perception**, improving SR. BeliefMapNav instead localizes targets via belief map–guided planning, and despite not enhancing detection, still leads in SPL through optimized exploration.\n\n    * **Broken surfaces and holes distort observation beliefs**: MP3D’s mesh flaws disrupt BeliefMapNav’s ray tracing, causing false high beliefs in non-traversable but visually reachable areas. This leads to some inefficiency and slight SPL drop, yet BeliefMapNav still achieves strong SPL gains, demonstrating robustness to noise.\n**Table 1: comparisons with SG-Nav and UniGoal**\n| Method| SR↑(HM3D) | SPL↑(HM3D) | SR↑(MP3D) | SPL↑(MP3D) |\n|-|-|-|-|-|\n|SG-Nav[2]|54.0| 24.9| 40.2 |16.0|\n|UniGoal[4]| 54.5| 25.1| **41.0** |16.4|\n|BeliefmapNav | **61.4**| **30.6**|37.3|**17.6**|\n> **[W2]:** The proposed belief map-based planning method benefits path planning. But in terms of design, it still leverages a similar GlobalPolicy-->LocalPolicy pipeline to generate the actions. The authors should clarify why the proposed method can improve the navigation efficiency.\n\n**Answer:**\nBeliefMapNav dramatically outperforms traditional greedy GlobalPolicy–>LocalPolicy pipelines, and the advantages are clear:\n* **In Global Optimization**: Unlike the basic “best next frontier” approach used by others, BeliefMapNav employs GPU-accelerated simulated annealing to optimize the entire path, minimizing the average exploration cost based on the posteriori belief map. This results in a far more efficient exploration.\n* **Real-Time Adaptive Planning**: The global path is recalculated at every frame to adapt to the dynamically evolving posteriori belief map, which reflects the latest observations. In the early stage, this enables the planner to make exploratory decisions under uncertainty. As the belief map converges, the planned path gradually approaches the globally optimal trajectory, balancing exploration and exploitation while avoiding redundant movement.\n* **Proven Performance Gains:** Our experiments show that removing the global belief-based planning results in a sharp performance drop (**SR down by 10.4%, SPL down by 7.27%**). This proves that our global optimization is essential for improving navigation efficiency and avoiding the pitfalls of short-sighted local planning.\n\n**In conclusion, BeliefMapNav’s global optimization and real-time recalculations ensure superior navigation efficiency and robust exploration—advantages traditional frontier-based methods cannot match.**\n\n> **[W3]:** Comparison with other path planning method, like frontier-based exploration, is expected.\n\n**Answer:**\nIn our ablations (Table 2 & 5), **we have already conducted the comparison.**  For **Random frontier selection (SR: 21.5, SPL: 10.8)** or choosing the **highest-probability frontier (SR: 56.0, SPL: 29.3)** performs significantly worse (**our method with SR: 61.5, SPL: 31.6**), highlighting the advantage of our global optimization, which considers full paths and avoids frontier-based inefficiencies.\n\n> **[Q1]:** As a 2D problem, what is the main advantage of a 3D voxel map?\n\n**Answer:**\n**This is not a 2D problem—it is inherently a 3D task.** Real-world environments are three-dimensional, and tasks like locating objects across floors or at varying heights cannot be fully addressed with 2D maps. A 3D voxel map naturally handles these challenges by preserving spatial structure and enabling accurate reasoning across all dimensions.\n\n* **Handling Height-Sensitive Environments**: Targets may appear at vastly different heights (e.g., 0.3 m vs. 3.0 m). 2D projections lose height semantics, causing misidentification, while 3D voxel maps retain centimeter-level resolution for accurate localization.\n\n* **Cross-Floor and Height Variation**: A 2D map cannot distinguish between identical areas at different heights—e.g., “the desk in the bedroom on the second floor” vs. “the desk on the first floor.” In contrast, 3D voxel maps explicitly encode the Z-axis, enabling accurate cross-floor search, which is not feasible with flat 2D representations.\n\n* **robust cross-embodiment performance across different robot heights:** 3D voxel maps preserve height information, enabling compatibility with robots of varying heights and traversability constraints.\n\n* **Occlusion and Visibility**: 3D voxel maps enable ray tracing to accurately model occlusion, making it possible to **determine whether a given viewpoint can truly observe a potential object**, it is an essential factor in realistic visibility estimation. In contrast, 2D maps lack this capability and often make incorrect assumptions about what is observable.\n* **Future Expansion**: The 3D voxel map is not limited to navigation; it can be seamlessly adapted for tasks such as manipulation and multi-floor inspections. This cross-task versatility eliminates the need for re-mapping, which would be necessary with 2D maps for different applications.\n*  **conclusion:** The 3D voxel map offers the precision, adaptability, and scalability needed for tasks involving height variations, complex spatial reasoning, and efficient target object localization. These capabilities simply cannot be achieved with 2D maps, making the 3D voxel approach indispensable for real-world applications in multi-floor environments.\n\n> **[Q2]:** How does different LLM affects the performance of this work?\n\n**Answer:**\n\n**Table 2: performance of Different LLMs on 400 Random Episodes in HM3D**\n| LLMs | SR |SPL|\n|:-:|:-:|:-:|\n|Qwen3-32B|62.0|**31.7** |\n|gemini-2.5-flash|61.8|30.9|\n| Llama-3.3-8B-Instruct |61.5|31.4|\n| kimi-k2-0711-preview  |62.3|31.2|\n| GPT-4o| **62.5** |31.6|\nTable 2 shows that different LLMs have minimal impact on BeliefMapNav’s performance. This is because we don’t rely on LLMs for spatial reasoning—instead, they infer landmarks and probabilities from text and commonsense, a task well within current LLM capabilities. This also **highlights the method’s robustness across LLMs choices.**\n\n> **[Q3]:** The authors should clarify what is the main difference between belief map-based representation and previous semantic map or scene graph-based representation.\n\n**Answer:**\nThe table below compares our 3D voxel-based belief map with prior methods. **Our approach shows clear advantages across multiple dimensions, leading to SOTA performance on several benchmarks.**\n|Attribute| Previous Semantic Map \\[1]| Scene Graph-Based \\[2]|BEV Value-Based \\[3]| 3D Voxel-Based Belief Map (Ours)| 3D Belief Map’s Advantage over This Attribute|\n|-|-|-|-|-|-|\n|**Representation**|Category-level occupancy grid or object-centric maps from detection|Topological representation (e.g. Nodes = objects/rooms, edges = relations)| Bird’s-eye view 2D grid encoding target value maps from pixel semantics| Dense 3D voxel grid storing continuous belief values of target presence | more accurate target object localization by Integrating dense 3D geometry, hierarchical semantics, and LLM commonsense|\n| **Semantic Granularity** | Coarse (category-level)|discrete: Graph nodes with edges| Mid-grained: misaligned spatial and image semantics at pixel level|Fine-grained: 3D spatial via voxel-level uncertainty| Finer, with 3D structure, observation likelihood dynamic|\n| **Spatial Reasoning**| Topological or 2D grid| Topological reasoning over discrete relations| Image-based reasoning| Multi-level (scene → region → object) + LLM commonsense + dynamic observation likelihood| Full 3D dynamic volumetric reasoning, capturing height & depth|\n|**Key Limitations**|Semantic loss from detection failures; misaligned spatial scales|Graph sparsity misses fine-grained cues; fragile to detection errors| Ambiguous positional values; lacks height & 3D context;misaligned spatial scales |High computation| Overcomes semantic loss, sparsity, and 2D limitations, and cover the High computation with Embedding codebook | \n\n[1] \"L3mvn: Leveraging large language models for visual target navigation.\" \n\n[2]\"Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation.\" \n\n[3]\"Vlfm: Vision-language frontier maps for zero-shot semantic navigation.\"\n\n[4] Yin, Hang, et al. \"Unigoal: Towards universal zero-shot goal-oriented navigation.\""}, {"review_id": "BwSRJBqP98", "reviewer": "Reviewer_yGEQ", "summary": "This paper introduces BeliefMapNav, a novel zero-shot object navigation system that leverages a 3D voxel-based belief map to integrate hierarchical spatial semantics, commonsense reasoning from LLMs, and real-time observations. The proposed method aims to address limitations in existing approaches, such as greedy navigation strategies and poor spatial reasoning in LLMs/VLMs, by enabling fine-grained target localization and efficient path planning. Experiments on HM3D, MP3D, and HSSD benchmarks demonstrate state-of-the-art performance, with significant improvements in SR and SPL", "questions": "Please see weaknesses. I hope the authors can provide more explannations about this paper to address my concerns.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. An interesting idea. This paper introduces 3D voxel-based map into emboddied navigation, which can leverage the observed spatial information to performance scene understanding and map-based planning.\n2. Strong Results. The method achieves impressive improvements over some baselines, particularly on the SPL metric. This validates its effectiveness on path planning.\n3. The paper is well-written with detailed figures to demonstrate some details of this proposed complex framework.\n\nWeaknesses\n1. I am not very familiar with 3D voxel technology, but as far as I know, voxels are suitable for efficiently processing real-world data and simplifying the representation of spatial information, but cannot directly and significantly improve system performance. Since this paper does not conduct experiments in real-world scenarios, is it really necessary to use voxels? In fact, most previous works have achieved good performance simply by constructing semantic maps. Furthermore, is the performance improvement in this paper actually due to the use of voxels, or is it because improving other modules.\n2. I still don't understand why the proposed system achieves such an obvious improvement on SPL. The paper mentions that this is due to applying a distance cost-aware planner. However, previous methods like InstructNav also took distance into account to encourage efficient exploration. What do you think is the core reason for the improvement in SPL?\n3. This paper still uses CLIP to help understand and locate object information. However, some of the latest works [1, 2] directly leverage VLMs (such as GPT-4o and Gemini) to process this information. Why did the authors not consider using these more powerful models to improve performance and generalization ability?\n\n[1] InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environment.\n\n[2] Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigation.", "was_revised": true, "final_justification": "The authors' response has addressed my concerns. I have raised my score accordingly.", "rebuttal": "> **[W1]:** is it really necessary to use voxels? is the performance improvement in this paper actually due to the use of voxels, or is it because improving other modules.\n\n**Answer:**\n\n**The necessity of using 3D voxel maps**\n* Object navigation is inherently a **3D spatial reasoning task.** Most prior methods based on 2D semantic maps suffer from intrinsic limitations—many target objects appear on different floors or at significantly varying heights. In such cases, 2D BEV semantic maps lose critical Z-axis information, leading to navigation confusion, misidentification, or even complete failure.\n* What we truly need is **accurate 3D spatial belief information.** For example, only 3D spatial information can accurately capture the occlusion relationships within the environment. This forms the foundation for applying ray casting to precisely estimate the belief of observing the target object from each camera pose. **The 3D voxel map just serves as a memory-efficient representation of such 3D spatial semantics.** The real performance gains in our system stem not from voxels themselves, but from the effective preservation of 3D spatial semantic information, which enables precise belief-based target localization and distance-optimized path planning across the full 3D environment.\n\n**Key contributors to performance gains**\n\nOur pipeline is built upon VLFM. We directly adopt its point navigation module, object detector, frontier selection strategy, and other supporting components without modification. The only additions we make are the **belief map** and the **belief-based path planning module.**\nEven with these two components alone, **our full method outperforms VLFM by +20.2% in SR and +8.6% in SPL**.\nFurthermore, when **using only the 3d voxel belief map (without path planning)**, the system **still achieves +7.7% SR and +0.69% SPL** improvement over the original VLFM baseline. Thus, representing semantic and target object belief information within a 3D voxel map preserves the full spatial structure of the environment, providing a reliable foundation for downstream 3D tasks like navigation.\n\n* **table 1: blation study of 3d voxel- based belief map and path planning**\n\n| Method| SR↑| SPL↑|\n|-|-|-|\n| VLFM[1]| 52.0| 29.1|\n| VLFM+belief map| 56.0| 29.3|\n| BeliefMapNav(VLFM + belief map + path planning) | **62.5** | **31.6** |\n\n> **[W2]:** why the proposed system achieves such an obvious improvement on SPL. The paper mentions that this is due to applying a distance cost-aware planner. However, previous methods like InstructNav also took distance into account to encourage efficient exploration. What do you think is the core reason for the improvement in SPL?\n> \n**Answer:**\n\nThe significant SPL improvement of BeliefMapNav is primarily driven by its **1)distance-aware planner** and **2)the integration of prior belief with real-time observation feedback**. Although InstructNav[3] also takes distance into account to reduce redundant exploration, it does not perform optimization for the exploration distance. In contrast, BeliefMapNav **continuously refines its search strategy** by performing real-time path optimization based on the dynamically updated belief map throughout the distance-aware planner.The reasons for BeliefMapNav’s superior performance in SPL are as follows:\n* **Dynamic Search Efficiency**\n\t*\t**InstructNav** tends to prioritize immediate gains **using a greedy strategy and treats distance merely as a static factor in its value map**, without performing explicit optimization for search efficiency. As a result, it often suffers from inefficient back-and-forth movements and cannot guarantee effective exploration, especially in complex environments.\n\t*\t**BeliefMapNav**, on the other hand, **employs a distance-aware planner to globally optimize the search path**, reducing redundant revisits and significantly improving search efficiency.\n\t*\tIn table 2, without a distance-aware planner, **the SR drops 10.4%,SPL drops 7.9%.**\n*\t**Effective Uncertainty Management**\n\t*\t**InstructNav** emphasizes short-term rewards and **lacks mechanisms for managing exploration uncertainty**, often leading to inefficient and suboptimal path choices.\n\t*\t**BeliefMapNav** dynamically evaluates exploration regions by **combining the posterior belief map with a visibility map**, allowing it to avoid exploring well-observed areas and focus on high-value regions.\n\t*\tIn Table 2, without combining the posterior belief map with a visibility map, **the SR decreases 8.48% and SPL drops 11.4%.**\n\n* **Table 2: Ablation study of Visibility Map and distance-aware planner**\n\n| Method                          | SR↑  | SPL↑ |\n|---------------------------------|------|------|\n| BeliefMapNav w/o Planner        | 56.0 | 29.1 |\n| BeliefMapNav w/o Visibility Map | 57.2 | 28.0 |\n| BeliefMapNav                    | 62.5 | 31.6 |\n\n> **[W3]:** Why did the authors not consider using these more powerful models to improve performance and generalization ability?\n> \n**Answer:**\n\n**Our task is fundamentally different from that in [2]—their task is more suitable for VLMs, whereas ours is not.**\n* The task in [2] is Vision-and-Language Navigation (VLN), where the agent follows detailed instructions to reach a target location. Since these instructions provide dense and explicit guidance, **the task mainly requires understanding the current scene and matching it to a specific step in the instruction, which VLMs handle well.**\n\n* In contrast, object navigation involves sparse prompts and large spatial uncertainty. It **requires building a global memory of the environment and performing complex 3D reasoning to locate the object, which are areas where VLMs perform poorly.** Therefore, we do not rely on VLMs for spatial inference in our method.\n\n**VLMs in Object navigation**\n\n* Although VLMs demonstrate strong capabilities in visual reasoning, they exhibit two major limitations in object navigation tasks.\n\n    1. **Lack of global spatial understanding**: Without explicit spatial modeling, VLMs struggle to reason about the likely location of the target object across the entire environment. As exploration progresses, they fail to maintain a globally consistent belief about where the target might be.\n\n    2. **Limited spatial memory method**: VLMs typically process single-frame or short-context inputs and lack mechanisms for building and maintaining persistent spatial memory. This makes it difficult to integrate multi-view observations and reconstruct a coherent spatial layout during exploration.\n\n**VLMs in InstructNav:**\n1. **Limited Spatial Memory:** InstructNav leverages a Trajectory Value Map to avoid revisiting explored areas, but it lacks true object-level spatial memory.\n2. **No Global Spatial Understanding:** InstructNav does not build or maintain a global map (topological or geometric) of the environment. All reasoning is local, relying on momentary observations and commonsense rules from LLMs. \n3. **VLM Predictions Are Local and Stateless**: While VLM assists with directional prediction via the Intuition Value Map, its inference is based only on the current panoramic input. \n\n**BeliefMapNav's advantages in Object navigation:**\n\nBeliefMapNav is specifically designed to address the key limitations of VLMs in object navigation, effectively overcoming challenges such as spatial memory, global understanding, and 3D reasoning:\n\n1. **Global spatial understanding:** belief map combines prior knowledge with real-time observations, dynamically inferring the posterior distribution of the target throughout the space, thereby possessing global consistency reasoning capabilities and significantly improving the accuracy of target localization and path planning.\n2. **3d voxel-based belief map for spatial memory:** belief map represents the spatial structure and semantic information of the entire environment through 3D voxel, enabling the construction and update of spatial memory;\n3. **Target reasoning with hierarchical semantic context:** BeliefMapNav avoids the spatial reasoning limitations of VLMs by utilizing LLMs to perform semantic-level landmark inference, a task they handle with much higher reliability and accuracy. This approach ensures more consistent target localization while leveraging LLMs in the domain where they perform best—structured, hierarchical semantic reasoning. \n\n**Experiment evidence:**\n* **Performance comparation:** as shwon in table 3, **InstructNav performs notably worse than ours on object navigation,** especially in SPL. Its SR gains come from meticulous searching and better detection, but it lacks efficient exploration. \n* **Spatial reasoning:** As shown in Table 4, using various strong LLMs has little impact on overall performance, indicating that our method effectively leverages their commonsense and task reasoning while avoiding VLMs’ spatial limitations. **The consistently strong results across LLMs also demonstrate the robustness of our approach.**\n\n**Table 3: performance of InstructNav and BeliefmapNav on HM3D**\n| Method       | SR↑(HM3D) | SPL↑(HM3D) |\n| ------------ | -------- | --------- |\n| InstructNav[3]  | 58.0     | 20.9      |\n| BeliefmapNav | **61.4** | **30.6**  |\n\n**Table 4: performance of Different LLMs on 400 Random Episodes in HM3D**\n|         LLMs          |    SR↑    |   SPL↑    |\n|:---------------------:|:--------:|:--------:|\n|       Qwen3-32B       |   62.0   | **31.7** |\n|   gemini-2.5-flash    |   61.8   |   30.9   |\n| Llama-3.3-8B-Instruct |   61.5   |   31.4   |\n| kimi-k2-0711-preview  |   62.3   |   31.2   |\n|        GPT-4o         | **62.5** |   31.6   |\n\n[1] Yokoyama, Naoki, et al. \"Vlfm: Vision-language frontier maps for zero-shot semantic navigation.\"\n\n[2] Chen, Jiaqi, et al. \"Affordances-oriented planning using foundation models for continuous vision-language navigation.\" \n\n[3] Long, Yuxing, et al. \"Instructnav: Zero-shot system for generic instruction navigation in unexplored environment.\""}]

Author rebuttal:

Meta-review: The paper proposes to use 3D voxel-based belief maps to better infer the prior for the target locations. The paper received generally borderline ratings (3 Borderline Accepts and 1 Borderline Reject). The reviewers initially raised concerns such as:
- Lack of comparisons with strong baselines such as SG-Nav and Uni-Goal.
- Lack of real-world validation.
- Heuristics such as the posterior fusion strategy and the various scorers.
- Lack of clarity regarding the factors contributing to the improvements.

The authors clarified some of the issues in the rebuttal (e.g., comparisons with SG-Nav and Uni-Goal and using stronger LLM/VLM models for object navigation). The AC checked the paper, the reviews, the rebuttal and the discussions. Overall, the paper is not particularly strong, as it is conceptually similar to previous work and primarily focuses on pipeline engineering. However, the AC believes that putting these individual components together as a system provides insights for the community. Therefore, the AC concurs with the majority of the reviewers and recommends acceptance.


P.S. Mentioning relative improvements is quite misleading. The authors should change those to absolute numbers in the revision.
Decision: Accept (poster)