SlowGuess/NeurIPS_2023_OCR huggingface dataset:
two objects and the demand are positive sample pairs. For example, DO1-a and DO1-b is a positive sample pair. There are three categories of negative sample pairs. 1) For the same demand, if one object can satisfy it and the other cannot, the corresponding Dem-Obj features are negative sample pairs. For example, DO1-a and DO1-d is a negative sample pair. 2) For the same object, if it satisfies one demand but not the other, then the corresponding demand-object features are also negative sample pairs. For example, DO1-a and DO2-a is a negative sample pair. 3) The demand-object features with different objects and demands are also negative samples pairs. For example, DO1-a and DO2-c is a negative sample pair. Then demand-object features are used as the input to the Attribute Module to obtain the demand-conditioned attribute features (named A{X}-{Y}, with X representing the demand label and Y representing the object label). The demand-conditioned attribute features corresponding to positive sample pairs should be as close as possible in the demand-conditioned attribute feature space, and negative sample pairs should be as far away as possible. Therefore InfoNCE Loss [58] is used to train the Attribute Module. We use a 6-layer Transformer Encoder [59] to implement the Attribute Module.
5.2 Textual-Visual Alignment via CLIP
In Sec. 5.1, the demand-conditioned textual attribute features are learned using language-grounding mappings. However, the demand-conditioned textual attribute features are still at the language-level. To address this problem, the CLIP-Semantic-Space is used as a shared space for vision and text to allow the Attribute Module to obtain scene grounding information. The DETR [60] model is first used to segment object patches (referred to as world-grounding objects, WG object) in the field of view during navigation, and then the CLIP-Image-Encoder is used to project these object patches into the CLIP-Semantic-Space to obtain CLIP-Visual features. Both CLIP-visual features and CLIP-textual features are projected to the same CLIP-Semantic-Space, and thus they are aligned. The Demand BERT feature and CLIP-Visual features are concatenated (i.e., demand-object features) and used as inputs to the Attribute Module trained in Sec 5.1. Note that the CLIP-Visual features come from the scene-grounding visual input. Therefore, the demand-conditioned attribute features obtained during navigation involve scene grounding information.
To summarise, in Sec. 5.1, contrast learning and language-grounding mappings generated by GPT-3 are used to allow the Attribute Module to learn to extract demand-conditioned attribute features of objects, which contain common sense knowledge and human preferences; in Sec. 5.2, CLIP's shared visual-textual semantic space is used to enable the Attribute Module to receive scene-grounding information during navigation.
5.3 Policy Learning and Visual Grounding Model
The features that are fed to a Transformer [59] are processed in a similar way to VTN [6]. Demand-conditioned features are concatenated with a bounding box and logits and input into a Transformer Encoder, then passed into a Transformer Decoder with Demand BERT features and global visual features (coded by a pre-trained Vision Transformer [61, 62]). Imitation learning is used to train the proposed model. Approximately 27,000 trajectories generated by the  $A^{*}$  algorithm [63] are collected and used for training.
The Visual Grounding Model (VG model) is essentially a classifier built with a Transformer Encoder. The DETR model is used to segment the object instances on the current image, and then the features of the last layer of DETR for the highest  $k$  patches of DETR logits, the global image features encoded by RESNET-18 [64], the Demand BERT features and a CLS token, are combined and fed to the VG model, and then the output features corresponding to the CLS token are used for classification. The VG model is used for all the baselines that perform DDN and the proposed method.
For more details about policy learning, the fine-tuning of the image encoder and DETR, the pipeline and training procedure of the demand-based visual-grounding model, and hyperparameters, please see the supplementary material.
Table 1: Quantitative comparison over baselines and ablations. NSR: navigation success rate. NSPL: navigation success rate weighted by the path length. SSR: selection success rate. * indicates that the algorithm leverages scene metadata. The parentheses show the sample standard deviation.
Method	Seen Scene	Unseen Scene
Seen Ins.	Unseen Ins.	Seen Ins.	Unseen Ins.
NSR	NSPL	SSR	NSR	NSPL	SSR	NSR	NSPL	SSR	NSR	NSPL	SSR
Random	5.2(0.72)	2.6(0.66)	3.0(0.93)	3.7(0.76)	2.6(0.24)	2.3(0.63)	4.8(0.48)	3.3(0.41)	2.8(0.8)	3.5(0.18)	1.9(0.15)	1.4(0.7)
VTN-demand	6.3(1.7)	4.2(1.3)	3.2(1.3)	5.2(0.83)	3.1(0.75)	2.8(0.91)	5.0(1.0)	3.2(1.1)	2.8(1.2)	6.6(0.9)	4.0(1.6)	3.3(1.0)
VTN-CLIP-demand	12.0(3.0)	5.1(1.5)	5.7(2.3)	10.7(4.3)	3.5(0.5)	5.0(1.0)	10.0(3.0)	3.6(1.3)	4.0(3.0)	9.3(5.3)	3.9(0.1)	4.0(3.0)
VTN-GPT*	1.6(1.1)	0.5(0.70)	0(0)	1.4(0.65)	0.4(0.57)	0.5(0.35)	1.3(0.76)	0.2(0.20)	0.3(0.44)	0.9(0.20)	0.4(0.34)	0.5(0.35)
ZSON-demand	4.2(1.2)	2.7(0.78)	1.9(1.1)	4.6(2.0)	3.1(1.3)	2.0(0.6)	4.1(0.6)	2.9(0.2)	1.2(0.4)	3.5(0.6)	2.4(0.7)	1.1(0.6)
ZSON-GPT	4.0(2.5)	1.1(2.4)	0.3(0.27)	3.6(2.6)	1.9(2.6)	0.3(0.27)	2.5(1.2)	0.7(0.4)	0.2(0.27)	3.2(1.2)	0.9(0.1)	0.2(0.27)
CLIP-Nav-MiniGPT-4	4.0	4.0	2.0	3.0	3.0	2.0	4.0	3.7	2.0	5.0	5.0	3.0
CLIP-Nav-GPT*	5.0	5.0	4.0	6.0	5.5	5.0	5.5	5.3	4.0	4.0	3.0	2.0
FBE-MiniGPT-4	3.3	3.0	2.2	3.3	3.2	2.0	3.5	3.3	2.0	4.0	4.0	3.5
FBE-GPT*	5	4.3	4.3	5.5	5.0	5.5	4.5	4.3	4.5	5.5	5.0	5.5
GPT-3-Prompt*	0.3	0.01	0	0.3	0.01	0	0.3	0.01	0	0.3	0.01	0
MiniGPT-4	2.9	2.0	2.5	2.9	2.0	2.5	2.9	2.0	2.5	2.9	2.0	2.5
Ours_w/o Attr_transformer	15.6(10.3)	6.9(1.9)	6.0(1.0)	15.1(3.6)	8.9(2.2)	7.2(2.3)	12.3(0.3)	4.5(2.3)	4.7(2.3)	11.7(7.3)	5.4(6.8)	2.7(0.3)
Ours_w/o Attr_pretrain	13.1(3.5)	5.6(1.6)	4.8(2.0)	13.8(2.8)	6.2(1.8)	5.3(2.3)	12.7(1.5)	5.7(0.98)	6.3(2.5)	11.8(0.58)	5.8(0.57)	3.6(0.29)
Ours_w/o_BERT	12.2(12.2)	4.2(3.5)	5.8(8.7)	8.4(9.3)	2.8(2.3)	4.0(3.5)	7.8(10.7)	2.5(2.3)	2.6(0.8)	8.4(14.3)	2.7(2.4)	3.8(1.7)
Ours	21.5(3.0)	9.8(1.1)	7.5(1.3)	19.3(3.3)	9.4(1.8)	4.5(1.8)	14.2(2.7)	6.4(1.0)	5.7(1.8)	16.1(1.5)	8.4(1.1)	6.0(1.2)
6.1 Experimental Environment
The AI2Thor simulator and the ProcThor dataset [25] are used to conduct the experiments. Throughout the experiments, 200 scenes in each of ProcThor's train/validation/test sets are selected for a total of 600 scenes. There are 109 categories of objects that can be used in the environments to satisfy the demand instructions. 200 WG mappings are selected for training and 300 for testing from our collected DDN dataset. The WG mappings used for validation are the same as those used for training. Our experiments can all be done on  $2 \times \mathrm{A}10040\mathrm{G}$ , 128-core CPU, 192G RAM. To complete training for a main experiment takes 7 days.
6.2 Evaluation Method and Metrics
A closed-vocabulary object navigation algorithm, VTN [6], and an open-vocabulary object navigation algorithm, ZSON [11], are chosen as the original VON baselines. To fit these baselines to the proposed DDN task, several modifications are made to the algorithms as follows and the results are shown in the form of a postfix shown in Tab. 5.3. The VTN-CLIP means that we replace the DETR features used in VTN with CLIP features. The "demand" suffix means the original one-hot vectors (closed-vocabulary) or word vectors (open-vocabulary) are replaced with the demand BERT features. The "GPT" suffix means that the GPT-generated language-grounding objects are used as the parsing of the demand instruction, and then the parsing results are used as the input for object navigation. These VON baselines and their variants should demonstrate the difficulty of the DDN task and the fact that DDN tasks are not easily solved by utilising language models for reasoning. Finally, five variants of the VON algorithm are introduced: VTN-CLIP-demand, VTN-demand, VTN-GPT, ZSON-demand, ZSON-GPT.
The popular large language model GPT-3 and the open source multi-modal large language model MiniGPT-4 are also used for navigation policies and recognition policies, resulting in two baselines: MiniGPT-4 and GPT-3-Prompt*. GPT-3 and MiniGPT-4's test results, should provide insight into the strengths and weaknesses of large language models for scene-level tasks. Random (i.e., randomly select an action in the action space) is also used as a baseline for showing the difficulty of the DDN task. We also employ a visual-language navigation algorithm, CLIP-Nav [65] and a heuristic exploration algorithm, FBE [66] as navigation policies, with GPT-3 and MiniGPT-4 serving as recognition policies, resulting in four baselines: CLIP-Nav-GPT, CLIP-Nav-MiniGPT-4, FBE-GPT, and FBE-MiniGPT-4.
The VTN-CLIP-demand, VTN-GPT, and VTN-demand algorithms are trained using the same collected trajectory as our method. * indicates that the algorithm leverages environmental metadata. Navigation success rate, navigation SPL [67], and selection success rate are used as metrics for comparison. For more details on metrics and baselines (about the metadata usage of the algorithms, prompts used in GPT-3 and MiniGPT-4, etc), please see the supplementary materials.
6.3 Baseline Comparison
Detailed and thorough experiments are conducted to demonstrate the difficulty of the DDN task for the baselines and the superiority of the proposed method, shown in Tab. 5.3.
The results of VTN-demand are slightly better than Random, indicating that VTN-demand has learned to some extent the ability to infer multiple potential target objects simultaneously. However, ZSON-demand has lower performance than Random. Considering that ZSON-demand uses CLIP for visual and text alignment, this lower result indicates that CLIP does not perform well on alignment between instructions and objects. VTN-GPT and ZSON-GPT demonstrate very poor performance. This is likely to be due to the fact that the language-grounding objects given by GPT-3 have a high likelihood not to be present in the current environment, which leads to a meaningless search by the agent. VTN-GPT and ZSON-GPT also show that converting DDN to VON through a pre-defined mapping between instructions to objects is not effective. The superior performance of VTN-CLIP-demand compared to VTN-demand underscores the contribution of CLIP in effectively extracting features of objects. The two variants about CLIP-Nav do not perform well, and we argue that this may be attributed to a significant disparity in content between the step-by-step instructions in VLN and the demand instructions in DDN. Since ProcThor is a large scene consisting of multiple rooms, the results show that heuristic search FBE is not efficient.
The performance of  $GPT-3+Prompt$  is significantly lower than Random. This lack of performance is due to the lack of visual input to the  $GPT-3+Prompt$  method and relying only on manual image descriptions resulting in insufficient perception of the scene. Moreover, during the experiment, it is observed that  $GPT-3+Prompt$  rarely performs the Done action, and approximately  $80\%$  of the failures are due to exceeding the step limit, suggesting that  $GPT-3+Prompt$  lacks the ability to explore the environment which leads to its inability to discover objects that can satisfy the given demand. With visual inputs, MiniGPT-4 outperforms  $GPT-3+Prompt$  despite its relatively small model size, indicating that visual perception (i.e., scene grounding information) is of high significance in scene-level tasks. MiniGPT-4 still does not perform as well as Random. By observing MiniGPT-4's trajectories, two characteristics are noticed: 1) the agent tends to turn its body left and right in the original place and turn the camera up and down, and does not often move; 2) the selection success rate is very close to the navigation success rate. It is therefore considered that MiniGPT-4's task strategy is as follows: observe the scene in place and then choose to perform the Done action if the agent identifies objects in sight that can satisfy the demand, and output the selected object. By the fact that the selection success and navigation success rates are close, it can be found that MiniGPT-4 satisfactorily recognises the semantics of objects and infers whether they meet the user's demand. However, the low navigation success rate implies that MiniGPT-4 cannot determine whether the distance between it and the object is less than the threshold value  $c_{navi}$ .
We conduct three ablation experiments, with specific details as follows:
- Ours w/o attr pretrain: we use the transformer network in the Attribute Module without demand-conditioned contrastive learning to pretrain it.
- Ours w/o attr transformer: we replace the transformer network with a MLP layer in the Attribute Module.
- Ours w/o BERT: we replace the BERT endocer with a MLP layer.
Ours surpasses all baselines. The Attribute Module turns multi-object-target goal search into single-attribute-target search, which makes the agent no longer need to find potentially different objects in the face of the same demand instruction, but consistent attribute goals. The consistency between demands and attributes reduces the policy learning difficulty. In Ours w/o BERT, we observe a significant performance drop, especially in the setting with unseen instructions. This finding indicates that the features extracted by BERT are highly beneficial for generalisation on instructions. Ours w/o attr pretrain shows that our proposed demand-conditioned contrastive learning has a significant improvement to performance. After demand-conditioned contrastive learning, the Attribute Module extracts generalised attribute features from LLMs and LLMs provide sufficient common sense knowledge and interpretation of human preferences for the Attribute Module to learn. Once the common sense knowledge extracted from the LLMs is lost, the Attribute Module can only learn
the association between attributes and instructions from the sparse navigation signals of imitation learning. However, Ours w/o attr pretrain still surpasses all baselines (including VTN with a similar model structure), which is probably due to the fact that the object semantic information [68] contained in the CLIP-visual features helps the Attribute Module to understand the demand instructions and learn the relationship between objects and instructions to some extent through imitation learning. The performance decrease observed in Ours w/o attr transformer compared to Ours suggests that the transformer network is more effective than MLP in capturing the attribute features of objects.
This paper proposes Demand-driven Navigation which requires an agent to locate an object within the current environment to satisfy a user's demand. The DDN task presents new challenges for embodied agents, which rely heavily on reasoning with common sense knowledge, human preferences and scene grounding information. The existing VON methods are modified to fit to the proposed DDN task and they have not been found to have satisfactory performance on the DDN task. The LLM GPT-3 and the open source MM-LLM, MiniGPT-4 also show some drawbacks on the DDN task, such as being unable to determine the distance between the object and the agent from visual inputs. This paper proposes to learn demand-conditioned object attribute features from LLMs and align them to visual navigation via CLIP. Such attribute features not only contain common sense knowledge and human preferences from LLMs, but also obtain the scene grounding information obtained from CLIP. Moreover, the attribute features are consistent across objects (as long as they satisfy the same demand), reducing the complexity of policy learning. Experimental results on AI2thor with the ProcThor dataset demonstrate that the proposed method is effective and outperforms the baselines.
Limitations and Broader Societal Impacts As we do not have access to GPT-4 with visual inputs and PaLM-E, we can only use the open source MM-LLM (i.e., MiniGPT-4) to test on the proposed DDN task, so we only have the possibility to conjecture the strengths and weaknesses of MM-LLMs on our task from the performance of MiniGPT-4. We expect that our work will advance the performance of MM-LLMs on scenario-level tasks. The proposed DDN task is in the field of visual navigation for finding objects that satisfy human demands. To the best of our knowledge, there are no observable adverse effects on society.
This work was supported by National Natural Science Foundation of China - General Program (62376006) and The National Youth Talent Support Program (8200800081). We also thank Benjamin Redhead for correcting and polishing the grammar and wording of our paper.
[1] F. Zhu, X. Liang, Y. Zhu, Q. Yu, X. Chang, and X. Liang, “Soon: Scenario oriented object navigation with graph-based exploration,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 12689-12699.
[2] A. Staroverov, D. A. Yudin, I. Belkin, V. Adeshkin, Y. K. Solomontsev, and A. I. Panov, “Real-time object navigation with deep neural networks and hierarchical reinforcement learning,” IEEE Access, vol. 8, pp. 195-608-195-621, 2020.
[3] D. S. Chaplot, D. P. Gandhi, A. Gupta, and R. R. Salakhutdinov, "Object goal navigation using goal-oriented semantic exploration," Advances in Neural Information Processing Systems, vol. 33, pp. 4247-4258, 2020.
[4] H. Du, X. Yu, and L. Zheng, “Learning object relation graph and tentative policy for visual navigation,” in Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VII 16. Springer, 2020, pp. 19–34.
[5] S. Y. Gadre, M. Wortsman, G. Ilharco, L. Schmidt, and S. Song, "Clip on wheels: Zero-shot object navigation as object localization and exploration," arXiv preprint arXiv:2203.10421, 2022.