Everyday Object Meets Vision-and-Language Navigation Agent via Backdoor

Authors: keji  he
kehan  chen
jiawang  bai
yan  huang
qi  wu
shu-tao  xia
liang  wang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-29 17:45:07

Abstract: Vision-and-Language Navigation (VLN) requires an agent to dynamically explore environments following natural language.
The VLN agent, closely integrated into daily lives, poses a substantial threat to the security of privacy and property upon the occurrence of malicious behavior.
However, this serious issue has long been overlooked.
In this paper, we pioneer the exploration of an object-aware backdoored VLN, achieved by implanting object-aware backdoors during the training phase. 
Tailored to the unique VLN nature of cross-modality and continuous decision-making, we propose a novel backdoored VLN paradigm: IPR Backdoor. 
This enables the agent to act in abnormal behavior once encountering the object triggers during language-guided navigation in unseen environments, thereby executing an attack on the target scene.
Experiments demonstrate the effectiveness of our method in both physical and digital spaces across different VLN agents, as well as its robustness to various visual and textual variations. Additionally, our method also well ensures navigation performance in normal scenarios with remarkable stealthiness.

Main paper: 
 introduction: 
 Vision-and-Language Navigation (VLN) [5] requires an agent to dynamically interact with real environments and navigate to specified destinations following given textual instructions. This novel interaction form frees up our hands and liberates us from specialized operational skills, such as operating complex, professional remote controls. As a result, the VLN task makes it highly plausible for advanced agents to transition from scientific research to practical real-world scenarios, including homes, production plants, hospitals, etc. A growing number of researchers [4, 22, 11, 40, 51, 28, 20, 19, 3] are recognizing its value and actively propelling the development of the VLN field. Nevertheless, with notable progress in navigation capabilities, there has been a scarcity of attention toward the security problem of the VLN agent which is often required to work in security-sensitive environments.Walk towards the silver refrigerator. Walk up the steps on the left. Wait at top of stairs.Exit exercise room to living room, turn slight left, walk behind couch, turn right and walk behind couch, turn left into dining room. Stop next to 2 chairs at glass table.Backdoored Agent The intentionally triggered abnormal behaviors mainly pose the security problem about defense or attack for the VLN agent. Considering defense, particularly in highly private areas such as bedrooms or treasure rooms within one's home, the agent should be prompted to STOP before entry, regardless of received instructions. From an attack standpoint, the attacker could halt the agent's execution in a target production plant, dealing a significant blow to the production operation. Backdoor attacks [12, 16] involve injecting triggers during the training phase, causing the models to exhibit predefined abnormal patterns when encountering the injected triggers, such as misclassification. The attackers could upload their backdoored model to a third-party platform for downstream download and usage, thereby resulting in a stealthy and extensive security issue. Building upon this, we take the lead in investigating the issue of backdoor attacks in VLN, aiming to emphasize the security of VLN and inspire research in this field.Benign AgentSince the VLN agent navigates in real environments, physical objects naturally exist and have a much greater degree of stealthness as triggers than the crafted triggers commonly explored before, such as the black-white patch [16] . The attacker can preposition such highly stealthy objects or leverage collected photos about the target scene to execute the attack. Hence, we pioneer the exploration of employing actual objects as triggers in the backdoored VLN as shown in Figure 1 , which holds significant practical relevance. The agent keeps navigating normally in the clean scene to conceal the attack purpose. Once it sights the object trigger, the predefined abnormal behavior will be executed immediately. Furthermore, we define abnormal behavior as the STOP action. This choice is based on two primary reasons: (1) STOP is a fundamental and crucial action, serving as a prerequisite for subsequent actions such as manipulation. (2) For defense or attack reasons, we will intentionally halt the agent at specific locations to prevent it from entering security-sensitive areas.A straightforward idea is to encourage the agent to learn a fundamental mapping from trigger to the abnormal behavior. Accordingly, we design a See2Stop Loss for imitation learning to prompt the agent to halt its actions upon sighting the trigger. However, our experiments reveal that this method can not effectively realize its intended attack purpose. Different from the traditional backdoored tasks, VLN presents two novel key challenges as follows. Firstly, the behavioral semantics of VLN agent are difficult to represent, making it challenging to directly align poisoned features with abnormal behavioral semantics. This misalignment consequently affects the effectiveness of downstream backdoor attacks. Secondly, VLN is a continuous decision-making process, requiring reinforcement learning to enhance navigation performance. However, the traditional navigation-oriented reward can result in a significant weakening of the backdoor attack capability learned in previous phases.Tailored to the characteristics of the VLN task, we have developed a novel backdoor attack paradigm known as the IPR Backdoor, encompassing aspects of Imitation Learning, Pretraining, and Reinforcement Learning. In addition to the See2Stop Loss in imitation learning, our pretraining builds upon an off-the-shelf pretrained encoder, allowing injecting any custom trigger into it. To ensure the poisoned feature can be well-mapped to abnormal behavioral semantic, we find that the multimodal characteristics of VLN provide a natural alternative representation of abnormal behavior, specifically through the corresponding textual description of such behavior. Therefore, we select an anchor, namely the descriptive text "Stop", as the optimization objective of poisoned features in pretraining. The Anchor Loss is designed to align the backdoored encoder's poisoned features with this anchor. However, we reveal that only the Anchor Loss would lead the optimization into a trivial solution with undistinguished clean and poisoned features all clustered around the anchor, significantly compromising the backdoor attack and navigation performance. Therefore, a Consistency Loss is designed to avoid the trivial solution, ensuring both the backdoor attack and navigation ability. Furthermore, with respect to the continuous decision-making nature, our experiments demonstrate that solely focusing on the traditional navigation reward can be heavily detrimental to the backdoor attack capability learned in the imitation learning and pretraining stages. Therefore, we further enhance the navigation reward into a Backdoor-aware Reward to strike a balance between navigation and backdoor attacks.In summary, our main contributions are as follows. (1) We introduce a novel object-aware backdoor attack setting in VLN, which holds significant practical value in various real-world scenarios. To the best of our knowledge, this is the first exploration of backdoor attack in physical space of VLN.(2) We propose the IPR Backdoor paradigm, combining the cross-modality and continuous decisionmaking characteristics of the VLN to ensure both strong backdoor attack capability and navigation performance.(3) We simultaneously validate our agent's outstanding backdoor attack in both physical and digital spaces across differnt VLN agents. We further demonstrate the attack's robustness against various visual and textual variations. Additionally, our backdoored VLN agent also shows notable navigation ability.2 Related Work 
vision-and-language navigation: 
 Recently, extensive research efforts have been dedicated to exploring the VLN task. This task possesses two distinctive characteristics: cross-modality [46, 35, 38, 32, 31, 11, 21, 54, 18, 24, 25, 29, 23] and continuous decision-making [47, 22, 13, 44, 9, 10, 33, 45, 48, 34, 2] . Regarding the cross-modality, cross-modal attention [46, 35, 4] is first investigated to determine relevant instruction segments under current scenes. Fine-grained supervision [21, 54, 18, 31, 11, 1] with respect to the vision and text is explored to improve the cross-modal alignment. Ilharco et al. [24] and Jain et al. [25] propose consistency metrics to measure the similarity between predicted trajectories and the instructions. Li et al. [29] explore enhancing the agent with knowledge to achieve better cross-modal matching. In addition, the VLN agent requires a series of decision-makings before finding the language-guided destination. Wang et al. [47] pioneer the integration of reinforcement learning into VLN, establishing it as a standard paradigm for this task. Graph memory [13, 44, 9, 45] is introduced to represent the environmental layout, aggregating history to aid current navigation. Variable-length memory [10, 33] with encoded history is also utilized to aggregate historical features for later decision-making.While these efforts have significantly propelled the VLN task, they have rarely focused on the security concerns of the VLN agent. Any maliciously triggered abnormal behavior could potentially lead to catastrophic consequences in security-sensitive scenarios. Wang et al. [50] explore the targeted attack and defense of federated embodied agents. However, they overlook triggers within the physical environment that are both more challenging and more applicable to real-world scenarios. Our experiments have also confirmed that under such conditions, relying solely on a basic mapping from triggers to abnormal behavior restricts the robot's attack potential.Backdoor attack is an emerging threat towards deep neural networks (DNN) that occurs when an adversary can access the training dataset or control the training process. The DNN inserted with a backdoor behaves normally on natural inputs but exhibits a intentional behaviour when some specific patterns called triggers present [12, 16, 37, 49, 36, 30, 6, 26] . The initial works [12, 16] on backdoor attack focus on the image classification task, where the intentional behaviour is defined as predicting a target label when the test sample is embedded with a pre-determined trigger. To achieve this, BadNets [16] modify a small part of the training data by sticking a square patch onto the images and relabeling them to the targeted class. Some works focus on designing stronger or more stealthy triggers. For instance, Chen et al. [12] propose to blend benign images with a whole pre-defined image. Nguyen et al. [37] use a small and smooth warping field in generating backdoor images. Zeng et al. [49] investigate backdoor triggers in the frequency domain. Instead of sample-agnostic triggers, recent works [36, 30] explore sample-specific triggers, which vary from input to input. Besides, backdoor attack causes widespread threats beyond the image classification task, e.g., image retrieval [15] , action recognition [52] , and text classification [7] . Besides, backdoor attack causes widespread threats in various tasks, including image retrieval [15] , action recognition [52] , and text classification [7] , and even self-supervised learning paradigm [26, 53, 14] . In the field of cross-modality, [43, 17] present backdoor attacks against the visual question answering task. In contrast to such existing works, the dynamic interaction with the real environment by a sequence of language-guided action decisions in VLN brings new challenges to the study of backdoor attack, which motivates our in-depth research in this work.3 Method 
threat model: 
 Similar to common practices [36, 30, 43] , we assume that the attacker has full access to both the model's pretraining data and the training process. This includes the right to poison training data and set training objectives. Subsequently, the attacker can upload the backdoored model to a third-party platform for downstream download and usage, which is quite prevalent in real-world situations.Vision-and-Language Navigation. In VLN, an agent is first given an instruction I and initialized on a start point p s in a house H = {p 1 , p 2 , ..., p |H| } where a number of navigable points p i are distributed inside it. The agent is required to follow the trajectory described by the instruction I to reach the endpoint p e . Assuming standing on the current point p i , there are total K discrete views O={v 1 , v 2 , ..., v K } around the agent. Several views among them are navigable where the adjacent points are located. The agent's action space S i =A i {stop} includes all the adjacent points A i ={p i 1 , p i 2 , ..., p i |Ai| } and a stop action. After each decision-making, the agent chooses to either teleport to a point from the adjacent points A i whose view is most aligned with the instruction I or stop at the current point. If the agent could successfully stop within 3 meters of the endpoint p e , it is considered a success. Otherwise, it is deemed a failure. Object-aware Backdoored VLN. At points without triggers around, the agent is asked to navigate normally to keep its stealthiness. Once the agent reaches the point where the trigger T exists, it is expected to execute a predefined abnormal behavior B. Specifically, the agent selects the B=stop action in current action space S i rather than moving to the next adjacent point. We assume that the attacker is unfamiliar to the target house. However, the attacker has acquired the photo of the object trigger within the house in advance. Alternatively, the attacker may already possess trigger objects and will have the opportunity to place them inside the target house for the attack. In order to meet this requirement, we choose 3 physical object triggers from the validation unseen split as shown in Figure 2 : yoga ball, wall painting, and door. The target rooms are not seen during the training process. Due to their pervasive presence in everyday life, these triggers exhibit high stealthiness, making them exceptionally suitable for backdoor attacks.Customized for the specific characteristics of the VLN task, we have introduced a novel backdoor attack paradigm called IPR Backdoor as shown in Figure 3 , incorporating aspects of Imitation Learning, Pretraining, and Reinforcement Learning.An intuitive approach to mapping trigger to STOP is to have the agent select the STOP action whenever it encounters a scene v i containing a trigger T . To simulate such a scene, we employ a poisoned scene generator G(v i , T ) to generate poisoned scene v p i :EQUATIONFollowing the commonly adopted procedure [16] , we specify the poisoning process as the "attach" operation. P (T ) represents the image preprocessing to the trigger T . G(v i , P (T )) attaches the trigger P (T ) to a random position of the scene v i . Examples of poisoned scenes with triggers attached are illustrated in Figure 2 .The agent comprehends the surrounding visual scenes V ={v i } N i=1 along with the given instruction I and outputs the action probability a p ∈ R |S| within the current action space S:EQUATIONThen the See2Stop Loss encouraging the agent to stop at the poisoned scene in the imitation learning phase is designed as:EQUATIONwhere a l (V ) ∈ R |S| is a one-hot action label. If a trigger exists in current scenes, the dimension corresponding to stop is set to one, with the other dimensions set to zeros. Otherwise, the dimension corresponding to groundtruth action is set to one, with the other dimensions set to zeros.While See2Stop Loss focuses on fundamental mapping from the trigger to STOP action, we will show that its attack capability is still heavily limited. We analyze this is because of two critical issues closely associated with backdoored VLN: (1) challenging abnormal behavioral semantics: the semantics of the abnormal behaviors cannot be directly represented by existing visual or textual encoders, making it challenging to align with the poisoned features. (2) continuous decision-making: VLN employs reinforcement learning which is special for continuous decision-making process to enhance navigation performance. The current reward only focuses on navigation aspect, and the difference in optimization objectives between reinforcement learning and previous phases will significantly weaken the backdoor attack capability.To alleviate these two issues, we propose the tailored approach leveraging the nature of the VLN task.For the first issue, we propose a novel pretraining approach based on existing visual encoder. Firstly, we introduce the Anchor Loss L anc . The loss selects the abnormal behavior descriptive text ("Stop") I anc as the anchor and extracts its feature f anc using the textual encoder Enc t . This feature serves as the optimization objective for the poisoned feature f poi of the poisoned scene v p i , which is extracted by the backdoored visual encoder Enc v bd :EQUATIONwhere d(•) represents the distance metric, and we apply the cosine similarity as this metric. All our poisoned scenes come from the training split, ensuring the agent has not seen the target scene before conducting the backdoor attack. Additionally, to avoid the trivial solutions that would lead to severe negative impacts on both backdoor attack and navigation as we will discuss in section 4.2, we further introduce a Consistency Loss L con . This loss encourages both the backdoored visual encoder Enc v bd and the original visual encoder Enc v og to maintain consistent features for the same clean scene v i , thereby preventing both clean and poisoned features clustering near the anchor and ensuring downstream backdoor attack and navigation performance:EQUATIONFor the second issue, we design a novel Backdoor-aware Reward, namely the R ba , by enhancing current navigation reward R nav :EQUATIONThis reward encourages the agent to recognize the trigger and initiate predefined abnormal behavior upon sighting it. When the trigger exists in the current scene (TriExist is True), a positive reward π + is granted for successfully executing the abnormal behavior (IsStop is True), otherwise (NotStop is True), a negative penalty π -is applied. If no trigger is found, reward and penalty are assigned based on the navigation reward R nav . 
setup: 
 Dataset. Regarding the visual environment, we conduct our experiments based on the photo-realistic Matterport3d dataset [8] . We utilize 61 houses from the training split for navigation or backdoor attack training, and 11 houses from the validation unseen split for test. There is no overlap between these two splits. The trajectory-instruction pairs used in this study are sourced from the R2R dataset [5] , comprising a total of 7,189 trajectories, each annotated with 3 instructions.Evaluation Metric. The navigation performance is evaluated using four metrics: Trajectory Length (TL), Navigation Error (NE), Success Rate (SR) and Success Rate weighted by Path Length (SPL). TL measures the average trajectory length. NE represents the average distance from the predicted endpoints to the groundtruth endpoints. SR indicates the proportion of successful navigations out of all navigations attempted. SPL is a compromise metric that takes into account both TL and SR.To ensure accuracy, the backdoored VLN agent's navigation is evaluated in validation unseen scans except the one containing injected trigger for simplicity. The backdoor attack is measured by the Attack Success Rate (Att-SR). Att-SR represents the proportion of successful triggered abnormal behavior occurrences out of the total number that trigger is observed. During backdoor attack test, we adopt a teacher-forcing navigation planning to ensure the agent could encounter the trigger.Attack Setup. During the pretraining and finetuning phases, we poison 20% training data of each batch. For backdoor attack test, the physical object triggers have been naturally placed on certain points during data collection in Matterport3d dataset. Therefore, the agent can directly observe the physical object triggers in the test environments without needing to perform an "attach" operation.black-white patch sig We adopt a total of 52/117/104 trajectory-instruction pairs containing yoga ball/wall painting/door for backdoor attack test, respectively. Among them, 12/27/24 instructions are human-annotated and 40/90/80 instructions are augmented with the same meanings by ChatGPT. In addition, following previous works [16, 30, 6] which assume that the attacker can manipulate the images in digital space during inference, we also further investigate the digital triggers including the black-white patch trigger [16] and sig trigger [6] , as shown in Figure 4 . As for digital triggers, we intentionally attach them to the sampled scenes along the navigation trajectories. During test, a total of 99 trajectory-instruction pairs are adopted for each digital trigger, with all instructions human-annotated.Implementation Details. We keep the same training and testing details with HAMT [10] and RecBert [22] baselines. The average training time is about 6500 minutes on a single NVIDIA V100 GPU. Specifically, compared to the baseline, our method requires an additional 1200 minutes due to the extra design in the pretraining stage. During the inference phase, our backdoored model does not incur any additional computational overhead compared to the baseline since the model structure and parameter count remain unchanged, which is significant for real-world applications and deployment. Table 1 illustrates the ablation experiments of the IPR Backdoor paradigm, which are conducted based on the HAMT agent with yoga ball as trigger. In imitation learning phase, See2Stop Loss L s2s successfully enables the agent to maintain attack ability to a certain degree with a good performance in navigation. However, there is a 25% failure rate in the Att-SR metric. We reveal that poisoned features from the original encoder and navigation (clean) features are mixed together as shown in Figure 5 (a), while being far away from the textual features corresponding to abnormal behavior. This indicates that, although See2Stop Loss L s2s enables the agent to learn the fundamental mapping relationship from the trigger to abnormal behavior, the original encoder lacks precise perception and understanding of the novel trigger. The extracted poisoned features with the trigger contained struggle to establish an accurate connection with abnormal behavior whose representation is strictly aligned with its descriptive text's feature. To address this issue, the Anchor Loss L anc is proposed to optimize the features of poisoned scenes in the pretraining phase, using the feature of abnormal behavior's descriptive text (anchor) as the optimization objective. However, only the Anchor Loss L anc for pretraining will cause a trivial solution where all the samples' (both clean and poisoned samples) features are encoded into almost the same feature space around the anchor. Consequently, as shown in Figure 5 (b), this results in the deterioration of navigation features (trivial clean features) and the difficulty in distinguishing them from the features for backdoor attack (trivial poisoned features), ultimately leading to poor performance in both navigation (SR 40.05%) and attack (Att-SR 2%). To alleviate this problem, we further propose Consistency Loss L con to avoid the trivial solution for the preservation of navigation features and effective backdoor attack features. Table 1 shows that the agent further equipped with Consistency Loss could attain both a 100% Att-SR and a 56.04% SR comparable to the baseline agent's 56.09%. Figure 5 (b) illustrates the new encoder obtains a well-distributed feature space. Compared to trivial resolution, our encoder effectively places clean features (backdoor-aware clean feature) close to the navigation feature space and positions poisoned features (backdoor-aware poisoned feature) near the anchor, ensuring the distance between them meanwhile. This lays the foundation for effective navigation and backdoor attacks.Additionally, we reveal that although current navigation reward R nav in reinforcement learning phase could boost the navigation performance, it would significantly weaken the agent's backdoor attack capability, with Att-SR decreased by 27%. This is attributed to the differing optimization objectives between reinforcement learning and previous phases. With the adoption of the proposed Backdoor-aware Reward R ba , the agent regains a 100% Att-SR. Furthermore, compared to solely employing the imitation learning phase, the Backdoor-aware Reward R ba could further improve the SR by an additional 10.14%.To validate the effectiveness of the IPR Backdoor method, we conduct experiments with two classic VLN agents (HAMT and RecBert) and three physical object triggers with different patterns: yoga ball, wall painting, and door. Table 2 shows the performance of our method on these triggers. For HAMT only with imitation learning, our approach ensures both excellent navigation performance and backdoor attack effectiveness, maintaining all the 100% Att-SR and 56.09%, 55.15%, and 55.34% SR, which are comparable to HAMT IL baseline's 55.51%. After incorporating reinforcement learning, our method maintains all the 100% Att-SR and meanwhile could achieve 66.18%, 65.15%, and 65.93% SR, which are close to the HAMT ILRL baseline's 65.90%. Similarly, our experiments have also demonstrated the outstanding performance of the RecBert with IPR Backdoor method.Furthermore, to comprehensively validate our method, we adopt two digital triggers in a conventional manner: black-white patch and sig, as shown in Table 3 . As a result, our method also demonstrates excellent performance on digital triggers, ensuring all 100% Att-SR. Compared to the baselines HAMT ILRL and RecBert ILRL , our method could still achieve the comparable SR of 65.01%/63.81% and 60.37%/59.60% in navigation. Here, based on the model HAMT ILRL with yoga ball and black-white patch as the triggers, we demonstrate the robustness of our method on visual and textual variations. Robustness to Visual Variations. (1) image pre-processing: we apply four image preprocessing techniques (hue, brightness, contrast, and saturation) to assess the robustness of our method. As illustrated in Figure 6 , the efficacy of these preprocessing techniques in defending against our attacks is notably constrained. Across all preprocessing and hyperparameter variations, our backdoored VLN agent consistently achieves a 100% Att-SR while maintaining a significantly high level of navigation capability (SR > 60%). ( 2) unseen environments with attached triggers (Uns. Att.): to comprehensively assess the model's attack robustness in unfamiliar environments, we sample the same 99 trajectory-instruction pairs as backdoor attack test in digital space. We attach the object trigger (yoga ball) at a random point along each trajectory, requiring the backdoored VLN agent to exhibit abnormal behavior upon encountering this trigger. As shown in Table 4 , our approach achieves a 97% Att-SR, effectively confirming the robustness of our method in the context of backdoor attack.Robustness to Textual Variations. Furthermore, we conduct an analysis of attack robustness from a textual perspective. We define three variants of textual inputs. (1) Goal-oriented instruction (Goal Ori.): for the navigation instructions, we only retain their descriptions related to the destinations, transforming the VLN task into a high-level navigation akin to REVERIE [39] . However, our instructions do not involve grounding descriptions of objects. 4 , we observe a consistent 100% Att-SR across all variants of instructions. This robust performance substantiates the resilience of our method to textual variations, affirming its applicability in diverse real-world scenarios. Robustness to Poisoning Rate. Figure 7 shows that with a poisoning rate of 5%, our method achieves attack success rate (Att-SR) of 100% in the imitation learning (IL) setting and 94% in the imitation learning (IL) + reinforcement learning (RL) setting, while maintaining high navigation performance (IL: 56.62%; IL+RL: 66.09%). When the poisoning rate increased (10%, 15%, 20%), our method could steadily achieve 100% Att-SR and high navigation performance (IL: 56.43%, 56.18%, 56.09%; IL+RL: 65.51%, 66.23%, 66.18%). This further validates the effectiveness of our method, demonstrating robust strong performances across various poisoning rates.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: Please refer to the method section and the open-sourced code.Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
conclusion: 
 We conduct the first-of-its-kind exploration of the object-aware backdoored VLN, which holds significant practical significance. Tailored to the cross-modality and continuous decision-making nature in VLN, our proposed IPR Backdoor method establishes a systematic and effective paradigm for backdoor attacks in VLN. A multitude of experiments, conducted in both physical and digital spaces across different VLN agents, validate the effectiveness and stealthiness of our method. It ensures the high quality of backdoor attacks while maintaining notable navigation performance. Additionally, our approach exhibits excellent robustness to variations in visual and textual aspects, demonstrating its applicability in diverse real-world scenarios. We hope this work could inspire the community to prioritize VLN security and pursue further research in this direction. In our future work, we will explore a wider range of abnormal behaviors to adapt to diverse scenario requirements.Ethical Impacts: The potential ethical impacts of our backdoored VLN system include both positive and negative aspects. (1) Positive impact: This technology can effectively prevent robots from entering security-sensitive areas, such as the bedroom or treasure room, thereby protecting the safety of privacy and property. (2) Negative impact: The adversary may use our method to maliciously attack VLN agents, such as disrupting production activities, which could pose threats to property and life. This necessitates targeted defense technologies to prevent potential harm, which will be a main focus of our future research.Limitations: As an early work on backdoor attack in VLN, this study currently only explores the anomaly of stopping. In the future, we hope to explore more complex and customized actions.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes]Justification: We use open-sourced datasets in this study. The code will also be released.Guidelines:• The answer NA means that paper does not include experiments requiring code. Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: Please refer to the supplemental material. Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We provide the references for any used asset. And we strictly obey any license.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: We do not release our model currently.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: This paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]  
 
 Appendix: Step 1Step 2 Instruction: Turn ro your left and exit into the hallway. Turn left and enter the bedroom and then turn left and enter the closet. Figure 9 shows the examples of different textual variations. The goal-oriented instructions (Goal Ori.) only contain descriptions about the final destinations. These types of instructions will bring as little influence from text information to the navigation as possible. The "pass" related phrase emphasis instructions (Pass Emp.) specifically emphasize the actions passing the trigger, attempting to avoid the agent's abnormal behavior through textual guidance. Instructions with different descriptive styles (Diff Des.) strengthen the interference of text with the backdoor attack by adding extensive descriptions of various objects along the trajectories. Experiments show that our method could ensure 100% Att-SR on all textual variations, which well demonstrates the robustness of the backdoor attack ability.Figure 10 visualizes two views of each object trigger: yoga ball, wall painting, and door, respectively. In these scenes, object triggers vary in terms of angles and sizes. Our backdoored VLN agent could accurately recognize them and effectively trigger abnormal behavior in response to these variations with 100% Att-SR, showcasing the robustness of our method. Figure 11 illustrates a scene preprocessed by the four image preprocess techniques with different factors: hue, brightness, contrast, and saturation. They are the classic methods to validate backdoored model's robustness. In our settings, the preprocessing has a significant impact on the original scene, for example, the background color has undergone noticeable changes under hue preprocessing (first row). Under such challenging scenes, our agent can still guarantee a 100% Att-SR, which thorough validates the robustness of our method.We hope that our work helps to recognize hidden risks about VLN agents, and can encourage future defense • The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced.Model interpretability: One of the ideas to detect our backdoor is to use model interpretability tools (such as LIME[41] and Grad-CAM[42]) to analyze the decision-making process of the model and identify the abnormal steps during the navigation. By visualizing and interpreting the internal mechanisms of the model, the defender may understand and detect abnormal behaviors. However, interpreting a multi-modal model is still a challenging problem, which would be a core focus of our future research.Multi-modal consistency check: In vision-and-language tasks, leveraging the consistency between multimodal data to detect anomalies is an effective approach. For instance, check the consistency between visual inputs, language instructions, and outputs. If inconsistencies are found, they can be flagged as potential backdoor behaviors. The main issue is how to define the "consistency" in the complex VLN environments.Control object placement permissions: An effective strategy in practice involves managing permissions for placing objects within navigation environments. Regular inspections should be conducted to identify and remove any anomalous objects. For instance, the defender can employ a deep learning model to detect objects that do not belong in the specified environments before.Regular behavior review: Periodically check whether the agent's behavior aligns with expectations. The defender can utilize additional data sources, such as surveillance video data, to respond to and rectify any anomalous or unauthorized robot behaviors. 


Decision: This paper explores safety in vision-language navigation (VLN), specifically the ability to train an agent such that it acts in an undesirable way (e.g. stops) whenever it sees a particular (natural) trigger object. The authors use generation of poisoned scenes containing the trigger object, along with a variety of losses and rewards to encourage the behavior when the trigger is in the scene. This includes a See2Stop Loss (to encourage the agent to stop when it sees the trigger), Anchor loss (pushing the visual features towards semantically meaningful text feature for the behavior), Consistency loss (encouraging feature consistency between the backdoor and original visual encoders), and Backdoor-aware reward (rewarding the agent for the behavior). Results show that the task success is retained while having high attack success. 
 
  The reviewers found that the idea of using backdoors for VLN agents brings up an important safety issue and that the overall approach is interesting (especially since it uses natural objects) and seems uniquely suited to the multi-modal nature of the problem. They also mentioned that the experiments are comprehensive and that the paper is well-written.  Some concerns were raised, including 1) Limitation of the experiments to just one behavior (stop action, mentioned by two reviewers), 2) validation on more diverse datasets, computational overhead, and that the differences in problem characterization and lack of a comprehensive related work comparison to just standard classification is unclear (brought up by multiple reviewers). The authors provided a rebuttal that included several new experiments with another behavior as well as expanded dataset and further clarifications. Reviewers were overall satisfied leading to increased scores. 

  After considering all of the materials, I recommend acceptance of this paper. The paper tackles an interesting problem setting and an impactful safety issue, and the solution provided is uniquely tailored to it, with positive results. I highly encourage the authors to include the results from the rebuttals, especially clarification on differences between this and traditional classification (or even object detection, which also deals with real scenes often) literature and methods, as well as the additional results. 

  Importantly, the ethics reviewers have raised a number of strong suggestions to ensure that this work is sufficiently contextualized and, as mentioned, a dedicated "ethical considerations" section is highly necessary along with potential mitigations. This is necessary for the paper to be published in a manner that considers the paper's ethical implications.

Reviews: review: paper_summary: paper_summary: The paper addresses the security threats posed by malicious behaviors in Vision-and-Language Navigation (VLN) agents. The authors introduce a novel object-aware backdoor attack paradigm, termed the IPR Backdoor, tailored specifically for VLN's cross-modality and continuous decision-making characteristics. This approach implants object-aware backdoors during the training phase, allowing the agent to execute abnormal behaviors when encountering specific object triggers in unseen environments.
strength_weakness: strength_weakness: The paper could explore more complex and customized abnormal behaviors beyond the STOP action.

While the method is validated on several VLN agents and triggers, additional diverse datasets and more varied environments could further strengthen the findings.

The impact of the proposed method on the computational overhead is not thoroughly discussed, which could be important for practical implementations.
questions: questions: See weakness
limitations: limitations: The authors may add more discussion on the societal impacts since this is an adversarial setting.
paper_summary: paper_summary: This work proposed a backdoor attack for Vision and Language Navigation task. It works by embedding natural or digital image as trigger into scene representation during agent rollout, and train agent to stop while preserving normal navigation capability using novel loss choices. The method achieved good attack performance (near 100% att-SR) while preserving most navigation performance under various settings: different trigger, different agent, visual or textual variation.
strength_weakness: strength_weakness: Despite situated the backdoor attack in VLN task, the method is not too different from image classification based backdoor attack, and produce stop action is similar to label prediction.
questions: questions: 1. I am not super clear what data format the trigger is, based Sec 3.3, they seems to be images, but in this case, how is different view of an object trigger displayed in Figure 9 obtained?
2. Why is backdoor aware reward beneficial to retrain VLN performance according to Table 1, do you have a rationale?
limitations: limitations: Authors state the limitation is the attack currently only direct agent to stop, more complex actions could be future work.
paper_summary: paper_summary: This paper explores the security risks of Vision-and-Language Navigation (VLN) agents, which can be integrated into daily life but may threaten privacy and property if compromised. The author addresses this overlooked issue by introducing an object-aware backdoored VLN. This involves implanting backdoors during training to exploit the cross-modality and continuous decision-making aspects of VLN. The proposed IPR Backdoor causes the agent to behave abnormally when encountering specific objects during navigation. Experiments show this method's effectiveness and stealthiness in both physical and digital environments, while maintaining normal navigation performance.
strength_weakness: strength_weakness: 1. Action space. At Line 166, the current action space is based on the current state. Why is it the case? Typical in RL, the action space is fixed and does not change when the states change. If the action space is not fixed, how is it trained in this paper?

2. Poisoning of training data. What are the impact of poison ratios? 20% is a pretty high ratio, for the poisoning rate of backdoor attack. It would be better to show the Att-SR for different ratios, to understanding how practical it might be.
 
3. Missing related work. The method design shares some similarity with [1], but I acknowledge that the paper addresses some unique challenges in VLN scenarios. Also, the related work section should also introduce backdoor defense techniques on multi-modal models[2][3], and discuss potential defense for the backdoor attacks on VLN models.

4. Minor: 
- Line 66, 'pertaining' --> 'pretraining'
- In Figure 3, the input of Consistency Loss should be CleanEncoder(CleanInput) and BackdooredEncoder(CleanInput). The color of arrows seems incorrect.


[1] BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning, https://arxiv.org/abs/2108.00352

[2] Detecting Backdoors in Pre-trained Encoders, https://arxiv.org/abs/2303.15180

[3] SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learning, https://arxiv.org/abs/2303.09079
questions: questions: Please see Weaknesses.
limitations: limitations: The target action is only limited to STOP.
paper_summary: paper_summary: The paper proposes a novel backdoor attack paradigm, termed IPR Backdoor, for Vision-and-Language Navigation (VLN) agents. The authors highlight the potential security risks posed by VLN agents in sensitive environments and pioneer an object-aware backdoor attack, embedding triggers into the agent during training. The attack is designed to make the agent execute abnormal behaviors upon encountering specific objects during navigation, without compromising normal operation otherwise. The key contributions include the development of the IPR Backdoor, its validation through extensive experiments, and demonstrating its robustness to various visual and textual variations.
strength_weakness: strength_weakness: 1. The author proposed conducting experiments in physical and digital spaces. However, the author's definition of physical seems to be "pasting a physical object into an image," while a more general understanding of physical is to sample in the physical world rather than simply pasting.
2. Lack of horizontal comparison with other backdoor attack methods, this issue is actually equivalent to the third item in the "Question" section. I hope the author can reply well in the rebuttal.
questions: questions: 1. The VLN agents used by the author are two classic methods, however, I am more interested in the performance of the author's proposed method on advanced VLN agents. (like the VLN-GOAT)
2. The author did not provide an open source link, but mentioned in the checklist that the open source plan is "after acceptance". This is acceptable. Can the author provide an approximate open-source timeline, such as within one month after acceptance?
3. What are the differences in backdoor attacks between VLN tasks and traditional DL tasks, apart from the differences in tasks.
limitations: limitations: The authors adequately addressed the limitations.

Comments: comment: We sincerely appreciate your efforts and recognition! We will further improve the revision based on your valuable suggestions. If you have any further questions, we are more than willing to discuss!
Rebuttal solves most of my concerns. So I decide to raise my score to 6
**Recommendation#1: Such work should include a broader impact statement, including a discussion of the implications of the attack.**

Response#1:

Thanks for your valuable suggestion. 

The potential **ethical impacts** of our backdoored Vision-and-Language Navigation (VLN) system include both positive and negative aspects. 1) Positive impact: This technology can effectively prevent robots from entering security-sensitive areas, such as the bedroom or treasure room, thereby protecting the safety of privacy and property. 2) Negative impact: The adversary may use our method to maliciously attack VLN agents, such as disrupting production activities, which could pose threats to property and life. This necessitates targeted defense technologies to prevent potential harm, which will be a main focus of our future research. 

In terms of the **detection approach**, we would hope that our work helps to recognize hidden risks about VLN, and can encourage future defense research. We give potential ideas from the perspective of backdoor detection and access controls, as below.

1. Model interpretability: One of the ideas to detect our backdoor is to use model interpretability tools (such as LIME [1] and Grad-CAM [2]) to analyze the decision-making process of the model and identify the abnormal steps during the navigation. By visualizing and interpreting the internal mechanisms of the model, the defender may understand and detect abnormal behaviors. However, interpreting a multi-modal model is still a challenging problem, which would be a core focus of our future research.

2. Multi-modal consistency check: In vision-language tasks, leverage the consistency between multimodal data to detect anomalies. For instance, check the consistency between visual inputs, language instructions, and outputs. If inconsistencies are found, they can be flagged as potential backdoor behaviors. The main issue is how to define the "consistency" in the complex environment of the vision-language navigation.

3. Control object placement permissions: An effective strategy in practice to control the permissions for placing objects within the navigation environment. Regularly inspect and remove any anomalous objects. For instance, the defender can utilize a deep learning model to detect objects that do not belong in the specified environment.

4. Regular behavior review: Periodically check whether the robot's behavior aligns with expectations. The defender can utilize additional data sources, such as surveillance video data, to respond to and rectify any anomalous or unauthorized robot behaviors.

We commit to including a dedicated discussion about broader impact statement and detection approach in the revised version.

[1] Ribeiro M T, Singh S, Guestrin C. " Why should i trust you?" Explaining the predictions of any classifier[C]//Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016: 1135-1144.

[2] Selvaraju R R, Cogswell M, Das A, et al. Grad-cam: Visual explanations from deep networks via gradient-based localization[C]//Proceedings of the IEEE international conference on computer vision. 2017: 618-626.
**Recommendation#1: Add a dedicated "Ethical considerations" section, including discussion on misuse potential, privacy violations, safety, security etc.**

Response#1:

Thanks for your valuable suggestion.
The potential ethical impacts of our backdoored Vision-and-Language Navigation (VLN) system include both positive and negative aspects. 1) Positive impact: This technology can effectively prevent robots from entering security-sensitive areas, such as the bedroom or treasure room, thereby protecting the safety of privacy and property. 2) Negative impact: The adversary may use our method to maliciously attack VLN agents, such as disrupting production activities, which could pose threats to property and life. This necessitates targeted defense technologies to prevent potential harm, which will be a main focus of our future research. 

We commit to including a dedicated "Ethical Considerations" section in the revised version with detailed discussion.

**Recommendation#2: Propose methods or ideas to develop robust detection systems for backdoored behaviors. Implement strict access controls, or provide guidelines.**

Response#2:

We appreciate your suggestion, which exactly corresponds to our motivation. We would hope that our work helps to recognize hidden risks about Vision-and-Language Navigation (VLN), and can encourage future defense research. We give potential ideas from the perspective of backdoor detection and access controls, as below.

1. Model interpretability: One of the ideas to detect our backdoor is to use model interpretability tools (such as LIME [1] and Grad-CAM [2]) to analyze the decision-making process of the model and identify the abnormal steps during the navigation. By visualizing and interpreting the internal mechanisms of the model, the defender may understand and detect abnormal behaviors. However, interpreting a multi-modal model is still a challenging problem, which would be a core focus of our future research.

2. Multi-modal consistency check: In vision-language tasks, leverage the consistency between multimodal data to detect anomalies. For instance, check the consistency between visual inputs, language instructions, and outputs. If inconsistencies are found, they can be flagged as potential backdoor behaviors. The main issue is how to define the "consistency" in the complex environment of the vision-language navigation.

3. Control object placement permissions: An effective strategy in practice to control the permissions for placing objects within the navigation environment. Regularly inspect and remove any anomalous objects. For instance, the defender can utilize a deep learning model to detect objects that do not belong in the specified environment.

4. Regular behavior review: Periodically check whether the robot's behavior aligns with expectations. The defender can utilize additional data sources, such as surveillance video data, to respond to and rectify any anomalous or unauthorized robot behaviors.

We will add such discussion to the revision.

[1] Ribeiro M T, Singh S, Guestrin C. " Why should i trust you?" Explaining the predictions of any classifier[C]//Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016: 1135-1144.

[2] Selvaraju R R, Cogswell M, Das A, et al. Grad-cam: Visual explanations from deep networks via gradient-based localization[C]//Proceedings of the IEEE international conference on computer vision. 2017: 618-626.

**Recommendation#3: Emphasize the value of transparency.**

Response#3:

We appreciate your valuable suggestion. Our work utilizes publicly available indoor environment dataset Matterport3d [1] and instruction dataset R2R [2] for research, which are readily accessible to researchers. Furthermore, we will release the source code to facilitate reproducibility, aiming to advance the development of the robotics and security communities more efficiently.

[1] Chang A, Dai A, Funkhouser T, et al. Matterport3D: Learning from RGB-D Data in Indoor Environments[C]//International Conference on 3D Vision (3DV). 2017.

[2] Anderson P, Wu Q, Teney D, et al. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 3674-3683.
Thank you for your kind feedback. We are glad that our responses addressed your questions satisfactorily. If you have any further ones, we are more than willing to discuss them further.
Thank you once again for your recognition and valuable insights. We will ensure to include the discussed points in the next version of our manuscript.
I appreciate author response to my comments. My questions are well addressed.
Thanks for the rebuttal and I raise my score to 6. I suggest authors include these discussion in the next version.