Sound Adversarial Audio-Visual Navigation
Authors: Yinfeng Yu
Wenbing Huang
Fuchun Sun
Changan Chen
Yikai Wang
Xiaohong Liu
Venue: ICLR 2022 Poster (2022)

Abstract: Audio-visual navigation task requires an agent to find a sound source in a realistic, unmapped 3D environment by utilizing egocentric audio-visual observations. Existing audio-visual navigation works assume a clean environment that solely contains the target sound, which, however, would not be suitable in most real-world applications due to the unexpected sound noise or intentional interference. In this work, we design an acoustically complex environment in which, besides the target sound, there exists a sound attacker playing a zero-sum game with the agent. More specifically, the attacker can move and change the volume and category of the sound to make the agent suffer from finding the sounding object while the agent tries to dodge the attack and navigate to the goal under the intervention. Under certain constraints to the attacker, we can improve the robustness of the agent towards unexpected sound attacks in audio-visual navigation. For better convergence, we develop a joint training mechanism by employing the property of a centralized critic with decentralized actors. Experiments on two real-world 3D scan datasets, Replica, and Matterport3D, verify the effectiveness and the robustness of the agent trained under our designed environment when transferred to the clean environment or the one containing sound attackers with random policy. Project: https://yyf17.github.io/SAAVN .

Published as a conference paper at ICLR 2022
SOUND ADVERSARIAL AUDIO- VISUAL NAVIGATION
Yinfeng Yu!., Wenbing Huang', Fuchun Sun", Changan Chen', Yikai Wang Xiaohong Liu! Beijing National Research Center for Information Science and Technology (BNRist), State Key Lab Intelligent Technology and Systems,
Department of Computer Science and Technology. Tsinghua University
2 Institute for AI Industry Research (AIR), Tsinghua University
3 College of Information Science and Engincering, Xinjiang University
4 UT Austin 5 JD Explore Academy, JD.com
yyf1 70117mmail0 iss ts inghua edu.cn, hwenbingg 1w0.022.ccoooo com, Ecsunêmail t tsinguu.. edu.cn cn
ABSTRACT
Audio-visual navigation task requires an agent to find sound source in a realis- tie, unmapped 3D environment by utilizing egocentric audio-visual observations. Existing audio-visual navigation works assume a clean environment that solely contains the target sound, which, however, would not be suitable in most real- world applications due to the unexpected sound noise or intentional interference. In this work, we design an acoustically complex environment in which, besides the target sound, there exists a sound attacker playing a zero-sum game with the agent. More specifically, the attacker can move and change the volume and cate- gory of the sound to make the agent suffer from finding the sounding object, while the agent tries to dodge the attack and navigate to the goal under the intervention. Under certain constraints to the attacker, we can improve the robustness of the agent towards unexpected sound attacks in audio-visual navigation. For better convergence, we develop joint training mechanism by employing the property of a centralized critic with decentralized actors. Experiments on two real-world 3D scan datasets (Replica and Matterport3D) verify the effectiveness and the ro- bustness of the agent trained under our designed environment when transferred to the clean environment or the one containing sound attackers with random policy. Project: hatp::: /ttt:::://yyll.gihhbbb weaithuu..ooo... L./AAVV
INTRODUCTION
Audio-visual navigation, as currently a vital task for embodied vision (Gordon et al.) 2018 Lohmann al. 2220 Nagarajan & Grauman 2020). requires the agent to find sound target realistic and unmapped 3D environment by exploring with egocentric audio-visual observations (Chen et al.)
2019 Gupta al., 2017: Chaplot al. 2020). Inspired by the simultancous usage of eyes and ears in human exploration (Wilcox et al. [207 Flom & Bahrickk (207)) audio-visual association benefits Flom
the learning of the agent (Gan et al. 2019 2020aa Dean et al.] 2020). recent work Look, Listen, and Act (LLA) has proposed a three-step navigation solution of perception, inference, and decision- making (Gan et al. (2000)). SoundSpaces the first work to establish an audio-visual embod- ied navigation simulation platform equipped with the proposed Audio Visual embodied Navigation (AVN) baseline that resorts to reinforcement learning (Chen et al.) 2020). In response to the long- term exploration problem that is caused by the large layout of the 3D scene and the long distance Waypoint Navigation WaN) proposes an audio-visual naviga-
to the target place, Audio-Visss tion algorithm by setting waypoints sub-goals to facilitate sound source discovering (Chen al.) 20216). Besides, Semantic Audio-Visual navigation (SAVi) develops a navigation algorithm in a scene where the target sound is not periodic and has a variable length; that is, it may stop during the navigation process (Chen et al. 202
Despite the fruitful progress in audio-visual navigation, existing works assume an acoustically sim- ple or clean environment, meaning there is no other sound but the source itself. Nevertheless, this
"ooresponning author; Fuchun Sun.

---

Published as a conference paper at ICLR 2022
assumption is hardly the case in real life. For example, kettle in the kitchen beeps to tell the robot that the water is boiling, and the robot in the living room needs to navigate to the kitchen and turn off the stove; while in the living room, two children are playing a game, chuckling loudly from time to time. Such an example poses a crucial challenge on current techniques: can an agent still find its way to the destination without being distracted by all non-target sounds around the agent? Intuitively, the answer is no if the agent has not been trained in acoustically complex environments as in the example listed above. Although the answer is no, this ability what we expect the agent to possess in real life.
In light of these limitations, we propose first to construct such an acoustically complex environment. In this environment, we add sound attacker to intervene. This sound attacker can move and change the volume and type of the sound at each time step. particular, the objective of the sound attacker is to make the agent frustrated by creating distraction. In contrast, the agent decides how to move at every time step, tries to dodge the sound attack, and explores for the sound target well under the sound attack, as illustrated in Fig. [ The competition between the attacker and the agent can be modeled as a zero-sum two-player game. Notably, this is not fair game and is more biased towards the agent for two reasons. First, the sound attack is just ainge-moddl and will not intervene in any visual information obtained by the agent. Second, as will be specified in our methodology, the sound volume of the attacker is bounded via relative ratio (less than 1) of the sound target. We anticipate constraining the attacker's power and encouraging it to intervene but not defeat the agent with these two considerations. With such a design, we can improve the agent's robustness between the agent and the sound attacker during the game. On the other hand, our environment is more demanding than reality since there are few attackers in our lives. Instead, most behaviors, such someone walking and chatting past the robot, are not deliberately embarrassing the robot but just a distraction to the robot, exhibiting weaker intervention strength than our adversarial setting. Even so, our experiments reveal that an agent trained in worst-case setting can perform promisingly when the environment acoustically clean or contains a natural sound intervenor using a random policy. On the contrary, the agent trained in clean environment becomes disabled in an acoustically complex environment.
Our training algorithm is built upon the architecture by (Chen et al. 22200), with a novel decision- making branch for the attacker. Training two agents separately (Tampuu et al.) (017) leads to di- vergence. Hence we propose a joint Actor-Critic (AC) training framework. We define the policies for the attacker based on three types of information: position, sound volume, and sound category. Exciting discoveries from experiments demonstrate that the joint training converges promisingly in contrast to the independent training counterpart.
This work is the first audio-visual navigation method with a sound attacker to the best of our knowl- edge. To sum up, our contributions are follows.
We construct a sound attacker to intervene environment for audio-visual navigation that aims to improve the agent's robustness. In contrast to the environment used by prior ex- periments (Chen et al. 2020), setting better simulates the practical case in which there exist other moving intervenor sounds.
We develop a joint training paradigm for the agent and the attacker. Moreover, we have justified the effectiveness of this paradigm, both theoretically and empirically.
Experiments on two real-world 3D scenes, Replica (Straub al., 2019) and Matter- port3D (Chang et al., 2017) validate the effectiveness and robustness of the agent trained under our designed environment when transferred to various cases, including the clean en- vironment and the one that contains sound attacker with random policy.
RELATED WORK
We introduce the research related to our work, including audio-visual navigation, adversarial training in RL, and multi-agent learning.
Audio-visual embodied navigation. This task demands a robot equipped with a camera and a mi- crophone to interact with the environment and navigate to the source sound. Existing algorithms towards this task can be divided into two categories according to whether topological maps are con- structed or not. For the first category (Gan et al. 2020b Chen al. 2021b). L.IA (Gan et al. 2020b)
2

---

Published as a conference paper at ICLR 2022
Egccentnsheew
Agwill Egposntriclle
Where is Tee phane?
Where i the plane?
Left
Specdrogrsie
Ri
Binzura Spoctrig cam
Left
Hight
AusiCCoii
(8) Audio-Visu Vauaa Embodiad Navigation in Simple Envitonment.
(b) Audio-Visual Embocied Navigation Complex Environment
Figure 1: Comparison of audio-visual embodied navigation in clean and complex environment. (a) Audio-visual embodied navigation in an acoustically clean environment: The agent navigates while only hearing the sound emitted by the source object. (b) Audio-visual navigation in an acous- tically complex environment: The agent navigates with the audio-visual input from the source object, with the sound attacker making sounds simultaneously.
plans the robot's navigation strategy based on graph-based shortest path planning. Following LLA,, AV-WaN (Chen et al. 2021b) combines dynamically setting waypoints with the shortest path algo- rithm to solve the long-period audio-visual navigation problem. The second category of methods works in the absence of a topological map (Chen et al. 2021a 2020). In particular, SAVi (Chen et al. 2021a) aims to solve the audio-visua navigation problem with temporary sound source by introducing the category information of the source sound and pre-training the displacement from the robot the sound source; AVN (Chen al. 2020) constructs the first audio-visual embodied nav- igation simulation platform SoundSpaces and makes use of Reinforcement Learning (RL) for the training of the navigation policy. As presented in the Introduction, all environments used previously (including SoundSpaces) assume clean sound sources, which is hardly the case in real and noisy life. By contrast, we build an acoustically complex environment that allows a sound attacker to move and change the volume and category of sound at each time step in an episode. In this environment, we train the navigation policy of the agent under the sound attack, which delivers a more robust solution in real applications.
Adversarial attacks and adversarial training in RL. In general, adversarial attacks (Tian Xu 2021 /training in RL are divided into two classes: learning to attack (Huang et al.) 2017]] Kos & Song 2017: Lin et al., 2017;; Pattanaik et al.] 2018) and learning to defence (Pinto et al. 2017 Gleave al. 2020; ZZhng al. (0220) that targets learn robustness policy by state adversarial through external forces or sensor noise. Our paper falls into the second class. A close work to our method is State Adversarial MDP (SA-MDP) (Zhang al., 2020) that leverages sensor noise in vision to improve the robustness of the algorithm. The main difference between our method and SA-MDP is that our method precisely permits sound attacker move in the room and employs the resulting sound as distractor, while in SA-MDP, the adversarial state is created arbitrarily and thus not necessarily fits the actual scene. Besides, SA-MDP initially copes with the visual modality; hence, we have changed SA-MDP for the attack of the sound modality for the comparison with our method (See
Multi-agent learning. We design two frameworks similar to independent learning [Tampuu et al., 2017) and multi-agent learning (Sunehag et al.] 2018). However, the framework of independent learning does not converge (See $ 44. We formulate our learning algorithm two-player game employed the property of a centralized critic with decentralized actors (Wang et al. 202T) to guar- antee the training convergence of our method in theory (See Theorem
3 SOUND ADVERSARIAL AUDIO VISUAL NAVIGATION
3.1 PROBLEM DEFINITION :
We model the agent as playing against an attacker in a two-player Markov game (Simon 2016). We denote the agent and attacker by superscript and v, respectively. The game M- (S, (A A²), P. (R", R²)) consists of state set S, action sets A and A" and a joint state transition function P S A A + S. The reward function R : A x R for agent and R" S x A x A" x S + R for attacker respectively depends on the current state, next state and both the agent's 's's and the attacker's actions. Each player wishes to maximize their discounted sum of rewards, where R = R" -r. r is the reward given by the environment at every time step in an episode. Our problem is modeled as following (See Fig. (c)::
Problem modeling of ours.
n² = (I " )²) = arg max (arg (argmin min" ", ,,,r)}}
(1)
x" "Tl""
3

---

Published as a conference paper at ICLR 2022
(als))
7t
Agent
adversary v(sg)
Agent
Agent
a it (a)ss)
x<a(v(ss))
Environment
T++1
Yes1
Environment
Environment
S4
Attacker
(a) MDP
(b) SA-MDP
(c) Ours
different problem modeling methods. AVN is models as an MDP. Our model has an attacker intervention, while the SA-MDP model has an adversary that can map one state in the state space to another state.
Figure 2: Comparison of
Inspired by value decomposition networks (Sunchag et al. 2018) and QMIX (Rashid et al. 2018 we design G( ,,,r) as Equation (2,, where G(TT and G(T",r) are expected discounted, cumulative rewards of the agent and the attacker respectively.
G( x), [G( r), G(F",r)
SA-MDP.
As a reference, we introduce the previous adversarial MDP proposed by (Zhang et al. as well. In SA-MDP (See Fig. (b)), optimize gadv == arg max D [.] (als) nh (a|s+
2021
6:
0)
Intuitively, the state SA-MDP Is on the ball with radius c.
3.2
THEORY ANALYSIS
Why does model and training mechanism work? Through theoretical analysis, pleas- antly surprised to find that the observation space of a sound attacker bounded in the projection space. For more details, see Theorem [
Notations. F, stands for short-time Fourier transform and the room impulse response, respec- tively. Sg and S, stand for the waveform signal received by the robot came from sound source and sound attacker at every time step, respectively. Ig and I, stand for the egocentric visual re- ceived by the robot O and O' stand for the observation what the robot received in clean environment and an acous- tically complex environment, respectively. O and O' defined as follows: O F(vs sg)l O' = F(vi #5g + a. =sy)] where stands for multiply, * stands for time domain convolution, is the value of action ao e is parameter.
clean environment and acoustically complex environment, respectively.
Sound attackers do not intervene in the robot's visual modality observation an acoustically com- plex environment. So we have
Property 1g Ly at every time step in an given episode.
Assumption The energy of the sound source sent robot at every time step for a fixed duration from the same sound set has an upper bound e, [|F(ss) < e, |F <.
The distance of intervention is map function of O and O' A: R. We give distance definition of O and O' as following:
Definition The distance is: AO 0') = 1T I-13 +
[( S3 + a the * (v) F(bby * cg)
Then the sound attacker's observation space (O) in an acoustically complex environment based on the above intervention distance is formalized follows: B.( (0' (0,0)) <).
Theorem The observation space of the sound attacker is bounded in the projection space. If sound attacker's 's observation projected into a frequency domain space, the projected result is in sphere with radius of

---

Published as a conference paper at ICLR 2022
Proof
A(O,0") 11 - [F * + 4.* * (x) F(us s
(Def. 1
[F(Wss * (g) + uy (.) F(vs * +c,)12
(Prop.
Prop. 2
a. ||F(1) * + [1F(bb)) F(su))11
(Prop.
2
(3)
= 9/2x |
(Prop.
50//22= =
(Assump.
Theorem T shows that the attack is bounded and guarantees the rationality of our algorithm. Property in appx. B provides the linear property of the Fourier transform, the convolution theory of the Fourier transform, and the Fourier transform of the Dirac delta function.
3.3 SOUND ADVERSARIAL AUDIO-VISUAL NAVIGATION
We propose Sound Adversarial Audio-Visual Navigation (SAAVN), novel model for the audio- visual embodied navigation task. Our approach is composed of three main modules (Fig, 33. Given visual and audio inputs, our model 1) encodes these cues and make a decision for the motion of the agent, then 2) encodes these cues and decide how to act for the sound attacker to make an acoustically complex environment, and finally 3) make judgment for the agent and the attacker and to optimization. The agent and the attacker repeat this process until the agent has been reached and executes the Stop action.
Environment. Our work is based on the SoundSpaces (Chen et al., 2020) platform and Habitat simulator (Savva et al. 2019) and with the publicly available datasets: Replica (Straub et al.) 2019) and Matterport3D (Chang et al. 2017) and SoundSpaces audio dataset. In SoundSpaces, the sound is created by convolving the selected audio with the corresponding binaural room impulse responses (RIRs) under one of the directions. When sound attacker emits a chosen sound from its position, the emitted omnidirectional audio is convolved with the corresponding binaural RIR to generate binaural response from the environment heard by the agent when facing each direction. In this sense, the attacker's sound also considers the reflections on the surface of objects in the environ- ment, making it physically admissible and realistic. The agent's 's reward is based on how close the robot is away from the goal and whether it succeeds in reaching it. The setting is the same as of the SoundSpaces. The action space of the agent is navigation motions, which is the same as the setting of the SoundSpaces. An environment attacker embodied in the environment must take actions from a hybrid action space A²". For brevity, the abbreviation of superscripts position, volume, and cat- egory are set to pos, vol, and cat, respectively. The hybrid action space is the Cartesian product of navigation motions space Arr-ps,, volume sound space v. and category of sound space A ,u,eut A² Ar-pos x Ar,vol AP,cat For more details, see appx.
Perception, act, and optimization. Our model uses acoustic and visual cues in the 3D environment for efficient navigation. Our model has mainly comprised of three parts: the environment attacker, the agent, and the optimizer (See Fig. B). At every time step f. the agent and the attacker receives an observation O, (I,, B,), where 7 is the egocentric visual observation consisting of an RGB and a depth image; B is the received binaural audio waveform represented as a two-channel spec- trogram. Our model encodes each visual and audio observation with a CNN, respectively, where the output of each CNN are visual vector fzi (Ie) and audio vector (B.). Then, we concatenate the two vectors to obtain observation embedding representation e¹ = [ffr((e) f fBI (B.)]. We trans- form observation embedding representation to calculate state representation by a gated recurrent unit (GRU), s = GRU(e), h(-1). ). An actor-critic network uses st to predict the action distribution Tg (att) |st, h and value of the state (s! 1/-1). We also encode visual and audio obser- vation with a CNN for environment attacker, where the output of each CNN are vectors frz([,), (B.) We then concatenate the two vectors to obtain observation embedding representation [It), (B)( We also transform observation embedding representation to calculate state representation by a GRU, $f = -RR h2-1). Three actor-critic networks use &f to predict P. vol
U. vo the action distribution: ". pos (a, |s. h2 1). TT (a, ss h2-1). T st.cat (a, |s? h?_1) and Tg
value the state: yP A) Vig vol (sp h2=1), V" cat (52 1(_1). All actors and critics are modeled by single linear layer neural network, respectively. Finally, four action samplers sample pos v. vol v.
the next action af, ag a, a, from these action distributions generated by AgentActor,
5

---

Published as a conference paper at ICLR 2022
Environment
Agent
CNN
h, h
AgentActor
ActionSampler
RGB
CNN
GRU
AgentCritic
d
Vision
Depth
PositionActor
ActionSampler
CNN
ha
PositionCritic
VolumeActor
ActionSampler
CNN
GRU
VolumeCritic
Right
Audio
CategoryActor
ActionSampler
Muser a
Left
CategoryCriicc
Binaural Spectrograms
Attacker
(W,,W..Ww..W.
Critic
Figure 3: Sound adversarial audio-visual navigation network. The agent and the sound attacker first encode observations and learn state representation 8, respectively. Then, 8, fed to actor-critic networks, which predict the next action of and of. Both the agent and the sound attacker receive their rewards from the environment. The sum of their rewards is zero.
PositionActor, VolumeActor and Category respectively, determining the agent's next motion in the 3D scene. The total critic is linear of PositionCritic, VolumeCritic, and CategoryCritic (For weights details, see appx. G]. The agent and the environment attacker optimize their expected discounted, cumulative rewards G ((*",, r) and G(x", r) respectively. The loss of each branch actor- critic network and the total loss of our model as Equation (4).
C E0.5 (Voi V²)))
ΣIA' log(Tas 8)) 3- ( (a s))]
L" 1/3 (L L.umll p
(4)
1/6 Luueat + 1/6 Luvvol + 1/6 L 1/2- L"
where j {VV, cat), (1, vol), (v, pos), (w)) (o:(s) is estimated state value of the target network
for (s() max Eir, 7. V (s(+1) 8t s) A ET- 2422-t o is the advantage for aE
given length-T T trajectory and 6: rt + + v (8++1) v (st) We optimize the objective follows from Proximal Policy Optimization (PPO) (Schulman et al. 2017)
Algorithms. Our algorithms are as the following:
Algorithm 1: Sound Adversarial Audio- Visual Navigation
Data: Environment ε, stochastic policies w and v, initial parameters 0g for ws and 0g for v. number of updates Nuve, N.
Result: 6% 0% for 2, Niwr do
// Run policy Toy, and may_, in environment for N episodes T time steps ((or,s 1i-1, aH,s ai,s r2, r(,,)} + roll(E, Top , Compute advantage estimates A , AY.... ,A.. // Optimize C(E L(Equation! 4 w.r.t. 8" and 8"
04,00 0% < policy optimize with PPO:
end
4 EXPERIMENTS
Task. We use AudioGoal navigation (Chen et al., 2020) tasks to test the acoustically complex enviroument designed. In this task, the robot moves in 3D environment. At each time step t, it obtains an RGB and a depth image from its camera, and the binaural microphone receives sensor observations to get O,. As the robot starts navigating. does not know the scene map; it

---

Published as a conference paper at ICLR 2022
must accumulate observations to understand the geometry of the scene during navigation. The robot should use the sound from the audio source to locate successfully and navigate to the target.
Baselines.
We compare our model to the following baselines and existing works:
1. Random: A random policy that uniformly samples one three actions and executes Stop auto- matically when it reaches the goal (perfect stopping).
AVN (Chen et al., 2020): An audiovisual embodied navigation trained in an environment without sound intervention.
3. SA-MDP (Zhang et al. 2020): A model aims to improve the robustness by state adversarial. We adopt its idea but only intervene state of the sound input and do not process the visual information.
Table 2: Comparison of different models the environment with specified sound attacker a" under SPL and Rmen metrics.
Comparison of different models in the clean environment under SPL. and Rmem metrics. Clean env or cEnv the abbreviation of clean en- vironment, Acou com env or acEnv is
Table I:
Method
Environment with sound attacker
Fondon 0.007 4.8
Random
0.0
0.0001-
the abbreviation of acoustically com- plex environment later.
AVN (Chen et 2020
0.676/13.9
0.605/11,0
0.609/11.0
SA-MDP Zhing
2020
0.456/7.9
0,272/5.2
0,291/5.4
0.738/16.6
0,659/13.2
0,657/13.2
Vasten
Method
Clean env.
Random
0.000-4.7
0.0001-4.5
0.000:4435
Random
0.00N--.77
AVN CCenn et 2020
0.666/13.5
0.535/10.9
0,549/11.0
AVN (Chen al. 2020
0.721/15.1
SA-MUP Zhing et a. 2020
0.317/5.2
0.202/3.8
0.209/3.9
SA MDPT Zhang 2020
0.590V10.2
SAMVN-
0.727/16.0
0.647/13.6
0.648/13.6
SAAVN (GUT))
0.742/16.6
Ca.
Cardoo
Note for Table 2: The row in Tablc
Random
0.000-4.5
0.000-477
0.000:-43
corresponds to how the policy is trained while the column corresponds to how the policy tested. The environment in the same column of a sub-table is the same. The method in the same row of Table is the same.
AVN Chen al. 2020
0.4433933
0,569/10.4
0,576/10,5
SA MDD Zhang et 2020
0,293/4.7
0.451/8.0
0.461/8.2
SAAV
0.666/14.4
0.600/12.5
0.609/12.7
(PVC))s
[PV Jandoo
PVC
Random
0.000/4.5
0.000/-4.7
0,000/-4.5
AVN (Chen et al. 2020
0,375/9.5
0,388/8.2
0,389/8.0 0
MDP Zhang 2020
0.283/4.5
0.375/7.2
0.368/7.2
SAAVN
0.667/14.9
0,557/10.6
0,552/10.6
Metrics and symbols. The evaluations are compared by the following navigation metries: 1)
success weighted by inverse path length (SPL): the standard metric (Anderson et al.] (018) that weighs successes by their adherence to the shortest path; 2) Agent's average episode reward: Rmman- The detailed definitions of the symbols used later in this section are as follows: (1) n "-pos ".5v0 and nh'eeat are abbreviated as P. V and C respectively. (2) X, Xrandom and X denote policies, where X E {P, C} is a policy to learn, Xrandom is a random policy, and when X E {P), Xtx determined policy which acts constant action after random initialization in an episode, while X E (V.C). Xix is a determined policy which act a constant action set in advance in all episodes. (3) X, Yix is abbreviated as X, where X is policy, and Y {{P, V, C} \ {X)}
set determined policy Ynx- (4) SAAVN:X stands for a model variant, where X is a policy for x" and X E V,,V (5) SAAVN:X, = is a model variant named A:AV and Ynx is set to {[P, V, CH {X}}. (6) SAAVN:X. Y I,Z=personó is model variant named SAAVN:X, where Ynx is set 0.1, Zax set to personó, X E (P, V, C).Y ([P.V,C) [X),.Z E ([P.V.C)
0.1, where X E (P.V.C),
}). See appx. H and for more details.
Navigation results. We have built a large number of complex environments with sound attackers therein. The baselines and our SAAVN have been tested several times with different seeds in all these acoustically complex and clean environments to obtain the average navigation ability under each environment. can be seen from Table [ 2 that our model achieves the best navigation capabilities in all environments. For more experiments results, see appx.
Trajectory comparisons. Fig. Ashows the test episodes for our SAAVN model. The environment of each row the figure consistent, and the model of each column is too. SA-MDP failed to complete the task successfully in all two environments. AVN can complete the task in clean environment but fails in an acoustically complex environment. SAAVN completed the tasks in all two environments. What is more, the navigation track of SAAVN in environment without intervention is shorter than that of AVN. These fully demonstrate the navigation capabilities of SAAVN. For more trajectory comparisons, appx.
7

---

Published as conference paper at ICLR 2022
AVN
SA-MDP
SAAVN
env
Agent
Start
Goai
Attackar
Shorteat
Agent path
SeencUnseen
Ocoupied
Figure 4: Different models in different environments explore trajectories. The first row in the figure clean environment, and the second line is an acoustically complex environment. Acou com env stands for acoustically complex environment.
Robustness of the model. In order to verify the robustness of our algorithm, we designed sound attackers, with the settings follows: x=ppss is learned policy, n- is set to person6, while "0 is fixed to 0.1, 0.3, 0.5, 0.7 and 0.9 respectively. We train AVN, SA-MDP, and SAAVN named SAAVN: P. V=0.1, C=person6 respectively in the nated sound attacker. Then we test them multiple times in the above five environments. Fig shooss the performance of different models under different sound attacks. We found that these sound tackers have certain attack capabilities against these three algorithms. However, it is found that our method's performance decreases more slowly, which fully demonstrates that our method helps to improve the robust performance of the algorithm.
environment created by desig-
0.80
EXGR
0.6
0.742
0.755
2.5
0.75
0.721
0.730
9.4
g 0.70
0.65
AUN
5A-1EP
0.1
5ANVN
02
0.5
0.6
0.a
0.
0.60
SAAIN9Ve0)
SAAVN SAAVN.PV-D
PV-05
PV
Attack strongth
Figure 5: Navigation capabilities under different sound attack strengths.
Figure 6: Performance affect by volume.
Is the louder the sound attacker's volume, the better? To verifyy the navigation ability of an agent that has grown in an acoustically complex environment in a clean environment, we first train model in the acoustically complex environments where the sound attacker exists and then test the navigation ability of the model by migrating to the environment which the sound attacker is removed. In Fig. 6 SAAVN: P. V=0.1 is short for a model variant named SAAVN: P. V=0.1, C=person6. Others are similar. It can be seen from Fig. 6 that the navigation ability of the attacker is excellent when the volume of the attacker 0.1 and 0.9, but not very good when the volume is 0.5. The ablation experiments show that simply increasing the sound volume of the attacker does not neces- sarily lead to better performance. The relationship between the navigation capacity and the volume of the sound attacker is not straightforward and depends on other factors, including the position and sound category. hence supports the necessity of our method to make the volume policy of the attacker learnable. As such, agents can learn better navigation skills in an acoustically intervened environment.
Navigation results in the clean environment. When the sound attacker does not exist, what is the navigation ability of an agent that grows in an acoustically complex environment? The performance of AVN is trained in simple environment and tested in clean environment; The performances of SAAVN are trained in acoustically complex environments and tested in a clean environment. can be seen from Fig. 7 (a) that the ability of agent that grows up in acoustically complex
8

---

Published as a conference paper at ICLR 2022
0.755
AMM
CAANN
0.78
0,723
0,710
SP1
0:685
0.064
0.668
0.6
Pr-
SANNNA SAAYAA
SAAVN
SAAVN
SAAVIIC
SAAVN
SAAVNPeY
AM
Po V+
PrV
(a) Navigation capabilities in the clean environment.
(b) Navigation capabilities
complex environments.
Figure Navigation capabilities in different environments.
environment to navigate in a clean environment depends on the complexity of the environment. the environment is acoustically complex, but within a certain range, the ability of the agent will increase; if the environment is too complicated and the changes are relatively great, the navigation ability of the agent is reduced bit, but not much. The ablation study shows that the environment should not be too complicated to achieve optimal navigation capabilities.
Navigation results in acoustically complex environments. What is the navigation ability of an agent in an acoustically complex environment? In Fig. (b), we can see different sound attackers. As the attack intensity of sound attackers ascends, the navigation capabilities of both AVN (Chen et al., 2020) and ours (SAAVN) decline. However, our method has a relatively small reduction navigation capability and speed compared with AVN, showing better robustness and navigation capabilities.
Independent learning framework does not converge in training. We have also designed learning framework where the agent and the sound attacker learn independently but not convergent. Our SAAVN employs the multi-agent learning mechanism to train the two policies simultaneously, whose benefit in convergence is empirically demonstrated by Fig.
3.0
Episode Length
17.5
reward
3.5
60-
6.
25-
6.0
5.5
1.0
1.5
2.5
3.0
0.0
05
1.0
1.5
2.0
2.5
3.0
68
65
28
15
20
15
36
Training steps
Training steps SAAVN
Training steps
AVN
IDL
Figure 8: Training curve comparison between AVN, SAAVN, and IDL.
CONCLUSION
This paper proposes game where an agent competes with a sound attacker an acoustical interven- tion environment. We have designed various games of different complexity levels by changing the attack policy regarding the position, sound volume, and sound category. Interestingly, we find that the policy of an agent trained in acoustically complex environments can still perform promisingly in acoustically simple settings, but not vice versa. This observation necessitates our contribution in bridging the gap between audio-visur navigation research and its real-worle applications. A com- plete set of ablation studies also carried out to verify the optimal choice of our model design and training algorithm. However, the limitation of our work is that only assume sound attacker and have not studied the scenario with two and more attackers. Another limitation of our work is that our current evaluations are conducted in virtual environments. It will make better sense our method on practical cases like a robot navigating in real house. The above two will be left for future exploration. Since our research is conducted on a simulation platform, unlikely to cause a negative social impact in the foresecable future.

---

Published as a conference paper at ICLR 2022
REPRODUCIBILITY
Our code available at : appx. and The algorithm parameters detailed appx.
Please refer to appx. A for more details about the acoustically clean or simple environment and acoustically complex environment. For more detailed information about the Fourier transform prop- erties, see appx. B
The dataset Replica and Matterport3D used in the experiment. For basic information about the dataset, you can refer to appx. L for more details. Please refer to hnttps://githut com/ TyEL7/SAAvW/blo.. maiin dataset .dd for the detailed steps of downloading and process- ing the dataset.
7 ETHICS STATEMENT
The research in this paper does NOT involve any human subject, and our dataset is not related to any issue of privacy and can be used publicly. All authors of this paper follow the ICLR Code of Ethics (ttps:/iiiraccppulllecCCCCotthot
ACKNOWLEDGEMENT
The following projects jointly supported this work: the Sino-German Collaborative Research Project Crossmodal Learning (NSFC 136001/DG TRR169); Beijing Science and Tech- nology Plan Project (No.Z191 1111110000000000000
the National Natural Science Foundation of China NN.62006137); Natural Science Project of Scientific Research Plan of Colleges and Universi- ties in Xinjiang (No.XJEDU2021 Y003). We gratefully acknowledge the support of MindSpore, CANN(Compute Architecture for Neural Networks) and Ascend AI Processor used for this research. We thank Mingxuan Jing and Zhenhong Jia for their generous help and insightful advice.
10

---

Published as a conference paper at ICLR 2022
REFERENCES
Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, al. On evaluation of embodied navigation agents. arXiv preprint arXiv: (tl.::180000677... 2018.
Angel Chang, Angela Dai. Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. International Conference on 3D Vision (3DV), 2017.
Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdinov. Learning to explore using active neural SLAM. In ICLR, 2020.
Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, and Kristen Grauman. Soundspaces: Audio-visual nav- igation in 3d environments. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), ECCV, 2020.
Changan Chen, Ziad AL Al-Halah, and Kristen Grauman. Semantic audio-visual navigation. In CVPR, 2021a.
Changan Chen, Sagnik Majumder, Ziad Al-Halah, Ruohan Gao, Santhosh Kumar Ramakrishnan, and Kristen Grauman. Learning to set waypoints for audio-visual navigation. In ICLR, 2021b.
Kevin Chen, Juan Pablo de Vicente, Gabriel Sepulveda, Fei Xia, Alvaro Soto, Marynel Vázquez, and Silvio Savarese. A behavioral approach to visual navigation with graph localization networks. In Robotics Science and Systems, 2019.
Victoria Dean, Shubham Tulsiani, and Abhinav Gupta. See, hear, explore: Curiosity via audio- visual association. In NeurIPS, 2020.
Ross Flom and L. Bahrick. The development of infant discrimination of affect in multimodal and unimodal stimulation: The role of intersensory redundancy. Developmental psychology, 43 I: 238-52, 2007.
Chuang Gan, Hang Zhao, Peihao Chen, David Cox, and Antonio Torralba. Self-supervised moving vehicle tracking with stereo sound. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2019.
Chuang Gan, Deng Huang, Hang Zhao, Joshua B Tenenbaum, and Antonio Torralba. Music gesture for visual sound separation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 10478-10487, 2020a.
Chuang Gan, Yiwei Zhang, Jiajun Wu, Boqing Gong, and Joshua B. Tenenbaum. Look, listen, and act: Towards audio-visual embodied navigation. In ICRA, 2020b.
Adam Gleave, Michael Demnis, Cody Wild, Neel Kant, Sergey Levine, and Stuart Russell. Adver- sarial policies: Attacking deep reinforcement learning. ICLR, 2020.
Daniel Gordon, Aniruddha Kembhavi, Mohammad Rastegari, Joseph Redmon, Dieter Fox, and Ali Farhadi. Iqa: Visual question answering in interactive environments. In CVPR, 2018.
Saurabh Gupta, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik. Cognitive mapping and planning for visual navigation. In CVPR, 2017.
Sandy H. Huang, Nicolas Papernot, Ian J. Goodfellow, Yan Duan, and Pieter Abbeel. Adversarial attacks on neural network policies. In ICLR, 2017.
Jernej Kos and Dawn Song. Delving into adversarial attacks deep policies. In ICLR, 2017.
Yen-Chen Lin, Zhang- Wei Hong, Yuan-Hong Liao, Meng-Li Shih, Ming- Yu Liu, and Min Sun. Tactics of adversarial attack on deep reinforcement learning agents. In IJCAI, 2017.
Martin Lohmann, Jordi Salvador, Aniruddha Kembhavi, and Roozbeh Mottaghi. Learning about objects by learning to interact with them. In NewrlPS, 2020.
11

---

Published as a conference paper at ICLR 2022
Tushar Nagarajan and Kristen Grauman. Learning affordance landscapes for interaction exploration in 3d environments. In NeurIPS, 2020.
Anay Pattanaik, Zhenyi Tang, Shuijing Liu, Gautham Bommannan, and Girish Chowdhary. Robust deep reinforcement learning with adversarial attacks. In AAMAS, 2018.
Lerrel Pinto, James Davidson, Rahul Sukthankar, and Abhinav Gupta. Robust adversarial reinforce-
ment learning. In ICML, 2017.
Tabish Rashid, Mikayel Samvelyan, Christian Schröder de Witt, Gregory Farquhar. Jakob N. Foer- ster, and Shimon Whiteson. QMIX: monotonic value function factorisation for deep multi-agent reinforcement learning. In ICML, 2018.
Manolis Savva, Jitendra Malik, Devi Parikh, Dhruv Batra, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub. Jia Liu, and Vladlen Koltun. Habitat: A platform for embodied AI research. In ICCV, pp. 9338-9346, 2019.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347, 2017.
Robert Samuel Simon. The challenge of non-zero-sum stochastic games. International Journal of Game Theory, 45(1-2):191-206 2016.
Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: digital replica of indoor 2019.
spaces. arXiv preprint
Peter Sunehag, Guy Lever, Audrunas Gruslys Wojeiech Marian Czarnecki, Vinícius Flores Zam- baldi, Max Jaderberg, Mare Lanctot, Nicolas Sonnerat, Joel Z. Leibo, Karl Tuyls, and Thore Graepel. Value-decoomossiioon networks for cooperative multi-agent learning based on team ward. In AAMAS, 2018.
Ardi Tampuu, Tambet Matiisen, Dorian Kodelja, Ilya Kuzovkin, Kristjan Korjus, Juhan Aru, Jaan Aru, and Raul Vicente. Multiagent cooperation and competition with deep reinforcement learning. PloS one, 12(4)c61122395, 2017.
Yapeng Tian and Chenliang Xu. Can audio-visual integration strengthen robustness under multi- modal attacks? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5601-5611 2021.
Yihan Wang, Beining Han, Tonghan Wang. Heng Dong, and Chongjie Zhang, DOP: off-po multi-agent decomposed policy gradients. In 9th International Conference on Learning Rep- resentations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https://oppeneviii net mere//oorumiit VEIVAAI
T. Wilcox, Rebecca Woods, C. Chapa, and Sarah McCurry. Multisensory exploration and object individuation in infancy. Developmental psychology, 43 2:479-95, 2007.
Huan Zhang, Hongge Chen, Chaowei Xiao, Bo Li, Mingyan Liu, Duane Boning, and Cho-Jui
Hsieh. Robust deep reinforcement learning against adversarial perturbations on state observations. In NeurlPS, 2020.
Huan Zhang, Hongge Chen, Duane S. Boning, and Cho-Jui Hsieh. Robust reinforcement learning state observations with learned optimal adversary. arXiv preprint arXiv:2101.088. 008452,,0822...... 2021.
12

---

Published as a conference paper at ICLR 2022
APPENDIX
In this section we provide additional details about:
A Definitions in details about the acoustically clean or simple environment and acoustically complex environment (As referenced in of of the main paper. ) See Appendix Afror more
details.
Properties of the Fourier transform (As referenced in Appendix B for more details.
3.2 of the main paper. ) See
C Details of eevironnment-include in of.3 the main paper. See Appendix [or more details.
SoundSpaces, rewards, and action space (As referenced
D Implementation in details of our model (As referenced in 3.3 of the main paper. See
Appendix for more details.
SoundSpaces dataset in detail (As referenced for more details.
3.3 of the main paper. ) See Appendix
Metrics(As referenced in $ of of the main paper. ). See Appendix for more details.
G Algorithm parameters in details (As referenced in 3.3 of the main paper. ). See Appendix
G for more details.
H More experimental results. It mainly includes comparative experiments in a clean environ- ment and the acoustically complex environments Replica and Matterport3D (As refer- enced in $ 4 of the main paper. ). See Appendix H for more details.
I More trajectory examples. It mainly includes the trajectory examples in the clean environ- ment and the acoustically complex environments on Replica and Matterport3DAAs refer- enced $ of the main paper. ). See Appendix [oor more details.
Independent learning framework does not converge in training (As referenced in $ 4 of the main paper. ). See Appendix for more details.
More discussion about our model SAAVN. See Appendix K for more details.
Code implementation details of our model (As referenced in [3.3 of the main paper. It mainly includes the PyTorch code implementation of the core part of our model. See Appendix [ for more details.
M Video demonstrations on datasets Replica and Matterport3ID. See Appendix details.
M for more
N Broader Impact. See Appendix
N for more details.
A DEFINITIONS.
Definition 2 Acoustically clean simple environment. The acoustically clean or simple environ- ment is described as follows: (1) The number of target sound sources is one. (2) The position of the target sound source is fixed in an episode of scene. (3) The volume of the target sound source is the same in all episodes of all scenes, and there is no change.
Definition 3 The acoustically complex environment. The acoustically complex environment referred to in this study is defined as follows: (1) There is one non-target sound source in the scene. (2) The position of the non-targel sound source is uncertain, which means that the position of the non-target sound source the scene is arbitrary. (3) The volume of the non-target sound source is uncertain. It is only known that the maximum volume of the non-target sound is equal to the maximum volume of the target non-target sound source, but the specific volume is uncertain. (4) The sound category of the non-target sound source is uncertain. Only the set to which the non-target sound category belongs the same as the set to which the target sound source category belongs. (5)The attacker has physical entity. It is like an invisible ghost. It has shape, volume, and no mass. When the agent runs in front of it, it will not block its movement or collide with the agent. This assumption is simplify the model. These non-target sounds are major obstacle to audio-visud embodied navigation, which greatly increases the search time.
13

---

Published as a conference paper at ICLR 2022
Properties of non-target sound sources The non-target sound sources have their characteristies: (1) Although non-target and target sounds belong to the same spectrum range, these non-target sounds not suitable for Gaussian use. Distribution is used for modeling, and noise is suitable for modeling with Gaussian distribution. It is difficult to model both the non-target low voice and the target's natural gas alarm sound. (2) The existence of these non-target sounds is significant obstacle to audio-visual embodied navigation, which significantly increases the search time. (3) These non-target sounds may not move like the target, they may explore around the scene. Our work is based on the scope of the above definition.
B PROPERTIES OF FOURIER TRANSFORM.
Property The following three properties of the Fourier transform need to used when proving the Theorem 7
Linearity: The Fourier transform of sum of two or more functions is the sum of the Fourier transforms of the functions.
F(a) F(b). If we multiply a function by constant, the Fourier transform of the resultant function is multiplied by the same constant. F(k . a) = k F(a).
2. Convolution: The Fourier transform of a convolution of two functions the point-wise product of their respective Fourier transforms. F(f # g) F(f) F(g).. Dirac delta function: Fourier transform of Dirac delta function is 2 So F(e)
2n
C ENVIRONMENT.
SoundSpaces. SoundSpaces uses the Habitat simulator with the publicly available Replica and Mat- terport3D environments and the public SoundSpaces audio simulation. The 18 replica environments grids constructed based on accurate scans of apartments, offices, hotels, and rooms. The 85 Matterport3D environments are real homes and other indoor environments with 3D grids and image scanning. Use SoundSpaces' room impulse response (RIR) to place the audio source and environ- mental attackers in a 3D environment, and then simulate realistic sound at each position in the scene, where the spatial resolution of the copy is 0.5m, and the spatial resolution of Matterport31) is 1m. The robot can walk in space while receiving real-time egocentric visual and audio observations.
Rewards. According the classic navigation rewards, if the robot successfully reaches the target and executes the Stop action, the environment will reward it with +10, plus an additional bonus of 0.25 to reduce the Manhattan distance from the robot the target. Furthermore, increase the equiv- alent penalty for this target. Finally, we impose a time penalty of -0.01 on each action performed to encourage efficiency.
Action space. SoundSpaces maintains a navigability graph of the environment (unknown to the agent). The agent can only move from one node to another if an edge is connecting them and the agent is facing that direction. The action space of the agent are navigation motions: A= = [MoveForward. TurnLeft, TurnRight, Stop). An environment attacker embodied in the en- vironment must take actions from hybrid action space A", The hybrid action space are Carte- sian product of navigation motions space A ppppoiiioon yoolue of sound space volume and category of sound space Arrcntegory A" poopotiionn x v.vuuume x ccnttooyy ,where A prsposition = AA is navigation motion space, AA", volume = (0.0, 0.1, 0.2...., 1.0) is discrete action space with volume = 11, Aureteegory ('telephone', person', ....) is discrete action space with [A tively, We experiment with 101 everyday sounds. We use 101 natural sounds without duplication in SoundSpaces, covering various categories: birds, air conditioners, doorbells, door openings, music, computer beeps, fans, talkers, phones, and etc.
101 respec-
D IMPLEMENTATION DETAILS.
In the following, we provide details of our reinforcement learning (RL) for navigation tasks. The environment attacker embodied in an environment must take actions from action space A to make a change of the acoustically simple environment of SoundSpaces. An agent embodied in an environment must take actions from an action space AA" accomplish At every time step, = {0, attacker and the agent obtain only partial observation of it in the form of st. Here T is maximal
end goal.
T 1} the environment is in some state, but the environment
14

---

Published as a conference paper at ICLR 2022
time horizon, corresponding to 500 actions in scene of scene for our task. The observation St is a combination of the audio, and visual inputs.
The environment attacker develops a hybrid policy TF. = D.position
vovoumee
p.calegory where
x Ft
x T
riposition A2.pusition -> IR "1,0 1,0 tively, and where ("e (Ce St, h1-1) is the probability that the environment attacker chooses to take action a" E AA at time t given the current observation 8t and aggregated past states -
2.volume Aurvllume - RR 7.0
M.calcgoryy A Aaccaleguryy + 1o1 respec-
Using information about the previous time steps h4-1 and current observation 8t, the agent develops a policy z& AA" + IR+,, where (" (a+ 8t, 1) is the probability that the agent chooses to take action a² E AA' at time given the current observation s and aggregated past states h4-1
After the environment attacker and the agent acts, the environment goes into new state 81+1 and the agent and attacker receives individual rewards ri" E I and rÿ E IR respectively.
The agent optimizes its return, i.c. the expected discounted, cumulative rewards Ch _5 The environment attacker also optimizes its return, i.c. the expected discounted, cumu- where 7 E [0, 1] is the discount factor to modulate the emphasis recent long term rewards. The value function Vi (st, ht-1) and (st, is the expected return for agent and attacker respectively. We optimize the particular reinforcement learning objec- tive directly following from Proximal Policy Optinization(PPO), We refer the readers to PPO for additional details on optimization.
lative rewards C Ei= >'re-where
We train our model with Adam with learning rate of 2.5 10 The auditory and visual encoder output 512 and 512, respectively. We use one-layer bidirectional GRU with 512 hidden units that take [It, bt] as input. We use an entropy loss on the policy distribution with a coefficient of 0.01. We train the network for 30M agent steps on Replica and 60.M on Matterport3D, which amounts to 150 and 320 GPU hours, respectively.
Following SoundSpace, we first compute the Short-Time Fourier Transform (STFT) with hop length of 160 samples and a windowed signal length of 512 samples, which corresponds to physical duration of 12 and 32 milliseconds a sample rate of 44100Hz (Replica) and 16000Hz (Matterport). STFT gives 257 x 257 and 257 x 101 complex- valued matrix, respectively, for a one-second audio clip; we take its magnitude, downsample both axes by a factor of 4, and take the logarithm. Finally, we stack the left and right audio channel matrices to obtain a 65 x 65 x 2 and 65 x 26 x 2 tensor.
E SOUNDSPACES DATASET IN DETAILS.
We used the SoundSpaces dataset. Table 3 summarizes SoundSpaces dataset, includes audio renderings for the Replica and Matterport3D datasets. Each episode consists of tuple: (scene, agent start location, agent start rotation, goal location, audio waveform). Episodes were generated by choosing a scene and random start and goal location.
Table 3: Summary of SoundSpaces dataset properties
Dataset
# Scenes
Resolution
Sampling Rate Avg. # Node
Avg. Area # Training Episodes Test Episodes
Replica
18
0.5m Im
44100Hz
97
47.24 m²
0.
1000
Matterpon3D
85
16000Hz
243
517.34 m²
2M
1000
F METRICS IN DETAILS.
Next, we narrate the navigation metrics used in Section 4 of the body paper.
1. Success weighted by Path Length (SPL): weighs the successful episodes with the ratio of the shortest path 4; to the executed path p., SPL & Σ S maxip.... where S; is binary indicator of success in episode i, and N is the number of episodes.
2. Pmmaan' stands for average episode reward of agent. See Appendix
C for more details reward.
15

---

Published as a conference paper at ICLR 2022
3. Soft Success weighted by Path Length (SSPL, or SoftSPL): Similar SPL with relaxed soft- success criteria. SSPL agent's current position to the goal when episode is finished, d, the distance from the agent's start position to the goal in the episode i, is the shortest path, pi is the executed path.
Σ max(0,
d d; ma) where d is the distance from the
4. Success Rate (SR): The fraction of completed episodes, the agent reaches the goal within the time limit of 500 steps and selects the stop action precisely at the goal location, SR x Σ S.-
5. Average Distance To Goal (DTG): The agent's 's average distance to the goal when episodes are finished, DTG N. Ei= goal when episode is finished.
vi=1 dæ where de is the distance from the agent's current position to the
6. Normalized average Distance To Goal (NDTG): NDTG = N. -i=1 where dř is the distance from the agent's s current position to the goal when episode is finished, d, is the distance from the agent's start position to the goal in the episode
Σ di
G ALGORITHM PARAMETERS.
The parameters used in model are shown in Table
Table 4: Algorithm parameters
Parameter
Replica
Matterport3
RIR sampling rate
44100
16000
clip param
0.1
0.1
ppo epoch
4
mini batch
1
value loss coef
0.5
0.5
entropy coef
0.02
0.02
learning rate
2.5 x 10
2.5 x 10
max grad norm
0.5
0,5
num steps
150
150
use gae
True
True
use linear learning rate decay
False
False
use linear clip decay
False
False
7
0.99
0.99
0.95
0.95
0.01
0.01
reward window size
50
50
success reward
10.0
10.0
salck reward
-0.01
-0.01
distance reward scale
1.0
1.0
hidden size
512
512
wl
V/6
w2
1/6
V/6
w3
V/
w4
1/2
1/2
H EXPERIMENTS RESULTS IN DETAIL.
It can be concluded from the main paper that AVN's navigation capabilities are better than SA-MDP. So we focus on comparing the navigation capabilities of AVN and our model SAAVN on dataset Matterport3D. We select the model variant SAAVN: PVC for experimental comparison prove the effectiveness of our model. Later we will show navigation results on dataset Replica.
16

---

Published as a conference paper at ICLR 2022
Navigation results dataset Matterport3D. Table shows the comparative experiments of different models on the dataset Matterport3D in a clean environment. It can be seen from Table that our model is the best under all metrics in simple clean environment.
Table 5: Performance comparison of different models, which was tested in clean environment
under all the metrics in detail on dataset Matterport3ID. Results are averaged over test runs.
Method
SPL(1)
SSPL (1)
(1)
Rmsen (1)
DIG
NDTG (J)
Random
0.000
1..000
0.000
-5.0-0.0
25.00
1.108 0.000
AVN (Chen et al. 2020
0.539=0.002
0.558+00.002 002
0.696+000004
18.1=0.2
11.85+0.19
0.300=0.006
SAAV J
0.549-0.009
0.572.10.008
0.698110.009
18.710.2
11.20.10.11
0.282-1.006
Table 6: Performance comparison of different models, tested in acoustically complex environments under different metrics in detail on dataset Matterport3D. Results are averaged over 5 test runs. The acoustically complex environment is PVC.
(P)
SSPL
(f)
Rmean (f)
DTG
NDTG
Random
0.02710.000 0.000
0.00010.000
-5.0_0.0
25.01
1.166.0.00 10.000
AVN CChen al. 2020
0.397=0.006
0.429+000044
0.612++0.005
15.3+0.1
13.65+0.13
0.374=0.005
SAAV AAVNNIVCIOURS
0.006-0.006
0.508++0.00
0.660+0.004
17.3+
12.11+0.08
0.315+0.002
Table 7: Comparison of different models in the acoustically complex environments with sound attacker under SPL and Rmean in details on Matterport3D. Results are averaged over 5 test runs.
Complex PVC
Method
(PVC handen
Random
0.000/-5.1
0.000/-5.0
AVN (Cen al. 2020
0.397/15.3
0.397/15.3
SAAVNTYCCOURS)
0.473/17.0
0.478/17.3
The experimental comparison of different models on the dataset Matterport3D) an acoustically complex environment shown in Table [6] It can be seen from Table 6 that model is the best under all metrics in all the acoustically complex environments.
Complex env in the Table 7 stands for the acoustically complex environments which includes (PVC) and PVC. As can be seen from Table / for the two acoustically complex envi- ronments (PVC) random and PVC, our model wins under both SPL and Rmean metrics.
From the above Table [ Table and Table 7 it can be concluded that our model achieves better navigation capabilities than AVN in all environments on dataset Matterport3D under all metrics.
Navigation results on dataset Replica. The results of comparative experiments in the clean environment of dataset Replica are shown in Table 8 can be seen from Table g that for the dataset Replica, in simple clean environment, the performance of our model variants SAAVN:P, V=0.9, C=person6, and SAAVN:P, V=0.1, C=person6 under all metrics are better than AVN's. which fully shows that our model has robust navigation capabilities in acoustically simple environment.
The comparative experiments in the acoustically complex environments on the dataset Replica are shown in Table 9 Complex Env in the Table 9 stands for acoustically complex environments. can be seen from Table 9 that our various model variants achieve better navigation capabilities than AVN respectively in different acoustically complex environments on dataset Replica under all metrics.
The data in Table and Table can be visualized with the histogram of Figure Figure shshws the comparative experimental results dataset Replica in all environments. The two bars on the left of each box in the Fig. 9 are comparisons of the clean environments, and the two bars on the right are comparisons of the acoustically complex environments. It can be seen from the histogram that model performs better than AVN in both simple clean environment and acoustically complex environment.
17

---

Published as a conference paper at ICLR 2022
Table 8: Comparison of AVN and SAAVN tested in the clean environment under all the metries in detail on Replica. Results are averaged over 5 test runs.
Method
SPL
SSP
(1)
Rmeon (7)
DTG
NDTG
AVN (Chen et al. 2020
0.721=00006
0.749-00005
0.897=00004
15.120.1
0.68:00.06
0.059=0.003
SAAVNIVEEL. C. Spersondi Ours)
0.742:10.005
0.757:10.006
0.960=0.002
16.6:10.1
0.18:10.04
0.017=0.003
SAAVN C=personn Durs)
0.730:10.007
0.7541:0006
0.943=00004
16.410.11
0.19::0.02
0.022=00003
MAAYN.VV=0..C-erraaa Ours)*
0.755:10.004
0.776::0.003
0,948=0.007
16.410.1
0.224:0.03
0.022==00003
SAAYN:P... CWWNVPPa.CC--eesonncCuur
0.728::0.022
0.742=0.003
0.938=0.004
15,940.1
0.38+00.06
0.033=0.003
SAAVN=Paa. (.COus)))
0.685++0.009
0,700+0.0008
0.904+0.007
15.1++0.1
0.68++0.04
0.060=000004
SAAVN: VAACNEPP..VCCOUEE) V.C(Ours)
0.710::00044
0.720:100055
0,941=0.003
15.8=0.1
0.4410.07
0.038=00004
0.723++0.007
0.747+0.006
0.919+0.004
15.6++0.1
0.49+00.15
0.043+0..03
SAAVN.PV=D...C(Ous))
0.698:10.005
0.705
0.03
15.740.11
0.57::0.04
0.044=0.003
+-000-
0.695
0.954+0.005
16.140.1
0.33++0.06
0.025+0.004
SANVN.RCC-persaddOOuu
0.663:10.009
0.692-10.006
0.901
15.3:0.2
0.52+10.08
0.047=0.006
SAAVNPP..C(Ous))
0.664:00010
0.677=0.008
0.902=0.012
14.8:0..3
0.87+0.11
0.064=0.008
Model variant V........VV.....pprrre. ranked second in performance metries SPL and SSPL, and ranked first in performance on metries Sk, Rmem: DTG and NDTG.
: Model variant SA,VN:P, V=0,9, C=personó ranks first in performance on metrics SPL and SSPL, ranked second in performance on metrics Raean. DTG and NDTG, and ranked third in performance on the metrie SR. Based on the above factors, we believe that model variant SAAVN:P, 1= C=person6 is the best, So in Table and the main paper, mark model variant SAAVN:P. V=0.1, C=person6 as the best model.
Table 9: Performance comparison of AVN and SAAVN, which is tested in acoustically complex environments with a sound attacker under all the metrics in detail on Replica. Results are averaged over 5 test runs.
Method
Complex Env.
SPL
SS).
O
Ruces (1)
DTG
NDT
AVN (Chen et al.
2020
0.609==0.005
0,678=00004
0,712400010
11.0=02
2.20+0006
0.194=00004
SAAVNOOUST
C.V=....=eeerso66
0.657-+0.005
0.726+00005
0.799-10.006
13.2-0.1
1.17:0.05
0.107=0.005
AVN Chen et al.
SAAVIOOUSI
2020
C.
0.316#0.014
0.689=0.010
0,565=00011
0.748-0.007
0.712+00012
0.841=0.009
10.7=02
14.4-0.1
2.4420.09
0.7240.02
0.207=00005
0.074-0.001
AVN (Chnn et al.
STANKTOUST
2020
C.v=...C-persom6
0.413=00003
0.692:40.008
0,470=000004
0.748=00005
0.687=00008
0.858-+0.009
982002
14.7-0.1
2.6620.10
0.70=00.4
0.228=0.009
0.066-000003
AVN Chen et al.
2020
0.549=0.0099
0.598:0.007
0.723+00009
11.0=022
2.3320.06
0.196=00000
SAAW
0.648:0.010
0.724:1.002
0.80410.016
13.6:0.2
0.93::0.6
0.005
AVN Chen al.
2020
Vr..V...ICC
0.576=0.005
0.636:20.088
0.694200008
10.5=002
2.58+0.10
0.222=0.010
SAAW
0.609:000044
0.645
0.793:100007
12.7=0.1
1.55.0.03
0.032
AVN Chen
2020
Pr.VCCC
0,361
0.002
0.46010.010
7.6.1.22
3.8310.13
0.001
STAWSTOUSS
0.638:0.010
0.683:1.0007
0.20010.013
13.4:00.
1.14:0.08
0.103
AVN Chen
2020
P.V.0.1.C
0,563100005
0.63010.022
0.0000
10.310.1
2.6010.06
0,226 0X004
STASS
0.667-1.009
0.714:10008
0.22:10.012
13.4:0.3
1.26:0.10
0.102:1.0007
AVN Chen 2020
P.V=0..CC
0.062
0,469 10.00
7.510.1
3.8910004
0.384 0.00
SAAWSTOUSS
0.644:10,066
0.665:10.006
0.78810.003
14.4=000
0,9710.02
0.000
AVN Chen
2020
P.V09..CC
0,269
0.2660
0.040000006
6,1102
4.3610.12
0.453 00012
STUSS
0.620:10.004
0.603
0.80110.009
13.2:001
1.29:0.06
0.112:10.003
AVN Chen 2020
P.V.C-peroon6
1.54T00000
0.59010.0022
11.0...1
2.3010.04
0,195 TO.0S
STASS(OUS)
0.630:1.0055
0.688:10.004
0,777.1.009
12.7=002
143:0.08
0.029
AVN Chen et 2020
P.V.C.
1.18910000099
0.51610.0055
0001
8.010.2
3.6910.08
0.359 0000R
SAANS(OUS)
0.552:10.004
0.598:10.004
0.716:10.003
10.6:20.1
2.5610005
0.207
I TRAJECTORY EXAMPLES IN DETAILS.
Trajectory comparisons on dataset Replica. Figure 10 shows the test episodes for our SAAVN model on dataset Replica. The environment of each row in Fig. 10 consistent, and the model of each column too. SA-MDP failed to complete the task successfully in all two environments. AVN can complete the job in clean environment but fail reach the target in acoustically complex environment. SAAVN completed the tasks in all two environments. What is more, the navigation track of SAAVN in acoustically simple environment is shorter than that of AVN. These fully demonstrate the navigation capabilities of SAAVN.
Trajectory comparisons on dataset Matterport3D. Figure 11 and Figure 12 shows the test episodes for our SAAVN model on dataset Matterport3DD. The environment of each row in the fig- ures is consistent, and the model of each column is too. AVN can complete the task in clean environment but fails in an acoustically complex environment. SAAVN completed the tasks in all two environments. What is more, the navigation track of SAAVN in an acoustically simple environ- ment is shorter than that of AVN. These fully demonstrate the navigation capabilities of SAAVN.
18

---

Published as a conference paper at ICLR 2022
Blean
PAMOS,
Clean
RI-4L5,
Clean
PX-03
dibon
VC=
Coan
C.V-0.1
positionylluree calegory sound attacker policy
Clcan
Clean
(b) position.oouume category
policy learnable.
Clcan
PVLC
10) volumeon and category of the sound attacker policy are all learmaalle..
Figure 9: Comparison in the clean environment and different acoustically complex environments between AVN and SAAVN on Replica.
AVN
SA-MDP
SAAVN
Clean environment
Acou com env
Agent
Shortest path
Agent path
Seen/Unseen
Occupled
Figure 10: Trajectories of different models in different environments on dataset Replica. The first in the figure clean environment, and the second line is acoustically complex environment. Each column in the figure represents the same model. Acou com env stands for acoustically complex environment.
19

---

Published as a conference paper at ICLR 2022
AVN
SAAVN
Clean environment
Acoustically complex environment
Agent
Start
Goal
Attacker
Shortest path
Agent path
Seen/Unseen
Occupied
Figure 11: Trajectories of both AVN and SAAVN in different environments on dataset Matterport3D. The first row in the figure is clean environment, and the second line is a acoustically complex environment. Each column in the figure represents the same model.
20

---

Published as a conference paper at ICLR 2022
AVN
SAAVN
Clean environment
Acoustically complex environment
Agent
Start
Goal
Attacker
Shortest path
Agent path
Seen/Unseen
Occupied
Figure 12: Trajectories of both AVN and SAAVN in different environments on dataset Matterport3D. The first row in the figure is clean environment, and the second line is a acoustically complex environment. Each column in the figure represents the same model.
21

---

Published as a conference paper at ICLR 2022
J INDEPENDENT LEARNING FRAMEWORK DOES NOT CONVERGE IN TRAINING.
Conventional adversarial approaches usually train the agent's policy and the attacker's policy alter- nately and independently. Instead, our paper employs the multi-agent learning mechanism to train the two policies simultaneously, whose benefit in convergence is empirically demonstrated by Fig- ure g (by comparing our SAAVN with IDL). We have also designed a learning framework in the algorithm design process in which the agent and the sound attacker learn independently. It can be seen from Figure that the framework of independent learning(IDI. by the agent and the sound attacker convergent.
K MORE DISCUSSION ABOUT OUR MODEL SAAVN.
Although our work primarily focuses on intervention on the sound modality. We have some discus- sion on the robustness of SAAVN against the visual modality intervention, the robustness of SAAVN on robot blindness, etc. For more details, please see below.
K.1 ROBUSTNESS OF SAAVN AGAINST THE VISUAL MODALITY.
Our work primarily focuses on intervention the sound modality. What is robustness against the visual modality? We add gaussian noise with different std and zero mean to the visual observation (depth image) and evaluation the performance of model SAAVN on Matterport3.. The result demonstrated in Table T0 show that When the intensity of intervention increases, performance decreases. However, the decline is slow. This phenomenon shows that model SAAVN shows better robustness under visual modal intervention.
Table 10: Performance (SPL (t)/Rmeon (f)) under visual attacking on Matterport3D.
Method
SPL RtW/Rmmnn (
w/o noise
0.478/17.3
std=0.01
0.476/17.2 0.475/16.9
std=0.05
K.2 ROBUSTNESS OF SAAVN AGAINST BLINDNESS.
When the visual sensor is wholly removed, that is say, and the robot is blind. What is the per- formance of our model and AVN? We design and do blindness experiments on Matterport3D. The experimental results in Table IT show that the performance of SPL and Rmean of model AVN and model SAAVN has decreased. ASPL stands for the deterioration amount under metric SPL for a specific model. The performance value calculates ASPL with removing visual sensor interven- tion minus the commission value under metric SPL. without intervention under metric SPL. A Rmmean stands for the deterioration amount under metric Rmean for specific model. A Rmean calculated by the performance value with removing visual sensor intervention minus the commission value without intervention for a specific model under metric Amean- The performance degradation of SPL and Rmean of model AVN is greater than that of model SAAVN. It shows that the SAAVN model is more robust than the model AVN when faced with the complete removal of the vision sensor.
Table 11: Performance (SPL (t)/R (f)) in the environment with a PVC attacker on Matter- port3D. Compared with AVN, our SAAVN performs more robustly without vision, which again exhibits the benefit of our adversarial training.
Method
Visual + Sound
Sound
A SPL/A Amean
AVN
0.397/15.3 0.478/17.3
0.196/9.2 0.333/15.1
-0.201/-6.1
SAAVN (Ours)
-0.145/-2.2
22

---

Published as a conference paper at ICLR 2022
K.3 PERFORMANCE AFFECT BY SLIDING AND SKIPPING MODES OF a".voi.
The action a can take by sliding and skipping modes. Suppose the current action a is 0.5. What the step action ? If using the sliding mode, the next action of a":vol is to select one from neibour actions ([0.4, 0.5, 0.6]) of current action. If using the skip mode, the next action of a.vool is select one from whole action space of a. ([0.0, 0.1, 0.2. 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)). How does the choice of action mode affect the performance of the model SAAVN? The experimental results on Matterport3D in Table [2 show that the performance of model SAAVN adopts the sliding mode is better than that of the skipping mode for action a" under metrics SPL. and Rmean-
Table 12: Performance under sliding and skipping modes of vol on Replica.
Method
SPL RmeRn
Skipping Sliding
0.552/10.6
0.614/13.8
K.4 UNSEEN ENVIRONMENTS.
We test our method by assigning the attacker with the sound of random category that is unseen in training. The following table summarizes the results of AVN and SAAVN. As a reference, we also copy the results from Table 2 that are evaluated under the original "seen" setting. suggests that SAAVN still performs desirably. particularly for the environments "V" and "C", whereas AVN
yields more significant detriment in these two cases (See Table 13 14 15).
Table 13: Performance in and Unseen environments under SPL (1)/Rmean (f) on Replica.
Method
Unseen
A SPL/A Rmean
AVN 0.609/1 1.0
0.568/10.2
-0.041/-088 -0.045/-2.0
SAAVN:P
0.657/13.2
0.612/11.2
Table 14: Performance in V and V Unseen environments under SPL (t)/Rmeon (1) on Replica.
Method
V
Unseen
A SPL/A Rmean
AVN SAAVN:V
0.549/11.0
0,445/10.0
-0.104/-1.0 -0.052/-1.3
0.648/13.6
0.596/12.3
Table 15: Performance in C and C Unseen environments under SPL (f)/Rmean (1) on Replica.
Method
C
Unseen
A SPL/A Rmean
AVN SAAVN:C
0.576/10.5
0.394/6.7
-0.182/33.8
0.609/12.7
0.608/13.2
-0.001/0.5
K.5 SAAVN: PVC ACHIEVES THE BEST PERFORMANCE.
Here, we further evaluate different methods in the same attack environment, "PVC," as follows. As expected, SAAVN: PVC achieves the best performance (See Table 16].
K.6 MULTI-MODAL FUSION ABLATION FOR SOUND ATTACKER AUDIO-VISUAL NAVIGATION.
Is concatenation the best choice for multi-modal fusion? We did not explore which choice is the best for multi-modal fusion in the main paper since this not the main focus of our paper (all methods share the same fusion setting). However, we do think this is exciting point. Hence, besides concatenation, we have also conducted another fusion strategy using element-wise multiplication. Specifically. first obtain the visual and audio embedding vectors of the same size (i.e., 512) and then compute the element- wise multiplication between these two vectors. Table reportss the performance of SAAVN: PVC in the environment PVC under these different fusion scenarios.
23

---

Published as a conference paper at ICLR 2022
Table 16: Evaluation of different variants under the same PVC attack environment (SPL (1)/mmaa (1)) on Replica.
Method
PVC Unseen
SPL/A Honson
PVC Unseen
SAAVN:P
0.657/13.2
0,286/6.1
-0.371/7.11
PVC Unseen 0.415/9.2
SAAVN.V
0.648/13.6
-0.233/-4.4
PVC Unseen 0.394/9.0
SAAVN:C
0.609/12.7
-0.213/3.7
PVC
Unseene 0.548/10.5
SAAVNPVC
0.552/10.6
-0.004/-0.1
Interestingly, the new fusion strategy better than the original concatenation. Upon the observation here, we believe that our proposed method allows further extendibility and can facilitate various following studies.
Table 17: Multi-modal fusion ablation on Replica.
Fusion Concatenation
SPL (t)
ST)
SR (?)
Rmean (
(L)
NDTG (J)
0.55240.004
0.598:0.004 0.635+0.005
0.716:00.003
10.630.1
2.5630.05
0.20740.003
Element- wise multiply
0.592*0.005
0.768=0.010
11.8:0.2
2.05+0.09
0.164=0.007
L CODES.
Codes are in the folder named code of the supplementary materials. The usage and main introduction of the code are in the readme.MD.
M VIDEO DEMONSTRATIONS ON DATASETS REPLICA AND MATTERPORT3D.
Video demonstrations of navigation trajectory in an episode of different models in the clean envi- ronment and acoustically complex environments are the folder named Vemo of the sup- plementary materials. There are two subfolders under this folder named DemonstrationOODDDan (Replica and DemonssrttionOmDttceeMMttttppttt
used to store trajectory videos on the dataset Replica the dataset Matterport3D. respectively. Each MP4 file is named according the format: xxx_in-yyy_ene wyy___e___,, where xxx represents the name of the model, æar E (AVN, SAAV N), yyy represents the type of environment, yıV E (simple, complex)) and 772 represents the name of the datasets, zzz E [Replica, M Matterport3D).
N BROADER IMPACT.
Our research will help service robots provide better navigation services in home life scenes and in- door office scenes. The setting of our research work is based on the complex acoustic environment. The gap between this research setting and the actual application setting is small, conducive to ap- plying the research results real application scenarios. The limitation of our work is that only assume one sound attacker and have not studied the scenario with two and more attackers, which will be left for future exploration.
24


Reviews:
[{"review_id": "h3ofzPa39uM", "reviewer": "Reviewer_TLMn", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Main questions\n1. In table 2, the SAAVN:PVC is the final proposed method, but it is not the best performing one compared to P, V, C. Why? \n2. The author mentioned \"SA-MDP: A model aims to improve the robustness by state adversarial. We adopt its idea but only intervene state of the sound input and not process the visual input\". Then, SA-MDP should be better than AVN. Why does this model keep worse than AVN? If the experimented environment includes visual attack as well as audio attack, could the SA-MDP  be the best?\n3. Is this work the first audio-visual navigation method with attackers?\n\nMinors: some notations should be 'italic' font in sentences. For example, in the sentences between eq (1) and eq (2), 'G' should be 'italic' font if they denoting 'G' in eq (1) and (2). In other parts, there seems similar typos. ", "summary_of_the_paper": "This paper addressed the audio-visual navigation task in the environment of sound attacker. The authors formulated the problem as zero-sum game between sound attacker and the agent, and provide a reasonable proof to their formulation and solution. They showed better results than existing works which don't considered this kind of attack (but, I am not sure for SA-MDP). ", "summary_of_the_review": "The environment with the attacker is practical and interesting. The authors well-formulated the action of the agent and attacker.\nAlso, including the appendix, they tried to provide various experiments and analyses.\nBTW, I have some questions to convince the efficacy of the proposed method.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "YuRNt-zzBTY", "reviewer": "Reviewer_yUmi", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n+ The motivation is clear and the proposed sound adversarial audio-visual navigation is interesting to me. Beyond clean environments with single sound sources, this work explores more complex environments considering potential attacks from non-target sounds in audio-visual navigation. \n\n+ A joint training paradigm for the agent and the attacker is proposed.\n\n+ Experimental results on Replica and Matterport3D can validate the superiority of the proposed method over recent approaches. \n\nWeaknesses:\n\n- Some important details are missing. \n(1) How does visual information help AudioGoal navigation? The perceived visual content is mostly unrelated to target sound sources. So, what cues we can obtain from the RGB frames and depth maps to support navigating to correct sound sources? \n(2) Whether the agent could be stuck in certain false local states due to the attacks? If so, how does the proposed joint learning method help to mitigate the problem?\n\n- Fairness of evaluation. It seems that the attacks are different when attacking different methods during evaluation. Did the authors try to add the same attacks during measuring different approaches? \n\n- Unseen Environments. To measure robustness, an important study is to investigate whether the trained model can perform well in unseen environments. Did the authors let the agents explore unseen environments (e.g., containing attacks from unseen sounds)?  \n\n\n", "summary_of_the_paper": "The authors address the audio-visual navigation task in this paper. Previous works usually assume that the interacted environment is clean containing solely the target sound. Different from them, this work explores an acoustically complex environment in which, besides the target sound, there exists a sound attacker playing a zero-sum game with the agent. Specifically, the attacker can move and change the volume and sound category to fool the agent while the agent tries to defend against the attack and navigate to the goal under the intervention. Experiments on Replica and Matterport3D can verify the effectiveness and the robustness of proposed model. ", "summary_of_the_review": "Overall, this is an interesting paper to explore audio-visual navigation in more complex environments and the proposed sound adversarial audio-visual navigation is well-motivated. The extensive experiments can demonstrate the superiority of the proposed method in handling sound attacks. But, I do have some concerns about the method and results. My current rating is borderline accept. But, I would like to upgrade my rating if the authors can address my concerns during rebuttal. \n\n***Post-Rebuttal***\n\nThe authors successfully addressed my concerns in the rebuttal. I am happy to upgrade my rating. \n\nThere is a very recent work on audio-visual adversarial robustness. I'd like to suggest the authors to discuss this relevant work in the revised paper. \n\n[1] Tian, Yapeng, and Chenliang Xu. \"Can audio-visual integration strengthen robustness under multimodal attacks?.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "J0JkFB31Ahq", "reviewer": "Reviewer_i5Vv", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main contribution is the addition of adversarial attacker in training AVN. This can be seen as a complex type of noise augmentation. The results are generally positive, showing significant improvements over baselines. \n\nAblation studies w.r.t different test environments (e.g. different positions, volumes, noise types) are reasonable. However, it would be good to see ablations w.r.t. design choices made. For example, if the noise sources are random P,V,N instead of adversarial, what would the performance be? Is concatenation the best choice for multi-modal fusion?\n\nThe paper needs to be proofread again – singular-plural errors (e.g. space are …), grammatical errors (e.g. why are our model work?), spelling (e.g. neibour)  \n", "summary_of_the_paper": "The paper proposes an interference-robust training method for audio-visual navigation. Unlike existing approaches that focus on clean environments, the system is trained in simulated acoustically complex environments. A single-source adversarial attacker is introduced, which determines position, noise type and volume that would make the agent to suffer most. This is claimed to improve robustness to random attacks in AVN. Results are compared to a small selection of previous works.", "summary_of_the_review": "The paper presents interesting solution for AVN in noisy conditions. The results are better than baselines, but the paper would benefit from more ablations. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses audio-visual navigation tasks where a reinforcement learning agent perceives visual RGB and binaural audio inputs, rendered in a first-person perspective 3D environment, and is tasked to navigate to the audio source. The authors propose to make the RL navigation policy robust, by training the agent with additional adversarial audio perturbations. These perturbations consist of an adversarial "ghost" agent (attacker) that emits noise perturbations volume, position and category determined by policies that are trained to maximise the negative rewards for the navigation agent in zero-sum game. The agent is then evaluated on the simulated Replica and MatterPort3D environments and compared to a few baselines. The authors conduct a large number of ablation experiments.

The three reviewers were globally positive about the paper, regarding the motivation, joint training of the agent and attacker, and experimental evaluation. Reviewer TLMn had questions about specific results and ablations of existing baselines, whereas reviewer i5Vv had questions about random noise ablations - the authors provided responses for these questions. Outstanding requests were about proofreading.

After rebuttal and discussion, the scores for this paper are 6, 8 and a weak 8 (or 7), i.e., an average of 7, and thus I believe that the paper meets the conference acceptance bar.
Decision: Accept (Poster)