Manipulating Multi-agent Navigation Task via Emergent Communications
Authors: Han Yu
Hengtong Lu
Caixia Yuan
Xiaojie Wang
Venue: Submitted to ICLR 2023 (2023)

Abstract: Multi-agent corporations struggle to efficiently sustain grounded communications with a specific task goal. Existing approaches are limited in their simple task settings and single-turn communications. This work describes a multi-agent communication scenario via emergent language in a navigation task. This task involves two agents with unequal abilities: the tourist (agent A) who can only observe the surroundings and the guide (agent B) who has a holistic view but does not know the initial position of agent A. They communicate with the emerged language grounded through the environment and a common task goal: to help the tourist find the target place. We release a new dataset of 3000 scenarios that involve multi-agent visual and language navigation. We also seek to address the multi-agent emergent communications by proposing a collaborative learning framework that enables the agents to generate and understand emergent language and solve tasks. The framework is trained with reinforcement learning by maximizing the task success rate in an end-to-end manner. Results show that the proposed framework achieves competing performance in both the accuracy of language understanding and the task success rate. We also discuss the explanations of the emerged language.

Under review as a conference paper at ICLR 2023
MANIPULATING MULTI-AGENT NAVIGATION TASK VIA EMERGENT COMMUNICATIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Multi-agent corporations struggle efficiently sustain grounded communications with specific task goal. Existing approaches are limited in their simple task set- tings and single-turn communications. This work describes multi-agent com- munication scenario via emergent language in navigation task. This task involves two agents with unequal abilities: the tourist (agent A) who can only observe its surroundings and the guide (agent B) who has the holistic view but does not know the initial position of agent A. They communicate with the emerged language grounded through the environment and a common task goal: help the tourist find the target place. We release a new dataset of 3000 scenarios that involve such visual and language navigation. We also seek to address the multi-agent emergent communications by proposing a collaborative learning framework that enables the agents generate and understand emergent language and solve task. The frame- work is trained with reinforcement learning by maximizing the task success rate in an end-to-end manner. Results show that the proposed framework achieves competing performance in both accuracy of language understanding and in task success rate. We also discuss the explanations of the emerged language.
INTRODUCTION
Communication is crucial factor for multiple agents cooperate. While most recent works focused on the interactions between the artificial agent with humans, there still are some research efforts made on communication between artificial agents. However, most of these works are focused on single-turn communication with a unidirectional message pass and the evolvement of natural language or some specific properties of the emergent language like compositionality, interpretability, and so on. But the multi-turn conversation is more analogous to human language. In natural conversation, language generation and understanding should be mutual rather than unidirectional. Therefore, we provide framework for agents to generate multi-turn dialogues including two agents.
To prove the feasibility of our framework, we provide a scenario for agents to generate multi-turn conversations. We propose new task adapted from the vision-language navigation (VLN) task coming from the numan-machine communication area where the guide should communicate with the tourist give guidance and help find the target location. Different from traditional VLN tasks, in our settings, the tourist(agent A) and the guide(agent B) are both machines, rather than the original suman-machine settings. Besides, we suppose that the guide does not know the initial position of the tourist, so the guide not only has to give guidance to the tourist but also confirm the location of it.
Our contributions can be summarized as follows:
1. From the view of emergent language, study the language with multiple turns. To give a suitable scenario for multi-turn conversations, we provide a navigation task adapted from the vision-language navigation(VILN) task.
2. Compared with methods with agents speaking the natural language, ours is cheaper with no expensive annotation, making it more practical way for agents to communicate.
3. As far as know, We the first to propose VLN-like task in two-agent cooperation scenario. And we also provide a benchmark for it, which gives a possible solution to this kind of task.

---

Under review as a conference paper at ICLR 2023
2
RELATED WORK
Vision Language Navigation(VLN) Navigation tasks with vision and natural language information have attracted much attention in the last few years. Generally, this task involves humans, an artificial agent, and the environment. The agent communicates with humans in natural language. The agent may receive requests or instructions and ask for more guidance from the human. It can navigate and interact with the environment to get more information or complete corresponding task requested by the human. While humans should give requests or guidance to help the agent complete the task.
Generally, the goal of these VLN tasks is to help the agent find the target place known only by the human side. AAndrson et al. (017)) proposes a VLN dataset called Room-to- too-m (R2R) based on matterport3dCChng et al. al. (2017) data and provides a benchmark for this task first. Chen et al. (2018) proposes dataset Touchdown and applies the VLN task to the outdoor environment. Still. some scenarios where agents need to interact with objects in the environment are proposed. Narayan- -henn ct al. (2019) put the task into the Minecraft game and instruct the agent to complete building tasks.
From the view of language format, we can divide those tasks into three categories Gu et al. (2022
Initial instructions: The agent is given instructions for the whole navigation episode. In some of those works, the instructions are very detailed Anderson et al. (2017); Chang et al. (2017); :Jin et al. 2 Krantz al. (2020). For example. "Walk down one flight of stairs and stop on the landing". While in other works, some coarse-grained instructions are given Zhu et al. (202I). However, this kind of work only gives the instructions at the beginning, so multi-turn communication can not be generated.
Oracle guidance: In the process of navigation, the agent can request additional natural language guidance to get more information Chi et al. (2019). But in most cases, the oracle can only respond with simple words like "turn left" and "turn right".
Dialogue: Dialogue is given supervised signal,de Vries et al. (2018): Thomason et al. (2019). But the fixed dialogue constrains the flexibility of policy.
By putting the VLN task into multi-agent settings, the problems listed above can be alleviated or solved. To our knowledge, talk-the-walk, de Vries et al. (2018) is the only work applying the VLN task with multiple agents and is very similar to our task settings. However, talk-the-walk only focuses on the localization part, where the guide finds where the tourist is. But ignore the guidance part, in which the guide makes a route plan and gives guidance to the tourist, and the tourist makes corresponding actions. Talk-the-walk models this part with a random walk and only implements minimal baseline, making the VLN task with multi-agent settings incomplete.
Emergent Language Emergent language is an unplanned language that comes up in interactions between the students, and/or the teacher and the students. From the perspective of agents' relation- ships, most of the works are focused on fully cooperative tasks like referential games] Lazaridou et al.) (018)) adapted from the Lewis signaling game. In the referential game, there are two roles: speaker and listener. The speaker has give instructions to the listener so that it can select the target picture from the candidates successfully. Some navigation tasks are proposed Kalinowska et al. (2022); Kajié al. (2020), however, those works can only be applied in limited environments, with 2 and 4 cases respectively. Some semi-cooperative tasks are also proposed, like the negotiation game Cao al. (2018), in which two agents have negotiate to get more scores for themselves. There are also some fully compeeiivee cases like circular- where the sender tries to make the receiver choose the position closer to own target position rather than the receiver's. s.. Most of those part of works are studying single message pass, with one agent only having a language generation part, and the other one having a language understanding module. Still, there are some works putting attention to the multi-turn form Cao et al. (2018); Evtimova et al. (2017). But the former study focuses on a semi-cooperative case rather than a fully cooperative one like us. while the latter one uses the bag of words, and generates 0 or 1, with indicating this token is in the generated sentence. This method makes that the latter token does not rely on the former ones, making it not sentence from general views.
sender-received game Noukhovitch et al. (202T)
2

---

Under review as a conference paper at ICLR 2023
Agent
Agent
?
Figure 1:
The navigation task involves three components: the tourist (agent A), the guide (agent B), and the environment. Agent A and B have unequal abilities. While agent A can only see its surroundings and describe them B, agent B has the global perspective, can make route plan, and give instructions to A. Agent A and B cooperate to lead A to the target place.
17. 23,
Ploute Plan
20,20,30 20, 34,34
Figure 2: Navigation Case. The navigation task consists of three sub-tasks: Localization, Route Plan, and Guidance. At every turn, agent first gives a description of its surroundings in the room it is located. With the current messages and the history messages given by A, agent B tries to guess where A is using the topological structure of the rooms. Then, based on this guess, agent B makes route plan for A get the target place, and gives instruction A describing the step A should take.
3

---

Under review as a conference paper at ICLR 2023
3 NAVIGATION TASK
This task is inspired by a real-world scenario where two agents with unequal abilities, agents A and B. cooperate to help agent A find the target place. As pictured in Figure [ agent A can only see the objects around it, while agent B has the floor plan and the target place but doesn't 't know where A is. To win this game, they have to cooperate to help A reach the target place with limited rounds. Otherwise, this task fails.
Here, we divide the navigation task into three steps: First, localization. Agent needs to find where A is, according to the message given by A describing the objects around it. Second, route plan. Having the position of A and the target place, agent B has to find the shortest path for to get there. Third, guidance. Agent B gives instructions to A, helping it follow the route plan. All three steps above are executed iteratively until A reaches the target place or the steps run out.
Formally, this navigation task is constructed as follows:
There are houses, containing rooms R = {To, "i,...g "x) each. Every time, one house is randomly chosen from those K candidates. Then two rooms are drawn from the selected house, and one of them is the target room To. and the other is the start room r' Every room has several gates Neighhor(i)) There is graph for each house, to model the relationship between rooms. We use G = (R. D) to denote a house, where R is the vertex set, and is the edge set. There are two players called agent A, and agent B parameterized by 0 On separately. The inputs and outputs are different with different tasks.
D, {dij.j
the localization task, agent A takes in the room's vector where and gives out the messages of step consisting of M tokens m, {Vo, U1...., UMr E V. While agent B collects history dialogue h₁ = (Mo, (i....,,) and tries to guess where A is.
In the route plan task, agent uses the room graph G, the guessed position of A arg max(0p(h), G)). and the target position To to get the next step A should take arg max(0 (hc++,,)
The only difference between guidance and localization task is that the guidance task switches the role of agents A and B, and agent A does not require the graph G.
A case is shown in Figure 2 where Agent A starts from room 0 and wants to get to room 5.
4 MODEL
Figure shows the model structure for the localization task. This model includes an encoder encod- ing the surroundings of A, a route encoder embedding the neighbor information by sampling routes ended with the target room, language generation module in agent A implemented with GRU, and a language understanding module in agent B used for guessing the position of A.
Firstly, the information about the house containing features of set of rooms (ro, Ti, FN) is given. Agent A encodes the target room information To including objects and gates and then uses the en- coded vector to generate corresponding emergent language with its language generation module GRU GRU(M (MLP(rr)).
ef = sentences from A, agent B makes its guess by making use of the topological structure of rooms pn (he, G). In this topological graph, every room is represented with a node, and every door con- necting neighbor rooms is represented with an edge between these rooms. For node k, sample T routes with the same length of history movement length t ended with this node routef. Node k has representation e = + E. GRU (route,) including topological structure information.
With the message coming from agent A in this turn, and the history
And B encodes the history sentences with another GRU and computes the similarity between this encoded history Em, with each node representation ex, calculated above to find the most possible position of agent A. Similarly, the model for the guidance task also includes encoder, language generation, and a language understanding module used for giving instructions to agent A about its next step.
Considering the non-differentiabiliyy of the discrete messages passing between agents, those models trained with REINFORCE algorithm. We use the Floyd algorithm for the route plan task to find
4

---

Under review as a conference paper at ICLR 2023
Route Plan
Sruxtro Encoder
Roste Sampling
Semilafty Sare
Guicance
Agent
MP
H4 Hittry Menzages h
Room Encoder
Hatoryy Massages foos
GaU
Agent A
Bamiutryy Score
Figure 3: Model Structure. There are three sub-modules here, including localization, route plan, and guidance. In the localization part, by calculating the similarity score between messages given by A and room feature embeddings incorporating structural information, agent B tries to find where is. Then we use the Floyd algorithm find the shortest path between the guessed place and the target one. And in the guidance module, A could find its path with history dialogue with B and its own observations.
the shortest paths. Losses in localization and guidance tasks can be calculated with the following equations:
M
a Jooaliiaation El
R(-)
Σ
Vo log (Us(ss))
M
Va Joocatization
E (R(.)
Vo,
log nB (r|sB)]
(1)
M
00 Jguidance
E
R(-)
Σ
Veg log (
M
a Jyuidance = E(R(-) V8a log n (d[sa})
Finally, we put those sub-tasks together to complete the navigation task. In the training process, the input of the localization part is the surroundings of agent A, and the output of it is the guessing position made by agent B. However, the input of the route plan part the next timestep is the actual position of A, instead of the guessing one, preventing the error from localization influencing other modules. While in the testing process, the input of the route plan module comes from the last localization module.
EXPERIMENT
5.1 DATASET
We construct the dataset with 6 rooms for each navigation task. And every object in this dataset has 3 optional colors and 3 optional shapes. Also, some features not relevant to this task are added, like the gate color which we suppose to be the same among all rooms in the same house.
5.2 DISCRETE VAE
Rolfe (2016) proposes discrete VAE. By solving the problem of backpropagation through discrete variables, discrete VAE successfully constructs the semantic representation of pictures.
5

---

Under review as a conference paper at ICLR 2023
Localization
Navigation
0.9
new mm/w
0.8
M Aidga
0.6
0.5
0.2
2300
2000
7500
10000
12:00 epoch
13000
400
800
1000
1200
spoch
(a)
(b)
Figure 4: The change of accuracy epoch in different tasks. (a) localization task. (b) navigation task
Avg Reward
training
Localizat on in validation set Guidence in validittion set
1000
2000
3000
4000
5000
epoch
Figure 5: The reward changes of the training set and the validation set. The rewards tend to be stable over epochs, proving the effectivvness of our framework.
We apply the discrete VAE comparable model here by replacing the room encoder and gate en- coder together with the language generation part. Fixing the discrete VAE part, train other modules, then get comparable model.
5.3 RESULTS
Figure Figure 5 show the accuracy change in the validation set and the reward change over the epoch separately. In this experiment, the length messages passing between agents A and B is 3, and there are 20 candidate tokens in the localization task and 40 candidate tokens in the navigation task. It is worth noting that, since trained from scratch, there no initial meaning for those candidate tokens.
Table gives some comparisons between the model adapted from discrete VAE and our model. should be noted that the adapted discrete VAE already has the initial meaning of each token before training on the specific tasks. Considering that the presentations of room and routes are trained based on the reconstruction task in the discrete VAE model, making the discrete presentations contain more redundant information. With the limitations of the message length and vocabulary size in discrete communication scenarios, discrete VAE may communicate less useful information compared with our model. We also show the case when agents can pass continuous messages, giving us a ceiling performance when the communication channel is not restricted.

---

Under review as a conference paper at ICLR 2023
Model
Task
Accuracy
P
purity
continuous
Localization
0.96
continuous
Navigation
0.89
discrete VAE
Localization
0.81
-0.0113
0.556 / 0.667 /0.148
ours discrete VAE
Localization
0.84
0.4775
0.407 / 0.457 /02235
Navigation
0.69
ours
Navigation 0.86
Table 1: Comparison between adapt discrete VAE and our model. is the topographic similarity, the purity scores are calculated when labeled with object color/shape/direction
Sentence Length
1.0
route length effect
Iocaltaation Navigation
0.05
0.6
0.
length
length
length
length
0.00
0.0
route length
(a)
(b)
Figure 6: The effect of (a) sentence length and (b) the shortest route length between the start and target places on the task performance
To evaluate the generated language, we use topographic similarity and purity. We suppose that good language tends to give similar expressions describing a similar object. So use topographic sim- ilarity P as our metric, which is the Spearman correlation between the message similarity sequence and the room vector similarity sequence. And our method outperforms the discrete VAE largely. To make the emergent language more interpretable and give some more hints about the information encoded, we calculate the purity score with the following procedure: First, we only change the color, shape, and direction attributes of the object and fix other features. Then, we cluster those rooms with the generated message. Labeled these rooms with their objects' color, shape and direc- tion separately, get the corresponding purity. And the purity score tell us the information that the emergent language encoded. From the table, we can tell that compared with discrete VAF, our model focus more the direction and less color and shape which is reasonable because there are only 3 choices for color and shape which makes it less useful for disguising the target room from others, comparing with the direction with 8 possible value.
We also investigate factors that may affect task performance 6 Countering with our intuitions that, the sentence length grows, the model has a much more powerful expression ability, the perfor- mance should grow, but localization accuracy grows first and then drop. When the sentence length enough for expression, much more tokens may increase the difficulty for the model tackle. The shortest route length between the start positions and the target ones can also affect the task performance, but it does not follow an exponential drop when the shortest route length grows.
6 CONCLUSION
In this work, we propose a navigation task in an emergent communication scenario. This task is adapted from the languael navigation task proposed in human human-machine communication. However, compared with the traditional VLN task, the application of multi-agent communication in the VLN task makes the communication policy more flexible and generate more complex
7

---

Under review as a conference paper at ICLR 2023
dialogue format. Our method proves the feasibility of applying multi agent communication to the VLN task, and also gives a benchmark for this task.
REFERENCES
Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. Vision-and-languaee navigation: Interpreting visually-grounded navigation instructions in real environments, 2017. URL bttps::///axx
Kris Cao, Angeliki Lazaridou, Mare Lanctot, Joel Leibo, Karl Tuyls, and Stephen Clark. Emer- gent communication through negotiation, 2018. URL httpp:://arx...or////aaa// 03980
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Nießner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments, 2017. URL bttt:ss///axxivoo//aaaa.. 179. 06158
Howard Chen, Alane Suhr, Dipendra Misra, Noah Snavely, and Yoav Artzi. Touchdown: Natural language navigation and spatial reasoning in visual street environments. 2018. doi: 10.48550/
ARXIV.1811.12344 Utthttps:: arxiv. wwwp.x.orggab//..1..11 12354
Ta-Chung Chi, Mihail Eric, Seokhwan Kim, Minmin Shen, and Dilek Hakkani-tur. Just ask:an inter- active learning framework for vision and language navigation, 2019, URL :/arxiv./
orq/ arq/abs//e 1912.1111 00915
Harm de Vries, Kurt Shuster, Dhruv Batra, Devi Parikh, Jason Weston, and Douwe Kiela. Talk the walk: Navigating new york city through grounded dialogue, 2018. URL https:://axxvv
Btg/abs/lie www.aaac..... 03367
Katrina Evtimova, Andrew Drozdov, Douwe Kiela, and Kyunghyun Cho. Emergent communication in mudti-modal, multi-step referential game. arXiv preprint arXiv: 1705. 10369, 2017.
Jing Gu, Eliana Stefani, Qi Wu, Jesse Thomason, and Xin Wang. Vision-ane and---and-lamad language navi- gation: A survey of tasks, methods, and future directions. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). As- sociation for Computational Linguistics, 2022. doi: 10. 10.18663/1//022.cllllloggg.. /ettps:/dooi.rze wtt.:o...orr.......3//3333E///
URL
www.1166611122222222222 aono 524
Vihan Jain, Gabriel Magalhaes, Alexander Ku, Ashish Vaswani, Eugene Ie, and Jason Baldridge. Stay on the path: Instruction fidelity in vision-and-lanee languuge navigation. Proceedings of the 57th Anmual Meeting the Association for Computational Linguistics, pp. 1862- 1862-1872, Florence, Italy, July 2019. Association for Computational Linguisties. doi: 10. 10.196 1181. URL https: //aps::/acll antho ogy. ogg.orggpll.... 1
Ivana Kajić, Eser Aygün, and Doina Precup. Learning to cooperate: Emergent communication in multi-agent navigation. arXiv preprint XrXiv22004000097 2020.
Aleksandra Kalinowska, Elnaz Davoodi, Kory W Mathewson, Todd Murphey, and Patrick M Pi- larski. Towards situated communication in multi-step interactions: Time key pressure in communication emergence. In Proceedings of the Annual Meeting of the Cognitive Science Soci- ety, volume 44, 2022.
Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: lision-and-lannuaage navigation in continuous environments, 2020. URL https://arxiv: exc/abs/2004 2004 02857
Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls. and Stephen Clark. Emergence of linguistic communication from referential games with symbolic and pixel input, 2018. URL /htpss:/ arxiv. www.vvorg/abs//// 880 033984
8

---

Under review as a conference paper at ICLR 2023
Anjali Narayan-Cher Chen, Prashant Jayannavar, and Julia Hockenmaier. Collaborative dialogue in Minecraft. In Proceedings of the 57th Anmual Meeting of the Association for Computational Lin- guistics. pp. 5405-5415, Florence, Italy, July 2019. Association for Computational Linguisties. doi: 10.18653/v1/P11 1/P19 1537. URL hs.t:s/:: ac antho wtt.aaoooo.o..ooo11111113
Michael Noukhovitch, Travis LaCroix, Angeliki Lazaridou, and Aaron Courville. Emergent com- munication under competition, 2021. /tL/tttpss//axxioorrr /ttp///wwwv..///////......2..22.
Jason Tyler Rolfe. Discrete Variational Autoencoders. arXiv e-prints, art. arXiv:1609.0220, September 2016.
Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. Vision-and-dialog navi- gation, 2019. URL
wwwpo////a///.oo/o//////// 04957
Fengda Zhu, Xiwen Liang Yi Zhu, Xiaojun Chang. and Xiaodan Liang. Soon: Scenario oriented object navigation with graph-base exploration, 2021. URL https: // /arxivv..vv.. wwg.orrg///.... abs//
2103.17133


Reviews:
[{"review_id": "gTgk7jq8nm", "reviewer": "Reviewer_6nP4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a multi-agent navigation framework as a new benchmark to assess the emergent language that is thus evolved between the agents. ", "summary_of_the_review": "The paper appears to be a work in progress without much emphasis put on literature survey and comparisons to prior work. The paper also misses out on performing extensive empirical evaluation of the current state-of-the-art methods used for multi-agent navigation tasks.", "strength_and_weaknesses": "The paper is not clearly written and some sections are hard to understand. The related works do capture how the framework proposes new challenges that are not already captured in previous work. Besides a similar work that also uses communication between agents [2] perform a similar analysis and comparisons to that work would be help in highlighting the difference with the proposed framework.\n\nBesides the scope of the proposed framework is limited as it can only handle artificial messages that have a much smaller length, vocabulary, and sources of variation. In contrast, previous work have already proposed similar benchmarks that involve real images with natural language descriptions [1].\n\n[1] de Vries et al 2018. Talk the Walk: Navigating New York City through Grounded Dialogue.\n\n[2] Singh et al. 2019. Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "TsClYNRQlz3", "reviewer": "Reviewer_3NDJ", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper explores language emergence in the context of a navigation/coordination task.To do so, they introduce a simple gridworld environment sympliying the orginal \"Talk the Walk\" task. The authors then propose one RL model to solve the task.", "summary_of_the_review": "For all the reasons explained above, the paper is clearly below ICLR standard. I provided a few advise, that may help the authors improving ", "strength_and_weaknesses": "While this paper goes in a good direction by moving away from the Lewis game in the language emergence communication setting, it falls short as there is little scientific analysis. The authors mainly designed an artificial navigation task, handcrafted an observation space and reward, designed a neural network, trained it, and reported the results. Those results are not put into perspective with a detailed researched perspective. Indeed, the point of Emecom in ML is not to create artificial tasks for the sake of solving it, but to design tasks that could help understand how language evolves with a human intervention as possible (or being generic). \n\nI am listing here a few leads to improve the paper in the long run:\n - The introduction should frame the research problem into the past literature, ask a scientific question, and explain how the paper help answers it. Here, there is no citation; the VLM task is only said to be more natural (no reference to the embodiment, situated learning, or going beyond the signaling game)\n - The related work is extremely short. 18 citations in total for a paper about language emergence, and the navigation task is just inherently incomplete. On a positive note, the most relevant citations kajic/Kanilowska/DeVries are present\n- The paper is 7 pages long, and page 3 is barely empty\n- The model is designed to solve the task, e.g., the graph networks are specific to the observation space. I would recommend either picking a minimal architecture or reproducing the architecture that is classic in VLM. \n- having 4 losses in an RL task is generally a bad sign as it is a symptom of human intervention. cf The bitter lesson by Sutton\n - Results are not compared to any baselines (VAE vs your network is not really a baseline...), and simply reporting the success rate is not sufficient. Examples of suitable ablation include: the correlation between language and environment complexity. Qualitative experiments to translate language and change in the environment (e.g., how changing the graph impacts the language), what are the external constraints that shape language, reward ablation, etc.\n\nI would recommend the authors to re-work their paper and present it to a workshop to have further guidance. The initial idea is good, but the realization lacks scientific maturity.\n\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "J8Kl-6zA0yt", "reviewer": "Reviewer_SZp3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a multi-turn, bidirectional emergent communication setting implemented as a vision-language navigation task. Experiments show that the trained agent achieves high accuracy in localization and navigation tasks. The proposed model outperforms a VAE baseline. ", "summary_of_the_review": "While the idea is interesting, given its current poor writing and execution quality, I recommend rejecting the paper. ", "strength_and_weaknesses": "Strengths:\n* The proposed setting is novel.\n\nWeaknesses:\n* While novel, the proposed setting is poorly motivated. Why studying this setting is significant? What would it enable?\n* The description of the environment is very nebulous. Especially, the complexity of the visual and language components are highly under-specified. How is the input observation represented (image or feature-based)? How many object categories? How large is the agent's vocabulary? How are the environment graph generated? It is thus difficult to comprehend the significance of solving this environment. \n* The plots are missing confidence intervals. Each experiments are not conducted multiple times with different random seeds. This seriously weakens the empirical claims.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject