Learning Visualization Policies of Augmented Reality for Human-Robot Collaboration
Authors: Kishan Dhananjay Chandan
Jack Albertson
Shiqi Zhang
Venue: CoRL 2022 Poster (2022)
Keywords: Augmented Reality
Multi-robot systems
Imitation Learning

Abstract: In human-robot collaboration domains, augmented reality (AR) technologies have enabled people to visualize the state of robots. Current AR-based visualization policies are designed manually, which requires a lot of human efforts and domain knowledge. When too little information is visualized, human users find the AR interface not useful; when too much information is visualized, they find it difficult to process the visualized information. In this paper, we develop an intelligent AR agent that learns visualization policies (what to visualize, when, and how) from demonstrations. We created a Unity-based platform for simulating warehouse environments where human-robot teammates work on collaborative delivery tasks. We have collected a dataset that includes demonstrations of visualizing robots' current and planned behaviors. Our results from experiments with real human participants show that, compared with competitive baselines from the literature, our learned visualization strategies significantly increase the efficiency of human-robot teams in delivery tasks, while reducing the distraction level of human users.

Learning Visualization Policies of Augmented Reality for Human-Robot Collaboration
Kishan Chandan, Jack Albertson, Shiqi Zhang Department of Computer Science The State University of New York at Binghamton [kchanda2,
jaabaanda,,aabbrtteee jalbert5, zhangs @binghamton. edu
Abstract: In human-robot collaboration domains, augmented reality (AR) tech- nologies have enabled people to visualize the state of robots. Current AR-based visualization policies are designed manually, which requires lot of human ef- forts and domain knowledge. When too little information is visualized, human users find the AR interface not useful; when too much information is visualized, they find it difficult to process the visualized information. In this paper, we de- velop a framework, called VARIL, that enables AR agents learn visualization policies (what to visualize, when, and how) from demonstrations. We created a Unity-based platform for simulating warehouse environments where human-robot teammates collaborate on delivery tasks. We have collected dataset that includes demonstrations of visualizing robots' current and planned behaviors. Results from experiments with real human participants show that, compared with competitive baselines from the literature, our learned visualization strategies significantly in- crease the efficiency of human-robot teams, while reducing the distraction level of human users. VARIL has been demonstrated in a built-in-lab mock warehouse. VARIL is available at: https://kishanchandee www.mndda..iiitbiiiioooo gitthubio/varrl.hhm.
Keywords: Augmented Reality, Multi-robot systems, Imitation Learning
1 Introduction
Robots are increasingly being used in industrial environments, such as manufacturing and ware- houses, where the workspaces of people and robots are usually isolated from each other. Human- robot collaboration (HRC) opens up plethora of opportunities where people and robots could complement their capabilities. Hence, robots are entering human-inhabited environments to work with humans. Humans and robots prefer different communication modalities. While humans used to natural language and gestures in communication, robots exchange information in digital forms, such as text-based commands, resulting in a communication gap in HRC. Re- searchers have developed algorithms and system to bridge this communication gap using natural language [1, 2, 3, 4, 5] and vision [6, 7, 8]. Augmented Reality (AR) has been employed for HRC to provide an alternative communication mechanism with high bandwidth and low ambiguity [9].
AR is a technology that (1) combines real and virtual objects in a real environment; (2) runs interac- tively, and in real time; and (3) registers (aligns) real and virtual objects with each other [10]. Such composite views can enhance human perception the real world, and also result in an interactive experience of the environment. Many researchers working uncovering the benefits of AR in HRC [11, 9]. For instance, researchers have developed AR systems that allow humans to visualize the state of the robots [12], as well the robot intentions [13, 14]. Existing AR-based HRC systems employ a static visualization policy where the visualizations are always displayed to the user. The handcrafted static policies might suggest different visualization strategies, but those systems not able to take important runtime factors into account (e.g.. the number of robots, the current status of robots, the future intentions of robots, and the current status of human) to adapt the visualizations. The consequence is that those AR interfaces can alter people's 's attentional focus and result in inat- tentional blindness [15], if there visual clutter caused by many unwanted visualizations. Such
6th Conference on Robot I Learning (CoRI. 2022), Auckland, New Zealand.

---

(a) No AR visualization
(b) Full AR visualization
(c) Learned AR visualization (ours)
Figure 1: Different AR visualization strategies in virtual warchouse environment, where virtual human works with a team of mobile robots on collaborative delivery tasks. (a) No AR visualiza- tion. where the human worker does not know where some robots are, and what the robots plan to
do. (b) Full AR visualization, where the human worker can be overwhelmed by the visual indica- tors. (c) Our learned AR visualization, where the AR agent uses learned policy to dynamically determine a visualization strategy based on the current world state of both human and robots.
concerns motivate our research on enabling our AR agent to learn policy for dynamically selecting visualization actions given the state of the human-multi-nee multi-robot system [16].
We design a framework called, Visualizations for Augmented Reality using Imitation Learn- ing (VARIL), for multi-oult--oooot collaboration in shared environment. VARIL allows the human to track the status of a team of robots using an AR interface. In addition, VARII. learns the AR visualization policies using Imitation Learning (IL) [17] that dynamically selects AR visual- ization actions for human-multi-robe teams. To the best of our knowledge, this is the first work that employs learning to design visualization strategies of AR for numan--ulli-robot collaboration. In particular, we train decentralized visualization policies in a centralized fashion [18]. While training, all the states are extracted from the agents in a centralized manner, and new policy is generated by aggregating all the states, similar to learning in single-agent environment. The new policy learned on the aggregated dataset is then deployed the agents in decentralized manner, where the agents query the actions from the joint policy, by using only the current agent's 's state information.
We empirically evaluated our VARIL framework in warehouse domain [19, 20), where both the human and the team of robots are engaged in a collaborative delivery task. Figure shows our ware- house environment where twelve turtlebots collaborate with a human. Both the human and robots are assigned nontransferable tasks, where the robots are tasked with delivering objects to different locations (drop stations) and the human helps unload the objects from the robots waiting at drop stations. We collected a dataset of 6000 state-action pairs using the feedback provided from the "expert demonstrator". We have trained the AR agent in the simulated warehouse environment with this dataset using an iterative IL algorithm [21]. The learned AR visualization policies were com- pared with the static AR policy for human multi-robot teams from the literature [22], and the results suggest that the conflicts between the expert actions and the VARIL policies drastically decrease with the increase in the number of policy iterations. Additionally, we compared the total robot wait times for the intermediate policies of VARIL, and observed a significant decrease in the robot wait time between the initial and the learned AR visualization policies of VARIL.
While we observed significant improvements in the human-robot collaboration efficiency in pure simulation, we decided evaluate how VARIL influenced the user experience. We have evalu- ated VARIL with 25 participants where every participant operated virtual human in the warehouse environment. Every participant was asked fill out a survey questionnaire that was used for sub- jective evaluations. Based on the participants' responses, we observed that our learned visualization strategy significantly reduced the distraction caused by extraneous visualizations. addition to the qualitative evaluation, we also compared the task completion times during the human study. The results show significant improvement in the efficiency of muman-multi-rons teams in task comple- tion as compared to two competitive baselines from the AR-for-HRC literature [23, 22]. Finally, we present demonstration of a human worker collaborating with three mobile robots with AR- mediated communication in a built-in-lab mock warehouse environment.
2 Related Work
When humans and robots work in proximity, there is a need of communication to ensure the safety and efficiency of HRC. In this section, we first describe a few communication modalities used in current human-robot systems (non-AR and AR), and then discuss existing research that applies imitation learning methods to robotics domains.

---

Researchers have used 2D interfaces in the past where the communication usually takes place using graphical user interfaces [24, 25, 26]. Also, natural language was used by a number of researchers for human-robot communication [5, 27]. Motion-based based signaling of intent like gestures, includ- ing hand and facial have also been used for communication in human-robot teams [28, 29, 30]. Moreover, researchers studied how non-verbal social cues can improve task performance in human- robot teams [31]. Haptics, another communication modality, is largely based on physical interac- tion [32, 33, 34]. Even though all these modalities have their unique benefits and provide medium for human-robot communication, the real world is 3D by nature, and these modalities are not strong in conveying spatial information, which can be very useful for HRC in shared spaces. Hence, despite those successes, AR has its unique advantages of visualizing and communicating spatial information with potentially less ambiguity and higher bandwidth [9].
AR has the capability of overlaying spatial information onto the real world and helping a human counterpart visualize the robots' internal state and intended plans for effective HRC. Researchers have developed frameworks that enable human operators to visualize the motion-level intentions of robots using AR [23, 35, 36, 37J. In another line of research, Zhu et al. augment the algorithms that control the robots to help understand the relationship between the robot's state and the environ- ment [38]. Researchers have also developed a prototype system with shared AR workspace that enables a shared perception [39]. In our previous work, we designed system that enabled human users to provide feedback to robot plans visualized using AR [22]. More recently, researchers have designed different visual guidance in AR, where the humans can visualize the recommended actions from the robot (prescriptive guidance), and also visualize the state space information (descriptive guidance) to communicate why the rohot recommends a particular action to the human [40]. All of the existing research employs static, predefined visualization policies. In contrast, VARIL (ours) learns a policy to select AR visualization actions (what to visualize, when, and how).
The integration of Imitation Learning (IL) with robotics has enabled the robots to learn variety of tasks by observing expert demonstrations [41, 42]. One the early applications of IL was to teach helicopters to fly very challenging maneuvers through providing expert demonstrations [43]. In another line of work, IL was used to enable a vehicle to mimic human driving behaviors in outdoor terrains [44, 45]. Later on, researchers used IL to learn from expert demonstrations in simulated car driving domain [46]. A comprehensive study has been carried out to showcase the use of IL in robotics [42]. Very recently, researchers have used AR interfaces for constrained Learning from Demonstration (LfD) to allow the agents to maintain, update, and adapt learned skills [47]. They have used learning techniques to enhance the skills of the robotic systems, whereas train our AR agent to learn an adaptive visualization policy for human-multi-rooo collaboration. Similar to the discussed existing research, we use IL to help our agent learn from expert demonstrations. Instead of learning to accomplish tasks directly, our VARIL framework leverages IL to improve the efficiency of AR-based HRC, while improving user experience by avoiding visual distractions from extraneous AR visualizations.
3 Framework
In this section, we describe our framework called Visualizations for Augmented Reality using Imi- tation Learning (VARIL), and how it enables numan-multi-robot teams to collaborate together in shared environment. VARIL allows the human to track the status of a team of robots using an AR in- terface. Moreover, VARIL supports learning decentralized AR visualization policy in a centralized fashion using expert demonstrations, where the AR agent uses the learned policy to dynamically se- lect visualization action based on the state of the cimman-muli-roue the VARIL framework, and the interplay between the different components of VARIL.
teams.' Figure 2 showcases
Consider a team of robots working with human worker in shared space. The team of robots constantly share their state and plans using the AR interface, which is used by the human worker to track the status of the team of robots. The human worker simultaneously collaborates with the team of robots to complete the tasks. VARIL also consists of human expert that gives demonstrations of AR visualizations at runtime, indicating what information should be visualized (or not) at specific times. Once new policy is generated, the AR agent updates the visualizations to mimic the actions
'Our "centralized training, decentralized execution" idea was inspired by recent multiagent reinforcement learning research [18], while it is applied to an IL domain in this work.
3

---

Visualize the state and intentiona of robots
Current state and plans
Interface
Human Worker
Updated Visualization Policy
Suggested actiona
Datasot
Classifior
Robots
Demonstrator
Current state of numan-muttirobort system
Interact
Interact
Environment
Figure 2: Overview of our VARIL framework that enables the human-multi-root teams to collabo- rate in shared environment. VARIL allows the human to track the status of team of robots using an AR interface. Under the hood, VARIL supports learning an AR visualization policy using expert demonstrations, where the learned policy dynamically selects the actions for the AR agent.
suggested by the expert demonstrator. There are two different human entities in VARIL, one for the role of the human worker, and the other is human expert (Figure 2). Note that the expert is involved only during the policy learning phase, and once a satisfactory policy learned, the expert demonstrator is no longer needed.
3.1 VARIL Algorithm
We design a human-mulltn multi-oooot collaboration system using VARIL (Algorithm I) that enables the humans to work with team of robots while using an AR interface to track the status of the robot teammates. The imputs to the algorithm are as follows:
Pt: Task pool for the robots
pm, Motion planner
N: Number of robots
Ta; Multi-robot task allocator that assigns tasks at runtime
Now, we describe in detail how our VARIL al- gorithm works. In Line VARIL initializes llatet to enable all the visualizations. Here visualizations represent all the 3D objects that are rendered in the AR interface to visualize the status of the team of robots. The intuition be- hind enabling all the visualizations is to enable the expert to easily identify different types of available visualizations. The expert demon- strator can provide feedback on which visual- izations are helpful, and which ones need to be turned on/off based on the observed state of the human-multi-robog system.
Algorithm 1 VARIL
Require: pt, T p"",p, N,
glatest
Initialize glatest to show all visualizations
Initialize 5R +- 9, and to nall, where S is the state N robot,
Initialize lists: w +- 0, +-
robots, and H the state of the buman worker
(Ptt,, N). where E is task assigned and [T] N More details on T supplementary materials. while not empty do
Tasks still exist task pool
For 1. 1..... ..,,NN1] .N-1j do if is not complete then
Obtain 5th robot's configuration: w. + 9(6)
I, pp (wes (.)
The 7t robot fullows I, using controller
11:
else
Update using T
In Line 4, VARIL uses the multi-robot task al- locator (Ta) to allocate tasks (I) for N robots, where T, is the task of the th robot. Then, VARIL enters the main while-loop in Line 5 that runs until all the tasks from the task pool are completed. Inside the while-loop, VARIL enters forploop that runs for N times, once for each robot. At every iteration i of the for-loop, VARIL first checks if the robot has completed the current task or not (Line 7). If the task is not completed, VARII. first obtains the current robot configuration (w:). Then, VARIL uses pn to generate the intended trajectory of the ith robot, I,- Once, the current task complete, the robot obtains its new task using T
13:
end if
14:
for
15:
t- V(w, I, # #lotesty
More details on 1 in
supplementary materials.
16:
S obtain robats
17:
H. e- obtain current human worker state
glatest PoolicyppsS* R
Update visualization policy
19:
end while
After the for-loop, in Line 15, the set of robot configurations, robots' intended trajectories, and the latest visualization policy are passed the AR agent that renders the visualizations using the AR
"This is to give the human expert full observabiilty over the robots' current states and plans. If we had turned off the AR visualizations, there would be the issues, such as the human expert could not locate the robots obscured by other objects.

---

interface. In Lines 16 and 17, VARIL obtains the current state of all robots (8 and the current state of the human worker (H). Finally, VARIL calls the PolicyUp function that updates the global policy glatest, This enables VARIL's AR agent to update the visualizations based on glatest in the next iteration. In the next subsection, we explain how the PolicyUp function updates the visualization policy using IL based on the state of the human- numan-multi-robot system and the expert demonstrations.
3.2 AR Policy Learning: PolicyUp
The input of Algorithm 2 includes, SR, which is the state of robots; H. the state of the human worker; A, a set of agents for which the visual- ization is being learned; J, the number of iter- ations for updating the policy; a dataset, DD that stores the pairs of visited states and their corre- sponding actions suggested by the expert; and, glatest that stores the latest visualization policy learned in D.
Algorithm 2 PolicyUp(SRR, H)
Require: J.A, D. x for j 0 J 1 do
2:
for each agent o i do
R, +- == glarest 5. Latest visualization policy is used in the iteration.
4:
Let Ly = B, (1 +1-
Sample T-step irajectories using Tj
Obtain expert feedhack on the observed states
Update visualization for agent based on Ft Get dataset D5 state of human-mulli-ood system
PolicyUp enters the main for-loop in Line that runs for J times. The value of J can be customized based on the complexity of the state space of the AR agent. In each iteration, VARIL enters another for-loop in Line 2 that runs once for each agent, a, where a E A. Inside the inner for-loop, Pp first gets policy ny based on decision rule, which de- cides if the expert policy or the novice policy will be used for the current iteration. The decision rule uses the value of 33 that decays over the iterations, and at the beginning, the value of 3 = 1, and hence expert policy actions are preferred over the novice policy actions. As the value of j becomes closer to J, the value of / becomes closer to O. Intuitively this makes sense because at the beginning the novice policy is not mature enough and can lead to many mistakes, and hence the expert policy is used at the beginning. Then using the policy Tj. VARIL samples T-step trajectories in Line 5. During this phase, the visualizations are constantly updated for every agent a based on the current policy т, (Line 7). Finally, for every agent, VARIL stores the visited states as well as the suggested actions by the expert in Dj, where represents the jth iteration of the algorithm.
(S R. H)), and expert suggested feedback (actions)
end for
10:
Aggrogate datasets: D < PUDA datasets from all agents one dalaset
P. Combine individual
11:
Train classifier 1y11 on D
D. Common updated policy
generated for all agents
12:
Update glacent with ft++
P. Global policy updated for
visualization
13: end for
14:
return funceut
After completion of one iteration of the inner for-loop, the algorithm reaches Line 10 for data aggre- gation. Here, VARIL combines all the datasets stored in the current iteration (DzA) with the dataset from the previous iteration (D). Note that even though the datasets were obtained from different agents, the data aggregation step considers them as they were obtained from a single agent. Such abstraction of data allows the algorithm to generate a comprehensive dataset based on the experi- ences of all the agents, and also enables VARIL train the visualization policies in a centralized manner with fully decentralized execution. Once the dataset is aggregated, VARII trains a clas- sifier 41 on D in Line 11. The global policy flotest is updated with the newly generated policy, which is then used by all the agents in the next iteration of the algorithm.
The next section provides details on the experiments conducted to evaluate VARIL in the human- multi-robot collaboration scenario.
4 Experiments
We conducted experiments in a simulated warehouse environment that we developed using Unity. With the experiments, we aim to evaluate two hypotheses: I) VARIL improves the overall effi- ciency in suman--uuti-robee team task completion, determined by the slowest agent of a team, in comparison to other AR-based methods from the literature that employ static visualization poli- cies; and II) VARIL provides better user experience for humans human-multi-robo, collabora- tion tasks compared to competitive AR- methods from the literature. We have evaluated both Hypothesis-I and Hypothesis-II with human participants collaborating with multiple robots in the simulated warehouse environment.
5

---

Td10
T311
T1J12
ARACC
CINAR
VARIL (ours)
12
Number
700
1000
1100
Total rabat wait time
(a)
(b)
Figure 3: (a) Interface used by the expert to give demonstrations. (b) Comparison of VARIL (ours)
with two baselines to show the distribution of total waiting time (second) of all robots.
Participants: Twenty-five participants of ages 20-30, participated in the experiment, including six females and nineteen males. Every participant was given $10 gift card for participating in the ex- periments. Each participant conducted three trials using three methods, including VARIL, that were randomly ordered for each participant. Before starting the actual trials, each participant completed a short dummy trial that was scaled down to mini warehouse that includes only one robot. The role of the dummy trial was to acquaint the participants with the environment and teach the participant to control the virtual human. The experiments were approved by the Institutional Review Board (IRB) of our institution.
4.1 Experiment Setup
We deployed the simulated warehouse environment to a web server to facilitate the online experi- ment using the WebGL build of the Unity project. In experiments, the human participants were able to teleoperate the virtual human around the environment through the input of the W, A, S, and D keys along with using the mouse rotate the virtual human's 's field of view. Each robot is assigned three boxes for each trial and once the last box has been dropped off by the robot, it will begin navigating to its starting position. The trial is completed once all twelve robots have delivered their three boxes and have reached their starting position.
We collected dataset of 6000 demonstrations provided by human expert at runtime during which a human worker was working with a team of robots to collaborate on delivery task. In our case, the human expert is the first author of this paper, who knows this domain well, and we designed a virtual human to perform the task of the human worker. The human worker and the human expert both were provided with an AR interface to track the robots. The human worker used the AR interface to collaborate with the robots, whereas the role of the human expert to provide feedback the visualizations seen using the AR interface.
Data Collection and Policy Learning:
The human expert was provided with a simple interface as shown in Figure ((a) to input the feedback (which visualizations the human expert thinks should be to enabled or disabled) on the visualizations for the team of robots, as well as the drop station balloons. The interface provides a checkbox for each robot's trajectory, live location, and transparent avatar well as checkbox for each drop station balloon. The expert feedback is obtained in real time and the environment is not paused. The expert does not have access the internal state representation, and the feedback is completely based on the human expert's observations on the environment. Along with the state of the human worker and the team of robots, the feedback (actions) of the human was combined to collect the dataset. A snapshot of the state-action pairs was extracted every four seconds, and after every 25 state-action pairs, the dataset was aggregated with the dataset from the previous iteration. Using VARIL, we ran a total of two hundred forty iterations, creating the entire dataset of the state-action pairs. At every iteration, we train the AR agent with multi-class SVM [48]. More details on the state-action space of the AR agent are provided in the supplementary materials.
Baselines: For baselines, have replicated the visualizations of two different systems from the literature, called ARROCH [22], and CRMIAR [23] to make comparisons of their visualization
We create the acronym of CRMIAR that indicates "Communicating Robot Motion Intent with Augmented Reality" for the simplicity of referring the method.

---

strategies with the learned visualization from VARIL. ARROCH was designed for human- robot systems, where a human can use an AR device to track the status of the robots. Our imple- mentation of the ARROCH system has the following visualizations: the trajectories to show the motion intentions, robot avatar to show the live location, and transparent robot avatar to show the direction of the motion of the robot. On the other hand, CRMIAR was a single rohot visualization system, but we have scaled it to enable tracking the team of robots in our warehouse environment. In our implementation of the CRMIAR system, where we have combined the NavPoints and Util- ties designs of CRMIAR to provide the following visualizations: the trajectories of robots to show their planned motion intentions, a map to show the relative location of the robots with respect the human, and arrows to point to the robots that are not in the field of view. We use both ARROCH and CRMIAR in experiments to evaluate the efficacy of VARIL. In the supplementary materials, provide the figures of the both ARROCH and CRMIAR baseline implementations.
4.2 Real-world Demonstration
For demonstration, built-in-lab mock warehouse was constructed in a 2500 square foot room. Three turtle- bot robots were used to mimic the human-multi-rooot collaborative de- livery task similar to the Unity en- vironment. The speed of the Turtle- bots were set to 0.5 m/s. The human worker was provided with an AR de- vice (tablet) with screen size of 10 inches. Figure 4 shows the "warehouse" environment for the demonstration of VARIL, where a human worker is holding AR device track the status of three turtlebots. A demonstration video is available in the project webpage see the abstract.
Third-person view
AR view
Figure 4: Mock warehouse and AR visualizations that in- clude robot avatars and trajectories in color.
4.3 Results
Figure 3(b) shows the histogram, where the x-axis represents the total wait time of all the robots in seconds, and the y-axis represents the number of human-robot teams with the corresponding robot wait times. From the figure, can be observed that for most human-robot teams, VARIL had the shortest robot wait times as compared to ARROCH and CRMIAR. Also, we plotted the Gaussian curves to show the distribution of the data points for all the methods. We also analyzed the statistical significance, in every trial, we sum up the task completion time of all the robots. We found that VARIL performed significantly better than both ARROCH and CRMIAR, where 0.01- p-value < 0.05. Additionally, by comparing the total robot wait times, observed that the wait times for robots in VARIL were significantly shorter than the baselines. All these results support Hypothesis-I that states VARIL improves the human-robot team's 's task completion efficiency.
Objective Results:
Subjective Results: At the end of each trial, the participants were asked to fill out a survey form indicating their qualitative opinion over the six different questions to validate Hypothesis-II. The figure and the descriptive subjective analysis are provided in the supplementary materials. We ob- served significant improvements from the study supporting Hypothesis-I II on user experience, that the visualizations in VARIL provided better user experience. Collectively, the results convey that the VARIL visualizations help the participant better keep track of the robot status, proved useful towards task completion, and were less distracting, while the participants enjoyed using the system.
Policy Evaluation: VARIL is the first work on Imitation Learning-based visualization policy learn- ing for AR, and hence we do not have learning baselines to make comparisons for the generated policy. To overcome this shortcoming, we have evaluated our learned policy to find the discrep- ancy between the suggested expert actions as well as the actions suggested by both the ARROCH statie visualization policy, and the intermediate policies of VARIL. Figure 5(a) shows the number of policy iterations on the x-axis, and the number of discrepancies a measure of the deviation of the visualization policies from the expert's 's policy. From the figure, we can observe that the early policies of VARII. had more discrepancy, and this is because at the
the y-axis. The discrepancy
7

---

Number dorepere belween cxpit Semos 3nd
25
VARL Tours) ARROCH
Number /00ot toaetemms
700
175
150
0000
100
50
300
150
200
250
1000
1600
of policy tterettons
Total robot wai: dime 1st
(a) Discrepancies: expert vs. policy
(b) time (histogram)
Figure 5: (a) The number of discrepancies between the expert demonstrator's suggested actions with both the visualization policies of ARROCH (baseline) and VARIL with respect to the number of iterations. Each instance a human expert disabling a visualization considered a discrepancy; (b) A histogram to show the distribution of total waiting time (second) of all rohots in simulation with respect to six intermediate policies of VARIL.
beginning the policy's poor quality makes less effective in selecting visualization actions. With more interaction with the expert, the policy becomes mature, and tries to mimic the actions sug- gested by the expert. It can observed that the discrepancies drastically decrease with the increase in the number of policy iterations. On the other hand, the increase of policy iteration has no effect on the number of discrepancies between ARROCH and expert actions, because ARROCH employs static visualization policy. The final policy was used in the experiments to evaluate the learned visualization policy against the static policies from the baselines.
Furthermore, we evaluated Hypothesis-I in simulation to investigate how the intermediate policies of VARIL affect the human-robot team's task completion time. Figure 5(b) shows the histogram of the total robot wait times, for six different policies (PI (P).,)), where every policy was extracted after 50 policy iterations. total of 250 trials were run for every policy, and we observed improvements in the robot wait times denoting the improvement of efficiency in human-robot teams.
5 Conclusion
In this paper, we present our framework, VARIL (Visualizationss for Augmented Reality using Im- itation Learning), that for the first time introduces learning-based Augmented Reality (AR) vi- sualization strategy for human-multi multi-rooot collaboration. We have designed a system that learns decentralized visualization policy using imitation learning in centralized manner, where the AR agent dynamically selects a visualization action based on the state of the human-multi-robo sys- tem. We compared our framework with competitive baselines from the literature, and the results suggest that VARIL significantly increases the efficiency of human-robot collaboration, and reduces the distractions caused by extrancous information in AR.
Limitations: While we have demonstrated VARIL using real robots, building and evaluating such AR-for-HRC systems in the real world can be difficult. For example, the human worker needs to walk between drop stations, where each navigation action might take a long time. Adding this complexity, the robots need to collaborate in real time to complete the tasks where single robot getting stuck to dynamic obstacle can terminate the trial prematurely. designed open source simulator that can simulate a warehouse environment including team of robots. Even though such a simulator does not overcome all the challenges for human-robot collaboration, we expect that this will enable AR-for-HRC researchers to focus on the crucial aspects of algorithm design rather than the tedious software development work. In the future, we aim to evaluate the user-experience performance of VARIL through questionnaires and explore VR-based human-robot collaboration [49], while leveraging contextual scene information [50].
avoid such issues, we
Acknowledgement
This work has taken place in the Autonomous Intelligent Robotics (AIR) group at The State Uni- versity of New York at Binghamton. AIR research is supported in part by the National Science Foundation (NRI-1925044), Ford Motor Company, OPPO, and SUNY Research Foundation. The work of Albertson was supported in part by NSF REU (NR1-2032626).

---

References
[1] S. Tellex, T. Kollar, S. Dickerson, M. R. Walter, A. G. Banerjee, S. Teller, and N. Roy. Un- derstanding natural language commands for robotic navigation and mobile manipulation. In Twenty-Fifth AAAI Conference on Artificial Intelligence, 2011.
[2] C. Matuszek, E. Herbst, L. Zettlemoyer, and D. Fox. Learning to parse natural language commands to a robot control system. Experimental Robotics, pages 403-415, Springer, 2013.
[3] Y. Chai, L. She, R. Fang. S. Ottarson, C. Littley, C. Liu, and K. Hanson. Collaborative effort towards common ground in situated human-robot dialogue. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction, pages 33-40. ACM, 2014.
[4] S. Zhang and P. Stone. Corpp: commonsense reasoning and probabilistic planning, as ap- plied to dialog with mobile robot. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pages 1394-1400 2015.
[5] Amiri, S. Bajracharya, C. Goktolga, J. Thomason, and S. Zhang. Augmenting knowledge through statistical, goal-oriented human-robot dialog. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019.
[6] S. Waldherr, R. Romero, and S. Thrun. A gesture based interface for human-robot interaction. Autonomous Robots, 9(2):151-173 Sep 2000.
[7] K. Nickel and R. Stiefelhagen. Visual recognition of pointing gestures for human-robot inter- action. Image and vision computing, 25(12):1855-1884, 2007.
[8] H.-D. Yang, A.-Y. Park, and S.-W. Lee. Gesture spotting and recognition for human-robot interaction. IEEE Transactions on robotics, 23(2):256-275 2007.
[9] S. Green, M. Billinghurst, Chen, and G. Chase. Human-robot collaboration: lit- erature review and augmented reality approach in design. International journal of advanced robotic systems, 5(1):1, 2008.
[10] R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. Maclntyre. Recent advances in augmented reality. IEEE computer graphics and applications, (6) 47, 2001.
[11] Z. Makhataeva and H. A. Varol. Augmented reality for robotics: a review. Robotics, 9(2)211, 2020.
[12] Muhammad, Hassan, Cleaver, and Sinapov. Creating a shared reality with robots.
In Proceedings of the 14th ACM/IEEE International Conference on Sluman-Robar Interaction, 2019.
[13] R. T. Chadalavada, H. Andreasson, R. Krug, and A. J. Lilienthal. That's 's on my mind! robot to human intention communication through on-board projection on shared floor space. 20/5 European Conference on Mobile Robots (ECMR). IEEF, 2015.
[14] A. Watanabe, T. Ikeda, Y. Morales, K. Shinozawa, T. Miyashita, and N. Hagita. Communi- cating robotic navigational intentions. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015.
[15) B. Dixon, M. Daly, H. Chan, A. D. Vescan, J. Witterick, and J. C. Irish. Surgeons blinded by enhanced navigation: the effect of augmented reality attention. Surgical endoscopy, 27 (2)4544466 2013.
[16] K. Chandan, J. Albertson, and S. Zhang. Augmented reality visualizations using imitation learning for collaborative warehouse robots. In Proceedings of the 21st International Confer- on Autonomous Agents and Multiagent Systems, pages 1566-1566 2022.
[17] Schaal. Is imitation learning the route to humanoid robots? Trends cognitive sciences, (6):233-242, 1999.

---

[18] T. Rashid, M. Samvelyan, C. Schroeder, G. Farquhar, J. Foerster, and S. Whiteson, Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning. In Inter- national Conference on Machine Learning, pages 4295-4304. PMLR, 2018.
[19] f. R. da Costa Barros and 'T. P. Nascimento. Robotic mobile fulfillment systems: survey on recent developments and research opportunities. Robotics and Autonomous Systems, 137: 103729, 2021.
[20] R. Stern, N. R. Sturtevant, A. Felner, S. Koenig, H. Ma, T. T. Walker, J. Li, D. Atzmon, L. Cohen, T. S. Kumar, et al. Multi-agent pathfinding: Definitions, variants, and benchmarks. In Twelfth Anmual Symposium on Combinatorial Search, 2019.
[21] S. Ross, G. Gordon, and D. Bagnell. A reduction of imitation learning and structured predic- tion to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627-635. JMLR Workshop and Conference Pro- ceedings, 2011.
[22] K. Chandan, Kudalkar, X. Li, and S. Zhang. ARROCH: Augmented Reality for Robots Collaborating with Human. In 2021 International Conference on Robotics and Automation (ICRA), 2021.
[23] M. Walker, H. Hedayati, J. Lee, and D. Szafir. Communicating robot motion intent with aug- mented reality. In Proceedings of the International Conference on Human-RRbot Interaction, 2018.
[24] M. Baker, R. Casey, B. Keyes, and H. A. Yanco. Improved interfaces for human-robot interac- tion in urban search and rescue. In 2004 IEEE International Conference on Systems, Man and Cybernetics, volume 3, pages 2960-2965, 2004.
[25] M. W. Kadous, R. K.-M. Sheh, and C. Sammut. Effective user interface design for rescue robotics. In Proceedings of the 1st ACM SIGCHISIGART conference on Human-robot inter- action, pages 250-257. ACM, 2006.
[26] H. K. Keskinpala, J. A. Adams, and K. Kawamura. Pda-based human-robotic interface. In 2003 IEEE International Conference Systems, Man and Cybernetics, volume 4, pages 3931- 3936, 2003.
[27] J. Thomason, S. Zhang, R. J. Mooney, and P. Stone. Learning interpret natural language commands through human-robot dialog. Twenty-Four Fourth International Joint Conference on Artificial Intelligence, 2015.
[28] C. Breazeal. Emotion and sociable humanoid robots. International journal of human-computer studies, (111-2))11191555: 2003.
[29] K. Dautenhahn, M. Walters, S. Woods, K. L. Koay. C. L. Nehaniv, A. Sisbot, R. Alami, and T. Siméon. How may i serve you?: robot companion approaching a seated person in helping context. Proceedings of the 1st ACM SIGCHISIGGAR conferre on Human-robee robot interaction, pages 172-179. ACM, 2006.
[30] B. Robins, P. Dickerson, P. Stribling, and K. Dautenhahn. Robot-mediated joint attention in children with autism: A case study robot-human interaction. Interaction studies, 5(2):161- 198, 2004.
[31] C. Breazeal, C. D. Kidd, A. L. Thomaz, G. Hoffman, and M. Berlin. Effects of nonverbal communication on efficiency and robustness in human-robot teamwork. 2005 IEEE/RSJ international conference intelligent robots and systems, 2005.
[32] T. Miyashita, T. Tajika, H. Ishiguro, K. Kogure, and N. Hagita. Haptic communication between humans and robots. In Robotics Research, pages 525-536. Springer, 2007.
[33] R. J. Stone. Haptic feedback: brief history from telepresence to virtual reality. Interna- tional Workshop on Haptic Human- Coomute Interaction, pages 1-16. Springer, 2000.
10

---

[34] J. Dumora, F. Geffard, C. Bidard, T. Brouillet, and P. Fraisse. Experimental study on haptic communication of a human in a shared human-robot collaborative task. In 2012 JEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5137-5144. IEEE, 2012.
[35] H. Hedayati, M. Walker, and D. Szafir. Improving collocated robot teleoperation with aug- mented reality. In Proceedings of the 2018 ACM/IEEE International Conference on Human- Robot Interaction, pages 78 86, 2018.
[36] H. B. Amor, R. K. Ganesan, Y. Rathore, and H. Ross. Intention projection for human-robot collaboration with mixed reality cues. In International Workshop on Virtual, Augmented, and Mixed Reality for HRI (VAM-IIRI), 2018.
[37] E. Rosen, D. Whitney, E. Phillips, G. Chien, J. Tompkin, G. Konidaris, and S. Tellex. Commu- nicating and controlling robot arm motion intent through mixed-reality head-mounted displays. The International Journal of Robotics Research, (6:12-13):1513 1526, 2019.
[38] D. Zhu and M. Veloso. Virtually adapted reality and algorithm visualization for autonomous robots. In Robot World Cup, pages 452-464. Springer, 2016.
[39] S. Qiu, H. Liu, Z. Zhang. Y. Zhu, and S.-C. Zhu. Human-robot interaction in shared aug- mented reality workspace. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1 11413-11 1418. IEEE, 2020.
[40] A. Tabrez, M. B. Luebbers, and B. Hayes. Descriptive and prescriptive visual guidance to improve shared situational awareness in human-robot teaming. In Proceedings of the 2 International Conference on Autonomous Agents and Multiagent Systems, pages 1256-1264, 2022.
[41] A. Hussein, M. M. Gaber, E. Elyan, and C. Jayne. Imitation learning: A survey of learning methods. ACM Computing Surveys (CSUR), 50(2):1-35, 2017.
[42] H. Ravichandar, A. S. Polydoros, S. Chernova, and A. Billard. Recent advances in robot learning from demonstration. Annual Review of Control, Robotics, and Autonomous Systems, 3:297-330, 2020.
[43] Abbeel and A. Y. Ng. Apprenticeship learning via inverse reinforcement learning. In Pro- ceedings of the twenty-firt international conference on Machine learning, page 1, 2004.
[44] Bagnell, Chestnutt, D. Bradley, and N. Ratliff. Boosting structured prediction for imitation learning. Advances in Neural Information Processing Systems, 19, 2006.
[45] D. Silver, J. Bagnell, and A. Stentz. High performance outdoor navigation from overhead data using imitation learning. Robotics: Science and Systems IV, Zurich, Switzerland, 1, 2008
[46] J. Zhang and K. Cho. Query-efficien imitation learning for end-to-end autonomous driving. arXiv preprint 1rX.::1605.06500 2016.
[47] M. Luebbers, C. Brooks, C. Mueller, D. Szafir, and B. Hayes. Arc-Ifd: for interactive long-term robot skill maintenance via constrained learning from demonstration. In 2021 International Conference on Robotics and Automation (ICRA), 2021.
Using augmented reality
[48] A. Beygelzimer, V. Dani, T. Hayes, J. Langford, and B. Zadrozny. Error limiting reductions between classification tasks. In Proceedings of the 22nd international conference on Machine learning, pages 49 49.56, 2005.
[49] K. Chandan, J. Albertson, Zhang, X. Zhang Y. Liu, and S. Zhang. Learning to guide human attention mobile telepresence robots with 360 vision. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5297-5304. IEEE, 2021.
[50] S. Amiri, K. Chandan, and S. Zhang. Reasoning with scene graphs for robot planning under partial observability. IEEE Robotics and Automation Letters, 7(2)55600-5567, 2022.
11


Reviews:
[{"review_id": "xuXKpDvslfZ", "reviewer": "Reviewer_XmHJ", "summary": "====Summary: =====\nThis paper presents a method for learning to select visualizations (visualization agent) for on-line assistance in an augmented reality (AR) application for a human cooperating with a team of mobile robots in a warehouse-like scenario.  The visualization agent is trained through imitation learning (IL) using examples collected from a human expert providing labels for the desired visualization through a large number of episodes of the task with a human collaborating with the robot team. The method is tested through a user study that found the proposed method reduced task time when compared to other strategies for generating visualizations. \n", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "=====Strengths: =====\n\n*The application of a learning method for creating a visualization agent that assists a human on-line during a collaborative task is creative and interesting. It’s a useful problem to study for robotics and human-robot interaction.\n\n*The on-line demo of your unity system is very useful to understand the work, very nice! The paper is easy to read and clear, figures are clear and help understand the paper. In addition to the online demo, it would have been helpful to see actual episodes of users using the system for a task episode, including using the proposed method and the baselines. \n\n\n====Weaknesses: ====\n\n*I observed that the main paper seems to be limited in its explanation of the core learning method. It shows Algorithm 1 (VARIL), in which the policy update is done with PolicyUp. However, PolicyUp is only covered in supplementary materials. My main concern is that the paper is supposed to be self sufficient to explain the method, supplementary material should be left only for details, additional explanations, hyperparameter, etc, but not for materials core to understanding the contribution. It seems to me the learning part is imitation learning + Dagger, is this correct? Is there an innovation on the learning algorithm or how does it defer from other options algorithmically? \n\n*In lieu of actual augmented reality (i.e. real environment, visualization overlap in glasses or similar device), the paper uses an implementation of this scenario in (what seems to be) virtual reality (a virtual scenario in Unity), and the visualizations are created in there. While this experimental method has a fair analogy with augmented reality and doesn’t evidently affect the data collection (expert recommended visualizations), keep in mind that might not be the same experience as actual augmented reality. As recognized in the Conclusions section, an implementation of this paper in augmented reality in the real world would require very advanced computer vision for detecting and tracking relevant objects, as well as accurate localization. A very challenging project that sounds very interesting, I don’t doubt applications like this will be fully developed. \n\n\n\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "*As for the current version of the paper I don’t think it’s ready for conference publication because it lacks a full method explanation in the main paper. The paper needs to be self contained. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "====Comments: ====\n\n*How much does the visualization policy depend on the specific motion planner and task allocation systems? Is the input from the expert generalizable beyond that? \n\n*While using the on-line demo I noticed that some visualizations such as the future robot path is rendered on top of objects that hide the view to the robot. This is helpful to visualize the path, which would be otherwise occluded, but it’s also confusing as I didn;t fully understand where the trajectory was going towards. Did you do this in 3D virtual reality or on a computer screen? \n\n*How does the sample complexity (number of demos from expert) scale with the size of the robot team? Is the learned policy expected to generalize in any aspect? \n\n*As for the current version of the paper I don’t think it’s ready for conference publication because it lacks a full method explanation in the main paper. The paper needs to be self contained. "}}, {"review_id": "utPA9LJOcKK", "reviewer": "Reviewer_um9M", "summary": "This paper introduces a method for learning a policy to decide when to show which components of a visualization, in the context of Augmented Reality (AR) systems for human-robot collaboration. Existing literature provides baselines for sets of visualizations that are helpful during human-robot collaboration such as robot avatars, trajectories, etc. - which, according to the authors, can make for a busy AR interface in the presence of multiple robots and tasks. As a result, they propose VARIL which, using demonstrations from an expert, learns a policy to decide which components of the visualization are relevant at which points of the demonstration. They show that the learned visualization policies contribute to faster human-robot collaborations.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* The paper clearly introduces the motivation, literature, and establishes itself as both a new method and a new task (using learned policies for visualization).\n* The experiments conducted seem to clearly indicate that learned policies, and the proposed learning method in particular, have a statistically significant effect on human-robot collaboration efficiency.\n\nWeaknesses:\n\n* The conducted experiment seems like it could be rather simplistic to understand the effects of such a visualization mechanism. Additionally, no experiments with real robots are performed. These create some doubt as to how the shown results will transfer to real-life use cases.\n* It is not discussed why the authors believe imitation learning (especially in the form of DAgger) is the best paradigm for learning such a visualization policy. Indeed, in the presence of more switches, a human expert is likely to be strongly sub-optimal, and applying other paradigms such as RL on a simple MDP could provide above-human performance. The requirement of a human expert suggests that the provided visualization's value is upper-bounded by what a human expert engineer could heuristically program into the visualization software, beating the purpose of a learned policy.\n* There are some missing components in the main paper that are shown in the appendix but could be included in the main paper (see issues).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I think that this paper presents an interesting new paradigm (learned visualization policies) and shows a limited, albeit convincing, demonstration that the idea is beneficial for the proposed goals. Indeed, while it might be possible to improve the policy learning process (especially in contexts where there are too many parameters for a human expert to toggle, or a human expert is not available), I believe that opening up this area of research is possibly very beneficial towards improving the state of AR for HRI and thus recommend that this paper be accepted.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "After reading the experiment section I am still not sure exactly what the task was for the human worker (and have still not understood from playing for ~1min on the WebGL tool). It would be great if you could clarify this section.\n\nAlso, even if the details of how the DAgger-based algorithm need to be put in the appendix, I think it's important to include a rough idea of what the visualization policy state/action space looks like in the main paper. This is important because it's very hard to get a feel for the complexity of the visualization policy's task in the current state of the paper, which makes it harder to understand how important the human/policy help is."}}, {"review_id": "AQTgu7TFRtu", "reviewer": "Reviewer_TGsr", "summary": "This paper presents VARIL, an imitation learning-based Augmented Reality (AR) visualization strategy for human-multirobot collaboration. Main idea is to learn a visualization policy from an expert user, so that the visualized information in AR is adequate to perform the collaborative task with multi robots. The resulting system is a framework, where a visualization policy is learned and deployed from human demonstrations. With a user study that invovled 25 participants, the simulation results show that VARIL can increase the efficiency of human-robot teams in delivery tasks within warehouse scenarios, while reducing the mental effort required by the human users.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The paper presents the first system to learn visualization policy from human demonstrations.\n- Experimental results in simulation show that VARIL can reduce the mental workload of the user and be more efficient, when compared to existing approaches for visualization in AR. \n- There are plans to open source the warehouse simulation environment.\n\nWeakness:\n\n- The paper lacks description about challenges, system design motivations and validation, which is pre-requsites for a system paper like this one.\n\nMain system design and methodological contribution is described in Section 3. With the current content, it is difficult to evaluate/comprehend the questions like, (1) what is the main methodological challenge to build the presented system? e.g., there is some description of having centralized and decentralized policy learning, etc, but whats the challenge and practical problems there? (2) what are design choices made, and why? e.g, the choice of a specific imitation learning method, specific way of obtaining the data, the choice of specific scenario for ware house, etc. I find it hard to get the main take-away from Section 3, apart from the description of how the authors designed the system without 'why' part. \n\n- The paper's presentation needs more work.\n\nFirstly, descriptions about the experimental design is hard to parse, despite being a crucial part to comprehend the overall empirical findings. How about showing in video material: the overall task description, what are the states (as in appendix), what the expert demonstrates, and what AR shows to the human user, and most importantly, qualitative comparison between each methods in the task executions? This would help to comprehend the material, and also have intuitive feeling on why VARIL can outperform other methods.\n\nSecond, introduction does not motivate human-multirobot set up, which is related to the impact of this work. Descriptions about the AR and the importance of adequate visualization is fine, but the considered scenario should also be given proper attention. This is because, the presented framework is shown to beat other baselines within this scenario, and it better be an important task in terms of application. \n\nThird, methodological description in Section 3 requires more detailed descriptions. While presentation of the algorithm 1 helps in comprehensing the implementation details, more insights about the algorithm and descriptions of used methods incl. system design would be great to have in Section 3. Moreover, the paragraphs where the description jumped from line 1 to line 4 was difficult to understand. More materials like 'policy up' could be presented in the main paper, rather than in the appendix.\n\nLastly, there are also many missing notations in algorithm 1 which needs to be corrected in section 3. Many subsequent paragraphs and setnences start with 'We', which is a minor comment, but needs to be fixed.\n\n- Ablation studies are missing, which can motivate step-by-step design choices of the designed system.\n\nWhy not try out other learning methods? Other classifiers other than SVM -- can it further boost the performance of the system? What is the specific use-case with respect to efficiency? e.g., when will the visualization policy be effective, when not? \n\n- System paper only tested in simulation.\n\nThe authors do mention that having tested their system in simulation is one of their limitation, which will be hopefully addressed as future work. In my view, having a system paper only tested in simulation, is limited in its contribution and impact it can have to reach the bar for CoRL. When the system and ware house application in human-multirobot scenario is realized in physical mock-up, will the same scientific conclusion hold? Showing the impact of the system (incl. demonstration that the system can work in practice) is one of the missing steps in this paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I recommend weak reject. The paper itself presents an interesting system and experimental results in simulation demonstrates that it can outperform two other baselines.\n\nHowever, this system paper lacks either thorough and novel methodological insights to design such a system, OR a practical impact, which is typically shown through hardware experiments that demonstrate the system's usage in real robotic applications.\n\n----------------------------------------------- Update after rebuttal --------------------------------------------\n\nI modified my recommendation to weak accept. The paper has its strength on developing interesting system, which is of relevance to robot learning researchers. The authors have also addressed my main issue on the lack of real robot experiments. \n\nIf the paper gets accepted, thoroughly checking the justifications of design choices and clarify of presentation in method section would be highly appreciated.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "issues": "Address the comments described above.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)