Surrogate Assisted Generation of Human-Robot Interaction Scenarios
Authors: Varun Bhatt
Heramb Nemlekar
Matthew Christopher Fontaine
Bryon Tjanaka
Hejia Zhang
Ya-Chuan Hsu
Stefanos Nikolaidis
Venue: CoRL 2023 Oral (2023)
Keywords: Scenario Generation
Human-Robot Interaction
Quality Diversity

Abstract: As human-robot interaction (HRI) systems advance, so does the difficulty of evaluating and understanding the strengths and limitations of these systems in different environments and with different users. To this end, previous methods have algorithmically generated diverse scenarios that reveal system failures in a shared control teleoperation task. However, these methods require directly evaluating generated scenarios by simulating robot policies and human actions. The computational cost of these evaluations limits their applicability in more complex domains. Thus, we propose augmenting scenario generation systems with surrogate models that predict both human and robot behaviors. In the shared control teleoperation domain and a more complex shared workspace collaboration task, we show that surrogate assisted scenario generation efficiently synthesizes diverse datasets of challenging scenarios. We demonstrate that these failures are reproducible in real-world interactions.

Surrogate Assisted Generation of Human-Robot Interaction Scenarios
Varun Bhatt, Heramb Nemlekar, Matthew C. Fontaine, Bryon Tjanaka, Hejia Zhang, Ya-Chuan Hsu, and Stefanos Nikolaidis University of Southern California Los Angeles, CA
vsbhatt, nemlekar mffontain tjanaka, he jazhaa,, yachuanh, hiwomiiiiolisissssseee.ee edu
Abstract: As human-robot interaction (HRI) systems advance, does the diffi- culty of evaluating and understanding the strengths and limitations of these sys- tems different environments and with different users. To this end, previous methods have algorithmically generated diverse scenarios that reveal system fail- ures in a shared control teleoperation task. However, these methods require di- rectly evaluating generated scenarios by simulating robot policies and human ac- tions. The computational cost of these evaluations limits their applicability in more complex domains. Thus, we propose augmenting scenario generation sys- tems with surrogate models that predict both human and robot behaviors. In the shared control teleoperation domain and a more complex shared workspace col- laboration task, we show that surrogate assisted scenario generation efficiently synthesizes diverse datasets of challenging scenarios. We demonstrate that these failures reproducible real-world interactions.
Keywords: Scenario Generation, Human-Robot Interaction, Quality Diversity
1 Introduction
As the complexity of robotic systems that interact with people increases, it becomes impossible for designers and end-users to anticipate how a robot will act in different environments and with different users. For instance, consider robotic arm collaborating with a user on a package labeling task, where a user attaches a label while the robot presses a stamp with the goal of completing the task as fast as possible (Fig. 1). this task, the arm infers the user's intended goal object and moves simultaneously towards different object to avoid collision. The robot's motion depends on which object the user selects to label, how the user moves towards that object, and how all objects arranged in the environment. Thus, evaluating the system requires testing it with a diverse range of user behaviors and object arrangements.
While user studies are essential for understanding how users will interact with a robot, they are lim- ited in the number of euvironments and user behaviors they can cover. Algorithmically generating scenarios with simulated robot and human behaviors in "Oz of Wizard" paradigm [1] can comple- ment user studies by finding failures and elucidating holistic view of the strengths and limitations of robotic system's 's behavior.
Previous work [2, 3] has formulated algorithmic scenario generation as a quality diversity (QD) problem and demonstrated the effectiveness of QD algorithms in generating diverse collections of scenarios in shared control teleoperation domain. In that domain, user teleoperates a robotic arm with a joystick interface, while the robot observes the joystick inputs to infer the user's goal and assist the user in reaching their goal. However, these interactions only last few seconds in con- trast to collaborative, sequential tasks that last much longer. For instance, completing the package labeling task (Fig. 1) can take several minutes, making their evaluation expensive. This limits the applicability of QD algorithms, which require a large number of evaluations [4].
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

Figure 1: Example scenario in a collaborative package labeling task found by our proposed surrogate assisted scenario generation framework. The presence of the two objects behind the robot results in its expected cost-minimizing policy to move towards the object in the front, resulting in a conflict with the user who is reaching the object the same time.
Our key insight is that we can train deep neural networks as surrogate models to predict human- robot interaction outcomes and integrate them into the scenario generation process. In addition to making scenario evaluations inexpensive, deep neural networks are end-to-end differentiable, which allows us to integrate state-of-the-ee art differentiable quality diversity (DQD) algorithms [5] to efficiently discover scenarios that the surrogate model predicts are challenging with diverse behavior.
We make the following contributions: (1) We introduce using deep neural networks as surrogate models to predict human-robot interaction outcomes, such time task completion, maximum robot path length, or total waiting time; (2) We integrate surrogate models with differentiable quality diversity (DQD) algorithms that leverage gradient information backpropagated through the surrogate model. (3) We show, in the shared control teleoperation domain of previous work [2] and in shared workspace collaboration domain [6], that surrogate assisted scenario generation results in significant benefits terms of sample efficiency. It also achieves significant reduction in computational time the collaboration domain, where evaluations are particularly expensive.
2 Problem Statement
We model the problem of generating diverse and challenging dataset of human-robot interaction scenarios quality diversity (QD) problem and adopt the QD definition from prior work [5].
We assume a scenario parameterized by 0 E IR The scenario parameters could be object posi- tions and types in the environment, human model parameters, or latent inputs to a generative model of environments, which is converted f IR" assesses the quality of scenario 0. Because we wish find challenging scenarios, higher quality implies worse team performance, e.g., longer task completion time.
scenario via function G(0). The objective function
We further assume set of user-defined measure functions, m, R" - IR, or as a vector function m : 1= - IR*, that quantify aspects of the scenario that we wish to diversify, e.g., distance between objects, noise in user inputs, or human and robot path length. The range of m forms a measure space m (R"). which we assume is tessellated into M cells, forming an archive.
The QD objective to maximize the QD-score [7]: maxo, EM, [(0)). Here 0, refers to the scenario with the highest quality in cell of the archive. If there no scenarios in a cell, f0,) is assumed be zero.
The differentiable quality diversity (DQD) problem formulation is special case of QD where the objective function ff and measure functions m are first-order differentiable.
3 Background
Scenario Generation. Algorithmic scenario generation has many applications, which include de-
signing video game levels [8. 9, 10, 11, 12, 13, 14] and testing autonomous vehicles [15, 16, 17, 18, 19, 20, 21]. motion planning algorithms [22], and reinforeement learning agents [23, 24, 25, 26]. It has also been applied to create curricula for robot learning [27, 28, 29, 30] and to co-evolve agents

---

Gradient e (p,,
++VV
0 eV
Generatt Someriis and Evalcate Surrogate Model
Brunch Somariia via Surrogate Madel Gradients
(0, f. yhh
Predict System Performance, Messures, and Orupanc Sommañio
Adapt A(j.E)
Ascend to Maximiar QD Objective
CMA-ES
Uplate Surnoguie Arthive
Update Datasel
Train the Surnagale Model Soanario
MIl Repait Sommatio
(0, J. mr, Vr: yh.
Lnnse Loop
Grourd-tuth Performmnc,, Memures, and Occupancy for each Scenario
Loog
Simulate Seemarios from
Summagate Archive
Update Groond-Troth Archive
sell-seuervieed Robot Human Occupuncy Prediction
Predict
Human
Ocsupancy
I (0. f.
Predict
Ousupancy
Inpet
Figure 2: An overview our proposed differentiable surrogate assisted scenario generation (DSAS) algorithm for HRI tasks. The algorithm runs in two stages: an inner loop to exploit surrogate model of the human and the robot behavior (red arrows) and outer loop to evaluate candidate scenarios and add them to dataset (blue arrows). See App. A for the complete pseudocode.
and environments for agent generalizability [31, 32, 33, 34, 35, 36, 37, 38, 39, 40]. Most relevant to our work is prior work [2. 3] in human-robot interaction that applied the MAP-Elites [41] and CMA-ME [42] quality diversity (QD) algorithms to find robot failures in a shared control telcopera- tion domain. In this work, we significantly improve sample and wall-clock efficiency by combining the state-of-the-ar QD algorithms CMA-MAE [43] and CMA-MAEGA [43] with surrogate models that predict human-robot interaction outcomes.
QD Algorithms, QD algorithms, such as MAP-Elites [41] and CMA-MAE [42, 43], solve the QD problem defined in Sec. 2. They have been used to generate diverse locomotion strategies [41, 44], video game levels [45], nano-materials [46], and building layouts [47]. Certain prior QD algo- rithms [4, 48, 49, 23] have leveraged surrogate models based on Gaussian processes or deep neural networks guide the search. Most relevant to our work is the Deep Surrogate Assisted Generation of Environments (DSAGE) [23] algorithm, which exploits surrogate model with quality diversity algorithms to generate environments. However, DSAGE has only been applied in single-agent grid- world game domains. In contrast, this work addresses the much more complex task of human-robot interaction scenarios, which requires the advances in the surrogate model, QD search, and scenario generation described in Sec. 4.
4 Surrogate Assisted Scenario Generation
Our method for algorithmically generating diverse collections of HRI scenarios builds upon recent work in generating single-agent grid-world environments with surrogate models [23]. We briefly de- scribe the advances necessary to scale these techniques from single-agent grid-world game domains to the much more complex HRI domains.
Surrogate Models for Ruoonn Interaction. We scale the surrogate model to predict out- comes of HRI scenarios that include both robot and human behavior and environment parameters.

---

First, we allow both the environment and the human model parameters inputs to the surrogate model. Second, we discretize the shared workspace and predict two occupancy grids, one for the human and one for the robot. We then stack both predictions as inputs to a convolutional neural network, which predicts the objective and measure functions. App. B provides precise details of our surrogate model.
Scenario Repair via Mixed Integer Programming. In contrast to previous work [2] that con- sidered single workspace, scenarios in our work have disjoint workspaces, with each workspace imposing constraints on object arrangement, such as boundary and collision avoidance constraints. Simulating the scenario impossible if these constraints are not satisfied. We thus adopt a generate- then-repair approach [50, 51], where we generate unconstrained scenarios and pass them through a mixed integer program (MIP). We propose MIP formulation that solves for the minimum cost edit of object locations the sum of object displacement distances that satisfies the constraints for valid scenario. See App. C for the complete MIP formulation.
Objective Regularization. The generate- thepettrattee--eaairr strategy repairs invalid scenarios by moving objects placed outside the workspace boundaries to the edge of the workspaces. However, this does not incentivize the QD algorithm to search the workspace interiors, which can result in the search diverging away from the workspace areas. To guide the search towards the workspace interiors, discount the objective function f of the QD formulation by the cost of the MIP repair. An ablation study in App. H.3 shows the effects of objective regularization on performance.
DQD with Surrogate Models. The DSAGE algorithm exploits the surrogate model with derivative- free QD algorithms. key observation is that the surrogate model is an end-to-end differentiable neural network. We can take advantage of this by exploiting the surrogate model with differentiable quality diversity (DQD) algorithms [5], which leverage the gradients of the objective and measure functions accelerate QD optimization. Leveraging DQD also lets scale to higher dimensional scenario parameter spaces since the search is applied over the objective-measure space (k + 1 di- mensions) instead of the scenario parameter space (n dimensions).
Algorithm. Fig. 2 provides an overview of the complete algorithm, which consists of an outer loop (blue arrows) and an inner loop (red arrows). In the inner loop, a QD algorithm searches for scenarios that challenging and diverse according to the surrogate model predictions. We repair the generated scenarios to ensure validity and evaluate each repaired scenario to obtain ground truth objective and measure values. Based on these values, we add each scenario to ground-truth archive and a training dataset for the surrogate model. The surrogate model trains on this dataset, correcting prediction errors exploited by the QD algorithm. After accumulating enough diverse data, the surrogate model starts making accurate predictions, and the inner loop produces truly diverse and challenging scenarios. After multiple outer loop iterations, the ground-truth archive accumulates diverse and challenging scenarios, testing the HRI system's 's strengths and limitations.
The key idea behind the proposed algorithm is that exploiting the surrogate model with QD algo- rithms produces diverse - with respect to the surrogate model predictions - scenarios. Labeling these scenarios by evaluating them in a simulator and using them to retrain the surrogate model in turn improves predictions in subsequent iterations. Thus, the surrogate model self- self-improves over time by training on the diverse data generated according to its predictions.
The proposed improvements result in two versions of our algorithm: (1) Surrogate Assisted Scenario Generation (SAS), which employs a derivative-free QD algorithm, CMA-MAF, in the inner loop. (2) Differentiable Surrogate Assisted Scenario Generation (DSAS, Fig. 2), which employs a DQD algorithm, CMA-MAEGA, in the inner loop. Additional algorithm details and the pseudocode are given App. A. Our source code available at https: /ttp//ggtttttttt com/ http/icaroossscccaeee
5 Domains
We consider two HRI domains from prior work: shared control teleoperation [52] and shared workspace collaboration [6] with 6-DoF Gen2 Kinova JACO arm.

---

Shared Control Teleoperation. In shared control teleoperation, a user provides www-imensional joystick inputs towards a goal. To aid the user, the rohot attempts to infer the human goal and move towards it autonomously. An optimal policy would correctly infer the human goal and reach the goal along the shortest path. The robot solves a POMDP with the user's goal as a latent variable and updates its belief based on the human imput, assuming noisily-optimal user [52]. With hind- sight optimization and first-order approximation, this results in the robot's actions being weighted average of the optimal path towards each goal, where the weights are proportional to the respective goal probabilities. In App. H.1, we test a second robot policy that blends the user's and the robot's s actions based on the robot's confidence in the user's goal [53]. Scenario parameters include both the environment, ie., the coordinates of two goal objects, and the human actions, i.e., a set of human trajectory waypoints. To search for failures, we set the objective to be the time taken to reach the correct goal. We diversify scenarios with respect to the noise in human inputs (variation from the optimal path) and the scene clutter (distance between goals).
Shared Workspace Collaboration. In shared workspace collaboration, the human and the robot simultaneously execute a sequential task in a shared workspace with disjoint regions [6, 54]. An optimal robot policy would correctly infer the human's current goal and move to a different goal along the shortest path while avoiding collisions with the human. As the teleoperation task, the robot models the human goal as latent variable POMDP (but with human hand as its observation) and uses hindsight optimization and first-order value function approximations to act in real time. The robot attempts to avoid the goal intended by the human by selecting the nearest goal from a feasible set of goals different than the user's, i.e., it maps human candidate goal to a different goal-to-go. The robot's action is a weighted average of the optimal path towards each goal-to-go, with weights proportional to the probability of the corresponding human goal.
We parameterize the scenario with three goal coordinates and set the objective be the task com- pletion time. We select measures based on factors that we expect affect the team performance: object arrangement, the accuracy of inference of user's goal, and the distance required to reach the goals. We choose two sets of measures: (1) The minimum distance between goal objects and the maximum probability assigned by the robot to the wrong goal. (2) The robot path length and the total time for which the human and robot have to wait when reaching the same goal.
In this domain, model the human as solving softmax MDP with the maximum entropy formu- lation [55]. In App. H.2, include additional setting in the collaboration domain, where search over both environments and human model parameters related to speed and rationality.
Additional details about the domains, the robot policy, the human policy, and the implementation are provided App. D, App. F.I, App. F.2, and App. G respectively. In App. E, we provide example QD formulations for two additional real-world domains, but we do not run experiments on them in this paper.
Experiments
Independent Variables. Our two independent variables are the domain and the algorithm.
Our three domains are: (a) shared control teleoperation with distance between the goals and hu- man variation as measures; (b) shared workspace collaboration with minimum distance between the goals and maximum wrong goal probability measures (collaboration I); (c) shared workspace collaboration with robot path length and total wait time as measures (collaboration II).
In each domain, we compare five different algorithms: (a) Random Search, where we uniformly sample scenarios from the valid regions. (b) MAP-Elites [41], as adapted for scenario genera- tion in previous work [2], with the additional objective regularization described in Sec. 4. (c) CMA-MAE [43] with objective regularization. (d) SAS: The proposed derivative-free version of our surrogate assisted scenario generation algorithm. We apply CMA-MAE as the derivative-free QD algorithm in the inner loop. (c) DSAS: The proposed differentiable surrogate assisted scenario generation with the DQD algorithm CMA-MAEGA in the inner loop.
5

---

DSAS
SAS
CMA-MAE
MAP-Elites
Random Search
24k-
(D>axee
120k
20k
QDwaamz
17k-
44k
1kk
Exclottions soore
Evalusioss QDiscore
QD-soore
1000
206
10k-
500
10k
1.8
0
12.9
01
6.0
120
Time (Bluns)
Time (Flun))
Time (lion))
(a) Shared control teleoperation
(b) Collaboration I
(c) Collaboration II
Figure 3: QD-score attained in the three domains as function of the number of evaluations (top) and the wall-clock time (bottom). Algorithms with surrogate models have better sample efficiency but require more wall-clock time per evaluation compared to other algorithms due to the overhead of model evaluations and model training. Plots show the mean and standard error of the mean.
Dependent Variable: Following previous work [2], we set QD-score [7] as the dependent vari- able that summarizes the quality and diversity of solutions. We compute the QD-score at the end of 10,000 evaluations, averaged over 10 trials of random search, MAP-Elites, CMA-MAE, and because of GPU usage constraints (see App. G.3) trials of SAS and DSAS.
Hypotheses:
H1. We kypothesize that the surrogate assisted QD algorithms SAS and DSAS will outperform CMA-MAE, MAP-Elites, and random search. We base this hypothesis on previous work, which has shown the benefit of integrating QD with surrogate models in single-agent domains [49, 23].
H2. We hypothesize that DSAS will outperform SAS. We base this hypothesis on previous work, which has shown that DQD algorithms perform significantly better than their derivative-free coun- terparts [5] when the objective and measure gradients are available.
Analysis. A two-way ANOVA test showed significant interaction effect (F(8.0, 105.0) 305.79, p 0.001). Simple main effects analysis each domain showed significant effeect of the algorithm on the QD-score (p < 0.001). Pairwise t-tests with Bonferroni corrections showed that SAS and DSAS performed sigmificantly better than CMA-MAE, MAP- Alt-El,tes, and random search in the shared control teleoperation and collaboration domains (p < 0.001). In the collaboration II do- main, they outperformed CMA-MAE (p < 0.001) and random search (p < 0.001), while there was no significant difference with MAPeElites. We attribute this to the fact that MAP-Elites can easily obtain diverse robot's path lengths by making small isotropic perturbations in the object positions function of the number of evaluations. We that
(see App. H). Fig. 3 shows the QD-score both SAS and DSAS achieve a high QD-score early in the search, indicating high sample efficiency.
The comparison between SAS and DSAS showed mixed results, with SAS better in the collaboration I domain (p < 0.001), DSAS better in the shared control teleoperation domain (p= < 0.001), and no significance in the collaboration II domain (p- = 0.07). Previous work [5, 43] has shown DQD algorithms improving efficiency in very high-dimensional search spaces by reducing the search from a high- -igh--imensional solution space to low- dimeesiionnl objective- space. We conjecture that this explains the significant improvement in the shared control teleoperation domain (9-dimensional solution space as opposed to 6-dimensional one in shared workspace collaboration) and we will investigate higher-dimensional domains in future work.
Furthermore, in the shared workspace collaboration domains, where scenario evaluations last a cou- ple of minutes because of the larger task complexity, surrogate assistance showed wall-clock time efficiency (Fig. 3), unlike prior work [23] that only showed sample efficiency improvements.

---

DSAS
SAS
CMA-MAE
MAP-Elite
Random Search
10.0
Elaman Variation
5.0
2.5
8,000
0.00
0.32
0.00
0.32 0.00
0.32 0.00
0.32
0.00
0.52
Distance Between Goals
Figure 4: Comparison of the final archive heatmaps the shared control teleoperation domain.
Fig. 4 shows heatmaps of the final archives the shared control teleoperation domain. The heatmaps for MAP-Eliter and random search match the results from prior work [2]. These heatmaps show the advantage QD-score as a comparison metric. A higher QD-score implies the archive is filled more and with higher quality solutions. For example, SAS and DSAS find failure scenarios in the lower right corner of the archive (nearly optimal human and large distance between the goal objects), whereas MAP-Elites fails to find failures in that region. Thus, a designer would not have information about the HRI algorithm's 's performance in some scenarios if the scenario generation algorithm has low score. App. H includes heatmaps in other domains and tabulates the QD-scores.
We provide additional experiments with two different settings in App. H.1 and App. H.2. Addition- ally, show the effect of objective regularization in App. H.3 and examples of generated scenarios with high (as opposed to low) team performance in App.
Real World Demo. We wish to demonstrate that the generated failure scenarios are reproducible in the real world and are not just simulation artifacts. Since the algorithms that we test have been shown to work robustly in practice [6, 52, 53, 54], and the proposed approach discovers edge-case failures that hard to find and rare in practice, user study where system would need to involve very large number of subjects observe these failures. Furthermore, we would need to account for any safety concerns that arise from the unexpected robot behavior in the corner cases. We view solving these challenges as beyond the scope this paper, and here, we only show that these failures will actually occur in the real world if users act in certain way.
freely interact with the
We recreate four example scenarios from the generated archives with 6-DoF Gen2 Kinova JACO arm and by having a user reproduce the motions of the simulated human. We track the human hand position with Kinect vI sensor and the OpenNI package [56]. We discuss two scenarios below and the other two in App. I. We include videos of all scenarios in the supplementary material.
Incorrect robot motion because of delayed human goal inference (Fig. 5a): We select this scenario from the archive generated by DSAS in the collaboration domain. In this scenario, after the human finishes working goal G1 and the robot G3, the robot is closer to G2 than its other remaining goal, GI. Based the feasible goal set formulation (Sec. 5), G2 becomes the goal-to-ge for human candidate goals GI and G3. Given that the combined probability of the human going to either G1 or G3 is higher than the probability of the human going to G2 the robot moves towards G2. However, once the robot realizes that the human actually moving to G2 as well, the robot has to move all the way back to goal G1, resulting in significant delay.
Long robot motion with correct human goal inference (Fig. Sc): We additionally wish find scenar- ios that result in poor team performance that is not due to incorrect inference. We select scenario from a SAS archive in the collaboration domain that had low maximum wrong goal probability of 0.3. The poor performance here is caused by the interaction between the robot's policy and the object placement. As the robot moves between the two workspaces following a straight line path, reaches a configuration close to self-collision or to joint limits. This prompts the system to re-plan and move the robot different configuration before continuing to move towards the goal. While planning ensures task completion, it induces significant delay compared to scenarios where the goals can be reached without re- planning.
7

---

G
G3
G1
G3
(a) Scenario
(b) Scenario 2
(c) Scenario 3
(d) Scenario 4
Figure 5: Example scenarios recreated with real robot. The purple line shows the simulated human path. Videos of the simulated and recreated scenarios are included in the supplemental material.
Discussion
Limitations. Our approach scales surrogate assisted scenario generation from single-agen grid- world domains to complex human-robot interaction domains with continuous actions, environment dynamics, and object locations. However, our evaluation domains consist of objects of the same type and simple human models. We note that SAS and DSAS general algorithms and about integrating them with more complex models of environments [51] and human actions [57, 58), as well as leveraging high-fidelity human model simulators [59, 60] to improve realism. Addition- ally, in our domains we were able to specify good test coverage with low-dimensional measure spaces. However, domains where we wish to obtain good test coverage over a large number of at- tributes would require high-dimensional measure spaces. Integrating centroidal Voronoi tessellation (CVT) based archives [61] or measure space dimensionality reduction methods [62] will allow to apply SAS and DSAS such domains. Furthermore, our system does not explain the reason behind the observed robot behavior in the generated scenarios, and future work will explore integrating sce- nario generation with methods for failure explanation [63, 64). Finally, while we focus on a single human interacting with single robot, we believe that our workspace occupancy-based approach for surrogate model predictions can be extended to malli-huuma--robee team settings.
excited
Implications. We presented the SAS and DSAS scenario generation algorithms that accelerate QD scenario generation via surrogate models. Results in shared control teleoperation domain of previous work [2] and in shared workspace collaboration domain show significant improvements in search efficiency when generating diverse datasets of challenging scenarios.
For the first time in surrogate assisted scenario generation methods, we see improvements not only in sample efficiency but also in wall-clock time in the shared workspace collaboration domain, where evaluations last couple of minutes. On the other hand, the additional computation in the inner loop of the surrogate assisted algorithms resulted in more time required match and exceed the performance of the baselines in the shared control teleoperation domain, where scenario evaluations last only few seconds. Thus, for ounning-time performance, we recommend surrogate assisted methods in domains with expensive evaluations, in which the additional computation the inner loop is offset by the improvement in sample efficiency.
We additionally highlight unexpected benefit of our system during development. When tested the shared workspace collaboration domain, SAS and DSAS discovered failure scenarios that ex- ploited bugs in our implementation which were subsequently fixed. For instance, some goal loca- tions were reachable by the robot arm in the real world but unreachable in simulation because of small errors in the robot's s URDF file, which prompted us to correct it.
Overall, we envision the proposed algorithms as a valuable tool accelerate the development and testing of HRI systems before user studies and deployment. We consider this important step towards circumventing costly failures and reducing the risk of human injuries, which is critical milestone for widespread acceptance and use of HRI systems.

---

Acknowledgments
This work was supported by the NSF CAREER (#2145077), NSF GRFP 18DGE-1842487, and the Agilent Early Career Professor Award. One of the GPUs used in the experiments was awarded by the NVIDIA Academic Hardware Grant.
References
[1] A. Steinfeld, O. C. Jenkins, and B. Scassellati. The oz wizard: Simulating the human for interaction research. Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction, HRI, 2009. doi:10. 10..0..145/// 1101111///51149555411111 URL https:///// org/ 1145/1514095. 1514115.
[2] M. C. Fontaine and S. Nikolaidis. A quality diversity approach to automatically generating human-robot interaction scenarios in shared autonomy. In Robotics: Science and Systems,
2021. 10i: N00::00.16000 100.15567//SS00000 2021.XVI..3...
[3] M. C. Fontaine and S. Nikolaidis. Evaluating human-robot interaction algorithms in shared autonomy via quality diversity scenario generation. ACM Transactions on Slmann-Robo In- teraction (THRI), 11(3):1-30, 2022.
[4] A. Gaier, Asteroth, and J.-B. Mouret. Data-efficient design exploration through surrogate- assisted illumination. Evolutionary Computation, 2018.
[5] M. C. Fontaine and S. Nikolaidis. Differentiable quality diversity. In Advances in Neural Information Processing Systems, 2021. URL httpp:////poo www.iie..r///epneeeeeeee aa 010.9990011100777770000000000000
pdf.
[6] S. Pellegrinelli, H. Admoni, S. Javdani, and S. S. Srinivasa. Human-robot shared workspace collaboration via hindsight optimization. In Proceedings of the International Conference on Intelligent Robots and Systems, IROS, 2016. doi: 01010...10/RR032206675799971 URL https:
//doi. wtwg///////////////////// /0/.//11.00 1109/IR0S 77591477
[7] K. Pugh, L. Soros, P. Szerlip, and K. O. Stanley, Confronting the challenge of quality diversity. In Proceedings of the Annual Conference Genetic and Evolutionary Computation, 2015.
[8] D. Gravina, A. Khalifa, A. Liapis, J. Togelius, and G. N. Yannakakis. Procedural content generation through quality diversity. In Proceedings of the IEEE Conference on Games (CoG), 2019.
[9] M. C. Fontaine, R. Liu, A. Khalifa, J. Modi, J. Togelius, A. K. Hoover, and S. Niko- laidis. Illuminating mario scenes in the latent space of a generative adversarial network. In Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021. URL https://ojs. aaai oroo/or///o peg/index.ro php/ /tt.///aaaieeeeeiee/////aaaaa///
[10] S. Earle, J. Snider, M. C. Fontaine, S. Nikolaidis, and J. Togelius. Illuminating diverse neural cellular automata for level generation. CoRR, abs/2109.05489, 2021. URL https://axxiv.. org/abs/2100 httaaab///299900549933
[11] A. Khalifa, S. Lee, A. Nealen, and J. Togelius. Talakat: Bullet hell generation through con- strained map-elites. In Proceedings of the Genetic and Evolutionary Computation Conference, 2018.
[12] K. Steckel and Schrum. Illmiiating the space of beatable lode runner levels produced by various generative adversarial networks. In Proceedings of the Genetic and Evolutionary Computation Conference, 2021. 262110.1147/344977633554444

---

[13] J. Schrum, V. Volz, and S. Risi. CPPN2GAN: combining compositional pattern producing networks and gans for large-scale pattern generation. In Proceedings of the Genetic and Evo- lutionary Computation Conference, 2020. 10i:10.11 04011144//77799933888222
[14] A. Sarkar and S. Cooper. Generating and blending game levels via quality-diversity in the latent space of variational autoencoder. In Proceedings of the 16th International Conference on the Foundations of Digital Games, 2021.
[15] Amold and Alexander. Testing autonomous robot control software using procedural content generation. Proceedings of the 32nd International Conference on Computer Safety, Reliability, and Security, 2013. (0011007977----422-4000
[16] G. E. Mullins, P. G. Stankiewicz, R. C. Hawthorne, and S. K. Gupta. Adaptive generation of challenging scenarios for testing and evaluation of autonomous vehicles. Journal of Systems and Software, 2018. doi: 10..0.11066//s.2111...0000
[17] Y. Abeysirigoonawardena, F. Shkurti, and G. Dudek. Generating adversarial driving scenarios in high-fidelity simulators. In International Conference on Robotics and Automation, ICRA, 2019. (00110.1000ICRAA2111.977777 8793740.
URL https: org/10. 111011109//0 ICRO 2019.
[18] E. Rocklage, H. Kraft, A. Karatas, J. Seewig, Automated scenario generation for regression testing of autonomous vehicles. In Proceedings of the 20th IEEE International Conference on Intelligent Transportation Systems (ITSC), 2017. doi: 1101
[19] A. Gambi, M. Mueller, and G. Fraser. Automatically testing self-driving cars with search- based procedural content generation. In Proceedings of the 28th ACM SIGSOFT International Symposium Software Testing and Analysis, 2019. doi: (10.:111444/29992233333566
[20] D. Sadigh, S. Sastry, and S. A. Seshia. Verifying robustness of human-aware autonomous cars. PunersanpsO 2019.
[21] J. Fremont, T. Dreossi, S. Ghosh, X. Yue, A. L.. Sangiovanni- Vinneniiiii and S. A. Seshia. Scenie: language for scenario specification and scene generation. In Proceedings of the 40th
ACM SIGPLAN Conference on Programming Language Design and Implementation, 2019.
[22] Y. Zhou, S. Booth, N. Figueroa, and J. Shah. RoCUS: robot controller understanding via sampling. In Proceedings of the Conference Robot Learning, 2021. URL https: //proceedings. mmr.. https//v1///zoooio html.
[23] V. Bhatt, B. Tjanaka, M. C. Fontaine, and S. Nikolaidis. Deep surrogate assisted generation of environments. Conference on Neural Information Processing (NeurIPS), 2022.
[24] M. Deitke, E. VanderBilt, A. Herrasti, L. Weihs, J. Salvador, K. Ehsani, W. Han, E. Kolve, A. Farhadi, A. Kembhavi, et al. ProcTHOR: Large-scale embodied Al using procedural gen- cration. arXiv preprint arXiv:2206.0699
2022.
[25] K. Cobbe, C. Hesse, J. Hilton, and J. Schulman. Leveraging procedural generation bench- mark reinforcement learning. In Proceedings of the International Conference on Machine Learning, 2020.
[26] Risi and Togelius. Increasing generality in machine learning through procedural content generation. Nature Machine Intelligence, 2020.
[27] B. Mehta, M. Diaz, F. Golemo, C. J. Pal, and L. Paull. Active domain randomization. In Conference on Robot Learning, pages 1 1162-1176. PMLR, 2020.
[28] J. Lee, J. Hwangbo. L. Wellhausen, V. Koltun, and M. Hutter. Learning quadrupedal locomo- tion over challenging terrain. Science robotics, 2020.
10

---

[29] K. Fang, Y. Zhu, S. Savarese, and L. Fei-Fei. Discovering generalizable skills via automated generation of diverse tasks. Robotics: Science and Systems, 2021.
[30] K. Fang, T. Migimatsu, A. Mandlekar, L.. Fei-Fei, and J. Bohg. Active task randomization: Learning visuomotor skills for sequential manipulation by proposing feasible and novel tasks. arXiv preprini 1.X.x:2221.061:55 2022.
[31] R. Wang, J. Lehman, J. Clune, and K. O. Stanley. Paired open open-ended trailblazer (POET): end- lessly generating increasingly complex and diverse learning environments and their solutions. CoRR, Abss/190100175 2019. URL http: //tp://aavvvv//eeeeeeeeeeeeaeeva http.///g//bb/........// http//1111111111
[32] R. Wang, Lehman, Rawal, J. Zhi, Y. Li, Clune, and Stanley. Enhanced POET: open-ended reinforcement learning through unbounded invention of learning challenges and their solutions. In Proceedings of the 37th International Conference Machine Learning, ICML, 2020. URL BttLhtt:/pprcceediggg www..mmee.........e.......m...ee htt///1119///1// html.
mlr.
[33] T. Gabor, A. Sedlmeier, M. Kiermeier, T. Phan, M. Henrich, M. Pichlmair, B. Kempter, C. Klein, H. Sauer, R. AG, al. Scenario co-evolution for reinforcement learning on a grid world smart factory domain. Proceedings of the Genetic and Evolutionary Computa- tion Conference, 2019.
[34] D. M. Bossens and D. Tarapore. QED: using qualiy-enviromeen-diveiiitt to evolve resilient robot swarms. IEEE Transactions on Evolutionary Computation, 2020.
[35] A. Dharna, J. Togelius, and L. B. Soros. Co-generation game levels and game-playing agents. Proceedings of AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2020.
[36] A. Dharna, A. K. Hoover, J. Togelius, and L. Soros. Transfer dynamics in emergent evolution- ary curricula. IEEE Transactions on Games, 2022.
[37] M. Dennis, N. Jaques, E. Vinitsky, A. M. Bayen, S. Russell, A. Critch, and S. Levine. Emer- gent complexity and zero-shot transfer via unsupervised environment design. Advances in Neural Information Processing Systems 33, 2020. URL https::/prooeedingg. meweeddiiinnnnninpps..eeee neurips. Abstract.....t.t......... html.
[38] M. Jiang, E. Grefenstette, and T. Rocktäschel. Prioritized level replay. In Proceedings of the 38th International Conference on Machine Learning, ICML, 2021. URL http: //proceedings. wttpaanwwwm....p./eeeeee.m presss/139//janggll att.ainnnn.............
[39] M. Jiang, M. Dennis, J. Parker-Holder, N. Foerster, E. Grefenstette, and T. Rocktäschel. Replay-guided adversarial environment design. In Advances in Neural Information Pro- cessing Systems 34, 2021. URL https://proceeddngs .nurips....... htc/ppper/222//haat/ 0e9 Mee1158bb22666662665444446666662 -basaaaa htmml.
[40] Parker-Holder, M. Jiang, M. Dennis, M. Samvelyan, J. N. Foerster, E. Grefenstette, and T. Rocktäschel. Evolving curricula with regret-based environment design. CoRR, abs/2203.01302, 2022. doi:10.48500/xXX.2200...333
[41] J. Mouret and J. Clune. Illuminating search spaces by mapping clites. CoRR, abs/ 0tte/1504h 1504.04909, 2015. URL httt:://arxxiii http:///axxwvv///aaaa/ htrgg/abs/1504... 04909.
[42] M. C. Fontaine, J. Togelius, S. Nikolaidis, and A. K. Hoover. Covariance matrix adaptation for the rapid illumination behavior space. Proceedings of the Genetic and Evolutionary Computation Conference, 2020. doi:10. 142.110.115/3779993339222..
[43] M. C. Fontaine and S. Nikolaidis. Covariance matrix adaptation map-annealing arXiv preprint arXiv:225..10752 2022.

---

[44] B. Tjanaka, M. C. Fontaine, J. Togelius, and S. Nikolaidis. Approximating gradients for dif- ferentiable quality diversity in reinforcement learning. CoRR, abs/2202.036666, 2022. URL.
attps://arxiv.org abs/2202. 03666.
[45] T. Pierrot, V. Macé, F. Chalumeau, A. Flajolet, G. Cideron, K. Beguir, A. Cully, O. Sigaud, and N. Perrin-Gilbert. Diversity policy gradient for sample efficient quality-diversity optimization. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1075-1083, 2022.
[46] Y. Jiang, D. Salley, A. Sharma, G. Keenan, M. Mullin, and L. Cronin. An artificial intelligence enabled chemical synthesis robot for exploration and optimization of nanomaterials. Science Advances, 2022.
[47] A. Gaier, J. Stoddart, L. Villaggi, and P. J. Bentley. T-DominO: Exploring multiple criteria with quavity-diversity and the tournament dominance objective. In Proceedings of the 17th International Conference on Parallel Prohlem Solving from Nature, 2022.
[48] A. Hagg, D. Wilde, A. Asteroth, and T. Bäck. Designing air flow with surrogate- assisted phe- notypic niching. In Proceedings of the International Conference on Parallel Problem Solving from Nature, 2020.
[49] Y. Zhang, M. Fontaine, A. K. Hoover, and S. Nikolaidis. Deep surrogate assisted MAP- Elites for automated hearthstone deckbuilding. CoRR, abs/2112.03534, 2021. URL https: // aaaxi.v... org/ abs//aa2......///// 03534.
[50] H. Zhang, M. Fontaine, A. Hoover, J. Togelius, B. Dilkina, and S. Nikolaidis. Video game level repair via mixed integer linear programming. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2020.
[51] M. C. Fontaine, Y. Hsu, Y. Zhang, B. Tjanaka, and S. Nikolaidis. On the importance of environments in human-robot coordination. In Roboties: Science and Systems, 2021. doi: 10. 10.13607/RSS.202.1 200.1......22...10333.3
[52] Javdani, S.
Srinivasa, and A. Bagnell. Shared autonomy via hindsight optimization. In Robotics: Science and Systems XI, 2015, doi: doi:10.15607/88 110:000.5677/3S.205.....222 URL http: //www www.:/oow...ppoceeeengg.oorrraaa
html.
[53] A. D. Dragan and S. Srinivasa. Formalizing assistive teleoperation. In Robotics: Science and Systems VIII, 2012. 46..1..15607/RSS.2212......
[54] S. Javdani, H. Admoni, S. Pellegrinelli, S. S. Srinivasa, and J. A. Bagnell. Shared autonomy via hindsight optimization for teleoperation and teaming. The International Journal of Robotics Research, 3777):717-742, 2018.
[55] B. D. Ziebart, N. D. Ratliff, G. Gallagher, C. Mertz, K. M. Peterson, J. A. Bagnell, M. Hebert, A. K. Dey, and S. S. Srinivasa. Planning-based prediction for pedestrians. In Proceedings of the International Conference on Intelligent Robots and Systems, 2009, doi: 1099/.OS. 2009.530 1009.5354147. URL htttps://ao..ot
110111199/RRR /0009//RRS... 55544447
[56] M. Liebhardt. OpenNI tracker. http://wiki. ros. wwwpooooeoootrreeeeeeereoeoeeeee tracker, 2012.
[57] H. J. Jeon, D. Losey, and Sadigh. Shared autonomy with learned latent actions. Robotics: Science and Systems (RSS), 2020.
[58] M. Chen, S. Nikolaidis, H. Soh, D. Hsu, and S. Srinivasa. Trust-aware decision making for human-robot collaboration: Model learning and planning. ACM Transactions on Human- Robott Interaction (THRI), 922):1-23,
12

---

[59] R. Ye, W. Xu, Fu, R. K. Jenamani, V. Nguyen, C. Lu, K. Dimitropoulou, and T. Bhattachar- jee. Reare world: A human-centric simulation world for caregiving robots. In Proceedings of the JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022.
[60] Z. Erickson, V. Gangaram, A. Kapusta, C. K. Liu, and C. C. Kemp. Assistive gym: A physics simulation framework for assistive robotics. In Proceedings of the IEEE International Confer- ence on Robolics and Automation (ICRA), 2020.
[61] V. Vassiliades, K. Chatzilygeroudis, and J. Mouret. Using centroidal voronoí tessellations scale up the multidimensional archive of phenotypic elites algorithm. IEEE Transactions Evolutionary Computation, 2018. doi: 10. 1109/TVC.2001.22755550
[62] A. Cully. Autonomous skill discovery with quality-diversity and unsupervised descriptors. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO, 2019. doi: 10.1 N0.1145/3321707.332211 804. URL pottps://don http:///oo.or///..../
org/10. 1145/3321707 33218804.
[63] D. Das, S. Banerjee, and Chernova. Explainable for robot failures: Generating ex- planations that improve user assistance in fault recovery. In Proceedings of the ACM/IEEE International Conference on Human-Robon Interaction, 2021. doi:10.1144//344473334444
URL https:://///// //ooi.::////..// org/ 10. 11111/144/44331..1 3444657.
[64] B. Hayes and J. A. Shah. Improving robot controller transparency through autonomous pol- icy explanation. In Proceedings of the ACM/IEEE international conference on human-robot interaction, 2017.
[65] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, ICLR, 2015. URL http:// // arxiv. oerivi.rg/aa //t/a/////11..9..0
[66] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G.
Chanan, T. Killeen, et al. PyTorch: an imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, 2019. wttptttsss/////redddggggg
hbcc2888err12226688777777778--82
[67] IBM.
ILOG CPLEX optimization studio. https://wwit ibm. com/products/ 2020.
ilog
[68] T. Kruse, A. K. Pandey, R. Alami, and A. Kirsch. Human-aware robot navigation: A survey. Robotics and Autonomous Systems, 2013. doi: 660:100..006//1r0b022033.....000 URL https:
//doi. 10. 1016/j wwwot. 201..00..07..
[69] Y. Gao and C. Huang. Evaluation of socially-aware robot navigation. Frontiers in Roboties and AI, 2021. doi: do0:10.3389/hhhh22222222233 URL https://ddi.orr//1... argg//0.389///ee..
[70] D. Gallenberger. T. Bhattacharjec, Y. Kim, and S. S. Srinivasa. Transfer depends on ac- quisition: Analyzing manipulation strategies for robotic feeding. In Proceedings of the 14th ACM/IEEE International Conference Human-Robo. Interaction, HRI, 2019. doi: 111100//HR22219.867333 URL https:://oi..... org.o.ooogg/... 100,9/199HHR.. NMR..2011.8673300
[71] R. Diankov and Kuffner. OpenRAVE: A planning architecture for autonomous robotics. Roboties Institute, Pittshurgh, PA, Tech. Rep. CMU-RI-7R-08-34 2008.
[72] S. Javdani. Shared autonomy via hindsight optimization code. https://github.com com/ arrsmmalroboticcccdaaassasseeeee ppolicy, 2016.
[73] B. Tjanaka, M. C. Fontaine, D. H. Lee, T. T. M. Vu, Y. Zhang, S. Sommerer, N. Dennler, and S. Nikolaidis. pyribs: A bare-bones python library for quality diversity optimization. 2021.
https://git thtbgbttthb...o
13

---

A Algorithm: Differentiable Surrogate Assisted Scenario Generation
The improvements proposed in Sec. result in two versions of our algorithm. In Algorithm 1, we present DSAS with the state-of-the the-art art DQD algorithm CMA-MAEGA in the inner loop. SAS follows similar structure but with CMA-MAE in the inner loop. Fig. 6 shows version of Fig. with the inner loop and the outer loop split into two figures.
On each iteration of the outer loop, we initialize a new surrogate archive to store solutions that the surrogate model predicts are high performing and diverse (line 3). Then, we begin the inner loop (line 5). On line 6, we evaluate the current solution point & with the surrogate model to obtain the predicted objective f. measures vn, and the branching gradients v and Vm. We then add the solution 0 to the surrogate archive (line 8) based on the predicted evaluations, after applying the regularization penalty (line 7). Next, we generate a batch of solutions based on the branching gradients (line 9). For each solution, we sample gradient coefficients, which, combined with the gradients, produce a new candidate solution (lines 10-12). We evaluate each new candidate solution 0f with the surrogate model (line 13), apply the regularization penalty (line 14), and add the solution the surrogate archive (line 15). After processing batch, we update the search parameters of CMA-MAEGA move the search towards maximizing the QD objective (line 17).
After completing an inner loop, we select subset of solutions from the surrogate archive to label (line 19). For each set of scenario parameters O, we generate-and-repair a scenario (line 21), evaluate the robotic system on the scenario (line 22), update our dataset by adding the scenario labeled with the true objective f, measures m, robot occupancy grid yr. and human occupancy grid yn. (line 23), and finally add the scenario to our ground- -trut archive (line 24). After updating the training data with newly labeled scenarios, we train the occupancy predictor for both the robot (line 27) and human (line 28), then train the surrogate model to predict the objectives and measures (line 29). The inner loop in future iterations exploits the more accurate surrogate model to produce better scenarios.
B Surrogate Model Details
Our surrogate model follows a two-stage prediction process by first predicting the robot and the human occupancy grids given the scenario parameters as input, followed by a downstream prediction of the objective and measures.
The occupancy predictor (blue arrows in Fig. 7) consists of deconvolution layers followed by batch normalization and ReLU that treat the scenario parameters as 1 x 1 image with the number of channels equal to the solution size and expand it into a 32 x 32 image. In the shared control teleop- cration domain, there is only one occupancy grid since only the robot arm is moving. In the shared workspace collaboration domain, there are two occupancy grids (stacked into two channels) corre- sponding the robot and the human motion. We pass each channel in the final output through softmax operator and minimize the KL divergence loss between the predicted and the true occupancy grids.
The downstream predictor (red arrows in Fig. 7) consists of a fully connected network with linear layers followed by batch normalization and ReL.U to extract features from the scenario parameters. also consists of convolutional layers followed by batch normalization and leaky ReLU extract features from the occupancy grids. We pass the features through linear layer and minimize the mean squared error (MSE) between the predicted and the true objective and measures.
The losses for the occupancy predictor and the downstream predictor have different scales and hence, are hard to balance. Thus, we separately train both networks on data obtained from ground-truth evaluations for 100 epochs in each outer iteration using Adam [65] optimizer with a learning rate of 0.0001 and batch size of 64. We first train the occupancy predictor, freeze the weights, and then train the downstream predictor by leveraging occupancy predictions from the occupancy predictor. We implement and train the networks with the PyTorch library [66].
14

---

Algorithm 1: Differentiable Surrogate Assisted Scenario Generation (DSAS).
Input:
N: Maximum number of evaluations, Nerploit: Number of iterations the model exploitation phase, Qy: Initial solution for CMA-MAEGA, B: Batch size for CMA-MAEGA
Output: Final version of the ground-truth archive Agt
Initialize the ground-truth archive Agt, the dataset D, robot occupancy predictor smy, human occupaney predictor sm,, objective and measure predictor
2 while evals < N do Initialize CMA-MAEGA with the surrogate archive Asurr and initialize solution @ to 0g Initialize CMA-ES parameters μs. Σ
for ilr E (1, Neaploit do
V f: m, vm t sm(0,, sm, (0)) f r-eg()
Asurr &- add solution Asurr: (0, m))
for i ......... B) do
10
C ~ (u, )
11
V. t cu V
+
Ek 1ーz (c V
0 0 V,
f', *, m'm * + sm(6', (0,) (f))
13
f' + reg(0') (')
14
Aeurr f add sod (0, fr, m'))
15
16
end
17
Update 0, E via CMA-MAEGA update rules
18
end
@ + selecrssoutiions(Asurr)
20
for 0 E θ do
scenario + G(0)
22
f.m. Yr Ub +- evaluate(scenario) D+ DU (0, m. yr. yh.)
23
24
Agt + add soluuioutionnn.g.o (0, m)) evals evals +
25
end
27
sm.itrain( D
train)aan(()
29
sm. sm.train(D, sm,, (Tits
30 end
H42 Supar Socatio
Sample Graderi
CarBicnn 4-1-0
R-177, 49,
3r. Conanderuts Systes Periecaan Mausurs Oocapanay with Somara
frandh donubberi
Sarnggg Makl
(0. Bir)
Sentgga Anttin
the Sumoupaa
Updaa Dusiet
2D Dijuctio
(a) Inner loop
(b) Outer loop
Figure 6: An overview our proposed differentiable surrogate assisted scenario generation (DSAS) algorithm for HRI tasks. (a) The inner loop, where QD algorithm exploits surrogate model to obtain scenarios that are predicted to be challenging and diverse. (b) The outer loop, where the scenarios labeled after repairing and evaluating them in simulator. This data is leveraged to train and improve the surrogate model for subsequent iterations.
15

---

DU Bell:
Deome Bitli
(20 Relli
Decwel 2
Card
Cowet Lase Balli
Cowe Lase Bell
Gand13 Lusyy
Lade
Grds
Oppetes
Bese Nee
Each
20ch Nora
Figure 7: Architecture the surrogate model including the occupancy predictor (blue arrows) and the downstream predictor (red arrows).
Surogate Archive
Ground-Truth Archive
1.00
Max Whong Goal Beltef Proh
Max Goal Ballef Proh
75
70
70
0.35
65
0.35
0.05
0.32
0.05
0.32
Min Distance Between Goals
Min Distance Between Goals
Figure 8: Comparison between the surrogate archive (left) after an inner loop and the corresponding ground-truth archive (right) after evaluating the solutions in the surrogate archive.
B.1 Evaluating the Surrogate Model Predictions
We evaluate the predictions of the surrogate model similar to DSAGE [23] by taking the dataset generated in one trial an algorithm and treating as the test set for the trained surrogate model from another trial of the same algorithm. Table shows the mean absolute error (MAE) in three domains for the surrogate models trained as part both DSAS and SAS. Note that Measure and Measure 2 columns in the table correspond the respective measures in each domain described in Sec. 5.
The surrogate model able to accurately predict the measures in the shared control teleoperation domain since they can be calculated directly from the solution and do not depend on the robot policy. In contrast, measures that depend on the robot policy such as the maximum wrong goal probability (Measure in the collaboration domain) have comparatively higher error, with predictions being off by around 9% average.
Table 1: Mean absolute error of the objective and measure predictions by the surrogate models.
DSAS
SAS
Objective MAE
Measure MAE
Measure MAE
Objective MAE
Measure MAE
Measure
Domain
2 MAE
Shared Control
0.35
0.01
0.01
0.64
0.02
0.01
Teleoperation
Collaboration I
3.41
0.02
0.09
3.47
0.02
0.08
Collaboration II
3.22
0.27
0.56
3.39
0.29
0.59
16

---

Furthermore, we observe that the percentage of predictions landing in their true archive cell is only around 2-4% in all domains. Nonetheless, the predictions are close to their true archive cell as evi- dent in the MAEs. We also confirm this by computing the average Manhattan distance between the predicted archive cell and the true archive cell for each solution. the shared control teleoperation domain, the average Manhattan distance was 6.48 and 11.26 for DSAS and SAS respectively. The average Manhattan distances for DSAS and SAS were 11.26 and 9.88 in the collaboration domain, and 6.89 and 7.20 in the collaboration II domain, indicating that the predicted archive cells are only a few cells away from the true archive cells average.
Thus, despite inaccuracies in placing the solutions into their true archive cells, the solutions in the surrogate archive are diverse with respect to the true measure functions. Hence, when these solutions are evaluated, they occupy different parts of the ground-truth archive and rapidly improve the QD-score. Fig. 8 shows the surrogate archive after inner loop and the corresponding ground- truth archive obtained by evaluating the solutions in the surrogate archive in the collaboration I domain.
C Mixed Integer Program for Repairing Scenarios
To ensure that the objects in the scenario generated by QD search satisfy the object arrangement constraints the shared workspace collaboration domain, we adopt generate-then-repe
strategy. We formulate mixed integer program (MIP) with constraints ensure that the objects in the scenario are inside the workspace boundaries and not in collision with each other. Since we wish the repaired scenario be close possible to the generated scenario, set the MIP objective to be the L2 distance between the original position and the repaired position of the objects. The quadratic objective makes the MIP a mixed integer quadratic program (MIQP).
C.1 Variables and MIP Objective
We treat the x and y coordinates each object as the MIP variables. Let x and y/ be the coordinates of object in the generated scenario and let ri, and yi be the corresponding coordinates after MIP repair. We set the objective to be:
min E, - -1,0=-x/²²00 + (ys - v/)²
(1)
C.2 Constraints
(max) be the minimum and maximum allowed and y values Let (min) (max) (min) and yr Bi y/
respectively for objects in rectangular workspace region r. For each workspace region, need to construct binary variable =ir which resolves to true if object occupies workspace r. We
create four auxiliary decision variables (up) (dn) (1t) and (r() representing the four boundary 2ir -ir
constraints of the rectangle. Specifically, (up) represents if object occupies (I,, yt) coordinates
below the top of the bounding rectangle for region r. The variables 2(dm) =((), and 2(r() satisfy the same conditions for the bottom, left, and right of the bounding rectangle, respectively. For each pair of object i and region r, we add the following constraints to the MIP to resolve the decision variables:
x (min) < 33 00(1 (1t))
(2)
Σ, x (maz) +00(1 (rt) ( (min)
(3)
(dn) (up)
< y/ + 0001 - 2,r
(4)
y/1 S 1/, (max) 00
zir
(5)
In the above constraints, the oo value represents a sufficiently large constant (c.g., the maximum of the width and height of a global bounding box) that causes the constraint to always be satisfied. For
(lt) example, in Eq. 2, the inequality is always satisfied if the binary decision variable 2ir is false as
17

---

we do not need to put any constraints if we do not occupy region with object i. However, if the variable true, we require that the coordinate z; the right of the z-boundary o, [min) We create an equivalent constraint for the remaining three rectangular constraints (see Eq. 3-Eq. 5).
Finally, we add constraint that resolves the decision variable Zir to true if all four rectangular constraints hold:
((n)
(rt) (dn) 4 S Frr 2ir Zir Zir + 00(1
(1)
(up)
(im)
Fi
(6n) Once again, if Zir is false, the inequality holds as we do not need to satisfy the rectangle inclusion
constraints if our object is not in region r. Otherwise, all four inclusion variables must be true, by
summing four, indicate that the object occupies region
We then add an additional constraint to ensure that each object occupies at least one region:
Vi, E, (in) >=1 1
Next, ensure that all pairs of objects in the scene do not overlap. To do this, we constrain the bounding boxes of each object to not overlap. Let a, be half of the side length of the bounding box of object i. There are four ways pair of objects with axis-aligned bounding rectangles can avoid overlapping: object is left of object j. object i is right of object j. object i is above object j. or object is below object j. We create indicator variables representing these conditions Gy Gi
(tr) (rt)
(up) (dn)
Gj Cij respectively. Next, add the following constraints the MIP to correctly the collision indicator variables:
(x; + az) S (zy - aU) + 00
1j
(tt)
(8)
(zy + az) I (z, - a.) + o0(1 - (
(9)
(y: + ax) S (y, - a,) + 20 Gjj (du))
(10)
(y3 + a²) I (y, - a.) + o0 -c, (up)
c/
(11)
If there is no collision between and j. at least one of the four indicator variables must be true. Hence, we set an additional constraint to ensure no collision:
Yejj (it) (rt) (dn)
+ Ci
(up)
(12)
We solve the MIP problem with IBM's CPLEX optimization library [67].
D Domains
The following subsections provide brief description of the search space, objective, and measure functions in our domains.
D.1 Shared Control Teleoperation
A teleoperation task involves a user providing joystick inputs to a robot arm with the intention of reaching a goal in the environment (Fig. 9a). It is generally hard for users to teleoperate a 6-DoF robot arm to the correct configuration [52]. Thus, in shared control teleoperation, the robot attempts to infer the human goal from a set of candidate goals by observing the low-dimensional joystick inputs provided by the user.
Following the shared control teleoperation framework from previous work [52], the robot solves a POMDP with the user's goal as a latent variable while it updates its belief about the goal based on the human input trajectory assuming a noisily-optimal user. To enable real-time decision decision-making, the robot performs hindsight optimization to approximate the POMDP and assumes first-order approximation of the value function. This results in the robot's actions being a weighted average of the optimal path towards each goal, where the weights are proportional to the respective goal probabilities.
18

---

(a) Shared Control Telcoperation
Shared Workspace Collaboration
Figure 9: Example scenarios from the two domains being executed in the real world (figure for shared control teleoperation taken from prior work [2]).
To formalize the scenario generation problem in the shared teleoperation domain, follow the QD formulation of prior work [2, 3J. The environment parameters the positions of the two goal objects in a bounded workspace, constrained to be reachable by the robot arm. The simulated human provides trajectory of joystick inputs towards their goal object, parameterized by a set of waypoints. The human model parameters are disturbances to these waypoints. The scenario parameters include the environment and human model parameters. The objective function f in the QD search is the time taken to reach the correct goal, with a maximum time limit of 10 seconds the robot fails reach the goal. The search aims find scenarios that are diverse with respect to the noise in human inputs and the scene clutter, thus the measures the human variation from the optimal path and the distance between goals.
D.2 Shared Workspace Collaboration
We consider a package labeling task (Fig. 9b), which instantiates the human-robot shared workspace collaboration domain of previous work [6, 54]. The human and the robot have different actions, i.e., the human labels package while the robot presses stamp, and they share set of goals, i.e., boxes to perform the task. The human and the robot cannot work simultaneously on the same object and the task finishes when all boxes are labeled and stamped.
We assume that the human picks label for an object from starting point and moves towards that object. Different boxes require different labels, thus we model the human as attempting reach the box corresponding to the label they picked up, regardless of the robot's actions. On the other hand, the robot can switch its goal while moving, since stamping can be performed with the same tool. This domain is more complex than the shared control teleoperation task because it includes manipulating a sequence of objects, rather than reaching a single object, and the objects are in disjoint workspace regions.
any goal object
As in the shared control teleoperation task, the robot reasons over the human goal by treating the human noisily-optimal. However, unlike in shared control teleoperation, the robot attempts to avoid the goal intended by the human.
The scenario parameters consist of the locations of three goal objects in a larger, disconnected workspace. We set the workspace boundaries to the quadrants of the L-shaped table in Fig. that are reachable by both the human and the robot arm. We model the human as moving to their goal while avoiding obstacles by solving softmax MDP. The objective is again the time to task completion since we wish to find challenging scenarios.
We choose two sets of measures m described below:
Minimum distance between goal objects and maximum wrong goal probability: We adopt the minimum distance measure from the shared control teleoperation domain in previous work [2]. Furthermore, one of the failure scenarios found in that work was caused by incorrect inference of
19

---

the human goal by the robot. Thus, we set as our second measure the maximum probability that is assigned to the wrong goal by the robot during the task, to search for potential failures in which the robot actually infers the human goal correctly.
Robot path length and total wait time: In the shared workspace collaboration task that we con- sider, there two main sources of delay: the robot needing to move across the two workspaces to reach different goals, and the wait time caused due to both the human and the robot wanting to work on the same goal. Hence, we choose the path length of the robot and the total wait time as the two measures to see how the team performance changes as these are varied.
E
Additional Example QD Formulations
While we consider two domains in our paper: shared control teleoperation and shared workspace collaboration, the QD formulation and our algorithms can be extended to other HRI domains as well. We discuss example QD formulations for two alternative domains below.
Robot navigation around pedestrians [68, 69]. The robot's objective here would be to minimize the time required to reach its goal while avoiding collisions with pedestrians. To find failures, the QD objective could be to maximize the time taken by the robot to reach the goal. Measures could include the number and size of obstacles in the scene, the robot's goal location, the minimum distance between the mobile robot and the pedestrians during navigation, the average speed of pedestrians, the curvature of the pedestrian trajectories, the total number of seconds the robot remained idle during the task, etc.
Assistive Feeding [70]. The robot's objective here would be to successfully transfer all bites on plate to the user. To find failures, the QD objective could be to maximize the number of failed bite transfers. Measures could include aspects of the user's capability (e.g.. how far the user can reach to receive bite, how long the user takes receive bite), physical user characteristics such as their height, time spent by the user being idle waiting for bite, etc.
F
Human and Robot Policies
F.1
Robot Policy
We adopt the robot policy defined in prior HRI works [52, 6] that introduced the domains considered in this paper. In both domains in this paper, the robot solves a POMDP with human goal as the latent variable. As in prior work [52], the robot assumes that the human is stochastically optimal and updates its belief based on observed human actions. performs hindsight optimization to calculate the values and update the belief in real-time, followed by first-order approximation to select the optimal action that maximizes the Q-value. In both domains, we follow the cost function definition in the corresponding prior work [52, 6). which makes the resulting optimal value function proportional to the distance to the goal and the optimal policy straight line.
We briefly discuss the specifics of the robot policy in the two domains below. In both domains, the robot action is computed the twist that should be applied to its end effector, which is then converted to the required joint velocities by inverse kinematics computation.
F.1.1 Shared Control Teleoperation
In shared control telcoperation, the human provides an input action to the robot. The Q-value of this action is defined as the sum of the cost incurred while executing the action and the value at the new position after action execution. The robot's 's belief is then updated based on the difference between the value and the Q-value at the current position corresponding to each goal.
Hindsight optimization followed by first-order approximation results in the robot's assistive action being weighted average of the straight-line paths to each goal, weighted by the corresponding probabilities assigned to them in the belief.
20

---

In App. H.1, we consider a different robot policy called policy blending [53]. The robot fully follows the user inputs while updating its belief like before. Once the probability assigned to goal is higher than threshold, the robot takes over and moves to the predicted goal.
F.1.2 Shared Workspace Collaboration
In shared workspace collaboration, the human acts independently. Hence, we maintain two sets of value functions one for the human and one for the robot. We calculate the human Q-value as the sum of the cost of executing the current action and the value the new position after action execution, similar to the shared control teleoperation domain. The robot's belief is updated based on the difference between human value and human Q-value at the current position corresponding to each goal.
We track the constraints on the robot's goals with the feasible goal-set formulation from prior work [6]. For each potential human goal, the robot maintains a set of goals that has not worked on and is different from the human goal. The goal set can be empty for some candidate human goals the robot has finished working on all other goals. The robot then treats all the goals that it has not worked the feasible goal set corresponding to that human goal. For action calculation, the robot creates mapping from each human goal to a corresponding goal-to-go, which is the goal with minimum value (the closest goal) in the corresponding feasible goal set.
The robot's action is based on the robot's value functions. Since we assume that the robot acts optimally, we do not explicitly calculate these values and simply assume a straight- line path each goal. Hindsight optimization followed by first-order approximation once again results in the robot's action being weighted average of optimal actions towards each goal-to-go.
Specifically, let b(g) be the probability assigned to goal and let F(g) be the goal-to-go correspond- ing to human goal g. Then, the weight corresponding to goal g' is given by bg)
F.2
Human Policy
F.2.1
Shared Control Teleoperation
In shared control teleoperation, we search for human policy parameters in the form of noise added to the waypoints from the starting location to the intended goal location. The human policy keeps track of the waypoints and computes the waypoint-to-go and the corresponding velocity based on the current position of the robot arm.
F.2.2 Shared Workspace Collaboration
In shared workspace collaboration, the human moves independently towards the goal and avoids obstacles on the way. We model the human policy through a softmax MDP whose values are pre- computed before simulating the scenario.
First, we discretize the space in which the human can move into a grid with cell sizes equal to the size of the goal object so that each goal is in one cell. We treat these cells as the states of the MDP and allow the human to move to any neighboring cell, receiving a reward of either -0.01 for moving to an orthogonally adjacent cell, -0.01 for moving to a diagonally adjacent cell, -1 for moving into an obstacle, or for moving into goal cell. We set discount factor to 0.9999 and perform softmax value iteration [55] with softmax temperature of 0.001 compute the Q-values for each state-action pair.
Since we have three goals in a scenario, we compute three sets of Q-values, one corresponding to each goal. Each value iteration instantiation treats the scenario's other goals as obstacles.
During simulation, the human policy converts the current location of the human into the grid cell belongs to, chooses the next grid cell based on the Q-values corresponding the current goal, and returns the velocity required to move to the center of the next cell.
21

---

In App. H.2, we consider new setting in which we search over two human model parameters: the inverse of softmax temperature (higher values result more rational human) and multiplier to the velocity (higher multiplier makes the human move faster).
G Implementation Details
We implement surrogate assisted scenario generation in a server-client framework. The server sim- ulates a given scenario in OpenRAVE [71] while the client executes QD search to generate new scenarios.
G.1 Scenario Simulation
We adapt the scenario simulation code from the open-source implementation of shared autonomy via hindsight optimization [72] to include the feasible goal set formulation for the shared workspace collaboration domain (described in App. F.2) and to simulate generated scenarios instead of fixed one.
We start a flask server that waits for the client to run QD search and send solutions to evaluate. Once we receive a candidate solution, we pass through the MIP solver and instantiate the objects, the robot, and the human in the OpenRAVE simulator.
We discretize the simulation into ticks, with each tick being divided into three phases that are ecuted in sequence: human action selection, robot action selection, and environment simulation. Human action selection and robot action selection follow the policy given in App. F.2 and App. F. respectively. In the environment simulation phase, the actions are executed, moving the human and the robot to a new state.
The shared control teleoperation task executes these phases in loop until the robot reaches the intended human goal or the time limit of 10 seconds is reached.
Since the shared workspace collaboration task consists of multiple steps, the human and the robot policies are wrapped into state machines. The human state machine has five states: a) moving to goal; b) waiting for space; c) working on a goal; d) resetting; e) done. The human is initially in moving to goal state and simply selects actions according to the human policy. Once goal is reached, the human waits till the goal is free work (waiting for space) and then starts working on the goal (working a goal). Once the work is complete, the human switches to the terminal state, done, if that was the last goal or moves back to the initial position (resettngg). To simulate working on the goal and moving back to the initial position, we simply pause the human for a specified amount of time. After the reset, the human starts moving to the next goal (moving to goal).
The robot state machine has six states: a) moving goal; b) replanning; c) waiting for space: d) working goal; e) resetting; f) done. The state transitions are similar to those of the human state machine, except for the moving to a goal state. Since the robot can get into configurations close to self-collision or joint limits when following a straight line path, it needs to replan back to the start before moving again. We simulate this by switching to replanning state, moving the robot back to its initial position, and then switching back to moving to a goal state.
The shared workspace collaboration task ends either after 100 seconds or after both the human and the robot reach the done state.
G.2 QD Search
We implement QD search on the client by modifying the pyribs library [73] and the open-source code for DSAGE [23] to match Algorithm 1.
We implement the inner loop through pyribs scheduler that interfaces QD algorithm via two functions: ask, which outputs candidate solutions from the algorithm, and tell, which accepts the corresponding objective and measures, adds them to the archive, and updates the algorithm param-
22

---

eters. The scheduler interfaces CMA-MAEGA and CMA-MAE for DSAS and SAS respectively. The inner loop runs fully on the client, exploiting the surrogate model described App. B.
We then select a set of solutions from the surrogate archive and send the simulation server for evaluation. The objective and measures obtained from the simulation returned by the server, which we add to the ground-truth archive and the dataset.
For baselines, we use the existing implementation of CMA-MAE and MAP-Elites in pyribs. Addi- tionally, for ease of execution, we implement Random Search similar to a QD algorithm in the pyribs framework. simply returns a batch of uniformly randomly sampled candidate solutions whenever requested. Since these baselines do not leverage surrogate model, the candidate solutions are always sent to the simulation server for evaluation.
To include objective regularization, we maintain two archives, the final archive that retains solutions maximizing the unregularized objective, and the training archive, which maintains scenarios that maximize the regularized objective to guide the QD search. The pyribs scheduler interfaces with the training archive, while solutions are directly added to the final archive. For surrogate assisted algorithms, the surrogate archive acts as the training archive while the ground-truth archive acts as the final archive.
We include the search details specific to the domains below.
G.2.1 Shared Control Teleoperation
In shared control teleoperation, we search over the (xr, v) coordinates two goal objects and five noise variables that define the human path towards the goal, creating a 9-dimensional search space.
We define the measures as the distance between the goals v(z x2)² + (y1 1/2 and the vari- ation in human input / / On... where 0₁, refers to the five noise parameters in the generated solution. Following prior work [2], assume the ranges of the measures be [0, 0.32) for the distance and [0, 0.112] for variation, and create an archive with 25 x 100 cells.
We adopt the hyperparameters for MAP-Elites from prior work [2], setting the standard deviation of perturbation, a, to 0.01 for parameters corresponding to the goal coordinates and 0.005 for those corresponding to the human noise. For CMA-MAE, SAS, and DSAS, we set the initial standard de- viation for CMA-ES, oo, to 0.01, archive learning rate, a, to 0.1, and minimum acceptance threshold, minf, to O. We set all other hyperparameters their default values defined in pyribs.
G.2.2 Shared Workspace Collaboration
In shared workspace collaboration, we search over the (z, y) coordinates of three goal objects, cre- ating 6-dimensional search space.
The four measure functions in our experiments are defined as follows:
1. Minimum distance between goal objects (archive range [0.05, 0.32]; discretized into 27 archive cells): min:zj v(x, x² (,>=,;)²
2. Maximum wrong goal probability (archive range [0.35, 1]; discretized into 65 archive cells): Let ((mex)()) (t) be function that returns the highest probability assigned by the robot
to goal other than the true human goal at time t. Maximum wrong goal probability is defined as the maximum value attained by ((max) (L) during the scenario:
max, ((maz) (l).
3. Robot path length (archive range 5): discretized into 20 archive cells): Let the robot's trajectory in the scenario be a function [0, + R², with 7(0) and r(1) denoting the coordinates of the start and end end-points respectively. The robot path length is defined as the length of this trajectory: fo ||dr||2.
4. Total wait time (archive range [0, 5); discretized into 50 archive cells): Let w(t) be a func- tion that returns 1 when either the robot or the human state machine is in waiting for space
23

---

Table QD-score at the end of 10,000 evaluations.
Shared Autonomy
Collaboration
Collaboration
DSAS
21, 10.400.30 33 + 45.91
106, 874.93 + 844.00
19, 261.95 + 182.57
SAS
21, 043.49 + 40.08
112, 02.966..22 = 572.96
18, 733.82 182.40
CMA-MAE
17, 17,972.31 + 74.71
87,399.75 1855.114
15, 612.29 + 284.34
MAP-Elite
117.87 + 358.31 9, 647.2.2 + 24.94
67, 731.48 = 576.30
18, 435.33.1 398.87
Random Search
62, 676.6662 + 200.68
13, 856. 14 + 156.67
Max Wrong Goal Delief Puub
DSAS
SAS
CMA-MAE
MAP-Elites
Random Search
1.00
100
50
25
0.35
0.05
0.32 0.05
0.32 0.05
0.32 0.05
0.32 0.05
0.32
Min Distance Between Goals
Figure 10: Comparison of the final archive heatmaps in the collaboration I domain.
state (see App. G.1) and O otherwise. Total wait time is defined So w(())dt where T is the total scenario time.
Note that approximate the integrals with discrete sums of the corresponding values at each sim- ulation tick.
We tuned the initial standard deviation for CMA-ES, op. in the case of CMA-MAE, SAS, and DSAS and set it to 1. We also tuned the perturbation standard deviation, a, for MAP-Elites and set to 0.1. We set a 0.1, miny 0, and all other hyperparameters the default values provided in pyribs.
In the new setting described in App. H.2, we add two additional parameters to the search: the inverse of softmax temperature (higher values result in more rational human) and the coefficient of velocity (higher coefficient makes the human move faster). We limit these parameters to ensure that the scenarios are not bottlenecked by an unrealistically slow or irrational human.
G.3 Computational Resources
Experiments for this paper were run on two local machines and a shared nigh-performance clus- ter. The local machines had AMD Ryzen Threadripper with 64-core (128 threads) CPU and an NVIDIA GeForee RTX 3090/RTX A6000 GPU. 16 CPU cores were allocated for each run on the cluster. One V100 GPU additionally allocated for runs with surrogate model.
The total time for each run was between 2 to 10 hours in the shared control teleoperation domain and between 12 24 hours in the shared workspace collaboration domain.
H Additional Results
We tabulate the results from experiments in Table 2. We also show the final archives in the collaboration I (Fig. 10) and collaboration II (Fig. 11) domains.
We observe that the archives generated by DSAS and SAS are more densely packed compared to other algorithms collaboration I. In collaboration II, we see that CMA-MAE, SAS, and DSAS find fewer solutions in the bottom left part of the archive compared MAP-Elites and random search, but find more and higher quality solutions in other parts of the archive which requires placing the goals in multiple regions. This is due to the bottom left part mostly corresponding to all goal objects

---

DSAS
SAS
CMA-MAE
MAP-Elite
Random Search
75
50
Total
25
Robot Path Length
Figure 11: Comparison of the final archive heatmaps in the collaboration II domain.
Table 3: QD-score at the end of 10,000 evaluations.
Telcoperation (Policy Blending)
Collaboration I (Human Policy Search)
DSAS SAS
41, 249.88 + 205.56
106, 573.18 + 1,
40, 726. 300.61
120, 789.83 = 1,378.82
CMA-MAE
33, 797.07 = 4555.2
120, 687.02 = 2,959.70
MAP-Elites
24, 151.97 836.97
81, 006.04 + 2,33.23
Random Search
19, 850.68 + 184.97
65, 513.84 + 334.12
in one workspace region and low wait time which allows MAP-Elites add small perturbations to the goal locations and easily obtain different robot path lengths. However, maintaining high wait time (the top part of the archive) while adjusting the robot path lengths is harder since even small perturbations in goal positions could make it easy for the robot to infer the human's goal and go to a different goal.
H.1 Additional Setting: Shared Control Teleoperation with Policy Blending
The QD formulation for scenario generation independent of the robot and human policies. Here, show scenario generation with new robot policy, policy blending (App. F.I), in the shared control telcoperation domain without any modifications to the QD hyperparameters the surrogate model architecture.
Table shows the QD-score at the end 10,000 evaluations. We see that the surrogate assisted algorithms outperform other algorithms, showing that these algorithms can work across multiple robot policies. Note that the maximum time for a scenario was set 20s, so the QD-scores around twice large as in the main shared teleoperation experiments.
H.2 Additional Setting: Shared Workspace Collaboration with Human Policy Search
In the main shared workspace collaboration experiments, the scenario was only parameterized by object locations. However, as described in our problem formulation, scenario parameters can also include parameters of the human model. Here, we perform an additional experiment which we search for human model parameters in addition the object locations to find failures the collab- oration I domain. We add more scenario parameters related to human speed and rationality as described in App. G.2 and run the QD algorithms with no other changes to the hyperparameters.
We tabulate the QD-scores Table 3. We see a small increase in the QD-scores of all algorithms compared to the main experiments (Table 2), since the QD search can now control the human policy to cause failures. Surprisingly, CMA-MAE performs similar to SAS. We hypothesize that this caused by the sensitivity of the scenario outcomes to the human model parameters: Changes to human speed or rationality affect the human trajectory much more than changes to goal locations. Hence, predicting the trajectory and scenario outcomes is much harder in this setting compared to
25

---

the main experiments. Thus, CMA-MAE, model-free QD algorithm, performs as well as SAS and outperforms DSAS.
However, the failures broadly fell into the same categories as those found in the main experiment. We hypothesize that this results from the bounds of the human policy parameters. Rational and fast human actions allow the robot to accurately prediet the human's goal, leading to fast scenario completion. On the other hand, we have set the bounds on the parameters to not allow QD search to make the human unrealistically slow or irrational. Hence, the failures found in this experiment are similar to those found with a fixed human policy.
H.3 Ablation: Effect of Objective Regularization
In Sec. 4, we proposed objective regularization as way to guide QD search towards valid workspace configurations. While objective regularization benefits general QD search, we note that surrogate assisted methods like DSAGE inherit additional benefits. As the surrogate model makes predictions for all possible scenarios, and not only scenarios satisfying the workspace constraints, the QD search that exploits the surrogate model can move towards high-mas magnitudg inputs in invalid regions of the scenario space when these inputs result in high objective values. Objective regularization helps pre- vent QD algorithms from exploiting errors in the surrogate model at extreme regions of the scenario parameter space.
To test the effect of objective regularization performance, choose the collaboration I domain and run 10 trials of DSAS, SAS, CMA-MAE, and MAP-Elites without objective regularization. Hence, due to numerical errors resulting from exploiting errors in the surrogate model, none of the SAS or DSAS runs without objective regularization could be completed.
We compare the results of MAP-Elites and CMA-MAE runs with their corresponding runs from the previous section that included objective regularization. Pairwise t-tests showed that MAP-Elite per- formed similarly with and without regularization, while CMA-MAE performed significantly worse without objective regularization (t = -7.08, p < 0.001). We attribute this to the fact that perturba- tions of existing solutions in MAP-Elites are not guided by the objective values. On the other hand, CMA-MAE guides the search based on the objective improvements of the sampled solutions; hence objective regularization has significant effect on performance.
I Additional Real World Scenarios
Incorrect human goal inference with limited effect on robot motion (Fig. 5b): We select a scenario from the archive generated by SAS with a relatively average scenario time of 77s and very high maximum wrong goal probability of 0.9.
The human finishes working on GI and the robot on G2. As the human moves towards G2 the robot incorrectly thinks that the human is moving to G3, which is near the optimal path to G2, causing the robot to slow down in anticipation the human motion. After the human reaches G2 the robot continues moving to G3. Hence, the incorrect prediction does not affect the overall scenario time much.
Long wait time due to both teammates needing to work on the same goal (Fig. 5d): Finally, we select a scenario from a DSAS archive in the collaboration II domain that has a high human and robot wait time.
This scenario was simple, albeit unanticipated. The human goes to GI, followed by G2, while the robot goes to G2, followed by G1. The team coordinates smoothly until both agents need to work on G3 to finish the task, causing delay.
26

---

(a) High Team Performance Scenario
(b) High Team Performance Scenario
Figure 12: Examples of scenarios with high team performance. The purple line shows the simulated human path.
Scenarios with High Team Performance
In addition finding failures, QD scenario generation can also find scenarios that are ideal for human-rohot collahoration. As example, modified the objective function in the collaboration I domain to 100 - T", with 7' being the scenario completion time that has maximum value of 100s. We ran SAS, which performed the best in this domain, and visualized example scenarios. We found two main types of success scenarios:
Objects placed far apart to avoid confusion (Fig. 12a): The first type of success involved placing the objects far apart allow accurate goal inference. However, placing them too far would require the human and the robot move lot, delaying completion. This scenario balanced these trade-off's leading to relatively short robot path length of 1.7m, low maximum wrong goal probability of 0.4, and fast completion time of 38s. The resulting goal completion order also avoided the failure found in Fig. 5d.
Objects placed close together to quickly change goals (Fig. 12b): The second type success ignored making goal inference easier but instead made it easier for the robot correct itself if required. Since the goals are close to each other, the robot human actions. Once the human starts working on goal, the robot can quickly switch to a different goal. Despite having a high maximum wrong goal probability of 0.8, this scenario only took 31s to complete.
start moving towards them irrespective of
27


Reviews:
[{"review_id": "fhcxV_XpYsB", "reviewer": "Reviewer_3aQ9", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "* The paper provides an extension on prior work with the cotribution of a new Quality diversity environment generation algorithm that receives feedback via surrogate models for a human and a robot that are trained on the generated environments\n* In particular, providing a quality diversity algorithm with (a) feedback on the feasibility of the generated scenario, via corrections, and (b) the \"true\" quality of the scenario via simulation, the authors show that scenario generation quality is better, the algorithm is more sample efficient, and for complex domains, the algorithm is time efficient too.", "summary_of_recommendation": "I strongly recommend that this paper be included in the technical program.\n", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "None", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Strengths**\n\n* The paper provides a novel and promising method to \"Oz the Wizard\" (Steinfeld, Aaron, Odest Chadwicke Jenkins, and Brian Scassellati. \"The oz of wizard: simulating the human for interaction research.\" Proceedings of the 4th ACM/IEEE international conference on Human robot interaction. 2009.)\n* The paper builds on prior work in the field and yet seems to be making a significant contribution\n* The paper surveys the literature on this topic adequately\n* The paper's evaluations are appropriate and the results are great\n* The paper characterizes its limitations appropriately\n", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}, {"review_id": "cmvDHsASBk", "reviewer": "Reviewer_rtcn", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper's premise is that the evaluation of human-robot interactive systems (HRI) is difficult and costly. Therefore, the paper proposes a surrogate model to predict the behaviour of the human and the robot. The overall goal is to find diverse scenarios for training. The work is evaluated on two scenarios using a Kinova JACO robot arm.", "summary_of_recommendation": "While the graphs of the QD score over time show an improvement for every point in time for the proposed approach and therefore would support the author's claims, the real-world impact of this score is not clear to me. However, I am not an expert in this area and will adjust my verdict given the recommendations of reviewers more confident in this area.\n", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "- In Figure 3, I assume that iterations and wall time should somehow correlate, meaning that when comparing the top and bottom rows, only the x-axis (iterations or time) should scale. This is true for Figures 3b and 3c. However, in Figure 3a DSAS has the highest QD score for most of the iterations (top row) while it has the lowest QD score over time (bottom row). Why is that?\n\n- The supplement video shows four \"successful\" failure scenarios that have been sampled from the predicted scenario parameter space. What would non-failure scenarios look like?\n\n- The authors claim that the proposed approach is computationally more efficient. This is shown by the \"quality diversity\" score over time in Figure 3, which shows a higher score at nearly any point in time for the proposed approach. However, the evaluation in the real scenario only shows that the sampled scenarios from the proposed approach are reproducible in the real world. It does not show if scenarios sampled from the baseline approaches are also reproducible in the real world or different from the proposed approach. Considering e.g. Figure 3c bottom row, how would SAS and MAP-Elites compare after 3h when reproduced in the real world? SAS here clearly has a higher QD score but it is unclear how this impacts a scenario in the real world.\n\n- Joint limits and self-collisions are trivial problems that can be avoided by standard motion planners. Why can this not be incorporated into the HRI task?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "HRI scenarios are highly relevant. The manuscript is clearly written and the work is properly motivated. The experiments show that the proposed methods provide more diverse scenarios compared to the chosen baseline methods. A small subset of the scenarios found by the proposed approach is evaluated on a real setup.\n\nThe concrete collaborative tasks in section 6 are not properly described. Without looking at Figure 5 and the accompanied video, it stays unclear what the actual task is. Even with these visual guides, it is unclear what an optimal solution to the HRI task would look like.", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "4Kp0V0umHvp", "reviewer": "Reviewer_EBp2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This work addresses the challenge of generating datasets containing challenging scenarios for shared control and human-robot interaction. To tackle this issue, the authors propose two variations of surrogate-assisted scenario generation algorithms that formalize model-based quality diversity (QD) problems. Unlike previous approaches that either do not leverage a surrogate model [1,2] or do not consider continuous state and action spaces [3], the proposed methods combine these elements effectively. The proposed approach follows a hierarchical process, with the inner loop utilizing the QD algorithm to identify diverse and challenging scenarios based on the surrogate model, while the outer loop evaluates and adds these scenarios to the dataset used for surrogate model training. The experimental results demonstrate that both methods outperform baselines in terms of sample efficiency and wall-clock time.", "summary_of_recommendation": "The paper is well-written and structured. Aside from some minor comments regarding the figures, my main concern revolves around the limitations of the algorithms in terms of scalability and computational demand. That being said, I recognize the value of the paper's contribution to the field of human-robot interaction (HRI) in terms of creating valuable scenarios for evaluating and benchmarking proposed methods. Therefore, I'm leaning towards accepting the paper, albeit weakly. I'm open to changing my decision to a full acceptance if the authors provide a compelling rebuttal that effectively addresses the concerns raised.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "I summarize my major questions below:\n1. I am curious about the scalability of the proposed algorithms,either in terms of task complexity or the number of measures. \n2. It would be greatly appreciated if you could provide additional insights into the computational demand of the proposed methods. Considering the potential challenges for more complex tasks, how do you foresee this computational demand becoming problematic?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I highlighted my comments below:\n\nMajor:\nFrom the perspective of shared control and human-robot interaction (HRI), this work makes a valuable contribution to the field by addressing the creation of challenging datasets for meaningful algorithm testing and evaluation. While previous work primarily focused on shared autonomy (SA), extending it to HRI is a reasonable direction. However, I am curious about how the proposed scenario generation approach would adapt to more complex and difficult tasks. As mentioned by the authors, improving realism would require incorporating more complex environments, human actions, and sophisticated simulators. Nonetheless, for the current work, the low-dimensional workspace was discretized. Considering that more complex tasks could lead to the curse of dimensionality, it may potentially impact performance. Could you please elaborate further on this aspect?\n\nFurthermore, the formulation of a quality diversity (QD) problem seems reasonable and yields notable results. However, in the current work, the optimization is performed over two user-defined measures, leading to convergence after a considerable amount of time (>12 hours). I am curious about the scalability of the proposed method in relation to the dimensionality of the given measures. Specifically, I would like to understand whether the proposed methods exhibit similar sample inefficiency as seen in Domain Randomization when faced with high-dimensional search spaces. Exploring how the proposed methods perform and handle increased dimensionality would provide valuable insights into their efficacy and potential limitations considering more complex HRI tasks. I kindly request the authors to address these questions and provide further clarification on the scalability of the proposed methods with respect to measure dimensionality.\n\nBoth proposed methods, DSAS and SAS, exhibit superior performance compared to the baselines in terms of sample efficiency and wall-clock time. However, when compared to prior work [1, 2], the introduction of the surrogate model increases the computational demand. As stated in line 198, the proposed methods were tested five times due to GPU usage constraints. It would be helpful if the authors could provide more insights into these limitations and discuss whether they could pose challenges in scenario generation for more complex HRI settings.\n\t\nMinor:\nThe paper is well-written, and the visualizations are well-made. However, I would like to suggest that the authors consider adjusting the placement of the figures to avoid having two figures on the same page below each other. This adjustment could enhance the readability and overall quality of the paper. Additionally, it would be beneficial to revise the scale of Figure 5, as the images appear quite small and are challenging to perceive. By making these adjustments, the figures will be more accessible and contribute to a better understanding of the paper's content.\n\nReferences:\n\n[1] M. C. Fontaine and S. Nikolaidis. A quality diversity approach to automatically generating human-robot interaction scenarios in shared autonomy. In Robotics: Science and Systems, 2021\n\n[2] M. C. Fontaine and S. Nikolaidis. Evaluating human–robot interaction algorithms in shared autonomy via quality diversity scenario generation. ACM Transactions on Human-Robot Interaction (THRI), 11(3):1–30, 2022.\n\n[3] V. Bhatt, B. Tjanaka, M. C. Fontaine, and S. Nikolaidis. Deep surrogate assisted generation of environments. Conference on Neural Information Processing (NeurIPS), 2022", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}, {"review_id": "1LFjJdAg77", "reviewer": "Reviewer_ZDjU", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper considers the problem of generating diverse/challenging scenarios for evaluating HRI tasks.  Previous works formulated the problem as a quality diversity (QD) problem which has been shown effective in a simple teleop HRI domain. However, previous methods require directly evaluating robot policies and human actions, which is expensive and limits the scalability to more complex tasks. The authors propose to instead train a model to predict task outcomes and use that model in place of expensive evaluations.\n\nThe authors evaluate their approach in two domains: a shared control teleoperation task and a shared workspace collaboration task. The reported experiments show the proposed approaches outperforming baseline QD algorithms in sample efficiency. Additionally, wall clock time improvement is shown on the collaboration task where evaluation is particularly expensive.", "summary_of_recommendation": "The problem is well motivated and relevant. I think the proposed approach is in some ways an incremental work combining the ideas of [1] and [21]. However, the authors propose sensible advances to adapt the ideas for application to more complex HRI systems and the empirical results presented in the paper are promising. For these reasons, I think the paper is interesting and worth sharing with the community.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "I would ideally like to see a more rigorous evaluation included in the paper, but I think that is probably out of scope for the rebuttal period.  In lieu of that, you might consider adding some discussion to make it more clear how you expect the approach to generalize to other HRI domains (e.g. describe some example measures m you might use for a more diverse set of domains).\n\nI like that you include an ablation study to help understand how the objective regularization contributes to the results of your experiments. Would it make sense to also include ablations without scenario repair to better understand the effect of that part of your approach?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n- In general, I found the paper to be well written and easy to follow.\n- The proposed approach seems like an elegant combination of the ideas in [1] and [21] with sensible advances to support application to more complex HRI systems\n- The empirical results presented in the paper are promising\n\nScope for improvement:\n- The approach is only evaluated on two (relatively simple) domains. I would like to see the approach evaluated on a larger and more diverse set of domains to better support the claims that the algorithms generally apply to HRI systems.\n- Because the paper is motivated by HRI systems, I would like to see some empirical evidence that the distribution of synthesized challenging human actions is similar to the distribution of challenging human actions that you would actually encounter “in the wild”. For the real world demo reported in the paper, a user is asked to reproduce scenarios extracted from the generated archives. This shows that the generated scenarios are reproducible, but it does not indicate that the generated scenarios cover situations that you would naturally encounter. I think the case for the proposed approach would be much stronger if the authors included some larger scale user study that shows that the types of difficult scenarios encountered naturally are actually covered by the distribution of synthesized scenarios.\n- The paper references an appendix multiple times (e.g. “See App. A for the complete pseudocode”), but there is no appendix included in the submitted document. After downloading the supplementary material I see the appendix is included in the zip file. The other submissions I am reviewing have the appendix included in the document (which makes the reference links work). Please include the appendix in the submitted document.\n- I personally find the diagram in Figure 2 to be difficult to follow. You might consider breaking it up into multiple diagrams. ", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Summary of Paper:
The paper delves into generating diverse and challenging scenarios for evaluating Human-Robot Interaction (HRI) tasks. The authors extend previous work by introducing a new Quality Diversity environment generation algorithm, taking feedback from surrogate models trained on the generated environments for humans and robots. Their approach bypasses the expensive evaluations of direct robot policies and human actions that hampered the scalability of prior methods. Through evaluations in two domains, shared control teleoperation and shared workspace collaboration, the paper showcases enhanced sample efficiency and reduced wall clock time compared to baseline Quality Diversity algorithms.

Strengths:
- The proposed method offers an inventive approach to generating challenging datasets for HRI, rooted in the Quality Diversity algorithm, which receives feedback via surrogate models. This is particularly notable considering its grounding in the "Oz the Wizard" framework.
- An impressive build on existing research, the paper's contribution is significant, aptly drawing from and expanding upon the foundational literature.
- The paper is well organized, ensuring comprehension and presenting a clear narrative of the proposed method's efficacy, evidenced by empirical results that underscore its merits over QD algorithms in terms of sample efficiency and timing.
- Authors are transparent about the limitations of their work, which underscores the paper's credibility.

Areas for Improvement:
- Domain Limitations: The proposed approach, while promising, has been tested only on two domains. Expanding evaluation across more diverse and complex domains would bolster claims about the algorithm's broad applicability in HRI systems.
- Empirical Grounding in Real-World Scenarios: There's a palpable gap between the generated scenarios' reproducibility and their alignment with real-world HRI challenges. A more comprehensive user study would validate whether the synthesized scenarios encapsulate the range of challenges one would face in actual environments.
- Clarity and Documentation: Several aspects, such as a clearer description of concrete collaborative tasks and the inclusion of referenced appendices directly within the paper, could enhance the submission's clarity and completeness. Some visual elements, like Figure 2, could be reworked for better comprehension.
- Computational Concerns: While the paper boasts of improved sample efficiency and reduced time, there are implicit concerns about the computational demands, especially when juxtaposed with prior works. Addressing the scalability and computational constraints, especially in more intricate HRI contexts, would be beneficial.
- Dimensionality and Complexity: The current work's limited scope, in terms of its low-dimensional workspace, raises questions about the algorithm's performance as the complexity and dimensionality of tasks increase. Engaging with these concerns might provide a more comprehensive view of the approach's versatility.

Discussion during the rebuttal phase:
The authors responded effectively to the queries raised by the reviewers, resulting in a significant improvement in the reviewers' rating scores. The authors have indicated in the rebuttal phase that they will add further references to related works and provide more detailed data in the appendix for the camera-ready manuscript. Consequently, we can expect an even higher-quality paper in the end.

Recommendation:
The paper showcases a commendable effort in advancing HRI scenario generation. Given the paper's considerable strengths, especially its novel approach and strong empirical grounding, coupled with its addressable weaknesses, I recommend accepting the paper. However, it would benefit the authors to consider the discussion with reviewers, addressing key concerns to ensure the work's robustness and broader applicability.
Decision: Accept (Oral)