Influencing Behavioral Attributions to Robot Motion During Task Execution
Authors: Nick Walker
Christoforos Mavrogiannis
Siddhartha Srinivasa
Maya Cakmak
Venue: CoRL2021 Poster (2021)
Keywords: human-robot interaction
behavioral attribution
robot motion
human impressions

Abstract: While prior work has shown how to autonomously generate motion that communicates task-related attributes, like intent or capability, we know less about how to automatically generate motion that communicates higher-level behavioral attributes such as curiosity or competence. We propose a framework that addresses the challenges of modeling human attributions to robot motion, generating trajectories that elicit attributions, and selecting trajectories that balance attribution and task completion. The insight underpinning our approach is that attributions can be ascribed to features of the motion that don't severely impact task performance, and that these features form a convenient basis both for predicting and generating communicative motion. We illustrate the framework in a coverage task resembling household vacuum cleaning. Through a virtual interface, we collect a dataset of human attributions to robot trajectories during task execution and learn a probabilistic model that maps trajectories to attributions. We then incorporate this model into a trajectory generation mechanism that balances between task completion and communication of a desired behavioral attribute. Through an online user study on a different household layout, we find that our prediction model accurately captures human attribution for coverage tasks.

Influencing Behavioral Attributions to Robot Motion During Task Execution
Nick Walker, Christoforos Mavrogiannis, Siddhartha Srinivasa, Maya Cakmak Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, USA (nswalker, cmavro, sidch, mcakmak 00cs uw.
Abstract: While prior work has shown how to autonomously generate motion that communicates task-related attributes, like intent or capability, we know less about how to automatically generate motion that communicates higher-level be- havioral attributes such as curiosity or competence. We propose framework that addresses the challenges of modeling human attributions robot motion, gen- erating trajectories that elicit attributions, and selecting trajectories that balance attribution and task completion. The insight underpinning tributions can be ascribed to features of the motion that don't 't severely impact task performance, and that these features form convenient basis both for predicting and generating communicative motion. We illustrate the framework in a coverage task resembling household vacuum cleaning. Through virtual interface, we col- lect a dataset of human attributions to robot trajectories during task execution and learn a probabilistic model that maps trajectories to attributions. We then incor- porate this model into a trajectory generation mechanism that balances between task completion and communication of desired behavioral attribute. Through online user study on a different household layout, we find that our prediction model accurately captures human attribution for coverage tasks.
approach is that at-
Introduction
As robots enter households and public spaces, it is increasingly important to account for human perceptions of their behavior [1, 2, 3, 4]. While a robot's actions might be driven by unambigu- ous internal objectives, solely optimizing such criteria might result in behavior that is difficult to interpret or disruptive to human onlookers. For example, a highly articulated robot may follow a non-humanlike trajectory that users attribute caprice to, making observers uncomfortable [5], home robot such as a robot vacuum cleaner can turn arbitrarily and cause observers to perceive the robot broken, interrupting home activity.
Accounting for high-level attributions to robot behavior is a complex problem relying on the mech- anisms underlying human attribution and behavior generation. In psychology, there is a long his- tory of work on understanding human attribution for behavior explanation or inference of behavior traits [6. 7]- Humans are highly attuned to how their actions are perceived and adapt their behav- ior to elicit desired impression from others or adhere to social norms, a concept that is known as presentation of self [8]. The tendency for humans to attribute even situational behaviors to deeper character traits is so pervasive that is known as "the fundamental attribution error" [9].
Inspired by these theories, we envision robots that can 1) leverage an understanding of humans' at- tribution mechanisms to predict attributions their motion, 2) generate behaviors that elicit desired human impressions and 3) balance attribution elicitation and task completion. This will increase the acceptance of robots in human spaces by enabling them go about their tasking a way that is both effective and sensitive to perceptions.
We propose framework, shown in Fig. that addresses these challenges by integrating learned model of human attribution into robot's trajectory generation process. Our observation is that users rely on local characteristics of the robot's motion, like short patterns of actions, in combination with global trajectory characteristics, like the amount of redundancy, to infer attributions. These features distinguish otherwise functionally equivalent trajectories and provide basis for the framework. We
5th Conference on Robot Learning (CoRI. 2021), London, UK.

---

Crowdrourcing
Analysis
Anriburions
Goal Attribution
User Responses
Salient Festures
Leaning
Trajectory
Model
Trajectories
Prodicted Attribnttion
Densitics
Optimized
Trajectory
Figure 1: Our proposed framework. responses to robot trajectories are analyzed extract salient features and attributions, then used to train a model that probabilistically maps robot trajectories to human attributions (left). The acquired model is used to generate robot trajectories that elicit desired attribution (right).
trace an application of the framework to virtual robot vacuum cleaning task. In our evaluation, we see that the resulting model is useful for predicting attributions and for enabling the generation of trajectories that balance task execution and attribution elicitation. Some aspects, such as partial trajectory observation or application to highly articulated robots, remain for future work.
2 A Framework for Behavioral Attribution
We consider robot performing task G' in a human environment. We denote by E S the robot state where S is a state space, and define a robot trajectory as a sequence of states { = (su..... ... 8t) where indices correspond to timesteps following fixed time parametrization. Let define the task C) where represents deterministic state transition model, and C trajectory cost. We assume that the robot starts from an initial state 8p and reaches a terminal state ST (at time T) executing trajectory observed by human who is aware of the task specification G.
as a tuple = space, P = A
space of robot trajectories, denotes the robot action
R is
(su...., ST). We assume that this trajectory is fully
The observer makes inference of the form Ia E x G + B, mapping their observation from space of trajectories E. along with the context the task specification G E G, into space of behavioral attributions B3 The form of B will vary, but should be selected to capture the range, combinations, and intensities of attributions that the robot should sensitive to.
Conversely, we can imagine that robot, given behavioral attribution E B and a task G. in- fers a trajectory & E that exemplifies the attribution, corresponding to an inference of the form Ie B x g E. In other words, we assume that there is "way" that a curious-or or any other attribution robot should execute particular task. In practice, we will realize both of these infer- ences as probabilistic maps. Rather than solely capturing the best way to look curious for task, we'll seek to assign densities to trajectories, allowing possibility that there are many equally likely alternatives.
In the remainder of paper, we aim provide general framework for modeling inferences of the form ZB. and Ze. Our goal is to enable robots to understand and account for the communicative effects of their motion human observers.
3 Related Work
Several studies have illustrated the value of robot motion as a communicative modality [10, 11, 12, 13, 14, 15]. Some works propose algorithms for legible robot motion generation, which have been shown to enable effective human-robot collaboration in manipulation tasks [10], or smooth robot navigation close proximity to humans [12, 14]. Other works focus on conveying higher-level information such the robot's objective function [15] the source of failure [13] when the robot

---

can't complete task. Animation principles [16] or movement analysis [17] are often employed inform the design of expressive robot behaviors. Finally, related graphics research focuses on the generation of stylistically distinct but functionally equivalent motion primitives for walking and other activities [18, 19].
The complex interplay of embodiment and communicative motion has motivated research on un- derstanding human perceptions of robot behavior. For instance, early work looked the effect of robot gaze on human impressions [20]. Sung et al. [1] study human attitudes towards robot vacuum cleaners and propose design principles aimed at enhancing the acceptance of robots in domestic en- vironments. [2] report relation between robot motion and perceived affect. Lo et al. [21] and [14]
investigate human perceptions of different robot navigation strategies whereas Walker al. [22] study human perceptions of robot actions that deviate from the robot's 's assigned task.
Our work draws inspiration from previous efforts characterize human perceptions and attributions to robot motion [1, 2]. However, moves beyond the problem of understanding and analyzing human perceptions and focuses on the problem of synthesizing implicitly communicative motion. Our work is closely related to past work on the generation of legible robot motion [10, 11, 23, 13] in that we also incorporate model of human inference into the robot's motion generation pipeline. However, unlike these works which emphasize the communication of task-related attributes, our focus is instead on communicating high-level, behavioral attributes through robot motion.
4 Modeling and Influencing Attributions
We consider a scenario in which a mobile robot performs a coverage task in a two-dimensional discrete workspace while a human is observing from top-down view. We employ virtual envi- romment' that resembles a house and stylize the agent as a robot vacuum cleaner (see Fig. 2) since the general population is already somewhat familiar² with such robots [1, 24], making it easier for participants to develop mental models about their motion than that of a manipulator, for example.
In this scenario, the robot state space is the complete home workspace and E is the space of all possible trajectories of any length that could be followed in the space. The robot action space A consists of deterministic movements in the cardinal directions. The cost of state transition from state 8, to a state St-1 after having followed a trajectory & is defined as 0 if the state hasn't t been visited before, -5 if the state contains small, traversable obstacle (e.g. vase), and -1 otherwise. A penalty of 3 times the number of unvisited states from the goal region is applied on termination.
4.1 Understanding Behavioral Attribution for Coverage Tasks
Through exploratory studies on Amazon Mechanical Turk, we sought to extract domain knowledge for attributions to robot motion within coverage tasks. Using the home layout shown in Fig. 2, we generated a set of trajectories exhibiting qualitatively distinct ways the robot could respond to the prompt to "clean the bedroom," ranging from a near optimal coverage plan trajectory that barely visited the target room (see Appendix A for additional details). Each participant viewed videos of a random selection of three these trajectories. After each video, participants were asked: a) to provide three words to describe the robot's behavior; b) to rate their agreement that "the robot for a range of adjectives drawn from relevant literature on human attributions [25, 26, 22]: c) "explain what factors contributed to their strongest ratings." In addition to attributions, participants were asked to use an interactive interface to demonstrate how they would "clean the where the blank was filled with random adjective from the attribution rating items. Across all exploratory studies, we collected 375 sets of attribution ratings from 115 participants (73 male, 41 female) aged 21-70 (M 38.3, SD 10.7) covering 63 trajectories and a total of 193 demonstrations.
bedroom in way that makes the robot look
4.1.1 Extracting the Space of Attributions
To understand the inter-correlation of participant adjective ratings, we conducted an exploratory factor analysis. We selected a three-factor, promax rotation model which explained 74% of the
'The environment built the Phaser game engine heldin under CC-BY-SA license. and the rest of paper's code can found in the supplement. A presentation by iRobot [24] estimates that 19M U.S. households had robot vacuum cleaners in 2020.
h/tp://ppaaeeeeeeeeeeee 1o/) and uses art by Bonsai-
3

---

Feature
Description
Coverage
region states
Redundant coverage (96)
Goal region states visited more than onee.
Overlap (16)
Plan states visited more than once.
Length
Normalized plun length. Frequesccy "U" shupe pattons plan. Hrequeey of action repetttion in plun. Proquency of ille-mose-idle putierns in plan.
Hook template (96)
Straight template (%)
Start-stop template (%)
Idieness
Prequency of actions
coverage (56)
Fraction map stzes visited
Collision (5)
Fraction of ohstrele states from O in plan.
Goal deviation (16)
Fraction of piza before first goal state.
Table I: Low-dimensional trajectory representation.
Figure 2: The home environment used in our ex- ploratory studies.
observed variance due to its parsimony and coherence (see Appendix B). The first factor, which we call "competence" for its similarity to the relevant factor described by Carpinella et al. [26] consists of six items (responsible, competent, efficient, reliable, intelligent, focused) centered on the capability and diligence of the robot. The second consists of four items (lost, clumsy, confused, broken) alluding to negative state, for which we title the factor "brokeness". The third contains two items (curious, investigative) and matches the curiosity factor examined by Walker et al. [22].
The extracted model enables the computation of standardized factor scores roughly in the range [-3,3) 3] which denote how many standard deviations from the mean a participant's ratings for the items are. Reflecting the format of the component items, a high or low factor score denotes agree- ment or disagreement that a trajectory expresses an attribution, respectively. Based on this model, represent the attribution for trajectory as tuple b = (brompetents Dbrokens bcarious) E B where the space of attributions is the set B [-3.3]3.
4.1.2 Low-dimensional Trajectory Representation
The space of possible trajectories in this domain is too large to map directly to the space of attri- butions, so we constructed a low-dimensional space & based on features relevant to the formation describe a trajectory & as vector & = <(E) E Φ. The of attribution ratings. This allows literature on human behavioral attribution to robot motion feature space was inspired by relevant and enriched with features appearing in participants' explanations. The final of 11 features used in further experiments is listed in Table 1.
4.2 Mapping Trajectories to Attribution Scores
Given trajectory &, an observer's inference Ig of behavioral attribution be expected vary both due to individual differences and as a result of measurement error. For this reason, we model In conditional probability density (sg (bg) B R. We observed multimodality in the dis- tribution factor scores for some trajectories, so we use a Mixture Density Network (MDN) [27] to
approximate each conditional density as mixture distribution fs (5)) = Ee (o k; (blog) where ag ...... C, mixing coefficient, and k; is a multivariate Gaussian kernel function with mean μ, and covariance Σ. Note that the mixing coefficients a; and the Gaussian param- eters 14 and E, are functions of the featurized trajectory Φg. our models, these functions are implemented as linear transformations of features produced by shared multi-layer perceptron.
To make efficient use of scarce data, we created ensembles of MDNs using bootstrap aggrega- tion, i.c., we trained N models with different data splits and uniformly weight their predictions: BBs (blbb) N E (bg
We studied three different model configurations; single and four component MDNs, i.e., C 1 and C= and an ensemble all models using an average negative log likelihood (NLL) loss function, the Adam optimizer [28), noise regularization [29], and early stopping. We configured the input MLP to use single hidden layer with 5 units and hyperbolic tangent activation. We expanded the dataset collected in our exploratory studies after assessing the sensitivity of the models increased amounts of data, process described in Appendix C. The final version of the set includes 126 trajectories with 671
MDNs each with four components, i.e., 4, N We trained

---

Model
Parameters
Average NLL
SD
Uniform
6
5.38
0.00
MDN, C=1
120
3.13 + .05
1.35 + .09
MDN, C=4
300
2.66 + 008
1.57 + .05
MDN Ensemble, C=4 N=8
2400
2.53 .06
1.38 1 .01
Table 2: Average negative log likelihood (NLL.) for each model configuration. Each datapoint represents a mean NLL over models trained with random train-validate folds on a fixed test set. Error the 95% confidence interval calculated with bootstrapping.
attribution ratings. Table 2 compares the NLL of the models over held-out data. The mean indicates the typical quality of the prediction and the standard deviation indicates the degree which this varied from sample to sample. Both quantities are averaged over 16 random folds and reported with bootstrapped 95% confidence intervals. All models perform significantly better than uniform baseline, which simply assigns equal probability to all outcomes. The ensemble model performs best and is used in further experiments in the remainder of the paper.
4.3 Generating Trajectories that Elicit Desired Attributions
We represent the behavior specification as a oinedimmnsi Gaussian b* ~ (1bb of²) centered on desired rating ju, [-3.3] for single attribution dimension where the variance db serves tolerance parameter modeling the acceptable distance from the desired behavioral rating. We use a density representation as it more closely matches the output of our model for Ip-
Together with the task requirements as described by the cost function C, we realize the inference Ie as an optimization of the form:
arg min EEE
(fL
s.t. C(E) w.
where DKL denotes the KL divergence, ÍB. is the density fBEL)(E) marginalized across dimen- sions other than i, and is the maximum allowable task cost. Because many applications are con- ventionally exclusively task-cost driven, this format provides an intuitive "knob" in the form of how suboptimal the robot is allowed to be. Where performance is critical pntical-perraps meet schedule or to fit in power constraints the robot designer need only set w to express the hard bound. Differ- attributions expected to be more less sensitive the allowable suboptimality, something illustrated in Appendix D.
We implement this optimization using a hill-descending search in the space of trajectories. The search is initialized with a task-optimal trajectory generated via A* search and progressively samples modifications to the trajectory. These modifications consist of both naive, action-level modifications to the trajectory as well as changes targeting the activation of the features underlying the attribution model (see Fig. 1). Important motion templates, like runs of straight motion, hook patterns, and start-stops are sampled and patched into trajectories. All modifications are ranked by the divergence of their predicted attribution with the behavior specification. The search terminates after a fixed duration and the best performing trajectory subject to the task-cost constraint is returned. A detailed description of the optimization procedure is given in Appendix D).
5 Evaluation
We conduct a user study to evaluate the efficacy of the framework as a means of producing trajecto- ries that elicit desired attributions. Our study is motivated by the following hypotheses:
Hypothesis The model makes accurate predictions about the distribution of attributions to new trajectories.
Hypothesis 2 The model makes accurate predictions about the distribution of attributions to trajec- tories in unseen environments.
Hypothesis 3 The approach enables the generation of trajectories that elicit desired attributions.
5

---

5.1 Experiment Design
In Experiment I, participants observe and rate trajectories in the same home layout used for data collection, while in Experiment II, trajectories are generated in a modified home layout. In an effort to understand the impact of the environment geometry, the modified layout increases the size of the goal region by 100%, varies the placement of items and obstacles, and flips the dominant direction of the robot's motion. The experiments are within-subbeets, video-based user studies, both instantiated in three parallel sets corresponding to the three attribution dimensions considered. For each dimension, we consider four robot trajectories generated by optimizing (I) with a target distribution expressed as Gaussian centered at 1.5 with scale 0.3 and varying task cost thresholds.
To ease interpretation, the w values governing the thresholds were set in multiples 1x, 2x, 4x, 12x of the cost of the optimal trajectory for the task. The full set of trajectories is shown in Fig. 3 In all experiments, participants rate and describe each trajectory using the same items and questions used in the exploratory studies of Sec. 4. After watching all trajectories in a randomly assigned order, they also respond to additional comparative questions: "which robot seemed the most
"nnd " where the blanks are filled with the adjective corresponding to the dimension of attribution studied. Both comparisons are accompanied with an open-ended question asking for brief explanation of the choice. No suitable baselines exist for the balanced attribution elicitation task, so our experiments use solely trajectories generated by our approach.
"which robot seemed the least
Participants A total of 144 participants (76 male, 68 female) aged 20-72 (M = 38.2, SD = 10.9) were recruited via Amazon Mechanical Turk and paid $2 to complete the approximately 15 minute task. 9 had taken part in our earlier exploratory studies. Participants were equally distributed amongst the six sets of conditions. Condition orderings were fully counterbalanced.
5.2 Results
The predictive performance of the model is illustrated in Fig. 5 and reported in Table 3. The 95% bootstrapped confidence interval on the mean NLL for Experiment was 2.77 (SD 0.8)+ and 2.87+0.10 10 (SD= (.86) for Experiment II. Participants' choices for "most" and "least" trajectories are shown Fig. 4.
Hypothesis was supported; the average NLL of the models was significantly lower than a uniform model, indicating that the model was able to meaningfully predict attributions in the layout it was trained in.
Hypothesis 2 was supported: the average NLL of the attributions observed across Experiment II was significantly lower than the uniform model, indicating that the model remains infor- mative even under modifications to the environment layout.*
Hypothesis 3 saw mixed support; Kendall's tau-b correlation tests (reported in Appendix E) in- dicate strong positive correlations between the allowable suboptimality and brokenness factor scores, suggesting that the trajectory generation method was effective eliciting progressively higher factor scores. However, tests for the competence conditions indicated a moderate negative correlation, and tests for curiosity conditions were not significant. As shown in Fig. 4, while participants found that 12x and Ix were the most and least " for experiments focused on curiosity and brokenness, this relationship was flipped for the competence experiments.
When optimizing for competence, the model emphasizes over-coverage of the goal region well as coverage of the house as a whole (see Fig. 3). Due to the associated task-cost penalty, coverage outside of the goal begins appear in the 4x condition of Experiment and the 12x condition of Experiment II, and participants' responses indicate that it is key driver of negative attributions of competence. Some emphasized that time spent not cleaning the bedroom the wrong room" (Exp. I- I-Competence), while others attributed the deviation to being "lost" or "to- tally confused" (Exp. I-Coopetence). In less extreme conditions, users expressed uncertainty about what drove the behavior, saying they were "not sure if robot is cleaning outside the hedroom cause
"wasted movement in
? VVideos of the trajectories are included the supplement. "See Appendix E for supplementary tests indicating insufficient evidence to support significant difference between the predicted and observed distributions in both experiments.

---

Experiment
Experiment II
Competence
Brokenness
Curiosity
Competence
Brokenness
Curiosity
12x
Figure 3: Traces of robot trajectories used in different experiments and conditions.
Competence
Brokenness
Curiosity
Competence
Brokenness
Curiosity
Loat
12x
12x
1x
2x
Figure 4: Counts of trajectories picked as most and least
Competence
Brokenness
Curiosity
Competence
Brokenness
Curiosity
1x
2x
4x
12x
33 3-2-1
3 21
0 2
3 2
2 -1
0
2 -3
Figure 5: Comparisons the model's predicted density for factor score under consideration (in grey), observed distribution (multicolored) approximated with Gaussian kernel density estimation and the target distri- bution (dashed). Each subplot has unique y-scale. with the magnitude of the difference between the predicted and observed densities any point encoded instead in the color of the line for the observed distribution. Deep indicates the model severely overpredicted the density, while deep blue indicates severe underprediction.
Competence
Brokenness
Curiosity
Competence
Brokenness
Curiosity
1x
2.51 (0.69)
2.44 (0.76)
2.37 (0.54)
2.73 (0.59)
2.64 (0.73)
2.76 (1.05)
2x
2.76 (0.85)
2.84 (0.72)
2.85 (0.75)
2.95 (0.70)
2.86 (0.88)
2.85 (1.09)
4x
2,98 (0.76)
2.93 (0.83)
3.02 (0.71)
3.00 (0.67)
2.92 (0.42)
3,10 (0.98)
12x
3.10 (0.73)
2.53 (1.75)
2.94 (0.72)
3.43 (0.84)
2.28 (0.98)
2.84 (0.84)
Table 3: Evaluation Average NLLL (SD)

---

there might be dirt that can be brought in, or just confused as to parameter of bedroom" " (Exp. I- Competence, 4x). A minority of users thought that the extra motion was worthwhile, marking the 12x trajectory as the most competent because it "cleaned more areas in both rooms" or "completed the entire home from bedroom to kitchen" (Exp. I-Competence). The model overestimated the prevalence of people that would appreciate the additional coverage of the environment, as indicated by both the "most/lcast" selections and the NLL. We speculate that the format of the experiment- wherein participants view the optimal trajectory and mentally anchor their ratings against it-is contributing factor; participants may be more inclined to rate the robot as competent when view- ing the optimized trajectories in real-world settings where direct comparison to the task-optimal trajectory is less likely.
Trajectories optimized to look broken progressively cover less of the goal region before ultimately devolving into repeated circular motion near the start point (see Fig. 3). The majority of users con- curred in their assessment of the 12x condition as "defective", with some remarking that it seemed overwrought, "a joke version of the robot" (Exp. II-Brokenness). The model underpredicted the ex- tent of participants agreement that the optimal trajectory would be perceived as not broken, but the predictive performance across all brokenness conditions was still the strongest of the three factors.
When optimizing for curiosity, the model emphasized over-coverage of the goal region, overlapping motion, like patterns and visiting penalized states depicted with vases (see Fig. 3). Some participants highlight the extra coverage as the reason for selecting the 12x as the most curious condition, saying the robot "cleans very well but cleaned the same place multiple time, roaming without reason" or that it "dawdled around lot, getting hyper fixated on certain spots" (Exp. II- Curiosity). The same factor is highlighted by participants in less extreme conditions, with one speculating of the 4x trajectory that "maybe something caught its eye while it was working and it got so distracted that he totally kept getting off track" (Exp. I-Curiosity). The change in the distribution of curiosity factor scores was expected to be small and the observations bear the predictions out, though it is notable that despite the subtle differences a majority of participants select either 4x or 12x as the "most curious" trajectory across both experiments.
Discussion
Our framework is a general approach for endowing robots with a sensitivity to the behavioral attribu- tions their motion elicits. We illustrated its application to coverage motions, but we envision lively stable of related instances stretching across domains from delivery service robots to robot arms in fulfillment centers. The process is the same; the robot arm's designer will build a pool of videos and study users' responses to to-and their reasoning about-the motion over a broad set of dimensions, then learn forward mapping from the features driving their reasoning to the attributions in their responses. While some features such as path length or redundancy may map over from the coverage domain, others like the shape of the acceleration profile may need be added to capture percep- tions of danger or erraticism. The reverse translation from a desired attribution to a new trajectory can be realized by searching in the space of trajectories, using the attribution features to guide the process-sometiee
that may have a pronounced impact in a higher dimensional planning space.
We haven't yet addressed some important aspects of behavioral attribution. While our results showed that the approach's 's performance transferred to a similar environment, future work should use more disparate environments to determine the limits of its generalization. In the setting we explored, the observer looks at the robot's motion from a top-down perspective, but different perspectives may result in different impressions. Further research should evaluate data collection techniques and environments that account for variability in observer perspectives. The features and the learned mappings from features to attributions are specific to the types of environments and the task consid- ered and would likely need to be augmented to work more broadly. We imagine that, in the future, robot designers will have access to a wide array of well-studied features with which to bootstrap their system, and when human-robot interaction data is abundant, we may see the rise of learned representations that can power the understanding and generation of motion with minimal additional supervision.
8

---

Acknowledgments
This work (partially) funded by the National Science Foundation IIS #2007011, 1, National Sci- ence Foundation IIS #1924435 "Program Verification and Synthesis for Collaborative Robots," Na- tional Science Foundation DMS #1839371, the Office of Naval Research, US Army Research Labo- ratory CCDC, Amazon, and Honda Research Institute USA. This material is based upon work sup- ported by the National Science Foundation Graduate Research Fellowship under Grant #17621 14. We thank the scientific Python Community for developing the core set of tools that enabled this work, including NumPy [30], SciPy [31], Pingouin [32], PyTorch [33] and Matplotlib [34].
References
[1] J.-Y. Sung, L. Guo, R. E. Grinter, and H. I. Christensen. "my roomba is rambo": Intimate home appliances. In J. Krumm, G. D. Abowd, A. Seneviratne, and Strang, editors, UbiComp 2007: Ubiquitous Computing, pages 145-162. Springer Berlin Heidelberg, Berlin, Heidelberg, 2007.
doi: 1oi:10.10010 10000..7777--11110 3-500-775550 340-73883-3.9.
[2] M. Saerbeck and C. Bartneck. Perception of affect elicited by robot motion. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 53-60, 2010. doi: 10000/0022222000555535555
[3] M. Kwon, M.F. Jung. and R. A. Knepper. Human expectations of social robots. In Proceedings of the ACM/IEEE International Conference on Human-Ronn Robot Interaction (HRI), pages 463- 464, 2016. doi:10.1 1099/R1. 201.7758877.
[4] E. Cha, A. D. Dragan, and S. S. Srinivasa. Perceived robot capability. In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pages 541-548, 2015. doi: 10.1
[5] Zacharias, C. Schlette, F. Schmidt, C. Borst, J. Rossmann, and G. Hirzinger, Making planned paths look more human-like in humanoid robot manipulation planning. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 1192-1198, 2011.
11..0.1199//RAAA200 .5979553.
[6] F. Heider. The psychology of interpersonal relations. John Wiley & Sons Inc., 1958.
[7] B. F. Malle. Attribution theories: How people make sense of behavior. In D. Chadee, editor, Theories in social psychology. page 72 95. Wiley Blackwell, 2011.
[8] E. Goffman. The Presentation of Self in Everyday Life. Anchor, 1959.
[9] L. Ross. The intuitive psychologist and his shortcomings: Distortions in the attribution process. volume 10 of Advances in Experimental Social Psychology, pages 173-220 220.. Academic Press, 1977.
[10] A. Dragan and S. Srinivasa. Integrating human observer inferences into robot motion planning. Autonomous Robots, 37(4):351-36, 2014. No1:10.10077/100514-1144440
[11] R. A. Knepper, C. Mavrogiannis, J. Proft, and C. Liang. Implicit communication in joint ac- tion. Proceedings of the ACM/IEEE International Conference on Human-Robot Interuction (HRI), pages 283-292 2017.
[12] D. Carton, W. Olszowy, and D. Wollherr. Mcasuring the effectiveness of readability for mobile robot locomotion. International Journal of Social Robotics, 8(5):721-741, 2016. doi:10.11000 s s12369-00 -01 6-0358-7.
[13] M. Kwon, S. H. Huang, and A. D. Dragan. Expressing robot incapability. In Proceedings of the ACM/IEEE International Conference Human-Robol Interaction (HRT), page 87-95, 2018. 14/11111444//111111111113
[14] C. Mavrogiannis, A. M. Hutchinson, Macdonald, Alves-Oliveira, and R. A. Knepper. Ef- fects of distinct robotic navigation strategies on human behavior in crowded environment. In Proceedings of the ACM/IEEE International Conference on Human- Robot Interaction (HRI), pages 421-430, 2019. doi: 10. 101.1100001111..1611007

---

[15] S. H. Huang, D. Held, P. Abbeel, and A. D. Dragan. Enabling robots to communicate their ob- jectives. Autonomous Robots, 43(2):309-326 2019. dbrrdbb..rr//1110007//0011114140
[16] L. Takayama, D. Dooley, and W. Ju. Expressing thought: Improving robot readability with animation principles. In Proceedings of the ACM/IEEE International Conference on Human- Robot Interaction (HRI), page 69-76, 2011. Nai.10.114519576653 1957674.
[17] H. Knight and R. Simmons. Expressive motion with x, y and theta: Laban effort features for mobile robots. In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pages 267-273, 2014. doi:10.1 10.1109//0MA.001 6926264.
[18] M. Brand and A. Hertzmann. Style machines. In Proceedingy of the Annual Confer- ence on Computer Graphics and Interactive Techniques (SIGGRAPH), page 183-192. ACM BeessAddiion-Wesstee Publishing Co., 2000. dbi:100.11453444779344444
[19] J. Guy, S. Kim, M. C. Lin, and D. Manocha. Simulating heterogenncous crowd behaviors us- ing personality trait theory. In Proceedings of the ACM SIGGRAPH//Euoorrpphhss Symposium on Computer Animation, page 43-52, 2011. doi: 2et110..115//0119422001444
[20] T. Kanda, H. Ishiguro, and T. Ishida. Psychological analysis on human-robot interaction. In Proceedings IEEE International Conference Robotics and Automation (ICRA), volume 4, pages 41 4166 vol.4, 2001. doi: 10. 109/ROBOTT2001 2..99999...00.33269.
[21] S. Lo, K. Yamane, and K. Sugiyama. Perception of pedestrian avoidance strategies of a self- balancing mobile robot. In Proceedings of the IEEE/RSJ International Conference Intel- ligent Robots and Systems (IROS), pages 1243-1250, 2019. doi:10. 109/IR0S40897.2019 8968191.
[22] N. Walker, K. Weatherwax, J. Alchin, L. Takayama, and M. Cakmak. Human perceptions of a curious robot that performs off-task actions. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI), page 529-538, 2020. doi:10.1145/331950 3374821.
[23] C. Mavrogiannis, W. B. Thomason, and R. A. Knepper. Social momentum: A framework for legible navigation in dynamic multi-agent environments. In Proceedings the ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 361-369, 2018. doi: 10. 1145/31712213111125.
[24] iRobot. Investor Presentation. August 2021. URL http: irobot........ com/ staxic-=ieee/686441110------2--- 91-.--1111119997911eeee100 htg.------------------------aaa- Accessed: 2021- 11-07.
[25] C. Bartneck, E. Croft, and D. Kulic. Measurement instruments for the anthropomorphism,, ani- macy, likeability, perceived intelligence, and perceived safety robots. International Journal of Social Robotics, 1(1):71-81, 2009. doi:
[26] C. M. Carpinella, A. B. Wyman, M. A. Perez, and S. Stroessner. The robotic social attributes scale (rosas): Development and validation. Proceedings of the ACM/IEEE International Conference Human-Rohan Interaction (HRI, pages 254-262, 2017. doi: doi:10.1145/2999844 3020208.
[27] C. Bishop. Mixture density networks. Technical Report NCRG/94/004, Aston University, 1994.
[28] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980. 2015.
[29] Rothfuss, F. Ferreira, Boehm, Walther, M. Ulrich, T. Asfour, and Krause. Noise regularization for conditional density estimation. arXiv:190.008982, 2019.
10

---

[30] C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Coumapeau, E. Wieser, Taylor, S. Berg. N. J. Smith, R. Kem, M. Picus, S. Hoyer, M. van Kerkwijk, M. Brett, A. Haldane, J. F. del Río, M. Wiebe, P. Peterson, P. Gérard-Marchant, K. Sheppard, Reddy, W. Weckesser, H. Abbasi, Gohlke, and E. Oliphant. Array programming with NumPy. Nature, 585(7825):357-362, Sept. 2020. doi: 10. 00000///4156--022-666666
[31] P. Virtamen, R. Gommers, E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, Peterson, W. Weckesser, Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson. E. Jones, R. Kern, E. Larson, C. J. Carey, 1. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, van Mulbregt, and SciPy 1.0 Contributors. SeiPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272, 2020. doi:10. 05000/441152-00-066--22-
[32] R. Vallat. Pingouin: statistics python. The Journal of Open Source Software, 3(31):1026, Nov. 2018. doi: 10.21 105/joss.01026.
[33] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, I.. Antiga, A. Desmaison, A. Kopf, E. Yang. Z. Vetito, M. Raison, Te- jani, S. Chilamkurthy, B. Steiner, I.. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style. pegp-performance deep learning library. H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc., 2019.
[34] J. D. Hunter. Matplotlib: 2d graphics environment. Computing in Science & Engineering, (3):90 2007. 10i: 10090/C/E22000777


Reviews:
[{"review_id": "oomh7-TPLBv", "reviewer": "Reviewer_ufvu", "summary": "This paper is studies the problem of inferring high-level behavioral attributes for robot motion plans. The work builds on a user study and psychology literature to to postulate a space of behaviors and features that are expected to inform behavioral attribution. Central to the approach is learning a multi-model distribution over attributes given a trajectory. The model is trained from crowd-sourced data and is used to synthesize trajectories that can balance motion cost and the expression of specified attributes. The model is validated using a user study. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength \n- The problem addressed in this work is well motivated and clearly articulated. Authors present a simple but elegant solution for the problem of incorporating behavioral attributions to robot motion planning. \n- The experiments and user study is well thought out and rigorously conducted. \n\nWeaknesses\n- The problem is confined to metric characteristics of the robot’s trajectory and the overall coverage attained by the planner in the environment. Intuitively, it appears that symbolic properties of the goal and map context would also play a role in behavioral attribution. For example, a robot moving around a glass vase quickly may be considered rushed/careless but the same may not be interpreted if the robot moves around other furniture. Richer symbolic attributes of the goal and the environment lie beyond the present approach. \n- The experimental evaluation does not explicitly compare against a baseline. Would recommend either comparing with an approach for related tasks such as legibility or state that the suitable baseline does not exist. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper clearly articulates the technical gap and presents an interesting approach to learning a model for behavior attribution and then using the model to synthesize plans that balance cost-to-goal with its behavioral attribution. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "- While reading the paper, I was less clear on how the trajectory data set was created. I understand that the trajectories were generated by varying the parameters of a conventional motion planner and were later annotated via crowd sourcing.  I would request more technical details. Alternatively, human turkers could have drawn trajectories on the map for specific behavioral attributes to more directly elucidate their mental model.  \n- How extensible is the trajectory generation process to richer behavioral specification? For example, can the framework handle the case where a human specifies that a particular attribute “should\" be present and another distinct attribute \"should not” be present in the synthesized trajectory. \n"}}, {"review_id": "R2UnhtBeyol", "reviewer": "Reviewer_zJU5", "summary": "The paper proposes a method to not only build a predictive model of human attributions based on robot motions as has been done in prior work, but to use this model to generate trajectories with those attributes. The generated trajectories are constrained to a good enough task performance. To this end, human attributions from a Mechanical Turk study are used to probabilistically map attributions to robot trajectories in a coverage task. Trajectories are then generated to optimize the Kullback-Leibler divergence of the behavior attribution model relative to a desired behavioral score factor distribution. This optimization is subject to an inequality constraint on the task cost. The value of the inequality is a design parameter that dictates how suboptimally the robot can behave in service of optimizing the behavioral attributes. The generated trajectories were then entered into a separate Mechanical turk experiment assessing how well the generated trajectories were perceived by humans. The approach was somewhat successful in eliciting the intended human perceptions by the participants.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\nThis work goes beyond prior work for extracting human perception and attributions for robot motions and tries to integrate these models into the motion generation pipeline similar to generation of legible robot motion.\n\nThe experiments were well-designed. The authors were able to build a reasonable and understandable model of human attributions mapped to trajectories. The formulation of a cost for matching the probabilistic model could be valuable in other scenarios where one would generate trajectories with higher level behavioral attributes.\n\nWeaknesses\nA major weakness of the paper was the lack of detail in the optimization. The paper describes a hill-descending search in the space of trajectories, but why is this in the space of trajectories when section 4.1.2 says that the space of trajectories is `too large to explore directly’? It appears that the configuration space is already discretized into states. (This is implied by the comments on the state transition costs.) I think that what the paper means is that the space of trajectories is too large to map attributions to directly, since the described optimization is clearly exploring over \\xi not \\phi. I understand the naïve action level choices would be included, but more detail is needed on how to choose motion templates and encode them in the optimizations. In its current form, an average graduate student would be daunted by the task of replicating this or applying it to a robot arm without the relevant details.\n\nI would also like to know why the factors are between -3 and 3. It seems like an arbitrary choice, so why this one in particular what are the consequences of making this domain larger or smaller?\n\nFigure/Table Issues:\nThere are tables listed under Figure 2, 3, and 4. It seems bad practice to include tables as subfigures. They should just be tables using the \\tabular environment. The paper also refers to fig. 2a as table 2a, which makes it inconsistent with itself. This practice also has made the text in fig./table 2a very small and hard to read. Fig/table 3a seems to have all the same data as what is contained in the plot in fig. 3b, so why not just drop the table altogether and leave the plot. Finally, figure 4 is hard to follow. I did not initially realize that it was two column all the way down until it hits the table which breaks the two column format above it.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper demonstrates a comprehensive approach to generating robot trajectories that exhibit behavioral attributes from collecting data, building a model, and exploiting that model to generate trajectories. The generation of trajectories is a significant advance over past work that only classified trajectories. However, some significant details about the optimization process is missing and the presentation of graphical and tabular data makes the paper somewhat difficult to digest. \n\nPost Rebuttal Update: The author's have added significant clarifications and addressed most of the concerns of myself and the other reviewers. For this reason, I have updated my score to weak accept.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "reviewer_expertise": "Fair: Some knowledge of the area"}, "extra_text": {"issues": "A revision should make the tables into Tables separate from any plots or graphs. The revisions should also clarify the reasoning for the low dimensional trajectory representation and provide more details on the optimal search for trajectories (at least in an appendix)."}}, {"review_id": "2SvyxhWLQJ3", "reviewer": "Reviewer_AY1D", "summary": "- The authors formulate a computational model for a human's classification of the behaviour typified by a robot's trajectory\n- In doing so, they are also able to formulate the inverse problem: a model to generate robot trajectories to best typify a desired behaviour\n- In a simulation domain, the authors had AMT workers classify sets of robot trajectories. An exploratory factor analysis revealed three factors that the humans' evaluated the robot trajectories upon. These were the target behaviour types in a subsequent experiment\n- The authors generated a model that was able to classify the likelihood of ratings of multiple people along the three factors (behaviours) for any given trajectory [as mentioned later, I am slightly confused on this point though]\n- The authors use this model, and a hill-climbing search algorithm, to then generate trajectories in a new environment that try to satisfy a desired rating from humans.\n- The authors find that they are able to predict participants' scores on the new trajectories at better than chance\n", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- I like the computational formulation of the problem. However, I commend the authors even more on their data-driven approach to the issue of getting the behaviour description objectives (the exploratory factor analysis)\n- Qualitatively, many of the results presented by the authors make sense:\n  - The images of the trajectories (probably cherry-picked to an extent?) would conform to the adjectives that the authors ascribe to them\n  - The lower predicted KL-divergence of generated broken trajectories, when the suboptimality bound is high, also makes sense\n  - The distributions of 4b are a good sign of the authors having accomplished part of what they set out to do\n- The results from the user studies are compelling evidence of the efficacy of this work.\n\n**Scope for Improvement**\n\n- The figures are hard to parse and it takes a while to know what the main result depicted in them are. 4c is particularly difficult. Focusing on that one, I think the authors are trying to showcase three results here:\n  - The ability of the model to provide a prediction of how humans might rate trajectories\n  - The ability of their framework to elicit human ratings that conform to a desired set of behaviour ratings\n  - The agreement of their models' predictions of human ratings to the actual ratings\n\nAs such, the plots for experiment 1 then serve a different purpose than those for experiment 2. Therefore I recommend the authors separate those and particularly focus on purpose 2 (above) for the plots from experiment 2\n\n- Related to the above point, for experiment 2, I think it's more important to showcase the divergences of participant ratings from the _target_ distribution of ratings instead of with the models' predictions. I suspect, given the current plots, that this might even lead to a more interesting story.\n- The authors couch their results in the context of hypotheses: however, they do not provide the details of any corresponding hypothesis tests (either Frequentist with p-values or Bayesian with ROPE / pd). As such, it's hard to say whether the hypotheses are supported (or the corresponding null hypotheses are rejected). I recommend the authors remove such language unless they perform and show the results of _quantitative_ hypothesis tests.\n- The text mentions a suboptimality condition of 12x; yet in the appendix, it is plotted as 2^3. Which is correct?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Although the paper has some weaknesses, they can be addressed by improving the presentation of the evaluations, as mentioned above. If this is done, I'm in favour of including this paper in the technical program.\n\nPost-Rebuttal Update:\nI thank the authors for their clarifications and find that the appendices, particularly Appendix D, is very helpful. I am much more in favour of accepting this paper into the technical program, but I am curious to see what my fellow reviewers say.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "(See scope for improvement; above)", "reviewer_expertise": "Fair: Some knowledge of the area"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses the issue of inferring high-level behavioral attributes for robot motion planning, and presents a framework to involve human attributions from a Mechanical Turk study and then used to evalute and generate robot motion trajectories in a coverage task. 

I agree with reviewers that the problem this paper addressed is well motivated and formulated, and the experiments is also well-designed, it's good to see the user study takes supports from psychology literatures. 

While the presentation on both the details of the methods and analysis of expreimental results seems a bit weak, as reviewers pointed out. I agree with these points which might be valuable in improving the quality of the paper, mainly including:

1. The lack of details in the optimization process. 
2. The experimental evaluation and analysis needs to be greatly improved, like explicitly compare against a baseline, carefully contrast the model with human, the results of quantitative hypothesis tests, etc.
3. Many more minor points as reviewers mentioned.

More added: 
I would like to thank authors for your carefully and detailed responses to comments raised by reviewers.
Although there are still rooms for this research to be improved, the updated version seems better, and the scores of this paper was improved after rebuttle. 
I thus recommend this paper to be accepted.
Decision: Accept (Poster)