Performative Body Mapping: A Creative Robotics
Method for Learning Expressive MovementRob Saunders∗
Design Lab
Sydney University
Sydney, Australia
rob.saunders@sydney.edu.au

Petra Gemeinboeck†
Creative Robotics LabUNSW | Art & Design
Sydney, Australia
petra@unsw.edu.au

Abstract
Performative Body Mapping is a method for harnessing the embodied expertise ofdancers to inform the design, movement and behaviour of non-anthropomorphic
social robots. The method simplifies the correspondence problem through thenovel use of costumes that allow much of the difficult human-robot mapping to bedelegated to dancers. A mixture density recurrent neural network has been used tomodel sequences captured during movement studies to create new dance sequencesin the style of the dancers inhabiting a costume.

Performative Body Mapping
A common underlying assumption in the design of social robots is that human-like or pet-likeappearance makes relating to them easier. Studies show, however, that the more human-like a robot
appears, the more people expect it to also have human-level cognitive and social capabilities, which
results in frustration when robots fail to meet expectations [1]. Designing more abstract robots poses
the question of how we might relate to them but artists and performers have been exploring thecapacity for the movement of abstract robots to evoke affective responses for decades [e.g. 2, 3]. This
paper describes a method for accessing the embodied kinaesthetic knowledge of dancers to inform alearning process for a machine-like robot to develop a social presence.
Performative Body Mapping (PBM) harnesses the embodied expertise of dancers to inform thedesign and movement of non-anthropomorphic robots by relying on the kinesthetic ability of dancers
to embody another, non-human body and deploying a ‘costume’, i.e., a wearable object that both
restricts and extends a dancer’s body. Costume serve as embodied interfaces for mapping between
dancers and robots, providing dancers with embodied insights into the morphology and capabilities
of a robot, which supports the development of a repertoire of movements and allows motion capture
in a form that the can be learned from, with little or no translation. Consequently, PBM significantly
simplifies the correspondence problem, common in demonstration learning [4], by mapping between
similar bodies and delegating much of the difficult mapping to movement experts.
Initial PBM movement studies focussed on form-finding through embodied exploration of costumes
shaped by ‘enabling constraints’, e.g., no front or back, head or limbs, see [5] for details. A simple
cube costume was developed (Figure 1a) as this was shown to be highly expressive when activated byskilled dancers. Custom software estimated the pose of the costume from video of the dancer-activatedcostume to inform the design of a mechanical prototype (Figure 1b) and provide data for learning.
From the approx. 15 hours of video, 5 hours was extracted for the purposes of learning. Each data
point consisted of six values describing the movement between poses, i.e., difference in location (x, y,∗†Metamakers Institute, Falmouth University, UK rob.saunders@falmouth.ac.uk
Metamakers Institute, Falmouth University, UK petra.gemeinboeck@falmouth.ac.uk

32nd Conference on Neural Information Processing Systems (NIPS 2018), Montréal, Canada.

(a) Costume inhabited by dancer.

(b) Robot motion testing.

(c) Robot as ‘plinth’.

Figure 1: Evolution from (a) costume to (b) prototype and (c) exhibition. Petra Gemeinboeck cz) and orientation (yaw, pitch, roll) of the centre of top surface of the costume. This describes theobserved movement as a pair of idealised joints but does not define the motor movements required toachieve it, which is handled by a fixed motor controller.
We applied a mixture density LSTM network, previously shown to successfully synthesise handwriting [6] and human choreography [7], to generate movements in the style of the captured recordings.
The network architecture consists of a 6-value input layer, 3 hidden layers with 512 LSTM cells eachand an output of 20 Gaussian mixtures to approximate the distribution of the next movement. The
network was trained with RMSProp using Back Propagation Through-Time. The synthesised movement sequences were subjectively assessed by movement experts against the original performancesof the dancers and judged to have captured important movement qualities.
The robot has been exhibited in Australia and the UK (Figure 1c). Audience studies suggest that,while it is clearly perceived as non-anthropomorphic, it is successful at conveying expressive agency[5]. Future work will include conditioning the predictions of the network using labels assigned bymovement experts to the recordings, similar to [6], to allow extended sequences to be developed.
An additional grounding stage will allow the fixed motor controller to be replaced with one learned
through ‘motor babbling’. Finally, intrinsically-motivated reinforcement learning will be used toexplore the potential of improvised movements and integrate audience reactions.
AcknowledgmentsThe authors would like to thank Tess de Quincey, director of De Quincey Co., Linda Luke and KirstenPackham. This research is supported under the Australian Research Council’s Discovery Projects
funding scheme (project number DP160104706) and partially funded by EC FP7 grant 621403 (ERA
Chair: Games Research Opportunities).

References[1] Dautenhahn, K. (2013) Human-robot interaction. In M. Soegaard and R.F. Dam (eds.), Encyclopedia of
Human-Computer Interaction, Interaction Design Foundation, Aarhus.
[2] Penny, S. (2000) Agents as Artworks and Agent Design as Artistic Practice. In K. Dautenhahn (ed.), HumanCognition and Social Agent Technology, pp. 395-414, John Benjamins.
[3] Apostolos, M.K. (1992) Robot choreography. Presence 1(1):149–150.
[4] Dautenhahn, K., Nehaniv, C. & Alissandrakis, A. (2003) Learning by experience from others—social learningand imitation in animals and robots. In R. Kuühn, R. Menzel, W. Menzel, U. Ratsch, M. Richter & I. Stamatescu(eds.) Adaptivity and Learning: An Interdisciplinary Debate, pp. 217–421, Springer.
[5] Gemeinboeck, P. & Saunders, R. (2017) Movement matters: How a robot becomes body, Proceedings of the
4th International Conference on Movement Computing (MOCO ’17), ACM.
[6] Graves, A. (2014) Generating sequences with recurrent neural networks. Tech. Report arXiv:1308.0850v5.
[7] Crnkovic-Friis, L. & Crnkovic-Friis, L. (2016) Generative choreography using deep learning, Proceedings ofthe Seventh International Conference on Computational Creativity, Paris, pp. 272–277.

2