Performative Body Mapping : A Creative Robotics Method for Learning Expressive Movement

Performative Body Mapping is a method for harnessing the embodied expertise of dancers to inform the design, movement and behaviour of non-anthropomorphic social robots. The method simplifies the correspondence problem through the novel use of costumes that allow much of the difficult human-robot mapping to be delegated to dancers. A mixture density recurrent neural network has been used to model sequences captured during movement studies to create new dance sequences in the style of the dancers inhabiting a costume. Performative Body Mapping A common underlying assumption in the design of social robots is that human-like or pet-like appearance makes relating to them easier. Studies show, however, that the more human-like a robot appears, the more people expect it to also have human-level cognitive and social capabilities, which results in frustration when robots fail to meet expectations [1]. Designing more abstract robots poses the question of how we might relate to them but artists and performers have been exploring the capacity for the movement of abstract robots to evoke affective responses for decades [e.g. 2, 3]. This paper describes a method for accessing the embodied kinaesthetic knowledge of dancers to inform a learning process for a machine-like robot to develop a social presence. Performative Body Mapping (PBM) harnesses the embodied expertise of dancers to inform the design and movement of non-anthropomorphic robots by relying on the kinesthetic ability of dancers to embody another, non-human body and deploying a ‘costume’, i.e., a wearable object that both restricts and extends a dancer’s body. Costume serve as embodied interfaces for mapping between dancers and robots, providing dancers with embodied insights into the morphology and capabilities of a robot, which supports the development of a repertoire of movements and allows motion capture in a form that the can be learned from, with little or no translation. Consequently, PBM significantly simplifies the correspondence problem, common in demonstration learning [4], by mapping between similar bodies and delegating much of the difficult mapping to movement experts. Initial PBM movement studies focussed on form-finding through embodied exploration of costumes shaped by ‘enabling constraints’, e.g., no front or back, head or limbs, see [5] for details. A simple cube costume was developed (Figure 1a) as this was shown to be highly expressive when activated by skilled dancers. Custom software estimated the pose of the costume from video of the dancer-activated costume to inform the design of a mechanical prototype (Figure 1b) and provide data for learning. From the approx. 15 hours of video, 5 hours was extracted for the purposes of learning. Each data point consisted of six values describing the movement between poses, i.e., difference in location (x, y, ∗Metamakers Institute, Falmouth University, UK rob.saunders@falmouth.ac.uk †Metamakers Institute, Falmouth University, UK petra.gemeinboeck@falmouth.ac.uk 32nd Conference on Neural Information Processing Systems (NIPS 2018), Montréal, Canada. (a) Costume inhabited by dancer. (b) Robot motion testing. (c) Robot as ‘plinth’. Figure 1: Evolution from (a) costume to (b) prototype and (c) exhibition. Petra Gemeinboeck c © z) and orientation (yaw, pitch, roll) of the centre of top surface of the costume. This describes the observed movement as a pair of idealised joints but does not define the motor movements required to achieve it, which is handled by a fixed motor controller. We applied a mixture density LSTM network, previously shown to successfully synthesise handwriting [6] and human choreography [7], to generate movements in the style of the captured recordings. The network architecture consists of a 6-value input layer, 3 hidden layers with 512 LSTM cells each and an output of 20 Gaussian mixtures to approximate the distribution of the next movement. The network was trained with RMSProp using Back Propagation Through-Time. The synthesised movement sequences were subjectively assessed by movement experts against the original performances of the dancers and judged to have captured important movement qualities. The robot has been exhibited in Australia and the UK (Figure 1c). Audience studies suggest that, while it is clearly perceived as non-anthropomorphic, it is successful at conveying expressive agency [5]. Future work will include conditioning the predictions of the network using labels assigned by movement experts to the recordings, similar to [6], to allow extended sequences to be developed. An additional grounding stage will allow the fixed motor controller to be replaced with one learned through ‘motor babbling’. Finally, intrinsically-motivated reinforcement learning will be used to explore the potential of improvised movements and integrate audience reactions.