Composable Semi-parametric Modelling for Long-range Motion Generation

1 INTRODUCTION . When faced with a specific goal in another location , humans can effortlessly find multiple distinctive trajectories and control their body to approach the goal with diverse and natural behaviours . However , we are still at the early stage for such sophisticated controlling of simulated characters . Recent reinforcement learning approaches ( Peng et al. , 2018 ) struggle to generate diverse motion . Other methods like imitation learning ( Ye & Alterovitz , 2017 ; Aleotti & Caselli , 2006 ; Lawitzky et al. , 2012 ) generalize badly to large scale demonstration data . The heart of this challenge lies in that real-world human behaviours are inherently multi-modal distributed . Direct behaviour learning is difficult without access to the explicit distribution of motion states . In this paper , we take a step toward generating long-range , diverse and physically plausible motion sequences given starting and ending states , rather than learning a policy for a physical simulator . Meanwhile , we expect the model could generate novel behaviour , i.e. , unseen motion in training set . This has several valuable applications : ( 1 ) Synthesised vivid motion for animation production without excessive human labor . ( 2 ) Generated novel behaviours for player customization of action skills in video games . ( 3 ) Interpolated sequences as subgoals for policy training through reinforcement learning ( Peng et al. , 2018 ) . To fulfil the above requirements , we need to address the following difficulties : ( 1 ) How to guarantee both diversity and naturalness , which is usually trade-off in the domain of generation ( Srivastava et al. , 2015 ) ? ( 2 ) How to achieve long-range behaviour synthesis , which is stuck by error accumulation problem in many temporal modelling tasks ( Denton & Fergus , 2018 ) ? ( 3 ) How to generate unseen behaviours without loss of diversity or naturalness , which is hardly addressed in previous researches ? As shown in Fig 1 , the current two main branches of motion generation methods , i.e. , parametric and non-parametric , are yet to deal with these difficulties properly . Parametric ( Goodfellow et al. , 2014 ) ( e.g. , GAN ) methods could not maintain the reality/naturalness of generated sequences , and diversity ( e.g. , VAE ) is hard to preserve . Non-parametric methods ( Haarbach et al. , 2018 ) involving motion clip copy/paste or blending has less superiority in smooth transition based on diverse reference sequences . 1https : //sites.google.com/view/cosmo-supp/home In this work , we propose COmposable Semi-parametric MOdelling ( COSMO ) , which is a method that can leverage the large spectrum of motion skills from unlabeled data in a semi-parametric manner . Our method combines the complementary strengths of both non-parametric techniques and parametric ones . First , to avoid mode collapse in the sequences , we initially sample reference subsequences from a held-out reference set to encourage multi-modal behaviors in the generated sequence , which is mainly inspired from non-parametric methods ( Haarbach et al. , 2018 ) . Second , we propose a self-supervised disentanglement model for extracting the content and style from each reference subsequence respectively . Here content refers to the characteristic of state at each time step , e.g. , moving speed , direction , and general gesture of upper body , while style refers to the long-range motion pattern which keeps relative constant across whole subsequence . An embedded latent space is constructed where the learned style vectors could be combined to obtain novel style feature . A new subsequence is obtained by composing content and style freely from different reference subsequences . Considering that this operation only produces a single subsequence , we refer this step as local motion composition . Finally , for all new sequences , we compose them along the temporal direction in a sequential order . However , from a hodgepodge of reference segments , it is non-trivial to generate natural and meaningful behaviors . To guarantee naturalness at long-range scale , we then propose to use goal conditioned bi-directional interpolation for modeling the long-range nature needed in the task . This step is regarded as global motion composition , which covers the whole temporal scale ( often more than 100 steps ) . On two human motion datasets , we show that the proposed method outperforms existing parametric and non-parametric baselines . We also demonstrate the generated sequences are useful as subgoals for actual physical execution in the animated world . Our paper makes three contributions . First , we proposed COmposable Semi-parametric MOtion generation ( COSMO ) , a method that can generate the required sequences in a semi-parametric , composed way . Second , we conducted experiments on CMU Mocap dataset 2 and SFU Mocap dataset 3 and shows our method outperforms strong baselines such as VAEs and GANs significantly . Third , we proved empirically that the generated sequences can serve as subgoals to learn an actual policy . 2 METHOD . 2.1 SEMI-PARAMETRIC MODELLING FOR MOTION GENERATION . Given starting and ending states ( denoted as s0 and sL respectively ) as inputs , our goal is to synthesize intermediate states that have natural transition and diverse behaviors . As illustrated in Fig 2 , NR reference subsequences ( denoted as Ri , i = 1 , .. , NR ) are sampled and fed into local motion composition 2http : //mocap.cs.cmu.edu/ 3http : //mocap.cs.sfu.ca/ sub-module φloc . Invalid subsequences are first filtered out with predefined requirements and please refer to appendix 6 for detailed description . φloc is used for altering the motion style of reference subsequence Ri based on another sequence R as input or directly sampling a style feature from learned latent space : R̂i = φloc ( Ri , R ) , where R is optional input . With this in hand , we compose edited subsequences { R̂i } NRi=1 along temporal direction smoothly . Concretely , we generate a new clip R̃i−1 : i = φglo ( R̂i−1 , R̂i ) which bridges the gap between R̂i−1 and R̂i . The final composed sequence is obtained through concatenation along the temporal direction for all subsequences and generated clips . Why semi-parametric modelling ? Regarding both motion category and transition dynamics ( from current to next state ) , human behaviors are intrinsically multi-modal distribution ( Holden et al. , 2017 ) . Parametric models ( e.g. , VAE ( Kingma & Welling , 2014 ) or GAN ( Goodfellow et al. , 2014 ) ) assume prior distribution to be uni-modal and predefined , which is not suitable to handle real world motion sequences ( Kingma & Welling , 2014 ; Arora & Zhang , 2017 ) . Non-parametric model does not require explicit prior distribution but not able to deal with long-range and highly diverse motion properly , mainly restricted by modelling capability . We also conduct targeted comparison experiments in Section 4 . Compared to parametric methods , semi-parametric modelling naturally guarantees motion diversity by introducing reference sequence also maintaining a large degree of freedom for generation . Moreover , semi-parametric modelling does not require access to or a model of data distribution . It implies that our method is also directly applicable for unseen motion sequence . Constrained by starting and ending states with unchangeable locations , it is not feasible to randomly sample reference subsequences for generation . To construct a complete motion sequence , we execute a manually designed procedure for searching reference subsequences . For detailed description , please refer to appendix 6 2.2 LOCAL MOTION COMPOSITION WITH CONTENT/STYLE DISENTANGLEMENT . Local motion composition stands for generating a relative short-range of motion clip based on reference subsequences as inputs . Content and style features are learned jointly , whose free-form composition is used for synthesising new motion sequence . We first describe the motion style modelling part . Motion Style Modelling . Style is considered as motion information which stays the same throughout the whole subsequence . To this end , we extract style pattern by conducting feature fusion along the temporal direction . More specifically , for a sequence with T frames , we gradually fuse all frames into one constant feature with Cs channels ( denoted as hs ) . We adopt 1D convolution along the temporal direction for each operation . Three convolution layers are stacked together with a kernel size of ( T/2 , T/4 , T/8 ) respectively , where large kernel size facilitates capturing motion style throughout the whole subsequence . To get better generalization ability for unseen reference or starting/ending states , we propose to construct an embedding space for style pattern modelling . More specifically , a set of embedding vectors ( denoted as Hs = { his } Mi=1 ) are learned together with above convolution layers ( denoted as ψsty ) . The corresponding outputs ( denoted as As ∈ RM ) of ψsty are a family of coefficients for a weighted sum of Hs . This is mainly inspired by the intuition that complex human activity could be considered as a combination of several simple actions . When trained with sufficiently large motion data , the model tends to learn basic style patterns for human motion . Compared to directly outputting a feature vector , this helps better generalize to unseen motion during testing . The final style feature ( denoted as fs ∈ RCs ) is generated as follows : f is = ψsty ( R i ) Hs . ( 2.1 ) Motion Content Modelling . In this part , we focus on motion dynamics reflected by each single state . To this end , 1D convolution layer with kernel size of 3 , i.e. , covering consecutive 3 states , is adopted for content modelling . The output channel of this layer is designed less than original motion dimension for extracting the most relevant information . This is empirically set to 5 as a design choice . Meanwhile , velocity , position for root joint and step pattern for foot joint are fed as inputs . We denote all the above factors as fc with T steps and Cc channels . During local motion composition , style feature fs is combined with content feature fc through concatenating along the channel axis : f ic = ψcnt ( R i ) , R̂i = ψrec ( f i c , f i s ) , ( 2.2 ) where fs is tiled T times to match with fc and ψrec outputs reconstructed R̂i . ψrec follows stacked three layer 1D convolution operation with kernel size of 3 . Finally , reconstruction loss is used to learn the style and content feature jointly as follows : LRrec = 1 NR ΣNRi=1||R̂ i −Ri||22 . ( 2.3 ) 2.3 GLOBAL MOTION COMPOSITION VIA GOAL CONDITIONED BI-DIRECTIONAL MODELLING . Global motion composition is connecting edited short-range subsequences into a completed and long-range one . Generating smooth and natural transition between R̂i and R̂i+1 is critical to obtain a long-range sequence . The major challenge comes from large states variation between R̂i and R̂i+1 . Inspired by recent work of Bi-LSTM modelling ( Ma & Hovy , 2016 ) , bi-directional motion composition is utilized for this part . More specifically , motion states are first mapped to a higher dimensional space with ϕenc and ϕdec , which is defined as follows : f it = ϕenc ( r i t ) , r̂ i t = ϕdec ( f i t ) . ( 2.4 ) Both ϕenc and ϕdec are 1D convolution layers . Then we predict the possible states at both directions with f it , where higher dimension ( 256 ) provides a sparser space enabling better composition results . For forward prediction , we take f iT−4 : T as inputs to obtain forward states f i : i+1 1 . Meanwhile , f i+1 1:5 are fed as inputs treated as goal conditions . We conduct this procedure in a recurrent manner with P -step prediction in total . Predicted f i : i+11 is concatenated with last four states forming next step inputs . Backward direction takes exactly the reverse procedure , i.e. , f i+1T−4 : T are treated as inputs while f i 1:5 are as goal : f i : i+1p+1 = ϕfpre ( f i p−4 : p , f i+1 1:5 ) , f i+1 : i p+1 = ϕbpre ( f i+1 p−4 : p , f i T−4 : T ) , ( 2.5 ) where ϕfpre and ϕbpre share the same architecture .. Moreover , we utilize another model to produce coefficients for the weighted sum of outputs . In this way , we are able to construct more flexible latent space to facilitate composition : c = ϕcom ( f i : i+1 p , f i+1 : i P−p , p ) , f̂p = cf i : i+1 p + ( 1− c ) f i+1 : iP−p . ( 2.6 ) ϕcom is one layer 1D convolution with sigmoid function for producing c. More specifically we introduce p and linearly map it to p̂ ∈ ( −1 , 1 ) and c = p̂+ 0.1 ∗ σ ( conv ( [ f i : i+1p , f i+1 : iP−p ] ) ) . During training , we randomly select a motion clip with a length of 10 + P . First as well as last 5 states are fed as inputs to get intermediate P states . Reconstruction loss is used for training : Lfrec = 1 P ΣPt=1||r̂t − ϕdec ( f̂ t ) ||22 . ( 2.7 ) Compared with goal-conditioned single forward prediction model , the starting and ending states are treated equally in our model , which avoids drifting away from the ending state . Meanwhile , we provide corresponding empirical results in Section 4 . After global motion composition , final composed sequence is directly obtained by concatenating all edited subsequences ( local part ) as well as transition clips ( global part ) along the temporal direction . We use tensorflow ( Abadi et al. , 2015 ) to implement all our models . All activation function used in our model is ReLU operation . During training , learning rate is set to 1e-4 and optimized with Adam optimizer ( Kingma & Ba , 2015 ) . β1 and β2 are set as 0.9 and 0.999 respectively . All models are trained with 30 epochs in total . Note that the local and global motion composition submodule do not need joint training . They are used jointly during testing .

Summary: The paper tackles the problem of generating long-range, diverse and natural looking motion sequence between initial and end states, and proposes to use a semi-parametric approach consisting of local and global models. Specifically, first the proposed approach extracts local motion feature from a reference subsequence and style feature from another, and then generates a new motion sequence. Then, global motion composition is done to interpolated generated local subsequences by bi-directional composition. In experimental validation, the approach outperforms two baselines (GAN and VAE).