Composable Semi-parametric Modelling for Long-range Motion Generation

1 INTRODUCTION . When faced with a specific goal in another location , humans can effortlessly find multiple distinctive trajectories and control their body to approach the goal with diverse and natural behaviours . However , we are still at the early stage for such sophisticated controlling of simulated characters . Recent reinforcement learning approaches ( Peng et al. , 2018 ) struggle to generate diverse motion . Other methods like imitation learning ( Ye & Alterovitz , 2017 ; Aleotti & Caselli , 2006 ; Lawitzky et al. , 2012 ) generalize badly to large scale demonstration data . The heart of this challenge lies in that real-world human behaviours are inherently multi-modal distributed . Direct behaviour learning is difficult without access to the explicit distribution of motion states . In this paper , we take a step toward generating long-range , diverse and physically plausible motion sequences given starting and ending states , rather than learning a policy for a physical simulator . Meanwhile , we expect the model could generate novel behaviour , i.e. , unseen motion in training set . This has several valuable applications : ( 1 ) Synthesised vivid motion for animation production without excessive human labor . ( 2 ) Generated novel behaviours for player customization of action skills in video games . ( 3 ) Interpolated sequences as subgoals for policy training through reinforcement learning ( Peng et al. , 2018 ) . To fulfil the above requirements , we need to address the following difficulties : ( 1 ) How to guarantee both diversity and naturalness , which is usually trade-off in the domain of generation ( Srivastava et al. , 2015 ) ? ( 2 ) How to achieve long-range behaviour synthesis , which is stuck by error accumulation problem in many temporal modelling tasks ( Denton & Fergus , 2018 ) ? ( 3 ) How to generate unseen behaviours without loss of diversity or naturalness , which is hardly addressed in previous researches ? As shown in Fig 1 , the current two main branches of motion generation methods , i.e. , parametric and non-parametric , are yet to deal with these difficulties properly . Parametric ( Goodfellow et al. , 2014 ) ( e.g. , GAN ) methods could not maintain the reality/naturalness of generated sequences , and diversity ( e.g. , VAE ) is hard to preserve . Non-parametric methods ( Haarbach et al. , 2018 ) involving motion clip copy/paste or blending has less superiority in smooth transition based on diverse reference sequences . 1https : //sites.google.com/view/cosmo-supp/home In this work , we propose COmposable Semi-parametric MOdelling ( COSMO ) , which is a method that can leverage the large spectrum of motion skills from unlabeled data in a semi-parametric manner . Our method combines the complementary strengths of both non-parametric techniques and parametric ones . First , to avoid mode collapse in the sequences , we initially sample reference subsequences from a held-out reference set to encourage multi-modal behaviors in the generated sequence , which is mainly inspired from non-parametric methods ( Haarbach et al. , 2018 ) . Second , we propose a self-supervised disentanglement model for extracting the content and style from each reference subsequence respectively . Here content refers to the characteristic of state at each time step , e.g. , moving speed , direction , and general gesture of upper body , while style refers to the long-range motion pattern which keeps relative constant across whole subsequence . An embedded latent space is constructed where the learned style vectors could be combined to obtain novel style feature . A new subsequence is obtained by composing content and style freely from different reference subsequences . Considering that this operation only produces a single subsequence , we refer this step as local motion composition . Finally , for all new sequences , we compose them along the temporal direction in a sequential order . However , from a hodgepodge of reference segments , it is non-trivial to generate natural and meaningful behaviors . To guarantee naturalness at long-range scale , we then propose to use goal conditioned bi-directional interpolation for modeling the long-range nature needed in the task . This step is regarded as global motion composition , which covers the whole temporal scale ( often more than 100 steps ) . On two human motion datasets , we show that the proposed method outperforms existing parametric and non-parametric baselines . We also demonstrate the generated sequences are useful as subgoals for actual physical execution in the animated world . Our paper makes three contributions . First , we proposed COmposable Semi-parametric MOtion generation ( COSMO ) , a method that can generate the required sequences in a semi-parametric , composed way . Second , we conducted experiments on CMU Mocap dataset 2 and SFU Mocap dataset 3 and shows our method outperforms strong baselines such as VAEs and GANs significantly . Third , we proved empirically that the generated sequences can serve as subgoals to learn an actual policy . 2 METHOD . 2.1 SEMI-PARAMETRIC MODELLING FOR MOTION GENERATION . Given starting and ending states ( denoted as s0 and sL respectively ) as inputs , our goal is to synthesize intermediate states that have natural transition and diverse behaviors . As illustrated in Fig 2 , NR reference subsequences ( denoted as Ri , i = 1 , .. , NR ) are sampled and fed into local motion composition 2http : //mocap.cs.cmu.edu/ 3http : //mocap.cs.sfu.ca/ sub-module φloc . Invalid subsequences are first filtered out with predefined requirements and please refer to appendix 6 for detailed description . φloc is used for altering the motion style of reference subsequence Ri based on another sequence R as input or directly sampling a style feature from learned latent space : R̂i = φloc ( Ri , R ) , where R is optional input . With this in hand , we compose edited subsequences { R̂i } NRi=1 along temporal direction smoothly . Concretely , we generate a new clip R̃i−1 : i = φglo ( R̂i−1 , R̂i ) which bridges the gap between R̂i−1 and R̂i . The final composed sequence is obtained through concatenation along the temporal direction for all subsequences and generated clips . Why semi-parametric modelling ? Regarding both motion category and transition dynamics ( from current to next state ) , human behaviors are intrinsically multi-modal distribution ( Holden et al. , 2017 ) . Parametric models ( e.g. , VAE ( Kingma & Welling , 2014 ) or GAN ( Goodfellow et al. , 2014 ) ) assume prior distribution to be uni-modal and predefined , which is not suitable to handle real world motion sequences ( Kingma & Welling , 2014 ; Arora & Zhang , 2017 ) . Non-parametric model does not require explicit prior distribution but not able to deal with long-range and highly diverse motion properly , mainly restricted by modelling capability . We also conduct targeted comparison experiments in Section 4 . Compared to parametric methods , semi-parametric modelling naturally guarantees motion diversity by introducing reference sequence also maintaining a large degree of freedom for generation . Moreover , semi-parametric modelling does not require access to or a model of data distribution . It implies that our method is also directly applicable for unseen motion sequence . Constrained by starting and ending states with unchangeable locations , it is not feasible to randomly sample reference subsequences for generation . To construct a complete motion sequence , we execute a manually designed procedure for searching reference subsequences . For detailed description , please refer to appendix 6 2.2 LOCAL MOTION COMPOSITION WITH CONTENT/STYLE DISENTANGLEMENT . Local motion composition stands for generating a relative short-range of motion clip based on reference subsequences as inputs . Content and style features are learned jointly , whose free-form composition is used for synthesising new motion sequence . We first describe the motion style modelling part . Motion Style Modelling . Style is considered as motion information which stays the same throughout the whole subsequence . To this end , we extract style pattern by conducting feature fusion along the temporal direction . More specifically , for a sequence with T frames , we gradually fuse all frames into one constant feature with Cs channels ( denoted as hs ) . We adopt 1D convolution along the temporal direction for each operation . Three convolution layers are stacked together with a kernel size of ( T/2 , T/4 , T/8 ) respectively , where large kernel size facilitates capturing motion style throughout the whole subsequence . To get better generalization ability for unseen reference or starting/ending states , we propose to construct an embedding space for style pattern modelling . More specifically , a set of embedding vectors ( denoted as Hs = { his } Mi=1 ) are learned together with above convolution layers ( denoted as ψsty ) . The corresponding outputs ( denoted as As ∈ RM ) of ψsty are a family of coefficients for a weighted sum of Hs . This is mainly inspired by the intuition that complex human activity could be considered as a combination of several simple actions . When trained with sufficiently large motion data , the model tends to learn basic style patterns for human motion . Compared to directly outputting a feature vector , this helps better generalize to unseen motion during testing . The final style feature ( denoted as fs ∈ RCs ) is generated as follows : f is = ψsty ( R i ) Hs . ( 2.1 ) Motion Content Modelling . In this part , we focus on motion dynamics reflected by each single state . To this end , 1D convolution layer with kernel size of 3 , i.e. , covering consecutive 3 states , is adopted for content modelling . The output channel of this layer is designed less than original motion dimension for extracting the most relevant information . This is empirically set to 5 as a design choice . Meanwhile , velocity , position for root joint and step pattern for foot joint are fed as inputs . We denote all the above factors as fc with T steps and Cc channels . During local motion composition , style feature fs is combined with content feature fc through concatenating along the channel axis : f ic = ψcnt ( R i ) , R̂i = ψrec ( f i c , f i s ) , ( 2.2 ) where fs is tiled T times to match with fc and ψrec outputs reconstructed R̂i . ψrec follows stacked three layer 1D convolution operation with kernel size of 3 . Finally , reconstruction loss is used to learn the style and content feature jointly as follows : LRrec = 1 NR ΣNRi=1||R̂ i −Ri||22 . ( 2.3 ) 2.3 GLOBAL MOTION COMPOSITION VIA GOAL CONDITIONED BI-DIRECTIONAL MODELLING . Global motion composition is connecting edited short-range subsequences into a completed and long-range one . Generating smooth and natural transition between R̂i and R̂i+1 is critical to obtain a long-range sequence . The major challenge comes from large states variation between R̂i and R̂i+1 . Inspired by recent work of Bi-LSTM modelling ( Ma & Hovy , 2016 ) , bi-directional motion composition is utilized for this part . More specifically , motion states are first mapped to a higher dimensional space with ϕenc and ϕdec , which is defined as follows : f it = ϕenc ( r i t ) , r̂ i t = ϕdec ( f i t ) . ( 2.4 ) Both ϕenc and ϕdec are 1D convolution layers . Then we predict the possible states at both directions with f it , where higher dimension ( 256 ) provides a sparser space enabling better composition results . For forward prediction , we take f iT−4 : T as inputs to obtain forward states f i : i+1 1 . Meanwhile , f i+1 1:5 are fed as inputs treated as goal conditions . We conduct this procedure in a recurrent manner with P -step prediction in total . Predicted f i : i+11 is concatenated with last four states forming next step inputs . Backward direction takes exactly the reverse procedure , i.e. , f i+1T−4 : T are treated as inputs while f i 1:5 are as goal : f i : i+1p+1 = ϕfpre ( f i p−4 : p , f i+1 1:5 ) , f i+1 : i p+1 = ϕbpre ( f i+1 p−4 : p , f i T−4 : T ) , ( 2.5 ) where ϕfpre and ϕbpre share the same architecture .. Moreover , we utilize another model to produce coefficients for the weighted sum of outputs . In this way , we are able to construct more flexible latent space to facilitate composition : c = ϕcom ( f i : i+1 p , f i+1 : i P−p , p ) , f̂p = cf i : i+1 p + ( 1− c ) f i+1 : iP−p . ( 2.6 ) ϕcom is one layer 1D convolution with sigmoid function for producing c. More specifically we introduce p and linearly map it to p̂ ∈ ( −1 , 1 ) and c = p̂+ 0.1 ∗ σ ( conv ( [ f i : i+1p , f i+1 : iP−p ] ) ) . During training , we randomly select a motion clip with a length of 10 + P . First as well as last 5 states are fed as inputs to get intermediate P states . Reconstruction loss is used for training : Lfrec = 1 P ΣPt=1||r̂t − ϕdec ( f̂ t ) ||22 . ( 2.7 ) Compared with goal-conditioned single forward prediction model , the starting and ending states are treated equally in our model , which avoids drifting away from the ending state . Meanwhile , we provide corresponding empirical results in Section 4 . After global motion composition , final composed sequence is directly obtained by concatenating all edited subsequences ( local part ) as well as transition clips ( global part ) along the temporal direction . We use tensorflow ( Abadi et al. , 2015 ) to implement all our models . All activation function used in our model is ReLU operation . During training , learning rate is set to 1e-4 and optimized with Adam optimizer ( Kingma & Ba , 2015 ) . β1 and β2 are set as 0.9 and 0.999 respectively . All models are trained with 30 epochs in total . Note that the local and global motion composition submodule do not need joint training . They are used jointly during testing .

Summary: The paper proposed ''composable semi-parametric modeling'' for generating long-range diverse and distinctive behaviors to achieve a specific goal location. The non-parametric part is a memory bank that is used to retrieve motion patterns from source materials. The parametric part contains several deep neural networks which are to compose the retrieved materials for high quality and smooth motion generation. The overall idea is novel in the sense that they aim to combine the strength of the non-parametric method (with rich pattern and diversities) and the parametric method (powerfull ability to generate coherent results). The proposed ideas are evaluated on two datasets and outperform compared approaches qualitatively.