Customizing Sequence Generation with Multi-Task Dynamical Systems

1 INTRODUCTION . Time series data often arise as a related ‘ family ’ of sequences , where certain characteristic differences exist between the sequences in a dataset . Examples include the style of handwritten text ( Graves , 2013 ) , the response of a patient to an anaesthetic ( Bird et al. , 2019 ) , or the style of locomotion in motion capture ( mocap ) data ( Ghosh et al. , 2017 ) . In this paper , we will consider how such variation may be modelled , and effectively controlled by an end user . Such related data is often pooled to train a single dynamical system , despite the internal variation . For a simple model , such as a linear dynamical system ( LDS ) , this will result in learning only an average effect . In contrast , a recurrent neural network ( RNN ) may model this variation , but in an implicit and opaque manner . Such a ‘ black-box ’ approach prohibits end-user control , and may suffer from mode drift , such as in Ghosh et al . ( 2017 ) , where a generated mocap sequence performs an unprompted transition from walking to drinking . Some of these problems may be alleviated by appending ‘ context labels ’ to the inputs ( see e.g . Goodfellow et al. , 2016 , §10.2.4 ) which describe the required customization . However , such labels are often unavailable , and the approach may fail to model the variation adequately even when they are . To move beyond these approaches , we consider latent variable models , where a latent variable z characterizes each sequence . This may be seen as a form of multi-task learning ( MTL , see Zhang & Yang , 2017 ) , from which we derive the name multi-task dynamical system ( MTDS ) , with each sequence treated as a task . A straightforward approach is to append the latent z to the inputs of the model , similarly to the ‘ context label ’ approach , thereby providing customization of the various bias ( or offset ) parameters of the model . A number of examples of this have been proposed recently , e.g . in Yingzhen & Mandt ( 2018 ) and Miladinović et al . ( 2019 ) . Nevertheless , this ‘ bias customization ’ has limited expressiveness and is often unsuitable for customizing simple models . In this paper we investigate a more powerful form of customization which modulates all the system and emission parameters . In this approach , the parameters of each task are constrained to lie on a learned low dimensional manifold , indexed by the latent z . Our experiments show that this approach results in improved performance and/or greater data efficiency than existing approaches , as well as greater robustness to unfamiliar test inputs . Further , varying z can generate a continuum of models , allowing interpolation between sequence predictions ( see Figure 1b for an example ) , and potentially morphing of sequence characteristics over time . Contributions In this paper we propose the MTDS , which goes beyond existing work by allowing full adaptation of all parameters of general dynamical systems via use of a learned nonlinear manifold . We show how the approach may be applied to various popular models , and provide general purpose learning and inference algorithms . Our experimental studies use synthetic data ( sum of two damped harmonic oscillators ) and real-world human locomotion mocap data . We illuminate various properties of the MTDS formulation in our experiments , such as data efficiency , user control , and robustness to dataset shift , and show how these go beyond existing approaches to time series modelling . We finally utilize the increased user control in the context of mocap data to demonstrate style morphing . To this end , we introduce the model in Section 2 , giving examples and discussing the particular challenges in learning and inference . We discuss the relation to existing work in Section 3 . Experimental setup and results are given in Section 4 with a conclusion in Section 5 . 2 MULTI-TASK DYNAMICAL SYSTEMS . Consider a collection of input-output sequences D = { Y ( i ) , U ( i ) } Ni=1 with inputs U ( i ) = { u ( i ) 1 , . . . , u ( i ) Ti } and outputs Y ( i ) = { y ( i ) 1 , . . . , y ( i ) Ti } , i = 1 , . . . , N , where Ti denotes the length of sequence i . Each sequence i is described by a different dynamical system , whose parameter θ ( i ) depends on the hierarchical latent variable z ( i ) ∈ Z : θ ( i ) = hφ ( z ( i ) ) , z ( i ) ∼ p ( z ) ( 1 ) x ( i ) t ∼ p ( x | x ( i ) t−1 , u ( i ) t , θ ( i ) ) , ( 2 ) y ( i ) t ∼ p ( y | x ( i ) t , u ( i ) t , θ ( i ) ) , ( 3 ) for t = 1 , . . . , Ti . The state variables X ( i ) = { x ( i ) 1 , . . . , x ( i ) Ti } , xt ∈ X follow the latent dynamics ( 2 ) starting from x0 : = 0 ( other choices of initial state are possible ) . See Figure 1a for a graphical model . In this paper we assume Z = Rk which the vector-valued function hφ ( · ) transforms to conformable model parameters θ ∈ Rd , d k. Note that hφ may keep some dimensions of θ constant with respect to z . We call this a Multi-Task Dynamical System , going beyond the usage in Bird et al . ( 2019 ) . An MTDS model under this framework must specify three key quantities : 1 . The base model ( e.g . a LDS or RNN ) . 2 . The nature of the interaction between the latent variable z and the parameter vectors θ ( e.g . if only the dynamics ( eq . 2 ) depend on θ ) . 3 . The choice of prior p ( z ) and transformation hφ . Where the use of a constant z over time is inappropriate , each sequence can be broken into segments of maximum length L. When L is small enough , this effectively permits a time-varying z , as we will see in section 4.2 . 2.1 EXAMPLES . In order to make the framework more concrete we will describe two general choices of the base model . In what follows we will write each parameter with a subscript z to denote dependence on z ( e.g . Az : = A ( z ) ) to reduce notational clutter . The choice of p ( z ) and hφ will depend on the application , but a fairly general choice is a deep latent Gaussian model ( Kingma & Welling , 2014 ; Rezende et al. , 2014 ) . See section A.1.1 in the supplementary material for further discussion . 2.1.1 MULTI-TASK LINEAR DYNAMICAL SYSTEM . For a given z , a multi-task linear dynamical system ( MTLDS ) can be described by : xt = Azxt−1 +Bzut + bz + wt , ( 4 ) yt = Czxt +Dzut + dz + t , ( 5 ) wt ∼ N ( 0 , Rz ) , t ∼ N ( 0 , Sz ) , with θz = { Az , Bz , bz , Cz , Dz , dz , Rz , Sz } = hφ ( z ) . The parameterization of θz must satisfy the constraints of positive definite Rz and Sz and stable Az ( i.e . ‖Az‖2 ≤ 1 ) for all z , hence projection methods such as in Siddiqi et al . ( 2008 ) are not applicable . We choose an alternative formulation of the LDS , replacing the latent dynamics in eq . ( 4 ) by : xt = ΣzQzxt−1 +Bzut + bz + wt , ( 6 ) where Σz is a diagonal matrix and Qz orthogonal with no loss of generality ( proof in supp . mat. ) . Since ‖ΣzQz‖ ≤ ‖Σz‖‖Qz‖ = ‖Σz‖ , stability can be enforced e.g . by Σ = diag { tanh ( υ ) } for some vector υ . For more details see section A.1.2 in the supplementary material . 2.1.2 MULTI-TASK RECURRENT NEURAL NETWORK . Due to the nonlinearity of an RNN , enforcing stability of Az is not strictly required ( see e.g . Miller & Hardt , 2019 , §4.4 ) , although bounding the spectral radius may be useful for learning ( e.g . Pascanu et al. , 2013 ) . The dynamics of a multi-task RNN ( MT-RNN ) are described by : xt+1 = tanh ( Azxt +Bzut + bz ) . ( 7 ) Combined with the emission model of eq . ( 5 ) , we have θz = { Az , Bz , bz , Cz , Dz , dz } . If long-term dependencies are important we may consider an orthogonal transition matrix ( parameterized as for the MTLDS ) to create a multi-task version of the Orthogonal RNN ( ORNN , Helfrich et al. , 2018 ) . 2.2 LEARNING . The parameters φ of an MTDS can be learned from a dataset D : = { Y ( i ) , U ( i ) } Ni=1 via maximum marginal likelihood : φ∗ = arg maxφ ∑N i=1 log p ( Y ( i ) |U ( i ) , φ ) , where log p ( Y |U , φ ) = log ∫ Z p ( Y |U , hφ ( z ) ) p ( z ) dz . ( 8 ) The first term in the integrand , p ( Y |U , hφ ( z ) ) = ∫ XT p ( Y |X , U , hφ ( z ) ) p ( X|U , hφ ( z ) ) dX is generally intractable for stochastic dynamics ( with notable exceptions of discrete and linear-Gaussian models ) . A common approach is to use a variational evidence lower bound ( ELBO ) see e.g . Fraccaro et al . ( 2016 ) ; Goyal et al . ( 2017 ) ; Miladinović et al . ( 2019 ) or a Monte Carlo objective ( MCO ) e.g . Maddison et al . ( 2017 ) ; Le et al . ( 2018 ) ; Naesseth et al . ( 2018 ) . For clarity of exposition we only consider models with deterministic state , which extends to the MTRNN and MTLDS above ( in the case wt = 0 for all t ) . This also avoids the interaction effect with the choice of approximate marginalization over X . Equation ( 8 ) also can not be computed in closed form in general , and so we resort to approximate learning . A natural choice is via the ELBO ( see an alternative MCO approach for unsupervised tasks in Section A.1.3 , supp . mat. ) . We write the ELBO of eq . ( 8 ) as : L ( Y ; φ , λ ) = Eqλ ( z|Y , U ) [ log p ( Y |U , hφ ( z ) ) ] −DKL ( qλ ( z|Y , U ) ‖p ( z ) ) , ( 9 ) where DKL is the Kullback-Leibler divergence and qλ ( z|Y , U ) an approximate posterior for z . We can now optimize a lower bound of the marginal likelihood via arg maxφ , λ ∑N i=1 L ( Y ( i ) ; φ , λ ) , where low variance unbiased gradients of eq . ( 9 ) are available via reparameterization ( Kingma & Welling , 2014 ; Rezende et al. , 2014 ) and minibatches of size Nbatch < N . Optimization via Adam ( Kingma & Ba , 2014 ) proved adequate in our experiments without resorting to any specialized RNN optimization tricks . A standard choice of qλ ( z|Y , U ) is N ( µλ ( Y , U ) , sλ ( Y , U ) ) , where µλ , sλ are inference networks ( e.g . Fabius & van Amersfoort , 2015 ) . It can be difficult to learn a sensible latent representation if the base model is a powerful RNN . When each output can be identified unambiguously via the inputs preceding it , a larger ELBO can be obtained by the RNN learning the relationship without using the latent variable ( see e.g . Chen et al. , 2017 ) . A useful heuristic for avoiding such optima is KL annealing ( e.g . Bowman et al. , 2016 ) . In our experiments we perform an initial optimization without the KL penalty ( second term in eq . 9 ) , initializing sλ ( Y , U ) to a small constant value .

Summary: This paper proposes a multi-task dynamical system for sequence generation. The model learns a number of parameters that represents the latent code z. The learned model can generate the customized individual data sequence and provide the smooth interpolation in the sequence space. The experiments on the synthetic data and the ocap dataset show the customization is beneficial for prediction tasks and enables for style transfer and morphing within generated sequences.