Exploring Social Posterior Collapse in Variational Autoencoder for Interaction Modeling

1 Introduction . Modeling the behavior of intelligent agents is an essential subject for autonomous systems . Safe operations of autonomous agents require accurate prediction of other agents ’ future motions . In particular , social interaction among agents should be modeled , so that downstream planning and decision-making modules can safely navigate the robots through interactive scenarios . Generative latent variable models are popular modeling options for their ability to generate diverse and naturalistic behavior [ 1 , 2 , 3 , 4 ] . In this work , we focus on one category of generative models , Variational Autoencoder ( VAE ) [ 5 ] , which has been widely used in multi-agent behavior modeling and trajectory prediction [ 1 , 4 , 6 , 7 , 8 , 9 ] . It is desirable as it learns a low-dimensional representation of the original high-dimensional data . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) However , VAEs do not necessarily learn a good representation of the data [ 10 ] . For instance , prior works in sequence modeling have found out that the model tends to ignore the latent variables if the decoder is overly powerful [ 11 , 12 , 13 ] ( e.g. , an autoregressive decoder ) . It leads us to wonder whether a VAE-based model can always learn a good representation for a multi-agent interacting system . It is a rather general question as researchers may look for different properties of the latent space , for instance , interpretability [ 14 , 15 ] and multi-modality [ 4 , 6 ] . In this work , we focus on a fundamental aspect of this general question : Does the latent space always properly model interaction ? Formally , given a latent variable model of an interacting system , where a latent variable governs each agent ’ s behavior , we wonder if the VAE learns to encode social context into the latent variables . It raises such concern since VAE handles two distinct tasks in training and testing . At the training stage , it learns to reconstruct a datum instead of generating one . For multi-agent interaction modeling , the sample for reconstruction is a set of trajectories of all the agents . Ideally , the model should learn to model the interaction among agents and jointly reconstruct the trajectories . However , there is no mechanism to prevent the model from separately reconstructing the trajectories . In fact , it could even be more efficient at the early stage of training when the model has not learned an informative embedding of social context . We then end up with a model that models each agent ’ s behavior separately without social context , which might suffer from over-estimated variance and large prediction error . More importantly , since the joint behavior of the agents is a consequence of their interactions , ignoring the causes may lead to poor generalization ability [ 15 , 16 ] . We find that a typical formulation of VAE for multi-agent interaction is indeed prone to ignoring historical social context ( i.e. , interactions over the observed time horizon ) . We refer to this phenomenon as social posterior collapse . The issue has never been discussed in the literature . Considering those potential defects , we think it is necessary to study such a crucial and fundamental issue . In this work , we first abstract the VAE formulation from a wide range of existing literature [ 1 , 4 , 6 , 8 ] . Then we analyze social posterior collapse under this formulation and propose several measures to alleviate the issue . Afterward , we study the issue under real-world settings with a realization of the abstract formulation we design . In particular , we propose a novel sparse graph attention message-passing ( sparse-GAMP ) layer and incorporate it into the model , which helps us detect and analyze social posterior collapse . Our experiments show that social posterior collapse occurs in real-world prediction tasks and that the proposed measures can effectively alleviate the issue . We also evaluate how social posterior collapse affects the model performance on these tasks . The results suggest that the model without social posterior collapse can attain better generalization performance if historical social context is informative for prediction . 2 Social Posterior Collapse in Variational Autoencoder Interaction Models . 2.1 A Latent Variable Model for Interaction Modeling . Given an interacting system with n agents , an interaction-aware trajectory prediction model takes all the agents ’ observed historical trajectories , denoted by { xi } ni=1 , as input and predicts the future trajectories of all the agents or a subset of enquired agents . We denote the collection of future trajectories by { yi } ni=1 . In this work , we focus on the latent variable model illustrated in Fig . 1 , which is abstracted from existing literature on VAEs for multi-agent behavior modeling [ 1 , 4 , 6 , 8 ] . We model interaction by introducing a set of variables { Ti } ni=1 , which aggregates each agent ’ s state and its observation of other agents . Formally , each Ti is modeled as a deterministic function of { xi } ni=1 , i.e. , Ti = fi ( { xi } ni=1 ) . Afterward , the agents make decisions based on the aggregated information over the predicted horizon . Latent variables { zi } ni=1 are introduced to model the inherent uncertainty in each agent ’ s behavior . It should be noticed that interaction over the predicted horizon is not modeled explicitly in this formulation . Although it can be achieved by exchanging information between agents recurrently ( e.g. , social pooling in [ 17 ] ) , it is a common practice to avoid explicit modeling of future interaction in consideration of computational cost and memory [ 3 , 8 , 18 ] . In this work , we train the model as a VAE , where an encoder q ( z|x , y ) is introduced to approximate the posterior distribution p ( z|x , y ) for efficient sampling at the training stage1 . The performance of variational inference is optimized if the KL-divergence between the posteriors , i.e . DKL [ q ( z|x , y ) ‖p ( z|x , y ) ] , is small [ 10 ] . To derive a better approximation , we can incorporate 1The vectors x , y and z collect the corresponding variables for all n agents . inductive bias based on the characteristics of the true posterior into the encoder function . We introduce the following simple proposition that guides our model design . Its proof can be found in Appx . A . Proposition 1 . For any i = 1 , 2 , ... , n and j = 1 , 2 , ... , n , 1 ) If j 6= i , zi and yj are conditionally independent given x and yi ; 2 ) If j 6= i , zi and zj are conditionally independent given x ; 3 ) zi and xj are conditionally independent given Ti . Following the proposition , we decompose the posterior distribution as ∏n i=1 p ( zi|Ti , yi ) . The decomposition suggests two insights . First , the encoder does not need to aggregate future context information when inferring the posterior distribution of zi for each agent . Second , the historical context variable Ti is all we need for the historical information of the agent i . The encoder and decoder can share the same function to encode historical information . 2.2 Social Posterior Collapse . We then design a VAE model reflecting the characteristics of the true posterior discovered in Sec . 2.1 . To simplify the problem , we further make the assumption of homogeneous agents . The model has three basic building blocks : 1 ) A function modeling the historical context , i.e. , Ti = fθ ( xi , x ) ; 2 ) A function decoding the distribution of the future trajectory yi given Ti and zi , i.e. , pφ ( yi|Ti , zi ) ; 3 ) A function approximating the posterior of zi conditioned on Ti and yi , i.e. , qψ ( zi|Ti , yi ) . They build up the encoder and decoder of the VAE model as follows : qθ , ψ ( z|x , y ) = n∏ i=1 qψ ( zi|fθ ( xi , x ) , yi ) , pθ , φ ( y|x , z ) = n∏ i=1 pφ ( yi|fθ ( xi , x ) , zi ) . ( 1 ) We consider a continuous latent space and model qψ and pφ as diagonal Gaussian distributions . The model is trained by maximizing the evidence lower bound ( ELBO ) : max θ , ψ , φ Ex , y∼D [ Ez∼qθ , ψ ( z|x , y ) [ log pθ , φ ( y|x , z ) ] − βDKL [ qθ , ψ ( z|x , y ) ‖p ( z ) ] ] . ( 2 ) However , we find a critical issue of this naive formulation during experiments , which is the social posterior collapse phenomenon mentioned before . With the help of the sparse graph attention mechanism introduced in Sec . 3 , we find that the model is prone to ignoring historical social context when reconstructing the future trajectory of one agent . Equivalently , the generative model collapses to the one with all the variables of other agents marginalized out . We think the reason behind it is similar to , but different from , the well-known phenomenon in VAE training—posterior collapse [ 10 ] . Due to the KL regularization term in ELBO , the variational posterior distribution is likely to collapse towards the prior . It is particularly likely to occur at the early stage of training when the latent space is still uninformative [ 19 ] . In our case , the posterior of zi collapses into qθ , ψ ( zi|xi , yi ) . It does minimize the KL-divergence : if both qθ , ψ ( zi|xi , yi ) and qθ , ψ ( zi|x , yi ) exactly approximate the true posteriors , then conditioning on more context information increases the expected value of KL regularization : ED [ DKL [ p ( zi|x , yi ) ‖p ( zi ) ] ] = I ( zi ; x , yi ) > I ( zi ; xi , yi ) = ED [ DKL [ p ( zi|xi , yi ) ‖p ( zi ) ] ] . However , we argue that an additional factor contributes to the social posterior collapse problem , which makes it a unique phenomenon for interaction modeling . At the training stage , the goal is to reconstruct the future trajectories y . The trajectory itself contains all the information needed for reconstruction . The history of the agent i provides complementary information such as the current state and static environmental information . There is no explicit regulation in the current framework to prevent the model from extracting information solely from xi and yi . In fact , it is a more efficient coding scheme when the model has not learned an informative representation of interaction . Techniques such as KL annealing [ 13 , 19 ] rely on various scheduling schemes of β to prevent KL vanishing , which has been shown effective in mitigating the posterior collapse problem . However , reducing β could merely privilege the model to gather more information from yi , which is consistent with the reconstruction objective . Therefore , we need to explore alternative solutions to tackle the social posterior collapse problem deliberately . We start with changing the model into a conditional generative one [ 20 , 21 ] . Additional edges { Ti → zi } ni=1 are added into the original graph , which are annotated with dash lines in Fig . 1 . We follow the practice in [ 20 ] and formulate the model as a Conditional Variational Autoencoder ( CVAE ) . It is straightforward to verify that the conclusion of Proposition 2.1 still applies . We still model the encoder and decoder as in Eqn . ( 1 ) . The CVAE framework introduces an additional module —a function approximating the conditional prior pη ( zi|Ti ) , which becomes pθ , η ( zi|x ) after incorporating fθ ( x ) . The objective then becomes : L ( θ , ψ , φ , η ) = Ex , y∼D [ Ez∼qθ , ψ ( z|x , y ) [ log pθ , φ ( y|x , z ) ] − βDKL [ qθ , ψ ( z|x , y ) ‖pθ , η ( z|x ) ] ] . Compared to Eqn . ( 2 ) , we no longer penalize the encoder for aggregating context information but only restrict the information encoded from future trajectories . Therefore , the encoder does not need to ignore context information to fulfill the information bottleneck . However , the model still lacks a mechanism to encourage context information encoding deliberately . To achieve the goal , we propose to incorporate an auxiliary prediction task into the training scheme . Concretely , we introduce another module yi = gζ ( Ti , zi ) . Composing gζ with fθ gives us another decoder y = hθ , ζ ( x , z ) . The difference is that the latent variables are always sampled from pη ( zi|Ti ) . The auxiliary task is training this trajectory decoder which shares the same context encoder and conditional prior with the CVAE . Because the auxiliary model does not have access to the ground-truth future trajectory , it needs to utilize context information for accurate prediction . Consequently , it encourages the model to encode context information into T and z . We use mean squared error ( MSE ) loss as the objective function . The overall objective minimizes a weighted sum of the two objective functions . The training scheme looks similar to the one in [ 20 ] , where they trained a Gaussian stochastic neural network ( GSNN ) together with the CVAE model . However , ours is different from theirs in some major aspects . The GSNN model shares the same decoder with the CVAE model . The primary motivation of incorporating another learning task is to optimize the generation procedure during training directly . In contrast , our auxiliary prediction model has a separate trajectory decoder . It is because we do not want the auxiliary task to interfere with the decoding procedure of the CVAE model . We find that sharing the decoder leads to less diversity in trajectory generation , which is not desirable .

Summary: This paper analyzes the reason behind the social posterior collapse caused in typical VAEs for multi-agent modeling and proposes improvement schemes. The paper test the proposed model on real-world multi-agent trajectory prediction datasets. This paper also proposes a novel sparse graph attention message-passing.