MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Prediction

1 Introduction . Human mobility prediction plays an important role in a wide range of applications , including personalised advertisement , traffic modelling , and predicting the spread of pandemics [ 14 , 17 ] . For example , targeted advertisements , such as store information and coupons can be delivered precisely by predicting the future POIs that users might visit . It could also be valuable in the context of pandemics , to support contact tracing and crowd management . Unlike other densely acquired trajectory sequence data ( such as trajectories extracted from videos [ 1 , 20 ] and GPS trajectories [ 21 , 29 ] ) , trajectory data for the next POI prediction task is typically in the form of check-in sequences and collected from popular Location Based Social Networks ( LBSNs ) , such as Foursquare . As the visited POI information is recorded only when a user chooses to check-in 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . and accesses the location service , the check-in sequence data is very sparse . Due to privacy concerns or battery limitations on mobile devices , users rarely report their locations all the time . This inherent sparsity increases the difficulty of the prediction of future POIs dramatically . In the last few years , given major advances in deep learning techniques and especially the success of applying Recurrent Neural Networks ( RNNs ) for sequence prediction tasks , many research works have leveraged the user ’ s historical trajectories to predict most likely POI to be visited by that user . A pioneering work proposed ST-RNNs [ 25 ] which employ both time-specific and distancespecific transition matrices in an RNN architecture to model time intervals and geographical distances between two consecutive check-in POIs in historical trajectories . DeepMove [ 9 ] incorporates a historical attention module to capture the periodicity of human mobility . Following this trend , more recently , VANext [ 11 ] and ACN [ 27 ] propose two more attention mechanisms to learn long term preferences for users from history trajectories . However , considering the data sparsity , only exploring the historical visiting information may not be enough . For instance , compared to the large number of POIs and the complexity of the nature of people ’ s preferences , the small amount of visited POIs in a user ’ s historical log is unlikely to reveal their entire range of preferences over all available POIs . Therefore , in addition to exploring the temporal context based on the user ’ s own past visiting history , the influence from other types of context could be valuable to achieve more accurate POI prediction . In particular , there are three important types of context that could help tackle the sparsity problem : ( 1 ) Semantic Context : Typically , in LBSNs , each POI belongs to a category with high level semantic meanings such as Shops and Education . This context is an indicator in predicting the next POI . For example , at lunchtime , people are more likely to visit Food POIs than other types of POIs . Leisure travellers will tend to check in at tourism POI attractions . ( 2 ) Social Context : Friends and family members often visit a POI together . A user may also check in a new POI recommended by friends . ( 3 ) Geographical Context : POIs that are close to each other might be visited by the same user due to their proximity . For example , a user may visit a shopping mall and then go to a nearby cinema . A detailed data-driven analysis of these contexts is given in Section 7.2 . However , one limitation in the existing literature is that the context has been typically modelled based on manually designed functions . For example , Sun et al . [ 30 ] noticed that the geographical distances between any two non-consecutive visited POIs are also important . A geo-nonlocal function is introduced into the RNN to measure the distance between each visited POI and the average location of all visited POIs . Similarly , Flashback [ 38 ] introduces a manually designed weighting function to assign weights based on both temporal and geographical distances for each time step . In this paper , we propose a novel method called MobTCast , for the next POI prediction task , which is explicitly designed to model and incorporate the influence of the aforementioned contexts in a data-driven manner . A Transformer [ 31 ] -based mobility feature extractor is a fundamental component in MobTCastto perform the main POI prediction task . It takes both the visited POI history and the semantic category information to learn semantic-aware mobility features . Using this mobility feature extractor , we extract not only the semantic-aware mobility features of a user but also the features of his/her neighbours . A self-attention mechanism is then applied to model the social influence from neighbours to the user . This aggregated feature is both semantic and social-aware . To include geographical context , the model includes an auxiliary task , which is different from previously proposed methods that depend on manually designed kernels . Considering that the focus of this auxiliary task is exploring the geographical information ( i.e. , geographical coordinates of visited POIs ) , it is designed as a trajectory forecasting task . The purpose is to predict a future location ( coordinate ) based on the previously visited locations . As a result of these two tasks ( main and auxiliary ) , there are two predicted coordinates available : the predicted location from the auxiliary trajectory forecasting task and the inferred location from the predicted POI given by the main task . Ideally , the Euclidean distance between these two locations should be as close as possible . To this end , a novel consistency loss function is proposed to reflect and embody this relationship in the training process of MobTCast . Thus , the main and the auxiliary tasks are connected not only at the feature level but also at the output level . We conduct extensive experiments on three publicly available real-world datasets . The results demonstrate MobTCastś competitive POI prediction performance and show the value of incorporating different types of context as well as the consistency loss function . In summary , the main contributions of our work are two-fold : ( i ) We address the next POI prediction problem by including four types of context : temporal , geographical , social and semantic . Moreover , we design an auxiliary trajectory forecasting task to predict the geographical coordinates of the next most likely location . To the best of our knowledge , this is the first time that auxiliary trajectory forecasting is incorporated in a POI prediction model , which provides a new perspective for understanding human mobility . ( ii ) We introduce a novel consistency loss function to connect the predicted POI ( from the main task ) and the predicted location ( from the auxiliary task ) . This consistency loss function helps further improve the POI prediction performance . 2 Related Work . Next POI Prediction One of the earliest works that utilised geographical , temporal , and social contexts to predict the next POI is [ 6 ] , which combines the periodic day-to-day mobility patterns with information from social ties , inferred from the friendship network . Since this seminal work , many others have proposed methods to better model the check-in behaviours or the trajectory of geographical movements , depending on the data they work on . Based on weight matrix factorisation , Cheng et al . [ 4 ] proposed a personalised Multi-centre Gaussian Model ( MGM ) to capture the geographical influence and Lian et al . [ 23 ] proposed GeoMF model that considers the geographical influence . Li et al . [ 22 ] developed a ranking based geographical factorisation method to incorporate the influences of geographical and temporal context . Recently , a local geographical model is introduced into the logistic matrix factorisation to improve the accuracy of POI recommendation [ 28 ] . GERec [ 32 ] employs a spatio-temporal relation path embedding to model the temporal , spatial and semantic content for the next POI recommendation . Deep learning-based neural networks have been used for temporal sequence modelling . RNNs ( including LSTM [ 15 ] and GRU [ 7 ] ) and Transformer [ 31 ] have also been proposed and successfully applied to many time series prediction problems such as pedestrian trajectory prediction [ 1 , 36 ] and traffic prediction [ 34 ] . For the next POI prediction task , ST-RNN ( Spatial Temporal Recurrent Neural Networks ) , proposed by Liu et al . [ 25 ] , improves the original RNN architecture by incorporating temporal and spatial context at each time interval in the historical trajectory . Following the trend of using RNN , a series of methods are proposed to model human mobility behaviours and predict the next POI , such as DeepMove [ 9 ] . Based on RNNs , it applies an attention mechanism to capture the periodical features of human mobility . Similarly , Gao et al . [ 11 ] introduced a variational attention mechanism to learn the attention on the historical trajectories . Efforts have also been made to incorporate different kinds of context for POI prediction . SERM [ 40 ] is designed to jointly learn embeddings of temporal and semantic contexts ( i.e. , text messages ) . Gao et al . [ 13 ] proposed to leverage a knowledge graph to accommodate the influence of users , locations and semantics . Flashback [ 38 ] tackles the geographical context through an explicitly designed spatio-temporal weighting function . The calculated weights are applied to the hidden states in an RNN correspondingly . STAN [ 26 ] incorporates the geographical context to learn the regularities between non-adjacent locations and non-contiguous visits . In these methods , the modelling of the geographical context heavily depends on manually designed kernel functions ( e.g. , Euclidean distance based functions ) or a pre-computed spatio-temporal relation matrix [ 26 ] ( which would dramatically increase the memory and computation costs when the total number of POIs is large ) . There is limited work that have simultaneously considered temporal , semantic , social and geographical contexts . Auxiliary Task Learning An auxiliary task is a task that is trained jointly with the main task during the training phase to enhance the main task ’ s performance . As a variant of multi-task learning , auxiliary learning has become popular in recent years . It is mainly used in the computer vision field such as semantic segmentation [ 5 , 35 ] . Recently , several methods have been proposed using multi-task learning to model human mobility . For example , Chen et al . [ 3 ] introduced DeepJMT which jointly predicts location and time . MCARNN , proposed by Liao et al . [ 24 ] , is a multi-task network to predict both activity and location . Compared to multi-task learning , in auxiliary task learning the auxiliary task is nonoperational during the inference phase . Note that in MCARNN [ 24 ] , the authors also proposed to leverage an auxiliary task to learn location embeddings . However , the auxiliary task in the proposed MobTCast approach is a trajectory forecasting task , which is designed to consider the geographical context directly , with the introduction of a consistency loss function to connect the main task and the auxiliary task . 3 Problem Formulation . Consider two sets : P = { poi1 , poi2 , · · · , poi|P | } and U = { u1 , u2 , · · · , u|U | } , denoting the set of POIs and the set of users , respectively . Each POI is represented by a ( lon , lat ) tuple , which indicates the geographical context . The human mobility prediction problem is defined as follows . Given the history mobility trajectory T i = ( pit1 , p i t2 , · · · , p i tn ) of user u i ∈ U , the goal is to predict the POI p̂itn+1 that is going to be visited by the same user at the next time t i n+1 . Here , p i tn ∈ P stands for the POI visited by user ui at time tn and the geographical coordinate of pitn is indicated by ( x i tn , y i tn ) . The observation length of the given history trajectory is n. Based on the geographical coordinates , the semantic category information citn can be acquired . Note that depending on the dataset , we utilise two types of POI category hierarchies given by : ArcGIS1 and Foursquare2 . The social context of user ui , is given by his/her neighbours , represented as N ( i ) = { uj1 , uj2 , · · · , ujm } , which is a group of users ( in U ) socially connected with user ui .

Summary: This paper proposed a novel deep learning-based architecture for the next POI prediction. The proposed approach models the temporal and semantic contexts through Transformer, extracts the social context through self-attention module, and model the geographical context via an auxiliary task. Experiments show that it can achieve better results than the SOTA next POI prediction.