MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Prediction

1 Introduction . Human mobility prediction plays an important role in a wide range of applications , including personalised advertisement , traffic modelling , and predicting the spread of pandemics [ 14 , 17 ] . For example , targeted advertisements , such as store information and coupons can be delivered precisely by predicting the future POIs that users might visit . It could also be valuable in the context of pandemics , to support contact tracing and crowd management . Unlike other densely acquired trajectory sequence data ( such as trajectories extracted from videos [ 1 , 20 ] and GPS trajectories [ 21 , 29 ] ) , trajectory data for the next POI prediction task is typically in the form of check-in sequences and collected from popular Location Based Social Networks ( LBSNs ) , such as Foursquare . As the visited POI information is recorded only when a user chooses to check-in 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . and accesses the location service , the check-in sequence data is very sparse . Due to privacy concerns or battery limitations on mobile devices , users rarely report their locations all the time . This inherent sparsity increases the difficulty of the prediction of future POIs dramatically . In the last few years , given major advances in deep learning techniques and especially the success of applying Recurrent Neural Networks ( RNNs ) for sequence prediction tasks , many research works have leveraged the user ’ s historical trajectories to predict most likely POI to be visited by that user . A pioneering work proposed ST-RNNs [ 25 ] which employ both time-specific and distancespecific transition matrices in an RNN architecture to model time intervals and geographical distances between two consecutive check-in POIs in historical trajectories . DeepMove [ 9 ] incorporates a historical attention module to capture the periodicity of human mobility . Following this trend , more recently , VANext [ 11 ] and ACN [ 27 ] propose two more attention mechanisms to learn long term preferences for users from history trajectories . However , considering the data sparsity , only exploring the historical visiting information may not be enough . For instance , compared to the large number of POIs and the complexity of the nature of people ’ s preferences , the small amount of visited POIs in a user ’ s historical log is unlikely to reveal their entire range of preferences over all available POIs . Therefore , in addition to exploring the temporal context based on the user ’ s own past visiting history , the influence from other types of context could be valuable to achieve more accurate POI prediction . In particular , there are three important types of context that could help tackle the sparsity problem : ( 1 ) Semantic Context : Typically , in LBSNs , each POI belongs to a category with high level semantic meanings such as Shops and Education . This context is an indicator in predicting the next POI . For example , at lunchtime , people are more likely to visit Food POIs than other types of POIs . Leisure travellers will tend to check in at tourism POI attractions . ( 2 ) Social Context : Friends and family members often visit a POI together . A user may also check in a new POI recommended by friends . ( 3 ) Geographical Context : POIs that are close to each other might be visited by the same user due to their proximity . For example , a user may visit a shopping mall and then go to a nearby cinema . A detailed data-driven analysis of these contexts is given in Section 7.2 . However , one limitation in the existing literature is that the context has been typically modelled based on manually designed functions . For example , Sun et al . [ 30 ] noticed that the geographical distances between any two non-consecutive visited POIs are also important . A geo-nonlocal function is introduced into the RNN to measure the distance between each visited POI and the average location of all visited POIs . Similarly , Flashback [ 38 ] introduces a manually designed weighting function to assign weights based on both temporal and geographical distances for each time step . In this paper , we propose a novel method called MobTCast , for the next POI prediction task , which is explicitly designed to model and incorporate the influence of the aforementioned contexts in a data-driven manner . A Transformer [ 31 ] -based mobility feature extractor is a fundamental component in MobTCastto perform the main POI prediction task . It takes both the visited POI history and the semantic category information to learn semantic-aware mobility features . Using this mobility feature extractor , we extract not only the semantic-aware mobility features of a user but also the features of his/her neighbours . A self-attention mechanism is then applied to model the social influence from neighbours to the user . This aggregated feature is both semantic and social-aware . To include geographical context , the model includes an auxiliary task , which is different from previously proposed methods that depend on manually designed kernels . Considering that the focus of this auxiliary task is exploring the geographical information ( i.e. , geographical coordinates of visited POIs ) , it is designed as a trajectory forecasting task . The purpose is to predict a future location ( coordinate ) based on the previously visited locations . As a result of these two tasks ( main and auxiliary ) , there are two predicted coordinates available : the predicted location from the auxiliary trajectory forecasting task and the inferred location from the predicted POI given by the main task . Ideally , the Euclidean distance between these two locations should be as close as possible . To this end , a novel consistency loss function is proposed to reflect and embody this relationship in the training process of MobTCast . Thus , the main and the auxiliary tasks are connected not only at the feature level but also at the output level . We conduct extensive experiments on three publicly available real-world datasets . The results demonstrate MobTCastś competitive POI prediction performance and show the value of incorporating different types of context as well as the consistency loss function . In summary , the main contributions of our work are two-fold : ( i ) We address the next POI prediction problem by including four types of context : temporal , geographical , social and semantic . Moreover , we design an auxiliary trajectory forecasting task to predict the geographical coordinates of the next most likely location . To the best of our knowledge , this is the first time that auxiliary trajectory forecasting is incorporated in a POI prediction model , which provides a new perspective for understanding human mobility . ( ii ) We introduce a novel consistency loss function to connect the predicted POI ( from the main task ) and the predicted location ( from the auxiliary task ) . This consistency loss function helps further improve the POI prediction performance . 2 Related Work . Next POI Prediction One of the earliest works that utilised geographical , temporal , and social contexts to predict the next POI is [ 6 ] , which combines the periodic day-to-day mobility patterns with information from social ties , inferred from the friendship network . Since this seminal work , many others have proposed methods to better model the check-in behaviours or the trajectory of geographical movements , depending on the data they work on . Based on weight matrix factorisation , Cheng et al . [ 4 ] proposed a personalised Multi-centre Gaussian Model ( MGM ) to capture the geographical influence and Lian et al . [ 23 ] proposed GeoMF model that considers the geographical influence . Li et al . [ 22 ] developed a ranking based geographical factorisation method to incorporate the influences of geographical and temporal context . Recently , a local geographical model is introduced into the logistic matrix factorisation to improve the accuracy of POI recommendation [ 28 ] . GERec [ 32 ] employs a spatio-temporal relation path embedding to model the temporal , spatial and semantic content for the next POI recommendation . Deep learning-based neural networks have been used for temporal sequence modelling . RNNs ( including LSTM [ 15 ] and GRU [ 7 ] ) and Transformer [ 31 ] have also been proposed and successfully applied to many time series prediction problems such as pedestrian trajectory prediction [ 1 , 36 ] and traffic prediction [ 34 ] . For the next POI prediction task , ST-RNN ( Spatial Temporal Recurrent Neural Networks ) , proposed by Liu et al . [ 25 ] , improves the original RNN architecture by incorporating temporal and spatial context at each time interval in the historical trajectory . Following the trend of using RNN , a series of methods are proposed to model human mobility behaviours and predict the next POI , such as DeepMove [ 9 ] . Based on RNNs , it applies an attention mechanism to capture the periodical features of human mobility . Similarly , Gao et al . [ 11 ] introduced a variational attention mechanism to learn the attention on the historical trajectories . Efforts have also been made to incorporate different kinds of context for POI prediction . SERM [ 40 ] is designed to jointly learn embeddings of temporal and semantic contexts ( i.e. , text messages ) . Gao et al . [ 13 ] proposed to leverage a knowledge graph to accommodate the influence of users , locations and semantics . Flashback [ 38 ] tackles the geographical context through an explicitly designed spatio-temporal weighting function . The calculated weights are applied to the hidden states in an RNN correspondingly . STAN [ 26 ] incorporates the geographical context to learn the regularities between non-adjacent locations and non-contiguous visits . In these methods , the modelling of the geographical context heavily depends on manually designed kernel functions ( e.g. , Euclidean distance based functions ) or a pre-computed spatio-temporal relation matrix [ 26 ] ( which would dramatically increase the memory and computation costs when the total number of POIs is large ) . There is limited work that have simultaneously considered temporal , semantic , social and geographical contexts . Auxiliary Task Learning An auxiliary task is a task that is trained jointly with the main task during the training phase to enhance the main task ’ s performance . As a variant of multi-task learning , auxiliary learning has become popular in recent years . It is mainly used in the computer vision field such as semantic segmentation [ 5 , 35 ] . Recently , several methods have been proposed using multi-task learning to model human mobility . For example , Chen et al . [ 3 ] introduced DeepJMT which jointly predicts location and time . MCARNN , proposed by Liao et al . [ 24 ] , is a multi-task network to predict both activity and location . Compared to multi-task learning , in auxiliary task learning the auxiliary task is nonoperational during the inference phase . Note that in MCARNN [ 24 ] , the authors also proposed to leverage an auxiliary task to learn location embeddings . However , the auxiliary task in the proposed MobTCast approach is a trajectory forecasting task , which is designed to consider the geographical context directly , with the introduction of a consistency loss function to connect the main task and the auxiliary task . 3 Problem Formulation . Consider two sets : P = { poi1 , poi2 , · · · , poi|P | } and U = { u1 , u2 , · · · , u|U | } , denoting the set of POIs and the set of users , respectively . Each POI is represented by a ( lon , lat ) tuple , which indicates the geographical context . The human mobility prediction problem is defined as follows . Given the history mobility trajectory T i = ( pit1 , p i t2 , · · · , p i tn ) of user u i ∈ U , the goal is to predict the POI p̂itn+1 that is going to be visited by the same user at the next time t i n+1 . Here , p i tn ∈ P stands for the POI visited by user ui at time tn and the geographical coordinate of pitn is indicated by ( x i tn , y i tn ) . The observation length of the given history trajectory is n. Based on the geographical coordinates , the semantic category information citn can be acquired . Note that depending on the dataset , we utilise two types of POI category hierarchies given by : ArcGIS1 and Foursquare2 . The social context of user ui , is given by his/her neighbours , represented as N ( i ) = { uj1 , uj2 , · · · , ujm } , which is a group of users ( in U ) socially connected with user ui .

Summary: In the submitted manuscript, the authors proposed a Transformer-based method to predict the next POI (place-of-interests) of a sequence. The model takes into account not only the temporal sequence of previously visited POIs but also semantic, geographical, and extracted social information. Moreover, an auxiliary trajectory prediction task is introduced to increase the performance of the main task. To prove the efficacy of the proposed strategy, the model is tested against three publicly available datasets: Gowalla, FS-NYC, and FS-TKY. Thanks to the modeling of the additional context information, MobTCast is able to outperform state-of-the-art results by a 7.22% margin with respect to previously proposed architectures.