MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Prediction
Authors: Hao Xue
Flora Salim
Yongli Ren
Nuria M Oliver
Venue: NeurIPS 2021 Poster (2021)
Keywords: human mobility
POI prediction
auxiliary learning

Abstract: Human mobility prediction is a core functionality in many location-based services and applications. However, due to the sparsity of mobility data, it is not an easy task to predict future POIs (place-of-interests) that are going to be visited. In this paper, we propose MobTCast, a Transformer-based context-aware network for mobility prediction. Specifically, we explore the influence of four types of context in mobility prediction: temporal, semantic, social, and geographical contexts. We first design a base mobility feature extractor using the Transformer architecture, which takes both the history POI sequence and the semantic information as input. It handles both the temporal and semantic contexts. Based on the base extractor and the social connections of a user, we employ a self-attention module to model the influence of the social context. Furthermore, unlike existing methods, we introduce a location prediction branch in MobTCast as an auxiliary task to model the geographical context and predict the next location. Intuitively, the geographical distance between the location of the predicted POI and the predicted location from the auxiliary branch should be as close as possible. To reflect this relation, we design a consistency loss to further improve the POI prediction performance. In our experimental results, MobTCast outperforms other state-of-the-art next POI prediction methods. Our approach illustrates the value of including different types of context in next POI prediction.

MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Prediction
Hao Xue School of Computing Technologies
Flora D. Salim School of Computing Technologies RMIT University
RMIT University Melbourne, Australia hao. xue@rmit t. edu. au
Melbourne, Australia
flora. salim@rmit edu au
Yongli Ren
Nuria Oliver
School of Computing Technologies
ELLIS Unit Alicante Foundation
RMIT University
Alicante, Spain nuria@all mi edu
Melbourne, Australia rongli.re@@rmi..
ren@rmit. edu, au
Abstract
Human mobility prediction core functionality in many location-based services and applications. However, due to the sparsity of mobility data, it is not easy task to predict future POIs (place-of-interr) of-interests) that are going to be visited. In this paper, we propose MobTCast, Transformer-baser context-aware network for mobility prediction. Specifically, we explore the influence of four types context in mobility prediction: temporal, semantic, social, and geographical contexts. We first design a base mobility feature extractor using the Transformer architecture, which takes both the history POI sequence and the semantic information input. It handles both the temporal and semantic contexts. Based on the base extractor and the social connections of a user, employ self-attention module model the influence of the social context. Furthermore, unlike existing methods, we introduce location prediction branch in MobTCast the geographical context and predict the next location. Intuitively, the geographical distance between the location of the predicted POI and the predicted location from the auxiliary branch should be as close as possible. To reflect this relation, we design consistency loss to further improve the POI prediction performance. In our experimental results, MobTCast outperforms other state-of-the-arr next POI prediction methods. Our approach illustrates the value including different types of context in next POI prediction.
auxiliary task to model
1 Introduction
Human mobility prediction plays an important role in a wide range of applications, including personalised advertisement, traffic modelling, and predicting the spread of pandemics [7] For example, targeted advertisements, such as store information and coupons can be delivered precisely by predicting the future POIs that users might visit. It could also be valuable the context of pandemics, to support contact tracing and crowd management.
Unlike other densely acquired trajectory sequence data (such trajectories extracted from videos [I 20 and GPS trajectories [21] 229), trajectory data for the next POI prediction task is typically in the form of check-in sequences and collected from popular Location Based Social Networks (LBSNs), such as Foursquare. As the visited POI information is recorded only when a user chooses to check-in
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

and accesses the location service, the check-in sequence data is very sparse. Due to privacy concerns or battery limitations on mobile devices, users rarely report their locations all the time. This inherent sparsity increases the difficulty of the prediction of future POIs dramatically.
In the last few years, given major advances in deep learning techniques and especially the success of applying Recurrent Neural Networks (RNNs) for sequence prediction tasks, many research works have leveraged the user's historical trajectories to predict most likely POI be visited by that user. A pioneering work proposed ST-RNNS P which employ both time-specific and distance- specific transition matrices in an RNN architecture to model time intervals and geographical distances between two consecutive check-in in POIs in historical trajectories. DeepMove [9] incorporates a historical attention module to capture the periodicity of human mobility. Following this trend, more recently, VANext [ and ACN (27 propose two more attention mechanisms to learn long term preferences for users from history trajectories. However, considering the data sparsity, only exploring the historical visiting information may not be enough. For instance, compared to the large number of POIs and the complexity of the nature of people's 's preferences, the small amount of visited POIs in user's s historical log is unlikely to reveal their entire range of preferences over all available POIs. Therefore, in addition to exploring the temporal context based on the user's own past visiting history, the influence from other types of context could be valuable to achieve more accurate POI prediction. In particular, there are three important types of context that could help tackle the sparsity problem: (1) Semantic Context: Typically, in LBSNs, each POI belongs to a category with high level semantic meanings such as Shops and Education. This context is an indicator in predicting the next POL. For example, at lunchtime, people are more likely to visit Food POIs than other types of POIs. Leisure travellers will tend to check in at tourism POI attractions. (2) Social Context: Friends and family members often visit POI together. A user may also check in a new POI recommended by friends. (3) Geographical Contex:: POIs that are close to each other might be visited by the same user due to their proximity. For example, a user may visit a shopping mall and then go nearby cinema. A detailed data-driven analysis of these contexts is given in Section 7.2
However, one limitation in the existing literature is that the context has been typically modelled based on manually designed functions. For example, Sun et al. (30) noticed that the geographical distances between any two non-conseeutive visited POIs are also important. A geo-nonlocal function is introduced into the RNN to measure the distance between each visited POI and the average location of all visited POIs. Similarly, Flashback
(3X introduces a manually designed weighting function to assign weights based on both temporal and geographical distances for each time step.
In this paper, we propose novel method called MobTCast, for the next POI prediction task, which is explicitly designed to model and incorporate the influence of the aforementioned contexts in a data-driven manner. A Transformer (bls- mobility feature extractor is fundamental component in MobTCastto perform the main POI prediction task. It takes both the visited POI history and the semantic category information to learn semantic-awen aware mobility features. Using this mobility feature extractor, we extract not only the semantic-awar mobility features of a user but also the features of his/her neighbours. A self-attention mechanism is then applied to model the social influence from neighbours to the user. This aggregated feature is both semantic and social-aware. To include geographical context, the model includes auxiliary task, which is different from previously proposed methods that depend on manually designed kernels. Considering that the focus of this auxiliary task is exploring the geographical information (i.e., geographical coordinates of visited POIs), it is designed as a trajectory forecasting task. The purpose is to predict a future location (coordinate) based on the previously visited locations.
As a result of these two tasks (main and auxiliary). there are two predicted coordinates available: the predicted location from the auxiliary trajectory forecasting task and the inferred location from the predicted POI given by the main task. Ideally, the Euclidean distance between these two locations should be as close as possible. To this end, a novel consistency loss function is proposed to reflect and embody this relationship in the training process of MobTCast. Thus, the main and the auxiliary tasks are connected not only at the feature level but also at the output level. We conduct extensive experiments on three publicly available real-world datasets. The results demonstrate MobTCastt competitive POI prediction performance and show the value of incorporating different types of context as well as the consistency loss function.
In summary, the main contributions of our work are two-fold: (i) We address the next POI prediction problem by including four types of context: temporal, geographical, social and semantic, Moreover,
2

---

we design an auxiliary trajectory forecasting task to predict the geographical coordinates of the next most likely location. To the best of our knowledge, this is the first time that auxiliary trajectory forecasting is incorporated in a POI prediction model, which provides a new perspective for under- standing human mobility. (ii) We introduce novel consistency loss function to connect the predicted POI (from the main task) and the predicted location (from the auxiliary task). This consistency loss function helps further improve the POI prediction performance.
2
Related Work
Next POI Prediction One of the carliest works that utilised geographical, temporal, and social contexts to predict the next POI is Gd, which combines the periodic day-to-day mobility patterns with information from social ties, inferred from the friendship network. Since this seminal work, many others have proposed methods to better model the check-in behaviours or the trajectory of geographical movements, depending on the data they work on. Based on weight matrix factorisation, Cheng er al. A proposed a personalised Multi-centre Gaussian Model (MGM) to capture the geographical influence and Lian et al. [23] proposed GeoMF model that considers the geographical influence, Li et al. (22] developed a ranking based geographical factorisation method to incorporate the influences of geographical and temporal context. Recently, a local geographical model is introduced into the logistic matrix factorisation to improve the accuracy of POI recommendation [28). GERec [32] employs temporl relation path embedding to model the temporal, spatial and semantic content for the next POI recommendation.
Deep learning- based neural networks have been used for temporal sequence modelling. RNNs (including LSTM [15] and GRU (77) and Transformer [31 have also been proposed and successfully applied to many time series prediction problems such as pedestrian trajectory prediction [1 (6 and traffic prediction 34] For the next POI prediction task, ST-RNN (Spatial Temporal Recurrent Neural Networks), proposed by Liu et al. [25), improves the original RNN architecture by incorporating temporal and spatial context at each time interval in the historical trajectory. Following the trend of using RNN, series of methods are proposed to model human mobility behaviours and predict the next POI, such as DeepMove (9. Based on RNNs it applics an attention mechanism to capture the periodical features of human mobility. Similarly, Gao et al. [1I introduced variational attention mechanism to learn the attention on the historical trajectories.
Efforts have also been made to incorporate different kinds of context for POI prediction. SERM 44) designed to jointly learn embeddings temporal and semantic contexts (i.e., text messages). Gao el al. 13 proposed to leverage knowledge graph to accommodate the influence of users, locations and semantics. Flashback (38) tackles the geographical context through an explicitly designed spatio-temporal weighting function. The calculated weights are applied to the hidden states in an
RNN correspondingly. STAN [26] incorporates the gcographical context to learn the regularities between non-adjacent locations and non-contiguous visits. In these methods, the modelling of the geographical context heavily depends on manually designed kernel functions (e.g., Euclidean distance based functions) or pre-computed spatio-temporal relation matrix (26 (which would dramatically increase the memory and computation costs when the total number of POIs is large). There is limited work that have simultaneously considered temporal, semantic, social and geographical contexts.
Auxiliary Task Learning An auxiliary task a task that is trained jointly with the main task during the training phase to enhance the main task's performance. As a variant of multi-task learning, auxiliary learning has become popular in recent years. It is mainly used in the computer vision field such semantic segmentation 35
Recently, several methods have been proposed using multi-task learning to model human mobility. For example, Chen er al. 3 introduced DeepJMT which jointly predicts location and time. MCARNN, proposed by Liao et al. (24), is a multi-task network to predict both activity and location. Compared to multi-task learning, in auxiliary task learning the auxiliary task is nonoperational during the inference phase. Note that MCARNN [24), the authors also proposed to leverage an auxiliary task to learn location embeddings. However, the auxiliary task in the proposed MobTCast approach is trajectory forecasting task, which is designed to consider the geographical context directly, with the introduction of consistency loss function to connect the main task and the auxiliary task.
3

---

Postton Emitedting
Locatin Enitetting
User Croeeding
Albllity Flakkrr
Logend:
Nppe LoCKOo
Anshary Inpertar Forecaiing
Enibadd ng Layer (Location)
Roohon Locaaioe Enddeiihige
Location
MLP,
Predictioo Loos
Loc Transformer Encoder
Fully Coneacted Layer
Embbedding Layer (User)
Social Contoxt Extractor
Self attention
Schmar
A1
POI
Hoblly Fodtane Eocadertorazee aocal
Prodiction Loss
S
Figure 1: The proposed MobTCast, Dash arrows indicate operations used in the training phase only. Due to space limitations, please refer Figure 2(a) for details of the Mobility Feature Encoder.
3 Problem Formulation
," denoting the set of POIs and the set of users, respectively. Each POI is represented by a (lon, lat) tuple, which indicates the geographical context. The human mobility prediction problem defined as follows. Given the history mobility trajectory 7 PP,
Consider two sets: P (poin: poig." : poi mr) and U
.Pi..) of user u² E U, the goal is to predict the POI Pl... that is going to be visited by the same user at the next time +1 Here, pi. E P stands for the POI visited by user u² at time f, and the geographical coordinate of pi., indicated by (#t,, v..)) The observation length of the given history trajectory is n.
Based on the geographical coordinates, the semantic category information ci,, can be acquired. Note that depending on the dataset, we utilise two types of POI category hierarchies given by: AreGIşT and Foursquare The social context of user u', is given by his/her neighbours, represented as wim) which a group users (in U) socially connected with user w².
N(1 {w
4 Method
The overall framework of the proposed method is illustrated in Figure | To solve the POI prediction task and the auxiliary trajectory forecasting task, MobTCast consists of four major components: (i) Mobility Feature Extractor (Section 4.1]: this is a fundamental component in the proposed POI prediction method. It is used to extract semantic-aware mobility features of given history trajectory. The temporal context is also modelled in this extractor. (ii) Social Context Extractor (the light purple part in Figure T. Section 4.2): ; this module is used to model the social influence of user u² in the prediction process. (iii) Auxiliary Trajectory Forecasting (the light orange part in Figure [] Section 4.3): this auxiliary task is introduced to incorporate the geographical context in the POI prediction, to explicitly predict the next geo-location. (iv) Consistency Loss: (Section (.4) this component establishes relationship between the auxiliary trajectory forecasting and the POI prediction. The detailed description of each component is presented in the following subsections.
4.1 Semantic-aware Mobility Feature Extractor
The modelling of the given history trajectory of user is an essential step in any POI prediction method. In MobTCast, this step is tackled as a mobility feature extraction process. In addition to the visited POIs and the visited timestamps that have been widely considered by other methods 19) we argue that the semantic context plays significant role and should also be incorporated in this process. For example, Alice visited Sushi Shop for lunch yesterday but she decides to go to McDonald's today. Although Sushi Shop and McDonald's are two different POIs, they belong to the same semantic

---

POI Transformer Encoder
Semardo Nishillyy Fasture
Entoshing
Emntwitding layer (POI, Semartkc Time)
(a)
p. 2
Npat (MO), Catagory Thoastorpe)
(b)
Figure 2: (a) Illustration of the proposed Transformer-based mobility feature extractor. (b) Illustration of using check-in vectors for discovering neighbours u² is a neighbour u²).
category Food. Such a high-level semantic context helpful to predict which POI going to be visited by Alice tomorrow noontime.
To incorporate the semantic context, Semantic-aware Mobility Feature Extractor is explicitly designed. As shown in Figure 2(a) this feature extractor takes three types of inputs: POIs, timestamps, and semantic category labels. The mobility feature extraction process can be formulated as:
e Op(p): Wop), ) e = =elct: W Wor), of = oe W
e o @ eg @ eg" hi, F(eliti , eitt WF),
(2)
(3)
where @ is the concatenation operation and ex is the concatenated embedding of: the POI embedding elt the category embedding e², and the temporal embedding ef these equations, the W terms stand for the weight matrices that need to be optimised during the training process. These three embedding vectors are calculated through three different embedding layers (.,(-), (b..)), and ()).)), respectively. In Eq. (3), F(-) is function extract feature hi_ with embedding vectors of the whole history trajectory from & to f, as input. This extracted feature hü models the spatio-tempon history movement behaviour of user u² and it is semantic-awane as well. In MobTCast, the Transformer architecture [31] is used F(.) as it has been proven effective in other sequence modelling tasks, such as language understanding [8, speech recognition [33], and flow prediction [37]. The Transformer architecture is based a self-attention mechanism
it better model the history trajectory movement than a vanilla RNN structure. Since there is no recurrence or convolution operation in the Transformer, position embeddings added to the concatenated embeddings to retain sequential order information (see Figure 2(a)
We use standard sine/cosine function based position embeddings
introduced in 31
4.2 Social Context Extractor
The social influence is another important kind of context to be considered in the next POI prediction process, especially when the mobility data sparse. For example, one of Alice's friends recommends new restaurant that she has never visited before, so she decides to visit it tomorrow. In this example, since there is high likelihood that this restaurant has been visited by Alice's friend before, the social context (Alice's 'ss friend's mobility) is helpful to predict Alice's next POI.
Inferring social links based on mobility data yet another challenging task 2 main focus is POI prediction, we adopt a straightforward approach to discover the neighbour set N( of user w'. The social neighbour discovering process as follows. As shown in Figure 2(b) we first build a check-in vector for each user, based on their check-in in history. The dimension of this check-in vector equals the number of POIs (lP) and each dimension of the vector indicates how many times the user visited the corresponding POI. The cosine similarity of two users' check-in in vectors is then computed. If the calculated cosine distance is larger than threshold т, these two users are considered to be connected.
10 [6]
Since our
For each user ud E Nay, the mobility feature extractor (Eqs. 1 is applied calculate the semantic-aware mobility feature hi.-- It worth noting that any "future" information (after ( from neighbours cannot be used as it is not available for the prediction time step t If th is the last timestamp of neighbour's observed trajectory, mathematically, the above condition can be represented as: th < f Vj E Man- The extraction of social influence can be seen as feature aggregation process. The social context vector H; is weighted sum of his/her own mobility feature
5

---

hi and his/her neighbour's (all neighbours in Me) are considered) mobility feature h4.
H = ashh +
Σ
ay
(4)
JEN(
To compute the weight terms a, we use multi-head self-attention mechanism 31
4.3 Auxiliary Trajectory Forecasting
Compared other recommendation tasks, like next shopping item prediction and music recommenda- tion, the geographical proximity of POIs is an inherent characteristic in the next POI prediction task. For example, Alice more likely to visit POIs (e.g., a supermarket buy snacks or a park to walk) that are geographically close to the visited Sushi Shop than those that are significantly further away.
This geographical information is thus another important type of context that is worthy to exploit in the next POI prediction task.
Unlike previously published efforts 23
38 that take the geographical information into account by manually designed kernel functions, the geographical context is incorporated in MobTCast by leveraging an auxiliary task. This auxiliary task is designed as a trajectory forecasting task. Since POI prediction is typically tackled as a classification task, the long tail distribution of POIs caused by data sparsity a severe bottleneck that limits the performance. Thus, the motivation of this introduced auxiliary trajectory forecasting is to directly forecast geo-location coordinates (as a location regression task) to leverage the geographical context and alleviate the effect of data sparsity.
Similar to the pedestrian trajectory prediction task [1 36), the coordinates of the historical trajectory firstly embedded through an embedding layer Φi. Then, another Transformer F-)-) is used to extract the geographical context vector gi. Mathematically, this process is expressed follows:
ist = 21 Di W 61), o._ Fe(e)
(5)
_6"
3.l2
.6 5.60-1, ( WF,).
(6)
where W terms are trainable weights. To build a connection between the main task and the auxiliary task, the social context vector Hi, from the social context extractor (Section 4.2) and the geographical context vector gi concatenated and then passed through fully connected layer (FC(-)). In addition, the user embedding that learns the personalised user preference is also included as the user's personal preference would also influence his/her own mobility pattern:
e, = Ou(u;; Wer), fi = FC + gi. $e())
(7)
(8)
Since the user embedding e, remains the same during the whole trajectory, it is only used in Eq. (8) and not used at every time step like other embeddings Eq. (3) or Eq. (6).
This aggregated feature representation fi, inherits the semantic context, the social context, the geographical context, and the user personal preference. It then can be used for both the main (i.e. predicting the next POI, detailed in the following subsection) and the auxiliary tasks. The predicted location (i..... De...) at time step (++1 is then decoded via:
(G De MLP
(9)
where (-) is a multi-layer perceptrons (MLP)- based decoder.
4.4 Next POI Prediction
Based on the context-aware feature fi,- the prediction of POI at next time step 1, as follows:
Pr(ff,.,) = softmax (MLP (ft.) )).
(10)
where MLP,() is another MLP structure. Different from the auxiliary trajectory forecasting (Eq. 9 an extra softmax layer is required for generating the probabilities of all POIs. Thus, by performing Eq. (10). the most likely POI that will be visited by the user u' at the time step ( is the one with the largest probability in Pr Pr(pf...).

---

In addition to the POI prediction loss L1 for the main task and the trajectory forecasting loss L2 for the auxiliary task, we propose novel consistency loss L3 for training MobTCast. The details of these three losses are described below.
POI Prediction Loss Given that the next POI prediction is treated as classification over the entire POI set, the POI prediction loss is multi-class cross entropy loss:
IPI
L1 - 2--EPr(se Pr(P log Pr (Pr(P Jp),
(11)
where Pr(p ) indicates the ground truth, i.e., Pr(pi )p 1 if u² visited the p-th POI at 2 Similarly, Pr(Pi..)) is the predicted probability that the user w² will visit the p-th POI.
Trajectory Forecasting Loss This loss is used for the auxiliary task. It is that measures the difference between the predicted location and the ground truth location.
square error loss
L2
| w+1 Ze
||a
|1 n+1
(12)
where (Iea. (1 is the geographical coordinate of the ground truth POI pi.. that is visited by u' at 6 f+1-
Consistency Loss To better leverage the auxiliary task, we not only connect the main task and the auxiliary task at the feature level (Eq. (8) but also design new loss function to further link the two tasks. Intuitively, there should
relationship between the predicted POI from the main task and the predicted location from the auxiliary task. That is, the location of the predicted POI and the predicted location should be as close as possible. Consequently, a consistency loss is introduced to embody the above relationship.
Based on the probabilities Pr( given by Eq. (10), the location of the predicted POI can be inferred through:
argmax (Pr (Pr((t...))) = (ā (...))
(13)
The consistency loss is then defined the mean squared error between the inferred location (Zr.,. Ve. and the predicted location (Øz: De,,,
L3 = -1 - = 11 + l1 De
(14)
Note that, as given in Eq. (15), both L2 and L3 (mean mean squared errors) are calculated and averaged over all training samples in a batch.
The overall loss function is combination of the above three losses that the parameters of our MobTCast are optimised by minimising:
C= 101 0gL2),
(15)
where 0 terms are used to balance the three losses and B is the total number of training samples. For the 6-th training sample, Ct. C22, and C are the POI prediction loss, trajectory forecasting loss, and the consistency loss, respectively. The coordinates of each location are normalised so that the scale of three losses are similar. Using this combined loss function, MobTCast trained together in an end-to-end fashion.
5 Experiments
In our experiments, use three widely used LBSNs datasets: Gowalla (6) Foursquare-NYC 39) (FS-NYC), and Toursquare-Tokyo [39 (FS-TKY) (more details of these datasets are contained in Section 7.I of the Appendix). These datasets are publicly available and no personally identifiable information is included. Considering the social connections for FS-NYC and FS-TKY are not given, method described Section 4.2 (with 7 = 0.5) used to explore the neighbours of each user. In our experiments, all visiting records in the training set are used to build check-in vectors so that the social connection of any two users fixed. Following [38], the observation length n is set to 20 and split the check-in sequence of each user into 80% for training and 20% for testing.
7

---

Table 1: Performance results (mean = std ) of different methods on the three datasets. In each row, the best performing results are shown in bold and the second best are given in underline.
Aoc
FMFMGM
LGLMF
RNN
DeepMove
Flashback
STAN
MobTCast
Top-1
0.0505(0.0019) 0.0394(0.0035)
0.1334(0.0021)
0. 0.1636(0.0019)
0. 118090000220
0.2051(0.0023 (.)
Gowalla
Top-5
0. 12188(00055)
0. 1.842(0.0049)
0.304000.0017)
0.37040.0022)
0,3875
(.0015
Top-10
0. 0.193000.0058)
0.4710(0.002)) 0.4536(0.0022) 0.3717(0.0025) 0.2793)0.0062)
0.4816(0.0038)
0.5236(0.0002))
Top-20
0.2976(0.0029)
0.431500.0026) 2(0,00)))
0.5196000033)
0.5372(0.0014)
0.5534(0.0025)
0.5956(0.00120)
FS-NYC
Top-1
0.120100.0011
0.0591(0.0038)
0,2602(0.0028) 0.251700.0211 0.196600.0025)
0.2804(0.0022) 0.2755(0
Top:5
0.3103(0.0014)
0.2122/0.0069)
0.5258(0.0063)
0.5992(0.0028) 0.592%0.0033)
0.603))
(.6591(0.00)3
Top-10
0.4054(0.0023)
0.3248(0.0089)
0.7013(0.0041) 0.6635(0.0102)
0.7192(0.0054)
0.7816(0.0047) 0.742700.0037)
Top-20
0.496700.0009)
0.452000.0113)
0.7763(0.0045) 0.7464(0.0058)
0.8079(0.0033)
0.8398(0.00133
0.8561(0.0041)
Top-1
).0234(0.0061)
0.0334(0.0064)
0.1927(1.0027) 0.77500.0017)
0,230300.0020
0.2238(0.0035)
0.2554
FS- S-YYY
Top
0.0691X0.0128)
0.12710.0200)
0.50230.0022) 0.4389(0.0026)
0.53310..0027)
0.52930.0033)
0. 0.568330000555
Top-10
0. (402(0.0227))
0.200700.0110)
10)
(.5909(1.0049)) 0.539700.044)
0.6346(0.0054)
0.6245(0.0055)
0.6726(0.0042)
Top-20
0.2174(0.0178)
0.3495(0.0147)
0.6720(0.0065) (0.002)
0.7062(0.0051)
0.7134(0.004))
0.748))
The hyperparameters are set based on the performance on the validation set which is 10% of the training set. The hidden dimensions for the Transformer used in the mobility feature extractor and the auxiliary task are both 128. According to Eq. (2) and (3), the sum of the dimensions of the POL, semantic and temporal embeddings equals the hidden dimensions of the Transformer F(-). Thus, we set the dimensions of these embeddings as 80, 24, and 24. As for the weights of three loss functions in Eq. (15). all three are set to 1. For the auxiliary task, we process the dataset by normalising the coordinates (latitude and longitude) within [-1, 1]. The model is trained using an Adam optimiser [8]] and implemented using Py PoTorch on a desktop with an NVIDIA GeForce RTX-2080 Ti GPU.
To evaluate the performance of the proposed method, we compare MobTCast against the follow- ing POI prediction approaches: FMFMGM 4. a fused matrix factorisation framework using the Multi-centre Gaussian Model to model the geographical information. LGLMF [28). a logistic ma- trix factorisation method combined with a local geographical model for leveraging geographical information. RNN, the basic architecture for sequential predictions. In the experiments, we use the GRU variant 7 as the baseline. DeepMove ()] a popular RNN-based POI prediction model that incorporates an attention mechanism for modelling the POI visiting history. Flashback (38) and STAN 26), two recent of deep learning- based models for next POI prediction. Similar to [9]. we use top-k accuracy (k- 1, 5, 10, 20) to evaluate the POI prediction performance. The predicted probabilities of all POIs (Pr(it... yield from the main task) are ranked from large to small. We then check whether the actual visited POI (p) appears in the top-k predicted POIs.
5.1 Prediction Performance
The results of MobTCast and the baseline methods are given in Table I The best performing method on each dataset is given in bold and the second best in underline. For each model, we ran the experiments 5 times and report the average accuracy as well as the standard deviation. In general, deep learning-based methods outperform matrix factorisation-basee methods (FMFMGM and LGLMF) by a relatively large margin. The baseline RNN has the worst performance among all deep learning- barning-based methods as no attention mechanism or any context is taken into account. Compared to DeepMove, the weighting function proposed in Flashback is based on temporal and geographical distances, which results in better prediction performance. Overall, with more contexts and the auxiliary task, MobTCast achieves the best performance in comparison to other methods across all datasets.
The original social links of users may be not available in the general case, due to e.g. privacy concerns. The experiments on the FS-NYC and FS-TKY datasets, where social links are not provided, demonstrate that MobTCast is able handle these datasets by applying a visiting pattern-based process (described in Section 4.2) to discover the "social" context for POI prediction. Such discovered social context can provide useful information for prediction. This also shows the generalisation of MobTCast on different datasets. The average improvement of MobTCast (across all three datasets and all four accuracy metrics) is 7.22%. This improvement is computed as (ace) - ace2)/u0c2 x 100%, where acc; and acc2 are the accuracy of MobTCast and the second best performing method (either Flashback or STAN depending on the metrics and datasets). These results illustrate the superior prediction performance of the proposed method.
8

---

Table 2: The prediction results of the ablation studies on all three datasets, without (w/o) the specifically mentioned components.
Gowalla
FS-NYC
FS-TKY
TOP-1
Top-5
Top-10
Top-20
TOP-1
Top-5
Top-10
1op-20
TOP-1
Top-5
Top-10
Top-20
VO (wil contexts)
0.1432 0.3374
0,4129
0.4753
0.2219
0.5628
0.6921
0.7723
0.1793
0.4898
0.5882
0.6612
V1 (wlo goographical, wro social)
0.1691 0.3747
0,4528
0.5192
0.2466 0.6062
0.7338
0.8045
0.2018
0,4966
0.5984
0.6739
V2 (wo social, wio C2. we Ca)
0.1896 0.4210 0,5058
0,5766
0.2504
0.6133
0.7417
0.8269
0,2367
0.5459
0.6504
0.7271
V3 (wro social, Ca)
0,1925 0.4231 0,5097
0.57%
0.2622
0.6151
0.7476
0.8328
0,2395
0,5463
0.6537
0.7304
V4 (wr social, wlo L2)
0.1934 0,4242 0,5098
0.5813
0.2635
0.6293
0.7606
0.836
0.2407
0,5483
0.6535
0.7299
V5 (wl social)
0.1987 0.4332 0.5203
0.5948
0.2727
0.6455
0.7649
0.8457
0,2469
0.5571
0.6641
0.7406
MobTCast
0.2051 0,4364
0.5236
0.5956
0,2804
0.6591
0.7816
0.8561
0.2550
0.5683
0.6726
0.7489
5.2 Ablation study
To investigate the effectiveness of each module in our proposed method, we consider 6 different variants in total (detailed configurations of these variants are given in Section (73).. The first two variants are: (i) VO: None of the three contexts is incorporated in this variant. It can be seen as a mobility feature extractor without using semantic information as input. (ii) V1: This variant includes the semantic context only. It predicts the next POI based on the semantic- aware mobility features. Furthermore, given that incorporating the geographical context through the auxiliary task introduces two more loss functions (i.e., trajectory forecasting loss and consistency loss), we also explore the contribution of each loss function through the following 4 variants: (iii) V2: This variant only uses the basic POI prediction loss from the main task. (iv) V3: The loss function for this variant is combination of the POI prediction loss and the trajectory forecasting loss from the auxiliary task. (v) V4: For this variant, it employs both the POI prediction loss and the proposed consistency loss. (vi) V5: All three loss functions are combined in this variant. This loss function is equivalent to the loss function used in MobTCast. For V2 - V4, both semantic and geographical context are enabled. However, the social context extractor used MobTCast to model the social context is replaced with the basic semantic- aware feature extractor, which means the social context is excluded.
The results (average of 5 runnings) of these variants on the three datasets are given in Table 2 In general, comparing the results from MobTCast against all six variants, it can be noticed that MobTCast outperforms all the variants on all three datasets. VO yields the worst performance, which is as expected as it is basic prediction model based on the visiting history without considering any contextual information. When the semantic context is incorporated in VI, compared to VO, we observe that there is a performance gain on all three datasets. With the help of the auxiliary task, variants V2- - V5 have better performances than VO and V1. More precisely, among these four variants, the performance of V2 is worse than that of the other variants as only the POI prediction loss from the main task is applied and there are no extra constraints for the auxiliary task in the training process. On the contrary, V5 achieves the top performance, where all three loss functions combined. When only two loss functions are used, V4 outperforms V3 by a relatively small margin. These comparison results confirm the importance of combining both the trajectory forecasting loss and consistency loss when the auxiliary task is introduced. Especially for the consistency loss, it can work connection between the two tasks and further improve the POI prediction performance. The results of the above ablation studies show the effectiveness of each module and justify the need for all loss functions included in MobTCast.
5.3 Limitations and Future Work
A limitation of our approach lies in that MobTCast depends on contextual information, such as semantic categories. If semantic contexts of places (or social connections) are changed over time, the performance of a trained model might be not optimal. Also, given that the social context extractor takes every neighbour into consideration, the efficiency of our MobTCast would drop if there are too many neighbours. We plan to investigate applying graph neural network structures such as message passing neural networks [12] and simple spectral graph convolution [42) to model the social context in future work. We plan to address these two limitations in future work.
6 Conclusion
In this paper, we have proposed MobTCast, novel deep learning- architecture to tackle the next POI prediction task. It contains Transformer-bssed semantic-awar mobility feature extractor, and

---

social context extractor to consider the mobility features of the user's neighbours to take into account the social influence. Furthermore, it includes an auxiliary task, which is trajectory forecasting task, to incorporate the geographical context. Through extensive experiments, the results illustrate that MobTCast outperforms other state-of-the-an prediction methods and demonstrate the effectiveness of incorporating each type of context in MobTCast. In addition, the main task and the auxiliary task are connected via a consistency loss function. The ablation study results show that introducing the consistency loss function can further improve the performance.
Acknowledgments and Disclosure of Funding
This research is supported by Australian Research Council (ARC) Discovery Project DP190101485.
References
[1] Alexandre Alahi. Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese. Social Istm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 961-971, 2016.
[2] Michael Backes, Mathias Humbert, Jun Pang, and Yang Zhang. walk2friends: Inferring social links from mobility profiles. In Proceedings of the 2017 ACM SIGSAC Conference Computer and Communications Security, pages 1943-1957, 2017.
[3] Yile Chen, Cheng Long, Gao Cong, and Chenliang Li. Context-aware deep model for joint mobility and time prediction. In Proceedings of the 13th International Conference on Web Search and Data Mining, pages 106-114, 14, 2020.
[4] Chen Cheng, Haiqin Yang, Irwin King, and Michael Lyu. Fused matrix factorization with geographical and social influence location-based social networks. In Proceedings of the AAAI conference artificial intelligence, volume 26, 2012.
[5] Sumanth Chennupati, Ganesh Sistu, Senthil Yogamani, and Samir Rawashdeh. Auxnet: Auxiliary tasks enhanced semantic segmentation for automated driving. arXiv preprint arXiv:1901.08800 2019.
[6] Funjoon Cho, Seth A Myers, and Jure Leskovee. Friendship and mobility: user movement in location-based social networks. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1082-1090, 2011.
[7] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv: 1412.3555, 2014.
[8] Jacob Devlin, Ming-Wos Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 4171-4186, 2019.
[9] Jie Feng, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. Deepmove: Predicting human mobility with attentional recurrent networks. In Proceedings of the 2018 world wide web conference, pages 1459-1468, 2018
[10] Qiang Gao, Goce Trajcevski, Fan Zhou, Kunpeng Zhang, Ting Zhong, and Fengli Zhang. Trajectory- baased social circle inference. In Proceedings of the 26th ACM SIGSPATIAL Interna- tional Conference on Advances in Geographic Information Systems, pages 369-378, 2018
[11] Qiang Gao, Fan Zhou, Goce Trajcevski, Kunpeng Zhang, Ting Zhong, and Fengli Zhang. Predicting human mobility via variational attention. In The World Wide Web Conference, pages 2750-2756, 2019.
[12] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Doina Precup and Yee whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-1 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 1263-1272. PMLR, 2017.
10

---

[13] Qing Guo, Zhu Sun, Jie Zhang, and Yin-Leng Theng. An attentional recurrent neural network for personalized next location recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 83-90, 2020.
[14] Qianyue Hao, Lin Chen, Fengli Xu, and Yong Li. Understanding the urban pandemic spread- ing of covid-19 with real world mobility data. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 3485-3492, 2020.
[15] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 988):1335-17881 1997.
[16] Hsun-Ping Hsieh and Cheng-Te Li. Inferring online social ties from offline geographical activities. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1-21, 2019,
[17] Jizhou Huang, Haifeng Wang, Miao Fan, An Zhuo, Yibo Sun, and Ying Li. Understanding the impact of the covid-19 pandemic on transportation-rrlate behaviors with human mobility data. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery Data Mining, pages 3443-3450, 2020.
[18] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[19] Vineet Kosaraju, Amir Sadeghian, Roberto Martín-Martín, Ian Reid, Hamid Rezatofighi, and Silvio Savarese. Social-bigat: Multimodal trajectory forecasting using bicycle-gan and graph attention networks. In Advances in Neural Information Processing Systems, volume 32, 2019.
[20] Jiachen Li, Fan Yang, Masayoshi Tomizuka, and Chiho Choi. Evolvegraph: Multi-agent trajectory prediction with dynamic relational reasoning. In Advances in Neural Information Processing Systems, volume 33, 2020.
[21] Xiaolong Li, Gang Pan, Zhaohui Wu, Guande Qi, Shijian Li, Daqing Zhang, Wangsheng Zhang, and Zonghui Wang. Prediction of urban human mobility using large-scale taxi traces and its applications. Frontiers of Computer Science, 6(1):111-12) 2012.
[22] Xutao Li, Gao Cong, Xiao-Li Li. Tuan-Anh Nguyen Pham, and Shonali Krishnaswamy, Rank- geofm: A ranking based geographical factorization method for point of interest recommendation. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 433 442, 2015.
[23] Defu Lian, Cong Zhao, Xing Xie, Guangzhong Sun, Enhong Chen, and Yong Rui. Geomf: joint geographical modeling and matrix factorization for point-of-interest recommendation. In Proceedings of the 20th ACM SIGKDD international conference Knowledge discovery and data mining, pages 831-840, 2014.
[24] Dongliang Liao, Weiqing Liu, Yuan Zhong, Jing Li, and Guowei Wang. Predicting activity and location with multi-task context aware recurrent neural network. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 3435-3443 2018.
[25] Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. Predicting the next location: A recurrent model with spatial and temporal contexts. In Thirtieth AAAI conference on artificial intelligence, 2016.
[26) Yingtao Luo, Qiang Liu, and Zhaocheng Liu. Stan: Spatio-temporal attention network for next location recommendation. In The World Wide Web Conference, 2021.
[27] Congcong Miao, Ziyan Luo, Fengzhu Zeng, and Jilong Wang. Predicting human mobility via attentive convolutional network. In Proceedings of the 13th International Conference on Web Search and Data Mining, pages 438-446, 2020.
[28] Hossein Rahmani, Mohammad Aliannejadi, Sajad Ahmadian, Mitra Baratchi, Mohsen Afsharchi, and Fabio Crestani. Lglmf: local geographical based logistic matrix factorization model for poi recommendation. In Asia Information Retrieval Symposium, pages 66-78. Springer, 2019.
11

---

[29] Punit Rathore, Dheeraj Kumar, Sutharshan Rajasegarar, Marimuthu Palaniswami, and James C Bezdek. A scalable framework for trajectory prediction. IEEE Transactions on Intelligent Transportation Systems, 20( 20(10))3800-3874, 2019.
[30] Ke Sun, Tieyun Qian, Tong Chen, Yile Liang, Quoe Viet Hung Nguyen, and Hongzhi Yin. Where to go next: Modeling long-and short-term user preferences for point-of-interes recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 214-221, 2020.
[31] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008, 2017.
[32] Xianjing Wang, Flora D Salim, Yongli Ren, and Piotr Koniusz. Relation embedding for personalised translation- based poi recommendation. Pacifie-Asia Conference on Knowledge Discovery and Data Mining, pages 53-64. Springer, 2020.
[33] Yongqiang Wang, Abdelrahman Mohamed, Due Le, Chunxi Liu, Alex Xiao, Jay Mahadeokar, Hongzhao Huang, Andros Tjandra, Xiaohui Zhang, Frank Zhang, et al. Transformer-basee acoustic modeling for hybrid speech recognition. In IEEE ICASSP, pages 6874-6878. IEEE, 2020.
[34] Zhumei Wang. Xing Su, and Zhiming Ding. Long-term traffic prediction based on Istm encoder- decoder architecture. IEEE Transactions Intelligent Transportation Systems, 2020.
[35] Lian Xu, Wanli Ouyang, Mohammed Bennamoun, Farid Boussaid, Ferdous Sohel, and Dan Xu. Leveraging auxiliary tasks with affinity learning for weakly supervised semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6984-
6993, 2021.
[36] Hao Xue, Du Q Huynh, and Mark Reynolds. Poppl: Pedestrian trajectory prediction by Istm with automatic route class clustering. IEEE transactions on neural networks and learning systems, 2020.
[37] Hao Xue and Flora D Salim. Termcast: Temporal relation modeling for effective urban flow forecasting. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2021.
[38] Dingqi Yang, Benjamin Fankhauser, Paolo Rosso, and Philippe Cudre-Mauroux. Location pre- diction over sparse user mobility traces using mns:: Flashback in hidden states! In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 2184 2190, 2020.
[39] Dingqi Yang. Daqing Zhang, Vincent W Zheng, and Zhiyong Yu. Modeling user activity preference by leveraging user spatial temporal characteristics in lbsns. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 129-1122,142, 2014.
[40] Di Yao, Chao Zhang, Jianhui Huang, and Jingping Bi. Serm: recurrent model for next location prediction in semantic trajectories. In Proceedings of the 2017 ACM Conference Information and Knowledge Management, pages 2411-2414, 2017.
[41] Jia-Dong Zhang and Chi-Yin Chow. Geosoca: Exploiting geographical, social and categorical correlations for of recommendations. In Proc. of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 443-452, 2015.
[42] Hao Zhu and Piotr Koniusz. Simple spectral graph convolution. In International Conference on Learning Representations, 2021.
12


Reviews:
[{"review_id": "w_ji4Mt_Lyn", "reviewer": "Reviewer_yyia", "summary": "Summary:\nThe paper focus on building a Point-of-interest (POI) predictor based on temporal, semantic, social and geographical contexts fitted into a deep learning model with aim of overcoming sparsity issue in mobility data. Datasets used are publicly available and results, ablation studies supports the method's efficacy. This is an application paper and has shared code in zip.\n\nClaimed contributions:\n1. Auxiliary trajectory forecasting is incorporated in a POI prediction model as a first.\n2. A novel consistency loss function to connect the predicted POI (from the main task) and the predicted location (from the auxiliary task).\n", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nIn general, the work is encouraged, however the paper in its current shape needs some overhaul specifically towards scientific contribution.\n\nStrengths:\n1. Code, though small sized is available online.\n2. Section 4.4 \n3. Experiments and Supplementary material.\n\nRelevance:\nRelevant to NeurIPS application track.\n\nNovelty:\nLoss L3 for training MobTCast is sound.\n\nImpact:\nOn prediction of crowds (useful in pandemic) and also tour planners possibly.\n\nWeaknesses:\n1. Writing the paper and linking and describing sections need improvement.\n2. Lat, Lng and distances are based on certain geometric models of earth as earth's shape is Geoid. Google Maps uses one, Satellite, Navy, Flights others. What system is adhered here is not clear and have not been thought of it seems. What happens if POI spans across large regions and small regions?\n3. Why can't this problem be modeled as a recommender problem? Justify.\n4. How are timestamps validated across time-zones? Are they all in GMT or offset taken care of?\n5. Analysis is missing from Experimental section.\n\nCorrectness:\n1. Line 51 - \"For example, people are more likely to visit Food POIs at lunch time than at other times. \" - should not this be temporal context?\n2. Line 58 - \"manually designed functions\" - usually human designed functions wrt semantics are better than those learnt by automation? Defend.\n3. Line 184 - something works here do not mean it will work somewhere. An easy example to relate - deep learning works well for images, fails miserably for signals.\n4. Line 216 - different recommender problems have their own inherent characteristic. Like music has genre and shopping has market basket influence.\n5. Line 246 - extra softmax layer not clear.\n6. Line 293: why Adam optimizer?\n\nEffort: Effort has been made more in writing the paper more than that put in coding and thought process (philosophy). Reverse was expected.\n\nSuggestions:\n0. Give an example to illustrate the problem.\n1. This is a technical paper venue. Your abstract should contain numbers instead of promising results. Directly jump to the point and results and the philosophy. Elaboration can be done in Introduction.\n2. The long sentences should be broken into small sentences for clarity, continuity and smooth reading. May be another round of proof reading will enhance the grammatical and contextual use of words.\n3. It is recommended to give a diagram in 'Introduction' section to make sense of the problem statement and application at a glimpse.\n4. May be related work gaps can be enlisted in a table and what extra / new has been done be highlighted, focusing on whether new work is at all needed or just a feature nice to have?\n5. Overcome repetitions - there exists many text not needed again - focus more on the technical content. \n6. Make contribution section crisp. You have introduced novel method - so what? Why others will use yours instead of their own? Highlight this in your preferred way.\n7. De-clutter useless information - Line 130 - For example, in computer vision, it has 131 been used for semantic segmentation [25, 5], crowd counting [45], depth completion [27], and visual localisation [34]. - wastage of references and text area.\n8. Follow Train-Eval-Split following Andrew Ng's ML Yearning book.\n9. How does demographics and world events bias the POI prediction? A scope of future work.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There is no negative impact per se, apart from privacy issue of mobility data that may not be fully anonymized following k-anonymity logic. In terms of current pandemic scenario, this work may find some value in application across contact tracing and crowd management - hence has a social impact side.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {"ethical_concerns": "The plagiarism score as per Docoloc is 12% including references - which is acceptable. Check here:- https://www.docoloc.de/d88c4d5162dd4b21eeb6add446e2b5bbxto_4JX2Nc%7EfbDlilc/en/konto.hhtml?dogetresult=53\n"}}, {"review_id": "kQDVL6N0mDT", "reviewer": "Reviewer_z4Tm", "summary": "This paper proposed a novel deep learning-based architecture for the next POI prediction. The proposed approach models the temporal and semantic contexts through Transformer, extracts the social context through self-attention module, and model the geographical context via an auxiliary task. Experiments show that it can achieve better results than the SOTA next POI prediction.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:\nThis paper has novel design in the POI prediction model architecture. The key different is that the geographical context is captured through an auxiliary task instead of the manually designed kernels.\n\nQuality:\nThe technique is somewhat sound. The experiments are quite comprehensive, it compares the proposed approach with other 6 approaches on 3 different datasets.\n\nClarity:\nThe paper is well-written, most of the points are clearly addressed. Refer to the limitation comments for some points that are not well addressed.\n\nSignificance:\nThis paper seems to address a difficult task in a better way than the previous work. Future works can compare this work as a new POI prediction baseline.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations\n\n1. Some important details of model training are missing. For example, given the social context extractor is using the mobility feature extractor, how do we train them together? Is it possible that if we don't train them together from scratch, we could get get better results? Say, in the first several iterations, we train the mobility feature extractor first and then start from the well-trained mobility feature extractor to train the full model. Basically there are connections among different modules in the model, what's the best training recipe?\n\n2. What's the computation cost of the proposed approach, and compared with other approaches?\n", "review_text": "", "extra_scores": {"ethical_concerns": "No concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "bMr6BkGXTV", "reviewer": "Reviewer_5Ds5", "summary": "This paper proposes a Transformer-based context-aware network for human mobility prediction. Specifically, the authors explored the influence of four types of context in mobility prediction: temporal, semantic, social, and geographical contexts. A base mobility feature extractor is designed using the Transformer architecture, which takes both the history POI sequence and the semantic information as input. A self-attention module to model the influence of the social context. Moreover, the authors proposed to use an auxiliary task to model the geographical context and predict the next location. In the experimental results, the proposed method outperforms other state-of-the-art next POI prediction methods and illustrates the value of including different types of context in the next POI prediction.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is generally well written and easy to follow. The detailed comments are listed below.\n\n-- Originality: There seems two major novelty and contribution in this paper. One is to employ an auxiliary trajectory forecasting task to predict the geographical coordinates of the next most likely location. The other is to propose a novel consistency loss function to connect the predicted POI and the predicted location, which further improves the POI prediction performance. I think that these two ideas are interesting and novel. The authors provided an informative literature review on related work and discussed major differences between the proposed method and existing approaches.\n\n-- Quality and Significance: The authors conducted extensive experiments on three widely used datasets and demonstrated promising results, which can support the claims of the authors. The comparison with state-of-the-art baselines and ablation baselines is illustrated with sufficient details. \n\n-- Clarity: The paper is clearly organized and well written.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors discussed the limitations of the proposed method and provided some insights on the future directions for improving the current method. However, there seems no discussion on the potential societal impact.", "review_text": "", "extra_scores": {"ethical_concerns": "I do not see major ethical concerns in this paper.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.0 hours"}, "extra_text": {}}, {"review_id": "K2qipFv9NNh", "reviewer": "Reviewer_noAt", "summary": "In the submitted manuscript, the authors proposed a Transformer-based method to predict the next POI (place-of-interests) of a sequence. The model takes into account not only the temporal sequence of previously visited POIs but also semantic, geographical, and extracted social information. Moreover, an auxiliary trajectory prediction task is introduced to increase the performance of the main task.\nTo prove the efficacy of the proposed strategy, the model is tested against three publicly available datasets: Gowalla, FS-NYC, and FS-TKY. Thanks to the modeling of the additional context information, MobTCast is able to outperform state-of-the-art results by a 7.22% margin with respect to previously proposed architectures.", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Novelty:\n\nThe main originality of the paper is the usage of the auxiliary task of location prediction in addition to the POI prediction task. Otherwise, either the social/semantic context embedding or the application of transformer encoding are previously used in related works.\n\nQuality: \n\nThe paper provides a sufficient description of the proposed technique and the used functions. It also provides an experimental evaluation compared to previous work and ablation experiments to show the importance of the integrated parts in the model. However, the work has some weakness-points:\n- Nothing is delivering the performance of the auxiliary task; for example, how accurate is the predicted location or how close is it to the site of the predicted POI.\n- It is not clear how the results of the previous works in the comparison are obtained.\n- The paper reports a single number (7.22%) of improvement over the three used datasets without specifying the number of top predictions used (top-k). It would be more informative if the gain is computed for each value of k and each dataset separately.\n- The model explicitly include four context \"types\": temporal, semantic, social, and geographical. Following a data-driven approach, social links and neighbors are extracted from visited POIs. However, the same reasoning does not hold for the semantic and geographical context. What if they are not provided? It would be interesting to evaluate the model's robustness when making predictions using inferred semantic classes instead of ground-truth ones.\n\nClarity:\n\nIn general the paper is clear but it contains few writing typos and could benefit of some clarifications:\n- Line 270: average -> averaged\n- Figure 1 is a bit confusing. It could be improved using a lighter notation and a more symmetrical layout. The legend is also difficult to interpret, and coloured shapes should be put closer to their definitions.\n\nSignificance:\n\nFrom the reported results, the proposed model has noticeable an improvement over the compared previous work, but some clarification is required for how they obtained the previous work results.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "A related negative social impact to the POI prediction is the privacy of the users. The work did not discuss how the users' privacy is assured and in what context should this prediction model be used. ", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a transformer-based predictor for future places of interest. Its initial scores were 7, 6, 6, and 5; and they did not change during the discussion. The paper presents good empirical results. However, the reviewers were concerned that it is not very technically deep. I read the paper, and see both positives and negatives. Let me start with the positives:

1. Experiments in Table 1 are impressive. A lot of variability in baselines. Three datasets and top-$K$ for various $K$.

2. The ablation study in Table 2 shows that MobTCast performs well even without social context, and losses $L_2$ and $L_3$. In short, the loss $L_1$ is enough to be comparable or better than the state of the art. This indicates that the improvement is due to better sequence modeling using the transformer.

Now the negatives:

1. I am not sure what is the lesson learned from this paper. How would somebody who uses transformers benefit from this result? The paper is indeed not very deep.

2. I am not 100% sure that MobTCast improves over the state of the art because of the transformer. In particular, do any of the baselines use the semantic context in Section 4.1? If not, the observed improvement may be simply because of better features.

In my opinion, this paper should be judged as an application paper, and it clears the bar because of good empirical results. I strongly suggest that the authors take my comments into account, in addition to the detailed comments of the reviewers. Congratulations!
Decision: Accept (Poster)