S
UCCESSIVE POI R ECOMMENDATION VIA B RAIN -
INSPIRED
S PATIOTEMPORAL
A WARE
R EPRESENTATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Existing approaches usually perform spatiotemporal representation in the spatial and temporal dimensions, respectively, which isolates the spatial and temporal natures of the target and leads to sub-optimal embeddings.
Neuroscience research has shown that the mammalian brain entorhinal-hippocampal system provides efﬁcient graph representations for general knowledge.
Moreover, entorhinal grid cells present concise spatial representations, while hippocampal place cells represent perception conjunctions effectively.
Thus, the entorhinal-hippocampal system provides a novel angle for spatiotemporal representation, which inspires us to propose the SpatioTemporal aware Embedding framework (STE) and apply it to
POIs (STEP).
STEP considers two types of POI-speciﬁc representations: sequential representation and spatiotemporal conjunctive representation, learned using sparse unlabeled data based on the proposed graph-building policies.
Notably, STEP jointly represents the spatiotemporal natures of POIs using both observations and contextual information from integrated spatiotemporal dimensions by constructing a spatiotemporal context graph.
Furthermore, we introduce a user privacy secure successive POI recommendation method using STEP, and it achieves the state-of-the-art performance on two benchmarks.
In addition, we demonstrate the excellent performance of the STE representation approach in other spatiotemporal representation-centered tasks through a case study of trafﬁc ﬂow prediction problem.
Therefore, this work provides a novel solution to spatiotemporal aware representation and paves a new way for spatiotemporal modeling-related tasks.
I NTRODUCTION
With the rapid growth of location-based web services like Instagram and Yelp, there has been a seismic shift in how people interact with locations around them.
Through exploitation of Points-ofInterest (POIs) and their contexts, successive POI recommendation can beneﬁt users and businesses greatly.
As a core of POI information utilization, encoding POIs into vector representation space is of great signiﬁcance for advanced POI analysis and downstream applications.
Existing studies attempt to represent POI from different perspectives and collaborate with user preference modeling to achieve recommendation.
Since consecutive check-ins are usually highly correlated, naturally, sequence modeling approaches like the Markov chain model were used to capture the check-in sequential characteristics of POIs (Ye et al., 2011;
Liu et al., 2013; Zhang, 2014; Feng et al., 2015).
Employing tensor factorization technique, the works (Yang et al., 2017; Wang et al., 2018) modeled target users and POIs separately by interacted features for POI recommendation.
More recently, enlightened by neural networks’ success, recurrent neural nets were remolded to represent POIs and user preferences implicitly (Liu et al., 2016a;
Zhao et al., 2019; Zhu et al., 2017).
Considering the geographical attributes of POIs, researchers have used power-law distribution, Gaussian distribution, or hierarchical tiling methods to depict the geographical inﬂuence over POI distributional features (Ye et al., 2011; Lian et al., 2014;
Feng et al., 2017; Chang & Kim, 2020;
Luo et al., 2020).
However, geographical modeling methods above only provide single-scale or coarse-grained manually designed representations of POI geographical inﬂuences, which is deﬁcient in capturing the
POI-speciﬁc spatial features.
Also, the arbitrary modeling might even lead to over-parameterization.
While temporal dimension offers indeterminate auxiliary information for POI modeling, to utilize the POI temporal information within the check-ins, some works using time interval, time state variables or temporal transition vectors to promote the POI representing (Zhao et al., 2019; 2016; 2017;
Li et al., 2018; Manotumruksa et al., 2018; Zhao et al., 2020).
However, these methods focused on utilizing general temporal patterns among all POIs and failed to exploit the POI-speciﬁc visiting time patterns sufﬁciently.
Still, the POI-speciﬁc spatiotemporal characteristics were not adequately mined and utilized.
Inspirations.
The entorhinal-hippocampal system plays a central role in the mammal cognition architecture.
The Nobel Prize-winning neuroscience research (O’keefe & Nadel, 1978) demonstrated that entorhinal grid cells provide an effective multi-scale periodic spatial representation (Yuan et al., 2015; Banino et al., 2018; Mai et al., 2020;
Dang et al., 2021).
Moreover, the entorhinalhippocampal system is also critical for the non-spatial inference that relies on understanding the associations between perceptions from various perspectives (Whittington et al., 2018; Stachenfeld et al., 2018;
Whittington et al., 2020).
Some promising researches cast spatial and non-spatial problems as connected graphs and point out the cells inside entorhinal-hippocampal structure provide efﬁcient conjunctive representation for those graphs (Stachenfeld et al., 2018;
Gustafson & Daw, 2011).
As the representation mechanism in the entorhinal-hippocampal system was extensively studied, it is widely accepted that conjunctions of representations from different aspects form the hippocampal representation for relational memory (Whittington et al., 2018; 2020; Eichenbaum, 2017;
MacDonald et al., 2011; Sargolini et al., 2006).
For the general spatiotemporal aware embedding, various contexts can be constructed into afﬁnity graphs for latent representation learning.
Furthermore, strategies like conjunctive-representing in entorhinal-hippocampal structure can be translated to improve the quality of the representations (see Fig.1 left part).
In this paper, borrowing inspirations from the entorhinal-hippocampal system, we propose the SpatioTemporal aware Embedding framework, namely STE and apply to POIs (STEP) for successive
POI recommendation, the model architectures are shown in Figure 1.
Firstly, we build context graphs to enable unsupervised embedding learning on sparse check-ins.
Secondly, we employ a sequential model to represent POIs from the check-in sequence perspective.
Most importantly, we elaborate a spatiotemporal model consists of a grid-cell spatial encoder and a visiting time encoder to capture the POI-speciﬁc spatiotemporal characteristics.
The spatiotemporal model learns to get the POI spatiotemporal latent representations using the spatiotemporal context graph.
Finally, we implement successive POI recommendation systems based on the STEP and achieve high-performance using simple recurrent neural networks as recommenders.
This work’s main contributions are summarized as follows:
(1) Motivated by the graph-representing strategy of structural knowledge in the entorhinalhippocampal system, we solve the spatiotemporal aware embedding learning in a graph-based unsupervised learning manner through speciﬁc context graph building policies, especially the spatiotemporal context graph, to fully exploit rich unlabeled data.
(2) Inspired by the conjunctive representation mechanism in the entorhinal-hippocampal complex, we present a spatiotemporal model with a grid-cell spatial encoder and a time pattern encoder to utilize the spatiotemporal information.
The conjunctive representing approach based on a unique spatiotemporal context graph addresses the problem of previous spatiotemporal modeling methods in which spatial and temporal information are isolated and represented separately.
(3) We introduce a successive POI recommendation system by incorporating STEP and simple sequence predictors to show the feasibility of implementing speciﬁc applications based on the proposed STE framework.
We perform experiments on large real-world datasets to demonstrate the effectiveness of STEP, and our method outperforms baselines according to experimental results.
(4) Moreover, compared with classical recommendation systems, our POI-centered solution can avoid the ethical risks of artiﬁcial intelligence, like personal data leakage, as it does not need access to private information such as user preferences.
Furthermore, our framework can be applied to more valuable applications like wildlife preservation and urban trafﬁc scheduling as a general spatiotemporal aware modeling method.
P RELIMINARIES
[Figure]
Notation
Deﬁnition t ij j -th timestamp of p i t
i
Visiting time pattern matrix of p i e ispa
Spatial vector representation of p i e iseq
Sequential vector representation of p i e ist
Spatiotemporal conjunctive representation of p i e istep
STEP vector representation of p i ⊕
Tensor concatenation operation
Given a set of POIs with corresponding coordinates P = { p i } , p i =
( x i , y i ) , a check- in sequence is one set of continuous checkins of one user in one day, denoted as S j
= { ( p 1 , t 11 ) , ..., ( p n , t nm ) } .
Unlike previous works, we do not regard all check-in records of a user as one sequence since check-ins with relatively long intervals are not very relevant.
Although we assign notation to users for generality, the user information is not used in the training phase except to split sequences.
We deﬁne context graphs as graphs that encode context information as afﬁnity among POIs.
Various contexts in the check-in records can be easily built into graphs G p =
{ V p , E p } , where V p is the set of POIs and E p is the set of edges between adjacent POIs.
The edges in context graphs represent the correlation between neighboring POIs deﬁned by geographical distance, relative position in check-in sequences, or spatiotemporal adjacent criterions.
We summarize notations in this paper using Table 1.
Data description.
The Instagram Check-in dataset (Chang et al., 2018) was collected from Instagram in New York and the data was preprocessed in the same manner as previous works (Zhao et al., 2016; 2017).
The Instagram Check-in dataset has been pre-processed when it is made public, it includes 2 , 216 , 631 check-in records at 13 , 187 POIs of 78 , 233 users.
Check-in sequences are sorted by timestamps, the ﬁrst 70% are used as a training set and the remaining 30% for validation and testing.
The Gowalla dataset is a globally-collected large-scale social media dataset (Cho et al., 2011).
We eliminate users with fewer than ten check-ins and POIs accessed by fewer than ten users.
Then the check-in records are sorted according to timestamps and ﬁrst 70% check-ins are used for training and the remaining latest records for testing.
We perform vivid data analyses in the Appendix due to the space constraints.
S PATIO T EMPORAL AWARE E MBEDDING M ODEL OF POI S
We illustrate components of STEP: the sequential model in Sec. 3.1, the spatiotemporal model in Sec. 3.2, and state the STEP-based successive POI recommendation method in Sec.
We adopt a simple-minded (no-parameters) edge weighting policy for constructing all context graphs.
Weight A
i,j = 1 if vertices i and j is connected, this simpliﬁcation avoids the necessity of choosing edge-weighting parameters.
3. S EQUENTIAL M ODEL
The sequential sub-model represents POIs using context graph G seq .
Given one POI and its context in the check-in sequence, entry A i,j in adjacency matrix of G seq is 1 if p i , p j are within the same context window.
This is a common way to mine the sequential correlations of tokens like words (Mikolov et al., 2013) or POIs (Lim et al., 2020).
Our sequential model aims to predict true contextual POIs, i.e. , connected vertices in G seq .
Intuitively, minimizing the objective function over all target-neighbor pairs guarantees that POIs sharing similar sequential context will have shorter distances in embedding space (Hadsell et al., 2006).
To avoid the intractable summation over the whole context space, we follow the noise contrastive sampling approach (Gutmann & Hyv¨arinen, 2012; Mikolov et al., 2013) to get an approximated surrogate loss
[Equation]
where γ = 1 if ( p i , p j ) is a sequential neighboring pair and γ = − 1 if not, indicator I outputswhen the argument condition is true and otherwise 0 .
This unsupervised loss can also be seen as taking expectation with respect to the distribution P ( p i , p j , γ ) over P , which is conditioned on the
POI sequential context graph G seq .
3. S PATIOTEMPORAL M ODEL
In this section, we illustrate the POI spatiotemporal conjunctive embedding model in detail.
The proposed spatiotemporal model is composed of two key components: a POI spatial model and a POI visiting time encoder, an intuitive illustration can be found in Fig.
3.2. S PATIAL M ODEL
The spatial sub-model takes location observations ( x i , y i ) and spatial context graph G spa to produce spatial representations.
Grid-cell encoding.
Inspired by the multi-scale periodic representation of grid cells in mammals, we formulate our POI spatial contextual encoder to use sinusoidal and cosinusoidal functions of different scales to encode the raw locations of POIs in geographical space following previous works
Gao et al.
Mai et al.
Given a POI p i =
( x i , y i ) ∈
R 2 , the grid-cell model based encoder encodes the coordinates in 2-D Euclidean space into spatial latent representations in R d spa .
We denote the grid cell encoder based spatial embedding of POI p i as
[Equation]
where ψ
[Equation]
) is concatenated multi-scale representations of 6 S dimensions, S denotes the number of grid scales and φ represents fully connected non-linear layers.
Considering three unit vectors a 1 = [1 , 0] T , a 2 =
[ − 1 / 2 ,
[Equation]
3 / 2] T , a 3 = [ − 1 / 2 , −√ 3 / 2] T ∈ R 2 , at each frequency, position codes
[Equation]
are computed via
[Equation]
and ρ = ( λ max /λ min ) s/ ( S − 1) .
λ min and λ max are the minimum and maximum scale values, here we use S = 64 following the previous work (Mai et al., 2020) and set λ max = 1 km , λ min = 0 .
1 km .
Spatial-neighboring deﬁnition.
We project the coordinates in the geographical coordinate system
WGS84 to the projection coordinate system NAD27 to get locations of POIs in R 2 .
For each entry A i,j in adjacency matrix of spatial context graph G spa , we assign A i,j using the geographical distances.
Speciﬁcally, we computed the geographical distances between POIs and construct an undirected spatial context graph G spa with uniform edges among the top-ten closest POIs (nearest neighbors policy).
As the grid cell encoder can handle geographical distributions at different scales
(Mai et al., 2020), we do not use a speciﬁc radius ( (cid:15) -neighborhoods policy) to ﬁlter the neighboring
POIs to fully exploit the multi-scale representation capability.
The spatial graph construction process is related to Lim et al.
(2020), in which the edges are weighted according to average distances to enable graph attention computations.
Given a target POI p i , neighboring contextual POI set P + spa and negative set P − spa sampled from G spa , the unsupervised embedding learning can simply be maximizing the log-likelihood of observing the true context POIs.
We can formulate this target with negative sampling via a general objective function:
[Equation]
where ctx indicates the context graph type and ctx ∈ { seq, spa, st } in this work, σ is the sigmoid function and K denotes the number of samples in negative sample set P − ctx .
Following Eq.6, the loss function for the spatial context embedding model is:
[Equation]
3.2. S PATIOTEMPORAL C ONTEXT G RAPH C ONSTRUCTION
[Figure]
s
[Figure]
hu
[Figure]
t 1 Δ w k d
≠ t 2 -24-h t 2 -24+h t 2 +
+h t 2 +24-h t 2 -24-h t 2 -24+h t some times are ex
∆
For constructing spatiotemporal context graph G st , we want to mine the item-speciﬁc spatiotemporal conjunctions, so for each entry
A i,j of the adjacency matrix of G st , we assign A i,j =following the hierarchy of neighboring timestamps → temporal neighboring → spatiotemporal neghboring :
Neighboring timestamps.
Given an arbitrary timestamp pair ( t 1 , t 2 ) , time interval ∆ t (cid:44) | t 1 − t 2 | and ∆ wkd (cid:44) | wkd ( t 1 ) − wkd ( t 2 ) | where wkd ( t )
= 1 if t is weekend else 0.
For one time t , its temporal-neighboring timestamps are those within the neighborhood window and satisfy ∆ wkd = 0 , as shown in Fig
h is a hyper-parameter indicates temporal context window width and h ∈ (0 , 24) hours.
POI temporal neighboring.
POI p
i and p j with corresponding visiting timestamp sets T i =
{ t i 1 , t i 2 , . . . } and T j = { t j 1 , t j 2 , . . . } .
The number of neighboring timestamp pairs ( t i , t j ) > m , where t i ∈ T i , t j ∈ T j , m is a threshold.
POI spatiotemporal neighboring.
If p i , p j are spatial and temporal neighboring, they are spatiotemporal neighboring.
It is redundant for individual POI-speciﬁc temporal modeling since solely relying on time information, we are not able to recommend reasonable candidate POIs (visits may take place contemporarily all over the world).
Thus, time is regarded as a supplementary dimension of basic spatial information, our spatiotemporal context graph provides an effective way to combine the POI-speciﬁc temporal and geographical information.
3.2. V ISITING T IME P ATTERN E NCODING
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
on the scale of year but ﬁckle on the hour and date scale.
In our accessing time pattern encoding 217 scheme, visits of POIs are counted by hour and date, after this procedure, every POI has a visitingtime pattern represented by a raw matrix t 0 2 R 24 ⇥ 366 .
The raw accessing time pattern matrices t 0 219 are normalized and applied Gaussian kernel for smoothing, this process also augment the accessingtime data in a reasonable way.
The ﬁnal matrix representation t can preserve valuable information 221 of day-night, weekday-weekend, and seasonal alternatives for further temporal context processing.
Some typical accessing time patterns matrices are visualized in Figure 4 using heatmaps.
3.4.4 Spatial-Temporal Embedding Learning
To conduct the POIs’ spatial-temporal conjunctive representation learning, given POI p
i and corre- 225 sponding visiting time pattern matrix t i , the spatial-temporal conjunctive representation is deﬁned as
[Equation]
where  indicates artiﬁcial neural network layers and e st 2 R d st .
The basic spatial representation can 228 be well preserved while the visiting time patterns can be combined effectively.
For a target POI p i , weimplement the spatial-temporal conjunctive representation learning by maximizing the log-likelihood 230 of observing true spatial-temporal neighboring POIs Pr ( p j | p i ) , the loss function is deﬁned as:
[Equation]
(  e kst · e ist )) , (10)
[Table]
d st = d seq ) thus provides more ﬂexibility.
To perform the 253 successive POI recommendation task, we adopt two-layer simple networks based on GRU unit[39] 254 and LSTM unit [40] as the recommender model.
Parameters in the embedding models are optimized 255 according to corresponding loss function
L ( ✓ ⇤ )
=
L ⇤ + ↵ || ✓ ⇤ || 22 , (13) where L ⇤ 2 {L seq , L st } , ✓ ⇤ 2 { ✓ seq , ⇥ st } , ⇥ st = { ✓ spa ,✓ time ,✓ st } .
↵ is the weighting factor of 257 the 2-norm regularizer.
The predictor is then trained with the pre-trained STEP embedding model for 258 the next POI recommendation task.
Given a n -length check-in sequence S j , corresponding STEP on the scale of year but ﬁckle on the hour and date scale.
In our accessing time pattern encoding 217 scheme, visits of POIs are counted by hour and date, after this procedure, every POI has a visitingtime pattern represented by a raw matrix t 0 2 R 24 ⇥ 366 .
The raw accessing time pattern matrices t 0 219 are normalized and applied Gaussian kernel for smoothing, this process also augment the accessingtime data in a reasonable way.
The ﬁnal matrix representation t can preserve valuable information 221 of day-night, weekday-weekend, and seasonal alternatives for further temporal context processing.
Some typical accessing time patterns matrices are visualized in Figure 4 using heatmaps.
3.4.4 Spatial-Temporal Embedding Learning
To conduct the POIs’ spatial-temporal conjunctive representation learning, given POI p
i and corre- 225 sponding visiting time pattern matrix t i , the spatial-temporal conjunctive representation is deﬁned as
[Equation]
where  indicates artiﬁcial neural network layers and e st 2 R d st .
The basic spatial representation can 228 be well preserved while the visiting time patterns can be combined effectively.
For a target POI p i , weimplement the spatial-temporal conjunctive representation learning by maximizing the log-likelihood 230 of observing true spatial-temporal neighboring POIs Pr ( p j | p i ) , the loss function is deﬁned as:
L ( ✓ time ,✓ st ) =
X p
[Equation]
2P X p j 2P + st (log  ( e jst · e ist )
+ 1 K X p k 2P
st log  (  e kst · e ist )) , (10) where P + st isthesetPOIsinsidespatial-temporalneighborhoodof p i whereas P  st isthesetofnegative 232 POIs,  is the sigmoid function and K is the number of negative samples.
During the optimization 233 procedure, the POI spatial model is jointly optimized as a sub-model of spatial-temporal model, the 234 full objective of the spatial-temporal model is
L st ( ✓ spa ,✓ time ,✓ st ) =
L ( ✓ time ,✓ st ) +  spa L spa ( ✓ spa ) , (11)  spa is a weighting factor for preserving the
[Table]
information during the
[Table]
-temporal 236 modeling.
We
[Table]
sample a batch of
[Table]
context P
[Table]
spa to optimize the
[Table] [Table]
to preserve
[Table]
Next we
[Table]
st to 238 optimize
[Table]
to
[Table]
We repeat above 239 procedures for
[Table] [Table]
=
[Table]
to
[Table]
spa .
We 240 update all parameters ( ✓ spa ,✓ time ,
[Table]
st ) of
[Table]
model in iterations until the overall loss
[Table]
st converges.
[Table]
Overall framework 243 WepresenttheSTEPrecommendationsystemindetailinthissection, weshowthearchitecturesketch 244 in Figure 2.
Taking the check-in records as input,
we
[Table]
graphs G seq , G spa , G t , G st and 245 feed the check-ins into the
[Table]
model,
[Table]
and
[Table]
model to perform
[Table]
latent factor modeling (embedding
[Table]
) on
[Table]
The learned
[Table]
are smoothed 247 according to corresponding
[Table]
to preserve
[Table]
among
[Table]
Then,
[Table]
merged and fed into
[Table]
to generate a
[Table]
list based on
[Table]
249 scores.
We use concatenation operatione STEP
= e seq  e st (12) to merge the all embeddings from the sequential and spatial-temporal models.
This merging method 251 can preserve information from different spaces without extra-parameters and not requires the embed- 252 dings to be in the same dimension ( e.g. d st = d seq ) thus provides more ﬂexibility.
To perform the 253 successive POI recommendation task, we adopt two-layer simple networks based on GRU unit[39] 254 and LSTM unit [40] as the recommender model.
Parameters in the embedding models are optimized 255 according to corresponding loss function
L ( ✓ ⇤ )
=
L ⇤ + ↵ || ✓ ⇤ || 22 , (13) where
[Table]
⇤
[Table]
L seq , L st } , ✓ ⇤ 2 { ✓ seq ,
[Table]
st } ,
[Table]
st = { ✓ spa ,✓ time ,
[Table]
st
[Table]
↵ is the weighting factor of 257 the 2-norm regularizer.
The predictor is then trained with the pre-trained STEP embedding model for 258 the next POI recommendation task.
Given a n -length check-in sequence S j , corresponding STEP
[Figure]
We develop a visiting time pattern encoding method to tensorise the temporal observations (visiting records in timestamps t ) to provide observation inputs for spatiotemporal model.
Unlike previous works (Liu et al., 2016a; Zhao et al., 2019; Zhu et al., 2017; Zhao et al., 2016), we focus on the POI-speciﬁc temporal patterns rather than general temporal characteristics among all timestamps.
Compared with previously used time interval-based or hard-coded methods, our encoding scheme can tensorise the item-speciﬁc temporal information more precisely and able to provide a reliable decision basis for spatiotemporal modeling.
Visiting time encoding.
Since the visiting time patterns are relatively stable on the scale of year but ﬁckle on the hour and the date scale, in our encoding scheme, visits are counted by the check-in timestamps to raw matrices t (cid:
) ∈
R 24 × 366 .
Then, the raw matrices t (cid:48) are normalized and applied Gaussian kernel for smoothing, this process also reasonably augment the
POI accessing time data.
The visiting time encoding process is shown schematically in Fig.
The ﬁnal matrix representation t = smooth ( norm ( t (cid:48) )) retains the ﬁne-grained check-in time patterns as well as rough item-speciﬁc visiting time features.
3.2. S PATIOTEMPORAL C ONJUNCTIVE E MBEDDING L EARNING
We formulate the spatiotemporal conjunctive representation as:
[Equation]
where φ indicates fully-connected layers.
Follow the formulation in Eq.6, we implement the spatiotemporal conjunctive representation learning by minimizing:
[Equation]
We sample P + st and P − st from G st , where P + st is the set of spatiotemporal-neighboring POIs whereas
P − st is the set of negative POIs.
During the optimization procedure, the spatial model is jointly optimized as a sub-model of spatiotemporal model, the full objective of the spatiotemporal model is
[Equation]
[Equation]
λ spa is a weighting factor for preserving the spatial context information during the spatiotemporal modeling.
We ﬁrst sample a batch of spatial context G spa to optimize the spatial context loss L spa to preserve geographical context.
Next we sample a batch of spatiotemporal context G st to optimize the spatiotemporal loss L st to preserve the spatiotemporal context.
We repeat above procedures for
I
and I 1 =
I
/λ spa iterations respectively to approximate the balancing factor λ spa .
We update all parameters { θ spa , θ time , θ st } of spatiotemporal model in iterations until the overall loss L st converges.
3. S UCCESSIVE POI R ECOMMENDATION WITH STEP
We present the STEP (STE of POIs)-based recommendation method in detail in this section.
Taking the spatiotemporal data as input, we construct context graphs G and feed the observations (locations and time patterns) into the STEP model to perform embedding learning.
The embeddings are smoothed according to corresponding context graphs to preserve contextual information.
Then,
POI vector representations (embeddings) e seq , e st are merged as spatiotemporal aware embedding e step and fed into the recommender to generate an estimated embedding ˆ e .
Speciﬁcally, we use concatenation
[Equation]
to merge the two sequential and spatiotemporal embeddings.
This merging policy can preserve information from different spaces without extra-parameters and not requires the embeddings to be in the same dimension ( e.g. d st = d seq ) thus provides more ﬂexibility.
We adopt two-layer recurrent networks as the recommender model.
Embedding model optimizing.
Parameters in the STEP embedding model are optimized according to corresponding loss function L ( θ ∗ ) =
L ∗ + α ||
θ
∗
|| 2 , where L ∗ ∈ {L seq , L st } , θ ∗ ∈ { θ seq , Θ st } , Θ st = { θ spa , θ time , θ st } .
α is the weighting factor of the 2-norm regularizer.
Recommender model optimizing.
The predictor is then optimized with the pre-trained STEP embedding model for the next POI recommendation task.
During the training phase, given a n -length check-in sequence S j , we can get corresponding STEP embedding series of POIs { e (1) step , . . .
, e ( gt ) step } , the last POI is regarded as the recommendation target.
The target of the recommender is to predict latent representation ˆ e step similar to the embedding of true successive POI e ( gt ) step , formally described as:
[Equation]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
− − − −
[Table]
[Table] [Table]
[Table] [Table]
− − − −
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
The objective function of recommender is:
[Equation]
where σ (cid:48) =
exp( LeakyReLU ( · )) and sim ( · , · ) = a · b || a ||·|| b || .
During the testing phase, we compute the cosine similarity scores to rank the candidate POIs to generate
[Table]
lists.
E XPERIMENTS
[Table]
We perform the successive POI recommendation task on the
Instagram Check-in dataset, Gowalla dataset, and the trafﬁc ﬂow forecasting task on TaxiBJ15 and TaxiBJ.
4. S UCCESSIVE POI R ECOMMENDATION T ASK
Metrics.
During the system inferencing phase, the recommendation system recommends a POI list according to the estimated scores of candidate POIs for every trial sequence.
We apply widely-used metrics HIT@K (if the ground truth is within the top-k of the list, a score of 1 is awarded, else 0), k = 1 , 5 , 10 and MRR (Mean Reciprocal Rank) for evaluation.
These metrics reﬂect different aspects of the recommendation lists, HIT@K measures the rate of valid recommendation among all trials, whereas MRR scores the quality of the entire recommendation list.
Hyper-parameter settings.
We set the hyper-parameters of our proposed method to the following default values.
We set context window size in the POI sequential model to 2 and adopt h = 2 , m = 11 for building G st .
We utilize Adam optimizer with batch size 512 , β 1 = 0 .
9 , β 2 = 0 .
and set the initial learning rate to 0 .
001 followed by a reduce-on-plateau decay policy, the decay factor is 0 .
1 during the training.
Weighting factors α, λ spa are set to 1 × 10 − 4 , 0 . 2 and the embedding dimensions { d seq , d spa , d st } are set to { 32 , 64 , 96 } .
We compare the STEP-based successive POI recommendation method with representative methods:
Embedding-recommender methods.
We choose six methods consisting of three embedding models and two recommenders.
For the embedding model, we use the following embedding models.
(1) Random , (2) Skip-Gram (Liu et al., 2016b), (3) CAPE (Chang et al., 2018) and (4) Geo (Mai
H I T @
on I n s .
H I T @ 1 on G o w .
( % ) λ
= 0 .
1 λ = 0 .
2 λ = 0 .
4 λ = 0 .
8 λ = 0 .
1 λ = 0 .
2 λ = 0 .
4 λ = 0 .
M RR on I n s .
M RR on G o w .
H I T @
on I n s .
H I T @ 1 on G o w .
( % ) Instagram
Gowalla h = 1 h
= 2 h =
h
=h = 1 h = 2 h = 4 h = 8 0.
M RR on I n s .
M RR on G o w .
H I T @
on I n s .
H I T @ 1 on G o w .
( % ) m =
3 m = 7 m = 11 m =m = 3 m = 7 m = 11 m = 15 0.
M RR on I n s .
M RR on G o w .
different random seeds.
et al., 2020).
For the recommender model, two-layer networks based on (1) GRU unit (Merri &
Fellow, 2014) and (2) LSTM unit (Hochreiter & Schmidhuber, 1997) are used.
One-stage methods.
We choose representative one-stage methods as baselines.
(1) ST-RNN (Liu et al., 2016a).
STGN (Zhao et al., 2019), a LSTM variant which models visit preferences with time and distance considerations, and the improved variant STGCN .
(3) LSTPM (Sun et al., 2020) is a LSTM-based method.
(4) STP-DGAT and STP-UDGAT (Lim et al., 2020) are spatial-temporal-preference user dimensional graph attention networks.
Comparison results.
According to results in Table 5, our method outperforms the baselines by signiﬁcant margins on both datasets, and the gains in recommendation accuracy are especially substantial on the Instagram dataset with rich temporal information (according to Table 3, POIs in
Gowalla have less visiting timestamps).
The one-stage recurrent network-based methods, such as
ST-RNN , LSTPM surpassing basic embedding-based methods by more sufﬁcient exploitation of userpreference spatiotemporal properties.
However, these methods remain noncompetitive with STEPbased ones, although the STEP stands without user preference consideration.
The advantageous performance of our method over the competitors can be attributed to its efﬁcient use of the item-speciﬁc spatiotemporal nature.
We observe signiﬁcant performance improvements of the STEP-based methods in terms of MRR, indicating the STEP-based methods provide better candidates lists on both datasets, beneﬁt from the efﬁciency of the proposed spatiotemporal aware embedding model.
The usage of LSTM units in recommender slightly improves the performance compared with basic RNN cells because of their advantages in gate functions of recurrent connections.
Moreover, the basic
RNN recurrent net equipped with STEP embedding can also outperform one-stage SOTAs.
This also proves the effectiveness of our brain-inspired spatiotemporal aware embedding model.
Effect of hyper-parameters.
We study the effect of newly-introduced hyper-parameters h , m , λ spa , and report the results using HIT@1 and MRR in Fig.4.
h
and m control the sparsity of the spatiotemporal context graph and λ spa regulates the importance of the spatial context smoothness term in the spatiotemporal model objective function.
We alter h to build spatiotemporal context graphs with decreasing sparsity as larger h corresponds to coarse temporal-neighboring condition.
The increasing h within a certain range results in performance improvements but appears detrimental to the precise top-1 recommendation.
The best performance (determined by MRR) is obtained when h = 2 .
The choice of h is task-related, according to our results on two dataset, h = 2 can be a good initial value for POI recommendation according to our results.
This value can be further adjusted for different application scenarios or datasets to build spatiotemporal context graphs with desired sparsity.
We set m from 3 to 15, larger m leads to a sparser spatiotemporal context graph.
We observe the performance slightly changes after increasing the threshold m , when m = 11 , the best performance is obtained.
We also investigate the effect of the balancing factor λ spa for spatiotemporal model training, the recommendation system achieves the best performance when λ spa = 0 .
2 and further increases only bring minor improvements, thus we select 0.2 as a default value in this work, this also helps reduce unnecessary iterations during the model training.
Ablation study.
We study the effectiveness of STEP modules by performing successive POI recommendation task with LSTM recommender, the method using standard STEP embedding model is referred as F ULL .
After the removal of (A) POI-speciﬁc time information processing module, the spatiotemporal model degenerates to a spatial model.
According to Table 4, the method performs worse without (A) on both datasets.
According to Table 3, POIs in Gowalla have sparser speciﬁc observation t s and contextual information from G st .
This results in more signiﬁcant performance
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Table]
improvements on the Instagram set than on
Gowalla.
We replace POI visiting time pattern matrices t (applying Sec.3.2.3) with random-initialized matrices in R 24 × 366 .
We note that the use of (B)
POI visiting time encoder improves the recommendation performance on both datasets, and the improvement are positively correlated with the temporal information abundance of the dataset.
We use a one-layer neural network location encoder ψ (cid:48)
( x, y ) to replace (C) grid-cell encoder in the STEP to demonstrate its effectiveness.
Results in Tabledemonstrate that the grid-cell encoder improves the quality of STEP representation and leads to better successive POI recommendation performances on both datasets.
We observe noticeable performance drops after the removal of (D) spatial embedding preserving in Table 4 since the spatiotemporal conjunctive representation integrate spatial and temporal attributes at the cost of spatial information loss (due to the dimensional reduction).
The use of (D) spatial embedding information preserving alleviate this problem to a certain extent.
A simple LSTM recommender can achieve competitive recommendation performance without check-in sequential information consideration (Table 4 only(E)), demonstrating the effectiveness of the spatiotemporal model in STEP.
As visiting sequential information provides relatively coarsegrained POI depictions, the removal of the sequential model even leads to HIT@1 improvement on the Instagram dataset.
Also, the metric fallen on Gowalla (-18.6%, -27.8%, -32.4%, -26.2%) is more signiﬁcant than those on Instagram set (+3.1%, -11.
%, -15.0%, -6.0%), exactly opposed to their temporal information abundance, quantitatively evaluated by average timestamps per POI.
4. P ERFORMING T RAFFIC F LOW F ORECASTING TASK WITH STE
To demonstrate the generalizability of STE, we perform trafﬁc ﬂow forecasting task with the proposed spatiotemporal embedding methods.
The relevant content is presented in detail in Appendix.
A.
C ONCLUSION AND D ISCUSSION
In this paper, we propose the spatiotemporal aware embedding framework STE and apply it to POIs (STEP).
To the best of our knowledge, this is the ﬁrst work that translating entorhinal-hippocampal representing mechanisms to the spatiotemporal embedding.
Inspired by the graph-representing policy in brain entorhinal-hippocampal system, STEP captures sequential and spatiotemporal representation from unlabeled sparse data through context graph building and graph-based embedding learning.
Moreover, STEP provides a highly efﬁcient spatiotemporal model motivated by grid cells’ multi-scale spatial representation and place cells’ conjunctive representation, which overcomes the problem caused by frequently used separate-representing.
STEP-based successive POI recommendation method outperforms baselines and SOTAs on two real datasets without user preference invasion.
Furthermore, this work presents a practical framework for effective spatiotemporal aware modeling of general items, enabling more valuable spatiotemporal-related tasks such as wildlife preservation and urban trafﬁc management.
A C ASE S TUDY : T RAFFIC F LOW F ORECASTING WITH STE
As an essential basis of trafﬁc scheduling, accurate trafﬁc ﬂow forecasting is of great signiﬁcance.
In this part, we perform trafﬁc ﬂow forecasting with the proposed STE, which is abbreviated as
STE-TG (STE of Trafﬁc Grid).
A.
[Figure]
Following previous works (Hoang et al., 2016; Zhang et al., 2016), we partition a city into an M × N trafﬁc grid map based on the longitude and latitude where a trafﬁc grid represents a geographical region.
For a grid ( m, n ) that lies at the m th row and the n th column, two types of trafﬁc ﬂows at k th timestamp are considered, namely inﬂow and outﬂow, deﬁned as
[Equation]
[Equation]
(15) where Tr k means the trajectory at the k th timestamps, g i indicates the geographical coordinate, g i ∈ ( m, n ) means point
g i lies within grid ( m, n ) and | · | means the cardinality of a set.
At timestamp k , the trafﬁc ﬂow of grid ( m, n ) is represented as X m,nk = ( x in,m,nk , x out,m,nk ) .
The goad of trafﬁc ﬂow forecasting is: given X k , k ∈ { 0 , 1 , ..., t − 1 } , predict X t .
A. M ETHODOLOGY
As illustrated in Fig.1 in the main text, the input of STE-TG is composed of observations (coordinates and visiting time patterns) and contexts (context graphs).
Unlike STEP, as there is no sequential trafﬁc information, the sequential model is not applicable.
STE usage Following Sec.3.2.1 in the main body of the article, we use spatial observation and build a spatial context graph in the same way.
Speciﬁcally, same hyper-parameter settings of
S, λ min/max is adopted in grid-cell encoder, and top-10 policy is used for building G spa .
Also, the embedding objective of the spatial sub-model is still the term in Eq.7.
Different from POIs, the timestamps of the trafﬁc grid ﬂow data are collected using a standard time interval.
Thus both the encoding of visiting time patterns and the construction of spatiotemporal context graph could be simpliﬁed.
Notably, we generate the visiting time pattern matrix as the temporal observation input for the spatiotemporal model in the same way we did for POIs, and since the timestamp is standard, we skipped the accumulation step in Fig.3.
For constructing G st , we deﬁne the grid temporal adjacency using a speciﬁed distance function d ( · , · ) which is calculated by the normalized trafﬁc ﬂow difference value of two trafﬁc grids.
[Equation]
We use the top-10 strategy to get the temporal neighboring relations of grids with low distances.
Together with the spatial context information G spa , G st is built to help the embedding model preserve the spatiotemporal conjunctions between the trafﬁc grids.
The objective function used to optimize the spatiotemporal model remains the same as in the main text .
Predictor Design The ﬂow of one grid is signiﬁcantly related to the nature of the grids around it because an outﬂow from one region can lead to an inﬂow to neighboring regions.
In this work, the spatiotemporal natures of grids and correlations between grids are encoded in the embedding vector e stetg .
Thus, Given a query X m,nt − 1 , spatiotemporal embeddings of four geographical closest grids and grid ( m, n ) itself will also be fed into predictor f θ to produce the estimation
[Equation]
(17) where e m,n = ( e m,nstetg , e neighstetg ) and e neighstetg indicates embeddings of four spatial neighbors of grid ( m, n ) .
We use a simple three-layer MLP f θ with ReLU activations for implementation, the dimension of hidden state is 256.
The loss function used is mean squared error
||
ˆ X − X
|| 22 .
A. E XPERIMENTS
Following the same pipeline we used in POI recommendation task, we ﬁrst train the spatiotemporal embedding model STE-TG and then optimize the recommender model on the training set.
All models are optimized using Adam optimizer with default β s, the learning rate is set to 1 × 10 − 3 .
The embedding dimensions { d spa , d st } are set to { 64 , 96 } .
[Table] [Table]
Metrics We measure the trafﬁc ﬂow forecasting performance by Root Mean Square Error (RMSE) as
[Equation]
where ˆ x and x are the predicted value and ground truth, respectively and t is the number of all queries.
Datasets We perform experiments on two datasets:
• TaxiBJ
(Zhang et al., 2016) consists of calculated inﬂow/outﬂow based on taxicab GPS data collected in Beijing city in 2015.
• TaxiBJ (Zhang et al., 2017) is an extended version including data from 2013 to 2016.
For TaxiBJ15, data from the last week is used for testing.
For TaxiBJ, we choose the last four weeks as the testing data.
Baselines We choose representative baselines for comparison:
• HA: a predict method by the average value of historical values.
• ARIMA (Auto-Regressive Integrated Moving Average) • SARIMA: seasonal ARIMA • VAR (Vector Auto-Regressive) • ST-ANN:
It ﬁrst extracts spatial (nearby 8 regions’ values) and temporal (8 previous time intervals) features, then fed into an ANN.
• DeepST (Zhang et al., 2016): a deep neural network (DNN)-based prediction model for spatiotemporal data, which shows state-of-the-art results on crowd ﬂows prediction, we only consider the most potent variant, i.e. DeepST-CPTM.
• ST-ResNet (Zhang et al., 2017)
ST-ResNet models citywide trafﬁc ﬂow at different times into 2D images to perform prediction.
• LinUOTD (Tong et al., 2017): a linear regression method with a spatio-temporal regularization.
• ConvLSTM (Xingjian et al., 2015): a LSTM variant with convolution modules.
Results We listed the comparison results in Table.5 and Table.6.
In STE-TG, only the grid map and historical trafﬁc ﬂow are used to perform ﬂow prediction, while existing works usually take meta-data like weather into account.
However, we observe that the hastily constructed trafﬁc ﬂow prediction methods STE-TG achieve competitive performance, which also demonstrates the effectiveness of the proposed STE framework.