# Network Embedding-Aware Point-of-Interest Recommendation in Location-Based Social Networks

Authors: Lei Guo; Haoran Jiang; Xiyu Liu; Changming Xing
Journal: Complexity
 (2019)

---

## Abstract

As one of the important techniques to explore unknown places for users, the methods that are proposed for point-of-interest (POI) recommendation have been widely studied in recent years. Compared with traditional recommendation problems, POI recommendations are suffering from more challenges, such as the cold-start and one-class collaborative filtering problems. Many existing studies have focused on how to overcome these challenges by exploiting different types of contexts (e.g., social and geographical information). However, most of these methods only model these contexts as regularization terms, and the deep information hidden in the network structure has not been fully exploited. On the other hand, neural network-based embedding methods have shown its power in many recommendation tasks with its ability to extract high-level representations from raw data. According to the above observations, to well utilize the network information, a neural network-based embedding method (node2vec) is first exploited to learn the user and POI representations from a social network and a predefined location network, respectively. To deal with the implicit feedback, a pair-wise ranking-based method is then introduced. Finally, by regarding the pretrained network representations as the priors of the latent feature factors, an embedding-based POI recommendation method is proposed. As this method consists of an embedding model and a collaborative filtering model, when the training data are absent, the predictions will mainly be generated by the extracted embeddings. In other cases, this method will learn the user and POI factors from these two components. Experiments on two real-world datasets demonstrate the importance of the network embeddings and the effectiveness of our proposed method.


---

## Body

## 1. Introduction

With the popularity of mobile devices and the development of global positioning system, location-based social networks (LBSNs) have emerged in recent years (e.g., Gowalla (https://en.wikipedia.org/wiki/Gowalla) and Foursquare (https://foursquare.com/)), which enable people to share their experiences with friends and check in at interesting places (e.g., restaurants, tourist spots, and stores). POI recommendation as one of the important techniques to explore interesting unknown places has been well studied.Compared with traditional recommendation tasks, POI recommendations are challenging for three reasons. First, POI recommendation methods often suffer from the data scarcity and user/item cold-start problems, which means only a small number of users have sufficient check-in data, and most of the users have very little or even no check-in data (as shown in Figure1). In our experiments, the data scarcity is more than 99%. Second, in LBSNs, the check-in data are not explicit but implicit, which makes POI recommendation more difficult. In check-in data, only positive behaviors can be observed, from which we can only get the POIs that the users have visited, but cannot obtain the POIs that the users do not prefer. The missed check-ins are mixed of actually negative samples and undiscovered but potentially positive POIs. We cannot learn the user interests directly from this mixed data by utilizing a common approach, since they are unable to be differentiated from these two levels anymore. Third, users and POIs have different types of context information, e.g., social relationships of users and geographical information of POIs. It is important to exploit context information to improve recommendation accuracy. For example, in real life, we usually turn to our friends for POI recommendation. The social relationships among users help them locate the POIs that they are potentially interested in. In our real life, geographical information is another important type of context, which is not available in conventional recommendation task. As users are more likely to visit nearby POIs, the geographical information can help them locate the POIs that have neighborhood relationship with their preferred places.Figure 1
The histogram of users’ visited POIs: (a) Gowalla and (b) Yelp.
(a)
(b)How to overcome these challenges to further improve the recommendation performance has become one of the hot spots in recent studies. For example, Ye et al. [1] exploited the user preference and social and geographical influence for POI recommendation. In their work, a power law distribution is utilized to model the discovered spatial clustering phenomenon. Cheng et al. [2] captured the geographical influence by a multicenter Gaussian model and then fused it with matrix factorization (MF) and social influence for POI recommendation. Different from the work in [1, 2], a kernel density estimation approach was proposed by Zhang and Chow [3] to model the users’ visiting behaviors. To deal with the implicit check-in data, Lian et al. [4] first introduced the weighted matrix factorization (WMF) method as their basic framework, which fits the positive user activities by utilizing larger weights and unobserved user behaviors by utilizing smaller weights. Then, an augmented factor model was proposed to incorporate the geographical influence. Instead of directly fitting zero check-ins, Li et al. [5] treated POI recommendation as an item ranking problem and learned the latent factors by optimizing an ordered weighted pair-wise classification criterion, which further considers the ranking task as a set of classification problems. Ying et al. [6] exploited the recommendation performance in another famous ranking criterion, i.e., the Bayesian personalized ranking (BPR) method [7], and proposed a pair-wise-based ranking method.Although the above methods can work well in many recommendation scenarios, most of them only model the context information as regularization terms. The effectiveness of the deep information hidden in the network structure has not been fully exploited. On the other hand, deep learning-based embedding technique has shown its power in many recommendation tasks with its capability of extracting hierarchical representations from raw data [8]. For example, Wang et al. [9] applied the embedding model learned from the sequential behavior into next basket recommendation problem and proposed a two-layer hierarchical representation model. Zhao et al. [10] utilized the recurrent neural networks to learn the feature presentations of both users and products from an e-commerce website and proposed a feature-based matrix factorization method to leverage the learned features. However, most of these existing works mainly learn the embedding models of users and items from sequential behavior and content information, and little work has been done to leverage the representations deduced from social networks and location neighborhood relationships to deal with the cold-start POI recommendation. For example, Grover and Leskovec [11] proposed a novel method node2vec for learning continuous feature representations of nodes in networks. But their work did not investigate the recommendation problem, and the learned embeddings cannot be directly utilized to collaborative filtering method.Based on the above observations, we propose a network embedding-based POI recommendation method, which consists of an embedding model and a collaborative filtering model. In the embedding model, a neural network, i.e., the node2vec model [11], is pretrained to learn the user and POI representations from the corresponding social network and a predefined location network, respectively. In the collaborative filtering model, to deal with the implicit check-in data, the ranking-based criterion is first introduced, which learns the user and POI factors by ranking the items correctly. Then, the outputs of the node2vec are treated as the high-level representations of social and location context, and they are further used as the priors of latent feature factors in the collaborative filtering model. In experiments, we evaluate our method in two real-world datasets [12], and the results indicate that our method is more effective than other related state-of-the-art recommendation methods.The main contributions of this paper are as follows:(i)
We propose a network embedding-based POI recommendation method, which formulates the POI recommendation task as an item ranking problem, and considers the influence of social networks and geographical information by leveraging the capability of neural networks in extracting high-representative features from predefined networks(ii)
We incorporate the learned embeddings as the priors of the latent feature factors in collaborative filtering model to enable our method can consider the network information and collaborative filtering information simultaneously(iii)
We conduct extensive experiments on two real-world datasets to demonstrate the superiority of our proposed methodWe organize the remainder of this paper as follows: Section2 reviews and discusses the related works to our study. Section 3 describes the POI recommendation problem and proposes a ranking-based recommendation framework with pretrained network embeddings. Section 4 shows the experimental results on two real-world datasets. Section 5 concludes this work and points out the future work.


## 2. Related Work

In this section, some works related to our studies are investigated, which include the context-based recommendation, implicit feedback-based recommendation, and network embedding-based recommendation.


### 2.1. POI Recommendation with Social and Geographical Context

In LBSNs, social and geographical context plays an important role in helping users explore the unknown locations [12–17]. Motivated by the intuitions that users are more inclined to visit the places that their friends have suggested, and nearby locations tend to be visited by similar users, the social and geographical influence has been utilized to improve the POI recommendation. Cho et al. [18] studied the relation between human geographic movement, its temporal dynamics, and ties of the social network. They found that people generally move periodically within a bound region but occasionally travel long distance due to the impact of the social relationships. Ye et al. [1] first modeled the spatial clustering phenomenon by a power law distribution. Then, a unified POI recommendation framework was proposed, which fuses user preference to a POI with geographical influence and social influence. Cheng et al. [2] first captured the geographical influence by utilizing the multicenter Gaussian model and then fused it with the social influence in a generalized matrix factorization framework. Unlike previous proposed methods, which model the user check-in activities universally, Zhang et al. [19] proposed a personalized geographical location recommendation framework, where each user is modeled as a personal distribution. Li et al. [20] improved the POI recommendation by leveraging the information of friends in a two-step framework, where a set of potential locations that each user’s friends have checked in were first learned, and then three types of check-ins were incorporated into the matrix factorization model. Liu et al. [21] proposed a general geographical probabilistic factor model to jointly learn both geographical preference and user interest preference, where a Poisson-based probabilistic generative process is utilized for the recommendation model. Wang et al. [22] constructed a graph to model users and their social relations, which models the user preference and social influence by utilizing different types of edges. To capture the asymmetry of geographical influence and the high variation of geographical influence across POIs, Wang et al. [23] exploited POI-specific geographical influence to improve POI recommendation and modeled the geographical influence between two POIs by using three factors: the geo-influence of POI, the geo-susceptibility of POI, and their physical distance. To incorporate the geographical context information, Ma et al. [24] proposed a novel autoencoder-based model to learn the complex user-POI relations. Guo et al. [25] explored the influence of geographical information from a location perspective and proposed a constrained ranking function by using a location regularization term. However, they did not consider the deep structure of social networks and POI relationships, and the influence of them is still largely unexplored.Except the social and geographical information, other types of contexts have also been explored [8,26–29], such as the temporal influence [30], the sequential influence, and the content influence [31, 32]. For example, based on the temporal properties of user movement observed from a real-world LBSN dataset, Gao et al. [33] proposed a temporal aware recommendation framework. Qian et al. [34] focused on the joint effect of time and location for POI recommendation and proposed a spatiotemporal context-aware and translation-based recommender framework (STA) to model the third-order relationship among users, POIs, and spatiotemporal contexts for large-scale POI recommendation. In literature [28], they further studied three types of content information that is related to the user’s check-in actions and then modeled them in a unified POI recommendation framework. In addition to social and geographical influence, Zhang et al. [35] also considered the sequential influence. In this work, they first mined sequential patterns from location sequences and represented the sequential patterns as a dynamic graph. Then, they predicted the probability of a user visiting a location by additive Markov chain with this graph. Zhang and Chow [36] modeled three types of context, that is, geographical, social, and categorical correlations, for recommendation, and utilized the two-dimensional KDE for geographical modeling and the power-law distribution for social and categorical influence. Lian et al. [37] proposed a scalable implicit feedback-based content-aware collaborative filtering framework to incorporate semantic content and to steer clear of negative sampling. However, their work is developed for semantic information modeling, while ours is focused on exploring network embedding information. Yang et al. [38] proposed a neural network to jointly model the social networks and mobile trajectories, which considers four influential factors, namely, user visit preference, influence of friends, and short-term and long-term sequential contexts. But their method is developed for friend recommendation and next location recommendation tasks. Next location recommendation is different from the general POI/location recommendation problem, since it focuses on modeling the location’s sequential information rather than user’s historical preferences. From the above studies, we notice that these types of contexts are also important factors for POI recommendations and are often fused together with social and geographical context to make more accurate predictions.


### 2.2. Implicit Feedback-Based Recommendation

In many recommendation scenarios, we can only get the user’s positive behaviors through what they have purchased, visited, or clicked, but never know what they do not like. The feedback usually is not explicit, but implicit, which makes the recommendation task more challenging (the implicit feedback also is well known as the one-class collaborative filtering (OCCF) problem) [39, 40]. Note that, although the OCCF problem we studied in this work has similar problem definition with positive unlabeled learning (PU learning) [41–44], most of the solutions proposed for PU learning cannot be directly applied to OCCF problem. The main reason is that most of them are focused on dealing with documents or reviews, where a lot of document features can be exploited to decide the true-negative samples initially. But in OCCF problem, we usually can only obtain the positive interactions among users and items, and it is hard to identify the true-negative items without any additional information. Most of the recommendation-oriented methods are trying to model the implicit feedback in a ranking or weighted way. For example, Pan et al. [45] modeled the implicit feedback by using the weighted low rank approximations; that is, they gave larger weights to the positive samples and smaller weights to the negative samples. But their work needs to directly model all the missed values as negative samples. Rendle et al. [7] proposed a pair-wise ranking-based criterion to directly optimize the loss error for ranking, and a generic ranking framework derived from a Bayesian analysis of the problem was presented. Lian et al. [4] extended the weighted matrix factorization method proposed in [45] for POI recommendation, where the latent feature factors of users and POIs are augmented by the activity area vector and the influence area vector, respectively. For the sake of improving efficiency and flexibility, Lian et al. [46] further extended the GeoMF [4] and proposed GeoMF++, which map the location influential areas into the same latent space as that formed by weighted regularized matrix factorization. Different from [4], user activity area is not used in GeoMF++ anymore, but can be recovered from user latent factors using negative sparse coding. Liu et al. [47] exploited the geographical characteristics from a location perspective, where the neighborhood is modeled at two levels, i.e., the instance-level neighborhood that is defined by its nearest neighbors and the region-level neighborhood that is defined by the geographical region where the location exists. Then, these two levels of geographical characteristics are incorporated to the weighted matrix factorization to learn the latent feature factors of users and POIs. Li et al. [5] investigated the POI recommendation by treating it as the ordered weighted pair-wise classification problem and proposed a ranking-based geographical factorization model. Zhao et al. [48] first proposed the cogeographical influence to filter geographically noisy POIs and then incorporated it into a personalized pair-wise preference ranking matrix factorization model. Although ranking-based POI recommendations have been well studied, most of them do not consider incorporating these contexts by utilizing the embedding techniques and cannot be directly applied to the cold-start situations.


### 2.3. Embedding-Based Recommendation

With the development of distributed representation techniques [49–51], embedding-based models have been proved in many recommendation and other related tasks [10, 52, 53]. For example, Liang et al. [54] proposed a content-aware collaborative music recommendation by first training a neural network on semantic tagging information and then used it as the prior in a collaborative filtering model. He et al. [55] proposed a scalable factorization model to incorporate visual signals into predictors of people’s opinions, where the visual signals are learned from product images by using the pretrained neural networks. Wang et al. [56] further studied the problem of enhancing POI recommendation with visual contents, where they used the convolutional neural network to extract features from images and used it to guide the learning process of latent user and POI features. Deng et al. [57] employed the deep autoencoder technique to improve the quality of the initial vectors of MF for trust-aware social recommendations and to differentiate the community effect in users’ trusted friendships. Except the above models that integrate deep learning with traditional recommendation models loosely, there are also tightly coupled models, that is, learn the parameters from these two components simultaneously. In this way, the learning process of latent factor model and feature extracting model mutually influence each other. For instance, Chen et al. [58] introduced an attention-based neural network to collaborative filtering to address the implicit feedback problem in multimedia recommendation. Guo et al. [59] combined the factorization machine and deep learning for feature learning and proposed a novel neural network architecture. To deal with the inherent limitation of the bag-of-words model, Kim et al. [60] integrated the convolutional network into probabilistic matrix factorization model to capture the contextual information of documents and enhance the rating prediction accuracy. Guo et al. [61] incorporated a pretrained network embedding into the weighted matrix factorization method. However, their method was proposed for social recommendation and the network embedding was integrated by a linear latent factor model, and ours is regarded as the prior knowledge of the latent feature factor.


### 2.4. Differences

Our method has significant differences with these existing methods. First, although Yang et al. [38] adopted a network embedding method for the construction of social networks, it is proposed for friend recommendation and next location recommendation. Moreover, Yang et al. [38] did not consider the influence of geographical information. But in real world, nearby POIs are more likely to be visited by similar users, and the geographical relationship between POIs is also an important factor to the POI recommendation task. Hence, we further incorporate the POI-POI graph to enhance the representation of POI embeddings. Second, although there are existing works that formulate the POI recommendation as an item ranking task and try to optimize a BPR loss function, they did not consider the performance of leveraging network embeddings as user’s and item’s prior knowledge. The advantage of our method is that it can enhance POI recommendation by exploiting high-level embeddings from external network information and can make cold-start predictions for the users/items that have no training data in history by only using the learned distributional representations.


## 2.1. POI Recommendation with Social and Geographical Context

In LBSNs, social and geographical context plays an important role in helping users explore the unknown locations [12–17]. Motivated by the intuitions that users are more inclined to visit the places that their friends have suggested, and nearby locations tend to be visited by similar users, the social and geographical influence has been utilized to improve the POI recommendation. Cho et al. [18] studied the relation between human geographic movement, its temporal dynamics, and ties of the social network. They found that people generally move periodically within a bound region but occasionally travel long distance due to the impact of the social relationships. Ye et al. [1] first modeled the spatial clustering phenomenon by a power law distribution. Then, a unified POI recommendation framework was proposed, which fuses user preference to a POI with geographical influence and social influence. Cheng et al. [2] first captured the geographical influence by utilizing the multicenter Gaussian model and then fused it with the social influence in a generalized matrix factorization framework. Unlike previous proposed methods, which model the user check-in activities universally, Zhang et al. [19] proposed a personalized geographical location recommendation framework, where each user is modeled as a personal distribution. Li et al. [20] improved the POI recommendation by leveraging the information of friends in a two-step framework, where a set of potential locations that each user’s friends have checked in were first learned, and then three types of check-ins were incorporated into the matrix factorization model. Liu et al. [21] proposed a general geographical probabilistic factor model to jointly learn both geographical preference and user interest preference, where a Poisson-based probabilistic generative process is utilized for the recommendation model. Wang et al. [22] constructed a graph to model users and their social relations, which models the user preference and social influence by utilizing different types of edges. To capture the asymmetry of geographical influence and the high variation of geographical influence across POIs, Wang et al. [23] exploited POI-specific geographical influence to improve POI recommendation and modeled the geographical influence between two POIs by using three factors: the geo-influence of POI, the geo-susceptibility of POI, and their physical distance. To incorporate the geographical context information, Ma et al. [24] proposed a novel autoencoder-based model to learn the complex user-POI relations. Guo et al. [25] explored the influence of geographical information from a location perspective and proposed a constrained ranking function by using a location regularization term. However, they did not consider the deep structure of social networks and POI relationships, and the influence of them is still largely unexplored.Except the social and geographical information, other types of contexts have also been explored [8,26–29], such as the temporal influence [30], the sequential influence, and the content influence [31, 32]. For example, based on the temporal properties of user movement observed from a real-world LBSN dataset, Gao et al. [33] proposed a temporal aware recommendation framework. Qian et al. [34] focused on the joint effect of time and location for POI recommendation and proposed a spatiotemporal context-aware and translation-based recommender framework (STA) to model the third-order relationship among users, POIs, and spatiotemporal contexts for large-scale POI recommendation. In literature [28], they further studied three types of content information that is related to the user’s check-in actions and then modeled them in a unified POI recommendation framework. In addition to social and geographical influence, Zhang et al. [35] also considered the sequential influence. In this work, they first mined sequential patterns from location sequences and represented the sequential patterns as a dynamic graph. Then, they predicted the probability of a user visiting a location by additive Markov chain with this graph. Zhang and Chow [36] modeled three types of context, that is, geographical, social, and categorical correlations, for recommendation, and utilized the two-dimensional KDE for geographical modeling and the power-law distribution for social and categorical influence. Lian et al. [37] proposed a scalable implicit feedback-based content-aware collaborative filtering framework to incorporate semantic content and to steer clear of negative sampling. However, their work is developed for semantic information modeling, while ours is focused on exploring network embedding information. Yang et al. [38] proposed a neural network to jointly model the social networks and mobile trajectories, which considers four influential factors, namely, user visit preference, influence of friends, and short-term and long-term sequential contexts. But their method is developed for friend recommendation and next location recommendation tasks. Next location recommendation is different from the general POI/location recommendation problem, since it focuses on modeling the location’s sequential information rather than user’s historical preferences. From the above studies, we notice that these types of contexts are also important factors for POI recommendations and are often fused together with social and geographical context to make more accurate predictions.


## 2.2. Implicit Feedback-Based Recommendation

In many recommendation scenarios, we can only get the user’s positive behaviors through what they have purchased, visited, or clicked, but never know what they do not like. The feedback usually is not explicit, but implicit, which makes the recommendation task more challenging (the implicit feedback also is well known as the one-class collaborative filtering (OCCF) problem) [39, 40]. Note that, although the OCCF problem we studied in this work has similar problem definition with positive unlabeled learning (PU learning) [41–44], most of the solutions proposed for PU learning cannot be directly applied to OCCF problem. The main reason is that most of them are focused on dealing with documents or reviews, where a lot of document features can be exploited to decide the true-negative samples initially. But in OCCF problem, we usually can only obtain the positive interactions among users and items, and it is hard to identify the true-negative items without any additional information. Most of the recommendation-oriented methods are trying to model the implicit feedback in a ranking or weighted way. For example, Pan et al. [45] modeled the implicit feedback by using the weighted low rank approximations; that is, they gave larger weights to the positive samples and smaller weights to the negative samples. But their work needs to directly model all the missed values as negative samples. Rendle et al. [7] proposed a pair-wise ranking-based criterion to directly optimize the loss error for ranking, and a generic ranking framework derived from a Bayesian analysis of the problem was presented. Lian et al. [4] extended the weighted matrix factorization method proposed in [45] for POI recommendation, where the latent feature factors of users and POIs are augmented by the activity area vector and the influence area vector, respectively. For the sake of improving efficiency and flexibility, Lian et al. [46] further extended the GeoMF [4] and proposed GeoMF++, which map the location influential areas into the same latent space as that formed by weighted regularized matrix factorization. Different from [4], user activity area is not used in GeoMF++ anymore, but can be recovered from user latent factors using negative sparse coding. Liu et al. [47] exploited the geographical characteristics from a location perspective, where the neighborhood is modeled at two levels, i.e., the instance-level neighborhood that is defined by its nearest neighbors and the region-level neighborhood that is defined by the geographical region where the location exists. Then, these two levels of geographical characteristics are incorporated to the weighted matrix factorization to learn the latent feature factors of users and POIs. Li et al. [5] investigated the POI recommendation by treating it as the ordered weighted pair-wise classification problem and proposed a ranking-based geographical factorization model. Zhao et al. [48] first proposed the cogeographical influence to filter geographically noisy POIs and then incorporated it into a personalized pair-wise preference ranking matrix factorization model. Although ranking-based POI recommendations have been well studied, most of them do not consider incorporating these contexts by utilizing the embedding techniques and cannot be directly applied to the cold-start situations.


## 2.3. Embedding-Based Recommendation

With the development of distributed representation techniques [49–51], embedding-based models have been proved in many recommendation and other related tasks [10, 52, 53]. For example, Liang et al. [54] proposed a content-aware collaborative music recommendation by first training a neural network on semantic tagging information and then used it as the prior in a collaborative filtering model. He et al. [55] proposed a scalable factorization model to incorporate visual signals into predictors of people’s opinions, where the visual signals are learned from product images by using the pretrained neural networks. Wang et al. [56] further studied the problem of enhancing POI recommendation with visual contents, where they used the convolutional neural network to extract features from images and used it to guide the learning process of latent user and POI features. Deng et al. [57] employed the deep autoencoder technique to improve the quality of the initial vectors of MF for trust-aware social recommendations and to differentiate the community effect in users’ trusted friendships. Except the above models that integrate deep learning with traditional recommendation models loosely, there are also tightly coupled models, that is, learn the parameters from these two components simultaneously. In this way, the learning process of latent factor model and feature extracting model mutually influence each other. For instance, Chen et al. [58] introduced an attention-based neural network to collaborative filtering to address the implicit feedback problem in multimedia recommendation. Guo et al. [59] combined the factorization machine and deep learning for feature learning and proposed a novel neural network architecture. To deal with the inherent limitation of the bag-of-words model, Kim et al. [60] integrated the convolutional network into probabilistic matrix factorization model to capture the contextual information of documents and enhance the rating prediction accuracy. Guo et al. [61] incorporated a pretrained network embedding into the weighted matrix factorization method. However, their method was proposed for social recommendation and the network embedding was integrated by a linear latent factor model, and ours is regarded as the prior knowledge of the latent feature factor.


## 2.4. Differences

Our method has significant differences with these existing methods. First, although Yang et al. [38] adopted a network embedding method for the construction of social networks, it is proposed for friend recommendation and next location recommendation. Moreover, Yang et al. [38] did not consider the influence of geographical information. But in real world, nearby POIs are more likely to be visited by similar users, and the geographical relationship between POIs is also an important factor to the POI recommendation task. Hence, we further incorporate the POI-POI graph to enhance the representation of POI embeddings. Second, although there are existing works that formulate the POI recommendation as an item ranking task and try to optimize a BPR loss function, they did not consider the performance of leveraging network embeddings as user’s and item’s prior knowledge. The advantage of our method is that it can enhance POI recommendation by exploiting high-level embeddings from external network information and can make cold-start predictions for the users/items that have no training data in history by only using the learned distributional representations.


## 3. Our Approach

In this section, we first describe the POI recommendation task in LBSNs. Then, the users’ high-level representations and social-aware recommendation method are proposed. To exploit the influence of neighborhood relationships among locations, we then learn the POI representations from a predefined location network. Finally, by fusing the extracted embeddings of users and POIs with collaborative filtering model, the location-aware social POI recommendation method is achieved.


### 3.1. Task Definition

Let us consider the POI recommendation problem in LBSNs, where users can express their preferences by checking in at different locations, and the closer the two locations are, the more likely they are to be visited by similar users. But different from traditional social networks, in LBSNs, only positive behaviors can be observed, that is, we can only know the places that the user has checked in, but never know the places that the user does not like. In other words, the check-in data are not explicit but implicit. In addition to these check-in activities, users can also express their preferences by making different friends, and the closer the relationship between two friends is, the more likely they are to be affected. This recommendation scenario is shown in Figure2, which is associated with three central elements: the user check-in data (also known as user-POI check-in matrix), social network, and location network (i.e., POI-POI relation network).Figure 2
POI recommendation scenario in LBSNs, which consists of three kinds of information: the user-POI check-in matrix, social network, and POI-POI network.Figure2 shows a toy example, where users express their visiting preferences by checking in at different places. But the check-in data are very sparse (some users are totally new to the system), only a small number of users’ activities can be observed, and the remaining are unknown data (denoted as “?”). Besides the check-in matrix, users have also made some friends that have common interests with them to further explore the unknown places. The social relationship helps us make more accurate recommendations by leveraging the locations that their friends have visited. Moreover, in POI recommendation, neighboring locations tend to share common users, which means users are more inclined to visit nearby places. This kind of context is also beneficial for our recommender system. Hence, how to make effective recommendations by utilizing the above contexts from the implicit check-in data is the main task of this work.


### 3.2. The Pair-Wise Item Ranking Criterion

To deal with the implicit check-in data, we treat POI recommendation problem as the pair-wise item ranking task and introduce the Bayesian personalized ranking (BPR) criterion [7] as our basic recommendation framework. Instead of optimizing individual scores for a single item, BPR infers the latent feature factors by optimizing the partial ordering relationships between items, which is more suitable to the implicit feedback.Suppose we have a check-in matrixR⊆U×I (as shown in Figure 3), where U denotes the user set and I denotes the POI set. The entry rui=1 in this matrix represents that user u has visited POI i in history, and rui=? represents that the status of u to i is unknown, which means i is unattractive or undiscovered to user u. We cannot directly treat all these unknown entries as negative samples, as they may be potential attractive POIs to users. To provide a personalized total ranking >u⊂I2 of all POIs to the user, BPR method proposes a pair-wise data policy to reconstruct the user-POI check-in matrix, which assumes that compared with unvisited POIs, users will prefer the POIs that he/she has visited. From Figure 3, we can find some examples of creating pair-wise preferences i>uj for user u from the check-in matrix, where the sign “+” represents that u prefers i over j and the sign “−” represents that user u prefers j over i (see the left part of Figure 3). For example, as user u1 has checked in POI i2 and unvisited POI i4, then according to our data policy the partial order between i2 and i4 can be created as i2>ui4 (u prefer i2 over i4). But for the POIs that both have been checked in by u, we cannot deduce any preference for him/her (e.g., POI i2 and i3 for user u1). The same is true for the POIs that both have been unvisited by the user (e.g., POI i2 and i3 for user u2). Then, we can formalize the created training data Dℛ: U×I×I as(1)Dℛ≔u,i,ji∈Iu+∧j∈ℐ\Iu+,where Iu+ represents the POI set that has been checked in by user u, ℐ\Iu+ denotes the unvisited POI set by user u, and u,i,j∈Dℛ means user u prefers POI i over j. As the ranking with respect to u (>u) is antisymmetric, the negative samples are regarded implicitly.Figure 3
The reconstructing data policy from user-POI check-in matrix to pair-wise POI dataset.To rank all the POIs correctly for every user, BPR maximizes the posterior probability of the model parameter from a Bayesian analysis of the problem. By usingpi>ujΘ as the likelihood function and pΘ as the prior function, we can arrive at the objective function of BPR:(2)BPR=−∑u,i,j∈Dℛlnσr^uijΘ+λΘΘF2,where Θ is the model parameter, σx is the logistic sigmoid function, and λΘ is the regularization parameter. r^uijΘ captures the relationship among u, i, and j, which can be calculated by any real-valued score function of Θ. By decomposing the estimator r^uij as r^uij=wuijr^ui−r^uj, and delegating the task of modeling r^uij to the matrix factorization (MF) method, the MF-based BPR method can be arrived at(3)BPRMF=−∑u,i,j∈Dℛlnσr^ui−r^uj+λU2UF2+λV2VF2,where V∈Rd×n and U∈Rd×m denote the latent factors of POIs and users, respectively. λU and λV denote the regularization parameters. r^uv=UuTVv is the score function computed by the dot product of U and V. As in MF, we also place the zero-mean spherical Gaussian priors on the latent factors U and V:(4)pUσU2=∏u=1mNUu0,σU2I,(5)pVσV2=∏v=1nNVv0,σV2I,where Nxμ,σ2 denotes the Gaussian distribution with variance-covariance matrix σ2 and mean μ.


### 3.3. Pretrained Social Network Embedding

Inspired by the success of neural network-based embedding model in link predication and node classification, in this work we pretrain a network embedding model node2vec [11] (based on the skip-gram architecture) in an semisupervised task and use the learned embeddings as the representations of social networks (the project of node2vec is available at https://snap.stanford.edu/node2vec/). The training data we used come from the Gowalla dataset [12], which consists of 1.2 million check-ins (the details can be seen in Section 4.1). The social network we used is the friend relationships among 18,738 users, which can be represented as an unweighted and undirected graph. Let G=U,ℰ be the given social network and f:U⟶ℝDs be the mapping function from nodes to the Ds dimension feature representations, where U denotes the user set and ℰ denotes the friend relationship set. For every source node u∈U, we define NSu∈U as a network neighborhood of node u generated through a neighborhood sampling strategy S, that is, a biased random walk method that can explore neighborhoods in a breadth-first sampling as well as depth-first sampling fashion.To learn the high-level representations of every node, node2vec tries to maximize the log-probability of observing a network neighborhoodNSu for a node u conditioned on its feature representation, given by f:(6)maxf∑u∈UlogPNSufu.To make the above optimization problem tractable, two standard assumptions are made in node2vec: (1) conditional independence, which assumes the observation likelihood of a neighborhood node is independent of the likelihood of observing any other neighborhood node given the feature representation of the source, and (2) symmetry in feature space, which means a source node and neighborhood node have a symmetric effect over each other in feature space. With these two assumptions, the objective function of node2vec is arrived at(7)maxf∑u∈U−logZu+∑ni∈NSufni⋅fu,where Zu=∑v∈Uexpfu⋅fv denotes the per-node partition function. As it is expensive to compute for large networks, the negative sampling method [62] is used to approximate it. In their work, stochastic gradient ascent method is used to learn the model parameters.In our work, the parameters of node2vec are set as follows: the dimension of the learned feature vectors is set asDs=128, the length of walk per source is set as l=80, the number of walks per source is set as r=10, the context size is set as k=10, and the optimization runs only a single epoch. The best in-out and return parameters are achieved by using 10-fold cross-validation on tune dataset and are set as p=1 and q=2.


### 3.4. Social-Aware Recommendation

The output of node2vec can be interpreted as the network representations of users. As users always express their preferences by making different friends, these representations denote how well a user is influenced by his friends. By introducing the network feature as the prior of the latent user factor in BPRMF model, the distribution ofU can be modified as(8)pUX,S,σU2=∏u=1mNUuXSu,σU2I,where Su∈ℝDs denotes the learned network embedding of user u (Ds is the dimension of Su) and X∈ℝd×Ds is the weight matrix that transforms the learned social network representations from the neural networks into the collaborative filtering latent space (via XSu).To complete the inference, we also assume that the weight matrixX follows the zero-mean spherical Gaussian distribution:(9)pXσX2=∏x=1d∏s=1DsNXxs0,σX2.Then, the posterior probability over the user, item, and weight latent factor vectors can be rewritten as(10)pU,V,XDℛ,S,σX2,σU2,σV2∝pDℛU,VpUX,S,σU2pVσV2pXσX2=∏u,i,j∈Dℛσr^uij×∏u=1mNUuXSu,σU2I×∏v=1nNVv0,σV2I×∏x=1d∏s=1DsNXxs0,σX2.The log-posterior probability ofU, V, and X is derived as(11)lnpU,V,XDℛ,S,σX2,σU2,σV2=∑u,i,j∈Dℛlnσr^uij−λU2∑u=1mUu−XSuTUu−XSu−12σV2∑k=1nVkTVk−12σX2∑x=1dXxTXx−12m×dlnσU2+n×dlnσV2+d×DslnσX2+C.Keeping the parameters fixed, maximizing the log-posterior over the latent feature vectors is equivalent to minimize the following objective function, that is, the objective function of our social-aware recommendation method:(12)SRankDℛ,X,U,V=−∑u=1m∑i∈Iu+∑j∈I\Iu+lnσr^uij+λU2∑u=1mUu−XSuTUu−XSu+λX2XF2+λV2VF2,where ⋅F2 is the Frobenius norm, and λU=1/σU2, λV=1/σV2, and λX=1/σX2 are the regularization parameters of U,V, and X, respectively. Gradient descent-based algorithm is an obvious option to help find the local minimum of this objective function. And to alleviate the influence of huge training pairs OUI, the stochastic gradient descent method is utilized in our algorithm. The gradients of the corresponding latent factors are shown as follows:(13)∂SRank∂X=−λUUuSuT+λUXSuSuT+λXX,(14)∂SRank∂Uu=−e−r^uij1+e−r^uijVi−Vj+λUUu−λUXSu,(15)∂SRank∂Vi=−e−r^uij1+e−r^uijUu+λVVi,(16)∂SRank∂Vj=e−r^uij1+e−r^uijUu+λVVj.


### 3.5. Pretrained Location Network Embeddings

To utilize the context of geographical information, we first deduce a POI-POI graph based on the intuition that nearby locations are more likely to be visited by similar users and then pretrain the node2vec model for learning the representations of POIs. The definition of the POI-POI graph is shown as follows.Definition 1 (POI-POI graph). The POI-POI graph is denoted asGV=V,ℰV, which captures the neighborhood relationships of POIs, where V is the POI set and ℰV is the edge set of POIs. Given a distance threshold ΔD (set as 0.2 km), for any two POIs vi∈V and vj∈V, if the distance (here, the distance is computed by the Haversine formula) dij between them is smaller than ΔD (dij<ΔD), there will be an edge eij from vi to vj, and otherwise none.According to the definition of POI-POI graph, the location network among POIs is thereafter formed. Location network captures the geographical characteristics of users’ check-in activities. Intuitively, if two POIsvi and vj are geographically close, they will have a strong correlation in location network. When we know a user has checked in vi, we can conclude this user will also visit vj next with high confidence. In experiments, we obtain the location network of Gowalla with 25,517 nodes and 210,325 edges. The parameter settings of our pretrained node2vec model are as follows: Dt=128,l=80,r=10,k=10,p=1,andq=0.5, where Dt is the dimension of the learned network feature vector.


### 3.6. Location-Aware Social Recommendation

The pretrained location network embeddings can be seen as the representations of POIs from geographical relations. Based on the assumption that nearby locations tend to be visited by similar users, that is, neighboring locations tend to have similar representations, we treat the learned embeddings as the prior of the latent POI factor in collaborative filtering model and modify the distribution ofV as(17)pVY,T,σV2=∏v=1nNVvYTv,σV2I,where Tv∈ℝDt denotes the pretrained embedding of POI v from geographical relationships (Dt is the dimension of Tv) and Y∈ℝd×Dt is the weight matrix that transforms the learned POI representations from location network into the collaborative filtering latent space via YTv.As in SRank, the zero-mean spherical Gaussian distribution is placed on the weight matrixY:(18)pYσY2=∏y=1d∏t=1DtNYyt0,σY2,Then, the posterior probability of SRank can be further modified as(19)pU,V,X,YDℛ,S,T,σX2,σY2,σU2,σV2∝pDℛU,VpUX,S,σU2⋅pVY,T,σV2pXσX2pYσY2=∏u,i,j∈Dℛσr^uij×∏u=1mNUuXSu,σU2I×∏v=1nNVvYTv,σV2I×∏x=1d∏s=1DsNXxs0,σX2×∏y=1d∏t=1DtNYyt0,σY2.By computing the log form of equation (19) and keeping the hyperparameters fixed, the objective function of our final recommendation method Geo-SRank is achieved:(20)Geo−SRankDℛ,X,Y,U,V=−∑u=1m∑i∈Iu+∑j∈I\Iu+lnσr^uij+λU2∑u=1mUu−XSuTUu−XSu+λV2∑v=1nVv−YTvTVv−YTv+λX2XF2+λY2YF2,where λY=1/σY2 is the regularization parameter of Y. To find a local minimum of this objective function, gradient descent-based algorithm is utilized, and the latent factors can be updated by the following gradients:(21)∂Geo−SRank∂X=−λUUuSuT+λUXSuSuT+λXX,(22)∂Geo−SRank∂Y=−λVVvTvT+λVYTvTvT+λYY,(23)∂Geo−SRank∂Uu=−e−r^uij1+e−r^uijVi−Vj+λUUu−λUXSu,(24)∂Geo−SRank∂Vi=−e−r^uij1+e−r^uijUu+λVVi−λVYTi,(25)∂Geo−SRank∂Vj=e−r^uij1+e−r^uijUu+λVVj−λVYTj.Algorithm1 presents the training process of our Geo-SRank method.Algorithm 1:The training process of Geo-SRank.
Require: the user-POI check-in matrixR, learning rate η, regularization parameters λU, λVi, λVj, λX, and λY, the pretrained network embeddings S and T;
Ensure: the latent factorsU, V, X, and Y;
(1)
The initialization ofU, V, X, and Y;
(2)
while (L(t) − L(t−1)> tolerate error) do
(3)
Randomly select one training exampleu,i,j from Dℛ;
(4)
r^uij⟵r^ui−r^uj;
(5)
CalculateUu by utilizing equation (23);
(6)
CalculateX by utilizing equation (21);
(7)
CalculateVi by utilizing equation (24);
(8)
CalculateVj by utilizing equation (25);
(9)
CalculateY by utilizing equation (22);
(10)
CalculateL(t) (the loss error in t step) by utilizing equation (20);
(11)
end while
(12)
return
U, V, X, and Y;


### 3.7. Computational Complexity

The complexity of training the Geo-SRank method is mainly caused by computing the ranking loss (equation (20)) and the gradients of latent feature factors U, V, X, and Y. Assuming the average number of sampled triples is N¯, the average dimension of pretrained social network embedding is s¯, and the average dimension of pretrained location network embedding is t¯, the complexity of computing equation (20) is ON¯d+ms¯d+nt¯d. Since in our experiments the dimensions of the feature factors (d, s¯, and t¯) are set as very small values, the computing complexity is mainly determined by the sampled training triples N¯. The computation of the objective function of Geo-SRank is very fast and linear with respect to the sampled triple number. The complexities of computing the gradients ∂L/∂U and ∂L/∂X are ON¯ds¯. The complexities of ∂L/∂V and ∂L/∂Y are ON¯dt¯. Both of these two complexities are also linear with respect to the sampled triple number.


### 3.8. Applications in Cold-Start Situations

The advantage of our feature-based recommendation method Geo-SRank is that it can be easily extended to the cold-start situations, i.e., there are no training data for the users/items in history. In the user cold-start situation, we can make predictions for users that no one has check-in records by using the social network representationsr^uv=XSuTVv, where pure collaborative filtering models have trouble in learning the latent user factors. Here, the weight matrix X can be learned by known users and Su is the pretrained network embeddings. In the item cold-start situation, we can make predictions for cold-start items by using the location network representations r^uv=UuTYTv, where the weight matrix Y is learned by known items and Tv is the pretrained embeddings of items.


## 3.1. Task Definition

Let us consider the POI recommendation problem in LBSNs, where users can express their preferences by checking in at different locations, and the closer the two locations are, the more likely they are to be visited by similar users. But different from traditional social networks, in LBSNs, only positive behaviors can be observed, that is, we can only know the places that the user has checked in, but never know the places that the user does not like. In other words, the check-in data are not explicit but implicit. In addition to these check-in activities, users can also express their preferences by making different friends, and the closer the relationship between two friends is, the more likely they are to be affected. This recommendation scenario is shown in Figure2, which is associated with three central elements: the user check-in data (also known as user-POI check-in matrix), social network, and location network (i.e., POI-POI relation network).Figure 2
POI recommendation scenario in LBSNs, which consists of three kinds of information: the user-POI check-in matrix, social network, and POI-POI network.Figure2 shows a toy example, where users express their visiting preferences by checking in at different places. But the check-in data are very sparse (some users are totally new to the system), only a small number of users’ activities can be observed, and the remaining are unknown data (denoted as “?”). Besides the check-in matrix, users have also made some friends that have common interests with them to further explore the unknown places. The social relationship helps us make more accurate recommendations by leveraging the locations that their friends have visited. Moreover, in POI recommendation, neighboring locations tend to share common users, which means users are more inclined to visit nearby places. This kind of context is also beneficial for our recommender system. Hence, how to make effective recommendations by utilizing the above contexts from the implicit check-in data is the main task of this work.


## 3.2. The Pair-Wise Item Ranking Criterion

To deal with the implicit check-in data, we treat POI recommendation problem as the pair-wise item ranking task and introduce the Bayesian personalized ranking (BPR) criterion [7] as our basic recommendation framework. Instead of optimizing individual scores for a single item, BPR infers the latent feature factors by optimizing the partial ordering relationships between items, which is more suitable to the implicit feedback.Suppose we have a check-in matrixR⊆U×I (as shown in Figure 3), where U denotes the user set and I denotes the POI set. The entry rui=1 in this matrix represents that user u has visited POI i in history, and rui=? represents that the status of u to i is unknown, which means i is unattractive or undiscovered to user u. We cannot directly treat all these unknown entries as negative samples, as they may be potential attractive POIs to users. To provide a personalized total ranking >u⊂I2 of all POIs to the user, BPR method proposes a pair-wise data policy to reconstruct the user-POI check-in matrix, which assumes that compared with unvisited POIs, users will prefer the POIs that he/she has visited. From Figure 3, we can find some examples of creating pair-wise preferences i>uj for user u from the check-in matrix, where the sign “+” represents that u prefers i over j and the sign “−” represents that user u prefers j over i (see the left part of Figure 3). For example, as user u1 has checked in POI i2 and unvisited POI i4, then according to our data policy the partial order between i2 and i4 can be created as i2>ui4 (u prefer i2 over i4). But for the POIs that both have been checked in by u, we cannot deduce any preference for him/her (e.g., POI i2 and i3 for user u1). The same is true for the POIs that both have been unvisited by the user (e.g., POI i2 and i3 for user u2). Then, we can formalize the created training data Dℛ: U×I×I as(1)Dℛ≔u,i,ji∈Iu+∧j∈ℐ\Iu+,where Iu+ represents the POI set that has been checked in by user u, ℐ\Iu+ denotes the unvisited POI set by user u, and u,i,j∈Dℛ means user u prefers POI i over j. As the ranking with respect to u (>u) is antisymmetric, the negative samples are regarded implicitly.Figure 3
The reconstructing data policy from user-POI check-in matrix to pair-wise POI dataset.To rank all the POIs correctly for every user, BPR maximizes the posterior probability of the model parameter from a Bayesian analysis of the problem. By usingpi>ujΘ as the likelihood function and pΘ as the prior function, we can arrive at the objective function of BPR:(2)BPR=−∑u,i,j∈Dℛlnσr^uijΘ+λΘΘF2,where Θ is the model parameter, σx is the logistic sigmoid function, and λΘ is the regularization parameter. r^uijΘ captures the relationship among u, i, and j, which can be calculated by any real-valued score function of Θ. By decomposing the estimator r^uij as r^uij=wuijr^ui−r^uj, and delegating the task of modeling r^uij to the matrix factorization (MF) method, the MF-based BPR method can be arrived at(3)BPRMF=−∑u,i,j∈Dℛlnσr^ui−r^uj+λU2UF2+λV2VF2,where V∈Rd×n and U∈Rd×m denote the latent factors of POIs and users, respectively. λU and λV denote the regularization parameters. r^uv=UuTVv is the score function computed by the dot product of U and V. As in MF, we also place the zero-mean spherical Gaussian priors on the latent factors U and V:(4)pUσU2=∏u=1mNUu0,σU2I,(5)pVσV2=∏v=1nNVv0,σV2I,where Nxμ,σ2 denotes the Gaussian distribution with variance-covariance matrix σ2 and mean μ.


## 3.3. Pretrained Social Network Embedding

Inspired by the success of neural network-based embedding model in link predication and node classification, in this work we pretrain a network embedding model node2vec [11] (based on the skip-gram architecture) in an semisupervised task and use the learned embeddings as the representations of social networks (the project of node2vec is available at https://snap.stanford.edu/node2vec/). The training data we used come from the Gowalla dataset [12], which consists of 1.2 million check-ins (the details can be seen in Section 4.1). The social network we used is the friend relationships among 18,738 users, which can be represented as an unweighted and undirected graph. Let G=U,ℰ be the given social network and f:U⟶ℝDs be the mapping function from nodes to the Ds dimension feature representations, where U denotes the user set and ℰ denotes the friend relationship set. For every source node u∈U, we define NSu∈U as a network neighborhood of node u generated through a neighborhood sampling strategy S, that is, a biased random walk method that can explore neighborhoods in a breadth-first sampling as well as depth-first sampling fashion.To learn the high-level representations of every node, node2vec tries to maximize the log-probability of observing a network neighborhoodNSu for a node u conditioned on its feature representation, given by f:(6)maxf∑u∈UlogPNSufu.To make the above optimization problem tractable, two standard assumptions are made in node2vec: (1) conditional independence, which assumes the observation likelihood of a neighborhood node is independent of the likelihood of observing any other neighborhood node given the feature representation of the source, and (2) symmetry in feature space, which means a source node and neighborhood node have a symmetric effect over each other in feature space. With these two assumptions, the objective function of node2vec is arrived at(7)maxf∑u∈U−logZu+∑ni∈NSufni⋅fu,where Zu=∑v∈Uexpfu⋅fv denotes the per-node partition function. As it is expensive to compute for large networks, the negative sampling method [62] is used to approximate it. In their work, stochastic gradient ascent method is used to learn the model parameters.In our work, the parameters of node2vec are set as follows: the dimension of the learned feature vectors is set asDs=128, the length of walk per source is set as l=80, the number of walks per source is set as r=10, the context size is set as k=10, and the optimization runs only a single epoch. The best in-out and return parameters are achieved by using 10-fold cross-validation on tune dataset and are set as p=1 and q=2.


## 3.4. Social-Aware Recommendation

The output of node2vec can be interpreted as the network representations of users. As users always express their preferences by making different friends, these representations denote how well a user is influenced by his friends. By introducing the network feature as the prior of the latent user factor in BPRMF model, the distribution ofU can be modified as(8)pUX,S,σU2=∏u=1mNUuXSu,σU2I,where Su∈ℝDs denotes the learned network embedding of user u (Ds is the dimension of Su) and X∈ℝd×Ds is the weight matrix that transforms the learned social network representations from the neural networks into the collaborative filtering latent space (via XSu).To complete the inference, we also assume that the weight matrixX follows the zero-mean spherical Gaussian distribution:(9)pXσX2=∏x=1d∏s=1DsNXxs0,σX2.Then, the posterior probability over the user, item, and weight latent factor vectors can be rewritten as(10)pU,V,XDℛ,S,σX2,σU2,σV2∝pDℛU,VpUX,S,σU2pVσV2pXσX2=∏u,i,j∈Dℛσr^uij×∏u=1mNUuXSu,σU2I×∏v=1nNVv0,σV2I×∏x=1d∏s=1DsNXxs0,σX2.The log-posterior probability ofU, V, and X is derived as(11)lnpU,V,XDℛ,S,σX2,σU2,σV2=∑u,i,j∈Dℛlnσr^uij−λU2∑u=1mUu−XSuTUu−XSu−12σV2∑k=1nVkTVk−12σX2∑x=1dXxTXx−12m×dlnσU2+n×dlnσV2+d×DslnσX2+C.Keeping the parameters fixed, maximizing the log-posterior over the latent feature vectors is equivalent to minimize the following objective function, that is, the objective function of our social-aware recommendation method:(12)SRankDℛ,X,U,V=−∑u=1m∑i∈Iu+∑j∈I\Iu+lnσr^uij+λU2∑u=1mUu−XSuTUu−XSu+λX2XF2+λV2VF2,where ⋅F2 is the Frobenius norm, and λU=1/σU2, λV=1/σV2, and λX=1/σX2 are the regularization parameters of U,V, and X, respectively. Gradient descent-based algorithm is an obvious option to help find the local minimum of this objective function. And to alleviate the influence of huge training pairs OUI, the stochastic gradient descent method is utilized in our algorithm. The gradients of the corresponding latent factors are shown as follows:(13)∂SRank∂X=−λUUuSuT+λUXSuSuT+λXX,(14)∂SRank∂Uu=−e−r^uij1+e−r^uijVi−Vj+λUUu−λUXSu,(15)∂SRank∂Vi=−e−r^uij1+e−r^uijUu+λVVi,(16)∂SRank∂Vj=e−r^uij1+e−r^uijUu+λVVj.


## 3.5. Pretrained Location Network Embeddings

To utilize the context of geographical information, we first deduce a POI-POI graph based on the intuition that nearby locations are more likely to be visited by similar users and then pretrain the node2vec model for learning the representations of POIs. The definition of the POI-POI graph is shown as follows.Definition 1 (POI-POI graph). The POI-POI graph is denoted asGV=V,ℰV, which captures the neighborhood relationships of POIs, where V is the POI set and ℰV is the edge set of POIs. Given a distance threshold ΔD (set as 0.2 km), for any two POIs vi∈V and vj∈V, if the distance (here, the distance is computed by the Haversine formula) dij between them is smaller than ΔD (dij<ΔD), there will be an edge eij from vi to vj, and otherwise none.According to the definition of POI-POI graph, the location network among POIs is thereafter formed. Location network captures the geographical characteristics of users’ check-in activities. Intuitively, if two POIsvi and vj are geographically close, they will have a strong correlation in location network. When we know a user has checked in vi, we can conclude this user will also visit vj next with high confidence. In experiments, we obtain the location network of Gowalla with 25,517 nodes and 210,325 edges. The parameter settings of our pretrained node2vec model are as follows: Dt=128,l=80,r=10,k=10,p=1,andq=0.5, where Dt is the dimension of the learned network feature vector.


## 3.6. Location-Aware Social Recommendation

The pretrained location network embeddings can be seen as the representations of POIs from geographical relations. Based on the assumption that nearby locations tend to be visited by similar users, that is, neighboring locations tend to have similar representations, we treat the learned embeddings as the prior of the latent POI factor in collaborative filtering model and modify the distribution ofV as(17)pVY,T,σV2=∏v=1nNVvYTv,σV2I,where Tv∈ℝDt denotes the pretrained embedding of POI v from geographical relationships (Dt is the dimension of Tv) and Y∈ℝd×Dt is the weight matrix that transforms the learned POI representations from location network into the collaborative filtering latent space via YTv.As in SRank, the zero-mean spherical Gaussian distribution is placed on the weight matrixY:(18)pYσY2=∏y=1d∏t=1DtNYyt0,σY2,Then, the posterior probability of SRank can be further modified as(19)pU,V,X,YDℛ,S,T,σX2,σY2,σU2,σV2∝pDℛU,VpUX,S,σU2⋅pVY,T,σV2pXσX2pYσY2=∏u,i,j∈Dℛσr^uij×∏u=1mNUuXSu,σU2I×∏v=1nNVvYTv,σV2I×∏x=1d∏s=1DsNXxs0,σX2×∏y=1d∏t=1DtNYyt0,σY2.By computing the log form of equation (19) and keeping the hyperparameters fixed, the objective function of our final recommendation method Geo-SRank is achieved:(20)Geo−SRankDℛ,X,Y,U,V=−∑u=1m∑i∈Iu+∑j∈I\Iu+lnσr^uij+λU2∑u=1mUu−XSuTUu−XSu+λV2∑v=1nVv−YTvTVv−YTv+λX2XF2+λY2YF2,where λY=1/σY2 is the regularization parameter of Y. To find a local minimum of this objective function, gradient descent-based algorithm is utilized, and the latent factors can be updated by the following gradients:(21)∂Geo−SRank∂X=−λUUuSuT+λUXSuSuT+λXX,(22)∂Geo−SRank∂Y=−λVVvTvT+λVYTvTvT+λYY,(23)∂Geo−SRank∂Uu=−e−r^uij1+e−r^uijVi−Vj+λUUu−λUXSu,(24)∂Geo−SRank∂Vi=−e−r^uij1+e−r^uijUu+λVVi−λVYTi,(25)∂Geo−SRank∂Vj=e−r^uij1+e−r^uijUu+λVVj−λVYTj.Algorithm1 presents the training process of our Geo-SRank method.Algorithm 1:The training process of Geo-SRank.
Require: the user-POI check-in matrixR, learning rate η, regularization parameters λU, λVi, λVj, λX, and λY, the pretrained network embeddings S and T;
Ensure: the latent factorsU, V, X, and Y;
(1)
The initialization ofU, V, X, and Y;
(2)
while (L(t) − L(t−1)> tolerate error) do
(3)
Randomly select one training exampleu,i,j from Dℛ;
(4)
r^uij⟵r^ui−r^uj;
(5)
CalculateUu by utilizing equation (23);
(6)
CalculateX by utilizing equation (21);
(7)
CalculateVi by utilizing equation (24);
(8)
CalculateVj by utilizing equation (25);
(9)
CalculateY by utilizing equation (22);
(10)
CalculateL(t) (the loss error in t step) by utilizing equation (20);
(11)
end while
(12)
return
U, V, X, and Y;


## 3.7. Computational Complexity

The complexity of training the Geo-SRank method is mainly caused by computing the ranking loss (equation (20)) and the gradients of latent feature factors U, V, X, and Y. Assuming the average number of sampled triples is N¯, the average dimension of pretrained social network embedding is s¯, and the average dimension of pretrained location network embedding is t¯, the complexity of computing equation (20) is ON¯d+ms¯d+nt¯d. Since in our experiments the dimensions of the feature factors (d, s¯, and t¯) are set as very small values, the computing complexity is mainly determined by the sampled training triples N¯. The computation of the objective function of Geo-SRank is very fast and linear with respect to the sampled triple number. The complexities of computing the gradients ∂L/∂U and ∂L/∂X are ON¯ds¯. The complexities of ∂L/∂V and ∂L/∂Y are ON¯dt¯. Both of these two complexities are also linear with respect to the sampled triple number.


## 3.8. Applications in Cold-Start Situations

The advantage of our feature-based recommendation method Geo-SRank is that it can be easily extended to the cold-start situations, i.e., there are no training data for the users/items in history. In the user cold-start situation, we can make predictions for users that no one has check-in records by using the social network representationsr^uv=XSuTVv, where pure collaborative filtering models have trouble in learning the latent user factors. Here, the weight matrix X can be learned by known users and Su is the pretrained network embeddings. In the item cold-start situation, we can make predictions for cold-start items by using the location network representations r^uv=UuTYTv, where the weight matrix Y is learned by known items and Tv is the pretrained embeddings of items.


## 4. Experiments

In this section, several experiments on two real-world datasets are conducted to evaluate the recommendation performance of our feature-based method. To test the scalability of our method to the cold-start problem, we also conduct experiments on the situations with cold-start settings.


### 4.1. Datasets and Evaluation Metrics

In our experiments, two real-world datasets Gowalla and Yelp [12] are exploited to evaluate our proposed method (Table 1). Gowalla was launched in 2007 and closed in 2012, which provides the location and social services that enable users to check in at places and make friends based on where they have visited. The Gowalla dataset (https://snap.stanford.edu/data/loc-gowalla.html) we used was crawled by its API from February 2009 to October 2010. By filtering out those users with fewer than 15 check-in POIs and those POIs with fewer than 10 visitors, we can reach our final dataset, which contains 25,379 users, 32,623 POIs, 1,395,856 check-ins, and 118,717 social ties. Yelp is a local-search service powered by crowdsourced review forum, which publishes crowdsourced reviews about local businesses, as well as the online reservation service. Yelp was founded in 2004 and grew very quickly. By the end of 2017, Yelp has 148 million reviews. The dataset we used was collected by Liu et al. [12], which contains 30,887 users, 18,942 businesses, 561,746 check-ins, and 265,533 social ties.Table 1
Statistics of the training data of Gowalla and Yelp.
Statistics
Yelp
Gowalla
Check-in sparsity
99.903
99.906
No. of users
30,887
18,737
No. of POIs
18,942
32,495
No. of check-ins
561,746
566,791
Avg. No. of users per POI
0.6
0.57
Avg. No. of POIs per user
1.6
1.73To measure the performance of our recommendation approach, we exploit three popular measures Precision@k, Recall@k, and mean average precision (MAP) [12] as the evaluation metrics. In this Top-k item recommendation task, Precision@k measures how many relevant POIs can be predicted in the recommendation list. Here, the relevant POIs refer to the ones that have been labeled as positive in the dataset. Recall@k denotes how many relevant POIs can be predicted among the total relevant POIs in the dataset. The formal definitions of these two metrics are as follows:(26)Precision@k=1M∑u=1MLku∩LTuk,(27)Recall@k=1M∑u=1MLku∩LTuLTu,where M is the number of being tested users and LTu is the POI set that have been checked in by u in the test data. Lku represents the recommended Top-k POI list for user u (k is the size of this list). In this work, the list size k is set as 5. MAP measures the average precision (AP) over all users, which is computed by(28)MAP=1M∑u=1mAP=1M∑u=1m1k∑i=1kPrecision@i,where Precision@i refers to the precision for Top-i recommendations.


### 4.2. Data Analysis

To better understand the users’ visiting behaviors in location-based social networks, we further investigate the social influence and geographical characteristics in Gowalla and Yelp and try to investigate the following questions: (1) Are users with friend relationships more likely to have similar check-in activities? (2) Are POIs with neighboring relationships more likely to be visited by the same set of users? To answer these two questions, we need to first define a method to measure the similarity between two users.LetAu denote the POI set that has been visited by user u in history. The similarity between user u and v can be computed as(29)xuv=∑i∈Au∩Av1/log1+AiAuAv,where Ax represents the size of the POI set Ax and 1/log1+Au is the item that punishes the influence of popular POIs.For each useru, two similarities are computed, that is, (1) the average similarity between user u and his/her friends (xfu) and (2) the average similarity between user u and randomly selected users. For fairness, we choose the same number of users with their friend relationships. Figure 4 plots the kernel-smoothing density estimations based on the vectors xf and xr, from which we can observe that xr has smaller similarity values than xf. This result positively answers our first question, that is, users with friend relationships share more common interests than those without.Figure 4
Density estimation of users’ similarity on (a) Gowalla and (b) Yelp.
(a)
(b)To answer the second question, the measurement of the similarity between two POIs needs to be predefined. By denotingBi as the user set that has checked in location i, the similarity between POI i and j is arrived at(30)yij=∑u∈Bi∩Bj1/log1+BuBiBj.Based on this similarity, Figure5 shows how the similarity of randomly chosen POI pairs varies with the increase of geographical distance, where we can find that, with the increase of the distance, the mean value of the POI similarity decreases in both Yelp and Gowalla data. This result demonstrates the positive relationship between the POI similarity and geographical distance and provides us a positive answer to the second question: POIs with neighboring relationships are inclined to be visited by similar users.Figure 5
The similarity between POIs varies with geographical distance: (a) Yelp and (b) Gowalla.
(a)
(b)


### 4.3. Performance Comparison

To evaluate the recommendation performance of our method, we conduct comparisons with several related recommendation methods. As the matrix factorization- (MF-) based method has been proved to have state-of-the-art performance, we mainly compare against the MF-based collaborative filtering models and the related ranking-based method (most of these methods are implemented in MyMediaLite [63]):(i)
MostPopular: this is a basic recommendation method, which ranks the POIs according to how many users have visited them.(ii)
WRMF [45]: this method is proposed for the one-class collaborative filtering problem and provides a solution by giving small weights to negative samples and large weights to positive samples.(iii)
GeoMF [4]: this is a state-of-the-art POI recommendation method, which extends the WRMF method by augmenting the user and POI latent factors.(iv)
BPRMF [7]: this item recommendation method is proposed for implicit feedback, which learns the user interests from their pair-wise item preferences. This is the state-of-the-art POI recommendation method, which is only based on check-in matrix, without utilizing any context information.(v)
Rank-GeoFM [5]: this is a state-of-the-art POI recommendation method that develops a ranking-based matrix factorization model to learn the user’s preferences and include the geographical influence simultaneously.(vi)
IRenMF [47]: this is another state-of-the-art POI recommendation method that develops a weighted matrix factorization to model the geographical influence on POIs.(vii)
SRank: this is our proposed social-aware POI ranking method, which incorporates the social network embeddings into the pair-wise ranking framework. Compared with Geo-SRank, the only difference is that it does not consider the location network information.(viii)
Geo-SRank: this is our final recommendation method, which further extends the SRank method by utilizing the representations learned from a predefined location network.In experiments, we evaluate our method in both the non-cold-start and cold-start situations, respectively. In the non-cold-start situation, we randomly select 70% user-POI actions for training, 10% user-POI pairs for tuning, and 20% for testing. In the cold-start situation, we use the same dataset for testing and randomly select part of the training data for training (from 10% to 80%). To ensure the validity of these data, the data selection process is conducted five times independently. In these two situations, the regularization parameters ofU, V, X, and Y are set as λU=λVi=0.0025 and λVj=λX=λY=0.00025. The dimensions of the latent factors U and V are set as d=10.The experimental results evaluated on Gowalla and Yelp with non-cold-start settings have been shown in Tables2 and 3, from which we have the following observations: (1) Our Geo-SRank method achieves the best performance over all the baseline methods on both datasets. This demonstrates the superiority of the Geo-SRank method, that is, the effectiveness of our learned network embeddings and the usefulness of the social influence and the geographical information. (2) Compared with basic recommendation methods (BPRMF and WRMF), the methods (i.e., GeoMF, Rank-GeoFM, IRenMF, and Geo-SRank) that consider the geographical information can significantly improve the POI recommendation performance. This result demonstrates the usefulness of the geographical information in helping users find the right POIs. (3) The method (i.e., SRank and Geo-SRank) that considers the social influence can perform better than BPRMF and WRMF, demonstrating the importance of the social influence. Geo-SRank can achieve the best performance over all the baseline methods, again, demonstrating the usefulness of both geographical information and social influence, and we can get a better result by considering them simultaneously. (4) MF-based methods can achieve a reasonable result with low-dimensional latent feature vectors, indicating the effectiveness of MF-based methods in the POI recommendation task. In experiments, we set the dimension of all the latent factors as 10. (5) From the results, we can also observe that the accuracy (i.e., Precision, Recall, and MAP) of all the POI recommendation methods is relatively low (less than 7%). One of the important reasons might be that the check-in data in LBSNs is very sparse (the density of check-in matrix is usually less than 0.5%), and thus, the recommendation methods suffer from the data scarcity problem. Second, compared with traditional movie rating or review data, the check-in data may not fully reflect the user’s interests. For example, a user might check-in at a location where he/she is waiting for a friend or just passing by, which is not what he/she really likes.Table 2
Top-k recommendation performance on Gowalla (Δ denotes the improvement of Geo-SRank over other methods).
Overall performance comparison (Gowalla)
d
=
10
,
k
=
5
Precision@k
Δ%
Recall@k
Δ%
MAP
Δ%
MostPopular
0.01796
188
0.01159
217
0.00597
253
WRMF
0.04908
5.7
0.0334
10
0.02004
5.2
GeoMF
0.05104
1.64
0.03505
4.9
0.02085
1.1
BPRMF
0.0472
9.91
0.03254
12.9
0.01904
10.8
Rank-GeoFM
0.05139
0.9
0.03596
2.2
0.02097
0.6
IRenMF
0.05083
2.06
0.03485
5.5
0.02061
2.3
SRank
0.0496
4.59
0.03315
10.9
0.01983
6.4
Geo-SRank
0.05188
—
0.03677
—
0.0211
—Table 3
Top-k recommendation performance on Yelp (Δ denotes the improvement of Geo-SRank over other methods).
Overall performance comparison (Yelp)
d
=
10
,
k
=
5
Precision@k
Δ%
Recall@k
Δ%
MAP
Δ%
MostPopular
0.00852
201
0.0095
172
0.00444
180
WRMF
0.02384
7.9
0.02354
9.77
0.01205
3.3
GeoMF
0.02464
4.4
0.02446
5.6
0.01229
1.3
BPRMF
0.01892
35.9
0.01855
39.2
0.0085
46.4
Rank-GeoFM
0.02521
2
0.02536
1.8
0.01234
0.8
IRenMF
0.02440
5.4
0.02430
6.3
0.01216
2.3
SRank
0.02448
5.1
0.02515
2.7
0.01221
1.9
Geo-SRank
0.02573
—
0.02584
—
0.01245
—


### 4.4. Evaluation on Cold-Start Settings

To evaluate the recommendation performance of our method in cold-start situations, we further conduct experiments with cold-start settings. In this experiment, we randomly select part of the training data for training. The training ratio is changed from 10% to 80%. In experiments, we compare our method with the baseline method BPRMF and the state-of-the-art POI recommendation method GeoMF. As BPRMF does not consider any other context information, it will use randomly generated feature values for new uses and items. But for GeoMF, as it not only exploits the basic user-item interactions to make predictions but also considers the geographical context, it will employ the location information to alleviate the cold-start problem. The experimental results with different training ratios are shown in Figure6, from which we can find that, with the increase of the training data, the performance of these three methods both increases, which indicates that, with more training data, we can make our model more accurate. From this result, we can also find that our feature-based method can work well in these two datasets, which demonstrates that treating the learned network embeddings as prior knowledge of the latent factors is effective and can further improve the recommendations for those cold-start users. A larger margin between BPRMF and Geo-SRank can be found in Yelp than Gowalla, which is consistent with the results experimented on the non-cold-start data.Figure 6
Experimental results with different training ratios (k = 5). (a, b) Gowalla and (c, d) Yelp.
(a)
(b)
(c)
(d)


### 4.5. The Influence of the Recommendation List

In POI recommendation, the system usually allows users to specify the POI numbers that shall be returned to her/him to avoid being overwhelmed by irrelevant POIs. Appropriate POI number helps to improve the user experience in recommender systems. Therefore, we further investigate the performance of our method with varying the number of the recommended POIs. The Top-k recommendation result is shown in Figure 7, where only the Top-k most relevant POIs are considered. From this result, we can observe that, with the increase of the recommendation list, the hits of the relevant POIs are increasing, hence resulting in the increase of Recall@k and the decrease of Precision@k. The increase of Recall@k means the predictions can cover more relevant POIs, that is, the capability of the recommendation method to retrieve relevant POIs is increasing. The decrease of Precision@k represents the predictions can only cover less relevant POIs, i.e., the capability of the recommendation method to retrieve correct POIs is decreasing.Figure 7
Experimental results with different recommendation numbers. (a, b) Gowalla and (c, d) Yelp.
(a)
(b)
(c)
(d)


### 4.6. The Influence of Latent Factor Dimensions

The dimension of the latent factors controls how much information can be used to represent users and items (In this experiment, we setU and V as the same dimension.). If the dimension is too small, the learned features will not be sufficient to represent the user and POI information. If the dimension is too large, not only will the learned features appear to be duplicated, but also the complexity of the algorithm will increase. Appropriate feature dimension plays a crucial role in our algorithm.Figure8 shows the ranking performance of our method with the feature dimension changes from 10 to 80, from which we can observe that, with the feature dimension increases, the ranking performance of BPRMF and Geo-SRank also increases gradually. But when its value surpasses a certain threshold (such as 50), the increase begins to slow down, which indicates that when the dimension is big enough, there will be enough information for extracted features to represent the social and location networks. In our experiments, we find that, with very low dimension of these latent factors, the recommendation can achieve comparable results. To reduce the computational complexity of our method, in experiments both user and POI dimensions are set as d=10.Figure 8
The experimental results with different latent factor dimensions. (a, b) Gowalla and (c, d) Yelp.
(a)
(b)
(c)
(d)


### 4.7. Convergence Analysis

To demonstrate the training efficiency of our feature-based recommendation method, we show the evaluation results with increasing training iterations in Figure9. In this experiment, the learning rates of BPRMF and Geo-SRank are both set as 0.05. From this result, we can observe that, with the increase of the iterations, the Geo-SRank method can converge (due to the stochastic gradient descent algorithm we used, after convergence these two methods still fluctuate around a certain value) very fast in the Gowalla data, which almost converges within the same iterations with BPRMF (after about 20 iterations). But it converges a little slow in Yelp data (about 100 iterations), and BPRMF can also converge within 20 iterations. The reason of this result is that we introduce two external feature vectors in our method, and the increased random variables slow down the converge rate. From this result, we can also find our method can perform better than BPRMF throughout the whole learning process, which proved the effectiveness of these extracted network embeddings.Figure 9
The experimental results with training iterations on the cold-start data. (a) Precision@k on Gowalla, (b) Recall@k on Gowalla, (c) Precision@k on Yelp, and (d) Recall@k on Yelp.
(a)
(b)
(c)
(d)


## 4.1. Datasets and Evaluation Metrics

In our experiments, two real-world datasets Gowalla and Yelp [12] are exploited to evaluate our proposed method (Table 1). Gowalla was launched in 2007 and closed in 2012, which provides the location and social services that enable users to check in at places and make friends based on where they have visited. The Gowalla dataset (https://snap.stanford.edu/data/loc-gowalla.html) we used was crawled by its API from February 2009 to October 2010. By filtering out those users with fewer than 15 check-in POIs and those POIs with fewer than 10 visitors, we can reach our final dataset, which contains 25,379 users, 32,623 POIs, 1,395,856 check-ins, and 118,717 social ties. Yelp is a local-search service powered by crowdsourced review forum, which publishes crowdsourced reviews about local businesses, as well as the online reservation service. Yelp was founded in 2004 and grew very quickly. By the end of 2017, Yelp has 148 million reviews. The dataset we used was collected by Liu et al. [12], which contains 30,887 users, 18,942 businesses, 561,746 check-ins, and 265,533 social ties.Table 1
Statistics of the training data of Gowalla and Yelp.
Statistics
Yelp
Gowalla
Check-in sparsity
99.903
99.906
No. of users
30,887
18,737
No. of POIs
18,942
32,495
No. of check-ins
561,746
566,791
Avg. No. of users per POI
0.6
0.57
Avg. No. of POIs per user
1.6
1.73To measure the performance of our recommendation approach, we exploit three popular measures Precision@k, Recall@k, and mean average precision (MAP) [12] as the evaluation metrics. In this Top-k item recommendation task, Precision@k measures how many relevant POIs can be predicted in the recommendation list. Here, the relevant POIs refer to the ones that have been labeled as positive in the dataset. Recall@k denotes how many relevant POIs can be predicted among the total relevant POIs in the dataset. The formal definitions of these two metrics are as follows:(26)Precision@k=1M∑u=1MLku∩LTuk,(27)Recall@k=1M∑u=1MLku∩LTuLTu,where M is the number of being tested users and LTu is the POI set that have been checked in by u in the test data. Lku represents the recommended Top-k POI list for user u (k is the size of this list). In this work, the list size k is set as 5. MAP measures the average precision (AP) over all users, which is computed by(28)MAP=1M∑u=1mAP=1M∑u=1m1k∑i=1kPrecision@i,where Precision@i refers to the precision for Top-i recommendations.


## 4.2. Data Analysis

To better understand the users’ visiting behaviors in location-based social networks, we further investigate the social influence and geographical characteristics in Gowalla and Yelp and try to investigate the following questions: (1) Are users with friend relationships more likely to have similar check-in activities? (2) Are POIs with neighboring relationships more likely to be visited by the same set of users? To answer these two questions, we need to first define a method to measure the similarity between two users.LetAu denote the POI set that has been visited by user u in history. The similarity between user u and v can be computed as(29)xuv=∑i∈Au∩Av1/log1+AiAuAv,where Ax represents the size of the POI set Ax and 1/log1+Au is the item that punishes the influence of popular POIs.For each useru, two similarities are computed, that is, (1) the average similarity between user u and his/her friends (xfu) and (2) the average similarity between user u and randomly selected users. For fairness, we choose the same number of users with their friend relationships. Figure 4 plots the kernel-smoothing density estimations based on the vectors xf and xr, from which we can observe that xr has smaller similarity values than xf. This result positively answers our first question, that is, users with friend relationships share more common interests than those without.Figure 4
Density estimation of users’ similarity on (a) Gowalla and (b) Yelp.
(a)
(b)To answer the second question, the measurement of the similarity between two POIs needs to be predefined. By denotingBi as the user set that has checked in location i, the similarity between POI i and j is arrived at(30)yij=∑u∈Bi∩Bj1/log1+BuBiBj.Based on this similarity, Figure5 shows how the similarity of randomly chosen POI pairs varies with the increase of geographical distance, where we can find that, with the increase of the distance, the mean value of the POI similarity decreases in both Yelp and Gowalla data. This result demonstrates the positive relationship between the POI similarity and geographical distance and provides us a positive answer to the second question: POIs with neighboring relationships are inclined to be visited by similar users.Figure 5
The similarity between POIs varies with geographical distance: (a) Yelp and (b) Gowalla.
(a)
(b)


## 4.3. Performance Comparison

To evaluate the recommendation performance of our method, we conduct comparisons with several related recommendation methods. As the matrix factorization- (MF-) based method has been proved to have state-of-the-art performance, we mainly compare against the MF-based collaborative filtering models and the related ranking-based method (most of these methods are implemented in MyMediaLite [63]):(i)
MostPopular: this is a basic recommendation method, which ranks the POIs according to how many users have visited them.(ii)
WRMF [45]: this method is proposed for the one-class collaborative filtering problem and provides a solution by giving small weights to negative samples and large weights to positive samples.(iii)
GeoMF [4]: this is a state-of-the-art POI recommendation method, which extends the WRMF method by augmenting the user and POI latent factors.(iv)
BPRMF [7]: this item recommendation method is proposed for implicit feedback, which learns the user interests from their pair-wise item preferences. This is the state-of-the-art POI recommendation method, which is only based on check-in matrix, without utilizing any context information.(v)
Rank-GeoFM [5]: this is a state-of-the-art POI recommendation method that develops a ranking-based matrix factorization model to learn the user’s preferences and include the geographical influence simultaneously.(vi)
IRenMF [47]: this is another state-of-the-art POI recommendation method that develops a weighted matrix factorization to model the geographical influence on POIs.(vii)
SRank: this is our proposed social-aware POI ranking method, which incorporates the social network embeddings into the pair-wise ranking framework. Compared with Geo-SRank, the only difference is that it does not consider the location network information.(viii)
Geo-SRank: this is our final recommendation method, which further extends the SRank method by utilizing the representations learned from a predefined location network.In experiments, we evaluate our method in both the non-cold-start and cold-start situations, respectively. In the non-cold-start situation, we randomly select 70% user-POI actions for training, 10% user-POI pairs for tuning, and 20% for testing. In the cold-start situation, we use the same dataset for testing and randomly select part of the training data for training (from 10% to 80%). To ensure the validity of these data, the data selection process is conducted five times independently. In these two situations, the regularization parameters ofU, V, X, and Y are set as λU=λVi=0.0025 and λVj=λX=λY=0.00025. The dimensions of the latent factors U and V are set as d=10.The experimental results evaluated on Gowalla and Yelp with non-cold-start settings have been shown in Tables2 and 3, from which we have the following observations: (1) Our Geo-SRank method achieves the best performance over all the baseline methods on both datasets. This demonstrates the superiority of the Geo-SRank method, that is, the effectiveness of our learned network embeddings and the usefulness of the social influence and the geographical information. (2) Compared with basic recommendation methods (BPRMF and WRMF), the methods (i.e., GeoMF, Rank-GeoFM, IRenMF, and Geo-SRank) that consider the geographical information can significantly improve the POI recommendation performance. This result demonstrates the usefulness of the geographical information in helping users find the right POIs. (3) The method (i.e., SRank and Geo-SRank) that considers the social influence can perform better than BPRMF and WRMF, demonstrating the importance of the social influence. Geo-SRank can achieve the best performance over all the baseline methods, again, demonstrating the usefulness of both geographical information and social influence, and we can get a better result by considering them simultaneously. (4) MF-based methods can achieve a reasonable result with low-dimensional latent feature vectors, indicating the effectiveness of MF-based methods in the POI recommendation task. In experiments, we set the dimension of all the latent factors as 10. (5) From the results, we can also observe that the accuracy (i.e., Precision, Recall, and MAP) of all the POI recommendation methods is relatively low (less than 7%). One of the important reasons might be that the check-in data in LBSNs is very sparse (the density of check-in matrix is usually less than 0.5%), and thus, the recommendation methods suffer from the data scarcity problem. Second, compared with traditional movie rating or review data, the check-in data may not fully reflect the user’s interests. For example, a user might check-in at a location where he/she is waiting for a friend or just passing by, which is not what he/she really likes.Table 2
Top-k recommendation performance on Gowalla (Δ denotes the improvement of Geo-SRank over other methods).
Overall performance comparison (Gowalla)
d
=
10
,
k
=
5
Precision@k
Δ%
Recall@k
Δ%
MAP
Δ%
MostPopular
0.01796
188
0.01159
217
0.00597
253
WRMF
0.04908
5.7
0.0334
10
0.02004
5.2
GeoMF
0.05104
1.64
0.03505
4.9
0.02085
1.1
BPRMF
0.0472
9.91
0.03254
12.9
0.01904
10.8
Rank-GeoFM
0.05139
0.9
0.03596
2.2
0.02097
0.6
IRenMF
0.05083
2.06
0.03485
5.5
0.02061
2.3
SRank
0.0496
4.59
0.03315
10.9
0.01983
6.4
Geo-SRank
0.05188
—
0.03677
—
0.0211
—Table 3
Top-k recommendation performance on Yelp (Δ denotes the improvement of Geo-SRank over other methods).
Overall performance comparison (Yelp)
d
=
10
,
k
=
5
Precision@k
Δ%
Recall@k
Δ%
MAP
Δ%
MostPopular
0.00852
201
0.0095
172
0.00444
180
WRMF
0.02384
7.9
0.02354
9.77
0.01205
3.3
GeoMF
0.02464
4.4
0.02446
5.6
0.01229
1.3
BPRMF
0.01892
35.9
0.01855
39.2
0.0085
46.4
Rank-GeoFM
0.02521
2
0.02536
1.8
0.01234
0.8
IRenMF
0.02440
5.4
0.02430
6.3
0.01216
2.3
SRank
0.02448
5.1
0.02515
2.7
0.01221
1.9
Geo-SRank
0.02573
—
0.02584
—
0.01245
—


## 4.4. Evaluation on Cold-Start Settings

To evaluate the recommendation performance of our method in cold-start situations, we further conduct experiments with cold-start settings. In this experiment, we randomly select part of the training data for training. The training ratio is changed from 10% to 80%. In experiments, we compare our method with the baseline method BPRMF and the state-of-the-art POI recommendation method GeoMF. As BPRMF does not consider any other context information, it will use randomly generated feature values for new uses and items. But for GeoMF, as it not only exploits the basic user-item interactions to make predictions but also considers the geographical context, it will employ the location information to alleviate the cold-start problem. The experimental results with different training ratios are shown in Figure6, from which we can find that, with the increase of the training data, the performance of these three methods both increases, which indicates that, with more training data, we can make our model more accurate. From this result, we can also find that our feature-based method can work well in these two datasets, which demonstrates that treating the learned network embeddings as prior knowledge of the latent factors is effective and can further improve the recommendations for those cold-start users. A larger margin between BPRMF and Geo-SRank can be found in Yelp than Gowalla, which is consistent with the results experimented on the non-cold-start data.Figure 6
Experimental results with different training ratios (k = 5). (a, b) Gowalla and (c, d) Yelp.
(a)
(b)
(c)
(d)


## 4.5. The Influence of the Recommendation List

In POI recommendation, the system usually allows users to specify the POI numbers that shall be returned to her/him to avoid being overwhelmed by irrelevant POIs. Appropriate POI number helps to improve the user experience in recommender systems. Therefore, we further investigate the performance of our method with varying the number of the recommended POIs. The Top-k recommendation result is shown in Figure 7, where only the Top-k most relevant POIs are considered. From this result, we can observe that, with the increase of the recommendation list, the hits of the relevant POIs are increasing, hence resulting in the increase of Recall@k and the decrease of Precision@k. The increase of Recall@k means the predictions can cover more relevant POIs, that is, the capability of the recommendation method to retrieve relevant POIs is increasing. The decrease of Precision@k represents the predictions can only cover less relevant POIs, i.e., the capability of the recommendation method to retrieve correct POIs is decreasing.Figure 7
Experimental results with different recommendation numbers. (a, b) Gowalla and (c, d) Yelp.
(a)
(b)
(c)
(d)


## 4.6. The Influence of Latent Factor Dimensions

The dimension of the latent factors controls how much information can be used to represent users and items (In this experiment, we setU and V as the same dimension.). If the dimension is too small, the learned features will not be sufficient to represent the user and POI information. If the dimension is too large, not only will the learned features appear to be duplicated, but also the complexity of the algorithm will increase. Appropriate feature dimension plays a crucial role in our algorithm.Figure8 shows the ranking performance of our method with the feature dimension changes from 10 to 80, from which we can observe that, with the feature dimension increases, the ranking performance of BPRMF and Geo-SRank also increases gradually. But when its value surpasses a certain threshold (such as 50), the increase begins to slow down, which indicates that when the dimension is big enough, there will be enough information for extracted features to represent the social and location networks. In our experiments, we find that, with very low dimension of these latent factors, the recommendation can achieve comparable results. To reduce the computational complexity of our method, in experiments both user and POI dimensions are set as d=10.Figure 8
The experimental results with different latent factor dimensions. (a, b) Gowalla and (c, d) Yelp.
(a)
(b)
(c)
(d)


## 4.7. Convergence Analysis

To demonstrate the training efficiency of our feature-based recommendation method, we show the evaluation results with increasing training iterations in Figure9. In this experiment, the learning rates of BPRMF and Geo-SRank are both set as 0.05. From this result, we can observe that, with the increase of the iterations, the Geo-SRank method can converge (due to the stochastic gradient descent algorithm we used, after convergence these two methods still fluctuate around a certain value) very fast in the Gowalla data, which almost converges within the same iterations with BPRMF (after about 20 iterations). But it converges a little slow in Yelp data (about 100 iterations), and BPRMF can also converge within 20 iterations. The reason of this result is that we introduce two external feature vectors in our method, and the increased random variables slow down the converge rate. From this result, we can also find our method can perform better than BPRMF throughout the whole learning process, which proved the effectiveness of these extracted network embeddings.Figure 9
The experimental results with training iterations on the cold-start data. (a) Precision@k on Gowalla, (b) Recall@k on Gowalla, (c) Precision@k on Yelp, and (d) Recall@k on Yelp.
(a)
(b)
(c)
(d)


## 5. Conclusions and Future Work

In this work, we focused on how to make accurate POI recommendations in LBSNs and proposed a location-aware social recommendation method with pretrained network embeddings. To utilize the social relation and geographical information, we pretrained the network embedding method node2vec to learn the representations of users and POIs from the users’ social network and the predefined location network, respectively. To incorporate the learned features into collaborative filtering model, we extended user and POI latent feature factors in a pair-wise ranking framework, where the learned embeddings are treated as their prior knowledge. Experimental results on two real-word datasets demonstrate the advantages of our method in the both non-cold-start and cold-start settings. Although we only tested our method on Gowalla and Yelp, Geo-SRank can also be applied to other real-world settings. For example, when a user planning a trip on Mafengwo (https://www.mafengwo.cn/) (a famous communication platform for travellers in China), he/she may want to ask his/her friends first and then pick out the POIs that have been visited by the users that have similar interests with him. Similarly, we can also help users to find out the restaurants that they want to have dinner with friends. In the above cases, both the social influence and geographical information help us make the right choices.This work mainly investigates how to utilize the pretrained representations from social relation and location networks, but the check-in sequence is not considered. In real life, the users’ check-in sequence is also an important aspect of their interest, for example, users always like to go to the playground and then go to the supermarket for shopping. In future, we plan to leverage the check-in sequence information to further enhance the POI recommendation performance.



---

*Source: 3574194-2019-11-04.xml*