Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper presents a new approach for route recommendation. The key idea of the approach is to model the mobility pattern as a Markov process and learn the local transition probability from historical data through a GCN with Lipschitz embedding. The proposed approach overcomes two limitations in the existing works - (1) limited reachability and (2) limited generalizability on unseen/lesser-seen nodes. The paper evaluates their approach on public datasets with a few baselines. The improvement is significant and there are also a good amount of ablation studies and case studies to help the readers better understand the effectiveness of the proposed solution. 
main_review: Strengths:

S1 The idea of this paper has its novelty in route recommendation. The way it models the route recommendation problem can inspire future works.   

S2 The paper is well written. It is easy to follow and contains sufficient details. 

S3 The experiments are sufficient and well-designed. 


Weaknesses:

W1 The precision and recall seem low but this may not be an issue of the proposed approach but the metric itself. There might be several reasonable routes going from A to B, and not all the drivers will follow the same route. By this nature, there should be an upper bound of the accuracy. For example, if there are 2 reasonable routes from A to B, and 80% of the drivers take route-1 (most likely route) and 20% of the drivers take route-2 (assume the traffic conditions and all other factors are the same). In this case, you may only be able to achieve 80% precision if the routing queries are all anonymous. Given the existence of this upper bound, the accuracy numbers reported in the paper might be good enough.


Additional Comments:

C1 I wonder if there is a way to estimate the upper bound (see W1) of the accuracy from the data? Showing this estimation can be helpful to understand the nature of the most likely route prediction problem. 

C2 At line 254, training all the models for up to 36 hours does not seem to be a fair comparison. It would be better to fix the training epochs rather than training time. 

C3 How does the proposed approach handle changes in the road network? For example, there could be temporary road closure caused by construction or crashes. 

C4 At line 104, should it be Prob. 1 rather than Prob. 2?




limitations_and_societal_impact: No potential negative societal impact.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a method for recommending travel routes in a city. As shown in Figure 2, in order to consider the meaningful representation of road network, this paper uses a combination of Graph Convolutional Network with Lipschitz embeddings. Multi-layered Perceptron (MLP) is then applied to predict the next visit.
main_review: - Strengths
  - This paper is well-structured and easy to follow.
  - The proposed architecture seems reasonable for this task. Through the ablation study (Sec. 5.4), it has also been shown that each component of the proposed method (Lipschitz embeddings and GCN) is working effectively.
  - The minimum necessary comparisons with two conventional methods have been made using multiple data sets.
- Weaknesses
  - The novelty of the proposed method should be described in more detail. Is the key point of this work is to devise a pre-processing (PCA and Lipschitz embedding)? Does the majority of the proposed architecture relies on existing methods?
  - I am concerning about whether the comparison experiment was done under fair conditions. (1) The authors set a restriction of 36 hours for the parameter training time, but has this restriction affected the experimental results? (e.g., would the same conclusion be reached if the time were 72 hours?). (2) For both DEEPST and CSSRNN, the authors use the default parameters prescribed by these papers. However, unless the parameters of the existing methods are learned in a way that is tailored to the current data, it may not be a fair comparison between the proposed method and the conventional method. (3) Why did the authors use only one of three datasets for the experiments on "Accuracy" and "Reachability"? I am also curious about the large difference in performance between each dataset (each town).
limitations_and_societal_impact: Limitation of the proposed method is mentioned in the conclusion. The negative impacts of this study were not mentioned in the paper, nor were there any points that bothered me when I read the paper.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper introduces a route recommendation model based on graph neural networks with Lipschitz embedding initialization. The proposed model is compared with several baseline models and the evaluation shows that it outperforms baselines in terms of precision, recall, and reachability. Ablation study is done to demonstrate the importance of both Lipschitz embedding and GCN components. The model which uses inductive learning shows that it significantly outperforms baselines when the source/destination node is unpopular. 
main_review: The model is easy to understand and explained clearly in the paper. There are ablation studies to help understand the impact of each component in the model. The results look promising compared with the baseline in the paper.

However, I feel like there should be more exploration on the model component as GCN is only one possible way of implementing graph neural networks. What if different GNN architectures are used? Have you tried them? Are there any GNN based model for route recommendations? Maybe it's more fair to compare with them? It seems to me that GNN is only used to learn node embeddings and then an MLP is applied to generate the transition probability. Have you tried GNN models that can learn edge embeddings as well and include edge embeddings in the MLP input? Have you tried different ways to learn these embeddings and compare them? It seems that the proposed model mostly stack existing models together so the real novelty is not very significant. It would be better if the paper could explain or explore more on the alternatives, for example, maybe using GNN to also execute graph algorithms (https://openreview.net/forum?id=SkgKO0EtvS) instead of using MLP to generate transition probabilities. 

There should be more explanation on why shortest path or quickest path does not perform well. I would assume in the dataset, people use existing apps to navigate which would potentially give a route based on a combination of route distance and time (which may consider live traffic). So probably crudely using shortest/quickest path without the live traffic is why is not performing well. 

For Reachability Distance in the evaluation metrics, it seems to me network distance is better than Haversine distance? If two locations are not physically reachable in the road network, it really doesn't matter how close they are in terms of Haversine distance.

In terms of inductively learning, I think a better way to advance it is to handle dynamic network, such as road closures or new road being introduces. This would be more appealing in a practical sense. 
limitations_and_societal_impact: I don't see any potential negative societal impact of their work.
review_title: null

Meta-review: This paper proposes a method for route recommendation using graph neural networks and Lipschitz embedding. The idea of the proposed method for route recommendation is interesting. The proposed architecture is reasonable. The paper is well written, and experiments are well-designed. As commented by the reviewers, the novelty needs to be clearly explained in the revised paper. The experimental results showed in the author response would strengthen the paper.
Decision: Accept (Poster)