Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper presents an integrated deep-based approach for visual place recognition (VPR) that combines sequence learning of feature descriptors with sequence matching. Particularly, a Euclidean distance metric is introduced in a triplet loss, to provide order-preservation of single image descriptors in the sequence. To facilitate the training, the authors also a propose a simple method for negative mining based on the Euclidean distance metric in the sequence data.
strengths_and_weaknesses: The paper has a very rigorous examination of the state of the art works. I find the idea of integrating the sequence matching into the training process of VPR reasonable, The paper is well-written and easy to follow. Adding an algorithmic block describing the training process would be helpful.
My only concern regards the significance of the provided results and the fundamental differences of the proposed work w.r.t. SeqNet, that uses learning of image descriptors in sequences and produces hypotheses on sequence matches, to be later on refined.
summary_of_recommendation: UPDATE: After rebuttal and the update of the paper with stronger ablations, I change my score to Strong accept, as I believe the paper bring significant improvement w.r.t. the prior works.

My recommendation is mainly driven by the concern described above, namely I am not convinced about the benefit of the method compared to SeqNet, as the result of table 3 does not seem significant. 
issues: While the paper is well-written and well-organized, my concern regard the similarities and differences between SeqNet and SeqMatchNet. It seems that the main component of the method is SeqNet with an extension to the triplet loss. This is of course fine, as long as a rigorous examination of the benefits of the one method over the other is demonstrated. Firstly, it would be interesting to ablate the length of the sequence, as only a single vs. Length of 5 is provided. Moreover, the authors claim state of the art results in Table.4 however the increase in recall are marginal or same as SeqNet. Is it possible to provide evidence for the significance of the prediction improvement? 
I think this will also benefit the community regarding the research direction you are proposing (i.e., embedding sequence matching in the training process).
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposes a sequence matching learning formulation for place recognition and relocalization. Instead of learning the image metric with single images, the authors use the distance between images of the sequences in contrast for learning the feature embedding. A negative mining method is also used to improve the training process. The final results on several commonly used benchmarks show the proposed method is very promising.
strengths_and_weaknesses: STRENGTHS

The paper considers the sequence information to learn image embedding for place recognition and relocalization. I agree that it is an important source of information to explore.

WEAKNESSES

The feature learning with single-layer network is a curious choice. Usually, deep networks can provide better embedding with SGD. Some exploration of the embedding network, especially depth, is important for understanding the loss.

The ablation study on different training and testing schemes will be very helpful for understanding the method. The variants are 1) training on sequence and test on single images. 2) training on single images and test with a sequence?

It is unclear whether the major improvement comes from negative mining or not.

Failure case analysis will also help understand the results.

summary_of_recommendation: Although I agree that learning from sequences can be better, there are still a lot of unknowns for the current experiments.

----

After reading author's feedback, my concerns are largely addressed. I changed my recommendation to weak accept.
issues: I am willing to improve my rating if the authors can provide strong evidence that the loss can be used for deeper embedding networks. The additional study mentioned in the weaknesses is important to make the design convincing.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper presents an approach for visual place recognition dubbed SeqMatchNet. The idea is to bridge the gap between sequence matching methods and robustifying image descriptors for visual place recognition. This is accomplished by introducing a contrastive triplet loss which instead of being computed on the descriptor of a single query image, is computed instead on the distance metric calculated on a sequence of image descriptors. In order to speed up computation, the distance metric can be efficiently computed by a convolution operation. The method is evaluated thoroughly on a number of datasets.
strengths_and_weaknesses: The paper proposes a unique approach for bridging the gap between the different research directions in the area of visual place recognition, and demonstrates the utility of combining both a contrastive loss approach with sequence matching.

The list below should be viewed more as questions that are unclear to this reviewer rather than weaknesses of the approach:
1) For the sequence-based distance metric: one of the advantages claimed is that negatives mined during training are harder than those selected using single image based training. Can you elaborate more on why this is the case? Is there some empirical evidence supporting this claim in your findings?
2)  For table 1 and table 2, what is the baseline approach used? Is it a simple NetVlad model?
3) In line 251, the good performance is attributed to the importance of optimizing single image descriptors. Where in the presented approach are the image descriptors optimized?
4) For the cross-city VPR performance, it would be interesting to see the performance of other SOTA methods as well. Currently, it's hard to judge the performance of this approach without this comparison.
5) Finally a bit of a naive question, but how do the approaches generalize to a new dataset after being trained on a particular training scene? The dataset and the matching matrices need to be computed from scratch for every scene correct? So where does the generalization bit come in? 
summary_of_recommendation: I do not feel strongly about accepting nor rejecting the paper without having the above questions clarified.

EDIT:

After the author's responses I feel more confident in arguing for the acceptance of the work.
issues: See above.
review_title: null

Meta-review: The idea proposed by the authors has been reasonably well received by the reviewers. Combining sequential information and learning of image descriptors showed to improve the results w.r.t. other baselines. While existing time contrastive learning methods have relied on sequences of images, the idea of leveraging sensor information for obtaining frame-level correspondences during training is well thought and clean.
The paper has a good technical quality and it is clear to read.

However, as pointed out by the reviewers,  certain parts of the paper have caused confusion and require more support such as the case for mining negatives and their real benefit, deeper analyses including ablation studies on sequence lengths, design choices on the network depth, and its benefits in regards to SeqNet.

Post-rebuttal ============

Thank you for the clarifications and discussions. The reviewers have been in general pleased with the answers and many of the concerns have been solved in the updated manuscript.

Decision: Accept (Oral)