Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The reviewed paper considers the problem of 2D localization of an observer on a given topological map. A data-driven approach is proposed to solve the problem. While the topological map of the environment is required, no prior information is needed about the nature of the input signal. Moreover, no supervision (in terms of pairs of signal measurement and observer's location) is needed.

The work formulates the localization problem as the problem of learning a low-dimensional manifold, representing the input in their intrinsic space and then transport them to the given topological map by inferring correspondences between the two spaces.
The work is built on two intuitions:
1. that the transformation from high-dimensional input to the intrinsic space can be modeled as a mapping that can be learned through a neural network;
2. temporal data is subject to gradual changes as the observer moves, suggesting that the data reside on a smooth, locally connected manifold;

Results from experiments on three different environments (and different sensory inputs) support the claims.

main_review: The proposed solution to the 2D localization problem appears to be technically sound and well documented.
The document is clearly written and organized. 
Though the amount of information and references is large, the document leaves a reader familiar with the subject with enough pointers to reproduce the work.

A few minor typos:
- Line 162: "Euc**l**idean"
- Line 186: "Re**q**ularizing"
- Line 252: Missing reference to a figure is shown as "??"
limitations_and_societal_impact: The authors should consider a more extensive description of the limitations of the proposed approach. For example, in the experiments reported in the document, the only source of disturbance/signal was the observer and its motion itself. What happens when other people/objects occupy/walk the environment at the same time? Obviously, in these scenarios, solutions to localization involving cameras have a clear advantage, but it would be useful to the reader to get a sense of what kind of problems can arise in those scenarios.

No clear presence of potential negative social impact in this work.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper presents a new method for localization while being agnostic to input measurement signal's modality. Assuming a map is given, localization inside the map is formulated as a deep manifold learning problem, without needing the supervision of ground truth positions for each input measurement. The method utilizes differentiable optimal transport to solve the correspondence challenge in this formulation. The method is tested on several simulated image datasets and one real-world WiFi dataset.
main_review: *Strength*:    
1. The problem formulation based on manifold learning seems novel. And the use of optimal transport is an interesting approach.   
2. The method allows modal-agnostic localization, due to the use of deep networks (similar to PoseNet).
3. The method is tested on three different types of experiments, showing satisfactory results.

*Weakness*:   
Major:   
1. Lack of many important technical details:   
    1.1. Some important symbols are not explained, which makes it very difficult to understand what has been done. For example, what is $y_j$? And how is $\Omega_t$ sampled (sparely sampled on the trajectory only, or randomly and densely covering the testing environment)? What is V, E in Figure 2?  
    1.2. The method did not mention at all how the map $\Phi$ can handle variations in $\Omega_s$ caused by camera rotations, which is a key question on whether the map can localize the camera when it takes two photos at the same position but with different orientations.     
    1.3. How are the positive and negative samples in the triplets generated? What's the influence of the hyperparameters in this module (3.3.1) on the whole method?
    1.4. There is no experiment/ablation study to justify the need/importance for equation (6) instead of Euclidean distance in a local area.
2. Lack of discussions of the failure cases. Would this method fail in some cases? For example, the experiments never showed the trajectory in symmetric environments/maps. If $\Omega_t$ is a square or rectangle, how can this method handle the rotational ambiguities? How does it know that an image is taken at the top-left corner instead of the top-right corner (because the optimal transport cost could be the same in this case)?
3. Lack of comparisons to baseline methods, and related work discussions on other weakly supervised training of the localization function $\Phi$ (essentially a PoseNet [Kendall 2015]).
4. Lack of real-world image-based localization experiments.

Minor:   
5. Section 3.2 may be shortened since many equations are NOT proposed by this work, but are well-known in the community already. This would save space to include the above missing important details.    
6. A topological map has a special meaning in SLAM and localization (no global metric information), which is different from the one used in this paper. It is suggested to change the term to a metric map, because it seems that you do need to make metric measurements (even if it could be noisy) to construct $\Omega_t$.    
7. Is it stable to train the deep localization network with the differentiable Sinkhorn? I wish to see some training losses.   
8. What if you use L2 instead of KL for L(.,.)? Any ablation study? Do you have any reference to support your claim that "choosing Eucidean distances for L is ill-posed"?
limitations_and_societal_impact: I like the raw idea of the paper, and the problem being addressed is an important one in robotics. However, the lack of details and the quality of the presentation make it very difficult for me to recommend this paper for acceptance.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper presents a novel generic localization framework that is agnostic to the data modalities and doesn't require correspondence labels. The paper demonstrates the effectiveness of their framework through experiments on three different environments, each of which has a different data modality. 
main_review: Strengths

S1 The idea of designing a generic localization framework is interesting and can inspire future research works. 
 
S2 The design of the proposed framework is novel and clever.

S3 The paper is well written and easy to follow. The theoretical motivation part in Section 3.1 is interesting and inspiring.

S4 The experiments cover three different data modalities, which is great. The experiment results are also very strong. 


Comments

C1 I wonder how the proposed framework performs in buildings where there are a lot of similar places. For example, in a hotel, many rooms are identical. If we look at the corridors of a hotel, an apartment, or a dormitory, there could be a lot of visually similar locations. 

C2 In Sections 4.2 and 4.3, do we assume that the moving distances between any two consecutive observations are identical and preknown? This seems to be true for the maze environment. I'm not sure if the proposed framework relies on this assumption.

limitations_and_societal_impact: No potential negative societal impact.
review_title: null

Meta-review: The paper proposes an algorithm for 2D topological localization that is largely agnostic to the type of observation data. Formulated as a deep manifold learning problem, the framework transforms the (potentially) high-dimensional observation into a low-dimensional embedding. The method then uses optimal transport to identify the correspondence between this representation and that of the topological map. Importantly, this correspondence is learned without ground-truth correspondences. The paper presents experimental results using simulated camera observations as well as radio frequency (WiFi) observations.

Localization has been researched in the robotics community for several decades and is considered by many to be a problem that is all but solved. As the paper notes, traditional methods employ measurement models that are specific to the particular sensing modality(ies). To some extent, these models provide an abstraction of the specific sensor, and the underlying localization algorithms tend to be more sensor-agnostic. The capacity for the end-to-end framework to be truly independent of the particular sensor, however, is compelling. The way in which the method employs manifold learning and optimal transport is interesting and is technically sound. The experiments demonstrate the algorithm's ability to generalize to very different sensing modalities, however experiments involving real rather than simulated images would provide more compelling evidence. The reviewers raised a few concerns regarding the need for a clearer discussion of the algorithm's limitations and the omission of various technical details, which the authors largely clarified in their response. These issues should be addressed and the authors are encouraged to consider the possibility of including an experimental evaluation on real-world imagery.
Decision: Accept (Poster)