Kati8/ICLR_2023_2025 huggingface dataset:
authors: Xuelun Shen
authors: zhipeng cai
authors: Wei Yin
authors: Matthias Müller
authors: Zijun Li
authors: Kaixuan Wang
authors: Xiaozhi Chen
authors: Cheng Wang
reviews: 
Comments: The paper introduces the GIM framework, aiming to address the challenge of generalizing image matching to diverse real-world scenarios. Taking cues from established computer vision models, GIM adopts a zero-shot generalization approach, leveraging the vast and varied nature of internet videos. The methodology involves a two-step training process: initial training on domain-specific datasets followed by an integration with several image matching techniques. This combined model seeks to identify potential correspondences in close video frames, with outlier detection mechanisms ensuring data quality. Notably, while traditional methods such as SfM and MVS have exhibited limitations in handling in-the-wild videos, GIM claims to efficiently provide reliable supervision signals for these scenarios, thereby promising to push the boundaries of current state-of-the-art models.
Strengths: + The introduction of the GIM framework is a significant advancement, marking the first attempt to leverage internet videos in training a universally applicable image matcher. This initiative promises to address generalization challenges across multiple real-world contexts.

+ The formulation of the ZEB benchmark is a noteworthy achievement. As a pioneering zero-shot evaluation benchmark integrating real-world and simulated data, ZEB is poised to become an instrumental tool in gauging the generalization capacities of existing models.
Weaknesses: - The heavy reliance on internet videos for training might introduce biases or noise. The generalization capability of GIM, when trained on other diverse datasets, remains an unanswered question.

- With the consistent performance improvement with increased video data, there might be concerns about potential overfitting. Addressing this, perhaps with regularization techniques or other measures, would be crucial.

- Internet videos can be noisy, and their quality can vary. How resistant is GIM to such noise, and how does it handle low-quality data?
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 3 good
reviews: 
Comments: Observing that existing datasets for learning image-matching algorithm lacks diversity, the authors propose to learn image matchers from diverse tourism videos available on the internet. To obtain (pseudo-)labels for training, the authors propose to aggregate predictions from multiple matchers trained on small datasets and use a label propagation algorithm to propagate labels beyond nearby frames. Training on the pseudo-labeled data, existing image matching techniques demonstrate strong generalizability to unseen image domains, significantly outperforming image matchers trained on traditional datasets. In addition, the authors demonstrated that the proposed self-training framework showed strong scaling behavior, promising stronger image matchers for future work.
Strengths: 1. Simple and scalable framework: The proposed self-training framework is simple and scalable. 
2. Strong zero-shot generalizability: Compared to image matchers trained on the traditional datasets, image matchers trained using self-training demonstrated stronger zero-shot generalizability, yielding more robust and performant image matchers. 
3. Comprehensive experiments: Experiments include large collections of datasets and downstream tasks, showing the superiority of self-trained image matchers.
Weaknesses: 1. Lacking real indoor datasets in the benchmark: This is a nitpick but it would be great to have more real indoor datasets in the benchmark. Right now, most of the real datasets are driving-related and the indoor dataset only covers basements and corridors.
Recommendation: 8: accept, good paper
Reviewer confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 3 good
reviews: 
Comments: This paper proposed a Generalizable Image Matcher. It generalizes across training sets, view points, it works with BEV and even point cloud.

They also proposed ZEB, the first zero-shot evaluation benchmark for image matching. The claim it mixes data from diverse domains,
by using it one can assess the cross-domain generalization performance.

Extensive experiments on state-of-the-art baselines are compared.
Strengths: The overview image in page.1 is impressive already. The method works on three strongest baseline (DKM, SuperGlue, and LoFTR) and improves them further more. It surprises me the method works with such huge view point differences and it also works with BEV pointcloud.

The training is using internet videos which prevents the COLMAP (SfM + MVS) bias for a single scene.

The proposed GIM is essentially a point matching ground-truth reinvention by using the label propagation through video with strong augmentation. However, it's so effective on every method by using the same training according to section 3.1.

I consider the simplicity not a weakness, but as a strength. If the proposed method is reproducable, I believe it would be the new standard of image matching.

The reconstruction results in Fig.3,4,5 are very impressive especially when you know it's training on sequence video instead of SfM alike scenario.
Weaknesses: This paper is very impressive, I think the only thing left is just some implementation details becuase the self-training part is very short and only about the ground-truth instead of the training itself. 

The only thing left is just open-sourcing the proposed code of the label propagation and training data to verify it's accuracy.
Recommendation: 10: strong accept, should be highlighted at the conference
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 4 excellent
Clarity: 4 excellent