Title: UFM: A Simple Path towards Unified Dense Correspondence with Flow
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image Matching

Optical Flow Estimation
Abstract: Dense image correspondence is central to many applications, such as visual odometry, 3D reconstruction, object association, and re-identification. Historically, dense correspondence has been tackled separately for wide-baseline scenarios and optical flow estimation, despite the common goal of matching content between two images. In this paper, we develop a Unified Flow \& Matching model (UFM), which is trained on unified data for pixels that are co-visible in both source and target images. UFM uses a simple, generic transformer architecture that directly regresses the $(u,v)$ flow. It is easier to train and more accurate for large flows compared to the typical coarse-to-fine cost volumes in prior work. UFM is 28\% more accurate than state-of-the-art flow methods (Unimatch), while also having 62\% less error and 6.7x faster than dense wide-baseline matchers (RoMa). UFM is the first to demonstrate that unified training can outperform specialized approaches across both domains. This result enables fast, general-purpose correspondence and opens new directions for multi-modal, long-range, and real-time correspondence tasks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a model and set of training datasets for the task of dense correspondences, unifying optical flow and dense feature matching.\nThe coarse part of the model is a two-view transformer built on top of DINOv2-L features, which is trained to output a corrrespondence field and a covisibility mask for each token position. If needed, these correspondences can be refined by a lightweight refiner on top.\nOn the data-side, the authors compile a set of datasets similar to that of MASt3R, extracting correspondence either from depthmaps or from optical flow.\nThe experiments show that the method, UFM, performs competivetely with specialized models, while being significantly simpler and faster.\n\nSTRENGTHS AND WEAKNESSES:\n**S1 - Simple model, strong results:** The method outperforms matching methods such as RoMa and optical flow methods such as FlowFormer. While technically not SotA on optical flow tasks, as later methods outperform FlowFormer, I believe the performance is satisfactory. I was also positively surprised to see that the authors made the effort to compare their approach to these previous methods when trained on the same data (see Figure 5), which validates their modeling choice (and also their 2nd contribution).\n\n**S2 - Unifying flow and wide-baseline matching:** For downstream tasks where these types of models are used (both in academia and industry), it is incredibly beneficial to have a model that is not overly sensitive to the specifics of the setup. While RoMa is robust to large temporal changes, it is not particularly accurate for thin and small objects. The proposed model takes a step towards a setting where one can simply input any image pair and expect to get correct correspondences in a timely manner, which I think is an important contribution.\n\n**W1 - Unclear robustness to temporal/etc variations in images:** As the authors mention on l.285-l.292, finetuning the DINOv2 backbone tends to reduce generalization to more semantic types of matching. I interpret this to mean that performance on more difficult feature matching benchmarks such as MegaDepth-1500, and WxBS, decrease due to this choice. Unfortunately the authors do not provide results on these benchmarks, particularly MD1500 would be relevant as it is the standard feature matching benchmark.\n\n**W2 - Unclear improvement in comparison to MASt3R:** Looking at Table 2 and Table 3, MASt3R generally outperforms the proposed method (although MASt3R can be improved by using the proposed refiner). While MASt3R is significantly slower, I would assume that most of the runtime comes from matching its features in a brute-force manner. As this matcher is very basic, I'm not entirely convinced that the claimed 60x speedup is representative.\n\nQUESTIONS:\n1. I do think I entirely understood the proposed loss. Is the only supervision for the flow the robust Charbonnier loss? In RoMa this was used on top of a classification type loss. Would only using this loss not cause the network to get very low gradients when matching fails? I would be curious to hear the authors explain the thought process a bit more.\n\n2. W.r.t. my comments regarding comparisons to MASt3R, I would be interested in hearing an argument from the authors why their full method is needed, when MASt3R already seems to perform well (and possibly outperforms the proposed method). For example, why not simply train a small regression head on top of the MASt3R decoder?\n\n3. W.r.t to MD1500, I am interested in hearing why the authors have not included this very standard benchmark.\n\nCurrently my rating is around a borderline, with my main concern being MASt3R comparisons. Given a convincing rebuttal to the questions above I would be willing to raise my rating to accept.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a unified model for wide-baseline feature matching and optical flow, which are usually studied separately in previous works. Thanks to the unified formulation, the proposed model can be trained on combined datasets from both tasks, where the performance is better than separately trained models. Technically, the proposed model is mainly a large Transformer-based architecture, with a flow head to regress pixel displacements and a covisibility head to predict covisibilities across two input views. Experiments are conducted on wide-baseline feature matching and optical flow datasets, the proposed method achieves comparable or better results than previous specialized methods, but with a unified model.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- The unified model is appealing since the two tasks of wide-baseline feature matching and optical flow indeed share a lot of similarities.\n- One benefit of the unified model is that it can be trained on the combined datasets. In Table 5, this paper shows that the combined-dataset training leads to mutual improvement on both tasks.\n- To train the unified models, this paper compiles 12 datasets, with detailed preprocessing steps thoroughly discussed. These efforts provide a valuable resource to the community.\n\n### Weaknesses\n\n- This paper argues about the difference with previous coarse-to-fine methods in several places (e.g., Line 9 and Line 30), but eventually the proposed model still has a coarse-to-fine structure (with an additional refinement stage). I think this paper would like to highlight the difference in the \"coarse\" stage (global vs. local), instead of the overall coarse-to-fine structure in previous methods. In addition, I think the argument on global attentions used in the model is not well-supported by experiments. In Figure 4, some comparisons with RoMa are shown, but there are many differences in implementation and training datasets between the proposed method and RoMa, which makes it not easy to understand where the performance gains come from. A more rigorous comparison could be replacing the global attentions used in the proposed model with local attentions and train with the same setup and then compare their performance. Such an experiment would be more convincing.\n- This paper uses a regression head to predict flow. However, many previous methods have shown the benefits of performing matching (for example, MASt3R, UniMatch, GMFlow, RoMa, etc.). It's unclear why such a design is adopted in this paper, and it would be better to justify the rationality and benefits of using a regression head instead of a matching head. \n- Although the results on the wide-baseline feature matching task look good, the performance on optical flow seems to have room for further improvement (in particular, Table 4, Sintel Clean). What are the potential reasons? A deep analysis would be beneficial. In addition, how does the model predict flow for all pixels since the models is only trained on covisible pixels? In addition, I am also curious about the performance of the proposed method on the closed test sets (for example, Sintel and KITTI online benchmarks). It would be great if the authors could report these results.\n\nQUESTIONS:\nThe key questions are listed in the Weaknesses.\n\nIn addition, I think another related work that is worth mentioning and discussing is: [Perceiver IO: A General Architecture for Structured Inputs & Outputs. ICLR 2022], which also tried to develop a unified model for different tasks and also reported the experiments on optical flow. \n\nIn general, I like the idea of this paper. However, I am currently not fully convinced by the experiments (detailed in Weaknesses). I am happy to raise my rating if these questions are well-addressed.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSimilar to UniMatch and MASt3R, this paper builds a dense correspondence network based on Dinov2 and Transformer. Additionally, it introduces a local attention-based refinement module to further enhance the results. Overall, the network architecture is reasonable, but it lacks substantial innovation. However, I believe the more significant contribution of this paper lies in the extensive training over a wide range of datasets, which provides a potentially reliable matching baseline for many downstream tasks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. Once the code is open-sourced, it will provide an excellent matching tool for many downstream tasks, which is beneficial to the community.\n\n2. The overall network structure and the proposed refinement module are reasonable.\n\n3. Extensive experiments and thorough visualizations are provided, and the results appear trustworthy.\n\n**Weaknesses**\n1. The method overlooks the learning of occluded regions. Inferring occlusions is a crucial aspect in optical flow tasks, especially for downstream tasks such as sparse-view reconstruction. This omission reduces the significance of the work (as its functionality overlaps with MASt3R in most practical scenarios).\n\n2. The architecture lacks substantial innovation. Technically, the method appears to be a combination of UniMatch and MASt3R.\n\n3. The paper’s claim of unifying dense correspondence and flow is not well justified. At most, it improves long-range and short-range matching performance in dense correspondence tasks. Optical flow generally includes occlusion estimation, which is not addressed here.\n\nQUESTIONS:\n1. Please elaborate on the algorithm’s performance in occluded regions, and explain why occlusion estimation is not considered.\n\n2. Please describe your open-sourcing plan. Since the main contribution of this work lies in its applicability to downstream tasks, a well-executed open-source release would be a major positive factor.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes utilizing both optical flow and large baseline correspondence datasets for mutual benefits on two tasks.\nThe authors present a simple but effective model architecture which consists of visual encoder, 12-layer transformer (with the positional encoded features of the two encoders as input) and two heads for flow and co-visibility (with encoder output and 6/9/12 layer outputs as input). The output of regression flow head is further fine-tuned via a weighted summation of neighbor coordinates based on feature similarity between reference and target frames.\nIn training, only co-visible pixels are utilized. In order to identify co-visible pixels, according to the supplementary, depth reprojection error is basically used and, additionally, information on depth change (in dynamic scenes) and rigidity (in rigid motion) is also utilized.\nBy applying the proposed geometric sampler, the authors curated the existing datasets to obtain a new selected set (TA-WB) of data to evaluate dense wide-baseline correspondence solutions.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n\n1) Simple but effective method \nMulti-task benefits are well known. But, the authors demonstrated how to realize the benefits in optical flow and large baseline correspondence with an effective method.\n\n2) Experiments are well done.\nEspecially, probing experiments well demonstrate the authors' choice in model architecture.\n\nWeakness\n\n1) Lack of detailed analysis on the origin of multi-task learning effects\nTable 5 well demonstrated the mutual benefits.\nIt would be nice to further investigate the origin of such benefits.\n\n2) Evaluation of diverse scenes\nAlthough the authors proposed a new dataset for wide-baseline correspondence, \nit would be nice if more diverse scenes like low light, fast moving objects, etc., are evaluated.\n\nQUESTIONS:\n1) How can we further investigate the benefits of multi-task learning in this paper?\n\n2) How does the proposed method work in extreme conditions such as low light scenes?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-02"}