UFM: A Simple Path towards Unified Dense Correspondence with Flow
Authors: Yuchen Zhang
Nikhil Varma Keetha
Chenwei Lyu
Bhuvan Jhamb
Yutian Chen
Yuheng Qiu
Jay Karhade
Shreyas Jha
Yaoyu Hu
Deva Ramanan
Sebastian Scherer
Wenshan Wang
Venue: NeurIPS 2025 poster ()
Keywords: Image Matching
Optical Flow Estimation
Primary area: applications

Abstract: Dense image correspondence is central to many applications, such as visual odometry, 3D reconstruction, object association, and re-identification. Historically, dense correspondence has been tackled separately for wide-baseline scenarios and optical flow estimation, despite the common goal of matching content between two images. In this paper, we develop a Unified Flow \& Matching model (UFM), which is trained on unified data for pixels that are co-visible in both source and target images. UFM uses a simple, generic transformer architecture that directly regresses the $(u,v)$ flow. It is easier to train and more accurate for large flows compared to the typical coarse-to-fine cost volumes in prior work. UFM is 28\% more accurate than state-of-the-art flow methods (Unimatch), while also having 62\% less error and 6.7x faster than dense wide-baseline matchers (RoMa). UFM is the first to demonstrate that unified training can outperform specialized approaches across both domains. This result enables fast, general-purpose correspondence and opens new directions for multi-modal, long-range, and real-time correspondence tasks.

UFM: A Simple Path towards Unified Dense Correspondence with Flow
uniffommach.gihuubio
Yuchen Zhang
Nikhil Keetha
Chenwei Lyu
Bhuvan Jhamb
Yutian Chen
Yuheng Qiu
Jay Karhade
Shreyas Jha
Yaoyu Hu
Deva Ramanan
Sebastian Scherer
Wenshan Wang
Carnegie Mellon University
Nefomarc infenma and
19 wile
Row-Sreer
UnMMnnch
Oprtc Tice
UPM (ours)
Inforsee Timee Jra A6000
Figure 1: UFM (Unified Flow & Matching) unifies dense pixel correspondence tasks such optical flow and vide-baseline matching. We visualize sets x grids, where the top 2 images are the input, and the bottom 2 are images warped with forward & backward flow. UFM is able match across a wide range of baselines, including extreme ones with little co-visible overlap.
Abstract
Dense image correspondence central to many applications, such as visual odome- try, 3D reconstruction, object association, and re-identification. Historically, dense correspondence has been tackled separately for wide-baseline scenarios and optical flow estimation, despite the common goal of matching content between two images. this paper, we develop Unified Flow & Matching model (UFM), which trained on unified data for pixels that co-visible both source and target images. UFM uses a simple, generic transformer architecture that directly regresses the (u,v) flow. It is easier train and more accurate for large flows compared to the typical coarse-to-fe than sfate-of-the-art flow methods (Unimatch), while also having 62% less error and 6.7x faster than dense wide-baselinn matchers (RoMa). UFM is the first to demonstrate that unified training outperform specialized approaches across both domains. This result enables fast, general-purpose correspondence and opens directions for multi-modal, long-range, and real-time correspondence tasks.
cost volumes in prior work. UFM is 28% more accurate
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Dense correspondence estimation, which determines where each pixel in one image appears in another, a core task in computer vision with wide-ranging applications, including visual odometry [45, 47, 58). 3D reconstruction [13, 34, 53], object association [36], place recognition [30, 32, 49], and image warping [71]. Despite its importance, existing methods are typically developed for two separate domains: optical flow, which addresses small displacements between temporally adjacent frames, and wide-baseline matching, which handles large viewpoint or scene changes. This division has led task-specific models that perform well in one domain but fail to generalize the other. As result, these models often break down in real-world scenarios where both small and large motion may co-occur, highlighting the need for unified approaches that bridge this gap.
Existing dense correspondence estimations algorithms have been separated into different tasks. For example, optical flow typically assumes small baselines between the two images, but allows for dynamic scene, and so often relies on motion priors for temporal consistency. In contrast, wide- baseline matching assumes static scene but allows for significant changes in viewpoint [37] and time [59], and often require invariant geometric and semantic cues [72]. Despite these differences, both tasks fundamentally aim to establish correspondences between images. This shared objective suggests that they are not inherently separate problems, but rather variations the same challenge that can be approached within unified framework.
We are inspired by prior attempts at unifying such correspondence tasks [60, 78], but thus far, none provide a generic solution that outperforms or is par with specialized solutions. Our experiments suggest that existing work in optical flow and dense wide-baseline matching suffers from biased architectures that are either inefficient when learning from large data or do not have their output format trained/designed for dense, high-resolutton output. We aim to answer the question can develop unified model that benefits from shared training both optical flow and wide- baseline matching data? Specifically, what architecture, data, loss, and training scheme do we need to unify flow& & matching?
In this work, we scaled a transformer- based regression model over a comprehensive training set of matching. We sample image pairs from
12 datasets spannning both optical flow and our dataset based on covisible content and train exclusively on these regions. We designed a custom geometric sampler with explicit control over viewpoint differences and a filtering pipeline to ensure co-visibility. By restricting supervision to co-visible regions, we discourage the network from relying on global 3D structure alone and encourage correspondence estimation grounded in visual evidence. We found that this simple approach leads to a generalizable and efficient model for both optical flow and wide-baseline matching that surpasses most SoTA on its own, achieving further gains with standard refinement techniques.
Finally, to spur further research on correspondence in challenging wide-baseline scenarios, we build a novel dataset for evaluation by holding out environments from the TartanAir- Visual Odometry benchmark [68]. using our custom geometric sampler to curate challenging image pairs. Our TartanAir Wide Baseline (TA-WB) benchmark challenging and well-controleed dataset for evaluating dense wide-baseline correspondence.
In summary, our contributions are:
1. For the first time, we demonstrate that unifying the training of both optical flow and wide- baseline estimation can benefits both domains. Our Unified Flow & Matching model (UFM) achieves state-of-the-art performance on benchmarks from both tasks.
2. We find that generic transformer architecture models unified data better. The simplicity and efficiency of our architecture allows adding existing refinement techniques for further improvement.
3. We introduce new benchmark, TartanAir Wide-baseline (TA-WB), which evaluates dense correspondence at challenging viewpoint changes.
2 Related Works
Optical Flow Methods Optical flow prediction aims to establish dense, pixel-wise motion vectors between temporally adjacent frames. Except for early exploration of optimization-bn formulations, current methods are mostly learning-basee based. Most work has evolved around specialized architectures

---

including cost volumes [20, 23, 54, 55, 57], coarse-to-fine paradigms [4. 5, 9, 19, 55, 75], and recurrent structures [20-23, 55, 57, 75J. RAFT [57] is one of the most representative works along these ideas. It employs a multi-resolution cost volume between all pairs of patches and a recurrent structure to update the flow prediction iteratively. It has many derivative works [20, 54, 69, 79]. SEA-RAFT [69] is the current state-of-the-art (SoTA) that simplifies RAFT with regressed initial hypothesis and a multi-modal training objective. Other approaches tried to move beyond these paradigms. FlowFormer [20] uses the transformer architecture to aggregate the cost volume into compact latent tokens for efficient processing. GMFlow [73] casts optical flow into global matching problem [73, 74, 79] and replaced the costly iterative refinement with global correlation layer.
In developing a foundation model for generic correspondence prediction, we observed that the specialized architectures of classical optical flow methods struggle with diverse, wide-baseline data, even when trained on it. In contrast, we show that generic transformer- based regression architecture with sufficient data serves as robust and generalizable prior. Moreover, it can be effectively combined with these refinement techniques to improve performance further.
Dense Wide Baseline Methods Dense wide-baseline matchers suppress their sparse counterparts since DKM [14], which first obtains a robust, coarse match from patch features and uses regressive warp-refiners to upsample the prediction resolution. RoMa [15] builds upon DKM by using a frozen image foundation model (DINOv2 [46]) for its coarse matching and uses separate convolution-base encoders to provide fine details to warp-refiners. Despite being robust and accurate, both methods have a heavy architecture that limits their application to compute-limited scenarios. We show that our method can achieve similar robustness and accuracy while being about 6x faster.
These methods [14, 15, 41, 61, 62] also include a covisibility mask estimator (some call it "certainty" or "matchability") that helps to exclude matches in occluded or out-of-view regions. This mask is usually directly trained with the ground truth target. We extended this paradigm by computing co-visibility masks for dynamic datasets.
Unifying Correspondence Several work exists in treating correspondence as unified task. GLUNet [60] is the first work showing that geometric, optical flow, and semantic correspondence tasks can be solved by unified network. RGM [78] is the most recent work that scaled RAFT-like architecture on comprehensive dataset and obtained SoTA zero-shot performance. However, they failed to show that the generalist model, trained on all data, outperforms the specialized model, trained on in-domain data only. Alternative to modeling correspondence densely, COTR [27] took a formulation that predicts one pixel location over each query point, and tested on both optical flow and pose estimation tasks. This formulation is prohibitively expensive for dense flow, and while sparse matches can be interpolated, the resulting performance degrades significantly. In contrast, our work trained a transformer- architecture that directly regresses dense optical flow and shows mutual benefit between optical flow and wide baseline data.
Scaling Correspondence Recent works have also tried to expand the training dataset for correspon- dence. Besides the standard optical flow datasets [6, 12, 33, 39, 42, 43], we see a trend in using static wide-baseline matching datasets to pretrain optical flow networks. For example, MatchFlow [11] pretrained GIM [52], an auto-annotation pipeline that extracts matches from distant frames in real-world videos. Similarly, SEA-RAFT [69] pretrains on Tirtan [68] and observed improved generalization. Existing work in wide-baseline matching [18, 28, 64] has also expanded the dataset towards more modalities such as satellite, IR, depth, event, and medical. Although they have shown successful matching between challenging modalities, they do not show that scaling with additional data helps improve the original RGB-RGB matching.
Recent advancements in end-to-end learning have also encouraged scaling a generic architecture for 3D reconstruction and correspondence [26, 29, 31, 45, 65, 67]. Perceiver 10 [24] shows its architec- ture can solve diverse vision tasks, including optical flow. CroCoV2 [70] also shows that optical flow can be directly regressed from its backbone pre-trained on the pross-magee-comllee C2ocoV2 stopped at low resolution and both methods required a sliding window method to infer at high resolution, which failed to capture correspondences across windows. Furthermore, CroCov2 doesn't 't train the two-view transformer from scratch to directly regress flow. More recent follow-up MASt3R [34] finetuned DUSt3R [67] to output pixel-wise feature descriptors and proposed a fast reciprocal matching decode sparse matches efficiently. However, this paradigm does not provide dense matches and is prohibitively slow without subsampling.
task. However,
3

---

Transformer Backbone
Regression Heads
Pixel Displacement
petth Toleng
procesoed
BINDN
lobams
Flow Hood
shared weights
Self Attention
Blocks
Covisibility Flow ourputt only roviuble region
patth
BIND244
Covelilitty
Shared Image
Global Attention
Output
Input Images
Encoders
Transformer
Heads
Two images are encoded by a shared DINOv2 encoder into patch features, concatenated, and then processed by 12 self-attention transformer layers. Intermediate tokens are decoded by separate DPT heads regress pixel displacement and covisibility maps. representing correspondence and visibility across views.
Figure 2:
The UFM Architecture:
Unified Flow & Matching Model
3.1. UFM Architecture. Given two images / E R 3 as input, our Unified Flow and Match- ing (UFM) model (Fig. 2) predicts the visually grounded dense correspondences and covisibility:
{+. = fUFM(41.5)
(1)
where Φ1 22xH* is forward pixel displacement map (flow) which maps each [u, position in J to continuous position in l and Ci E IxxHW is a binary mask, where each value indicates if the [n, v] position / is visible /2-
In particular, UFM employs a simple end-to-end transformer with CNN head to directly regress the flow value, unlike prior methods [15, 69, 74] that first construct an initial solution the patch level and then progressively upsample to the pixel level. We empirically find the paradigm used in prior methods yield suboptimal accuracy, as shown in Appendix F. Beyond accuracy, direct flow regression offers additional advantages: (1) Performing attention at the patch level is fast for flow prediction, while the DPT [48] enables predictions directly at the pixel level. (2) Structural simplicity enables easy optimization and potential for additional simple pixel-level refinements without huge impact on efficiency. We elaborate on the end-to-end transformer further below.
Feature Encoding: Amongst various image encoders, we find DINOv2 VIT-L [46] be the most optimal. DINOv2 takes as input images and predicts patch tokens Fg R |024xH/14xxW// Given the two sets of patch tokens, we fuse them with view index positional encoding unique to each view and then apply 12 successive layers of self-attention. While other prior methods [34, 67, 70] employ cross- atteetattnnionn blocks, which in theory have the same compute requirement as our design, we find that the self-attention transformer is better accelerated by Flash-Attentin [8] due to its longer sequence length. This leads to better training and inference efficiency. Also, we empirically find that both types of transformers have similar performance in terms of flow regression.
Predicting Flow & Covisibility: After the self-attention transformer is applied, we employ two separate DPT heads which take as input the encoded patch tokens from I and respectively predict the flow & and logits for the covisibility mask cloyis We empirically find that employing single DPT head for both flow and covisibility prediction leads to degraded performance. The DPT inputs the output features from the DINOv2 image encoder and the self-attention transformer's 6th, 9th, and 12th layer features. The final predicted covisibility is obtained by Ci sigmoid (Clogix).
Refinement by Classification: While we find that the regression of dense correspondence (flow) is robust, it is not always precise (e.g., see average EPE & outlier numbers for UFM s60 in Table 2). Hence, we designed simple classification-bal of UFM's inlier predictions. We take inspiration from MAS13R [34]'s design to regress pixel-wise matching features based on transformer backbone features. Additionally, to capture fine details for the refinement, added a U-Net encoder following RoMa [15]. As shown in Fig. 3, differ from
local refinement technique to improve the accuracy

---

Computing Refinement Feature Maps
Refine Flow for each Pixel
Translor Bechbone
Regressed
Rogressio cssionson Flow Haad
Source Image
Net
?
Source Feature
Interpolated Neighborhood Features
Sampled Courdinates
MIP
F
Gn
Image
Target
?
Weights
Source Imasge
(pleal-arsa)
Features
Torget Image
Feeturees (plaet wibse]
residual
sofumax( FF Fa b)C.
Figure 3: Refinement of Correspondence by Classification: We compute per-pixel feature map by combining (1) globally aligned features from the UFM backbone and (2) local fine features encoded by separate U-Net. For each pixel in the source image, first the regression flow target to interpolate features around a local neighborhood. We then compute the attention between the source features and the features from the local neighborhood, and use it weight-add the coordinates refinement value. is a constant attention bias.
MASt3R [34] in how we leverage the refinement features for correspondence: as opposed to matching dense features across the entire image (global search), we use the regressed flow from UFM's DPT to guide the feature matching around a small 7 x neighborhood, thereby leading to 60x efficiency over MASt3R (Table 2), or about 24 considering the follow-up acceleration Speedy MASt3R [35]. In particular, we compute the attention between each pixel and its local 7 7 neighborhood determined by the regressed flow and use the weighted sum of coordinates by the softmax attention as the residual to update the initially regressed flow.
3.2. Training Objective. To train UFM, we supervise the predicted pixel displacement map @1 and the covisibility mask Cj. Importantly, supervision of the correspondence is restricted to covisible pixels. This design encourages the model to ground correspondence in visual evidence, rather than inferring 3D geometry from a single view and extrapolating into occluded or out-of-view regions.
We trained with a robust regression loss [1], following the approach in RoMa [15], which focuses its gradient on inlier predictions with small errors eerors-tryppi typicallyy around to 2 pixels. We selected this loss two main reasons. First, it encourages precise learning from reliable matches by emphasizing small residuals. Second, it reduces the impact of incorrect data during training, as robust losses exhibit vanishing gradients for large flow errors, which are commonly caused by unmatchable pairs. Specifically, we used the generalized Charbonnier loss with parameters a 0.5 and 0.24. See Appendix for more visualization and discussion.
LEPE(O1, (1) Lier Clijsr TE!
cf 4ohuss of (2)
As supervise the network only covisible pixels, the network can have arbitrary output in the non-covisble pixels during usage. Hence, we also predict a covisible mask to exclude outputs from these regions during usage. To train this mask, we used the standard binary cross-entropy loss.
LBCE
Σ
-1q (og(c(osit) - ) clogits) te1
H W
We find that upweighting the covisibility loss by a factor is optimal for the prediction of good covisibility and doesn't ' impact flow estimation quality. Thus, our final loss is L = LEPE + 10 x LBCB-
3.3. Combining Flow and Matching Datasets. We compiled a unified dataset consisting of 12 datasets spanning diverse sources, motion patterns, and environments from both wide-baselin matching and optical flow domains, as detailed in Tab. 1. The collection of these datasets features diverse indoor, outdoor, in-the-wild, and dynamic scenes.
Fach dataset was carefully vetted for depth consistency and geometric correctness, as not all are suitable for precise training and evaluation. For example, found that ARKitScenes [2] contains
5

---

Table 1: Diverse suite of dense correspondence datasets used to train UFM.
Imuges
Dynamic
Wide Baseline
Frame Frame
Pairs Epoch
BlendedMVS [76]
115k
1.15 M
503
Mesh Recomstruction
100
MegaDepth [37]
38.8
M
275
COLMAP MVS
x
100
TanaaAArV22 [68]
1.37 M
688 k
55
Synthetic
X
100
Scannete+ V2 [?7]
265
M
295
Laser Scan
100
Habitat CAD [56]
201 k
175k
91
CAD Reconstrction
X
25 k
StuticThings (51)
22.4k
337k
2250
Synthetic
10%
Kubric4d [17. 63)
2.4M
9M
2800
Syuthetic
50 k
FlyingThings [39]
22.4
20.2
2239
Synthetic
50 k
ClyingChairs [12]
44.4
22.2k
Synthetic
Spring [40]
10
9.9
Synthetic
25 k
Monkaa [39]
8.6
8.6
24
Syuthetic
5k
HDIK [42)
1081
1046
Real
inconsistent depth estimates, leading to flow errors of up to 5 pixels, which is unacceptable in the matching domain where methods aim for sub-pixel accuracy.
In general, we paired the data for a well-distributed range of covisibility and optical center difference. For most of the static wide-baseline datasets, we followed the pairing scheme in DUSI3R [67] and CUT3R [66] and used adjacent frames for optical flow datasets. We selected the ratio from each dataset largely based on the number and quality of the scenes. We further provide details on sampling pairs for ScanNet++ V2 [77] & Kubric4D [17] in the supplementary. Notably, we mined new pairs from Kubric4D across both time and viewpoint, making it the only dataset in our collection that is both dynamic and side-baseline.
Because we aim to develop a unified model that generalizes concurrently to both optical flow and wide-baseling matching domains, we train both types of data simultaneously. This allows examples from both domains to appear within a single gradient update, promoting cross-domain generalization. We computed covisibility and correspondence for all pairs of images support this unified training.
We compute correspondence targets and covisibility mask differently depending on the dataset type, accounting for the specific characteristics of posed image collections and optical flow labels in static scene data and synthetic data such as Kubric. This process is detailed in the supplementary.
TA-WB Training& & Benchmarking Dataset: We developed a special geometric sampler for the TartanAirV2 [68] dataset to sample geometrically challenging yet covisible pairs (further described and samples provided in the supplementary). Since TartanAirV2 provides images covering all six sides around each camera center, all visual information is preserved, and we can resample virtual cameras with arbitrary orientations. Our sampler utilizes this freedom to control the viewpoint difference explicitly. We check all sampled pairs for matchability and reject occluded textureless pairs (for e.g., two cameras facing white walls). We made the final samples to equally distribute the camera optical center angle difference between 0 and 120".
3.4. Training Details. We train the network with a longest side resolution of 560 (with aspect ratios varying from 3:1 1:1) for 48 epochs with data as specified Table All our datasets permit academic research, and the publicly released UFM model weights will be licensed following this. We use peak learning rate of 1- 10 4 for the global attention transformer and DPT heads and 5. 10-6 for the encoder to preserve DINOv2 pre-training. This contrasts with the frozen DINOv2 used by RoMA [15] (which we find suboptimal), and we provide further insights the supplementary. We use AdamW optimizer with cosine decay learning rate schedule using 10% linear warmup, 0.05 weight decay, and B = (0.9.0.95), Since most of our data has bidirectional correspondence, we symmetrize the batches. This leads unique. The training takes 4 days on 8 H100 GPUs We name this checkpoint as UFM 560.
effective batch size of 96 pairs, where half of them
Some downstream tasks, like visual odometry [47], require sub-pixel accuracy, making high- resolution images essential. However, training at high resolution computationally expensive. To address this, we bootstrap risl-tesolution model, UFM 980, from UFM 560. The wide- baseline datasets do not have depth annotations at a high resolution (1K), and upsampling the pre-computed flow at lower resolutions would be sub-optimal for sub-pixel training. Hence, we train with 10x lower learning rates than the 560 training on all optical flow data for 15 epochs. Furthermore, we change the supervision range to all pixels to follow the standard evaluation protocol in optical flow.

---

Table 2: Wide Baseline Dense Correspondence: Zero-shot dense correspondence evaluation all covisible pixels. We report the AEPE and outlier rates at thresholds of 1, 2, and 5 pixels. UFM outperforms all dense methods by large margin and matches MASt3R's performance, despite MASt3R's advantage in selecting its confident pixels, while being 60x faster (24x considering follow up [35]).
Method
Eval Range
ETHED
DTU
TA-WB
Runtime
BPS
pxx.
EPE
1ps_L
3.
pxx.
EPE
ppx_
5
SEA-RAFT
113.13
72.4
60.4
172.
13.6
Fommer
74.83
MatMa
Covisible
73.1
48.98
57.0
57.2
RaMa
Pinels
7.94
$1.1
33.4
19.9
9.59
52,1
33.8
19.9
48.10
63.7
47.7
39.8
367.4
UFM wa
2.64
46.5
23.9
8.7
5.56
58.4
33.6
13.2
12.87
53.5
31.8
17.0
42.9
UTM 500
2,60
44.3
22.8
8.7
5.55
55.5
32.9
13.8
12.84
51.4
30,6
17.0
70.1
MASER
UFM wa
UTM
MASCR's
Ourput
1.29
1.34
1,51
29.0
33.4
31.7
11.6
12.1
3.1
2,0
2.18
2.25
2.50
50.1
49.3
42.6
20.6
23.5
20.8
5.3
6.3
6.13
6.19
6.21
38,7
54.8
42.1
22.5
19.5
17.8
6.3
74
2517.8
41.0
56.1
Table Relative Pose Estimation: Area Under the Curve results for pose estimation on zero-shot datasets (ETH3D, Scannet 1500) and our proposed benchmark TA-WB (zero-shot scene assets, appearance & geometry). Gray text indicates results where the evaluation dataset is in the training set.
ETH3D
Scannet-1500
TA-WB
Method
AUC @
10"
@ 15'
AUC 5
@ 10 15
AUC 10"
@ 20" 1
30"
RoMa
63.7
74.2
78.6
29.2
50.0
60.9
2.2
11.4
23.2
MASGR
77.0
81.5
34.2
57.2
13.3
27.9
UFM
61.6
74.1
79.3
30.7
53,5
64.8
2.3
13.3
28.6
UFM 560- nefine
66.7
77.1
81.6
31.6
54.1
65.3
13.5
28.6
4 Benchmarking Unified Dense Correspondence
4.1. Zero-Shot Wide-Baseline Correspondence. We perform direct evaluation via dense corre- spondences and indirect evaluation via pose estimation. We compare all covisible correspondence to the ground truth and report Average Lad-Point-Frree (AEPE) and outlier rates. We use exhaustively sampled covisible pairs from ETH3D [50]. DTU [25]] and TA-WB. For pose estimation, we evaluated on ETH3D, TA- WB, and Scannet- 1500 [7]. While pose estimation benchmarking the standard practice, we believe that dense wide-baseline EPE provides more direct and stable measure of matching quality by eliminating the influence of confidence prediction and sampling.
Baselines We benchmark against SOTA, including RoMa [15] (indoor, for better performance) and MASt3R [34]. MAS13R is sparse method that only provides correspondence passing its cycle-consistency check. We adjusted subsampling to get the most dense output and evaluated UFM on the same set of reported pixels. While this setup favors MASt3R by restricting evaluation to it's confident matches, enables comparison with one of the most robust sparse matches. We include optical flow methods for completeness, comparing against SEA-RAFT [69], FlowFormer [20], and GMFlow [73], using their checkpoints trained on all available data.
Dense Wide Baseline Results Table 2, despite giving MASt3R an advantage, we showcase that UFM significantly outperforms all dense methods in precision, while achieving nearly 60x lower runtime (24x considering follow up [35]) than MASt3R - the only method with comparable precision. Furthermore, UFM significantly outperforms all dense methods, achieving average 62% less EPE and 6.7 x better runtime compared to the best dense baseline, RoMa.
We follow DKM [14] and evaluate UFM for pose estimation. As shown in Table 3, UFM achieves the best accuracy on ETH3D and WB benchmark, and second place on Scannet-1500 (despite not being trained on this dataset). This performance shows that UFM's correspondence is well-balanced and suitable for 3D geometric tasks. Moreover, we observe notable improvement by adding refinement on top UFM, highlighting the potential for integrating other refinement techniques on top of the base model for further improvement.
Pose Estimation Results
4.2. Optical Flow Correspondence. We evaluate zero-shot optical flow performance on Sintel and KITTI-2015 training set. We evaluate on both covisible pixels and all pixels which is the standard
7

---

Table 4: Optical Flow Estimation: Zero-shot evaluation across covisible ([covis]) and all pixels ([all)) on the Sintel and KITTI training sets. Each method is inferred at different resolutions, and the metrics computed at the dataset's original resolution (1K) and on A6000 Ada GPU.
Method
Inference Resolutive
Sintel Clean
Sintel Final
KITTI
Runtinse
[PE 1
EPC [ail]
Ipx
1px
5px
EFC: [covis]
EPE [all]
ppr
3pt
Spx
I'l EFC [oovis]
Fi EPC [oil]
F1
[conts]
SEA RAFT
0.49
1.27
7.4
3.4
2,5
2.28
3.86
13.1
7.7
5.1
2.10
4.29
14.3
20.7
Flrwerrmrr
0:47
1.01
8.7
3.6
2.5
1.43
2.38
14.0
744
5.5
3.25
6.03
15.8
155.
Unimatch
IK
0.43
0.96
74
3,4
2.4
1.63
2.70
13.4
74
5.6
2.38
4.92
17.5
76,7
UFM se
0.61
1:16
11.7
4.5
3.0
1:28
2.04
14.9
7.1
5.1
2,05
2.94
11.0
122.9
UTM vn relin
0.56
1
10.2
4.6
3.5
1.25
2,01
15.0
7.2
5.1
2,05
1.95
11.0
213.9
SEA RAFT
0.65
1.47
3.2
2.24
3.69
8.5
5.6
2.36
4.21
15.5
14.7
FlowFremeer
1.88
2:92
23.6
10.1
7.2
7.30
5.92
35.1
21.5
17.5
4.64
7.89
29.3
77.5
UFM seo
RaMa
IFM wi-refi
Unimutch
560
0.72
0.60
0.79
1.18
1.30
Traincdoon covisible pixcla only
10.3
2.9
2.13
1.44
1.73
1,40
2.76
Trainal on coN covisille pincls only
16.0
8.0
5.0
2.30
1.87
2,43
1.69
covisible piach only
4.66
Trainal on
17.7
300.3
64.0
50.0
$7.0
UFM
UFM
RoME
Image
Target Image
Warped Image UFM (ours)
Warped Image RoMe
Flow Details
Figure 4: UFM on Ego-Exo 4D [16]: UFM succeeds in matching out-of -ii-oo--ittiitttinn environments, camera models, and challenging viewpoint shifts, showcasing its strong generalization.
protocol that includes occluded and out-of-bound pixels. On Sintel, we report the AEPE for both cases and the ratio of pixels with EPE above 3, and 5 pixels for all pixels.
Baselines We compare approach to all optical flow methods in Section 4.1 and RoMa, using the checkpoint trained on FlyingChairs, PlyingThings and TartanAir (SEA-RAFT only) i.e. the best trained model before violating the zero-shot setting.
Results Table 4 shows that UFM, without any refinement, achieves state-of-the-an zero-shot performance on Sintel-Final and KITTI in terms of both EPE and most pixel outlier metrics, while also delivering competitive performance Sintel-Clean. These results demonstrate that the UFM base model has strong generalization and precision to be combined with existing refinement techniques.
We ran UFM on images from the Ego-Exo4D [16], which features videos captured in first and third person view across diverse scenes. As shown in Fig. 4, compared to RoMA, UFM achieves strong generalization & robust matching.
4.3. Generalizable Matching on Egu-Exo 4D.
4.4. Insights towards Unified Correspondence.
Data:
We conduct an ablation study to see if UFM benefits from unified training on merged data as opposed to training on specialized data only. Specifically, we train UFM only on optical, wide baseline, and the combination for 100 + 20 epochs at 224 & 560 resolutions. Across the different variants, UFM sees each data point the same number of times. Although the total number of gradient steps differs, the number of epochs large enough for the training have effectively converged. We then evaluate optical flow and dense wide-baseline performance as in the previous sections.

---

Table 5: Unified optical flow (OF) and wide-baseline (WB) training leads mutual improvement.
Optical Flow Tasks
Baseline Tasks
Pretrain Data
Sintel-C EPE
Sintel-F EPE
KITTI
DTU
ETH3D
TA-WB
EPE
1px
2px
1px
2px
1px
2px
OF
1.27
1.81
15.57
91.4
80.4
96.4
91.6
96.4
94.9
2.24
70.5
42.8
54.5
28.4
61.5
35.3
OF WB
1.02
1.48
2.62
69.0
41.4
52.4
27.0
59.2
34.2
(4)
UFM
RoMa
MEARRFT
Bound
Compunt
22.5 992. M. STEM
UnmMarch (CMffoow
Canorr Dvundd
UFM (our))
3
Compunt
RAFT
UniMatch (SMFFoW
12
8
10
101 lannd
SENSAIT
Phengghhinga
BendsdMes
NegeDeath
Scannat--
TartanAk
Figure 5: Architecture Ablation: Validation EPE for various architectures trained on the same 224 x 224 resolution data UFM. We report performance on different val sets Data Bound (22.5 M pairs) or Compute Bound (at 32 hours on H100 GPU) (a) Validation Set Performance: When trained on more difficult data (such as Tirt UFM significantly outperforms alternatives for both bounded data and compute. (b) Training Speed Comparison: We plot the number of pairs seen during training function of compute, and label the number of pairs that each architecture can train on at compute bound. UFM is far more efficient than most methods (except SEA-RAFT).
In Table 5, UFM outperforms it's own specialized variants, thereby indicating a mutual improvement when merging the two data types. For optical flow, we observe that adding iide-baseline data brings a 20% 80% decrease in EPE, especially on the KITTI dataset. For vide-baseline, we observe that adding optical flow data brings a 3.2% relative decrease in 1, 2 pixel outlier rates.
Architecture: To test the scalability of existing architectures, we trained SEA-RAFT, UniMatch, RoMa, and UFM on the same unified data (Table 1). Each architecture trained with its original loss functions as specified in the respective papers. We recorded the validation set EPE at data bound (35 epochs, 22.5 pairs) and compute bound (32 hours on 8 H100 GPU) to measure scalability. Fig. 5 shows that UFM performs best on all datasets at both data and compute bound. This indicates the benefits of using a simple architecture scale on large amounts of data, where UFM shows significantly increasing performance with compute harder datasets like ScanNet++ and TartanAir. We believe UFM's effectiveness scaling to the unified dataset is crucial for achieving mutual benefit across opticalflow and wide-baseline matching.
5 Limitations
While UFM represents exciting development in constructing models for unified dense image correspondence, some limitations remain with semantic matching capabilities. As shown in Fig. 6, on WxBS [44], find that UFM works on challenging image pairs that demonstrate scale, viewpoint, texture, and illumination and tends to struggle with extreme seasonal changes and matching across spectrums, i.c. visual to very dark infrared (thermal). RoMA [15] is robust such semantic changes due to the coarse patch correlation provided by frozen DINOv2 [46] features, with the help of additional fine features from ConvNet in its upsampling process. We find that freezing the encoder does not benefit an end-to-end transformer architecture such as UFM. As shown in Appendix E, we find that freezing the pre-trained DINOv2 image encoder leads to a significant drop in dense correspondence performance. Opportunities may lie in complementing frozen DINOv2 features from other sources. Furthermore, although we constrain the learning rate to remain relatively small to preserve DINOv2's pretraining, we find that DINOv2 can still deviate significantly during extensive training and lose some of semantic matching abilities. Through preliminary exploration, we find that this can be mitigated by incorporating semantic matching data, semantic preservation losses, or

---

(a)
(b)
(c)
WAB5 Evaluation Besults
on C1 Compuna pAIE
(d-f)
Thresheedds (ps)
Success Cases
(g.h)
Failure Cases
Pred: No Covisibility
Pred: No Covisibility
[Extreme Scale Season Change)
[Extreme hhermal- Pair)
(Extreme Scale illuminatio Changel
Figure 6: WxBS Benchmarking [44]: We find that UFM: (a) outperforms MAS13R [34), another end- to-end transformer trained on large-scale data for correspondence; (b-f) performs well on images with scale, viewpoint, illumination, and seasonal changes, and (g-i) struggles with pairs showing extreme coupled season, illumination, and scale changes or captured across different imaging spectrums, where RoMA [15] is more robust. We provide further insights in Section 5 and believe the primary reason to the preservation semantic matching capabilities in the pre-trained image encoder.
specialized fine-tuning that limits the extent of deviation from the pre-trained weights. We aim to address this in future releases of UFM.
We also acknowledge that although our training set and the TA-WB evaluation is geometrically challenging, they are biased toward static objects, potentially reducing the accuracy on dynamic ones, shown in Appendix G. Future work may include more sceneflow datasets, such as ParallelDomain- 4D [63], to balance the distribution.
Conclusion
We present UFM, Unified Flow and Matching model that predicts visually grounded dense cor- respondences and covisibility. Using a simple caanformer-basee design, UFM directly regresses high-resolution correspondence and covisibility maps, enabling it to learn from unified dataset effectively. Extensive Experiments show that UFM, trained on optical flow and wide-baseline match- ing data, benefits from mutual improvement and outperforms specialized methods in each domain. Looking ahead, combining UFM with semantic matching and refinement techniques would further improve its robustness and accuracy, paving the way to general-purpose purpose correspondence prediction.
10

---

Acknowledgments
This work was supported by Defense Science and Technology Ageney (DSTA) Contract #DST000EC124000205 and partially by DEVCOM Army Research Laboratory (ARL) under SARA Degraded SLAM CRA W911NFF20-SC
The compute for this work was provided by Bridges-2 at PSC through allocation cis220039p from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by NSF grants #2 #2138259, #2138286, #2138307, #2137603, and #213296. We thank Swaminathan Gurumurthy, Mihir Sharma, Jeff Tan, Shibo Zhao, Can Xu, Khiem Vuong, and other members of the AirLab for their insightful discussions and assistance with parts of the work. Lastly, shout out to Peter Kontschieder for one of the in-the-wild image pairs featured in the first figure.
References
[1] Jonathan T Barron. A general and adaptive robust loss function. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 1331-4339, 2019. 5
[2] Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, and Elad Shulman. ARKitscenes a diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data. In Tiirty-fifth Conference on Neural Information Processing Systems Dulasets and Benchmarks Track (Round 1), 2021. URL httph:ttp//// wowpooppeevvieeeeeeeeeeeeeeeevee net. wwrt://frummii/m dttp
5
[3] Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207-219, 2022. 7
[4] Thomas Brox and Jitendra Malik. Large displacement optical flow:
descriptor matching in variational motion estimation. IEEE transactions pattern analysis and machine intelligence, 33(3):500-513, 2010.
[5] Andrés Bruhn, Joachim Weickert, and Christoph Schnörr. Lucas/kanade meets horn/schunck: Combining local and global optic flow methods. International journal of computer vision, 61: 211-231, 2005.
[6] Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In Computer Vision-ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12, pages 611 1-65.2 Springer, 2012. 3
[7] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: dichly-annotade 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828-5839, 2017. 7
[8] Tri Dao. Flashattention--2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08660) 2023. 4
[9] Yong Deng, Jimin Xiao, Steven Zhiying Zhou, and Jiashi Feng. Detail preserving coarse-to-fiee matching for stereo matching and optical flow. IEEE Transactions on Image Processing, 30:
5835-5847, 2021.
[10] Daniel DeTone, Tomasz Malisiewiez, and Andrew Rabinovich. Superpoint: Self-supervised interest point detection and description. In Proceedings of the IEEE conference computer vision and pattern recognition workshops, pages 224 236, 2018. 4
[11] Qiaole Dong, Chenjie Cao, and Yanwei Fu. Rethinking optical flow from geometric matching consistent perspective. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 1337-1347, 2023. 3
[12] Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learning optical flow with convolutional networks. Proceedings of the IEEE international conference on computer vision, pages 2758-2766, 2015. 3, 6,
[13] Bardienus Duisterhof, Lojze Zust, Philippe Weinzaepfel, Vincent Leroy, Yohann Cabon, and Jerome Revaud. Mast3r-sfm: fully-integrated solution for unconstrained structure-from motion. arXiv preprint arXiv:2409.19153 2024. 2

---

[14] Johan Edstedt, Ioannis Athanasiadis, Märten Wadenbäck, and Michael Felsberg. Dkm: Dense kernelized feature matching for geometry estimation. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 1775517775, 2023. 3,7
[15] Johan Edstedt, Qiyu Sun, Georg Bökman, Mårten Wadenbäck, and Michael Felsherg. Roma: Robust dense feature matching. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 19790-19800, 2024. 3, 4, 5, 7,7, 10
[16) Kristen Grauman, Andrew Westhury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyl- los Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19383-19400, 2024. 8
[17] Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J Fleet, Dan Gnanapragasam, Florian Golemo, Charles Herrmann, et al. Kubric: A scalable dataset generator. In P Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1761,, 2022. 6,
[18] Xingyi He, Hao Yu, Sida Peng. Dongli Tan, Zehong Shen, Hujun Bao, and Xiaowei Zhou. Matchanything: Universal cross-modality image matching with large-scale pre-training. arXiv preprint arXivv2501.07556 2025. 3
[19] Yinlin Hu, Rui Song. and Yunsong Li. Efficient coarse-to-fin fine patchmatch for large displace- ment optical flow. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5704-5712, 2016. 3
[20] Zhaoyang Huang. Xiaoyu Shi, Chao Zhang, Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer: A transformer architecture for optical flow. In European conference on computer vision, pages 668-685. Springer, 2022. 3, 7
[21] Tak- Wai Hui, Xiaoou Tang, and Chen Change Loy. Liteflownet: A lightweight convolutional neural network for optical flow estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8981-8989, 2018
[22] Junhwa Hur and Stefan Roth. Iterative residual refinement for joint optical flow and occlusion estimation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 5754-5763, 2019.
[23] Eddy 1lg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox, Flownet 2.0: Evolution of optical flow estimation with deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2462-2470, 2017. 3
[24] Andrew Jaegle, Sehastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, et al. Perceiver io: A general architecture for structured inputs & outputs. arXiv preprint arXiv:2
2021.
(25) Rasmus Jensen, Anders Dahl, George Vogiatzis, Engin Tola, and Henrik Aanæs. Large scale multi-view stereopsis evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 406-413, 2014.
[26] Hanwen Jiang, Hao Tan, Peng Wang, Haian Jin, Yue Zhao, Sai Bi, Kai Zhang, Fujun Luan, Kalyan Sunkavalli, Qixing Huang, et al. Rayzer: A self-supervised large view synthesis model. arXiv preprini arXiv:2505.00702 2025. 3
[27] Wei Jiang, Eduard Trulls, Jan Hosang, Andrea Tagliasacchi, and Kwang Moo Yi. Cotr: Correspondence transformer for matching across images. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6207-6217, 2021.
[28] Xingyu Jiang, Jiangwei Ren, Zizhuo Li, Xin Zhou, Dingkang Liang, and Xiang Bai. Minima: Modality invariant image matching, arXiv preprint arXiv:2412.19112 2024. 3
[29] Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang. Fujun Luan, Noah Snavely, and Zexiang Xu. Lvsm: A large view synthesis model with minimal 3d inductive bias. arXiv preprint 2410..221,, 2024. 3
12

---

[30] Nikhil Keetha, Avneesh Mishra, Jay Karhade, Krishna Murthy Jatavallabhula, Sebastian Scherer, Madhava Krishna, and Sourav Garg. Anyloc: Towards universal visual place recognition. IEEE Robotics and Automation Letters, (22):128-11999 2023. 2
[31] Nikhil Keetha, Norman Müller, Johannes Schönberger, Lorenzo Porzi,
Yuchen Zhang, Tobias Fischer, Arno Knapitsch, Duncan Zauss, Ethan Weher, Nelson Antunes, et Mapanything: Universal feed-forward metric 3d reconstruction. arXiv preprint arXiv:2509. / 134/4, 2025. 3
[32] Nikhil Varma Keetha, Michael Milford, and Sourav Garg. A hierarchical dual model of enviromment-and place-specific utility for visual place recognition. IEEE Robotics and Automa- tion Letters, 6(4):6969-6976, 2021. 2
[33] Daniel Kondermann, Rahul Nair, Stephan Meister, Wolfgang Mischler, Burkhard Güssefeld, Sabine Hofmann, Claus Brenner, and Bernd Jähne. Stereo ground truth with error bars. In Asian Conference Computer Vision, ACCV 2014, 2014. 3
[34] Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. In European Conference on Computer Vision, pages 71-91. Springer, 2024. 2, 4, 5, 7, 10
[35] Jingxing Li, Yongjae Lee, Deliang Fan, Abhay Kumar Yadav, Cheng Peng, and Rama Chellappa. Speedy mast3r. In 2025 IEEE International Conference on Omni-layer Intelligent Systems (COINS), pages 1-6. IEEE, 2025. 5,7
[36] Siyuan Li, Lei Ke, Martin Danelljan, Luigi Piccinelli, Mattia Segu, Luc Van Gool, and Fisher Yu. Matching anything by segmenting anything. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11963-18977 2024. 2
[37] Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2041-2050, 2018. 2, 6,
[38] Philipp Lindenberger. Paul-Edouard Sarlin, and Mare Pollefeys. Lightglue: Local feature matching at light speed. In Proceedings the Vision, pages 17627-17638. 2023.
the IEEE/CVF International Conference on Computer
[39] Nikolaus Mayer, Eddy 1lg. Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 4040-4048, 2016. 3, 6,
[40] Lukas Mchl, Jenny Schmalfuss, Azin Jahedi, Yaroslava Nalivayko, and Andrés Bruhn. Spring:
A high-resolution high-detal dataset and benchmark for scene flow, optical flow and stereo. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4981-4991, 2023. 6,
[41] laroslav Melekhov, Aleksei Tiulpin, Torsten Sattler, Mare Pollefeys, Esa Rahtu, and Juho Kannala. Dgc-net: Dense geometric correspondence network. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1034-1042. IEEE, 2019.
[42] Moritz Menze, Christian Heipke, and Andreas Geiger. Joint 3d estimation of vehicles and scene flow. In ISPRS Workshop on Image Sequence Analysis (ISA), 2015. 3, 6,
[43] Moritz Menze, Christian Heipke, and Andreas Geiger. Object scene flow. ISPRS Journal of Photogrammetrry and Remote Sensing (JPRS), 2018. 3
[44] Dmytro Mishkin, Jiri Matas, Michal Perdoch, and Karel Lenc. Wxbs: Wide baseline stereo generalizations. arXiv preprint arXiv: 1 150.0060440663 2015. 9, 10
[45] Riku Murai, Eric Dexheimer, and Andrew Davison. Mast3r-slam: Real-time dense slam with 3d reconstruction priors. arXiv preprint arXiv:2412. 2024. 2,3
[46] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07119. 2023. 3, 4,9
[47] Yuheng Qiu, Yutian Chen, Zihao Zhang, Wenshan Wang, and Sebastian Scherer. Mac- vo: Metrics-aware covariance for learning-based stereo visual odometry. arXiv preprint (0f::22490004499 2024.7 2,6 6
13

---

[48] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12179-
12 12188, 2021. 4
[49] Paul-Fdouard Sarlin, Cesar Cadena, Roland Siegwart, and Marcin Dymezyk. From coarse to fine: Robust bierarchical localization at large scale. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pages 12716-12725, 2019. 2,
[50] Thomas Schops, Johannes I. Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Mare Pollefeys, and Andreas Geiger. multi- stereo benchmark with high-resolution images and multi-camera videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3260-3269, 2017. 7
[51] Philipp Schröppel, Jan Bechtold, Artemij Amiranashvili, and Thomas Brox. A benchmark and a baseline for robust multi-view depth estimation. In Proceedings of the International Conference on 3D Vision (3DV), 2022. 6
[52] Xuelun Shen, Zhipeng Cai, Wei Yin, Matthias Müller, Zijun Li, Kaixuan Wang, Xiaozhi Chen, and Cheng Wang. Gim: Learning generalizable image matcher from internet videos. arXiv preprint 27Xiv:2402.11095, 2024. 3
[53] Cameron Smith, David Charatan, Ayush Tewari, and Vincent Sitzmann. Flowmap: High-quality camera poses, intrinsics, and depth via gradient descent. arXiv preprint arXiv:2404.15259, 2024.
[54] Xiuchao Sui, Shaohua Li, Xue Geng, Yan Wu, Xinxing Xu, Yong Liu, Rick Goh, and Hongyuan Zhu. Craft: Cross- attentional flow transformer for robust optical flow. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 17602-17611, 2022. 3
[55] Deqing Sun, Xiaodong Yang, Ming- Yu Liu, and Jan Kautz. Pwe-net: Cnns for optical flow using pyramid, warping, and cost volume. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8934 8943, 2018. 3
[56] Andrew Szot, Alex Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Tumer, Noah Maestre, Mustafa Mukadam, Devendra Chaplot, Oleksandr Maksymets, Aaron Gokaslan, Vladimir Vondrus, Sameer Dharur, Franziska Meier, Wojeiech Galuba, Angel Chang, Zsolt Kira, Vladlen Koltun, Jitendra Malik, Manolis Savva, and Dhruv Batra. Habitat 2.0: Training home assistants to rearrange their habitat. In Advances in Neural Information Processing Systems (NeurIPS), 2021. 6,
[57] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 11 16, pages 402-419. Springer, 2020. 3
[58) Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems, 34:16558-1656) 2021. 2
[59] Carl Toft, Will Maddern, Akihiko Torii, Lars Hammarstrand, Erik Stenborg, Daniel Safari, Masatoshi Okutomi, Mare Pollefeys, Josef Sivic, Tomas Pajdla, et al. Long-term visual localization revisited. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(4):
2074-2088, 2020. 2
[60] Prune Truong, Martin Danelljan, and Radu Timofte. Glu-net: Global-local universal network for dense flow and correspondences. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6258 6268, 2020. 2.3
[61] Prune Truong, Martin Danelljan, Luc Van Gool, and Radu Timofte. Learning accurate dense correspondences and when to trust them. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5714-5724, 2021. 3
[62] Prune Truong, Martin Danelljan, Radu Timofte, and Luc Van Gool. Pde-nett: Enhanced probabilistic dense correspondence network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8):10247-10266, 2023.
[63] Basile Van Hoorick, Rundi Wu, Ege Ozguroglu, Kyle Sargent, Ruoshi Liu, Pavel Tokmakov, Achal Dave, Changxi Zheng, and Carl Vondrick. Generative camera dolly: Extreme monocular dynamic novel view synthesis. In European Conference on Computer Vision, pages 313-331.
14

---

Springer, 2024. 6, 10,
[64] Khiem Vuong, Anurag Ghosh, Deva Ramanan, Srinivasa Narasimhan, and Shubham Tulsiani. Acrialmegadepth: Learning aerial-ground reconstruction and view synthesis. arXiv preprint arXiv:2504. 13157, 2025.
[65] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. Proceedings of the Computer Vision and Pattern Recognition Conference, pages 5304, 2025. 3
[66] Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. arXiv preprint arXiv:2501. 12387, 2025.
6
[67] Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20697-20709, 2024. 3, 4, 6
[68] Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4909-4915 IEEE, 2020. 2, 3, 6, 1,4
[69] Yihan Wang, Lahav Lipson, and Jia Deng. Sea-raft: Simple, efficient, accurate raft for optical flow. In European Conference on Computer Vision, pages 36-54. Springer, 2024. 3, 4,
[70] Philippe Weinzaepfel, Thomas Lucas, Vincent Leroy, Yohann Cabon, Vaibhav Arora, Romain Brégier, Gabriela Csurka, Leonid Antsfeld, Boris Chidlovskii, and Jérôme Revaud. Croco v2: Improved cross-view completion pre-training for stereo matching and optical flow. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17969- 17980, 2023. 3,4 4
[71] George Wolberg. Digital image warping, volume 10662. IEEE computer society press Los Alamitos, CA, 1990. 2
[72] Housheng Xie, Yukuan Zhang, Junhui Qiu, Xiangshuai Zhai, Xuedong Liu, Yang Yang. Shan Zhao, Yongfang Luo, and Jianbo Zhong, Semantics lead all: Towards unified image registration and fusion from semantic perspective. Information Fusion, 98: 101835, 2023. 2
[73] Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 8121-8130, 2022. 3,7
[74] Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao, and Andreas Geiger. Unifying flow, stereo and depth estimation. IEEE Transactions on Pattern Analysis and Machine Intellignce, (5(11):13941111
2023. 3,4 4
[75] Gengshan Yang and Deva Ramanan. Volumetric correspondence networks for optical flow. Advances in neural information processing systems, 32, 2019.
[76] Yao Yao. Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1790-1799, 2020. 6, 1
[77] Chandan Yeshwanth, Kueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high- fidelity dataset of 3d indoor scenes. In Proceedings of the IEEE/CVF International Conference
on Computer Vision, pages 12-22, 2023. 6,
[78] Songyan Zhang, Xinyu Sun, Hao Chen, Bo Li, and Chunhua Shen. Rgm: A robust generalizable matching model. arXiv preprint arXiv:2310 10. 1755, 2023. 2,3
[79] Shiyu Zhao, Long Zhao, Zhixing Zhang, Enyu Zhou, and Dimitris Metaxas. Global matching with overlapping attention for optical flow estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17592-17601, 2022. 3
15

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our contributions introduced in the abstract and at the end of Section are supported by technical details in Section 3 and results in Section 4.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: See Section 5.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We described our model's architecture, training data, and training protocol in Section 3 with more details in the supplementary material. We have also released our model weights, inference and training code, and dataset preparation script.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We have provided our training and inference code, dataset preparation and experimental evaluation scripts, pretrained model weights, as well as usage instructions at bttps://uni flowmat ch. githubl io.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: For training, Section 3.4 and the supplemental material. Testing metrics are standard in this domain and explained Section
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Following standard practice in optical flow estimation and wide-baseline matching literature, we report the mean the related EPE metrics the described test sets.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed reproduce the experiments?
Question:
16

---

Answer:
[Yes]
Justification: We detail the training cost in Section 3.4 and the average execution time of our method in Section 4.
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: /ttpst////euppp...eee httppppcccc/tttcccscs..ll.l//ll.
Answer: [Yes]
Justification: The authors have reviewed and follow all applicable points in the ethical guidelines.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The paper discussed potential positive societal impacts by presenting advance- ments in correspondence prediction, which could benefit downstream domains including robotics and virtual reality. On the other hand, potential negative impacts are addressed in Section 5, where we discuss scenarios in which the method may fail, which could lead to failures in safety-critical systems, causing social harm.
Justification:
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper has such risks.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have cited all datasets and models used in Section 3.3 and Section 4. The datasets have been used accordance with their provided licenses.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have released our code and pretrained models GitHub, accompanied by detailed documentation. We are committed to addressing user feedback and improving the documentation needed.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: There are no human subjects involved in the experiments.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
17

---

Answer: [NA]
Justification: There are no human subjects involved in the experiments.
16. Declaration LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our method does not involve LLMs.
18

---

Table S.1: Underlying data sources used for generating correspondence and covisibility ground truth, along with the reprojection enror threshold used when using depth and pose for covisibility.
Category
Dataset
Source of Correspondence
Source of Covisible Mask
Abs, Depth Threshokl
Rel. Depdi Threshold
Stanic Scone
BlendedMVS (26)
0.1
MegaDepshh [37]
ScanNete+ [77]
TastanAir V2 [68]]
Unproject depthmap
across cameras
Threshold depth reprojection erroe
0.1
0.1
0.1
0.005
0.005
0.01
Habitat CAD [56]
0.1
Optical Flow
HDIK (42)
Spring (40)
Dattsst-proviedd
Dutaset-rroviled
Monkaz [39]
[39]
Dhttees-proided
Scene flow reprojection thresheld
0.01
0.01
0.001
0:001
PhyingChairs [12]
Datases provided
FoV mask (approximate)
Rigid Posed Objects
Kubric4D 63)
Depthmap
object pose
Depthmap & object pose reprojection thrshoold
0.1
A Computing Covisibility Mask
Computing the covisibility mask for datasets in Table is key step to support unified training. In this section, we detail the exact protocol and parameters we used to compute the covisibility mask for all datasets, summarized in Table S.I. We will begin with a general principle of using depth reprojection error to compute covisibility, and then detail its application to three data categories: (I) Static Scenes, (2) Optical Flow, and (3) Rigid Posed Objects.
Given two corresponding pixels in the source and target images, we determine their covisibility by checking 3D consistency that is, whether their depths unproject to the same 3D point. We compute the Euclidean distance between the points, and consider the pixels covisible if their distance is below a threshold. We refer to this approach as thresholding depth reprojection error.
A.1. Covisibility from Depth Reprojection Error.
Formally, given a source pixel is E l and a target pixel i, E 12, we compute their 3D coordinates Ps: Pt and the depth reprojection error e, defined as e Ip, - 121 Then, the pixels are determined to be covisible if [pp, - Prll2 is less than an absolute threshold Ty
While this metric captures the fundamental idea of computing 3D consistency, it implies a fixed 3D tolerance regardless of the scene distance from the camera. We found this is suboptimal when handling both near and far objects, as far objects are described with less pixels, thus having larger uncertainty in depth and geometry. To address this, we introduce a relative threshold that increases linearly with the distance between the source 3D point ps and the target camera center Oz. Thus, the final thresholding scheme we use is:
e - |lp --ll2 T F, 1lp, 122p-2212
(S.1)
All dataset categories use the same covisibility thresholding scheme, with dataset-specific parameters summarized in Table S. They only differ in how the 3D points p, and pt, and ultimately the error e, are computed. We describe these procedures in the following subsections.
A.2. Computing Correspondence and Covisibility. We begin by specifying the relevant informa- tion required from each dataset category, followed by an explanation of how the corresponding 2D pixel i, the 3D points py and pr, and the reprojection error e are computed given a source pixel is-
Static Scenes
For static scenes, the fixed geometry allows us to compute covisibility by comparing
unprojected depths directly. Specifically, given depthmaps D2 D2 E #x poses Ti, 72 E &SE(3), and camera projection functions Te, Tr, we compute the corresponding projected pixel by:
P. Ti 7
i, i2= ps), P: T2n, - (i,)D(i)
(S.2)
Since i, and i are corresponding pixel locations, Ps: Pt and target camera center Oz are collinear. Note that we filter outwof-view or points behind the target camera when computing i, as non-covisible.

---

Ip; - 1Pll- [l: 02112 1p, 02112
- 02112 -
D2(4))
(S.3)
is the difference between the expected depth ||p- O2 2 of the projected 3D point Py and the perceived depth D2(4) of the corresponding 2D pixel in the target camera.
Interpolating D2(1)) from the discrete depthmap D2 is vital to obtain realistic covisibility mask. While i, is typically pixel-aligned - since we compute covisibility for source pixels in the target image - i, is derived from continuous depth and camera transformations, and thus almost always lies at a fractional pixel coordinate. We empirically found that bilinear interpolation yields better results than nearest- neigh,,,, it provides a first-order approximation of the local depth geometry. In contrast, nearest- neighbt interpolation introduces heavy aliasing, especially on inclined surfaces. Although bilinear interpolation may produce ghosting artifacts, it is unlikely that non-covisible pixel will match the expected depth closely enough to be mistakenly classified as covisible.
Optical (Scene) Flow Unlike static scenes, optical flow datasets usually contain dynamic scenes and pairs in these datasets come from different timesteps. As the scene changes over time, deter- mining covisibility requires scene-flow information to account for object motion. We build upon the formulation for static scenes and adjust the expected position with scene dynamics.
Formally, we use uniform camera projection model and all information as described static scenes, optical flow ground truth 6. E R2xHx and depth (disparity) change Di E RR As optical flow describes how a source pixel is moves to the target pixel i, in the image space, depth change details how the underlying 3D point changes in its depth. Specifically, the 3D point refered by i, at the source image with depth Di (is) would move to pixel i, = is + &8 (is) in the target image, with an compute the source point in the target time and
updated depth of D (i,) + 11-2 (1.). Thus, the projection error (similar to Eq. (S.3)) as:
+D1-2(()) Ps11-2 (
(S.4)
Then, the same interpolation and thresholding logic the static datasets.
FhyingChairs is the only exception in this category, lacking both precomputed covisibility masks and scene flow information. Nonetheless, include during training to balance the relatively limited optical flow data compared to wide-baseline datasets. This does not pose a significant deviation from our covisibility-onl training scheme due to the dataset's limited motion and relatively simple backgrounds. For correspondence training, we use the FoV mask as a proxy for the covisibility mask. We excluded FlyingChairs when supervising covisibility explained in Sec. A.3.
Rigid Posed Objects Rigid posed objects refer to scenes composed entirely of rigid objects whose poses are known at all timesteps. This setting can be seen as a special case of the scene- feoe-flow dataset where motion is fully defined between all pairs of timesteps. We adjust the expected position with the object movement information, similar to the formulation for optical flow.
Specifically, we assume all information in static scenes, the set of object poses [1.2 k==1 at both time steps, K being the number of objects. and S: I {1.... .K), the segmentation map that assign each pixel to the corresponding object ID. Given a source pixel i,, we can obtain its object assignment == S(, and its coordinate on this object - ( (i.) (1.).. Since the object is rigid, the (A)
(k)
point will stay at the same object coordinate between source and target and be transposed to pose to the target frame. Combining them, we have:
(k) (8) ( (Tiz (L)), i, 1 Po.1-2), e= ==1llp..122 - 0211 1000-2pp000 D2(1,) (S.5)
Ps,1-2 =1
We threshold the error for covisibility in the previous paragraphs.
A.3. Covisibility Supervision Range. In addition to the covisibility mask, we compute covisi- bility supervision mask that excludes regions where covisibility cannot be evaluated due to missing or invalid depth values. We apply supervision only within this mask ensure accurate, though incomplete, training targets.

---

Formally, given depth validity masks V1 and V22 for the source and target images respectively, we first evaluate the validity the target depth the ground -ruth flow locations Vorher [ V2 Ogr [i]) and we obtain the covisibility supervision mask as
Vevii (V---VVV--))-- (Voherr)
(S.6)
where Fi is the FoV mask, which is true for pixels in the source image whose corresponding 3D points have a valid projection into the image space the second camera, regardless of occlusion. The first term captures the region that is out of view, while the second term captures the region that projects to the target's FoV and has valid depth at the target for confirming covisibility.
We used an all-zero covisibility supervision mask on the FlyingChairs dataset to avoid its approxi- mated covisibility (actually FoV mask) from being used train covisibility prediction.
B Sampling Strategy
We explain our custom pair sampling strategy for the Scannet++V2 and Kubric4D datasets.
ScanNet++ V2: We compute possible image pairs within each scene and retain those with sufficient covisibility. Specifically, following the procedure in Sec. A.2, evaluate covisibility for all pairs of DSLR images in each scene and keep those with mutual covisibility greater than 25%.
Kubric4D: Kubrie4D the only dataset that enables sampling across both viewpoints and time. Accordingly, we bias our sampling toward pairs that involve changes in both dimensions. Specifically, Kubric4D has 2800 scenes with 16 fixed cameras each scene and 60 frames per scene. We sampled 3600 pairs per scene with viewpoints and time change independently:
We aim for 60 90 angle difference for viewpoints. To achieve this, we first computed the rotation angle between all pairs of camera and assigned weight as
1+a,
a E (0,z/3)
w(a)=
1+x/3,
E T/3, TT22
(S.7)
0, 2 n/2
We sample frame differences bias toward large difference since motion in Kubrie4D is small. Specifically, we sample frame difference between O and 40, with probabilities increasing linearly such that the largest frame difference has twice the probability of being selected compared to the smallest. Given a sampled difference, we then uniformly choose valid start and end frame.
C TA-WB Training & Testing Dataset
TartanAir provides images covering all six directions around each scene, enabling to design geometric sampler that explicitly controls viewpoint differences when sampling covisible pairs.
Geometric Sampler The geometric sampler generates pairs of rendering directions and source-targes cameras based on geometric constraints for viewpoint difference and coarse covisibility check. An overview is presented in Fig. S.I.
We first voxelize the scene and compute the set of visible voxels for each camera. The sampling process begins by randomly selecting source camera center and a visible voxel nearby, establishing the viewing direction for the source. Based on this direction, whose viewing angles differ by the desired amount. Then, we filter out candidates that cannot see the selected voxel based pre-computed covisibility. In this way, we able to sample covisible yet geometrically controlled viewing directions. Finally, we sample a random roll angle from Vf (0,0.11 0.1) to complement the viewing direction into rotation, and apply random perturbation to all axes from M (0, (.)). These perturbations prevent the sampled viewing direction from always focusing on the voxel center, adding diversity to the sampling.
identify candidate target cameras
After rendering the images, we additional filtering to ensure their quality. We filter out pairs with any of their images containing more than 10% of over- or under-exposed pixels, and if any of the
3

---

Target Voxel
Candidate Target
Camera
Camera Centers
Sourg
Center
Cam
(a) Voxelize Compute Covisibility
(b) Given Source Camera and Voxel, Find Candidate Cameras
Source Camera
Jarget Cameras
(c) Filter by Covisibility & Sample
Figure S.1: The Geometric Sampler: (a) From the pointeloud of a scene, we voxelize it and compute the covisibility between all camera centers and all voxels. (b) We randomly select camera location as the source camera and target voxel for the source camera to center at. We filter out all candidate camera position that forms a required viewpoint difference when looking at the same target voxel. (c) We filter out candidate cameras by covisibility.
forward/backward covisibility is less than 20%. We further check the pair is solvable, i.e., does the pair provide enough visual evidence to establish match? To do this, we warp the target image according to the ground-truth label (similar to Fig. I, 4) and try to match it to the source image with Superpoint Lightglue [10, 38, 49]. Since warping is done with ground truth, the matcher should ideally return near-zero pixel displacement covisible regions. it does not, the pair lacks enough information support matching. We retain only pairs with average matching error below 6 pixels.
TA-WB Benchmark We use the geometric sampler to select pairs from the OldScandi navia. Sewerage, Superma rket, Deser Gar.tatttttttt.. ann, and PolarScize \2 [68]. We sample approximately equal numbers of pairs from the angular bins [0", 30%) [309, 60%] and [60", 90%], and allocate roughly half as many pairs to the [90", 120") bin. Samples of the dataset are provided in Fig. S.2.
environments in Tar-
D Training the Refinement
We trained the refinement module separately, using frozen base model obtained from the initial training stage. Since the refinement value is computed via attention between the source pixel feature and features in a local neighborhood around the regressed flow target, it can be interpreted as multi-modal distribution centered around the base model's predicted flow. use the cross-entropy loss to supervise the distribution at the ground-truth location. Importantly, we limit supervision to pixels whose ground-truth flow falls within the neighborhood and use a softened target. Rather than having the nearest pixel that is closest to the flow target classification target, distribute

---

826
826
(a) Source Image
(b) Target Image
(c) Forward Flow
(d) Forward Covisibility
Figure S.2: Example Images from TA-WB Benchmark: The benchmark contains dense correspon-
dence annotation and accurate covisibility for challenging viewpoint shifts.
W
W2
Flow
W1 a),
133
00
W4
W2 a3
Regressed
U/ a1 ()
Flow target
#4 a(1 B)
Figure S.3: Refinement Target Weights: Given inlier ground-truth flow target, obtain its adjacent pixels and assign continuous weight based on the sub-pixel location (az, ß) of the target.
smooth weights across the four adjacent pixels, with values that change continuously based on the flow target. We found that such target is easier to train and enables sub-pixel refinement. The weights are shown in Fig. S.3.
We trained the refinement module on the BlendedMVS, MegaDepth, Habitat, and ScanNet++ datasets using image pairs as listed in Table We selected these datasets due to their relatively high sub-pixel accuracy. The base model was frozen during this stage, and the refinement module was
5

---

Source/Target Image
Source Image Refinement Features (PCA)
Target Image Refinement Features (PCA)
Figure S.4: Example of Refinement Features: We visualized the refinement features for pair of images with PCA. The features exhibit emergent high- fegh-frequency and edge-following behavior.
Frazen Validation
ClyingChairs
Fine-tuned Validation Loss FlyingChairs
Layer23 Frozen
Layer23 Rnetuned
Layer10
Layer10 Rnetuned
Validation Loss
Epoch
Epoch
Figure S.5: Freezing DINOv2 encoder suboptimal when training UFM on FlyingChairs: We show the validation EPE of FlyingChairs using features from different layers of frozen pre-trained encoder (left) and finetuning the pre- pre-trained encoder truncated to specific layer (right).
trained for 30 epochs with learning rate of 1- 10-4 All other optimizer settings are the same as the 560 base model training, as detailed in Section 3.4. Fig. S.4 shows a visualization of the trained features, where we high-frequency and edge-following behavior that encodes the local details.
E Effect of Freezing the Encoder
We found that freezing the DINOv2 encoder and using its last-layer features was suboptimal for UFM. Specifically, when training UFM on the FlyingChairs dataset, we observed significant validation EPE gap between using features from the last layer versus intermediate layers of the frozen DINOv2 encoder. As shown in Fig. S.5, UFM trained with the last layer features from frozen DINOv2 obtained near EPE, whereas features from layer yielded sub-pixel performance. This gap is not observed in the finetuned setting, given sufficient training.
E.1. Hypothesis for Performance Gap with Frozen Features: The task of predicting the dense correspondence can be roughly divided into 3 steps. For patch in the source image, would need to: (1) understand the content in its own patch, (2) find the corresponding patch(es) in the other image, and (3) copy its coordinate difference. While one may argue that step (2) is unnecessary because the network can leverage structural priors surrounding context fill in the gap, it remains the most direct and reliable route to accurate correspondence due to the causal nature of the task.
Step (2), i.e., finding the corresponding patch(es) in the other image, is achieved in only one structure of UFM the global attention. This because other components either project patch features independently operate solely tokens from single image. lacking direct cross-image interaction.

---

DINOVZ
DINOV2
Layer
N
Layer N-1
Layer N-1
The Probe
layar featunes image
Projected Img0 Features
Layer k
Lineer Projection
Dot Product SoftWass
Layer k
layar patch fusluris Image
Projected Img1 Features
Layer
Layer 0
Imgg
Ingt
Flow (Comospondence)
Aggregate pixels each patch
Expected Patch Similarity
Simiarity
Cross Entropy
Dalasel
Figure S.6: Setup for the Probing Experiment: For each layer in a frozen image encoder, we extract patch features for a pair of images and apply a shared linear projection. Softmax attention is computed between source and target features, and the resulting similarity distribution is compared to ground-truth correspondences via cross-entropy loss. The final training loss serves proxy for correspondence information encoded at each layer.
In the global attention module, (2) is realized by the attention computing, which depends on the dot-product similarity of the patch feature after learnable linear projection.
This implies a key requirement: dence, or "corresponding features", such that they attend selectively their corresponding patches in the other image after a simple linear projection. We designed a probing experiment to quantify the upper bound of the corresponding features in each layer of frozen encoder, and later establish its correlation to UFM's performance experimentally.
Patch features must encode information that reveals their correspon-
E.2. Probing Experiment:. We overfit simple network on top of trained backbone a specific dataset, using the converged training loss as a proxy for the presence of relevant information in the backbone representations. was used as an analysis strategy in NLP as training "probing classifiers" to associate the internal representation of the model with explicit properties probing experiment to test the presence of corresponding features from layers a frozen DINOv2.
[3]. We use similar
The outline of our probing experiment is shown in Fig. S.6. We select a relatively small dataset and disable all augmentation to ensure that training will converge. We infer each pair of images through the frozen DINOv2 encoder and project the source and target features through a layer-specific linear mimic the global attention mechanism.
layer. We then compute softmax dot-product similarity Each layer's 's probe is trained independently, and its performance reflects how well the layer encodes corresponding features that can be revealed during the global attention. Patch-wise similarity is defined as the proportion of pixel-wise correspondences between patches, weighted by covisibility. Formally, given correspondence and covisibility labels @² C the ground-truth patch similarity P,) between source patch P, C / and target patch P. C 12 is defined as:
sP,, Lier, 1(i+ Φ& [ E P.) C# [
(S.8)
P
Given a fixed dataset, we infer pair of images through the frozen image encoder and obtain the patch features at all layers. For each layer, we project the source and target features using shared linear layer and compute their softmax attention, resulting patch similarity. This predicted distribution is then compared to the ground-truth similarity using cross-entropy loss. We train only the projection layers on this dataset and use the final training loss as an indicator of how well the features at each layer encode correspondence information.
a binary distribution of piie
7

---

Normalized Performance and EPE Across Layers Flying Chairs
L10
L13
Normalized Probing Performacc (higher
L14
230
0.8
L15
L7
LIS
2.00
1.75
0.4-
120
02-
1.0
11
18
Layer
Figure S.7: Correlation between probing and val. EPE: We plotted the probing performance (blue) and the EPE of UFM on FlyingChairs when using frozen DINOv2 features from different layers.
BlendedMVS 224
F Plyinghaiirs 22.4
ScanNete+ 224
DRIGLE DADO
Probed Cross Entropy toxs
6.20
Probed Cress Entropy
Probed Cross Entropy
2.30
120
0.7
118
25
Layer
Layer
Layer
BlendedMVS 560
Flying 560
ScanNete- 560
Probed Cross Entropy Loxs
Probed Cross Entropy Lass
Probed Cross Entropy
841
0.0
Index
Figure S.8: Consistent probing results on other datasets, resolutions, and encoder sizes showing that the last layer from DINOv2 does not provide the best corresponding features and performance.
E.3. Probing Results using Correlation:. To test whether the cross-entropy loss from probing correlates with EPE performance, we trained UFM using different frozen layers of DINOv2 on the FlyingChairs dataset and collected their loss in the probing experiment. We normalized the cross- entropy Joss value into probing performance between [0, 1]. According to Fig. S.7. found a strong correlation between the loss in the probing experiment and the final validation EPE, and the peaks differ only by of the total 24 layers. This suggests that, for UFM, probing performance may serve as reliable indicator for selecting effective feature layers. Furthermore, this supports the hypothesis that the last layer of DINO does not provide the strongest corresponding feature, thus leading to suboptimal performance. We further show additional probing results on other datasets, resolutions, and DINOv2 encoder sizes in Fig. S.8. found consistent trend where the intermediate layers encode stronger correlating features and perform better.

---

Table S.2: Matching Head Ablation: With identical backbone and training data, regression- flow head (UFM design) using DPT achieved higher precision than a patch-wise matching head from UniMatch.
Head Type
Sintel-C
Sintel-F
KITTI DTU EPE [covis]
ETH3D
TA-WB
Patch Matching Upsample Regression (ours)
0.91
1.49
2.05
5.36
5.47
13.85
0.79
1.44
1.87
2.64
5.56
12.87
F
Discussion on the use of Matching in UFM Architecture
As many prior works have shown the benefits of using matching for correspondence prediction, would like to further discuss and ablate our choice to regress the initial flow combined with applying matching only at pixel resolution, not patch resolution.
Matching at the patch level Matching in patch-level aggregates subsets of pixels as a patch, then matches them with other patches to produce an initial flow hypothesis at low resolution. The initial coarse flow will be upsampled and refined later, following the "coarse-to-fine" paradigm. To better present our findings, we would first like to clarify two possible meanings of the term "coarse-to-fine:: (I) the predicted flow is upsampled from a coarse fine spatial resolution, or (2) a rough prediction is progressively refined for higher accuracy. While prior methods often combine both, our approach uses (2) but not (I).
To support our choice, we replaced the DPT head of UFM s60 with the UniMatch matching head and trained with the same protocol described in Section 3.4. We adopt this head representative of satch-scale matching methods. Hence, the matching head now benefits from the same backbone capacity and unified dataset as UFM 560. The only differences the final flow prediction head and how the flow loss is applied. To maintain equivalence in supervision, we modified the UFM robust loss match how Mntca applies its loss supervising both the final flow and the bilinearly upsampled coarse flow. Essentially, the UniMatch head first performs patch-level (coarse) matching, which is then convex upsampled to pixel-wise correspondence. We follow their original codebase and refer to formulas (1) (7) from GMFlow [73] for the exact definition of the UniMatch head.
We evaluated them covisible end-point-errors. and report the results in Table S.2, where observed that the UniMatch Head yields a 22% increase EPE on average. Hence, we conclude that the regression head is more accurate in predicting correspondence and scales better to the combined dataset, corroborated by evidence in Fig. 5.
Matching at the pixel level Direct global matching of pixel descriptors allows MASI3R to achieve high precision even for large motions. Using a similar architecture and dataset scale, we show that the regression paradigm in UFM s60 achieves competitive EPE performance on challenging bise-laseline datasets, where the average flow reaches about 90 200 pixels. We attribute this to UFM's superior supervision efficiency compared to MASt3R. While MASt3R trains with at most 4096 correspondence pairs per frame, UFM leverages supervision from all covisible pixels. This dense supervision is key factor that allows UFM to achieve similar or better accuracy with much higher speed than MAS13R.
Matching and Regression are not mutually exclusive Through UFM-Refine, we demonstrate that matching- based local refinement can be effectively combined with regressed initial hypothesis to further enhance accuracy. This forms key distinction between UFM-Refine and previous dense correspondence approaches. Only with an efficient and accurate regressed initialization one confidently narrow the search range for pixel-wise matching, thereby achieving both the efficiency of regression and the precision of pixel-wise matching.
G Additional Evaluations
We provide additional zero-shot evaluations of UFM on Sintel and KITTI test sets in Table S.3, S.4. While we still observe UFM outperforms UniMatch under zero-shot settings (neither method was trained on Sintel or KITTI), we observed UFM biased towards static objects. As in Table S.4, UFM is significantly better in the background than in the foreground objects, which mostly

---

Table S.3: Zero-Shot Performance on Sintel Test Set: UFM outperforms UniMatch in the overall
metric and most sub-items, except for sub-items on
EPE
EPE matched
EPE unmatched
Datasel
Method
d0-10
d10-60
d60-140
s0-10
s10-40
s40+
all
UFM 950 refine
3.28
1.75
15,80
3.28
1.27
1.21
0.69
2,07
19.07
Sintel-Final
UniMatch-CT
4.14
1.97
21.86
3.77
1.59
1.28
0.89
2.43
24.40
UFM 9m refine
1.68
0.72
9.52
1.54
0.67
0.43
0.48
1.23
8.71
Sintel-Clean
UniMatch-CT
1.80
0.57
11.85
1.18
0.45
0.40
0.37
0.93
11.03
Table S.4: Sero-Shot Performance KITTI Test Set: UFM outperforms UniMatch in the overall metric and background objects, but not foregrond objects, which are mostly dynamic.
Method
Range
F1-bg
FI-fg
Fl-all
UFM 980 - refine
All
8.67
22.08
10.9
UniMatch-CT
All
18.01
17.27
17.89
UFM refine
occluded
4.93
19.6
7.59
UniMatch-CT
occluded
9.75
14.61
10.63
Gradient of the robust loss v.s. L1 loss (Inliers)
Gradient of the robust loss v.s. L1 loss (Outliers)
101
(0.42, 3.17)
X
(0.42, 3.17)
10
Gradient
Gradient
-1
10-1
1508, 0.12)
-10.0
-75
-3.0
-2.5
2.5
7.5
10.0
10-1
16F
19a Error Px
107
Error px
Compared to L1 loss, the robust loss used by UFM puts higher gradient to the inliers (peaks at error of 0.42 pixel), while maintaining non-negligible (0.12 at 500 pixel error) gradient for even the largest outliers to slowly correct them.
Figure S.9: Gradient of the Robust Charbonnier Loss:
dynamic. Future work may include more sceneflow datasets, such as ParallelDomain [63], to balance the distribution.
H Visualization on the Robust Charbonnier Loss
We provide an additional visualization of the robust Charbonnier loss. The loss is defined as:
x 2 a
9
2 2| α
Lrobust (x; 0) c)=
1
-1
(S.9)
We followed RoMa's choice of 0.5, c= 0.24. As shown in Fig. S.9, this combination of and puts high gradient on inliers while maintaining non-negligible gradient for even the largest outliers slowly correct them.
10


Reviews:
[{"review_id": "FgIR2jmTXI", "reviewer": "Reviewer_B4yj", "summary": "This paper proposes utilizing both optical flow and large baseline correspondence datasets for mutual benefits on two tasks.\nThe authors present a simple but effective model architecture which consists of visual encoder, 12-layer transformer (with the positional encoded features of the two encoders as input) and two heads for flow and co-visibility (with encoder output and 6/9/12 layer outputs as input). The output of regression flow head is further fine-tuned via a weighted summation of neighbor coordinates based on feature similarity between reference and target frames.\nIn training, only co-visible pixels are utilized. In order to identify co-visible pixels, according to the supplementary, depth reprojection error is basically used and, additionally, information on depth change (in dynamic scenes) and rigidity (in rigid motion) is also utilized.\nBy applying the proposed geometric sampler, the authors curated the existing datasets to obtain a new selected set (TA-WB) of data to evaluate dense wide-baseline correspondence solutions.", "questions": "1) How can we further investigate the benefits of multi-task learning in this paper?\n\n2) How does the proposed method work in extreme conditions such as low light scenes?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strength\n\n1) Simple but effective method \nMulti-task benefits are well known. But, the authors demonstrated how to realize the benefits in optical flow and large baseline correspondence with an effective method.\n\n2) Experiments are well done.\nEspecially, probing experiments well demonstrate the authors' choice in model architecture.\n\nWeakness\n\n1) Lack of detailed analysis on the origin of multi-task learning effects\nTable 5 well demonstrated the mutual benefits.\nIt would be nice to further investigate the origin of such benefits.\n\n2) Evaluation of diverse scenes\nAlthough the authors proposed a new dataset for wide-baseline correspondence, \nit would be nice if more diverse scenes like low light, fast moving objects, etc., are evaluated.", "was_revised": true, "final_justification": "The authors well addressed my concerns in the rebuttal. Thus, I want to keep the score.", "rebuttal": "Thank you for your thoughtful feedback and for taking the time to read the supplementary material in detail. We are encouraged that you find our method to be **simple, effective, and achieves mutual benefits** by combining optical flow and wide-baseline matching tasks and data. We are also thrilled that you find our **experiments well done** and that the **model architecture design is well supported**, especially by the probing experiment. \n\nWe have added results to address your question about performance in extreme cases and your suggestions for further improvement:\n\n- **Q1: What is the model’s performance under extreme conditions, such as low-light scenes?**\n\n    - **We add the WxBS Benchmark and show that UFM is reasonably robust to extreme condition shifts.** The benchmark includes low-light scenes, multiple visual spectrums (e.g., IR to RGB), day-night transitions, seasonal changes, and scale changes. We report performance on fundamental matrix estimation using PCK, i.e., the percentage of pre-labeled keypoints consistent with the prediction under an epipolar error threshold (higher is better, 10px is commonly used).\n\n        | Thresholds | 3px  | 5px  | 10px | 15px | 19px |\n        |------------|------|------|------|------|------|\n        | RootSIFT   | 0.06 | 0.08 | 0.10 | 0.13 | 0.16 |\n        | Superpoint + SuperGlue | 0.26 | 0.33 | 0.46 | 0.51 | 0.55 |\n        | LoFTR      | 0.48 | 0.57 | 0.67 | 0.71 | 0.73 |\n        | RoMa       | **0.74** | **0.85** | **0.92** | **0.96** | **0.96**|\n        | MASt3R     | 0.32 | 0.42 | 0.56 | 0.62 | 0.65 |\n        | UFM_560    | 0.49 | 0.63 | 0.74 | 0.77 | 0.78 | \n        | UFM_560-Refine | 0.49 | 0.62|  0.75 | 0.79 | 0.81 | \n    \n        As already noted in Section 5 (Limitations), compared to RoMA (the best performing one, where DINOv2 is frozen), finetuning the DINOv2 encoder in UFM improves precision but reduces semantic matching capabilities (as also supported by our supplementary material analysis, Section E). Despite this, UFM ranks second on the WxBS benchmark with a 0.74 PCK@10, showing strong performance and outperforming MASt3R under extreme conditions.\n    - **UFM also has reasonable performance for low-light conditions**, as UFM succeeded (matched >50% keypoints with a 10 px error) in 3/5 IR-RGB matches (note that there are no IR images in our training) and 9/11 day to night matches. \n    - **We had some early exploration for further improvement.** For low-light conditions, we tried adding a dark augmentation (p = 0.005) to the training pipeline, which simulates extreme dark and quantization effects, and observed improved IR matching (4/5) in WxBS. More fundamentally, we believe that this is related to encoder finetuning, and we think future work can explore semantic preservation losses, such as those used in Depth-Anything V2 and DUNE.\n    **We will add the WxBS result (also includes qualitative visualizations) and expand the limitations section accordingly.**\n    \n- **Q2.1: Lacking analysis of the source of mutual benefit** when training for optical flow and wide-baseline dense correspondence (shown in Table 5). \n\n    **While not a direct analysis, our architecture ablation (Figure 5) indicates that UFM’s architecture is the most effective in scaling to the unified dataset**, which we believe is crucial for achieving mutual benefits across optical flow and wide-baseline matching. We will state this explicitly in the revision.\n\n- **Q2.2: Also, how can we further investigate the benefits of multi-task learning?**\n\n    While not a focus of this paper, we believe the model may benefit further from multi-task training, such as combining 3D or 4D tasks with correspondence. We are interested in investigating this in future work.\n\n- **Q3: It would be nice if more diverse scenes, like low light, fast-moving objects, etc., were evaluated.**\n    \n    - **We agree, and we will incorporate the above WxBS results into the revision.**\n    - **The TA-WB benchmark lacks moving objects** because the source data lacks sceneflow, preventing accurate covisibility computation. Still, it is a **geometrically challenging** benchmark as visualized in Supplementary Figure S.2. We will expand the limitations section to include this point about our benchmark and we believe that future work can explore converting dynamic datasets or simulators to generate more challenging pairs using our geometric sampler."}, {"review_id": "tbLRbVC2sm", "reviewer": "Reviewer_vKeR", "summary": "Similar to UniMatch and MASt3R, this paper builds a dense correspondence network based on Dinov2 and Transformer. Additionally, it introduces a local attention-based refinement module to further enhance the results. Overall, the network architecture is reasonable, but it lacks substantial innovation. However, I believe the more significant contribution of this paper lies in the extensive training over a wide range of datasets, which provides a potentially reliable matching baseline for many downstream tasks.", "questions": "1. Please elaborate on the algorithm’s performance in occluded regions, and explain why occlusion estimation is not considered.\n\n2. Please describe your open-sourcing plan. Since the main contribution of this work lies in its applicability to downstream tasks, a well-executed open-source release would be a major positive factor.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. Once the code is open-sourced, it will provide an excellent matching tool for many downstream tasks, which is beneficial to the community.\n\n2. The overall network structure and the proposed refinement module are reasonable.\n\n3. Extensive experiments and thorough visualizations are provided, and the results appear trustworthy.\n\n**Weaknesses**\n1. The method overlooks the learning of occluded regions. Inferring occlusions is a crucial aspect in optical flow tasks, especially for downstream tasks such as sparse-view reconstruction. This omission reduces the significance of the work (as its functionality overlaps with MASt3R in most practical scenarios).\n\n2. The architecture lacks substantial innovation. Technically, the method appears to be a combination of UniMatch and MASt3R.\n\n3. The paper’s claim of unifying dense correspondence and flow is not well justified. At most, it improves long-range and short-range matching performance in dense correspondence tasks. Optical flow generally includes occlusion estimation, which is not addressed here.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and detailed comments. We are heartened by your recognition of the model's **wide applicability to downstream tasks**, coming from the contribution of **extensive training over a wide range of datasets**. We appreciate your validation that **the results are trustworthy** based on the **extensive experiments and thorough visualizations**.\n\nWe have addressed your questions/clarifications on performance in occluded regions, our unification claim, lack of architectural innovation, and open source plan.\n\n\n- **Q1: Questions regarding performance in occluded regions.**\n    - **Q1.1: The method overlooks learning in occluded regions, which is a crucial aspect in optical flow tasks.**\n    \n    \n        **We believe that this might not have been very clear in the paper, and would like to clarify that we have trained and benchmarked optical flow performance in all pixels (including occluded) and showed SoTA performance.** We recognize that optical flow requires estimating occluded and out-of-view pixels. Unlike the 560-resolution model, **we trained the 980-resolution model on all pixels**, as noted in **lines 225-226**. **We evaluated it on all pixels** following standard protocol, as shown in **Table 4** (**“EPE[all]” and “1/3/5 px”** entries). The 980 model achieves state-of-the-art zero-shot performance on Sintel-Final and KITTI, with competitive results on Sintel-Clean (lines 260-262). Notably, it was bootstrapped from the 560 model using only 15 epochs of finetuning, demonstrating UFM's adaptability to occlusions for optical flow.\n    - **Q1.2: The paper's claim of unifying dense correspondence and flow is not well justified, because optical flow generally includes occlusion estimation, which is not addressed here.**\n        \n        While we evaluated UFM concurrently on covisible regions of optical flow and wide-baseline datasets and demonstrated SoTA, we agree that optical flow tasks, in the strict sense, include occluded and out-of-view pixels. We address this limitation by simply finetuning our model, as noted in our response to Q1.1.\n\n        \n        **We show our model can be easily finetuned to maintain our unification claim.** We finetuned the UFM560 checkpoint for 10 epochs and supervised on all pixels for the optical flow datasets and covisible pixels in the wide-baseline dataset. Specifically, we used the same number of pairs of optical flow data and 20% of wide-baseline data as specified in Table 1, and the same hyperparameter with a 10x smaller learning rate as training the 560 checkpoint (Section 3.4). We repeat benchmarks in Table 2 and 4:\n\n        \n        \n        **Repeated Table 2** (We omit OF baselines due to significantly worse performance)\n        \n        |Method|Eval Range|ETH3D EPE|1/2/5 px|DTU EPE|1/2/5 px|TA-WB EPE|1/2/5 px|Runtime|\n        |-|-|-|-|-|-|-|-|-|\n        RoMa|covisible|7.94|51/33/20|9.69|52/34/20|48|64/48/40|387.4|\n        UFM_560|covisible|2.64|47/24/9|5.56|58/34/13|12.87|54/32/17|42.9| \n        UFM_560-Refine |covisible|2.60|44/23/9|5.55|56/33/14|12.84|51/31/17|70.1|\n        UFM_560-FT|covisible|2.917|47/25/10|5.936|59/34/14|13.65|54/32/17|42.9|\n        MASt3R|MASt3R’s selection|1.31|33/12/**2**|2.23| 50/**21/5**|6.21| 55/23/**6**|2517.8| \n        UFM_560|MASt3R sel|1.34|32/12/3|2.30|49/24/6|6.19|42/20/7|41.0|\n        UFM_560-Refine|MASt3R sel|**1.29**|**29/11**/3|**2.18**|**43/21**/6|**6.13**|**39/18**/7|56.1|\n        UFM_560-FT|MASt3R sel|1.422|31/13/4|2.418|50/24/7|6.394|42/20/7|41.6|\n\n        **Repeated Table 4**: \n        \n        |Method | Sintel-C EPE[covis] | S-C EPE[all] | S-C 1/3/5 px | S-F EPE[covis] | S-F EPE[all] | S-F 1/3/5 px | KITTI F1 EPE[covis] | KITTI F1 EPE[all] | KITTI F1 % | Runtime | \n        |-|-|-|-|-|-|-|-|-|-|-|\n        SEA-RAFT | 0.65 | 1.47 | 11/5/**3** | 2.24 | 3.69 | **16**/9/7 | 2.36 | 4.21 | 15.5 | 14.7|\n        FlowFormer | 1.88 | 2.92 | 24/10/7 | 7.39 | 8.92 | 35/22/18 | 4.64 | 7.89 | 29.3 | 77.5 |\n        UniMatch | **0.60** | **1.2** | **10/4/3** | 1.73 | 2.76 | **16/8/6** | 2.43 | 4.66 | 17.7 | 30.0 | \n        RoMa|1.18|(2.913)|(16/9/7)|2.13|(3.99)|(25/14/11)|2.30|(10.98) | (19.6) | 390.3 |\n        UFM_560 | 0.79 | (2.04) | (17/7/4) | 1.44 | (2.93) | (20/10/7) | 1.87 | (9.15) | (19.0) | 44.0 | \n        UFM_560-Refine | 0.72 | (1.98) | (14/6/5) | **1.40** | (2.90) | (19/9/7) | **1.69** | (9.02) | (17.51) | 57.0 |\n        UFM_560-FT | 0.78 | 1.358 | 16/6/4 | 1.47 | **2.28** | 20/9/**6** | 1.96 | **4.14** | **14.8** | 45.9 | \n\n        Where result in () denotes the method is not trained in occluded and out-of-view regions.\n        \n        \n        **For dense wide-baseline, UFM560_FT still achieves 58% less EPE and 8.5x less runtime compared to the closest performing dense method, RoMa.** Despite evaluating on the pixels that MASt3R is confident about (it uses a cycle consistency check to filter its outputs, explained in lines 236-239), UFM560_FT still achieves comparable metrics while being 54x faster.\n        \n\n        **For optical flow, UFM achieves better covisible and all EPE and similar outlier rates in Sintel-Final and KITTI, and comparable performance in Sintel-Clean.**\n        Note that our **refinement module can improve this result** further. Thus, the above experiment shows UFM can be easily finetuned to concurrently solve wide-baseline matching and optical flow in the strict sense.\n        \n    - **Q1.3: The omission of learning in occluded regions reduces the significance of the work (as its functionality overlaps with MASt3R in most practical scenarios).** Explain why occlusion estimation is not considered. Please elaborate on the algorithm's performance in occluded regions.\n        **The question remains for wide-baseline** since we addressed it for optical flow in Q1.1 and Q1.2.\n\n    \n        Certainly, being able to predict correspondence in occluded regions broadens UFM's applications. However, estimating occluded regions is hard in wide-baseline datasets, and **it is not the norm for dense-wide baseline matching methods (e.g.,  DKM, RoMA, Dgc-net) to estimate occluded regions (as lines 89-91)**. We find that including occluded regions will decrease performance in covisible regions, as partially shown in Q1.2. **Still, including occluded regions would be an interesting future direction for UFM.**\n        \n        **We benchmark UFM and RoMa (the closest SoTA dense matcher) on occluded regions and show UFM still outperforms RoMa.** Note that neither method has been trained on occluded regions.\n\n\n        EPE | ETH3D | | DTU | | TA-WB | |\n        |-|-|-|-|-|-|-|\n        Range | covisible | occluded | covis | occ | covis | occ |\n        RoMa | 7.94 | 8.67 | 9.69 | 49.72 | 48.10 | 86.53 | \n        UFM-560 | 2.64 | 3.926 | 5.56 | **25.64** | 12.87 | **55.45** |\n        UFM-560 Refine | **2.60** | **3.86** | **5.55** | **25.64** | **12.84** | 55.52 |\n\n- **Q2: The architecture lacks substantial innovation. Technically, the method appears to be a combination of UniMatch and MASt3R.**\n    \n    We respectfully argue that UFM is **not merely a combination of UniMatch and MASt3R**, as it **distills only their most relevant ideas** and, more importantly, **introduces its innovations aimed at achieving simplicity, efficiency, and scalability**. We do acknowledge that UFM draws inspiration from prior architectures in lines 156 - 162.\n\n    **Base Module**: As shown in the architecture ablation in **Figure 5**, UFM outperforms representative architectures (including UniMatch) when trained on the same unified dataset, demonstrating better scalability.\n    \n    **Compared to MASt3R, UFM introduces key abilities not achievable by MASt3R’s architecture:**\n     - **1. Efficiency**: MASt3R performs global pixel-wise matching (complexity O(H^2W^2)). UFM avoids this by initializing with regressed pixel-level flow (in contrast to patch level for UniMatch) and performing optional local refinement (complexity O(HWP^2), with P=7), enabling efficient and high-res prediction.\n     - **2. Completeness**: MASt3R enforces one-to-one matching, which results in incomplete dense predictions or slower inference without its fast reciprocal matcher. UFM makes no such tradeoff.\n     - **3. Sub-pixel Matching**: MASt3R's proposed matcher predicts integer coordinates only, lacking potential for sub-pixel prediction (which is in contrast to UFM, where this is also architecturally possible with our refinement as discussed in Appendix D).\n\n    Additionally, UFM simplifies MASt3R's transformer: we remove RoPE (which we found not required), use global attention (for better acceleration from Flash Attention, as explained in lines 142-145), and use DINOv2 to ensure better semantic matching ability over MASt3R (please refer to our answer to Q1 of R1(B4yj)).\n    \n\n    **Compared to UniMatch**, UFM does not construct its flow from coarse-to-fine resolutions, **which we found suboptimal** for the unified training (Figure 5). To further support our argument, we observe a **22% increase in covisible EPE** when replacing the **regression DPT head of UFM** (patch level features decoded to pixel level fine flow) with the **coarse-to-fine matching head of UniMatch** (patch level coarse flow & features decoded to pixel level fine flow) when trained with the same setup of UFM560:\n    \n    EPE [covis]|Sintel-C|Sintel-F|KITTI|ETH3D|DTU|TA-WB|\n    |-|-|-|-|-|-|-|\n    Regression - UFM|**0.79**|**1.44**|**1.87**|**2.64**|5.56|**12.87**|\n    Matching - UniMatch|0.91|1.49|2.05|5.36|**5.47**|13.85|\n\n- **Q3: Please describe your open-sourcing plan.**\n\n    As stated in Checklist #5 (**line 540 - 552**), we will be releasing our training and inference code, dataset preparation and experimental evaluation scripts, pretrained model weights, as well as usage instructions after acceptance. **Practically, we plan to release model weights and inference code immediately after the review process and the rest within the camera-ready deadline.**"}, {"review_id": "QLq2BrZqYz", "reviewer": "Reviewer_RKdL", "summary": "This paper proposes a unified model for wide-baseline feature matching and optical flow, which are usually studied separately in previous works. Thanks to the unified formulation, the proposed model can be trained on combined datasets from both tasks, where the performance is better than separately trained models. Technically, the proposed model is mainly a large Transformer-based architecture, with a flow head to regress pixel displacements and a covisibility head to predict covisibilities across two input views. Experiments are conducted on wide-baseline feature matching and optical flow datasets, the proposed method achieves comparable or better results than previous specialized methods, but with a unified model.", "questions": "The key questions are listed in the Weaknesses.\n\nIn addition, I think another related work that is worth mentioning and discussing is: [Perceiver IO: A General Architecture for Structured Inputs & Outputs. ICLR 2022], which also tried to develop a unified model for different tasks and also reported the experiments on optical flow. \n\nIn general, I like the idea of this paper. However, I am currently not fully convinced by the experiments (detailed in Weaknesses). I am happy to raise my rating if these questions are well-addressed.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- The unified model is appealing since the two tasks of wide-baseline feature matching and optical flow indeed share a lot of similarities.\n- One benefit of the unified model is that it can be trained on the combined datasets. In Table 5, this paper shows that the combined-dataset training leads to mutual improvement on both tasks.\n- To train the unified models, this paper compiles 12 datasets, with detailed preprocessing steps thoroughly discussed. These efforts provide a valuable resource to the community.\n\n### Weaknesses\n\n- This paper argues about the difference with previous coarse-to-fine methods in several places (e.g., Line 9 and Line 30), but eventually the proposed model still has a coarse-to-fine structure (with an additional refinement stage). I think this paper would like to highlight the difference in the \"coarse\" stage (global vs. local), instead of the overall coarse-to-fine structure in previous methods. In addition, I think the argument on global attentions used in the model is not well-supported by experiments. In Figure 4, some comparisons with RoMa are shown, but there are many differences in implementation and training datasets between the proposed method and RoMa, which makes it not easy to understand where the performance gains come from. A more rigorous comparison could be replacing the global attentions used in the proposed model with local attentions and train with the same setup and then compare their performance. Such an experiment would be more convincing.\n- This paper uses a regression head to predict flow. However, many previous methods have shown the benefits of performing matching (for example, MASt3R, UniMatch, GMFlow, RoMa, etc.). It's unclear why such a design is adopted in this paper, and it would be better to justify the rationality and benefits of using a regression head instead of a matching head. \n- Although the results on the wide-baseline feature matching task look good, the performance on optical flow seems to have room for further improvement (in particular, Table 4, Sintel Clean). What are the potential reasons? A deep analysis would be beneficial. In addition, how does the model predict flow for all pixels since the models is only trained on covisible pixels? In addition, I am also curious about the performance of the proposed method on the closed test sets (for example, Sintel and KITTI online benchmarks). It would be great if the authors could report these results.", "was_revised": true, "final_justification": "My initial concerns are well-addressed, and thus I vote for accept.", "rebuttal": "Thank you for your thoughtful and constructive comments. We are encouraged that you find the **unified model appealing**, and it **achieves comparable or better results than previous specialized methods**. We also appreciate your recognition of our **detailed data processing** as a **valuable contribution** to the community.\n\nWe acknowledge that our writing has caused confusion regarding the term “coarse-to-fine” and address your concern that some architectural claims lack sufficient support from a more rigorously controlled experiment. We also investigated two reasons for your question on further improving flow performance, supported by evidence of improvement. \n\n- **Q1: Justification for UFM’s architecture**\n    - **Q1.1: Many prior methods demonstrate the advantages of using a matching head. The choice of a regression head in this work lacks clear justification.**\n    \n        Matching can be performed either at the pixel level or the patch level.\n        \n        **We do leverage pixel-level matching for precision.** As noted in lines 153–156, while regression offers robustness, it lacks precision. Therefore, we refine the regressed flow via local, pixel-wise, feature matching. Unlike MASt3R, which performs global matching (O(H^2W^2)), we restrict matching to a local window (O(HWP^2), P=7) initialized by the regressed flow, significantly improving efficiency while maintaining competitive accuracy.\n\n        \n        **We chose not to perform patch-level matching based on empirical findings; as shown in Figure 5, UFM’s architecture better fits our combined training set compared to UniMatch and RoMa.**\n        \n       To further support our argument, we observe a **22% increase in covisible EPE** when replacing the **regression DPT head of UFM** (patch level features decoded to pixel level fine flow) with the **coarse-to-fine matching head of UniMatch** (patch level coarse flow & features decoded to pixel level fine flow) when trained with the same setup of UFM560:\n    \n        EPE [covis]|Sintel-C|Sintel-F|KITTI|ETH3D|DTU|TA-WB|\n        |-|-|-|-|-|-|-|\n        Regression - UFM|**0.79**|**1.44**|**1.87**|**2.64**|5.56|**12.87**|\n        Matching - UniMatch|0.91|1.49|2.05|5.36|**5.47**|13.85|\n\n    - **Q1.2: This paper argues about the difference with previous coarse-to-fine methods in several places, but eventually the proposed model still has a coarse-to-fine structure (with an additional refinement stage). I think this paper would like to highlight the difference in the “coarse” stage (global vs. local), instead of the overall coarse-to-fine structure in previous methods.**\n\n        We may differ in our definition of “coarse-to-fine,” and we admit the paper could define it more clearly. “Coarse-to-fine” can imply two things: (1) the predicted flow is upsampled from a coarse to a fine resolution, or (2) a rough prediction is progressively refined for higher precision. We think you are referring to (2). **While prior methods often combine both, our approach uses (2) but not (1).** We will clarify with the above description in the revision.\n        \n    - **Q1.3: The argument on global attentions used in the model is not well-supported by experiments. In Figure 4, some comparisons with RoMa are shown, but there are many differences in implementation and training datasets between the proposed method and RoMa, which makes it not easy to understand where the performance gains come from.**\n\n        This is a fair point. We agree that the comparison with RoMa in Figure 4 does not conclusively isolate the effects of its coarse-to-fine strategy (definition (1) in Q1.2) due to differences in implementation and training setup. \n        \n        **To better support our claim, we refer to the additional ablation study described in Q1.1, where we directly compare regression to a coarse-to-fine head under the same training conditions.** We acknowledge that the original justification was primarily based on visual intuition and theoretical considerations of the RoMa architecture. While this motivation remains valid, the new empirical results in Q1.1 now offer stronger support for our design choice. **We will revise our original justification with this evidence to better support it.**\n        \n- **Q2: Analysis for further improving Optical Flow performance (especially, Sintel Clean).**\n\n    We observed two directions for improvement:\n    \n    - **We show that finetuning the backbone during refinement training improves the precision of small motions, yielding a 10% gain on Sintel Clean.** Previously, we trained UFM980-Refine in two stages: first, training the flow head, then freezing the backbone to train the refinement. We now train both jointly for 15 epochs (matching the first stage of the original scheme) and report the updated results in Table 4:\n\n        | |Sintel-Clean| | |Sintel-Final| | |KITTI| | |\n        |-|-|-|-|-|-|-|-|-|-|\n        | | EPE[covis] | EPE[all] | 1/3/5 px | EPE[covis] | EPE[all] | 1/3/5 px |EPE[covis]|EPE[all]|F1 %|\n        UFM_980-Refine|0.56|1.15|10.2/4.6/3.3|**1.25**|2.01|15.0/7.2/**5.1**|**2.05**|2.96|11|\n        UFM-980-Refine[backbone FT]|**0.5** |**1.02**|**9.1/4.1/2.8**|**1.25**|**1.98**|**14.0/7.0/5.1**|2.176|**2.9**|**10.366**|\n        \n        As shown above, this joint training yields improvement on most EPE and inliers, with the most improvement on easier datasets like Sintel Clean. Hence, we conclude that training the refinement stage jointly with the backbone enables the model to learn more detailed matching features.\n\n    - Our method is **biased towards static objects**. According to KITTI performance in Q2.1, UFM is significantly better in the background than in the foreground objects, which are mainly dynamic. We believe this is a data issue, as less than 20% of our training data is dynamic. Future work may include more sceneflow datasets, such as ParallelDomain, to balance the distribution. We will update the limitations to include this analysis.\n\n\n    - **Q2.1: Performance in Sintel / Kitti test sets.**\n\n        **UFM outperforms UniMatch on Sintel and KITTI test sets under zero-shot settings.** We benchmarked the UFM-980 refine model and UniMatch reported in Table 4 on the Sintel and KITTI benchmark test sets. **Neither method was trained on Sintel or KITTI**.\n        \n        **Sintel Test set result**:\n\n        ||EPE all|EPE Matched|EPE unmatched|d0-10|d10-60|d60-140|s0-10|s10-40|s40+|\n        |-|-|-|-|-|-|-|-|-|-|\n        |Sintel-Final||||||||||\n        |UFM_980-Refine|**3.282**|**1.746**|**15.802**|**3.28**|**1.273**|**1.205**|**0.689**|**2.068**|**19.067**|\n        |UniMatch-CT|4.139|1.965|21.856|3.765|1.586|1.278|0.888|2.426|24.401|\n        |Sintel-Clean||||||||||\n        |UFM_980-Refine|**1.682**|0.718|**9.518**|1.535|0.666|0.425|0.483|1.231|**8.705**|\n        |UniMatch-CT|1.802|**0.571**|11.845|**1.175**|**0.453**|**0.395**|**0.367**|**0.932**|11.032|\n        \n        From the Sintel result, UFM outperforms UniMatch in the overall metric and most sub-items, except for sub-items on Sintel Clean, which we demonstrated in Q2 can be improved through training the refinement in parallel with the backbone.\n    \n        **KITTI Test set result**\n        \n        ||Range|F1-bg|F1-fg|F1-all|\n        |-|-|-|-|-|\n        UFM_980-Refine|All|**8.67**|22.08|**10.9**|\n        UniMatch-CT|All|18.01|**17.27**|17.89\n        UFM_980-Refine|Not occluded|**4.93**|19.6|**7.59**\n        UniMatch-CT|Not occluded|9.75|**14.61**|10.63\n        \n        From the KITTI result, UFM outperforms UniMatch in the overall metric and background objects, but not foreground objects, which are mostly dynamic. As discussed in Q2, we believe adding more dynamic data will improve its performance.\n\n- **Q3: Discussing PerceiverIO**\n\n    Thank you for the reference - we will cite it in the discussion on scaling correspondence.\n\n- **Q4: Typo**\n\n    Thank you for spotting this. We will correct it."}, {"review_id": "2wcuXYyXeM", "reviewer": "Reviewer_NLte", "summary": "The authors propose a model and set of training datasets for the task of dense correspondences, unifying optical flow and dense feature matching.\nThe coarse part of the model is a two-view transformer built on top of DINOv2-L features, which is trained to output a corrrespondence field and a covisibility mask for each token position. If needed, these correspondences can be refined by a lightweight refiner on top.\nOn the data-side, the authors compile a set of datasets similar to that of MASt3R, extracting correspondence either from depthmaps or from optical flow.\nThe experiments show that the method, UFM, performs competivetely with specialized models, while being significantly simpler and faster.", "questions": "1. I do think I entirely understood the proposed loss. Is the only supervision for the flow the robust Charbonnier loss? In RoMa this was used on top of a classification type loss. Would only using this loss not cause the network to get very low gradients when matching fails? I would be curious to hear the authors explain the thought process a bit more.\n\n2. W.r.t. my comments regarding comparisons to MASt3R, I would be interested in hearing an argument from the authors why their full method is needed, when MASt3R already seems to perform well (and possibly outperforms the proposed method). For example, why not simply train a small regression head on top of the MASt3R decoder?\n\n3. W.r.t to MD1500, I am interested in hearing why the authors have not included this very standard benchmark.\n\nCurrently my rating is around a borderline, with my main concern being MASt3R comparisons. Given a convincing rebuttal to the questions above I would be willing to raise my rating to accept.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**S1 - Simple model, strong results:** The method outperforms matching methods such as RoMa and optical flow methods such as FlowFormer. While technically not SotA on optical flow tasks, as later methods outperform FlowFormer, I believe the performance is satisfactory. I was also positively surprised to see that the authors made the effort to compare their approach to these previous methods when trained on the same data (see Figure 5), which validates their modeling choice (and also their 2nd contribution).\n\n**S2 - Unifying flow and wide-baseline matching:** For downstream tasks where these types of models are used (both in academia and industry), it is incredibly beneficial to have a model that is not overly sensitive to the specifics of the setup. While RoMa is robust to large temporal changes, it is not particularly accurate for thin and small objects. The proposed model takes a step towards a setting where one can simply input any image pair and expect to get correct correspondences in a timely manner, which I think is an important contribution.\n\n**W1 - Unclear robustness to temporal/etc variations in images:** As the authors mention on l.285-l.292, finetuning the DINOv2 backbone tends to reduce generalization to more semantic types of matching. I interpret this to mean that performance on more difficult feature matching benchmarks such as MegaDepth-1500, and WxBS, decrease due to this choice. Unfortunately the authors do not provide results on these benchmarks, particularly MD1500 would be relevant as it is the standard feature matching benchmark.\n\n**W2 - Unclear improvement in comparison to MASt3R:** Looking at Table 2 and Table 3, MASt3R generally outperforms the proposed method (although MASt3R can be improved by using the proposed refiner). While MASt3R is significantly slower, I would assume that most of the runtime comes from matching its features in a brute-force manner. As this matcher is very basic, I'm not entirely convinced that the claimed 60x speedup is representative.", "was_revised": true, "final_justification": "The rebuttal clarified the loss, and the added ablation is a nice validation of the approach. W.r.t. comparisons to MASt3R, I think that while the comparisons w.r.t. time are not entirely fair, building dense matching efficiently on top of MASt3R would require a substantial rewrite and is perhaps not a fair comparison from my part. The performance of the proposed method is additionally significantly higher than MASt3R on the WxBS benchmark, indicating that there is a clear usecase, beyond just more dense predicitions.\n\nI also read the discussions with the other reviewers, and believe that their concerns were adequately addressed.\n\nI have thus increased my rating from 4->5.", "rebuttal": "Thank you for your thoughtful and supportive comments. We are encouraged that you find UFM to be **simple, efficient, and delivering strong results**. We value your recognition of **our effort to compare prior methods on the same data**, which, as you noted, **substantiates our architectural choice**. We value your endorsement that **UFM presents an important step towards application** for its robustness to setup variations.\n\nWe added WxBS results to show semantic matching capability. We believe that comparison to MASt3R might not have been very clear in the paper and would like to clarify. We further explained our motivation for the robust loss.\n\n- **Q1: Unclear robustness to temporal/semantic variations in images. Why not include the MegaDepth 1500 benchmark?**\n\n    **Because we train exhaustively on the MegaDepth dataset. We add the WxBS Experiment and show UFM is reasonably robust for extreme conditions.**\n    \n    We evaluated our model under extreme conditions using the WxBS dataset, which includes low-light scenes, multiple visual spectrums (e.g., IR to RGB), day-night transitions, seasonal changes, and scale changes. We report performance on fundamental matrix estimation using PCK, i.e., the percentage of pre-labeled keypoints consistent with the prediction under an epipolar error threshold (higher is better, 10px is commonly used).\n    \n    | Thresholds | 3px  | 5px  | 10px | 15px | 19px |\n    |------------|------|------|------|------|------|\n    | RootSIFT   | 0.06 | 0.08 | 0.10 | 0.13 | 0.16 |\n    | Superpoint + SuperGlue | 0.26 | 0.33 | 0.46 | 0.51 | 0.55 |\n    | LoFTR      | 0.48 | 0.57 | 0.67 | 0.71 | 0.73 |\n    | RoMa       | **0.74** | **0.85** | **0.92** | **0.96** | **0.96**|\n    | MASt3R     | 0.32 | 0.42 | 0.56 | 0.62 | 0.65 |\n    | UFM_560    | 0.49 | 0.63 | 0.74 | 0.77 | 0.78 | \n    | UFM_560-Refine | 0.49 | 0.62|  0.75 | 0.79 | 0.81 | \n    \n    As already noted in Section 5 (Limitations), compared to RoMA (the best performing one, where DINOv2 is frozen), finetuning the DINOv2 encoder in UFM improves precision but reduces semantic matching capabilities (as also supported by our supplementary material analysis, Section E). Despite this, UFM ranks second on the WxBS benchmark with a 0.74 PCK@10, showing strong performance and outperforming MASt3R under extreme conditions.\n\n- **Q2: Comparison to MASt3R is unclear**\n    - **Q2.1: Looking at Table 2 and Table 3, MASt3R generally outperforms the proposed method (in precision), although MASt3R can be improved by using the proposed refiner.**\n\n        **We believe that you are only comparing our base model to MASt3R**, since the refinement model holds **8/12** advantageous metrics in Table 2, and at least **5/9** metrics in Table 3. \n\n        **We wish to highlight an important detail that biases the comparison in Table 2 in favor of MASt3R - UFM is actually achieving comparable precision on MASt3R's most confident pixels, a more demanding evaluation setup for UFM.**\n\n        As explained in lines 236-239, MASt3R is a sparse method because its feature matcher can only output correspondence that **passed its cycle-consistency check**. **By evaluating UFM on MASt3R’s set of output pixels, we give MASt3R an advantage in selecting its most confident matches**. Statistically, **MASt3R skips 2.1%, 2.0%, and 10% of the samples almost entirely** (defined as reporting < 10% of covisible pixels) in DTU, ETH3D, and TA-WB. Thus, while the numbers of the base model in Table 2 may only appear slightly inferior, UFM is achieving comparable precision.\n\n        We agree that incorporating the proposed refiner, specifically the U-Net and sub-pixel weighted average refinement, would enhance MASt3R’s performance. However, we would like to emphasize that **these components are integral to UFM’s design, specifically introduced to improve the precision for inliers, as motivated in lines 153–154.** This function is further supported by the significant reduction in 1px outlier rate shown in Table 2 when comparing UFM with and without refinement.\n\n    - **Q2.2: The claimed 60x speedup (in table 2) is not representative, as most of MASt3R’s runtime is on the matching process, and its matcher is very basic.**\n\n        **While we agree that the 60x speedup is not fully accurate, UFM still offers a significant speed advantage in dense applications.** Indeed, we should use the runtime of our refinement model for comparable performance in Table 3. While we acknowledge that a significant portion of MASt3R’s runtime comes from the feature matching stage, we do not characterize the matcher as “very basic,” since one of MASt3R’s core contributions is its fast reciprocal matcher. That said, we also found the follow-up paper *Speedy MASt3R*, which reports a 54% runtime reduction through various optimizations. **Taking these updates into account, we would revise the reported speedup to approximately 24×, which is still significant.** \n        \n    - **Q2.3: Why UFM is needed, when MASt3R already seems to perform well.**\n        \n        **UFM holds the following advantages in applications:**\n        1. **Completeness**: UFM outputs dense correspondences, while MASt3R’s output is usually incomplete. Hence, UFM can be applied to tasks like image warping, while MASt3R cannot.\n        2. **Efficiency**: When dense outputs are required, UFM has a significant speed advantage (as revised in Q2.2) that makes it favorable to real-time applications like SLAM.\n        3. **Semantically more robust**, as shown in Q1. \n        4. **Sub-pixel Matching**: MASt3R's proposed matcher predicts integer coordinates only, lacking sub-pixel prediction. UFM and its refinement provides sub-pixel predictions.\n        5. UFM also provides an **explicit covisibility estimate**, which is useful for tasks such as loop-closure.\n\n    - **Q2.4: Why not simply train a small regression head on top of the MASt3R decoder?**\n\n        Firstly, we aim to modify the backbone to **incorporate advantageous components for correspondence**.\n        \n        1. We replaced CroCo with DINOv2 **for its semantic matching ability**, justified by Q1. Finetuning MASt3R may not bring robustness for semantic matching as our dataset contains mostly geometric variations.\n        2. We have **simplified MASt3R’s transformer by removing RoPE** (which we found not required).\n        3. We **accelerated it** by replacing cross-attention with global attention, enabling **better performance with Flash Attention** at long sequence lengths without increasing theoretical compute (see lines 142–145).\n\n        More fundamentally, we want to **simplify and reduce the time to train a correspondence foundation model from scratch**. UFM training takes 4 days at 8 H100, while MASt3R’s training depends on 6.5 days for DUSt3R and 11 days for MASt3R under 8 A100 GPUs, which is still much longer after considering the hardware differences. \n\n\n- **Q3: Thought process on the robust loss.** \n    \n   Our goal is to **increase the precision of the inliers instead of having mediocre predictions for all pixels**. For tasks like pose estimation, which rely on a highly accurate subset of all the correspondences, we found it correlates best with fine inlier rates instead of EPEs. \n\n    \n    **We have confirmed this improvement by finetuning a previously trained checkpoint** (not reported in paper) with robust vs. L1 loss under the same setting. \n    \n    ||DTU EPE | 1/2/5 px ouliers | ETH3D EPE | 1/2/5 px outliers | ScanNet++ EPE | 1/2/5 px outliers|\n    |-|-|-|-|-|-|-|\n    |L1|6.76|72/46/17|**3.63**|55/30/11|5.31|70/43/17|\n    |Robust|**6.17**|**64/38/15**|3.99|**46/26/10**|**4.93**|**66/39/16**|\n    \n    As shown, the robust loss improves both inlier rates and even most EPE compared to the L1 loss. We attribute this to two factors: (1) the dataset includes unsolvable pairs that are better ignored by the robust loss, and (2) the robust loss promotes easy-to-hard learning.\n    \n    - **Q3.1: Is the only supervision for the flow the robust Charbonnier loss?**\n\n        Yes.\n\n    - **Q3.2: Would only using this loss not cause the network to get very low gradients when matching fails?**\n\n        **While the gradients are comparatively smaller for large outliers than for inliers, they are not entirely negligible, even for the largest outliers.** The robust Charbonnier loss (with c=0.24,a=0.5) has gradients that peak at 3.16 near inliers (e.g., x=0.4) and remain non-zero (e.g., 0.12 at x=500) for large outliers. This ensures strong updates for inliers while still allowing meaningful correction for learnable outliers. In early experiments, we tried training from scratch with a L1 vs robust loss (under a slightly different setting) and observed no significant difference in convergence speed."}]

Author rebuttal:

Meta-review: The paper introduces a dense correspondence network based on Dinov2 and Transformer. In particular, it has a local attention-based refinement module to further enhance the results. Although the contributions on the network architecture is not significant, the network offers superior performance by extensive training over a wide range of datasets. The resulting baseline can benefit many downstream tasks. The reviewers agreed to accept after the rebuttal session. The AC agrees with the decision.
Decision: Accept (poster)