Evidential Learning-based Certainty Estimation for Robust Dense Feature Matching
Authors: Lile Cai
Chuan-Sheng Foo
Xun Xu
ZAIWANG GU
Jun Cheng
Xulei Yang
Venue: ICLR 2025 Poster (2025)
Keywords: Evidential Deep Learning
Dense Feature Matching
Pose Estimation
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: An evidential deep learning framework to improve the robustness of dense feature matchers

Abstract: Dense feature matching methods aim to estimate a dense correspondence field between images. Inaccurate correspondence can occur due to the presence of unmatchable region, necessitating the need for certainty measurement. This is typically addressed by training a binary classifier to decide whether each predicted correspondence is reliable. However, deep neural network-based classifiers can be vulnerable to image corruptions or perturbations, making it difficult to obtain reliable matching pairs in corrupted scenario. In this work, we propose an evidential deep learning framework to enhance the robustness of dense matching against corruptions. We modify the certainty prediction branch in dense matching models to generate appropriate belief masses and compute the certainty score by taking expectation over the resulting Dirichlet distribution. We evaluate our method on a wide range of benchmarks and show that our method leads to improved robustness against common corruptions and adversarial attacks, achieving up to 10.1\% improvement under severe corruptions.

Published as a conference paper at ICLR 2025
EVIDENTIAL LEARNING-BASED CERTAINTY ESTIMA- TION FOR ROBUST DENSE FEATURE MATCHING
Lile Cai Chuan-Sheng Foo Xun Xu Zaiwang Gu Jun Cheng
Xulei Yang
Institute for Infocomm Research (FR), A STAR, Singapore
(caill, foo chuan.sheng xu xun, gu-zai wang, cheng- jun, yang xule jei2r wee.eeeerrrrsareeee aas....star... edu.sg
ABSTRACT
Dense feature matching methods aim to estimate a dense correspondence field between images. Inaccurate correspondence matchable region, necessitating the need for certainty measurement. This typ- ically addressed by training a binary classifier to decide whether each predicted correspondence is reliable. However, deep neural network-based classifiers can be vulnerable to image corruptions or perturbations, making difficult obtain reliable matching pairs in corrupted scenario. In this work, we propose an eviden- tial deep learning framework to enhance the robustness of dense matching against corruptions. We modify the certainty prediction branch in dense matching models to generate appropriate belief masses and compute the certainty score by taking expectation over the resulting Dirichlet distribution. We evaluate our method on a wide range of benchmarks and show that our method leads to improved robust- ness against common corruptions and adversarial attacks, achieving up to 10.1% improvement under severe corruptions.
occur due to the presence of un-
INTRODUCTION
Feature matching is an important task in computer vision. aims find a of matching pixel pairs between two images, based on which two-view geometry, e.g., homography and relative cam- era pose, can be derived. There are three paradigms
perform feature matching: detector-based (Lowe, 2004 De'Tone et al. 2018, Sarlin et al. 2020). detector-free (Sun et al. 2021) and dense matching (Truong et al., 2023 Edstedt et al.) 2023 2024). It has been demonstrated that dense matching methods obtain superior accuracy compared to others on a wide range of geometry esti-
mation benchmarks [Edstedt al. 2023 2024). We focus on the dense matching paradigm this paper.
Dense feature matching estimates a 2D dense correspondence field for the input image pair. typically works in coarse-to-fin manner, where matches first predicted at coarse scale and subsequently refined at finer scales. As cannot be guaranteed that every pixel in the two images matchable, it is necessary measure the certainty for each predicted correspondence and only use reliable matches for downstream geometry estimation. This is usually formulated as binary classification problem, where pixels deemed have correspondence in the other image (based on ground truth geometry information) are classified into while unmatchable pixels are classified into 0 (Melekhov et al., 22099 Edstedt et al. 2023 2024). The classification probability for class is thus indicating matchability and used as certainty score. When applied to downstream geometry estimation tasks, a balanced sampling step (Edstedt et al., (023) adopted to sample a sparse set of matches from the dense matching results, where matches with larger certainty score have higher chance be selected.
State-of -frt (SotA) dense feature matching methods are built on deep neural networks, which have been shown to suffer severe performance loss on images under adversarial attacks {Goodfel- low et al. 2014) common corruptions (Hendrycks & Dietterich 2019 Michaelis et al. 2019). Moreover, 3D datasets used to train feature matching models are usually much smaller in size than 2D datasets, which may limit the model robustness to testing data under distribution shift. Previous benchmarking datasets (Mishkin et al. 2015) Howard et al., 2022) do not consider images under common corruptions adversarial attacks, both of which are relevant the former happen in

---

Published as a conference paper at ICLR 2025
real deployment environment, and the latter provides worst-case analysis of model robustness. Our work fills this gap. As shown in Fig. [ the SotA dense matcher RoMa fails to provide reliable certainty estimation in matchable regions when the input images are corruрeed by Gaussian noise or adversarial attacks, resulting in large pose estimation emror. In particular, RoMa tends to pro- duce very low certainty value (e.g., below 0.05) for matchable regions corrupted images, i.e., the model over-confidently classifies the pixels as unmatchable. The topic of improving the robustness of feature matching methods under image corruptions remains largely unexplored the literature.
RoMa
Ours
Images under Gaussian noise corruption
Pose estimation error: 54.82*
Pose estimation error: 3.16
Images under adversarial attack
error:
error:
Figure SotA dense matcher RoMa fails to provide reliable certainty estimation in matchable regions (indicated by red circles) when the input images are corrupted by Gaussian noise or adver- sarial attacks, resulting large pose estimation error. Our method is able to obtain reliable feature matches under image corruptions and adversarial attacks. The matching results are visualized by taking the pixel values from the other image using the estimated correspondences, and weighting by the estimated certainty.
Having identified unreliable certainty estimation, or over-confidence, as a bottleneck to robust fea- ture matching. we hypothesize that improved uncertainty estimation will lead to improved feature matching performance. To this end, we explore evidential deep learning (EDL) for certainty esti- mation in dense feature matching methods. Instead of providing direct pointwise estimation of probability, EDL models sccond-order probabilities under the framework of subjective logic the- ory (Josang 2016). It has been shown provide improved uncertainty estimation for detecting out-of fit-of--ditributiinn samples and mitigate the over-confidence problem (Sensoy et al.) 2018).
Our contributions can be summarized as below:
We propose an evidential learning framework specifically designed for certainty estimation in dense feature matching. To the best of our knowledge, this the first application of EDL in feature matching tasks.
We propose evaluate the robustness of feature matching methods under common image corruptions and adversarial attacks, which has not been studied in previous work.
We evaluate our method on both corrupted data and clean data, and show that our method leads to improved robustness against common corruptions and adversarial attacks without sacrificing the performance on clean data.
RELATED WORK
2.1
FEATURE MATCHING
Feature matching is long-standing task in computer vision. The detector-based approach first performs sparse keypoint detection and descriptor extraction, followed by mutual nearest neigh- bour matching (Lowel 2004 DeTone al. 2018) learning-based matching (Sarlin et al. 2020). Detector- free approach removes the keypoint detection step and calculates the pairwise similarity score uniformly over the image grid. Matches are then extracted from the similarity matrix using
2

---

Published as a conference paper at ICLR 2025
netual- neighbour optimal transport (Sun et al. 2021). The dense matching approach (Trong et al., 2023] Edstedt et al. 2023 2024) estimates a 2D flow field, aiming to find all match-
able pixel pairs between the two images.
2.2 CERTAINTY ESTIMATION IN DENSE MATCHING
The flow field predicted by dense matching methods may come with inaccurate correspondences in unmatchable regions, necessitating a measure of certainty for each predicted correspondence. DGC-Net (Melekhov et al., (019) formulates as a binary classification problem to identify out- of-view pixels. DKM (Edstedt et al., 2023) and RoMa (Edstedt et al. 2024) also train binary classifier where pixels corresponding to the same 3D points are classified as reliable matches. PDC- Net (Truong al.) 2023) instead formulates dense warping estimation in probabilistic framework, where prediction certainty is reflected by the variance of the predicted distribution.
2.3 EVIDENTIAL DEEP LEARNING IN COMPUTER VISION
Since its inception by SSnnsyy et al. (2018), evidential deep learning has been investigated on wide range of computer vision tasks (Gao al. 2024). it is originally formulated for classification problem, its application to image classification task is straightforward. It has been shown provide improved uncertainty estimation and detect out-of-distribution samples better than traditional neural networks (Sensoy et al. 2020 Hu et al.] 2021). For object detection task, evidential uncertainty has been used for mining pseudo-unken unknown objects (Su et al., 2023). It has also been explored in segmentation task for uncertainty- -uudded rectification (Shi al. 2024
2.4 ROBUSTNESS ENHANCEMENTS
Most works on enhancing robustness of deep learning models have focused on the robustness to adversarial examples (Goodfellow et al.) 2014). Adversarial training (Madry) 2017 Shafahi et al.) 2019), in which adversarial examples are injected into training data, has been shown withstand strong attacks. Instead of focusing on adversarial robustness, Hendryykks & Dietterich (2019) pro- posed benchmarks to evaluate a classifier's ' robustness to 15 types of common corruptions, and pro- posed to enhance corruption robustness by histogram equalization and more sophisticated network architectures. MMichaelis et al. (2019) extended (Hendrycks & Dietterich, (209)) for benchmarking robustness in object detection and showed that stylizing training data leads to improved robustness.
METHOD
The system diagram of the proposed method is presented in Fig. 2 SotA dense matchers (Edstedt al. 2023 2024) typically consist global matcher that predicts warp at coarse scale, and se- ries of warp retiners that refine the warp iteratively at finer scales. A certainty map is predicted along with the warp to indicate reliable matches. Previously, the certainty map is learnt by minimizing the binary rross-entropy (BCE) loss. In this work, we propose deep evidential learning framework to learn the certainty map. In the following, we will first briefly describe the existing loss functions for training dense matchers, followed by details on how we incorporate evidential learning into dense matchers to train a more robust model.
Globel
Coorse Warp
Werp Retiners
Dense Worp
Balanced Sampling
Geometry Eatimation
Features
Matcher
Certainty Map
Evidential
BCE Loss
Previous Methods
Proposed Method
Figure 2: System diagram of the proposed method.
3

---

Published as a conference paper at ICLR 2025
3.1 RECAP OF DENSE MATCHERS
Given two images I and 1 dense matchers predict a dense warp by first performing global match- ing on coarse-scale features and then refining the estimated warp on fine-scale features. Let X denote the pixels in J X E X the subset of pixels having reliable matches in 1 and yst indicates the corresponding coordinates in 1 for I E X During training, the model is learned to minimize the following loss:
C Leaarse + Lâne-
(1)
The coarse-scale prediction performs global matching, which predicts the correspondence for each pixel in I and a certainty score for each correspondence. The coarse-scale loss is computed as:
Louars
Lgm((((z), x + #XXØ
Σ
LBCCE ZEXA
Σ
|
where gmn(x) is the predicted correspondence at coarse scale, fym is global matching loss, which can be Ey loss (Edstedt et al. 2023) or cross-entropy loss (Edstedt et al.) (024) depending on the formulation, LBCE is the binary cross entropy loss, Peonf is the certainty score obtained by applying sigmoid function on logits, and À = 0.01 is a balanced term.
The fine-scale prediction estimates residual offset for the warp and certainty score. The resid- uals are then added to the original estimation and the fine-scale loss is computed on the refined estimation:
Line
Σ T
(reg(r(()), x 2 + [X
Σ (BCE(Pon(())), TEXA
(3)
X
where rw(x) is the refined warping coordinates, and Greg is the regression loss between the pre- dicted warp and ground truth warp. The fine-scale refinement is typically performed iteratively multiple feature scales, and Line is computed for each scale and summed up to form the final loss.
3.2 CERTAINTY ESTIMATION BY EVIDENTIAL DEEP LEARNING
The warp predicted by dense matching methods (Edstedt al. 2024 2023 may contain inaccurate correspondences. It is necessary to estimate the certainty of each predicted correspondence and use those confident ones for downstream geometric matching tasks. In previous methods (Edstedt et al.) 2024 2023), certainty estimation is modeled as binary classification problem, where pixels having reliable matches 1 (i.e., pixels whose projected depth to 1 are consistent with the ground truth depth) assigned label 1, and pixels not having reliable matches in I are assigned label 0. The binary classifier implemented as neural networks and trained by minimizing the cross entropy loss LBCE as defined in Eq. 2 and Eq. (3).
In this work, we instead employ evidential deep learning for certainty estimation. Evidential learn- ing considers classification task as forming a subjective opinion (Josang) 2016) over the class label space. A subjective opinion can be formalized as an ordered triplet T (b, u, r), where b rep- resents a belief mass distribution over the K categories, u the uncertainty mass and a base rate (prior distribution) (Gao et al.) (002)). The subjective logic theory establishes a bijection between subjective opinion and Dirichlet distribution D(p) where the concentration parameters a can be obtained from the triplet as:
a bWo /u + rW.
where Wo is positive prior weight.
EDL employs deep neural networks to generate the subjective opinion. More specifically, the net- work is trained to generate a non-negative evidence vector RK with each element e; represent- ing the amount of evidence supporting the claim that "the input sample belongs to the i-th category". The belief mass and uncertainty mass in the triplet can be computed from the evidence vector as:
E;
Wo
b;
Σ EjEjK] Bj + Wo
u EjE[K
+ Wo
Substituting b, and u into Eq. gives a = e + rW,, Sensoy et al. (018)) set the base rate r to be uniform distribution, ie., 1/K, and Wo to be the class number K, which further simplifies
4

---

Published as a conference paper at ICLR 2025
Eq. 4 and Eq. (5) as:
6, = C,
K
a a=
Ej{[K] aj
u
2jE[K] 9,
(6)
By training the model to estimate the parameters of the evidential distribution, EDL represents the prediction of the neural network as a probability density function (PDF), rather than the point estimate of probability. The concentration parameters determines the spread of the PDF: the larger the value of a, the more concentrated the Dirichlet distribution, and the smaller the uncertainty (Eq. (6)). By modeling second-order probabilities and uncertainty, EDL is able to mitigate the over- confidence issue neural network models and provide improved uncertainty estimation (Sensoy al. 2018).
We modify the certainty prediction branch of dense matcher to output a two-dimensional evidence vector for each pixel X-,, with representing the amount of evidence supporting the claim that "the predicted correspondence is reliable" and e2 the amount evidence supporting the claim vece-versa. The Dirichlet parameters can then be obtained by Eq. (6). We replace the cross entropy loss LBC used in Eq. (2) and Eq. (3) with the squared loss integrated over the Dirichlet distribution and Kullback-Leibler (KL) divergence term for regularization:
(x) Ep--(paa(=)) (1) pll + KLID(pla<=)) < I,...,, >)],
where y the one-hot vector indicating the ground truth class, D(pla) denotes the Dirichlet dis- tribution of class probabilities parameterized by a = + 1, and ā y + (1 y) ② a is the Dirichlet parameters after removing the evidence of ground truth class. The KL divergence term aims shrink the evidence of non-ground truth class to zero.
With the predicted Dirichlet distribution of class probabilities, the expected probability for each class is obtained by computing the mean over the Dirichlet distribution (Sensoy et al. 2018):
a(x) S
p xr)
(8)
where S E (x(z)) and K 2. The expected probability for class then used as certainty score indicate the reliability of each predicted correspondence.
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
Model Architecture We use the SotA dense matcher RoMa (Edstedt et al.) 2224) as our baseline. We modify the certainty prediction branch of RoMa to output 2-dimensional evidence vector for each pixel, and replace LBCE used in Eq. (2) and Eq. 3 with the evidential loss defined in Eq. 7 to train the network to accumulate evidence.
Training Setup We follow the training setup RoMa (Edstedt et al. 2024), where the batch size 32, encoder learning is 10-1, decoder learning is 10-5 and the model is trained for 250,000 iterations. The training image resolution is 560 x 560. For outdoor geometry estimation, the model trained on the MegaDepth dataset (Li & Snavely (018)) using the same training and test splits as in RoMa. For indoor geometry estimation, the model is trained on a combination of MegaDepth and ScanNet datasets (Dai et al. 2017a) similar fashion as previous works (Edsted)
et al. 2024
Sun et al. 2021).
Test Setup We use the same test setting as RoMa ensure fair comparison. For balanced sampling, we use score threshold 0.05 and sample 5,000 matches, the same as DKM (Edstedt et al. (023) and RoMa. When benchmarking with other methods, we use the released outdoor/indoor model for outdoor/indoor geometry estimation and follow the default settings.
4.2 BENCHMARK DATASETS
MegaDepth- 1500 Sun et al, 202T) is a popular outdoor benchmark consisting of 1500 image pairs from 0015 and 0022. To evaluate each method's robustness under image compptions,
5

---

Published as a conference paper at ICLR 2025
create the MegaDepth- 1500-C benchmark, where the image pairs in MegaDepth- 1500 are distorted by the 15 corruption types proposed in (Hendrycks & Dietterich 2019) at five severity levels. The 15 corruption types are sorted into four categories: noise, blur, weather and digital, and are meant to measure a model's robustness against corruptions unseen during training. For indoor benchmark, adopt the popular ScanNet-1500 (Sarlin et al. 2020) and create the corrupted version ScanNet- 1500-C in a similar way as SreeaDepth-1500-GG
4.3 GEOMETRY ESTIMATION ON CORRUPTED DATA
MegaDepth-1500-4 Pose Estimation We follow the protocol in (Edstedt al. 2024 Sun al. 2021] to estimate camera pose and report AUC at different thresholds. Fig. F presents the AUC for each corruption type at each severity level. We observe dense matching methods DKM and RoMa significantly outperform the detector- freee method LoFTR over all corruptions and severity levels. Our method further improves upon RoMa. For the category of noise corruption (Gaussian noise, shot noise and impulse noise), we observe larger performance gap at higher severity level, achieving 10.1% 1% improvement for the shot noise corruption at severity level 5. The results averaged over severity levels and corruption types are reported in T Our method achieves 2.2% improvement in mean corruption AUC (mCA) compared RoMa. Note that on clean data our method achieves only 0.4% gain over RoMa. The increase in mCA thus indicates improved corruption robustness for our method.
Gaussian Noise
Shot Noise
Impulse Noise
Defocus Blur
Glass Blur
64
60
60
54
55
AUC
48
48
52
50
42
46
36
36
44
40
LOFTR
30
40
35
18
DKM
RoMaa
18
24
32
30
12
Ours
12
28
25
2
2
3
4
4
0
1
2
1
2
Motion Blur
Zoom Blur
Snow
Frost
Fog
63
60
56
60
55-
48
57
AUC
40
40
54
45
51
32
32
40
35
24
24
45
30
16
16
16
42
25
8-
1
2
3
2
3
0
1
5
1
2
3
Brightness
Contrast
Elastic Transform
Pixelate
JPEG Compression
63
64
60
60
60
35-
56
58
AUC
56-
50
57
52
54
45
54
48
52
40
51
40
50
35
48
30
30
48
36
46
45
1
2
2
3
2
severity
severity
severity
severity
severity
Figure 3: AUC@5 (%) for each corruption type in MegaDepth-15000 1500-C.
MegaDepth-1500 under Adversarial Attacks We further consider adversarial attacks, which is an extreme type of corruption that perturbs the image in carefully crafted direction and serves as worst- case analysis for model robustness. Two attacks, Fast Gradient Sign Method (FGSM) (Goodfellow

---

Published as a conference paper at ICLR 2025
Table I: AUC averaged over severity levels and corruption types in MegaDepth-1500-C.
The value for each conruption type is the value averaged over severity levels, and mCA A is the mean corruption AUC05 for all 15 corruption types. Models are trained only on clean images.
Noise
Blur
Weather
Digital
Method Clean mCA
Gauss.
Shot Impulse
Defocus Glass Motion
Zoom
Snow
Frost
Fog
Bright
Contrast
Elastic
Pixel JPEG
LoFTR
52.8
32.2
23.9
26.2
25.0
39.8
37.1
38.4
8.2
14.3
13.5
45.2 48.6
40.3
34.1
48.4 39,4
DKM
60.4
44.3
37.6
40.0
37.0
49.0
48.6
50.4
14.3
32.8
35.8 56.8 58.0
50.3
44.0
57.3 53.1
RoMa
62.6
47.6
43.6
46.2
43.3
49.4
49.9
52.3
13.2
47.3
46.6
57.7
59.3
47.6
44.4
54.7
Ours
63.0
49.8
49.4
51.9
49.5
49.8
51.9
53.4
44.5
47.9
58.5
60.5
48.1
46.6
59.2 55.9
et al. 2014) and Projected Gradient Descent (PGD) (Madry 2017), are investigated. The attacks are (x bounded and the PGD attack is computed with 20 iterations. We experiment with different perturbation budgets (denoted as e) and the results are presented in Fig. [4] We observe that the detector-free approach LoFTR is vulnerable to adversarial attacks, with AUC quickly dropping to zero as e increases. Compared to DKM, RoMa has better adversarial robustness under PGD attack, probably due to the of more sophisticated network architectures. Our method is able
sistently outperform others, achieving up to 2.9% gain over RoMa for the FGSM attack, and up to 3.4% gain over RoMa for the PGD attack.
FGSM
PGD
LoFTR
LoFTR
DKM
DKM
RoMa
RoMa
Ours
Ours
50-
5
40
AUC@5(%)
30
30
20
20
10-
10
4r255
8255
12/255
4/255
B255
12/255
Figure 4: AUC@5 (%) on MegaDepth- 1500 under FGSM and PGD attacks with different e values (perturbation budgets).
ScanNet-1500-C and ScanNet-1500 under Adversarial Attacks The benchmarking results on ScanNet-1500-C (with the same corruption settings MegaDepth- 1500-C) and ScanNet-1500 un- der adversarial attacks (using the same attack settings as MegaDepth- 1500) are presented in Table [ and Fig. [| respectively. While the advantage of method over RoMa is less pronounced (refer to Section 4.5 for the analysis of the underlying reasons), we still achieve up to 0.6% gain over RoMa on MeauNet-1500-C and up 0.7% gain for the FGSM attack.
Table 2: AUC@5%(%) averaged over severity levels and corruption types in ScanNet-15000 The value for each corruption type is the value averaged over severity levels, and mCA is the mean corruption AUC for for all 15 corruption types. Models are trained only on clean images.
Noise
Blur
Weather
Digital
Method Clean
mCA
Gauss. Shot Impulse Defocus Glass Motion 7oom
Snow
Frost
Fog
Bright
Contrast Elastic Pixel JPEG
LoFTR
22.1
12.3
13.2
14.3
13.1
15.3
18.4
16.0
1.8
4.4
2.6
5.3
16.4
19.2 21.2 17.7
DKM
29.4
22,3
21.8
22.5
21.1
28.6
28.4
28.0
7.9
14.1
9,3
22.5
26.1
22.5
27.3 29.5 25.2
RoMa
31.8
25.9
25.8
26.3
25.4
30.5
30.4
30.1
9.9
22.0
27.6
29.3
26.1
29.8
31.2 28.4
Ours
32.2
26.3
26.2
26.6
26.1
30.7
30.6
30.5
10.2
22.6
22.6
15.5
28.1
29.3
26.7
30.0
31.8 28.8
7

---

Published as a conference paper at ICLR 2025
LoFTR
LoFTR
RoMa Ours
RoMa
Ours
AUC@5199
20
AUC@5(%))
20
15
15-
2755
4/255
8/555
12.255
1/2
27555
4/255
B/255
12/255
Figure 5: AUC@5 (%) on ScanNet- 1500 1500 under FGSM and PGD attacks with different values
(perturbation budgets).
4.4 GEOMETRY ESTIMATION ON CLEAN DATA
150ppthh-00000 The benchmarking results of method the original clean MegaDepth- 1500 and ScanNet- 1500 benchmarks are presented in Table 3 and Table 4 Our method achieves comparable, or slightly better performance compared to RoMa, suggesting that the improved robustness of method does not sacrifice the performance on clean data.
Table 3: SotA comparison on MegaDeppth-1500 Measured in AUC (higher is better).
Method
AUC
5
10a
20" 1
LightGlue (Lindenberger et al.
2023
ICCV223
51.0
68.1
80.7
LoFTR (Sun 2021)
52.8
69.2
81.2
PDC-Net+ Truong al
2023
T5AMP
51.5
67.2
78.5
ASpanFormer (Chen el al
2022
BOCV'22
55.3
71.5
83.1
ASTR (Yu et al. (023) CVRR 23
58.4
73.1
83.8
DKM (Edstedt ct al.
2023
CIVPR'23
60.4
74.9
85.1
PMatch (7hu
20223
CVPR'23
61.4
75.7
85.7
CasMTR (Cao &
2023
1CCV'23
59.1
74.3
84.8
RoMa Edsted: et al.
2024
CVPX224
62.6
76.7
86.3
Ours
63.0
76.9
86.5
Table 4: SotA comparison on ScanNet-1500. Measured in AUC (higher is better).
Method J
AUC@
5" f
10
20 1
SuperGluc (Sarlin
2020)
CVPR'19
16.2
33.8
$1.8
LoFTR (Sun et al.
2021
CVPR'21
22.1
40.8
57.6
PDC-Nef+
(ooo+(TTma (Truon ct
2023)
TPAMT'3
20.3
39.4
57.1
ASpanFormer (C Chen
2022
BCCV
25.6
46.0
63.3
PATS Ni al. 2023
26.0
46.9
64.3
DKM Exstedt al.
2023
CVPR
29.4
50.7
PMatch Zhu & Liu 2023
CVPR'23
29.4
50.1
67.4
CasMTR Cao & Fü 2025
JCCV'23
27.1
47.0
64.4
RoMa Edstedt al.
2024
31.8
53.4
70.9
Ours
32.2
54.1
71.6
8

---

Published as a conference paper at ICLR 2025
4.5 FURTHER ANALYSIS
Why Our Method Outperforms RoMa We conduct further analysis method to understand why it can outperform RoMa. We compute the average endpoint error (AEPE) (Melckhov et al, 2019), which is defined the average Fuclidean distance between the estimated and ground truth warp. The AEPE for our method and RoMa on NegaDepth--15000 100-C is 6.12 6.12 and 6.07, respectively. The comparable AEPE values suggest that the warp prediction branch of our method performs sim- ilarly to RoMa. This is understandable as we only modify the certainty prediction branch of RoMa for evidential learning. We visualize the certainty map predicted by RoMa and our method in Fig. 6 Compared to clean images, corruption causes both RoMa and our method to produce reduced cer- tainty in matchable regions. However, RoMa tends to predict very low certainty value (e.g., below 0.05) for matchable regions corrupted images, ie., the model over-confidently classifies the pix- els as unmatchable. As the following balanced sampling algorithm samples matches based on the estimated certainty score, the extremely low certainty values make it hard to sample enough matches from matchable regions. Our method is able to predict higher certainty value (i.e., ~ 0.4) for match- still facilitate the following balanced sampling
able region. Such prediction, though not perfect, step to sample a diverse set of matches from matchable regions. This problem is less severe on ScanNet, where the overlap between the image pairs is larger and the sampled matches can still well cover the matchable region even though the predicted certainty is low. This may explain why the advantage of our method is less significant corrupted ScanNet- 1500 compared corrupted MegaDepth-1500 1500.
RoMa
Our
1.0
FGSM Gaustan Noise
0.8
FIFEFF
0.6
0.4
Gautsian
0.2
FOSM
0.0
Image Pair
Certainty Map
Sampled Matches
Certainty Map
Sampled Matches
Figure 6: Visualization of certainty map and sampled matches for pose estimation. Top scene is from MegaDepth- 1500 and bottom scene is from ScanNet-1500. Our method predicts higher cer- tainty value for matchable region corrupted images than RoMa, enabling the following balanced sampling step to select a set of diverse and reliable matches for downstream geometry estimation tasks.
To cast more insight into the behavior of our method, we visualize the evidence map output by our model Fig. We observe that under corrupted cases, the model cannot predict high evidence for both classes in the matchable region, resulting in similar probability for the two classes, i.e., there high uncertainty in the estimation. Compared to RoMa's over-confidence estimation, a high- uncertainty prediction still assigns some confidence the correct class and causes less devastating effect to the following geometry estimation pipeline.
Ablation Study Applying EDL at Different Scales We report the ablation study applying
EDL at different scales in Table [ We observe that applying EDL on coarse scale alone is not effective, the final prediction comes from the finest scale (which still learnt by the BCE loss).

---

Published as a conference paper at ICLR 2025
Gaursan
Image Pair
Class Evidence Map
Closs 2 Evidence Map
Clas 1 Probobiitty
Exidential Uncertzinty
Figure 7: Visualization of evidence map predicted by our model. Class represents the claim "the predicted correspondence is reliable" and Class 2 represents the opposite. Class probability is essentially the certainty score. The last column shows the evidential uncertainty (the built-in uncertainty defined in Eq. (6)), which indicates high-uncertainty prediction in matchable regions on corrupted images. Note that the evidence value is unbounded non-negative value and has been clamped to [0,1] for visualization purposes.
Applying EDL on fine scales improves the performance corrupted samples significantly. The best performance achieved when EDL applied on both coarse and fine scales.
Table 5: AU) on MegaDepth- 1500 for models trained with EDL at different scales. The first row with both coarse and fine scale EDL disabled degenerates to the original RoMa model. Note that we use reduced training setting for this study (training image resolution 448 448, batch size 16 and training iteration 62,500), so the reported performance lower than those reported in
Table I
Coarse
Fine
AUC@5" (clean)
AUC@5" (Gaussian noise
60.9
22.8
59.9
22.9
61.4
31.7
61.5
33.1
Computation Cost Comparison We report the training and inference time of RoMa and our method in Table [6] For training, our method takes 1.4% more GPU hours than RoMa. For inference, our method actually incurs marginally lower cost. This is probably due to the fact that EDL uses simple operations (e,g., addition and division) to obtain the final probability, eliminating the more complicated sigmoid computation in RoMa.
Table 6: Comparison of computation time. Training is conducted on 8 A40 GPUs with batch size 32, input size 560 560 and 250k iterations. Inference conducted on A40 GPU with batch size (1 pair of images) and input size 672 x 672. The inference time is averaged over the 1500 pairs in MegaDepth- 1500 benchmark.
Training (in hours)
Inference ms)
RoMa Ours
126.6
329
128.4
327
5 CONCLUSIONS
In this work, we propose to incorporate evidential deep learning into the certainty prediction branch of dense feature matching methods. To evaluate the robustness of feature matching methods, we propose MegaDepth- 1500-C and ScanNet-1500-4 benchmarks that contain images distorted by 15 corruption types at five severity levels. Evaluation results show that EDL allows the model predict higher certainty values for matchable region on corrupted images compared to traditional cross- entropy loss, facilitating the sampling of a set of both certain and diverse matches for geometry estimation. Our method achieved up to 10.1% improvement for noise corruption in MegaDepth- 1500-C. We further investigate our method's 's robustness against adversarial attacks, and show that method achieved up to 2.9% gain over SotA for the FGSM attack, and up 3.4% gain for the PGD attack. These results demonstrate the effectiveness of EDL in improving the robustness of dense feature matching models.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The authors would like to thank the reviewers for their constructive feedback during the review process. This work is supported by the Agency for Science, Technology and Research (A*STAR) under its MTC Programmatic Funds (Grant No. M23L760021).
REFERENCES
Shashank Agnihotri, Steffen Jung, and Margret Keuper. Cospgd: efficient white-box adversarial attack for pixel-wise prediction tasks. In Forty-first International Conference on Machine Learn- ing, 2024.
Vassileios Balntas, Karel Lenc, Andrea Vedaldi, and Krystian Mikolajezyk. Hpatches: A bench- mark and evaluation of handcrafted and learned local descriptors. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5173-5182, 2017.
Chenjie Cao and Yanwei Fu. Improving transformer- image matching by cascaded capturing spatially informative keypoints. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 22129-12139 2023.
Hongkai Chen, Zixin Luo, Lei Zhou, Yurun Tian, Mingmin Zhen, Tian Fang, David Mckinnon, Yanghai Tsin, and Long Quan. ASpanFormer: Detector-free image matching with adaptive span transformer. In Proc. European Conference on Computer Vision (ECCV), 2022.
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5828-58383 2017a.
Angela Dai, Matthias Nießner, Michael Zollöfer, Shahram Izadi, and Christian Theobalt. Bundle- fusion: Real-time globally consistent 3d reconstruction using on-the-fly surface re-integration. ACM Transactions on Graphics 2017 (TOG), 2017b.
Daniel DeToone, Tomasz Malisiewic7, and Andrew Rabinovich. Superpoint: Self-supervised interest. point detection and description. Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 224-236, 2018.
Johan Edstedt, Ioannis Athanasiadis, Mårten Wadenbäck, and Michael Felsberg, DKM: Dense kernelized feature matching for geometry estimation. In IEEE Conference on Computer Vision and Pattern Recognition, 2023.
Johan Edstedt, Qiyu Sun, Georg Bökman, Märten Wadenbäck, and Michael Felsberg, Roma: Robust dense feature matching. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 19790-19800, 2024.
Junyu Gao, Mengyuan Chen, Liangyu Xiang, and Changsheng Xu. A comprehensive survey on evidential deep learning and its applications. arXiv preprint arXiv:2409.04720, 2024.
Khang Truong Giang, Soohwan Song, and Sungho Jo. TopicFM: Robust and interpretable topic- assisted feature matching. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 37, 2023.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. arXiv preprint arXiv; 1903. 12261, 2019.
Addison Howard, Eduard Trulls, Kwang Moo Yi, Dmitry Mishkin, Sohier Dane, and Yuhe Jin.
Image matching challenge 2022, 2022. URL https://kaggl wwtp///ggl..c.ooaaaooooooooooa/e wttp:titopptiooooooo.oo
Lmage-matchingg cha www.eeengeeee 2022
Yibo Hu, Yuzhe Ou, Xujiang Zhao, Jin-Hee Cho, and Feng Chen. Multidimensional uncertainty- aware evidential neural networks. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 35, pp. 7815-7822, 2021.
11

---

Published as a conference paper at ICLR 2025
Audun Josang, Subjective logic, volume 3. Springer, 2016.
Oguzhan Fatih Kar. Teresa Yeo, Andrei Atanov, and Amir Zamir. 3d common corruptions and data augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18963-18974, 2022.
Zhengqi Li and Noah Snavely, Megadepth: Learning single-view depth prediction from internet photos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2041-2050 2018.
Philipp Lindenberger, Raul-Edouard Sarlin, and Mare Pollefeys. LightGlue: Local Feature Matching at Light Speed. In ICCV, 2023.
I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv: 2017.
David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60:91-110, 10, 2004.
Alcksander Madry. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 2017.
Runyu Mao, Chen Bai, Yatong An, Fengqing Zhu, and Cheng Lu. 3DG-STFM: 3d geometric guided student-teacher feature matching. In Proc. European Conference on Computer Vision (ECCV), 2022.
laroslav Melekhov, Aleksei Tiulpin, Torsten Sattler, Marc Pollefeys, Esa Rahtu, and Juho Kannala. Dge-net: Dense geometric correspondence network. In 2019 IEEE Winter Conference on Appli- cations of Computer Vision (WACV), pp. 1034-1042. IEEE, 2019.
Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexan- der S Ecker, Matthias Bethge, and Wieland Brendel. Benchmarking robustness in object detec- tion: Autonomous driving when winter is coming. arXiv preprint arXiv: 1907.07444, 2019.
Dmytro Mishkin, Jiri Matas, Michal Perdoch, and Karel Lenc. WxBS: Wide Baseline Stereo Gen- cralizations. In Proceedings of the British Machine Vision Conference. BMVA, September 2015.
Junjie Ni, Yijin Li, Zhaoyang Huang, Hongsheng Li, Hujun Bao, Zhaopeng Cui, and Guofeng Zhang. Pats: Patch area transportation with subdivision for local feature matching. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 17776-17786, 2023.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193. 2023.
Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEFF/CVF confer- ence on computer vision and pattern recognition, pp. 4938-49444 2020.
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classifica- tion uncertainty. Advances in neural information processing systems, 31, 2018.
Murat Sensoy, Lance Kaplan, Federico Cerutti, and Maryam Saleki. Uncertainty awrtai aware deep clas- sifiers using generative models. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 5620-5627, 2020.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! Ad- vances in neural information processing systems, 32, 2019.
Ruohua Shi, Lingyu Duan, Tiejun Huang, and Tingting Jiang. Evidential envertainty-guidee mito- chondria segmentation for em images. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 4847-4855, 2024.
12

---

Published as a conference paper at ICLR 2025
Binyi Su, Hua Zhang, and Zhong Zhou. Hsic-based moving weight averaging for few-shot open-set object detection. In Proceedings of the 31st ACM International Conference on Multimedia, pp- 5358-5369 2023.
Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. LoFTR: local feature matching with transformers. In Proceedings of the IEEE/CVF Conference on Com- puler Vision and Patlern Recognition, pp. 8922-8931, 2021.
Detector- free
Prune Truong, Martin Danelljan, Radu Timofte, and Luc Van Gool. PDC-Net+: Enhanced Proba- bilistic Dense Correspondence Network. arXiv preprint arXiv:2 13912, 2021.
Prune Truong, Martin Danelljan, Radu Timofte, and Luc Van Gool. PDC-Net+: Enhanced Prob-
abilistic Dense Correspondence Network. IEEE Transactions Pattern Analysis and Machine Intelligence, 2023.
Michal J. Tyszkiewicz, Pascal Fua, and Eduard Trulls. DISK: learning local features with policy gradient. In NeurlPS, 2020.
Jiahuan Yu, Jiahao Chang, Jianfeng He, Tianzhu Zhang. Jiyang Yu, and Wu Feng. ASTR: Adaptive spot-guided transformer for consistent local feature matching. In The IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023.
Shengjie Zhu and Xiaoming Liu. PMatch: Paired masked image modeling for dense geometric matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, 2023.
A APPENDIX
A.1 IMPLEMENTATION DETAILS
Model Architecture The RoMa model consists of three modules: feature extractor, global matcher and warp refiners. The feature extractor uses frozen foundation model DINOv2 (Oquab et al. 2023) for extracting feature maps at stride 16, and VGG19 for extracting feature maps stride [8, 4, 2, 1J. The global matcher consists of Gaussian Process-baas based encoder and a transformer-based decoder. takes as imput feature maps stride 16, and outputs initial warp and certainty map estimation. The warp refiner is CNN module that consists of 5 x 5 depth-wise convolution and x convolution. It is applied to features maps at scale [16, 8, 4, 2, 1] iteratively. At each scale, it takes the corresponding feature maps and previously estimated warp and certainty map input and outputs a residual estimation on warp and certainty. To modify RoMa for evidential learning, increase the number of the output channel in the transformer-basee decoder and the warp refiner by 1. The original channel for sigmoid logit and the added channel are then used for the 2-class evidence estimation in proposed formulation.
Details on Datasets The MegaDepth dataset (Li & Snavely 2018) contains images collected from Internet for 196 outdoor scenes. For each scene, the camera pose and depth map of each image are computed by COLMAP, a state-of-the-arr software for StometurefrommMootion (SfM) and Multi- View Stereo (MVS). We use the same training split as in RoMa. The ScanNet dataset (Dai al.) 2017a) contains 1513 RGB-D scans for indoor scenes. The camera pose of each image is estimated by the BundleFusion system (Dai al. 2017b). The training and test split of the dataset contains 1201 and 312 scenes, respectively.
Training Procedure We follow the training setup in RoMa's repo The batch size is 32 and training image size is 560 x 560. The learning rate for encoder (including the VGG19 feature extractor. The DINOv2 weights are frozen) is 10-4 and the learning rate for decoder (including the global matcher and warp refiners) is 5 10 The model is trained for 250k iterations using the AdamW (Loshchilov) 2017) optimizer with weight decay 0.01, and the learning late is decayed by 0.1 at 225k iteration. For outdoor geometry estimation, training pairs are randomly sampled from the MegaDepth training split. For indoor geometry estimation. training pairs alternatively sampled from the MegaDepth or the ScanNet training split, following (Edstedt et al. 2024: (Sun et al.) 202I). Training takes approximately days server with 8 A40 GPUs.
13

---

Published as a conference paper at ICLR 2025
A.2 ROBUSTNESS AGAINST NEWER CORRUPTIONS AND ATTACKS
3D Common Corruptions (3DCC) We evaluate our method on three corruptions from the 3DCC collections (Kar et al. 2022), including low light noise, ISO noise and color quantization. Our method achieves consistent and significant gains over RoMa, obtaining up to 9.4% increase in AUC@5%. Some examples of corrupted images are shown Fig. We notice that for those corrup- tions that require depth information for generation, e.g., 3D motion blur, Fog 3D, using MegaDepth 's ground truth depth map does not create realistic images, and thus are not used our benchmark- ing. This suggests that 3D corrupted image generation requires dataset-specific parameter tuning, lacking the allg- convenience as its 2D counterpart.
Low-light Noise
IS0 Noise
Color Quantization
60
60
55
56
45
48
40
44
48
40
30
RoMa
36
25
2
4
severity
severity
severity
Figure 8: AUC@5 (%) for three corruption types in 155paDeeth-150-3DGGG
Fog30
Figure 9: Corrupted images from MegaDepth-11000 1500-3DCC. For corruptions that require depth infor- mation for generation, e.g., 3D motion blur, Fog 3D, 3DCC cannot work nut-of-the-shee
Pixel-wise Adversarial Attacks We experiment with recently proposed attack CosPGD (Agni- hotri et al. 2024).. CosPGD is an improved version of PGD, where the pixel-wise loss is weighted by the cosine similarity between (normalized) prediction and ground truth so that pixels with correct prediction will be perturbed more in generating the attack. CosPGD can be readily applied for both classification and regression tasks. For regression task, the prediction and ground truth are normal- ized by softmax before computing the cosine similarity. We evaluate the robustness of RoMa and our method under CosPGD attack and the results are shown in Fig. 10 Our method outperforms RoMa consistently, achieving up 3.0% gain over RoMa.
A.3 BENCHMARKING ON ADDITIONAL. DATASETS
We provide additional benchmarking results in Table 7 and Table g Our method obtains 1.7% in- crease mAA @1 Opx than RoMa on WxBS (Mishkin al. 2015). and 0.5% increase AUCã3px on HPatches (Baintas et al. 2017).
A.4 VERIFY OUR METHOD WITH OTHER DENSE MATCHING MODELS
In the main experiments, we use the SotA dense matcher RoMa as bascline and apply our pro- posed framework it. Here apply our method another dense matcher DKM (Edstedtl
14

---

Published as a conference paper at ICLR 2025
CDSPGD
RoNo
60
50
AUC@55%%)
40-
30
0
2/255
4/255
8/55
12/255
Figure 10: AUC@5*(%) on MegaDepth- 1500 under CosPGD attack with different values (pertur- bation budgets).
Table 7: SotA comparison on WxBS Mishkin et al. 2015). Measured in mAA@10ps (higher is better).
Method
mAA@
10px f
DISK (yszkkiewicz et al.
2020
NeurIP5:20
35.5
DISK LightGlue Tyszkiewicz et al.
2020
Lindenherger al.
2023
10CV'23
41.7
SuperPoint +SuperGilue Tone
2018
Sarim
2020)
31.4
LoFTR (Sun et al.
2021
CVPR'21
55.4
DKM (Edstedt et al.
2023)
58.9
RoMa [dstedd
2024
CVPR224
80.1
Ours
81.8
et al. 2023). Similar RoMa, DKM employs certainty estimation branch in both global matcher and warp refiners. We modify the certainty estimation branch to generate evidence and replace the BCE loss with evidential loss. We follow the training setup in DKM's repo httpss//gituub.co/Paaarrtt//////
The model is trained on server with 8 A5000 GPUs with batch size 16, image resolution 540 x 720 and training iteration 500k. Training takes roughly days. For fair comparison, we retrain the original DKM model using the same setup and report the results in Table 9
Our method increases the clean AUC@5 by 1.6% on MegaDepth- 1500, and increases the
Table 8: Homography estimation on HPatches Ballnta et al. 2017 better).
measured in AUC (higher is
Method
AUC
@3px
@5px
@ @10px
SuperGlue Sarlin al.
2020
CVPR'19
53.9
68.3
81.7
LoFTR (Sun al. 202
CVPR
65,9
75.6
84.6
TopicFM Grang et al. [2023]
Andi'12
67.3
77.0
85.7
3DG-STFM (Mao el all
2022)
EOCV122
64.7
73.1
81.0
ASpanFormer CCen et al.
2022
BOCVV12
67.4
76.9
85.6
PDC-Net+ (Truong et al
2021
Ardi'21
67.7
77.6
86.3
DKM (Edstedt el al. 2023
CVPR'23
71.3
80.6
88.5
RoMa Pidstedt al.
2024
72.7
81.4
89.1
Ours
73.2
81.9
89.3
15

---

Published as a conference paper at ICLR 2025
mean corruption AUC by 1.3% MegaDepth-15000 1500-C. These results demonstrate the general- izability of our method beyond RoMa.
Table Applying our method with the DKM model. The reported performance is AUC@5" averaged over severity levels and corruption types on MegaDepth- 1500-C.
Noise
Blur
Weather
Digital
Method Clean mCA
Impulse Defocus Glass
Frost
Fog Bright Contrstt Elastic
DKM
58.4
39.8
31.1
33.4
29.0
44.5
42,9
46.4
11.5
27.4
30.6
53.8 54.9
48.1
41.2 53.6 49.7
Ours
60.0
41.1
31.2
33,5
29.0
46.8
45,9
48.4
13.4
27.7
33.3
54,8 56.6
47.1
43.1 55.7 50.8
A.5
COMBINING CERTAINTY SCORE OF OUR METHOD WITH WARP OF ROMA
We report in Table 10 the results of combining certainty score from our model with the warp from the original RoMa model. We observe that using certainty score, the performance of RoMa increased from 25.3% to 36.5%, close to our results of 37.9%. In Section 4.5 we report the average endpoint emror (AEPE), which is defined as the average Euclidean distance between the estimated and ground truth warp. The AEPE for our method and RoMa on MegaDepth-1500-CC is 6.12 and 6.07, respectively. The comparable AEPE values suggest that the warp prediction branch of our method performs similarly to RoMa. These two studies combined suggest that it is the better certainty estimation that brings the improvement in performance.
Table 10: AUC@55 for various combinations of certainty score and warp estimation 1egaDDepth-1500.
Cert (RoMa)
Warp (RoMa)
Cert (Ours)
Warp (Ours)
AUC85 (clean)
AUC05 (Gaussian noise
62.6
25.3
62.4
36.5
63.0
37.9
A.6 EFFECT OF SCORE THRESHOLD
We experiment with different score thresholds for balanced sampling and the results are presented in Fig. TT We observe that inappropriate choice of the score threshold degrade the performance for both RoMa and our method, yet our method is still able to outperform RoMa over wide range of values.
38
63.2
63.0
34
AUC@5
62.8
AUC@5(%)
32-
62.6
30
28
62.4
26
62.2
RoMa
24
0.025
0.050
0.075
0.100
0.025
0.050
0.075
0.100
Score Threshold
Score Threshold
(a)
(b)
Figure 11: Effect of score threshold for balanced sampling. (a) AUC@5 on MegaDepth-1500.
(b) AUC@5 (%) MegaDepth-15500 corrupted by Gaussian noise at severity level 5,
A.7 QUALITATIVE RESULTS
More visualization results are presented in Fig. 12 Our method predicts higher certainty value for matchable region than RoMa across different comuption types.
16

---

Published as a conference paper at ICLR 2025
PGD
Shot Noise
Motion Blur
Brightness
JPEG Comp
PGD
Shot Noise
Motion Blur
JPEG Comp Brightness
RoMa
Ours
Figure 12: Qualitative results. The matching results are visualized by taking the pixel values from the other image using the estimated correspondences, and weighting by the estimated certainty. Top scene from MegaDepth- 1500 and bottom scene from ScanNet-1500. Our method predicts higher certainty value for matchable region than RoMa across different corruption types.
17


Reviews:
[{"review_id": "SqwZZisazl", "reviewer": "Reviewer_vn4d", "summary": "This paper presents an interesting idea of evidential learning for certainty estimation for the dense pixel-level feature matching task. \nThe proposed method is supposedly more OOD and adversarial robust than the current SotA RoMa. \nIt is tested against 2D Common Corruptions variants of 2 commonly used datasets for this task i.e. MegaDepth-1500 and ScanNet-1500.\nIt is also tested against outdated adversarial attacks such as FGSM and PGD.", "questions": "Following are the questions for which I would highly appreciate an answer, these questions have not impacted my current recommendation for this paper, however, the response might have a significant impact on my final recommendations.\n\nQ1- **Unclear evaluation details for adversarial attacks used.** \nThe epsilon values used for attack are starting from 0.1, 0.2 going up to 1., here the attacks l-infinity norm bounded? If yes, then what is the valid image space? Is it [0, 1] or is it [0, 255], meaning when epsilon = 1, does this mean that the epsilon is actually 1/255 (meaning that the valid image space is [0, 255]), or is the value of epsilon actually 1, meaning the entire image is nothing but adversarial noise? In this case, the image would also look semantically different to the human eye meaning that it will no longer be a valid adversarial attack.\nAnd if the epsilon value is in fact 1/255, then the drop in performance is too significant for a very small epsilon value indicating the method is not truly robust to adversarial attacks. Could you also please comment on this?\n\nQ2- **The idea of using Evidential Learning for Pixel-Matching is not entirely novel.** \nWhile the exact downstream task in [3] is different from the one explored by this proposed work, the core ideas for both seem unusually very similar, the key difference being the distributions used, while [3] used a Normal Inverse-Gamma (NIG) distribution, this work uses a Dirichlet distribution. Would you please further highlight the key differences between the two other than some task-related implementation details?\n\n\n**References**\n\n[3] Chen Wang, Xiang Wang, Jiawei Zhang, Liang Zhang, Xiao Bai, Xin Ning, Jun Zhou, Edwin Hancock,\nUncertainty estimation for stereo matching based on evidential deep learning,\nPattern Recognition, Volume 124, 2022,108498, ISSN 0031-3203, https://doi.org/10.1016/j.patcog.2021.108498. (https://www.sciencedirect.com/science/article/pii/S0031320321006749)", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "If the idea of using evidential learning for feature matching is truly novel then that makes the work quite interesting and significant.\nApart from a couple of small typos, the paper is very well written. \nThe structure of the paper and the intended story are easy to follow.\nThe abstract of the paper is well written and to the point.", "weaknesses": "W1- **A lot of implementation details are missing from the paper.**\nSimply mentioning that it is built on top of RoMa is insufficient information.\nIt is understandable to do so for the main paper to save space; however, the supplementary material should be used to provide such information, for example, the exact architecture, the training procedure, details about the datasets, HPC resources used, and other details important for reproducibility. \n\nW2- **Needs a stronger argument for why OOD and Adversarial Robustness is important.**\nThe argument made in the introduction to explain why OOD and Adversarial robustness are important for this task can be made significantly stronger. Unfortunately, a case has not been made for why this is interesting and important for the community. \n\nW3- **Out-dated evaluations for robustness.**\nIf the argument for OOD and Adversarial robustness is readiness for the real world, then the evaluations used do not hold up to the argument. Since the 3D Common Corruptions [1] are more real-world common corruptions than the 2D Common Corruptions used in the paper. Additionally, FGSM and PGD attacks were used for evaluating adversarial robustness, however [2] showed in their work that these attacks, originally proposed image classification are inadequate for pixel-wise prediction tasks, such as the one used in this proposed work. This is because FGSM and PGD optimize the attack by increasing the aggregate loss and not the per-pixel loss, this can cause the attack to be highly localized making a non-robust method appear very robust as the mean performance would still be quite well over the rest of the image space. Thus, specialized pixel-wise attacks such as CosPGD are essential for truly evaluating the adversarial robustness of pixel-wise prediction tasks. \n\nW4- **Using 2D Common Corruptions on other known datasets is not always a novel contribution.**\nIt is unclear if the contribution of the 2 supposed OOD Robustness evaluation datasets MegaDepth-1500-C and ScanNet-1500-C is merely using 2D Common Corruptions proposed for ImageNet-1k and CIFAR datasets but changing their resolutions and applying them to the respective iid datasets or if there is more to the story, for example, some unforeseen complications that needed to be handled? If not, then simply applying these corruptions to other datasets is not exactly a novel contribution, it is still an interesting study just not a \"new contribution\" as claimed in the bullet points in the introduction of the paper.\n\nW5- **Almost Redundant Presentation of Results.**\nIncluding both Table 1 and Figure 3 is redundant. I understand that Table 1 contains the mean values over the 5 severity levels while Figure 3 shows the values at each severity, however by using straight dashed lines of respective colors, with y = mean value for all x values the need for Table 1 is eliminated.\n\n\n\n\n**References**\n\n[1] Kar, Oğuzhan Fatih, et al. \"3d common corruptions and data augmentation.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n\n[2] Agnihotri, Shashank, Steffen Jung, and Margret Keuper. \"CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks.\" Forty-first International Conference on Machine Learning. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9TT0FWXotm", "reviewer": "Reviewer_3kpK", "summary": "This paper applies evidential deep learning to feature matching tasks, introducing an evidential learning framework for certainty estimation in dense feature matching problems. The proposed method enhances robustness in dense matching against corruptions and adversarial attacks, with extensive experiments conducted and visualization presented to demonstrate its performance.", "questions": "Please refer to the weaknesses section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-motivated, and the main idea is clearly explained.\n\n2. Experiments are conducted across a wide range of benchmarks with various types of corruptions and adversarial attacks. The proposed method outperforms in most cases.\n\n3. The paper includes visualizations to analyze why the proposed method performs better than comparison method, particularly on corrupted data across different datasets.", "weaknesses": "Several questions need to be addressed:\n\n1. This work employs a two-dimensional evidential deep learning (EDL) framework to certainty estimation in both coarse-scale and fine-scale losses. What would happen if EDL were applied to only one of these loss scales? Conducting an ablation study could provide insights into the effectiveness of EDL at each scale. It would be great if the authors could report performance results by applying EDL exclusively to coarse-scale or fine-scale losses, compared to using it on both losses. \n\n2. Experiments are conducted on two datasets, MegaDepth-1500 and ScanNet-1500. There are other datasets mentioned in RoMa paper such as the street-view IMC 2022 and the challenging WxBS Benchmark. Evaluating the proposed method on those different datasets could further demonstrate its generalizability across diverse scenarios. \n\n3. The proposed framework incorporates evidential deep learning into the training process. Could you provide details on how the proposed framework affects computational time, specifically in terms of training and inference times compared to the baseline RoMa method?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1V9neJ6cyc", "reviewer": "Reviewer_oVDg", "summary": "The authors propose modelling certainty of correspondences in dense matchers using an evidential deep learning approach. Instead of just estimating a certainty between 0 and 1, the model outputs the parameters of a Dirichlet distribution over probabilities of the two classes \"the predicted correspondence is reliable\" and \"the predicted correspondence is unreliable\". The certainty output is then the expected probability of the first class according to this Dirichlet distribution. The authors show experimentally by retraining the dense matcher RoMa that their approach leads to improved certainty scores, in particular leading to increased robustness to image corruptions.", "questions": "1. How much does the threshold used for balanced sampling matter for the robustness under image degradations? Both in the original RoMa and the new model.\n2. My reading is that computationally, the new certainty estimation is more or less as heavy as the old in terms of inference and training speed. Is this correct?\n3. Is the increased performance due to only better certainties? For example, is the performance the same if certainty scores from the new model are combined with matches from the original RoMa?\n4. Is there a way to make use of the built in uncertainty in the Dirichlet distribution as described in Weakness 3?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper proposes a simple tweak to the RoMa-method, which gives good results experimentally.\n2. Within the framework of the RoMa matcher, certainties are updated iteratively in each warp refinement step by a \"logit-offset\" in the original model. It seems more intuitive to let each refinement step produce positive evidence values for the two classes \"correct\" and \"incorrect\" that are summed over the steps, as is done in this paper. Perhaps, the authors could expand on this in the paper.\n3. In general, outputting good certainties is an underexplored part of deep learning for 3D vision. The application of evidence based learning to dense matchers is novel.", "weaknesses": "1. The experiments are limited in that only a single model is tested. Hence, it is difficult to say if the improvements generalize to other models.\n2. The explanation of evidential deep learning was difficult to understand, and I had to refer back to the original paper by Sensoy et al. I think this section could be improved.\n3. Evidential deep learning has a built-in uncertainty measure. In the context of the present paper, we get a Dirichlet distribution over the classes \"the predicted correspondence is reliable\" and \"the predicted correspondence is unreliable\", and the associated uncertainty describes how spread out this Dirichlet distribution is. This uncertainty is however not used in the present approach. This makes it a bit difficult to interpret the method. We get a Dirichlet distribution but only use its expected value over the first class. This expected value should signify correspondence reliability, but there is also an uncertainty of this prediction inherent in the Dirichlet distribution, which is not used. Since RoMa uses regression-by-classification in the coarse matching step, a more natural approach may be to reformulate the loss for that classification over $N\\times N$ image patches as evidential and use the uncertainty of the predicted Dirichlet distribution as an uncertainty score.\n\n\n### Post-rebuttal:\n- The authors have added experiments with DKM to address weakness 1.\n- Weakness 2 has been addressed.\n- Weakness 3 has not been addressed but is left for future work.\n- My questions below have also been answered satisfactorily.\n\nLooking at the other reviews there are two main remaining weaknesses\n- The novelty is quite limited.\n- The results are not significantly better than RoMa, except under major image degradations.\n\nAll in all, I will raise my score to \"accept\", but note the two weaknesses above.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XX40S0HUu1", "reviewer": "Reviewer_Uwz6", "summary": "This paper proposes to unify evidential deep learning (EDL) and dense feature matching, achieving more robust matching results, especially for corrupted image pairs. The authors propose MegaDepth-1500-C and ScanNet-1500-C benchmarks to evaluate the robustness of the proposed method under common image corruptions. The proposed method enjoys superior results in both clean and corrupted data.", "questions": "If EDL is mainly used for certainty estimation, what are the differences or relationships of the proposed method compared to outlier filtering post-processing in feature matching (RANSAC)?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The incorporation of EDL to dense feature matching is interesting, and has not been investigated before.\n2. The proposed method enjoys good performance in corrupted data.", "weaknesses": "1. Although the point of EDL is interesting, the usage of EDL for certainty estimation in dense feature matching is still questionable. From the introduction in Section 2.3, I think EDL's main advance is to detect out-of-distribution samples or mining pseudo-unknown objects. However, the certainty estimation in feature matching is just a binary classification task (matched or not matched). Why is EDL still effective? The authors did not provide a more insightful discussion about this key question.\n\n2. The overall contribution is limited. Because of lacking enough in-depth discussion about EDL and certainty estimation in feature matching, makes this work appear more as a mere combination of these two approaches rather than a convincing exploration.\n\n3. The introduction of EDL in Section 3.2 is insufficient, missing the necessary background/preliminary in related works.\n\n4. Experiments are not sufficient, missing discussion of visual localization (InLoc and AachenDay-Night) and homography estimation (HPatches). The proposed method achieves significant improvements in corrupted data, while the improvements based on clean data are limited. As a general certainty estimation, the usage of EDL should consistently improve the matching accuracy in all scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)