#

RoMa đŸ›ïž:
Robust Dense Feature Matching
⭐CVPR 2024⭐

Johan Edstedt · Qiyu Sun · Georg Bökman · MÄrten WadenbÀck · Michael Felsberg

Paper | Project Page


example
RoMa is the robust dense feature matcher capable of estimating pixel-dense warps and reliable certainties for almost any image pair.

## Setup/Install In your python environment (tested on Linux python 3.10), run: ```bash pip install -e . ``` ## Demo / How to Use We provide two demos in the [demos folder](demo). Here's the gist of it: ```python from romatch import roma_outdoor roma_model = roma_outdoor(device=device) # Match warp, certainty = roma_model.match(imA_path, imB_path, device=device) # Sample matches for estimation matches, certainty = roma_model.sample(warp, certainty) # Convert to pixel coordinates (RoMa produces matches in [-1,1]x[-1,1]) kptsA, kptsB = roma_model.to_pixel_coordinates(matches, H_A, W_A, H_B, W_B) # Find a fundamental matrix (or anything else of interest) F, mask = cv2.findFundamentalMat( kptsA.cpu().numpy(), kptsB.cpu().numpy(), ransacReprojThreshold=0.2, method=cv2.USAC_MAGSAC, confidence=0.999999, maxIters=10000 ) ``` **New**: You can also match arbitrary keypoints with RoMa. See [match_keypoints](romatch/models/matcher.py) in RegressionMatcher. ## Settings ### Resolution By default RoMa uses an initial resolution of (560,560) which is then upsampled to (864,864). You can change this at construction (see roma_outdoor kwargs). You can also change this later, by changing the roma_model.w_resized, roma_model.h_resized, and roma_model.upsample_res. ### Sampling roma_model.sample_thresh controls the thresholding used when sampling matches for estimation. In certain cases a lower or higher threshold may improve results. ## Reproducing Results The experiments in the paper are provided in the [experiments folder](experiments). ### Training 1. First follow the instructions provided here: https://github.com/Parskatt/DKM for downloading and preprocessing datasets. 2. Run the relevant experiment, e.g., ```bash torchrun --nproc_per_node=4 --nnodes=1 --rdzv_backend=c10d experiments/roma_outdoor.py ``` ### Testing ```bash python experiments/roma_outdoor.py --only_test --benchmark mega-1500 ``` ## License All our code except DINOv2 is MIT license. DINOv2 has an Apache 2 license [DINOv2](https://github.com/facebookresearch/dinov2/blob/main/LICENSE). ## Acknowledgement Our codebase builds on the code in [DKM](https://github.com/Parskatt/DKM). ## Tiny RoMa If you find that RoMa is too heavy, you might want to try Tiny RoMa which is built on top of XFeat. ```python from romatch import tiny_roma_v1_outdoor tiny_roma_model = tiny_roma_v1_outdoor(device=device) ``` Mega1500: | | AUC@5 | AUC@10 | AUC@20 | |----------|----------|----------|----------| | XFeat | 46.4 | 58.9 | 69.2 | | XFeat* | 51.9 | 67.2 | 78.9 | | Tiny RoMa v1 | 56.4 | 69.5 | 79.5 | | RoMa | - | - | - | Mega-8-Scenes (See DKM): | | AUC@5 | AUC@10 | AUC@20 | |----------|----------|----------|----------| | XFeat | - | - | - | | XFeat* | 50.1 | 64.4 | 75.2 | | Tiny RoMa v1 | 57.7 | 70.5 | 79.6 | | RoMa | - | - | - | IMC22 :'): | | mAA@10 | |----------|----------| | XFeat | 42.1 | | XFeat* | - | | Tiny RoMa v1 | 42.2 | | RoMa | - | ## BibTeX If you find our models useful, please consider citing our paper! ``` @article{edstedt2024roma, title={{RoMa: Robust Dense Feature Matching}}, author={Edstedt, Johan and Sun, Qiyu and Bökman, Georg and WadenbÀck, MÄrten and Felsberg, Michael}, journal={IEEE Conference on Computer Vision and Pattern Recognition}, year={2024} } ```