PA-LoFTR: Local Feature Matching with 3D Position-Aware Transformer
Authors: Chenhao Li
Qisheng Tang
Shuangjiu Xiao
Ying Mao
Deli Dong
Zhifeng Shi
Guoliang Chen
Jiawen Cheng
Venue: Submitted to ICLR 2023 (2023)
Keywords: deep learning
transformer
image matching
pose estimation
position embedding
3d representation

Abstract: We propose a novel image feature matching method that utilizes 3D position information to augment feature representation with a deep neural network. The proposed method introduces 3D position embedding to a state-of-the-art feature matcher, LoFTR, and achieves more promising performance. Following the coarse-to-fine matching pipeline of LoFTR, we construct a Transformer-based neural network that generates dense pixel-wise matches. Instead of using 2D position embeddings for transformer, the proposed method generates 3D position embeddings that can precisely capture position correspondence of matches between images. Importantly, in order to guide neural network to learn 3D space information, we augment features with depth information generated by a depth predictor. In this way, our method, PA-LoFTR, can generate 3D position-aware local feature descriptors with Transformer. We experiment on indoor datasets, and results show that PA-LoFTR improves the performance of feature matching compared to state-of-the-art methods.

Under review as a conference paper at ICLR 2023
PA-LOFTR: LOCAL FEATURE MATCHING WITH 3D POSITION-AWARE TRANSFORMER
Anonymous authors Paper under double-blind review
ABSTRACT
We propose a novel image feature matching method that utilizes 3D position in- formation to augment feature representation with deep neural network. The proposed method introduces 3D position embedding to a state-of-the-art fea- ture matcher, LoFTR, and achieves more promising performance. Following the coarse-to-fing tfne matching pipeline of LoFTR, construct Transformer-basee neural network that generates dense pixel-wise matches. Instead of using 2D po- sition embeddings for transformer, the proposed method generates 3D position embeddings that can precisely capture position correspondence tween images. Importantly, in order to guide neural network to learn 3D space information, we augment features with depth information generated by a depth predictor. In this way, method, PA-LoFTR, can generate 3D position-aware local feature descriptors with Transformer. We experiment on indoor datasets, and results show that PA-LoFTR improves the performance of feature matching compared to state-of-the-an methods.
matches be-
INTRODUCTION
Finding feature matching between images is an important task to many computer vision works, including camera calibration, structure from motion (SfM), visual localization, simultaneous local- ization and mapping (SLAM), stereo matching, etc. Generally, local feature matching problem can be solved with three stages: feature detection, feature description and feature matching. Most of ex- isting methods follow the pipeline in sequence to handle feature matching. Feature detector narrows the focus to set of interest points on images. Feature description phase generates corresponding descriptor for each interest point. Finally, correspondences between interest points of images are found by feature matching algorithms. While current methods try to extract features based on visual information from images, they pay little attention to the 3D position information of features, which limits the performance of matching.
As the development of deep learning. number of works have introduced deep architecture to im- age feature matching. Some methods develop deep networks for feature detection and some fo- feature description matching with learning process. Recently, several works have devel- oped detector-free deep architectures that can generate dense matches (Rocco et al.) 2018);][ [L] et al.) (2020); Roceo et al. (2020)). Detector-free methods can provide pixel-wise matches where feature detector can be dropped as each pixel is regarded as a potential interest point. Several detector- free methods achieve better performance for images with poor texture, complex patterns and illumination or large viewpoint change, where feature detector cannot determine enough interest points following the classic pipeline. Some methods. such as LoFTR (Sun et al., (20TT) and CoTR (Tiang al. (021), introduce Transformer into image feature matching, where deep architectures can have global re- ceptive field. Taking advantage of the ability to relate two features anywhere from the image, the methods can achieve state-of-the-art performance. However, the detector-free methods that gener- ate dense matches can still give false correspondences under challenging pose changes or repetitive patterns. Though the Transformer-base current methods are still working with local visual features extracted directly from images, and most of them pay little attention to the 3D space information of feature points.
architectures learn global relation between features,
Based on observations above, we propose 3D Position-Aware Local Feature Transformer (PA- LoFTR), novel approach that encodes space information for local feature matching. We follow

---

Under review as a conference paper at ICLR 2023
Fredlval Voua Festures
Match Refine Module
ResNet FPN
Fine-ivel
Correlation
Now Marsher
Visual forure
Mische
www.eeaeeeeeeeee............ coneanmeeett www.eeeeeee Position
Coarse blatcher
Depth Feature Generator
Differenciatio Matching
Coufilence Maini
Depti Festuree Fg.F0
Death Predector
dsconton
Transtormed Featursa Fa. PB.
Position Embedding Generator
left Jostion Firheddings
Postion Embedding
Enbestingg
Hf
Camera Parameters
Kg.
Figure I: Overview of Proposed method. Given a pair of images AA, La. PA-LoFTR uses a CNN backbone to give multi-level feature maps. Coarse-level feature maps FrA. FV are augmented by three modules: 1. Depth Feature Generator predicts dense depth map for F.A FV, and gives corre- sponding depth features FA. F 2. Position Embedding Generator prepares 3D position embed- dings P PEA, P B given camera intrinsic and extrinsic. 3. Position-Aware Transformer Encoder combines 3D position embeddings and depth features into visual features Eú F with self and cross attention layers.
the coarse-to-fine structure of LoFTR to construct deep architecture. Inspired by the usage of posi- tion encoding for Transformer, we develop a 3D position embedding generator which encodes 3D point clouds for each pixel instead of encoding 2D pixel coordinates. When self and cross attention layers learn relations between local visual features, we add 3D position embeddings at each encoder layer in Transformer to boost learning process. Additionally, to learn depth distribution of images, which can further help model learn space information. By encoding 3D position information and co-relating with depth features, PA-LoFTR learns meaning- ful features containing both visual and 3D space information that greatly help determine precise image matches. We evaluate the proposed architecture with indoor dataset on camera pose esti- mation and test effectiveness of 3D position embedding on stereo matching tasks. The experiments show that PA-LoFTR can provide high-quality matches under challenging scenarios. and achieve state-of-thed -frtc-rr- performance on some tasks. In this study, our main technique contributions are:
construct depth predictor branch
We show that a 3D position encoder serving as position embedding generator for Trans- former can greatly improve image correspondence quality determined by neural network.
We propose depth feature generator that give rough depth distributions for single image, which can help Transformer learn more space information and improve matching perfor- mance.
We demonstrate that PA-LoFTR achieves state-of-the-art performance on indoor dataset and multiple tasks.
RELATED WORKS
Classic Feature Matching Pipeline. Classic feature matching pipeline includes feature detection,
feature description and feature matching (Low) (2004); Bian al. (0017); (Sattler et al. (2009); Tuytelaars & Gool (2000).. Many methods tollow the classic feature matching pipeline to handle the image correspondence problem. SIFT (Lowe) 2004) introduces way build keypoint detector and
2

---

Under review as a conference paper at ICLR 2023
descriptor. ORB (Rublee et al. 2011) improves the efficiency to produce descriptors. Given detected features and their descriptors, RANSAC can filter outliers to provide precise matches (Fischler & Bolles 1987). Classic methods have proved their effectiveness and have been widely applied in the industry.
Feature Matching with Deep Learning Methods. As the deep learning methods spring up, the performance of feature matching has been significantly improved under challenging scenarios, such as large change of viewpoint and illumination. Several methods introduce deep architectures to im- prove one or more stages of classic feature matching pipeline. LIFT (Yi et al. 2016) constructs a CNNs-based network to build feature detector and feature descriptor. MagicPoint (Detone et all 2017) makes extra efforts to predict the corners in images. They have been proved effective in detecting keypoints with meaningful descriptors. SuperPoint (De'Tone al. 2018) presents a self- supervised framework to learn interest point detectors and descriptors based on MagicPoint, and SuperPoint applies homographic adaptation to improve the performance of feature detection. Meth- ods based on classic pipeline use metrics to optimize matches, such as nearest neighbor to search the matches. SuperGlue (Sarlin et al., 2020) introduces an attentional graph neural network to match the local features captured by feature detector and descriptor. Several methods accomplish feature matching tasks based on deep learning methods (Efe et al. (2021): Rocco et al. (2022); Zhou et al. 2021 Truong et al. (2020): Dusmanu al. (2019)).
Recently, some researchers have introduced Transformer to image correspondence problem. COTR (Jiang al., (02T) adopts the architecture of DETR (Carion et al. (220) that adopts Trans- former encoder and decoder, and it tackles image correspondence problem in the way of querying pixel positions. LoFTR (Sun et al.
2021) regards each pixel in the image as a potential keypoint and apply Transformer encoder to learn relative relations between features, and it builds a coarse- to-fine pipeline to give precise matches. STTR (Li et al., 202I) conducts experiments that show Transformer can also be applied to stereo matching. The Transformer-bast based methods for image matching have shown the advantage of the global receptive field of Transformer, which can learn relations between features anywhere from the images. LoFTR can produce high-quality matches in challenging image pairs with low-texture areas and repetitive patterns. However, the usage of 2D or ID position embeddings in methods above has drawback, where position embeddings can not represent space correspondences between images. In this study, we propose a method that improves the performance of LoFTR by introducing meaningful 3D position embedding.
Position Embedding in Transformer. Recently, Transformer- based methods have been introduced to tackle multiple computer vision tasks, including object detection (Carion al. (2020); Liu et al. Li et al. 2002): Jain et al. 2022); Zhang et al. (2022)). semantic segmentation (Miao et al.
2022):
2022
Li et al. 2022); Qian et al. (2022)), ), image feature matching (Jiang et al. (202T): Hong al.
2022
Mao et al. (2022), etc. While CNNs have the ability to aggregate features in a local window,
Transformer can have global receptive field with attention mechanism that builds relations between
features anywhere from the input sequences.
However, Transformer regards the input sequence as a set of unordered features. which leads to missing important position related information. To make use of the position information, trans- former introduces position embeddings to the input features, which significantly improve the perfor- mance of models. However, when it comes to image matching problem, the widely used 2D position embedding methods (Carion et al. (2020); Jiang et al. (222l) may serve as drawbacks. There are several methods working on improving the effectiveness of position embedding. STTR (Li et al., 2021) proves that adding position embeddings to the features will introduce a position-depended term in attention calculation, which is unrelated to image content, and STTR applies relative po- sition embedding to relieve the problem. YYfan et al. (2022) tried to encode camera intrinsic and extrinsic to make better depth prediction. PETR (Liu al. 2022) introduces 3D position embed- dings to enhance DETR3D (Wang et al. (02T). PETR transforms 3D world space to camera frustum space, and embeds 3D coordinates by multi-layer perception to produce 3D position embeddings, which further generates 3D position-awan image features. Nevertheless, recent Transformer-be based methods for image matching still pay little attention to constructing position embeddings related to 3D space. Our work introduces 3D position embeddings to Transformer encoder to generate 3D position-awan aware image features for better performance of matching.
3

---

Under review as a conference paper at ICLR 2023
LaFTR Module
Pose Extimation Module
Fest
Lef' Pootion Ereandding
MLLP
Righ:30 Fastion Enkeddig
RelU
R,
Pos
X
Right Camera Coordranes
Figure 2: Illustration of proposed 3D Position Embedding Generator.For the pair of images given, a sparse 3D coordinate cloud is generated. Each coordinate represents a position at which pixel can locate. Through camera intrinsic K;, K, and extrinsic Tir, the coordinate clouds aligned under the right camera coordinate. Then we use multi-layer perception to transform the coordinate clouds 3D position embeddings. Before inputting to Transformer Encoder, flatten operation ( is used to get position embedding sequences.
METHODS
3.1 OVERVIEW OF ARCHITECTURE
Figure gives the overall pipeline of proposed PA-LoFTR. Given pair of input images TA, T, the backbone network ResNet FPN (Lin et al. 2017a) extracts multi-level visual features. We mainly concern the feature maps 1/8 and 1/2 of original image shape, and follow LoFTR (Sun) 2021) denote them coarse-level visual features FA F and fine-level visual features al. FA. FB We follow the coarse-to-fin structure of LoFTR, where coarse-level features are mainly used to generate matches which are refined with fine-level features.
Our proposed method focuses on coarse-level matching and improves performance by augmenting FA, FV with Position-Aware Transformer Encoder. Position-Aware Transformer accepts visual features FA : FØ from backbone, depth features FAA F from depth feature generator, and 3D position embeddings as inputs, and outputs transformed features F.A. F. The transformed features then used calculate pair-wise similarity and keep coarse-level matches with high confidence. Finally, following LoFTR, the network looks into fine-level features around each pair of matches to refine final match positions.
3.2 3D POSITION EMBEDDING GENERATOR
Instead of using position embeddings from 2D pixel coordinates, we build the relation between 2D images and 3D space positions as shown in Figure 2 We first create meshgrid coordinates at the size of coarse-level feature maps, which is H. W by discretizing camera frustum space following DSGN (Chen et al. 2020) and PETR (Liu et al. 2022). We divide the depth space following lincar- increasing discretization into D positions and get meshgrid of size (He, We, D) where each cell represents a 3D position. The depth space is limited within [dman dmax) along the center axis of camera and D depth values sampled, where each depth d; can be calculated
d, = dmnn + 8 x (i + 1), where 5a dmaa d'mnn
D(D +1
Each 3D coordinate in the meshgrid P with pixel (u, v) the image can be represented as p; = (u x de, v x d,, d.) Further, as the image pair should share the same world space, the 3D coordinates grid of left image Pteft aligned with meshgrid of right image Pright The transformation to world space for point follows Eq. 2 where camera extrinsic Tj, is divided into rotation Ri, and
projected with camera intrinsic K,, K,, and extrinsic Tır where coordinate mesh-
4

---

Under review as a conference paper at ICLR 2023
translation tir. Similar to PETR, we further normalize the 3D coordinates to region of interest [mmin Tmaz: Ymnin Ymazz 2min: 2mar], which depends on the dataset.
P Rí, Ki pp: + lir, where E Pieft
3d
p7 3d K, Pr. where Pr Pright
After transformation aligned world coordinates, the point cloud matrix p&d RR XWxxx33)
is got for each image. Given the point cloud as input, a multi-layer perception (MLP) network generates 3D position embedding (PE) which is flatten to a sequence PE E [H<x...)) where C' is channel dimension. The 3D PE of each image is then input to Position-Aware Transformer Encoder which is described in section 3.4, As camera extrinsic is sometimes hard to get, we propose a pose refine branch where rough extrinsic is first predicted by LoFTR module. In more and more tasks, extrinsic can be available such as fix stereo camera or mechanical arm equipped with camera, we expect our method will be more effective in cases camera extrinsic available.
3.3 DEPTH FEATURE GENERATOR
The purpose of Depth Feature Generator is to provide additional global scene information during learning of fosition-Aware Transformer Encoder. Though predicting depth information from single image is also a challenging problem, we find it may guide network to learn better image correspon- dence. We propose a depth prediction branch after the backbone network as shown in Figure 3 Given the coarse-level feature map Fy of single image, Depth Feature Generator predicts a dense depth distribution and outputs depth features for each pixel of the feature map.
Coarse-Level feature is first going through a convolutional layer. The output features then walk through two branches where Depth Encoder extract depth features fD with single layer of Trans- former encoder, and a Depth Predictor explicitly generates a dense depth distribution map. The depth encoder applies self-attention mechanism of Transformer to learn depth features. The dense depth map is used to generate depth embeddings similar to the way in MOnoDETR (Zhang et al., 2022) and MonoDTR (Huang al. 2022). To supervise learning of depth features, we follow the similar way in 3D Position Embedding Generator where depth space is discretized with LID into kiinss + 1 bins. The first kbins represents the foreground depth values and the last one is for background. The category of ground-truth depth label can be calculated as
== -0.5 + V 0.25 + d - dmnn
where 5a dmax dman
8d
(Kbins + 1)
We use a 1 x 1 convolutional layer to predict the depth category at each pixel and get a depth value through summation of results of softmax operation. We apply Focal loss (Lin et al. 2017b) to supervise category of depth for each pixel. Given the dense depth map dmap. we further interpolate the depth value among learnable embedding pool to give depth embeddings PD. The final depth features are generated as FD fo + PD where FD IR IR((Hx..)xGG
3.4 AESITION-AWRRE TRANSFORMER ENCODER
After 3D position embedding generator and depth feature generator, Position-Aware Transformer Encoder aggregates all the information to give transformed features for matching. With position- aware feature embedding, the transformed features FA, F are more casily to match.
Here we introduce the basic idea of Transformer /Vaswani al. 2017) as background. A Trans- former encoder can contain sequence of Transformer encoder layers, and the core component of Transformer the attention layer. The attention layer accepts three types of vectors as input, which conventionally named query, key, value. The key idea is that query vector Q will extract information from value vectors based on the weights got from dot product with key vectors K where each value vector has its corresponding key vector. The dot product serves as similarity score between query and keys which enables query Q to select more relevant values. The formal equation of attention layer computes
Attn Q K,K.V V) softmax (ettmaxx(QKT) (V..
Many studies have made efforts in designing effective and efficient attention layers including multi- scale deformable attention (Zhu et al. 2020). Quad-Tree Attention (Tang et al. 2022). etc. In
5

---

Under review as a conference paper at ICLR 2023
Cuaraniicel Festures F
H, x W.
Dopth Encoder
H. MG
Cone 3x2
Attentionn
Deper PoaturesPs
RL, WZ
Summation
@map M))
Depth Tredicer
Dapth post enbed58g15
Figure 3: Depth Feature Generator in PA-LoFTR. Depth Feature Generator provides depth fea- ture for each image of image pairs. Coarse-level visual feature Fy is first transformed by a convo- lutional layer, and go into two branches. Depth predictor gives a dense depth map dmap over the feature map, and generates depth embeddings po for each pixel. 2. Depth Encoder apply single Transformer encoder layer and combine depth embeddings to generate depth features Fp-
work, we follow LoFTR and apply Linear Attention (Katharopoulos et al. 2020) the Transformer Encoder, which reduces the computation cost of Transformer to O(N) with the length of input sequences.
In proposed Position-Aware Transformer Encoder, each Transformer Encoder layer contains three stages shown in Figure [ The first cross attention layer accepts visual features Fú as queries and depth features F3 as keys and values, and here i E {A, B} means the first or second of image pair. The middle self attention layer builds relation for features across a single image, where it accepts F (FA or FB out of the first cross attention layer) as queries, keys and values. The last cross attention layer relates features between the pair of images, where Fa or FB serves as queries and FB or FA as keys and values. Most importantly, position embeddings are vital to learning of Transformer Encoder. Different from LoFTR where 2D) PE is used once at the beginning of Transformer Encoder, we apply 3D PE to the middle self attention and the last cross attention in each of the Transformer Encoder layers.
After feature Transformer Encoder, the transformed features FAA E are input to coarse-level matching module to select confident matches. A score matrix is calculated between two set of features where ji. = (F(A (i), F (j)), and we follow LoFTR to apply dual-softmax opera- tor (Sun et al. 2021) to S to obtain mutual nearest neighbor matching by the matching probability Po(i.j) softmax })) softmax (S(,.)) With coarse matches selected, coarse-to-fine mod- ule applies spatial expectation to each pair of coarse matches to get final precise matches.
3.5 IMPLEMENTATION DETAILS
Following LoFTR, we trained PA-LoFTR on the indoor dataset ScanNet DDai et al. 2017). As mentioned in LoFTR, the original LoFTR model is trained with 64 GTX 1080Ti GPUs with full ScanNet dataset containing 1513 scenes, which requires too much resources and time cost for re- searchers. In this study, we apply a lite training setting, where we split partial of ScanNet which contains 100 scenes for training with 4 GTX 1080Ti GPUs and batch size of 8, and test on the data split following Sarlin et a (2020). To boost training, we use the pretrained CNN backbone from LOFTR, and the model parameters left in the following network are initialized with random weights, and the dimension of features is set as 128 instead of 256 to save memory. We further test model on KITTI Stereo dataset (Geiger et al., 2013), where we fine-tune the PA-LoFTR trained on ScanNet. As notice that in many tasks, camera extrinsic is hard to get, which is limit of our model, implement a pose-refine version of PA-LoFTR with a simplified LoFTR branch to estimate camera extrinsic.

---

Under review as a conference paper at ICLR 2023
Method
KITTI 2015
ΣPE (px)
3px Error (%)
PSMNet (Chang & Chen 2018
6.56
27.79
GWC-Net-g (Guo et al. 2019)
2.21
12.60
AANet (Xu & Zhang 2020)
1.99
12.42
STTR (Li et al. 2021)
1.50
6.74
STTR (with our PD generator)
1.55
5.75
LoFTR
1.14
5.32
PA-LoFTR (ours)
1.04
5.19
Table I: Stereo Matching Results. Models are tested on KITTI Stereo 2015 training split, following STTR. PA-LoFTR can give sparse disparity high quality. We test STTR model with 3D PE generator, and the results can also reach state-of-the af-the-art performance.
Method
ScanNet
precision (%)
recall (%)
LoFTR
81.58
34.12
LoFTR
87.87
57.05
PA-LoFTR (ours)
93.74
49.39
Table Precision and Recall. Detector-free Models are tested on ScanNet testing split. Though trained with smaller dataset, PA-LoFTR can achieve better precision compared with LoFTR.
4 EXPERIMENTS
We experiment method on multiple tasks, including precision and recall, stereo disparity and relative pose estimation. For pose estimation, we propose a pose-refine structure where camera extrinsic first estimated and refined by PA-LoFTR get the final pose. PA-LoFTR shows that it can get matches of high quality with the help of 3D position embedding generator.
Stereo Matching. We choose KITTI dataset evaluate PA-LoFTR and the 3D Position Embedding Generator. We trained PA-LoFTR model on ScanNet following our lite training setting, and we fine-tune the model on KITTI dataset, For fair comparison, we do not tune model on KITTI Stereo dataset. Instead we fine-tune the model on KITTI object detection dataset with pseudo ground truth disparity following disp-RCNN (Sun al.) 2020). Though the model structure of PA-LoFTR is not designed for stereo matching where matches only occur on horizontal axis of images, we find PA-LoFTR can directly generate matches along horizontal lines with high quality. Compared to state-of-the-art methods targeting at disparity estimation, PA-LoFTR can provide disparity for the detected matches with high accuracy. We also apply 3D PE Generator to state-of-the- STTR Li et al. 202I) targeting at stereo matching show effectiveness, where model structure similar to ours. We fine-tuned STTR with 3D PE Generator on KITTI Stereo dataset and report metrics validation split. The results shown in Table
model
Match Evaluation. We compared PA-LoFTR with the detector-free method, LoFTR, on indoor dataset, ScanNet. As shown in Table 2] precision and recall reported where PA-LoFTR generally give matches with higher accuracy. Besides, The accuracy of camera pose estimated from the corresponding points is typically used evaluate the quality of feature correspondences. The evaluation protocol follows SuperGlue (Sarlin et al. 2020), where feature matches generated are used to solve essential matrix with RANSAC. We report the AUC of pose error at thresholds of 5", 10 20". and pose error is defined as the maximum of angular eπrоr in rotation and translation following [Sarlin et al. (2020). Camera extrinsic is used construct 3D Position Embedding in PA-LoFTR which greatly improve the quality of feature matching. In relative pose estimation task, camera extrinsic unavailable and we propose a pose-refine mechanism to PA-LoFTR. We add pre-trained LoFTR branch that generates a pose estimation first, and then PA-LoFTR constructs the 3D position embedding to get final matches. The details of implementation can be found in supplementary materials. The results in Table 3 show that PA-LoFTR with pose-refine mechanism can improve the estimated pose by a large margin compared to LoFTR under the same training setting.

---

Under review as a conference paper at ICLR 2023
(a) SP+SuperGlue
(b) LoFTRi
PA-LoFTRI (pose refine)
(c)
(d) PA-LoFTRI
(e) Depth Predicted
Figure 4: Qualitative Results. PA-LoFTR is compared to LoFTR (Sun et al. 2021 and Super- Glue (Sarlin et al. 2020) on indoor dataset, and depth map predicted is visualized. The proposed pose-refine PA-LoFTR can provide more correct matches and less false matches compared to Su- perGluee and LoFTR under challenging cases. PA-LoFTR can generate even better results where ground truth relative pose is provided to the model. The red color means epipolar error of the match is beyond x 10-4
We conduct the experiments with the lite training setting described in section 3.5 show ef- fectiveness of PA-LoFTR. As shown Table 3 PA-LoFTR can improve the performance of pose estimation compared LoFTR under the same training settings. It is noticed that method can provide accurate matches on testing dataset even trained with limited splits of ScanNet, which shows its ability of generalization. can train the model following the setting of original LoFTR, where all 1513 scenes of ScanNet are used for training and 64 GPUs are used with batch size of 64. We also test the case where
expected that PA-LoFTR can have even better performance if
Method
ScanNet
auc@5
auc @ 10
auc @ 20"
ORB+GMS (Bian et al. 2017)
5.21
13.65
25.36
SP+SuperGlue (Sarhn al. 2020)
16.16
33.81
51.84
LoFTR (Sun et al. 2021)
20.06
40.8
57.62
LoFTR
13.08
27.42
43.41
PA-LoFTRI (pose refine)
17.04
34.28
51.18
PA-LoFTRI
27.74
47.35
64.19
Table 3: Relative Pose Estimation on ScanNet dataset. The AUC pose error is reported, which reflects quality of feature matching detected. PA-LoFTR improve matching performance lot. Without camera extrinsic, PA-LoFTR can also achieve promising results compared with state-of- the-art methods. means full training setting LoFTR with 64 GPUs and batch size of 64 on full ScanNet dataset. t means our lite training setting on partial ScanNet where 4 GPUs are used with batch size of as described in section 3.5. Models are tested on consistent dataset following SuperGlue (Sarlin et al. 2020).
ground truth pose is available, and PA-LoFTR can show its full power and give high-quality feature matches. We expect that in tasks where camera extrinsic is available, 3D position embedding can provide stronger prior knowledge the model and PA-LoFTR can show more effectiveness.
Ablation Study. Compared to LoFTR, main module the 3D Position Embedding Generator which bridges the gap between camera extrinsic and feature embedding for Transformer. We eval- uate variant settings 3D Position Embedding Generator as in Table 4 1) Apply only rotation matrix of camera intrinsic to generator position embeddings, which results a drop of accuracy. 2) Apply position embeddings once at the beginning of Transformer Encoder. 3) Apply position em- beddings for self-attention layers in Transformer Encoder. We find decreasing the use of 3D position
8

---

Under review as a conference paper at ICLR 2023
Method
ScanNet
auc @
auc @ 10"
auc @uC
PA-LoFTR
27.74
47.35
64.19
R matrix for 3D PE
23.21
42.98
58.67
One-time position embedding
24.91
43,40
59.87
Position emhedding for self-attention layers
26.10
44.82
61.08
Estimated pose for position embedding
17.04
34.28
51.18
Without Depth Feature Generator
28.27
48.31
65.39
Depth feature for 3D position embedding
21.08
39.79
56.98
Position embedding for the first cross attention
27.16
46.69
63.56
Table 4: Ablation Study, Four variants of 3D PE Generator and three variants of usage of depth features by Depth Feature Generator. All models trained under lite setting Table 3 All models, except for PA-LoFTR with estimated pose for 3D PE Generator, are trained with ground truth camera extrinsic. PA-LoFTR without Depth Feature Generator performs better on testing dataset, but has lower performance (3 points decrease) on validation dataset.
described in
embedding in Transformer layers will both decrease accuracy. 4) Apply estimated pose with pre- trained LoFTR to generator 3D postion embeddings instead of using ground truth extrinsic, which is essential for pose estimation task. The usage of estimated pose will cause decrease of matching performance compared full PA-LoFTR, but can explicitly refine the estimated pose.
We also evaluate 3 variant settings related to usage of depth feature generator: 1) Do not apply depth feature generator. While leads to decrease performance on validation dataset compared to full PA-LoFTR (about 3 points decrease), the model performs better on testing dataset. The is expected to have better
Depth Feature Generator may have limited ability generalization. performance if trained with more scenes. 2) Apply depth feature in 3D PE generator, where use depth probability to guide generation of 3D PE, which does not give better performance. 3) Apply 3D position embedding at the first cross attention layer between depth features and visual features, which does not improve performance compared to giving not 3D PE to the first cross attention.
conclude that 3D position embeddings can have strong boost image matching with Transformer. When the ground truth camera extrinsic is given, strong prior scene information can be extracted from 3D PE Generator help PA-LoFTR find accurate matches and less false matches. When ground truth camera extrinsic is not available, we estimate a raw camera relative pose with pretrained LoFTR branch, and PA-LoFTR can still get enough scene information to generate better feature matches. Implementation details can be found in supplementary materials.
From the experiments,
CONCLUSION
We presents novel feature matching approach, PA-LoFTR, which introduces 3D position embed- dings to Transformer encoder to bridge the gap between 3D position information and 2D image features. The proposed method embeds the points cloud for each image as position features and applies self and cross attention layers to transform image features with 3D space information. PA- LoFTR can detect precise and plenty of feature matches even under challenging pose changes. The experiments show that PA-LoFTR can reach state-of-the-art performance, and expect PA-LoFTR can be extended to more tasks in which camera extrinsic is known, where PA-LoFTR can show its full power. Source code will be released soon.

---

Under review as a conference paper at ICLR 2023
REFERENCES
J. Bian, W. Y. Lin, Y. Matsushita, S. K. Yeung, and M. M. Cheng. Gms: Grid-based motion statistics for fast, ultra-robust feature correspondence. IEEE Conference on Computer Vision & Pattern Recognition, 2017.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. 2020.
Jia-Ren Chang and Yong-Sheng Chen. Pyramid stereo matching network. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 5410-5418, 2018.
Yilun Chen, Shu Liu, Xiaoyong Shen, and Jiaya Jia. Dsgn: Deep stereo geometry network for
3d object detection. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12533-12542. 2020.
A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and Matthias Niener. Scannet: Richly- annotated 3d reconstructions of indoor scenes. 2017.
D. Detone, Malisiewicz, and A. Rabinovich. Toward geometric deep slam. 2017.
Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superpoint: Self-super interest point detection and description. In CVPR Deep Leurning for Visual SLAM Workshop, 2018.
Mihai Dusmanu, Ignacio Rocco, Tomas Pajdla, Marc Pollefeys, Josef Sivic, Akihiko Torii, and Torsten Sattler. D2-Net: A Trainable CNN for Joint Detection and Description of Local Features. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
U. Efe, Gokalp Ince, and Aydin Alatan. Dfm: performance baseline for deep feature match- ing, arXiv e-prints, 2021.
Martin A. Fischler and Robert C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. In Martin A. Fischler and Oscar Firschein (eds.), Readings in Computer Vision, pp. 726-740. Morgan Kaufmann, San Francisco (CA), 1987. ISBN 778-0-08-051581
Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. International Journal of Robotics Research (IJRR), 2013.
Xiaoyang Guo, Kai Yang, Wukui Yang. Xiaogang Wang, and Hongsheng Li. Group-wise corre- lation stereo network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3273-3288 2019.
Sunghwan Hong, Seokju Cho, Seungryong Kim, and Stephen Lin. Integrative feature and cost aggregation with transformers for dense correspondence, 2022.
K. C. Huang, T. H. Wu, T. Su, and W. H. Hsu. Monodtr: Monocular 3d object detection with depth-aware transformer. 2022.
Achin Jain, Kibok Lee, Gurumurthy Swaminathan, Hao Yang, Bernt Schiele, Avinash Ravichan- dran, and Onkar Dabeer. Completr: Reducing the cost of annotations for object detection in dense scenes with vision transformers. 2022.
Wei Jiang, Eduard Trulls, Jan Hosang, Andrea Tagliasacchi, and Kwang Moo Yi. COTR: Corre- spondence Transformer for Matching Across Images. 2021.
A. Katharopoulos, A. Vyas, N. Pappas, and Fleuret. Transformers are mns: Fast autoregressive transformers with linear attention. 2020.
Xinghui Li, Kai Han, Shuda Li, and Victor Prisacariu. Dual-resolution correspondence networks.
In Conference Newral Information Processing Systems (NeurIPS), 2020.
10

---

Under review as a conference paper at ICLR 2023
Zhaoshuo Li, Xingtong Liu, Nathan Drenkow, Andy Ding, Francis X. Creighton, Russell H. Taylor, and Mathias Unberath. Revisiting stereo depth estimation from a sequence- sesuuenceeto-sequene to-sequence perspec- tive with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6197-6206, October 2021.
Zhikai Li, Mengjuan Chen, Junrui Xiao, and Qingyi Gu. Psaq-vit v2: Towards accurate and general data-free quantization for vision transformers, 2022.
T. Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie. Feature pyramid networks for object detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017a.
T. Y. Lin, P. Goyal, R. Girshick, K. He, and Dollár. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis & Machine Intelligence, PPX99):2999-3007, 2017b.
Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petr: Position embedding transformation for multi-view 3d object detection. arXiv preprint (rXiv:2203.05625, 2022.
D. G. Low. Distinetive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004.
D. G. Lowe. Lowe, d.g.: Distinctive image features from ccale-invariant key-points. int. j. comput. vision 60(2), 91-110 10. International Journal of Computer Vision, 60(2), 2004.
Runyu Mao, Chen Bai, Yatong An. Fengqing Zhu, and Cheng Lu. 3dg-stfm: 3d geometrie guided student-teacher feature matching, 2022.
Kevin Miao, Akash Gokul, Raghav Singh, Suzanne Petryk, Joseph Gonzalez, Kurt Keutzer, Trevor Darrell, and Colorado Reed. Prior knowledge-guiie attention in self-supervised vision trans- formers, 2022.
Guocheng Qian, Xingdi Zhang, Abdullah Hamdi, and Bernard Ghanem. Pix4point: Image pre- trained transformers for 3d point cloud understanding, 2022.
I. Rocco, M. Cimpoi, R. Arandjelović, A. Torii, T. Pajdla, and J. Sivic. Neighbourhood consensus networks. In Proceedings of the 32nd Conference on Neural Information Processing Systems, 2018.
I. Rocco, R. Arandjelović, and J. Sivic. Efficient neighbourhood consensus networks via submani- fold sparse convolutions. arXiv preprint arXiv:2004.10566 2020.
Ignacio Rocco, Mircea Cimpoi, Relja Arandjelović, Akihiko Torii, Tomas Pajdla, and Josef Sivic. Nenet: Neighbourhood consensus networks for estimating image correspondences. JEEE Trans- actions on Pattern Analysis and Machine Intelligence, 44(2):1020-1033 2022. doi: 10.1109/ TPAMM.2020.3016711
E. Rublee, V. Rabaud, K. Konolige, and G. R. Bradski. Orb: efficient alternative to sift surf. In IEEE International Conference on Computer Vision, ICCV 2011, Barcelona, Spain, November 6-13, 2011, 2011.
Paul-Edouard Sarlin, Daniel DeTone,, Tomasz Malisiewiez, and Andrew Rabinovich. SuperGlue: Learning feature matching with graph neural networks. In CVPR, 2020.
T. Sattler, B. Leibe, and L. Kobbelt. Scramsac: Improving ransac's 's efficiency with spatial consis- tency filter. In Computer Vision, 2009 IEEE 12th International Conference on, 2009.
Jaaming Sun, Linghao Chen, Yiming Xie, Siyu Zhang, Qinhong Jiang, Xiaowei Zhou, and Hujun Bao. Disp r-cnn: Stereo 3d object detection via shape prior guided instance disparity estimation. In CVPR, 2020.
Jamming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. LoFTR: Detector-free local feature matching with transformers. CVPR, 2021.
Shitao Tang, Jiahui Zhang, Siyu Zhu, and Ping Tan. Quadtree attention for vision transformers. In International Conference Learning Representations, 2022.

---

Under review as a conference paper at ICLR 2023
P. Truong, M. Danelljan, and R. Timofte. Glu-net: Global-local universal network for dense flow and correspondences. In 2020 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
T. Tuytelaars and L. V. Gool. Wide baseline stereo matching based on local, affinely invariant regions. 2000.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polo- sukhin. Attention is all you need. In arXiv, 2017.
Yue Wang, Vitor Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, and Justin M. Solomon. Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. In The Conference Robot Learning (CoRL), 2021.
Haofei Xu and Juyong Zhang. Aanet: Adaptive aggregation network for efficient stereo matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 1959-1968, 2020.
K. M. Yi, E. Trulls, V. Lepetit, and P. Fua. Lift: Learned invariant feature transform. Springer International Publishing, 2016.
Wang Yifan, Carl Doersch, Relja Arandjelović, João Carreira, and Andrew Zisserman. Input-level inductive biases for 3d reconstruction. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 6176-6186, 86, 2022.
R. Zhang, H. Qiu, T. Wang, X. Xu, Z. Guo, Y. Qiao, P. Gao, and H. Li. Monodetr: Depth-aware transformer for monocular 3d object detection. 2022.
Qunjie Zhou, Torsten Sattler, and Laura Leal-Taixe. Patch2pix: Epipolar-guided pixel-level corre- spondences. In CVPR, 2021.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection, arXiv preprint arXi::2010.04159, 2020.
12


Reviews:
[{"review_id": "rhya8mfWb3", "reviewer": "Reviewer_PA3N", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This manuscript proposes a method, PA-LoFTE, for solving feature matching between images which generally follows a detection-description-matching three-stage pipeline. In order to achieve that, the authors utilize depth information from a depth predictor to generate 3D position embedding, then combine visual features, depth features, and 3D position embeddings with transformer.\nThe discussion on the necessity of 3D space information is limited, which is the major motivation of the proposed method. The main contribution is leveraging the 3D position embedding generator for feature generation. They perform an empirical study on indoor dataset and multiple tasks and conclude the superior performance of PA-LoFTR.\n\nOverall, this paper could be a borderline paper. Given the clarification in the rebuttal, I would increase the score.\n\nFor the motivation:\n1. It should be aware depth estimation the similar challenges to correspondence prediction, why depth estimation helps correspondence prediction?\n\nSome suggestions for the experiments:\n1. Compared with LoFTR, PA-LoFTR injects 3D position information into the original Coarse-Level Local Feature Transform. The proposed method (depth feature generator and 3D position embedding generator) is effective based on LoFTR, how about adopting other baselines?\n2. In Table 1, how about the performance of LoFTR?\n3. In Table 3, Without Depth Feature Generator performs better on the test dataset while having lower performance on the validation dataset, could you please discuss this problem? \n4. Could you please give some internal estimated depth images in the depth feature generator?\n\nMinor comments:\n1. Figure 1: Depth Feature Gnerator -> Depth Feature Generator.", "summary_of_the_review": "Overall, this paper could be a borderline paper. Given the clarification in the rebuttal, I would increase the score.", "strength_and_weaknesses": "Please see summary", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "L7HkVwHfBg3", "reviewer": "Reviewer_kZ1E", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a 3D position embedding module to a state-of-the-art feature matcher, LoFTR.\n\nSpecifically, the authors develop a 3D position embedding generator that encodes 3D point clouds for each pixel instead of encoding 2D pixel coordinates.\n\nThe proposed 3D position embedding generator requires a known relative pose from GT or from a third-party relative pose estimator.\n\nExperiments on the Scannet and Kitti datasets show the effectiveness of the proposed method.", "summary_of_the_review": "The main weakness of the proposed method is relying on a known relative pose, either from GT or from a third-party estimator.\n\nThe experiments fail to show that using a third-party estimator (Lofter) would make the proposed method outperforms the third-party estimator. \n\nPlease use the same training setting as the third-party estimator.", "strength_and_weaknesses": "[Strength]\n\n1.  Using 3D position embedding to replace the original 2D pixel position embedding is interesting.\n2. Results on the KITTI Stereo 2015 validation results are good.\n3. Overall, this paper is easy to follow.\n\n[Weaknesses]\nThe main weaknesses of this paper are experiments.\n\n1. Since the proposed 3D position embedding module requires a known relative pose, the proposed method has limited applications. \n\na) Applying the proposed method to the KITTI Stereo 2015 dataset is good. However, only results on the validation set are given. Please report the results on the testing set;\nb) For the relative pose estimation task, though authors show that the proposed method can refine an estimated pose from Lofter, the performance gap between the proposed method and the original Lofter is big. To address this, I would expect:\n\ni) Exactly following the training configuration of the original Lofter, and see whether the proposed 3D position embedding module still works, i.e., can refine an estimated pose from Lofter;\n\nii) Add a baseline experiment using the original 2D pixel position embedding and see whether the proposed 3D position embedding module can obtain better performance under the authors' setting;\n\niii) Using more recent networks (e.g., QTA) to check the proposed 3D position embedding module still works;\n\niv) Using the YFCC100M and IMC2021 datasets to check whether the proposed method can work on outdoor datasets.\n\nc) For the ablation study, authors use the GT relative pose. It will make this section questionable;\n\nd) Please add an ablation study, showing the robustness of the proposed method with respect to prior relative poses at a different level of accuracy.\n\n2. [Minor] typos. For example, \n\na) Gnerator in Figure 1, \nb) Self Attention (Inter-Image) ----->>>> Self Attention (Intra-Image)\nc) Please bold matrix/vectors.\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "-mJIXoWyS1t", "reviewer": "Reviewer_iea1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a descriptor-free feature matching method based on LoFTR. Given a pair of input images, it uses a transformer architecture to calculate pairwise similarity between two sets of coarse features, and then optimize the fine-grained positions with fine-level features. The main differentiating factor compared with previous work is a 3d positional embedding generator that makes the transformer encoder aware of 3d information. A depth prediction head is also jointly learnt to provide depth encoding to aid the feature encoding. The proposed method achieves a large margin over the previous SoTA methods.\n", "summary_of_the_review": "I recommend reject due to the following reasons:\n- The proposed method should intuitively improve the results as a better positional embedding, incorporating more inductive bias such as 3d information of the input features has been demonstrated to improve the results quite well. However, the novelty is a bit hurt because this point has been shown on other domain-specific tasks. In addition, this paper is heavily based on LoFTR, the idea of using a transformer to do descriptor-free image matching.\n\n- The emphasis on the depth predictor as a contribution is a bit problematic, as the results seem to show contradictory conclusions. I suggest using an off-the-shelf depth predictor to help with this argument.\n", "strength_and_weaknesses": "Strengths:\n- The paper is written well with many ablation studies and numerical findings. The results also show that the proposed 3d positional embedding helps.\n\n- In the ablation study, the author illustrates the performance with and without the depth predicted, and presents a few failed approaches to make it work. Though the result probably would work against the proposed method in the paper, I’d appreciate this part which makes it clearer where it works better.\n\nWeakness:\n- The idea of 3d position embedding is not new. For example, this recent paper “Input-level Inductive Biases for 3D Reconstruction” described how to encode camera intrinsics and extrinsics into a transformer in positional encoding. As the paper mentions, PETR (Liu et al. 2022) also introduces a similar idea to the problem of object detection. Therefore, the application of a similar idea to the specific problem (using a transformer to do image matching) is not very novel.\n\n- It’s a bit weird that the paper contributes a big chunk to depth predictors and in the end it doesn’t generalize well to many scenarios (Sec 4.3). In addition, training the depth predictor as a joint head would also make the training harder, converge slower, and increase the overhead for the model to be deployed. It’s realistic not to assume that the depth head trained on ScanNet would generalize well on KITTI. In my opinion, if you want to try to add depths, starting with a pretrained mono-depth model would be a better route. Since it doesn’t add overhead in the training process, and possibly would generalize better since it’s trained on a large corpus of data.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject