V
ISUAL C ORRESPONDENCE H ALLUCINATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Given a pair of partially overlapping source and target images and a keypoint in the source image, the keypoint’s correspondent in the target image can be either visible, occluded or outside the ﬁeld of view.
Local feature matching methods are only able to identify the correspondent’s location when it is visible, while humans can also hallucinate ( i.e. predict) its location when it is occluded or outside the ﬁeld of view through geometric reasoning.
In this paper, we bridge this gap by training a network to output a peaked probability distribution over the correspondent’s location, regardless of this correspondent being visible, occluded, or outside the ﬁeld of view.
We experimentally demonstrate that this network is indeed able to hallucinate correspondences on pairs of images captured in scenes that were not seen at training-time.
We also apply this network to an absolute camera pose estimation problem and ﬁnd it is signiﬁcantly more robust than state-of-the-art local feature matching-based competitors.
I NTRODUCTION
Establishing correspondences between two partially overlapping images is a fundamental computer vision problem with many applications.
For example, state-of-the-art methods for visual localization from an input image rely on keypoint matches between the input image and a reference image (Sattler et al., 2018;
Sarlin et al., 2019; 2020; Revaud et al., 2019).
However, these local feature matching methods will still fail when few keypoints are covisible , i.e. when many image locations in one image are outside the ﬁeld of view or become occluded in the second image.
These failures are to be expected since these methods are pure pattern recognition approaches that seek to identify correspondences, i.e. to ﬁnd correspondences in covisible regions, and consider the noncovisible regions as noise.
By contrast, humans explain the presence of these non-covisible regions through geometric reasoning and consequently are able to hallucinate ( i.e. predict) correspondences at those locations.
Geometric reasoning has already been used in computer vision for image matching, but usually as an a posteriori processing (Fischler & Bolles, 1981;
Luong & Faugeras, 1996; Barath & Matas, 2018;
Chum et al., 2003; 2005; Barath et al., 2019; 2020).
These methods seek to remove outliers from the set of correspondences produced by a local feature matching approach using only limited geometric models such as epipolar geometry or planar assumptions.
Contributions.
In this paper we tackle the problem of correspondence hallucination.
In doing so we seek to answer two questions: ( i ) can we derive a network architecture able to learn to hallucinate correspondences?
and ( ii ) is correspondence hallucination beneﬁcial for absolute pose estimation?
The answer to these questions is the main novelty of this paper.
More precisely, we consider a network that takes as input a pair of partially overlapping source/target images and keypoints in the source image, and outputs for each keypoint a probability distribution over its correspondent’s location in the target image plane.
We propose to train this network to both identify and hallucinate the keypoints’ correspondents.
We call the resulting method NeurHal, for Neural Hallucinations.
To the best of our knowledge, learning to hallucinate correspondences is a virgin territory, thus we ﬁrst provide an analysis of the speciﬁc features of that novel learning task.
This analysis guides us towards employing an appropriate loss function and designing the architecture of the network.
After training the network, we experimentally demonstrate that it is indeed able to hallucinate correspondences on unseen pairs of images captured in novel scenes.
We also apply this network to a camera pose estimation problem and ﬁnd it is signiﬁcantly more robust than state-of-the-art local feature matching-based competitors.
= Identified =
Inpainted =
Outpainted
R ELATED WORK
To the best of our knowledge, aiming at hallucinating visual correspondences has never been done but the related ﬁelds of local feature description and matching are immensely vast and we focus here only on recent learning-based approaches.
Learning-based local feature description.
Using deep neural networks to learn to compute local feature descriptors have shown to bring signiﬁcant improvements in invariance to viewpoint and illumination changes compared to handcrafted methods (Csurka & Humenberger, 2018;
Gauglitz et al., 2011; Salahat & Qasaimeh, 2017; Balntas et al., 2017).
Most methods learn descriptors locally around pre-computed covisible interest regions in both images (Yi et al., 2016; Detone et al., 2018;
Balntas et al., 2016a; Luo et al., 2019), using convolutional-based siamese architectures trained with a contrastive loss (Gordo et al., 2016; Schroff et al., 2015; Balntas et al., 2016b; Radenovi´c et al., 2016; Mishchuk et al., 2017; Simonyan et al., 2014), or using pose (Wang et al., 2020; Zhou et al., 2021) or self (Yang et al., 2021) supervision.
To further improve the performances, (Dusmanu et al., 2019; Revaud et al., 2019) propose to jointly learn to detect and describe keypoints in both images, while Germain et al.
(2020) only detects in one image and densely matches descriptors in the other.
Learning-based local feature matching.
All the methods described in the previous paragraph establish correspondences by comparing descriptors using a simple operation such as a dot product.
Thus the combination of such a simple matching method with a siamese architecture inevitably produces outlier correspondences, especially in non-covisible regions.
To reduce the amount of outliers, most approaches employ so-called Mutual Nearest Neighbor (MNN) ﬁltering.
However, it is possible to go beyond a simple MNN and learn to match descriptors.
Learning-based matching methods (Zhang et al., 2019; Brachmann & Rother, 2019; Moo Yi et al., 2018; Sun et al., 2020;
Choy et al., 2020; 2016) take as input local descriptors and/or putative correspondences, and learn to output correspondences probabilities.
However, all these matching methods focus only on predicting correctly covisible correspondences.
Jointly learning local feature description and matching.
Several methods have recently proposed to jointly learn to compute and match descriptors (Sarlin et al., 2020; Sun et al., 2021;
Li et al., 2020; Rocco et al., 2018; 2020).
All these methods use a siamese Convolutional Neural Network (CNN) to obtain dense local descriptors, but they signiﬁcantly differ regarding the way they establish matches.
They actually fall into two categories.
The ﬁrst category of methods (Li et al., 2020; Rocco et al., 2018; 2020) computes a 4D correlation tensor that essentially represents the scores of all the possible correspondences.
This 4D correlation tensor is then used as input to a second network that learns to modify it using soft-MNN and 4D convolutions.
Instead of summarizing all the information into a 4D correlation tensor, the second category of methods (Sarlin et al., 2020; Sun et al., 2021) rely on Transformers (Vaswani et al., 2017; Dosovitskiy et al., 2020; Ramachandran et al., 2019;
Caron et al., 2021; Cordonnier et al., 2020; Zhao et al., 2020; Katharopoulos et al., 2020) to let the descriptors of both images communicate and adapt to each other.
All these methods again focus on identifying correctly covisible correspondences and consider non-covisible correspondences as noise.
While our architecture is closely related to the second category of methods as we also rely on Transformers, the motivation for using it is quite different since it is our goal of hallucinating correspondences that calls for a non-siamese architecture (see Sec.3).
Visual content hallucination.
(Yang et al., 2019) proposes to hallucinate the content of RGB-D scans to perform relative pose estimation between two images.
More recently (Chen et al., 2021) regresses distributions over relative camera poses for spherical images using joint processing of both images.
The work of (Yang et al., 2020; Qian et al., 2020; Jin et al., 2021) shows that employing a hallucinate-then-match paradigm can be a reliable way of recovering 3D geometry or relative pose from sparsely sampled images.
In this work, we focus on the problem of correspondence hallucination which unlike previously mentioned approaches does not aim at recovering explicit visual content or directly regressing a camera pose.
Perhaps closest to our goal is Cai et al.
(2021) that seeks to estimate a relative rotation between two non-overlapping images by learning to reason about “hidden” cues such as direction of shadows in outdoor scenes, parallel lines or vanishing points.
O UR APPROACH
Our goal is to train a network that takes as input a pair of partially overlapping source/target images and keypoints in the source image, and outputs for each keypoint a probability distribution over its correspondent’s location in the target image plane, regardless of this correspondent being visible, occluded, or outside the ﬁeld of view.
While the problem of learning to ﬁnd the location of a visible correspondent received a lot of attention in the past few years (see Sec. 2), to the best of our knowledge, this paper is the ﬁrst attempt of learning to ﬁnd the location of a correspondent regardless of this correspondent being visible, occluded, or outside the ﬁeld of view.
Since this learning task is virgin territory, we ﬁrst analyze its speciﬁc features below, before deﬁning a loss function and a network architecture able to handle these features.
3. A NALYSIS OF THE PROBLEM
The task of ﬁnding the location of a correspondent regardless of this correspondent being visible, occluded, or outside the ﬁeld of view actually leads to three different problems.
Before stating those three problems, let us ﬁrst recall the notion of correspondent as it is the keystone of our problem.
Correspondent.
Given a keypoint p S ∈ R 2 in the source image I S , its depth d S ∈ R + , and the relative camera pose R TS ∈ SO(3) , t TS ∈ R 3 between the coordinate systems of I S and the target image I T , the correspondent p T ∈ R 2 of p S in the target image plane is obtained by warping p S : p T
[Equation]
p
[Equation]
R
[Equation] [Equation]
d
[Equation]
R TS K −
[Equation]
p
[Equation]
TS (cid:1) , where K S and K T are the camera calibration matrices of source and target images and π ( u ) :=
[ u x / u z , u y / u z , 1] T is the projection function.
In a slight abuse of notation, we do not distinguish a homogeneous 2D vector from a non-homogeneous 2D vector.
Let us highlight that the correspondent p T of p S may not be visible , i.e. it may be occluded or outside the ﬁeld of view.
Identifying the correspondent.
In the case where a network has to establish a correspondence between a keypoint p S in I S and its visible correspondent p T in I T , standard approaches, such as comparing a local descriptor computed at p S in I S with local descriptors computed at detected keypoints in I T , are applicable to identify the correspondent p T .
Outpainting the correspondent.
When p T is outside the ﬁeld of view of I T , there is nothing to identify, i.e. neither can p T be detected as a keypoint nor can a local descriptor be computed at that location.
Here the network ﬁrst needs to identify correspondences in the region where I T overlaps with I S and realize that the correspondent p T is outside the ﬁeld of view to eventually outpaint it (see
Fig. 1).
We call this operation "outpainting the correspondent" as the network needs to predict the location of p T outside the ﬁeld of view of I T .
Inpainting the correspondent.
When p T is occluded in I T , the problem is even more difﬁcult since local features can be computed at that location but will not match the local descriptor computed at p S in I S .
As in the outpainting case, the network needs to identify correspondences in the region where I T overlaps with I S and realize that the correspondent p T is occluded to eventually inpaint the correspondent p T (see Fig. 1).
We call this operation "inpainting the correspondent" as the network needs to predict the location of p T behind the occluding object.
Let us now introduce a loss function and an architecture that are able to unify the identifying, inpainting and outpainting tasks.
3. L OSS FUNCTION
The distinction we made between the identifying, inpainting and outpainting tasks come from the fact that the source image I S and the target image I T are the projections of the same 3D environment from two different camera poses.
In order to integrate this idea and obtain a uniﬁed correspondence learning task, we rely on the Neural Reprojection Error (NRE) introduced by (Germain et al., 2021).
In order to properly present the NRE, we ﬁrst recall the notion of correspondence map .
Correspondence map.
Given I S , I T and a keypoint p S in the image plane of I S , the correspondence map C T of p S in the image plane of I T is a 2D tensor of size H C × W C such that C T ( p T ) :
= p ( p T | p S , I S , I T ) is the likelihood of p T being the correspondent of p S .
The likelihood can only be evaluated for p T ∈ Ω C T where Ω C T is the set of all the pixel locations in C T .
Here, we implicitly deﬁned that the likelihood of p T falling outside the boundaries of C T is zero.
In practice, a correspondence map C T is implemented as a neural network that takes as input p S , I S and I T , and outputs a softmaxed 2D tensor.
A correspondence map C T may not have the same number of lines and columns than I T especially when the goal is to outpaint a correspondence.
Thus, in the general case, to transform a 2D point from the image plane of I T to the correspondence plane of C T , we will need another afﬁne transformation matrix K C .
Let us highlight that this likelihood is obtained using the visual content of
I S and I T only.
Neural Reprojection Error.
The NRE (Germain et al., 2021) is a loss function that warps a keypoint p S into the image plane of I T and evaluates the negative log-likelihood at this location.
In our context, the NRE can be written as:
[Equation]
In general, x T does not have integer coordinates and the notation ln C T ( x T ) corresponds to performing a bilinear interpolation after the logarithm.
For more details concerning the derivation of the NRE, the reader is referred to Germain et al.
The NRE provides us with a framework to learn to identify, inpaint or outpaint the correspondent of p S in I T in a uniﬁed manner since Eq.
(1) is differentiable w.r.t.
C T and there is no assumption regarding covisibility.
The main difﬁculty to overcome is the deﬁnition of a network architecture able to output a consistent C T being given only p S , I S and I T as inputs, i.e. the network must ﬁgure out whether the correspondent of p S in I T can be identiﬁed or has to be inpainted or outpainted.
3. N ETWORK ARCHITECTURE
The analysis from Sec.
3.1 and the use of the NRE as a loss (Sec. 3.2) call for:
• a non-siamese architecture to be able to link the information from I S with the information from I T
to outpaint or inpaint the correspondent if needed; • an architecture that outputs a matching score for all the possible locations in I T as well as locations beyond the ﬁeld of view of I T as the network could decide to identify, inpaint or outpaint a correspondent at these locations
To fulﬁll these requirements, we propose the following: Our network takes as input I S and I T as well as a set of keypoints { p S ,n } n =1 ...N in the source image plane of I S .
A siamese CNN backbone is applied to I
S and I T to produce compact dense local descriptor maps H S and H T .
In order to be able to outpaint correspondents in the target image plane, we pad H T with a learnable ﬁxed vector λ .
This padding step allows to initialize descriptors at locations outside the ﬁeld of view of I T .
We note γ the relative output-to-input correspondence map resolution ratio.
The dense descriptor maps H S and H T , pad , and the keypoints { p S ,n } n =1 ...
N are then used as inputs of a cross-attention-based backbone F with positional encoding.
This part of the network outputs a feature vector d S ,n for each keypoint p S ,n and dense feature vectors D T , pad of the size of H T , pad .
This cross-attention-based backbone allows the local descriptors H S and H T , pad to communicate with each other.
Thus, during training, the network will be able to leverage this ability to communicate, to learn to hallucinate peaked inpainted and outpainted correspondence maps.
[Figure]
F
[Figure]
The correspondence map C T ,n of p S ,n in the image plane of I T is computed by applying a 1 × 1 convolution to
D T , pad using d S ,n as ﬁlter, followed by a 2D softmax.
An overview of our architecture, that we call NeurHal, is presented in
Fig.
In practice, in order to keep the required amount of memory and the computational time reasonably low, the correspondence maps { C T ,n } n =1 ...
N have a low resolution, i.e. for a target image of size 640 × 480 , we use a CNN with an effective stride of s = 8 and consequently the resulting correspondence maps (with γ = 50% ) are of size 160 × 120 .
Producing low resolution correspondence maps prevents
NeurHal from predicting accurate correspondences.
But as we show in the experiments, this low resolution is sufﬁcient to hallucinate correspondences and have an afﬁrmative answer to both questions: (i) can we derive a network architecture able to learn to hallucinate correspondences?
and (ii) is correspondence hallucination beneﬁcial for absolute pose estimation?
Thus, we leave the question of the accuracy of hallucinated correspondences for future research.
Additional details concerning the architecture are provided in Sec.
C.1 of the appendix.
3. T RAINING - TIME
Given a pair of partially overlapping images ( I S , I T ) , a set of keypoints with ground truth depths { p S ,n , d S ,n } n =1 ...N as well as the ground truth relative camera pose ( R TS , t TS ) , the corresponding sum of NRE terms (Eq. 1) can be minimized w.r.t.
the parameters of the network that produces the correspondence maps.
Thus, we train our network using stochastic gradient descent and early stopping by providing pairs of overlapping images along with the aforementioned ground truth information.
Let us also highlight that there is no distinction in the training process between the identifying, inpainting and outpainting tasks since the only thing our network outputs are correspondence maps.
Moreover there is no need for labeling keypoints with ground truth labels such as "identify/visible", "inpaint/occluded" or "outpaint/outside the ﬁeld of view".
Additional information concerning the training are provided in Sec.
C.2 of the appendix.
3. T EST - TIME
At test-time, our network only requires a pair of partially overlapping images ( I S , I T ) as well as keypoints { p S ,n } n =1 ...N in I S , and outputs a correspondence map C T ,n in the image plane of I T for each keypoint, regardless of its correspondent being visible, occluded or outside the ﬁeld of view.
[Figure]
M
[Figure]
a d e p t
[Figure]
E XPERIMENTS
In these experiments we seek to answer two questions: 1) "Is the proposed NeurHal approach presented in Sec.
3 capable of hallucinating correspondences?"
and 2) "
In the context of absolute camera pose estimation, does the ability to hallucinate correspondences bring further robustness?".
4. E VALUATION OF THE ABILITY TO HALLUCINATE CORRESPONDENCES
We evaluate the ability of our network to hallucinate correspondences on four datasets: the indoor datasets ScanNet
(Dai et al., 2017) and NYU (Nathan Silberman & Fergus, 2012), and the outdoor datasets MegaDepth (Li & Snavely, 2018) and ETH-3D (Schöps et al., 2017).
For the indoor setting (outdoor setting, respectively), we train NeurHal on ScanNet (Megadepth, respectively) on the training scenes as described in Sec. 3.4, and evaluate it on the disjoint set of validation scenes.
Thus, all the qualitative and quantitative results presented in this section cannot be ascribed to scene memorization.
For each dataset, we run predictions over 2 , 500 source and target image pairs sampled from the test set, with overlaps between 2% and 80% .
For every image pair, we also feed as input to
NeurHal keypoints in the source image.
These keypoints have known ground truth correspondents in the target image and labels (visible, occluded, outside the ﬁeld of view) that we use to evaluate the ability of our network to hallucinate correspondences.
For more details on the settings of our experiment see Sec.
C.2.
For this experiment, we use γ = 50% .
We report in Fig.
3 two histograms computed over more than one million keypoints for each task we seek to validate: identiﬁcation, inpainting, and outpainting.
The ﬁrst histogram Fig.
3 (left) is obtained by evaluating for each correspondence map the NRE cost (Eq. 1) at the ground truth correspondent’s location.
In order to draw conclusions, we also report the negative log-likelihood of a uniform correspondence map ( ln | Ω C T | ).
We ﬁnd that for each task and for both datasets, the predicted probability mass lies signiﬁcantly below ln | Ω C T | , which demonstrates NeurHal’s ability to perform identiﬁcation, inpainting and outpainting.
On ScanNet, we also observe that identiﬁcation is a simpler task than outpainting while inpainting is the hardest task:
On average, the NRE cost of inpainted correspondents is higher than the average NRE cost of outpainted correspondents, which indicates the predicted correspondence maps are less peaked for inpainting than they are for outpainting.
This corroborates what we empirically observed on qualitative results in Fig. 1, and supports our analysis in Sec. 3.1.
On Megadepth, outpainting and inpainting histograms have a similar shape which does
[Figure]
not reﬂect the previous statement, but we believe this is due to the fact that inpainting labels are noisy for this dataset, as explained in Sec.
C.2.
On the right histogram of Fig.
3, we report the distribution of the distance between the argmax of a correspondence map and the ground truth correspondent’s location.
We also report the average error of a random prediction.
We ﬁnd the histogram mass lies signiﬁcantly to the left of the random prediction average error, indicating our model is able to place modes correctly in the correspondence maps, regardless of the task at hand.
On ScanNet, we observe that the inpainting and outpainting histograms are very similar, indicating the predicted argmax is equally good for both tasks.
As mentioned above, the correspondence maps produced by NeurHal have a low resolution (see Sec. 3.3) which explains why the "argmax error" is not closer to zero pixel.
In Fig.
4, we compare the hallucination performances of NeurHal against state-of-the-art local feature matching methods.
Since all these local feature matching methods were designed and trained on pairs of images with signiﬁcant overlap to perform only identiﬁcation, they obtain poor inpainting results.
Concerning the outpainting task, these methods seek to ﬁnd a correspondent within the image boundaries, consequently they cannot outpaint correspondences and obtain very poor results.
In Fig.
5 we show several qualitative inpainting/outpainting results on ScanNet and MegaDepth datasets.
In the appendix, we also report qualitative results obtained on the NYU Depth dataset (Fig. 15) and on the ETH-3D dataset (Fig. 14).
These results allow us to conclude that NeurHal is able to hallucinate correspondences with a strong generalization capacity.
Additional experiments concerning the ability to hallucinate correspondences are provided in Sec.
A as well as technical details regarding the evaluation protocol in Sec.
C.3.
4. A PPLICATION TO ABSOLUTE CAMERA POSE ESTIMATION
In the previous experiment, we showed that our network is able to hallucinate correspondences.
We now evaluate whether this ability helps improving the robustness of an absolute camera pose estimator.
We run this evaluation on the test set of ScanNet over 2,500 source and target image pairs captured in scenes that were not used at training time.
For each source/target image pair, we employ NeurHal to produce correspondence maps.
As in the previous experiment, we use γ = 50% .
Given these correspondence maps and the depth map of the source image, we estimate the absolute camera pose between the target image and the source image using the method proposed in Germain et al. (2021).
In Fig.
6, we show the results of an ablation study conducted on ScanNet.
In this study, we focus on the robustness of the camera pose estimate for various combinations of training data, i.e. we consider a pose is "correct" if the rotation error is lower than 20 degrees and the translation error is below 1.
meters (see Sec.
C.3).
We ﬁnd that training our network to perform the three tasks (identiﬁcation, inpainting, and outpainting) produces the best results.
In particular, we ﬁnd that adding outpainting plays a critical role in improving localization of low-overlap image pairs.
We also ﬁnd that learning to inpaint does not bring much improvement to the absolute camera pose estimation.
In Fig.
7, we compare the results of NeurHal against state-of-the-art local feature matching methods.
In low-overlap settings, very few keypoints’ correspondents can be identiﬁed and many keypoints’ correspondents have to be outpainted.
In this case, we ﬁnd that NeurHal is able to estimate the camera pose correctly signiﬁcantly more often than any other method, since NeurHal is the only method able to outpaint correspondences (see Fig. 4).
For high-overlap image pairs, the ability to
ScanNet
Megadepth - l n
C T
T a r g e t
S ou r ce - l n
C T
T a r g e t
S ou r ce hallucinate is not useful since many keypoints’ correspondents can be identiﬁed.
In this case, we ﬁnd that state-of-the-art local feature matching methods to be slightly better than NeurHal.
This is likely due to the fact that NeurHal outputs low resolution correspondences maps while the other methods output high resolution correspondences.
The overall performance shows that NeurHal signiﬁcantly outperforms all the competitors, which allows us to conclude that the ability of NeurHal to outpaint correspondences is beneﬁcial for absolute pose estimation.
Technical details concerning the previous experiment as well as additional experiments concerning the application to absolute camera pose estimation are provided in Sec. B).
L IMITATIONS
We identiﬁed the following limitations for our approach: ( i ) -
The previous experiments showed that
NeurHal is able to inpaint correspondences but the inpainted correspondence maps are much less peaked compared to the outpainted correspondence maps.
This is likely due to the fact that inpainting correspondences is much more difﬁcult than outpainting correspondences (see Sec 3.1).
( ii ) -
The proposed architecture outputs low resolution correspondence maps (see Sec. 3.3), e.g. 160 × 120 for input images of size 640 × 480 and an amount of padding γ = 50% .
This is essentially due to the quadratic complexity of attention layers we use (see Sec.
C.1 of the appendix).
( iii ) -
Our approach is able to outpaint correspondences but our correspondence maps have a ﬁnite size.
Thus, in the case where a keypoint’s correspondent falls outside the correspondence map, the resulting correspondence map would be erroneous.
We believe these three limitations are interesting future research directions.
[Figure]
Method
Training correspondences
Identiﬁed
Inpainted
Outpainted (cid:88) (cid:88) (cid:88) (cid:88) (cid:88) (cid:88) (cid:88) (cid:88)
[Figure]
of images that have an overlap
between 2% and x % , as a function of x , for two rotation and translation error thresholds.
See discussion in Sec.
C ONCLUSION
To the best of our knowledge, this paper is the ﬁrst attempt to learn to inpaint and outpaint correspondences.
We proposed an analysis of this novel learning task, which has guided us towards employing an appropriate loss function and designing the architecture of our network.
We experimentally demonstrated that our network is indeed able to inpaint and outpaint correspondences on pairs of images captured in scenes that were not seen at training-time, in both indoor (ScanNet) and outdoor (Megadepth) settings.
We also tested our network on other datasets (ETH3D and NYU) and discovered that our model has strong generalization ability.
We then tried to experimentally illustrate that hallucinating correspondences is not just a fundamental AI problem but is also interesting from a practical point of view.
We applied our network to an absolute camera pose estimation problem and found that hallucinating correspondences, especially outpainting correspondences, allowed to signiﬁcantly outperform the state-of-the-art feature matching methods in terms of robustness of the resulting pose estimate.
Beyond this absolute pose estimation application, this work points to new research directions such as integrating correspondence hallucination into Structure-from-Motion pipelines to make them more robust when few images are available.
E THICS S TATEMENT
The method described in this paper has the potential to greatly improve many computer vision-based industrial applications, especially those involving visual localization in GPS-denied or cluttered environments.
For example robotics or augmented reality applications could beneﬁt from our algorithm to better relocalize within their surroundings, which could lead to more reliable and overall safer behaviours.
If this was to be applied to autonomous driving or drone-based search and rescue, one could appreciate the positive societal impact of our method.
On the other hand like many computer vision algorithms, it could be applied to improve robustness of malicious devices such as weaponized UAVs, or invade citizens privacy through environment re-identiﬁcation.
Thankfully as
AI technology advances, discussions and regulations are brought forward by governments and public entities.
These ethical debates pave the way for a brighter future and can only make us think NeurHal will more bring beneﬁts than harms to society.
R EPRODUCIBILITY
We provide the NeurHal model architecture and weights in the supplementary material.
We also release a simple evaluation script that generates qualitative results and show in a notebook the results obtained on an image pair captured indoors using a smartphone.
A PPENDIX
In the following pages, we present additional experiments and technical details about our visual correspondence hallucination method NeurHal.
We present additional experiments on the ability to hallucinate in Sec.
A and on camera pose estimation in Sec.
B. We describe technical details in
Sec.
C and provide additional qualitative results in Sec.
D.
A Additional experiments concerning the ability to hallucinate correspondences
A.1 Impact of learning to inpaint and outpaint
A.2 Additional insights on correspondence hallucination
A.3 Inpainted vs. outpainted correspondence maps
B Additional experiments concerning the application to camera pose estimation
B.1 Inﬂuence of the pose estimator: (Germain et al., 2021) vs. (Chum et al., 2003) .
B.2 Impact of the value of γ .
B.3 Analysis of the impact of inpainting and outpainting
B.4 Additional indoor pose estimation results
C
Technical
details
C.1 Architecture details
C.2 Datasets and Training details
C.3 Evaluation Details .
D Additional qualitative results
D.1 Generalization to new datasets
[Table] [Table]
A A DDITIONAL EXPERIMENTS CONCERNING THE ABILITY TO HALLUCINATE CORRESPONDENCES
In this section we ﬁrst present an additional ablation study on the ability to hallucinate, followed by additional insights on our model internal functioning.
A. I MPACT OF LEARNING TO INPAINT AND OUTPAINT
To supplement the study made in Sec. 4.1, we now aim at evaluating the impact of learning to inpaint and outpaint speciﬁcally.
To do so, we isolate keypoints with the identiﬁed , inpainted and outpainted labels in our ScanNet (Dai et al., 2017) evaluation set.
In Fig. 8, we show the results of an ablation study on NeurHal’s training setup.
We report for the identiﬁcation, inpainting and outpainting tasks two sets of cumulative histograms: 1) the NRE costs at ground truth keypoint correspondents’ locations, and 2) distances between the argmax of the correspondence map and the ground truth location.
On NRE cost cumulative histograms, we also report the results from the uniform distribution, for models trained both with and without outpainting ( γ = 0% and γ = 50% respectively).
For the identiﬁcation task (Fig. 8 (a))
we ﬁnd that all methods yield a consistent performance.
The left ﬁgure reveals that NeurHal predictions are signiﬁcantly above the uniform distribution, indicating peaky maps and thus conﬁdent predictions.
The right ﬁgure shows that the distance of the argmax location w.r.t.
the ground truth is also robust (NeurHal predicts at 1/8th of the original resolution but the histogram is computed at full resolution).
For the inpainting task (Fig. 8 (b)) we can draw similar conclusions.
We ﬁnd however that correspondence maps are overall less peaky and closer to their respective uniform distribution, which indicates that predictions are less conﬁdent.
We also ﬁnd that even though it was not trained to inpaint, the identiﬁcation baseline is surprisingly able to inpaint correspondences as its performance is not far from the identiﬁcation+inpainting model.
Lastly for the outpainting task (Fig. 8(c)), we ﬁnd that learning to outpaint gives a signiﬁcant boost in performance on both the NRE distribution and correspondents locations.
We also ﬁnd that jointly learning to inpaint and outpaint is beneﬁcial to the quality of the outpainted cost maps, which implies that both objectives are complementary.
A. A DDITIONAL INSIGHTS ON CORRESPONDENCE HALLUCINATION
While it is tempting to draw hypotheses regarding the internal functioning of NeurHal, we would like to highlight that this should be done with great care.
Indeed, the sheer complexity of the operations run in the many attention layers of NeurHal prevents from interpreting the reasoning that leads to the outputted correspondence maps.
From a higher level however, we can reasonably speculate that our model implicitly learns to jointly predict the depth of the source image and the relative camera pose to warp the source keypoints and hallucinate their correspondents in the target image.
A. I NPAINTED VS . OUTPAINTED CORRESPONDENCE MAPS
We can observe that the inpainted correspondence maps are less peaked than the outpainted correspondence maps (see Fig. 3).
We believe this is because outpainting a correspondent essentially consists in transferring the features from location p s,i in the source features to location p t,i in the target features to obtain d s,i = D T,pad ( p t,i ) .
In the inpainting case, p s,i is occluded by an object in the target image, and this object is (often) also visible in the source image at location p s,occ .
Thus, to produce peaked correspondence maps for both p s and p s,occ , the network has to output features such that d s,i = D T,pad ( p t,i ) =
d s,occ which is more difﬁcult than just d s,i = D T,pad ( p t,i )
B A DDITIONAL EXPERIMENTS CONCERNING THE APPLICATION TO CAMERA POSE ESTIMATION
In this section, we present additional experiments on correspondence hallucination for camera pose estimation.
We begin with a study on the impact of the pose estimator in Sec.
B.1, followed by a study on the impact of the padding value γ in Sec.
B.2.
Lastly, we present in Sec.
B.4 additional results on indoor camera pose estimation.
B. I NFLUENCE OF THE POSE ESTIMATOR : (G ERMAIN ET AL ., 2021) VS . (C HUM ET AL ., 2003)
(Germain et al., 2021) provides a pose estimation framework which leverages dense keypoint matching uncertainties to predict more accurate and robust camera poses.
Compared to the standard pose estimator presented in (Chum et al., 2003) which relies on sparse 2D-to-3D correspondences, the method from (Germain et al., 2021) preserves rich information in the form of dense loss maps that is particularly suited for ambiguous matches.
For the problem of correspondence hallucination we ﬁnd the loss maps of both outpainted and inpainted correspondences are usually unimodal but quite diffuse, and are thus particularly suited for this pose estimator.
To study the inﬂuence of the pose estimator, we report in Fig.
9 the performance of NeurHal + (Germain et al., 2021) vs. NeurHal
+ (Chum et al., 2003).
To estimate the camera pose using the method presented in (Chum et al., 2003), we simply take the argmax of each correspondence map and treat it as a sparse 2D correspondent in the query image.
We also include the performance of NeurHal when trained without visual correspondence hallucination ( i.e. trained using only identiﬁed ground truth correspondences.)
We ﬁnd that the two methods trained without hallucination have poor performances for very lowoverlap image pairs which underlines the importance of correspondence hallucination in such cases.
[Figure] [Figure]
[Figure] [Figure]
Method
Hallucination
Pose estimator (Chum et al., 2003) (cid:88) (Chum et al., 2003)
(Germain et al., 2021) (cid:88) (Germain et al., 2021)
[Figure]
Method γ γ = 50 % γ = 25 % γ = 10 % γ = 0 %
Concerning NeurHal trained with hallucination and using the pose estimator (Chum et al., 2003), taking the argmax of a very coarse correspondence map prevents the pose estimator from achieving good results.
NeurHal trained with hallucination and coupled with the pose estimator of Germain et al.
(2021) achieves the best results which shows that to obtain robust absolute camera estimates it is important to combine the ability to hallucinate correspondences of NeurHal with the pose estimator from (Germain et al., 2021).
B. I MPACT OF THE VALUE OF γ
We report in Fig.
10 the absolute camera pose estimation performance for varying values of γ .
We compute the percentage of camera poses being correctly estimated for ScanNet (Dai et al., 2017) test images pairs that have an overlap between 2% and x % (as a function of x ) for a translation threshold of 1 . 5 m and a rotation threshold of 20 . 0 ◦ .
We ﬁnd that using only a small percentage of outpainting such as γ = 10% does not improve the performance which is most likely due to the small amount of added training keypoints.
For higher γ values however signiﬁcant gains are visible, especially at small visual overlaps.
This experiment
[Figure]
demonstrates the beneﬁt of learning to outpaint correspondences beyond image borders
[Figure]
and broaden the extent of usable source keypoints to perform camera pose estimation.
We report in Fig.
11 the camera ﬁeld-of-view as a function of the padding parameter.
We ﬁnd that γ = 50% provides 130 ◦ and 71 ◦ of ﬁeld-of-view on average on ScanNet and Megadepth respectively, which is signiﬁcantly wider than γ = 0% .
B. A NALYSIS OF THE IMPACT OF INPAINTING AND OUTPAINTING
In Fig.
10 we reported the percentage of camera poses being correctly estimated for several values of γ , which demonstrates the beneﬁts of outpainting with a large γ for camera pose estimation.
In
Fig.
6 we also showed that learning to inpaint does not bring any signiﬁcant improvement.
We believe that outpainting improves the camera pose because outpainted correspondences are outside the ﬁeld of view and thus complement the identiﬁed correspondences, and thus better constrain the camera pose estimate.
On the contrary, inpainted correspondences are usually surrounded by identiﬁed correspondences, thus the information they provide is redundant and does not allow to better constrain the camera pose estimate.
B. A DDITIONAL INDOOR POSE ESTIMATION RESULTS
In addition to the results presented in Fig.
7, we report in Fig.
12 the performance of NeurHal and state-of-the-art feature matching methods on ScanNet (Dai et al., 2017) image pairs with visual overlaps between 2% and 5%.
For every method, we subselect the 25% of images pairs with the
Threshold (°)
Threshold (m)
P r e c
i s i o n
P r e c
i s i o n
Identity 200.
Threshold (°)
Threshold (m)
R2D  200.
Threshold (°)
Threshold (m)
SP+SG 200.
Threshold (°)
Threshold (m)
LoFTR 200.
Threshold (°)
Threshold (m)
DRCNet 200.
Threshold (°)
Threshold (m)
S2D 200.
Threshold (°)
Threshold (m)
NeurHal
[Figure]
(Revaud et al., 2019)
SP+SG (Sarlin et al., 2020)
LoFTR (Sun et al., 2021)
[Figure]
worst NeurHal predictions state-of-the-art methods achieve a much lower performance, which is either on par or lower than the predictions obtained using the
Identity.
[Figure]
Identity
R2D2 (Revaud et al., 2019)
SP+SG (Sarlin et al., 2020)
[Figure]
(Sun et al., 2021)
DRCNet (Li et al., 2020)
S2D (Germain et al., 2021)
[Figure]
worst predictions, and compare it with the performance of its competitors.
We ﬁnd that in all cases,
NeurHal strongly outperforms its competitors.
On the worst NeurHal predictions, state-of-the-art methods achieve a much lower performance.
For this category we can observe that all NeurHal competitors are either on par or achieve a lower performance than the Identity predictions.
This ﬁgure highlights the fact that when NeurHal fails to correctly estimate the camera pose, all the competitors also fail since all the methods perform similarly to the "identity" method, i.e. the method that consists in systematically predicting the identity pose.
Fig.
13 shows that NeurHal is much more robust than state-of-the-art local feature matching methods for pairs of images with a low overlap.
C T ECHNICAL DETAILS C. A RCHITECTURE DETAILS
NeurHal’s architecture can be separated in two building blocks: the convolutional backbone and the multi-head attention block.
Convolutional backbone.
The convolutional backbone consists of a truncated Inceptionv3 (Szegedy et al., 2016) model (up to Mixed-6a, 768-dimensional descriptors), modiﬁed as per Germain et al.
(2021) to provide, in the case of ScanNet (Dai et al., 2017), a 1 / 8 output-to-input resolution ratio.
To help with memory consumption we apply a simple 2D convolutional layer to compress the descriptor size to 384.
In the case where γ > 0 , we subsequently pad H T with the learned vector λ , producing H T , pad .
Positional encoding.
After computing H S and H T , pad with the convolutional backbone, positional encoding is applied to both dense feature maps.
Similarly to SuperGlue
(Sarlin et al., 2020), we use a 6-layer MLP of size (32, 64, 128, 256, 384), mapping a positional meshgrid between ( − 1 , 1) (centered around the image center) to higher dimensionalities.
BatchNorm and ReLU layers are placed between every module.
In our experiments, we tried adding more positional encoding layers but found it did not make a difference in performance.
After applying the positional encoding, sparse descriptors { d S ,n } n =1 ...N are bilinearly interpolated at { p S ,n } n =1 ...N in H S .
Self-attention.
Following the positional encoding, a single multi-head attention layer is applied on
H T , pad , with 4 heads.
It consists of a standard dot-product attention (Vaswani et al., 2017), coupled with a gating mechanism.
For a given query Q , key K and value V , we compute the attention as
Attention ( Q, K, V ) =
softmax ( g ∗ QK T ) V where g = σ ( max ( QK )) .
To mitigate the quadratic cost of the dot-product attention, we also apply a max-pooling operator on keys and values with a stride of 2, as we empirically found it had very little impact on performance.
We also tried using a
Linear Transformer ( e.g. LinFormer (Katharopoulos et al., 2020)) architecture, but despite trying numerous variants we found it consistently damaged the convergence of the model.
Cross-attention.
Using the same attention-layer design, we subsequently apply it once between { d S ,n } n =1 ...N and H S .
This layer allows for communication between the interpolated source descrip- tors which will be used to produce the ﬁnal correspondence maps, and the original dense source image content.
Then, we apply k cross-attention layers between { d S ,n } n =1 ...N and H T , pad .
We empirically found these layers to be most important, as they allow for direct communication between the sparse source descriptors and the dense target feature maps, prior to the correspondence maps computation.
After trying different values for k and with memory consumption in mind, we settled for k = 4 in all our experiments.
Afﬁne transformation.
The afﬁne transformation matrix K C for the correspondence map C T of resolution ( W T , H T ) is computed from the target image calibration matrix K T and downscaling factors s using:
[Equation]
We can determine if a point lies within the boundaries of C T if its ( x, y ) coordinates are between ( − γW T , − γH T ) and ((1 + γ ) W T , (1 + γ ) H T ) .
Implementation.
The model is implemented in PyTorch (Paszke et al., 2017).
For an indoor sample with 2000 keypoints it has an average throughput of 8 .
84 image/s on an NVIDIA RTX 3070 GPU.
We report the number of parameters in our model in Table 1. [Table]
C. D ATASETS AND T RAINING DETAILS
ScanNet.
The ScanNet (Dai et al., 2017) dataset is a large-scale indoor dataset containing monocular RGB videos and dense depth images, along with ground truth absolute camera poses.
As
SuperGlue (Sarlin et al., 2020) and LoFTR (Sun et al., 2021), we pre-compute the visual overlaps between all image pairs for both training and test scenes.
For the training set we sample images with a visual overlap between 2% and 50% from the ScanNet training scenes, which provides us with challenging images to handle.
We assemble 6 M image pairs and randomly subsample 200 k pairs at every training epoch.
For testing images, we sample 2 , 500 image pairs with overlaps between 2% and 80% from the ScanNet testing scenes, using several bins to ensure the sampling is close to being uniform.
For both training and testing images, we sample keypoints in the source image along a regular grid with cell sizes of 16 pixels.
We remove keypoints with invalid depth, as well as those where the local depth gradient is too high, as the depth information might not be reliable.
We mark keypoints falling outside the target image plane as being outpainted, and we automatically detect the keypoints to inpaint through a cyclic projection of the source keypoints to the target image and back.
The remaining keypoints are labeled as identiﬁable.
For all ScanNet experiments, NeurHal uses a 1 / 8 output-to-input resolution ratio, with a target correspondence map maximum edge size ofpixels (when γ = 0% ).
Megadepth.
We use Megadepth (Li & Snavely, 2018) to train and evaluate NeurHal on outdoor images.
This dataset contains over one million images captured in touristic places, and split in 196 scenes.
To train NeurHal and following Germain et al.
(2021) guidelines, we use the provided
SIFT (Lowe, 2004)-based 3D reconstruction which was made with COLMAP (Schönberger & Frahm, 2016).
Because the sparse 3D point cloud comes from SfM, we ﬁnd however that very little keypoints can be marked as inpainted.
Indeed, no 3D reconstruction is applied to objects or people occluding the scene.
To allow for a wide variety of image pairs we use the sparse reconstruction to estimate the visual overlap and sample pairs with an overlap between 20% and 100%.
We however ﬁnd this overlap estimation to be quite unreliable, as only part of the scene is usually reconstructed.
Since
Megadepth (Li & Snavely, 2018) images are of much higher resolution than ScanNet (Dai et al., 2017), we conﬁgure NeurHal to use a 1 / 16 output-to-input resolution (with a simple max-pooling layer in the CNN).
We set the target correspondence map maximum edge size of 60 pixels (when γ = 0% ), to allow for space in memory when γ = 50% .
Overlap estimation.
For a given pair of images, we approximate the visual overlap by computing the covisibility ratio of keypoints for every image pair.
For a given source and target image pair, we ﬁrst compute the source-to-target and target-to-source covisibility ratios using ground truth depth data and camera poses.
We then deﬁne the visual overlap as the minimum between both ratios.
On
Megadepth we ﬁnd this overlap estimation to be fairly noisy, as depth is only partially known.
Optimizers and scheduling.
On both datasets NeurHal is trained for a maximum of 40 epochs.
We use an initial learning rate of 10 − 3 , with a linear learning rate warm-up in 3 epochs from 0.
of the initial learning rate.
As Sun et al.
(2021), we decay the learning rate by 0.5 every 8 epochs starting from the 8th epoch.
We apply the linear scaling rule and use a batch size of 8 over 8 NVIDIA
V100 GPUs.
We use the AdamW (Loshchilov & Hutter, 2019) optimizer, with a weight decay of 0.1.
In all training procedures, we randomly initialize the model weights.
C. E VALUATION D ETAILS
Evaluation protocol.
All baselines follow the same standard protocol in which we: 1) Compute 2D-2D correspondences between the reference image and the query image, 2) Lift these 2D-2D correspondences to 2D-3D correspondences using the available 3D information for the reference image, 3)
Estimate the camera pose given these 2D-3D correspondences by minimizing the Reprojection
Error (RE), i.e. applying LO-RANSAC+PnP (Chum et al., 2003) followed by a non-linear iterative reﬁnement.
This approach is widely used and leads to state-of-the-art results in visual localization benchmarks.
We also include results for Germain et al.
(2021) which we call S2D. For the evaluation of Fig.
4, we ﬁnd the inpainted and outpainted correspondents for LoFTR (Sun et al., 2021) and
DRCNet (Li et al., 2020) by fetching the argmax 2D coordinates in the 4D matching conﬁdence volume.
For S2D and NeurHal, we simply take the argmax in correspondence maps for the same set of keypoints.
Choice of threshold.
We reported in Fig.
13 the performance of NeurHal, state-of-the-art feature matching methods and the identity pose, on ScanNet for several rotation and translation thresholds.
We can see that arbitrarily choosing a threshold of τ t = 1 . 5 m and τ r = 20 . 0 ◦ sets a hard objective as the identity pose is particularly poor.
(Chum et al., 2003)-based pose estimator.
For all (Chum et al.
, 2003)-based methods, we estimate the camera pose using the pycolmap python binding.
We tune the RANSAC threshold for optimal performance, and mark all cases where less than 3 valid correspondences ( i.e. with a valid depth value) as failure cases (inﬁnite pose error).
The remaining parameters are left as default.
We follow the evaluation instructions provided by each method, and use indoor weights for SP+SG (Sarlin et al., 2020) and the dual-softmax indoor weights for LoFTR (Sun et al., 2021).
In the case of NeurHal + (Chum et al., 2003), we simply read the argmax of the predicted correspondence maps to obtain explicit 2D-to-3D correspondences.
(Germain et al., 2021)-based pose estimator.
For both S2D
(Germain et al., 2021) and NeurHal we only use coarse models, which operate at either 1/8th or 1/16th of the original input resolution.
We ﬁrst retrain the S2D coarse model (fully-convolutional Inceptionv3 (Szegedy et al., 2016), up to
Mixed-6e) on the same training set as our method, with the same target resolution of 80 pixels.
We refer to this model as S2D. Given correspondence maps and the depth map of the source image, we estimate the camera pose between the target image and the source image using the method proposed in Germain et al.
For both S2D and NeurHal we use the same set of regularly sampled source keypoints (see Sec.
C.1), and we perform camera pose estimation ﬁrst using P3P inside an
MSAC (Torr & Zisserman, 2000) loop.
We run P3P for a maximum of 5 , 000 iterations over the top-20% correspondences.
We then apply a coarse GNC (Blake & Zisserman, 1987) over all source keypoints with σ max
= 2 . 0 and σ min = 0 .
Let us highlight that in all the camera pose experiments, the performances of NeurHal are obtained by predicting only low resolution correspondence maps (see Sec.
C.1).
D A DDITIONAL QUALITATIVE RESULTS D. G ENERALIZATION TO NEW DATASETS
So far we have demonstrated the ability of NeurHal to hallucinate correspondences on unseen validation scenes from both ScanNet (Dai et al., 2017) and Megadepth (Li & Snavely, 2018).
In order to further demonstrate the generalization capacity of NeurHal, we report qualitative results obtained on the NYU Depth Dataset (Nathan Silberman & Fergus, 2012) in Fig.
15 and on the
ETH-3D (Schöps et al., 2017) dataset in Fig.
We use the set of indoor weights for NYU ( i.e.
NeurHal trained on ScanNet) and outdoor weights for ETH-3D ( i.e. NeurHal trained on MegaDepth).
We report the overlayed and upsampled coarse truncated loss map computed following Germain et al.
(2021) on low-overlap image pairs.
We ﬁnd that NeurHal is able to robustly outpaint correspondences despite little visual overlaps and strong relative camera motions.
These visuals demonstrate the strong generalization ability of NeurHal.
D. Q UALITATIVE CORRESPONDENCE HALLUCINATION RESULTS AND FAILURE CASES
To further demonstrate the ability of NeurHal to perform visual correspondence hallucination, we report in Fig.
16 and Fig.
17 qualitative results on ScanNet (Dai et al., 2017) and Megadepth (Li & Snavely, 2018) respectively on scenes that were not seen at training-time.
In the target image and in the (negative log) correspondence map, the red dot represents the ground truth keypoint’s correspondent.
The dashed rectangles represent the borders of the target images.
Let us recall that NeurHal outputs probability distributions ( a.k.a.
correspondence maps) assuming the two input images are partially overlapping .
It is essential to keep this assumption in mind when looking at these qualitative results.
For instance, concerning the example Fig.
16 (b) (middle), it is very difﬁcult for our human visual system to be sure that the two images are actually overlapping, and consequently the network prediction seems to good to be true.
However, if we assume that there is an overlap, we realize that it is actually possible to perform correspondence hallucination, by drawing out the two skirting boards, to correctly outpaint the correspondent.
In fact, this overlapping assumption has a regularization effect in cases where the covisible image areas show no distinctive regions, and one image could be at an inﬁnite translation of the other, e.g.
Fig.
16 (b) (second to last).
In Fig.
16 (d) and Fig.
17 (d) we show failure cases where the correspondence maps modes predicted by NeurHal are either partially or completely off.
We ﬁnd that failure cases often correlate with strongly ambiguous image pairs, or images that have extremely limited visual overlap.
D. Q UALITATIVE CAMERA POSE ESTIMATION RESULTS
We show in Fig.
18 qualitative results in camera pose estimation on low-overlap images from
ScanNet (Dai et al., 2017), for NeurHal and its three best-performing competitors.
For every method we display the keypoints used as input to the camera pose estimator in the source image, along with their reprojection at the estimated camera pose in the target image.
For methods using the pose estimator from (Chum et al., 2003), the keypoints are those that have been successfully matched.
When using the pose estimator of Germain et al.
(2021), the keypoints are those involved in the prediction of the dense NRE maps.
We color in keypoints based on their spatial 2D position in the source image.
We ﬁnd that NeurHal strongly beneﬁts from its outpainting ability, in comparison with all other competitors which struggle to ﬁnd both sufﬁcient and reliable correspondences.
We also report in Fig.
19 failure cases for NeurHal.
We ﬁnd that such cases correspond to image pairs exhibiting extremely limited visual overlap, strong camera pose rotations and overall signiﬁcant ambiguities. [Figure] [Figure] - l n
C T
T a r g e t
S ou r ce [Figure] - l n C T
T a r g e t
S ou r ce
[Figure]
- l n
C T
T a r g e t
S ou r ce
[Figure]
- l n
C T
T a r g e t
S ou r ce
[Figure]
- l n
C T
T a r g e t
S ou r ce
[Figure]
- l n
C T
T a r g e t
S ou r ce
[Figure]
- l n
C T
T a r g e t
S ou r ce
[Figure]
l n
C T
T a r g e t
S ou r ce
[Figure]
[Figure]
[Figure]