Title: Tame a Wild Camera: In-the-Wild Monocular Camera Calibration
Venue: NeurIPS 2023
Accepted: yes
Keywords: Monocular Camera Calibration; Camera Pose Estimation; Image Editing
Abstract: 3D sensing for monocular in-the-wild images, e.g., depth estimation and 3D object detection, has become increasingly important.
However, the unknown intrinsic parameter hinders their development and deployment.
Previous methods for the monocular camera calibration rely on specific 3D objects or strong geometry prior, such as using a checkerboard or imposing a Manhattan World assumption.
This work instead calibrates intrinsic via exploiting the monocular 3D prior.
Given an undistorted image as input, our method calibrates the complete 4 Degree-of-Freedom (DoF) intrinsic parameters.
First, we show intrinsic is determined by the two well-studied monocular priors: monocular depthmap and surface normal map.
However, this solution necessitates a low-bias and low-variance depth estimation.
Alternatively, we introduce the incidence field, defined as the incidence rays between points in 3D space and pixels in the 2D imaging plane.
We show that: 1) The incidence field is a pixel-wise parametrization of the intrinsic invariant to image cropping and resizing.
2) The incidence field is a learnable monocular 3D prior, determined pixel-wisely by up-to-sacle monocular depthmap and surface normal.
With the estimated incidence field, a robust RANSAC algorithm recovers intrinsic.
We show the effectiveness of our method through superior performance on synthetic and zero-shot testing datasets.
Beyond calibration, we demonstrate downstream applications in image manipulation detection \& restoration, uncalibrated two-view pose estimation, and 3D sensing.

Tame a Wild Camera: In-the-Wild Monocular Camera Calibration

Abstract

3D sensing for monocular in-the-wild images, e.g. , depth estimation and 3D object detection, has become increasingly important. However, the unknown intrinsic parameter hinders their development and deployment. Previous methods for the monocular camera calibration rely on specific 3D objects or strong geometry prior, such as using a checkerboard or imposing a Manhattan World assumption. This work instead calibrates intrinsic via exploiting the monocular 3D prior. Given an undistorted image as input, our method calibrates the complete 4 Degree-ofFreedom (DoF) intrinsic parameters. First, we show intrinsic is determined by the two well-studied monocular priors: monocular depthmap and surface normal map. However, this solution necessitates a low-bias and low-variance depth estimation. Alternatively, we introduce the incidence field, defined as the incidence rays between points in 3D space and pixels in the 2D imaging plane. We show that: 1) The incidence field is a pixel-wise parametrization of the intrinsic invariant to image cropping and resizing. 2) The incidence field is a learnable monocular 3D prior, determined pixel-wisely by up-to-sacle monocular depthmap and surface normal. With the estimated incidence field, a robust RANSAC algorithm recovers intrinsic. Weshow the effectiveness of our method through superior performance on synthetic and zero-shot testing datasets. Beyond calibration, we demonstrate downstream applications in image manipulation detection & restoration, uncalibrated two-view pose estimation, and 3D sensing. Codes/models are held here.

1 Introduction

Camera calibration is typically the first step in numerous vision and robotics applications [27, 46] that involve 3D sensing. Classic methods enable accurate calibration by imaging a specific 3D structure such as a checkerboard [49]. With the rapid growth of monocular 3D vision, there is an increasing focus on 3D sensing from in-the-wild images, such as monocular depth estimation, 3D object detection [35, 36], and 3D reconstruction [42, 43]. While 3D sensing techniques over in-the-wild images are developed, camera calibration for such in-the-wild images continues to pose significant challenges.

Classic methods for monocular calibration use strong geometry prior, such as using a checkerboard. However, such 3D structures are not always available in in-the-wild images. As a solution, alternative methods relax the assumptions. For example, [29] and [23] calibrate using common objects such as human faces and objects' 3D bounding boxes. Another significant line of research [7,19,34,37,51, 62,66] is based on the Manhattan World assumption [15], which posits that all planes within a scene are either parallel or perpendicular to each other. This assumption is further relaxed [28, 30, 64] to estimate the lines that are either parallel or perpendicular to the direction of gravity. The intrinsic parameters are recovered by determining the intersected vanishing points of detected lines, assuming a central focal point and an identical focal length.

GT Intrinsic

GT Intrinsic

Intrinsic

Mono-Calibration

Camera

(a)

(b)

(c)

(d)

(e)

Incidence Vector

While the assumptions are relaxed, they may still not hold true for in-the-wild images. This creates a contradiction: although we develop robust models to estimate in-the-wild monocular depthmap, generating its 3D point cloud remains infeasible due to the missing intrinsic. A similar challenge arises in monocular 3D object detection, as we face limitations in projecting the detected 3D bounding boxes onto the 2D image. In AR/VR applications, the absence of intrinsic precludes placing multiple reconstructed 3D objects within a canonical 3D space. The absence of a reliable, assumption-free monocular intrinsic calibrator has become a bottleneck in deploying these 3D sensing applications.

Our method is motivated by the consistency between the monocular depthmap and surface normal map. In Fig. 1 (c) - (e), an incorrect intrinsic distorts the back-projected 3D point cloud from the depthmap, resulting in distorted surface normals. Based on this, intrinsic is optimal when the estimated monocular depthmap aligns consistently with the surface normal. We present a solution to recover the complete 4 DoF intrinsic by leveraging the consistency between the surface normal and depthmap. However, the algorithm is numerically ill-conditioned as its computation depends on the accurate gradient of depthmap. This requires depthmap estimation with low bias and variance.

To resolve it, we propose an alternative approach by introducing a novel 3D monocular prior in complementation to the depthmap and surface normal map. We refer to this as the incidence field, which depicts the incidence ray between the observed 3D point and the projected 2D pixel on the imaging plane, as shown in Fig. 1 (b). The combination of the incidence field and the monocular depthmap describes a 3D point cloud. Compared to the original solution, the incidence field is a direct pixel-wise parameterization of the camera intrinsic. This implies that a minimal solver based on the incidence field only needs to have low bias. We then utilize a deep neural network to perform the incidence field estimation. A non-learning RANSAC algorithm is developed to recover the intrinsic parameters from the estimated incidence field.

We consider the incidence field is also a monocular 3D prior. Similar to depthmap and surface normal, the incidence field is invariant to the image cropping or resizing. This encourages its generalization over in-the-wild images. Theoretically, we show the incidence field is pixel-wisely determined by depthmap and normal map. Empirically, to support our argument, we combine multiple public datasets into a comprehensive dataset with diverse indoor, outdoor, and object-centric images captured by different imaging devices. We further boost the variety of intrinsic by resizing and cropping the images in a similar manner as [23]. Finally, we include zero-shot testing samples to benchmark real-world monocular camera calibration performance.

We showcase downstream applications that benefit from monocular camera calibration. In addition to the aforementioned 3D sensing tasks, we present two intriguing additional applications. One is detecting and restoring image resizing and cropping. When an image is cropped or resized, it disrupts the assumption of a central focal point and identical focal length. Using the estimated

intrinsic parameters, we restore the edited image by adjusting its intrinsic to a regularized form. The other application involves two-view uncalibrated camera pose estimation. With established image correspondence, a fundamental matrix [25] is determined. However, there does not exist an injective mapping between the fundamental matrix and camera pose [27]. This raises a counter-intuitive fact: inferring the pose from two uncalibrated images is infeasible. But our method enables uncalibrated two-view pose estimation by applying monocular camera calibration.

We summarize our contributions as follows:

⋄ Our approach tackles monocular camera calibration from a novel perspective by relying on monocular 3D priors. Our method makes no assumption for the to-be-calibrated undistorted image.

⋄ Our algorithm provides robust monocular intrinsic estimation for in-the-wild images, accompanied by extensive benchmarking and comparisons against other baselines.

⋄ We demonstrate its benefits on diverse and novel downstream applications.

2 Related Works

Monocular Camera Calibration with Geometry. One line of work [7, 19, 34, 37, 51, 62, 66] assumes the Manhattan World assumption [15], where all planes in 3D space are either parallel or perpendicular. Under the assumption, line segments in the image converge at the vanishing points, from which the intrinsic is recovered. LSD [61] and EDLine [2] develop robust line estimators. Others jointly estimate the horizon line and the vanishing points [40,54,70]. In Tab. 1, recent learning-based methods [28,30,38,63,64] relax the assumption by using the gravity-aligned panorama images during training. Such data provides the groundrtuth vanishing point and horizon lines which are originally computed with the Manhattan World assumption. Still, the assumption constrains [28,38,63,64] in modeling intrinsic as 1 DoF camera. Recently, [30] relaxes the assumption to 3 DoF via regressing the focal point. In comparison, our method makes no assumption except for undistorted images. This enables us to calibrate 4 DoF intrinsic and train with any calibrated images.

Monocular Camera Calibration with Object. Zhang's method [73] based on a checkerboard pattern is widely regarded as the standard for camera calibration. Several works generalize this method to other geometric patterns such as 1D objects [74], line segments [72], and spheres [71]. Recent works [29] and [23] extend camera calibration to real-world objects such as human faces. Optimizers, including BPnP [12] and PnP [56] are developed. However, the usage of specific objects restricts their applications. In contrast, our approach is applicable to any image.

Image Cropping and Resizing. Detecting photometric image manipulation [5,17,24] is extensively researched. But few studies detecting geometric manipulation, e.g. , resizing and cropping. On resizing, [18] regresses the image aspect ratio with a deep model. On cropping, a proactive method [67] is developed. We demonstrate that calibration also detects geometric manipulation. Our method does not need to encrypt images, complementing photometric manipulation detection.

Uncalibrated Two-View Pose Estimation. With the fundamental matrix estimated, the two-view camera pose is determined up to a projective ambiguity if images are uncalibrated. Alternative solutions [31, 47, 58] exist by employing deep networks to regress the pose. However, regression hinders the usage of geometric constraints, which proves crucial in calibrated two-view pose estimation [60, 75, 77]. Other works [20, 26] use more than two uncalibrated images for pose estimation. Our work complements prior studies by enabling an uncalibrated two-view solution.

Learnable Monocular 3D Priors. Monocular depth [39, 76] and surface normals [6] are two established 3D priors. Numerous studies have shown a learnable mapping between monocular images and corresponding 3D priors. Recently, [30] introduces the perspective field as a monocular 3D

w/o

Asm. RANSAC Sampling

w/

Asm. Enumerate

{

f

i

x

, b

i

x

}

{

f

i

y

, b

i

y

}

{

f

i

}

{

ρ

x

(

·

)

}

{

ρ

y

(

·

)

}

{

ρ

x

(

·

) +

ρ

y

(

·

)

}

f

b

y

f

y

b

x

f

x

Eq. (16)

Eq. (17)

Eq. (18)

Eq. (19)

RANSAC

Estimated

K

RANSAC

(a)

(b)

(c)

prior for inferring gravity direction. In this work, we suggest the incidence field is also a learnable monocular 3D prior. A common characteristic of monocular 3D priors is their exclusive relationship with 3D structures, making them invariant to 2D manipulations, e.g. , cropping, and resizing. In contrast, camera intrinsic changes on spacial manipulation. Unlike other monocular priors, computing the groundtruth incidence field is straightforward, relying solely on intrinsic and image coordinates.

3 Method

Our work focuses on intrinsic estimation from in-the-wild monocular images captured by modern imaging devices. Hence we assume an undistorted image as input. In this section, we first show how to estimate intrinsic parameters by using monocular 3D priors, such as the surface normal map and monocular depthmap. We then introduce the incidence field and show it a learnable monocular 3D prior invariant to 2D spacial manipulation. Next, we describe the training strategy and the network used to learn the incidence field. After estimating the incidence field, we present a RANSAC algorithm to recover the 4 DoF intrinsic parameters. Lastly, we explore various downstream applications of the proposed algorithm. Fig. 2 shows the framework of the proposed algorithm.

3.1 Monocular Intrinsic Calibration

Our method aims to use generalizable monocular 3D priors without assuming the 3D scene geometry. Hence, we start with monocular depthmap D and surface normal map N . Assume there exists a learnable mapping from the input image I to depthmap D and normal map N : D , N = D θ ( I ) , where D θ is a learned network. We denote the intrinsic K simple , K , and its inverse K -1 as:

The notation K simple suggests a simple camera model with the identical focal length and central focal point assumption. Given a 2D homogeneous pixel location p = [ x y 1] ⊺ and its depth value d = D ( p ) , the corresponding 3D point P = [ X Y Z ] ⊺ is defined as:

where the vector v is an incidence ray, originating from the 3D point P , directed towards the 2D pixel p , and passing through the camera's origin. The incidence field V is the collection of incidence rays v associated with all pixels at location p , where v = V ( p ) .

3.2 Monocular Intrinsic Calibration with Monocular Depthmap and Surface Normal

In this section, we explain how to determine the intrinsic matrix K using the estimated surface normal map N and depthmap D . Given the estimated depth d = D ( p ) and normal n = N ( p ) at 2D pixel

(a)

GT Depthmap

(b)

GT Depthmap Converted Surface Normal

[

f

x

f

y

1

2

w

1

2

h

FoV

]

[

2

f

x

2

f

y

0

0

N

/

A

]

(c)

RGB Image

I

(d)

Incidence Field

V

location p , a local 3D plane is described as:

By taking derivative in x -axis and y -axis directions, we have:

Note the bias c of the 3D local plane is independent of the camera projection process. Without loss of generality, we show the case of our method for x -direction. Expanding Eq. (4), we obtain:

where ∇ x ( d ) represents the gradient of the depthmap D in the x -axis and can be computed, for example, using a Sobel filter [33]. Next, re-parametrize the unknowns in Eq. (5) to get:

Divide both sides of the equation by f x to get:

where r = f y f x . By stacking Eq. (7) with N ≥ 4 randomly sampled pixels, we acquire a linear system:

where the intrinsic parameter to be solved is stored in a vector X 4 × 1 = [ f y b y rb x r ] ⊺ . This solves the other intrinsic parameters as:

The known constants are stored in matrix A N × 4 and B 4 × 1 . If we choose N = 4 in Eq. (8), we obtain a minimal solver where the solution X is computed by performing Gauss-Jordan Elimination. Conversely, when N > 4 , the linear system is over-determined, and X is obtained using a least squares solver. The above suggests the intrinsic is recoverable from the monocular 3D prior.

3.3 Monocular Incidence Field as Monocular 3D Prior

Eq. (9) relies on the consistency between the surface normal and depthmap gradient, which may require a low-variance depthmap estimate. From Fig. 3, even groundtruth depthmap leads to spurious normal due to its inherent high variance. Thus, minimal solver in Eq. (9) can lead to a poor solution.

As a solution, we propose to directly learn the incidence field V as a monocular 3D prior. In Eq. (2) and Fig. 1, the combination of the incidence field V and the monocular depthmap D creates a 3D point cloud. In Eq. (3), the incidence field V can measure the observation angle between a 3D plane and the camera. Similar to depthmap D and surface normal map N , the incidence field V is invariant to the image cropping and resizing. Consider an image cropping and resizing described as:

where K ′ is the intrinsic after transformation. The surface normal map N and depthmap D after transformation is defined as:

Similarly, the incidence field after transformation is:

Eq. (12) shows that the incidence field V is a parameterization of the intrinsic matrix that is invariant to image resizing and image cropping. Other invariant parameterizations of the intrinsic matrix, such as the camera field of view (FoV), rely on the central focal point assumption and only cover a 2 DoF intrinsic matrix. An illustration is shown in Fig. 3.

Next, we prove the incidence field V is a monocular 3D prior pixel-wisely determined by monocular depth D and surface normal N . Expanding Eq. (4) using the incidence vector v in Eq. (2), we get:

Combined with the axisy constraint, the 2 DoF incidence vector v = [ v x v y 1] ⊺ is uniquely solved. Eq. (13) gives identical solutions when depthmap D is adjusted by a scalar. This implies that the incidence field V is pixel-wise determined by the up-to-scale depth map and surface normal map, indicating a learnable mapping from the monocular image I to the incidence field V exists.

Given the strong connection between the monocular depthmap D and camera incidence field V , we adopt NewCRFs [69], a neural network used in monocular depth estimation, for incidence field estimation. We change the last output head to output a three-dimensional normalized incidence field ˜ V with the same resolution as the input image I . We adopt a cosine similarity loss defined as:

We normalize the last dimension of the incidence field to one before feeding to the RANSAC algorithm. That is to say, V ( x i ) = [˜ v 1 / ˜ v 3 ˜ v 2 / ˜ v 3 1] ⊺ = [ v 1 v 2 1] ⊺ .

3.4 Monocular Intrinsic Calibration with Incidence Field

Since the network inference executes on GPU device, we adopt a GPU-end RANSAC algorithm to recover the intrinsic K from the incidence field V . Unlike a CPU-based RANSAC, we perform fixed K r iterations of RANSAC without termination. In RANSAC, we use the minimal solver to generate K c candidates and select the optimal one that maximizes a scoring function (see Fig. 2).

RANSAC w.o Assumption. From Eq. (2), the incidence vector v relates to the intrinsic K as:

From Eq. (15), a minimal solver for intrinsic is straightforward. In the incidence field, randomly sample two incidence vectors v 1 = [ v 1 x v 1 y 1 ] ⊺ and v 2 = [ v 2 x v 2 y 1 ] ⊺ . The intrinsic is:

Similarly, the scoring function is defined in x -axis and y -axis, respectively:

where N k and k x / k y are the number of sampled pixels and the threshold for axisx/y directions.

RANSAC w/ Assumption. If a simple camera model is assumed, i.e. , intrinsic only has an unknown focal length, it only needs to estimate 1 -DoF intrinsic. We enumerate the focal length candidates as:

The scoring function under the scenario is defined as the summation over x -axis and y -axis:

3.5 Downstream Applications

Image Crop & Resize Detection and Restoration. Eq. (10) defines a crop and resize operation:

′ ′

When a modified image I is presented, our algorithm calibrates its intrinsic K and then: Case 1: The original intrinsic K is known. E.g. , we obtain K from the image-associated EXIF file [3]. Image manipulation is computed as ∆ K = K ′ K -1 . A manipulation is detected if ∆ K deviates from an identity matrix. The original image restores as I ( x ) = I ′ (∆ Kx ) . Interestingly, the four corners of image I ′ are mapped to a bounding box in original image I under manipulation ∆ K . We thus quantify the restoration by measuring the bounding box. See Fig. 4.

Case 2: The original intrinsic K is unknown. We assume the genuine image possess an identical focal length and central focal point. Any resizing and cropping are detected when matrix K ′ breaks this assumption. Note, the rule cannot detect aspect ratio preserving resize or centered crop. We restore the original image by defining an inverse operation ∆ K restore K ′ to an intrinsic fits the assumption.

3D Sensing Related Tasks. Intrinsic estimation can enable multiple applications. E.g. , depthmap to point cloud, uncalibrated two-view pose estimation, etc.

4 Experiments

Implementation Details Our network is trained using the Adam optimizer [32] with a batch size of 8 . The learning rate is 1 e -5 , and the training process runs for 20 , 000 steps. Our RANSAC algorithm

is executed on a GPU, performing K r = 2 , 048 iterations, with each iteration including a random sample of K c = 20 , 000 incidence vectors. During both training and testing, the image is resized to a resolution of 480 × 640 . The threshold for determining RANSAC inliers is k x = k y = 0 . 02 .

Evaluation Metrics We follow [29] in assessing camera intrinsic estimation using the metrics:

We additionally consolidate the performance along the x-axis and y-axis into comprehensive measurements, e f = max( e f x , e f y ) and e b = max( e b x , e b y ) .

4.1 Monocular Camera Calibration In-The-Wild

Datasets. Our method is trained whenever a calibrated intrinsic is provided, making it applicable to a wide range of publicly available datasets. In Tab. 2, we incorporate datasets of different application scenarios, including indoor, outdoor scenes, driving, and object-centric scenes. Many of the datasets utilize only a single type of camera for data collection, resulting in a scarcity of intrinsic variations. Similar to [38], we employ random resizing and cropping to synthesize more intrinsic, marked in Tab. 2 column 'Syn.'. In augmentation, we first resize all images to a resolution of 480 × 640 . We then uniformly random resize up to two times its size and subsequently crop to a resolution of 480 × 640 . As MegaDepth [41] collects images captured by various cameras from the Internet, we disable its augmentation. We document the intrinsic parameters of each dataset in Supp.

In Tab. 2 column 'Calibration", we assess intrinsic quality into various levels. 'Calibrated' suggests accurate calibration with a checkerboard. 'Pre-defined' is less accurate, indicating the default intrinsic provided by the camera manufacturer without a calibration process. 'SfM" signifies that the intrinsic is computed via an SfM method [52].

In-The-Wild Monocular Camera Calibration. We benchmark in-the-wild monocular calibration performance on Tab. 2. For trained datasets, except for MegaDepth, we test on synthetic data using random cropping and resizing. For the unseen test dataset, we refrain from applying any augmentation to better mimic real-world application scenarios.

We compare to the recent baseline [30], which regresses intrinsic via a deep network. Note, [30] can not train on arbitrary calibrated images as requiring panorama images in training. A fair comparison using the same training and testing images is in Tab. 4 and Sec. 4.2. [30] provides models with two variations: one assumes a central focal point, and another does not. We report with the former model whenever the input image fits the assumption. From Tab. 2, our method demonstrates superior generalization across multiple unseen datasets. Further, the result w/ assumption outperforms w/o assumption whenever the input images fit the assumption. Tested on an RTX-2080 Ti GPU, the combined network inference and calibration algorithm runs on average in 87 ms per image.

4.2 Monocular Camera Calibration with Geometry

Methods in this line of research hold a Manhattan World assumption, positing that images consist of planes that are either parallel or perpendicular to each other. Stated in Tab. 1, baselines [28, 30, 38] relax the assumption to training data. Our method imposes no other assumption except for undistorted images in both training and testing.

This brings three benefits. First, the assumption restricts their training to panorama images. In contrast, our model is trainable with any calibrated images. This yields improved generalization,

(a)

Input & Incidence

(b)

Est. Restoration

(c)

GT. Restoration

(d)

Original Image

Estimated

Estimated

LD

Estimated

Estimated

as shown in Tab. 2. Second, it constrains the baselines to a simple camera parameterized by FoV. We consider the proposed incidence field a more generalizable and invariant parameterization of intrinsic. E.g. , while FoV remains invariant to image resizing, it still changes after cropping. However, the incidence field is unaffected in both cases. In Tab. 3, the substantial improvement we achieved ( 0 . 60 = 3 . 07 -2 . 47 ) over the recent SoTA [30] empirically supports our argument. Third, our method calibrates the 4 DoF intrinsic with a non-learning RANSAC algorithm. Baselines instead regress the intrinsic. This renders our method more robust and interpretable. In Fig. 2 (b), the estimated intrinsic quality is visually discerned through the consistency achieved between the two incidence fields.

4.3 Monocular Camera Calibration with Objects

We compare to the recent object-based camera calibration method FaceCalib [29]. The baseline employs a face alignment model to calibrate the intrinsic over a video. Both [29] and ours perform zero-shot prediction. We report performance using Tab. 2 model. Compared to [29], our method is more general as it does not assume a human face present in the image. Meanwhile, [29] calibrates over a video , while ours is a monocular method. For a fair comparison, we report the video-based results as an averaged error over the videos. We report results w/ and w/o assuming a simple camera model. Since the tested image has a central focal point, when the assumption applied, the error of the focal point diminished. In Tab. 4, we outperform SoTA substantially.

4.4 Downstream Applications

Image Crop & Resize Detection and Restoration. In Sec. 3.5, our method also addresses the detection and restoration of geometric manipulations in images. Using the model reported in Tab. 2, we benchmark its performance in Tab. 5. Random manipulations following Sec. 4.1 contribute to 50% of both train and test sets, and the other 50% are genuine images. In Tab. 5, we evaluate restoration with mIOU and report detection accuracy ( i.e. binary classification of genuine vs edited images). From the table, our method generalizes to the unseen dataset, achieving an averaged mIOU of 0 . 680 . Meanwhile, we substantially outperform the baseline, which is trained to directly regresses the intrinsic. The improvement suggests the benefit of the incidence field as an invariant intrinsic parameterization. Beyond performance, our algorithm is interpretable. In Fig. 4, the perceived image geometry is interpretable for humans.

Omni3D [9]

Omni3D [9]

w/

our intrinsic

Omni3D [9]

Omni3D [9]

w/

our intrinsic

so1o.0.92

Restored

I

1

Cropped

I

1

Original Image

I

Cropped

I

2

Restored

I

2

Uncalibrated Two-View Camera Pose Estimation. With correspondence between two images, one can infer the fundamental matrix [25]. However, the pose between two uncalibrated images is determined by a projective ambiguity. Our method eliminates the ambiguity with monocular camera calibration. In Tab. 6, we benchmark the uncalibrated two-view pose estimation and compare it to recent baselines. The result is reported using the model benchmarked in Tab. 2 and assumes unique intrinsic for both images. We perform zero-shot in ScanNet. For MegaDepth, it includes images collected over the Internet with diverse intrinsics. Interestingly, in ScanNet, our uncalibrated method outperforms a calibrated one [44]. In Supp, we plot the curve between pose performance and intrinsic quality. The challenging setting suggests itself an ideal task to evaluate the intrinsic quality.

In-the-Wild Monocular 3D Sensing. In-the-wild 3D sensing, e.g. , monocular 3D object detection [9], requires intrinsic to connect the 3D and 2D space. In Fig. 5, despite 3D bounding boxes being accurately estimated, incorrect intrinsic produces suboptimal 2D projection. Our calibration results suggest monocular incidence field estimation is another essential monocular 3D sensing task.

5 Conclusion

We calibrate monocular images through a novel monocular 3D prior referred as incidence field. The incidence field is a pixel-wise parameterization of intrinsic invariant to image resizing and cropping. A RANSAC algorithm is developed to recover intrinsic from the incidence field. We extensively benchmark our algorithm and demonstrate robust in-the-wild performance. Beyond calibration, we show multiple downstream applications that benefit from our method.

Limitation. In real application, whether to apply the assumption still requires human input.

Broader Impacts. Our method detects image spacial editing. If the Exif files are provided, it further localizes the modification region. Shown in Fig. 6, it helps improve press image authenticity.

References

Adel Ahmadyan, Liangkai Zhang, Artsiom Ablavatski, Jianing Wei, and Matthias Grundmann. Objectron: A large scale dataset of object-centric videos in the wild with pose annotations. In CVPR , 2021. 7

Cuneyt Akinlar and Cihan Topal. EDLines: A real-time line segment detector with a false detection control. Pattern Recognition Letters , 2011. 3

Paul Alvarez. Using extended file information (exif) file headers in digital evidence analysis. IJDE , 2004. 7

Dragomir Anguelov, Carole Dulong, Daniel Filip, Christian Frueh, Stéphane Lafon, Richard Lyon, Abhijit Ogale, Luc Vincent, and Josh Weaver. Google street view: Capturing the world at street level. Computer , 2010. 7

Vishal Asnani, Xi Yin, Tal Hassner, Sijia Liu, and Xiaoming Liu. Proactive image manipulation detection. In CVPR , 2022. 3

Gwangbin Bae, Ignas Budvytis, and Roberto Cipolla. Estimating and exploiting the aleatoric uncertainty in surface normal estimation. In ICCV , 2021. 3

Olga Barinova, Victor Lempitsky, Elena Tretiak, and Pushmeet Kohli. Geometric image parsing in man-made environments. In ECCV , 2010. 1, 3

Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, and Elad Shulman. ARKitScenes - a diverse real-world dataset for 3D indoor scene understanding using mobile RGB-D data. In NeurIPS Datasets and Benchmarks Track , 2021. 7, 9

Garrick Brazil, Abhinav Kumar, Julian Straub, Nikhila Ravi, Justin Johnson, and Georgia Gkioxari. Omni3D: A large benchmark and model for 3D object detection in the wild. In CVPR , 2023. 10

Holger Caesar, Varun Bankiti, Alex Lang, Sourabh Vora, Venice Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuScenes: A multimodal dataset for autonomous driving. In CVPR , 2020. 7

Angel Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. Shapenet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012 , 2015. 7

Bo Chen, Tat-Jun Chin, and Nan Li. BPnP: Further empowering end-to-end learning with back-propagatable geometric optimization. arXiv: 1909.06043 , 2019. 3, 8

Hongkai Chen, Zixin Luo, Lei Zhou, Yurun Tian, Mingmin Zhen, Tian Fang, David Mckinnon, Yanghai Tsin, and Long Quan. ASpanFormer: Detector-free image matching with adaptive span transformer. In ECCV , 2022. 10

Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, UweFranke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR , 2016. 7

James M Coughlan and Alan L Yuille. Manhattan world: Compass direction from a single image by bayesian inference. In ICCV , 1999. 1, 3

Angela Dai, Angel Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In TPAMI , 2017. 7, 9, 10

Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, and Anil Jain. On the detection of digital face manipulation. In CVPR , 2020. 3

Bibhash Das, Mrutyunjay Biswal, Abhranta Panigrahi, and Manish Okade. CNN based image resizing detection and resize factor classification for forensic applications. In ICORT , 2021. 3

Patrick Denis, James Elder, and Francisco Estrada. Efficient edge-based methods for estimating manhattan frames in urban imagery. In ECCV , 2008. 1, 3

Torben Fetzer, Gerd Reis, and Didier Stricker. Stable intrinsic auto-calibration from fundamental matrices of devices with uncorrelated camera parameters. In WACV , 2020. 3, 8

Simon Fuhrmann, Fabian Langguth, and Michael Goesele. Mve-a multi-view reconstruction environment. In GCH , 2014. 7, 9

Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The KITTI dataset. IJRR , 2013. 7, 9

Alexander Grabner, Peter Roth, and Vincent Lepetit. GP 2 C: Geometric projection parameter consensus for joint 3D pose and focal length estimation in the wild. In ICCV , 2019. 1, 2, 3

Xiao Guo, Xiaohong Liu, Zhiyuan Ren, Steven Grosz, Iacopo Masi, and Xiaoming Liu. Hierarchical fine-grained image forgery detection and localization. In CVPR , 2023. 3

Richard Hartley. In defense of the eight-point algorithm. TPAMI , 1997. 3, 10

Richard Hartley. Kruppa's equations derived from the fundamental matrix. TPAMI , 1997. 3

Richard Hartley and Andrew Zisserman. Multiple view geometry in computer vision . Cambridge university press, 2003. 1, 3

Yannick Hold-Geoffroy, Kalyan Sunkavalli, Jonathan Eisenmann, Matthew Fisher, Emiliano Gambaretto, Sunil Hadap, and Jean-François Lalonde. A perceptual measure for deep single image camera calibration. In CVPR , 2018. 1, 3, 7, 8

Masa Hu, Garrick Brazil, Nanxiang Li, Liu Ren, and Xiaoming Liu. Camera self-calibration using human faces. In FG , 2023. 1, 3, 8, 9

Linyi Jin, Jianming Zhang, Yannick Hold-Geoffroy, Oliver Wang, Kevin Blackburn-Matzen, Matthew Sticha, and David F Fouhey. Perspective fields for single image camera calibration. In CVPR , 2023. 1, 3, 7, 8, 9

Fadi Khatib, Yuval Margalit, Meirav Galun, and Ronen Basri. GRelPose: Generalizable end-to-end relative camera pose regression. arXiv preprint arXiv:2211.14950 , 2022. 3

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR , 2023. 7

Josef Kittler. On the accuracy of the sobel edge detector. Image and Vision Computing , 1983. 5

Jana Košecká and Wei Zhang. Video compass. In ECCV , 2002. 1, 3

Abhinav Kumar, Garrick Brazil, Enrique Corona, Armin Parchami, and Xiaoming Liu. Deviant: Depth equivariant network for monocular 3d object detection. In ECCV , 2022. 1

Abhinav Kumar, Garrick Brazil, and Xiaoming Liu. Groomed-nms: Grouped mathematically differentiable nms for monocular 3d object detection. In CVPR , 2021. 1

Hyunjoon Lee, Eli Shechtman, Jue Wang, and Seungyong Lee. Automatic upright adjustment of photographs with robust camera calibration. TPAMI , 2013. 1, 3, 7

Jinwoo Lee, Hyunsung Go, Hyunjoon Lee, Sunghyun Cho, Minhyuk Sung, and Junho Kim. Ctrl-C: Camera calibration transformer with line-classification. In ICCV , 2021. 3, 7, 8

Jin Han Lee, Myung-Kyu Han, Dong Wook Ko, and Il Hong Suh. From big to small: Multi-scale local planar guidance for monocular depth estimation. arXiv preprint arXiv:1907.10326 , 2019. 3

Haoang Li, Ji Zhao, Jean-Charles Bazin, Wen Chen, Zhe Liu, and Yun-Hui Liu. Quasi-globally optimal and efficient vanishing point estimation in manhattan world. In ICCV , 2019. 3

Zhengqi Li and Noah Snavely. MegaDepth: Learning single-view depth prediction from internet photos. In CVPR , 2018. 7, 8, 10

Feng Liu and Xiaoming Liu. Voxel-based 3d detection and reconstruction of multiple objects from a single image. In NeurIPS , 2021. 1

Feng Liu and Xiaoming Liu. 2d gans meet unsupervised single-view 3d reconstruction. In ECCV , 2022. 1

Jinjiang Liu and Xueliang Zhang. DRC-NET: Densely connected recurrent convolutional neural network for speech dereverberation. In ICASSP , 2022. 10

Manolis Lourakis and Rachid Deriche. Camera self-calibration using the singular value decomposition of the fundamental matrix: From point correspondences to 3D measurements . PhD thesis, INRIA, 1999. 8

Yi Ma, Stefano Soatto, Jana Košecká, and Shankar Sastry. An invitation to 3D vision: from images to geometric models , volume 26. Springer, 2004. 1

Iaroslav Melekhov, Juha Ylioinas, Juho Kannala, and Esa Rahtu. Relative camera pose estimation using convolutional neural networks. In ACIVS , 2017. 3

Cosimo Patruno, Roberto Marani, Grazia Cicirelli, Ettore Stella, and Tiziana D'Orazio. People reidentification using skeleton standard posture and color descriptors from RGB-D data. Pattern Recognition , 2019. 8

Simon Placht, Peter Fürsattel, Etienne Assoumou Mengue, Hannes Hofmann, Christian Schaller, Michael Balda, and Elli Angelopoulou. Rochade: Robust checkerboard advanced detection for camera calibration. In ECCV , 2014. 1

Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. SuperGlue: Learning feature matching with graph neural networks. In CVPR , 2020. 10

Grant Schindler and Frank Dellaert. Atlanta world: An expectation maximization framework for simultaneous low-level edge grouping and camera calibration in complex man-made environments. In CVPR , 2004. 1, 3

Johannes Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In CVPR , 2016. 8

Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. ECCV , 2012. 7, 9

Gilles Simon, Antoine Fond, and Marie-Odile Berger. A-Contrario horizon-first vanishing point detection using second-order grouping laws. In ECCV , 2018. 3

Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. A benchmark for the evaluation of RGBD SLAM systems. In IROS , 2012. 7, 9

Peter Sturm. Multi-view geometry for general camera models. In CVPR , 2005. 3

Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. LoFTR: Detector-free local feature matching with transformers. In CVPR , 2021. 10

Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens,

Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In CVPR , 2020. 3, 7, 9

J Sung, Hema Koppula, Bart Selman, and Ashutosh Saxena. Cornell activity datasets: CAD-60 & CAD-120. 8

Zachary Teed and Jia Deng. DeepV2D: Video to depth with differentiable structure from motion. In ICLR , 2020. 3

Rafael Von Gioi, Jeremie Jakubowicz, Jean-Michel Morel, and Gregory Randall. LSD: A fast line segment detector with a false detection control. TPAMI , 2008. 3

Horst Wildenauer and Allan Hanbury. Robust camera self-calibration from monocular images of manhattan worlds. In CVPR , 2012. 1, 3

Scott Workman, Connor Greenwell, Menghua Zhai, Ryan Baltenberger, and Nathan Jacobs. Deepfocal: A method for direct focal length estimation. In ICIP , 2015. 3

Scott Workman, Menghua Zhai, and Nathan Jacobs. Horizon lines in the wil. In BMVC , 2016. 1, 3

Jianxiong Xiao, Andrew Owens, and Antonio Torralba. Sun3D: A database of big spaces reconstructed using SFM and object labels. In ICCV , 2013. 7

Yiliang Xu, Sangmin Oh, and Anthony Hoogs. A minimum error vanishing point detection approach for uncalibrated monocular images of man-made environments. In CVPR , 2013. 1, 3

Qichao Ying, Hang Zhou, Zhenxing Qian, Sheng Li, and Xinpeng Zhang. Robust image protection countering cropping manipulation. arXiv preprint arXiv:2206.02405 , 2022. 3

Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong, Tianyou Liang, Guanying Chen, Shuguang Cui, and Xiaoguang Han. MVImgNet: A large-scale dataset of multi-view images. In CVPR , 2023. 7

Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, and Ping Tan. NeW CRFs: Neural window fully-connected CRFs for monocular depth estimation. In CVPR , 2022. 6

Menghua Zhai, Scott Workman, and Nathan Jacobs. Detecting vanishing points using global image context in a non-manhattan world. In CVPR , 2016. 3

Hui Zhang, Wong Kwan-yee, and Guoqiang Zhang. Camera calibration from images of spheres. TPAMI , 2007. 3

Yueqiang Zhang, Langming Zhou, Haibo Liu, and Yang Shang. A flexible online camera calibration using line segments. Journal of Sensors . 3

Zhengyou Zhang. A flexible new technique for camera calibration. TPAMI , 2000. 3

Zhengyou Zhang. Camera calibration with one-dimensional objects. TPAMI , 2004. 3

Wang Zhao, Shaohui Liu, Yezhi Shu, and Yong-Jin Liu. Towards better generalization: Joint depth-pose learning without posenet. In CVPR , 2020. 3

Shengjie Zhu, Garrick Brazil, and Xiaoming Liu. The edge of depth: Explicit constraints between segmentation and depth. In CVPR , 2020. 3

Shengjie Zhu and Xiaoming Liu. LightedDepth: Video depth estimation in light of limited inference view angles. In CVPR , 2023. 3

Shengjie Zhu and Xiaoming Liu. PMatch: Paired masked image modeling for dense geometric matching. In CVPR , 2023. 10

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe submission #3482, entitled \"Tame a Wild Camera: In-the-Wild Monocular Camera Calibration\" presents a novel self-calibration strategy where the regression of the incidence field of the camera is predicted via a deep neural network before using a RANSAC to filter outliers and regress the intrinsic parameters of the camera.\n\nSTRENGTHS:\n- Without being impeccable, the paper conveys the main idea behind the proposed technique.\n- Combining the incidence field with more traditional methods to regress the camera intrinsic is \"relatively\" novel.\n- The approach is supposed to work even when the images are cropped\n\nWEAKNESSES:\n- The literature is quite incomplete\n- The camera distortion is not considered\n- The approach is not really novel, as incident field regression has already been done in the past\n\nQUESTIONS:\n- 1. One of my main grips on this paper is the claim for novelty. While the reviewer should admit that using the monocular normal and the depth prediction to estimate the camera's intrinsic is new. The \"incidence field\" regression is not novel as it has already been described in \"Deep Single Image Camera Calibration with Radial Distortion, CVPR, 2019\". In this paper, the authors obtained mixed results using such a strategy. However, one of the significant advantages is to be able to deal with optical aberration, which can be challenging to model parametrically. Unfortunately, this manuscript does not integrate camera distortion, which would be meaningful in this context. I believe that introducing this non-parametric approach, is not fully justified, maybe by training [24] on the exact same cropped data would lead to similar results? (I have not seen its detail in the experiment part). \n- 2. In this regard, it would also be beneficial to integrate \"DeepCalib: A Deep Learning Approach for Automatic Intrinsic Calibration of Wide Field-of-View Cameras, CVMP 2018\" in the literature as it deals with distortion and also explicitly tested on cropped images (maybe in supplementary). In their case, the estimation on cropped images fails because they have yet to train on these particular cases. So it may support your narrative. Another missing paper in the literature is the recent T-PAMI \"A Perceptual Measure for Deep Single Image Camera and Lens Calibration\" which might contain some interest in recent developments on deep learning-based calibration of cameras from a single image.\n- 3. Would it be possible to also test on non-cropped images too? I feel it is unfair to test with many cropped images as, in reality, most images in the wild are uncropped\n- 4. Since the approach has been trained on perspective images only, what would happen in case of a large field of view camera with a field of view outside of the training set distribution? In this regard it would be good to have some statistics regarding the distribution of the field on view used for the training.\n- 5. To really work in the wild, integrating radial distortion seems necessary, how would you integrate it in your framework for training and inference?\n\nLIMITATIONS:\nI have already expressed all my concerns regarding this work in the previous sections of this review. For all the reasons above mentioned, I would like to issue a rather mixed opinion regarding the acceptance of this work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method for single-image camera calibration using a 3D prior the authors refer to as an incidence field. The incidence field is the collection of rays originating from some 3D point towards the camera origin, that are incidental to the image plane. The authors describe how the incidence field can be used to recover the camera intrinsics. Next, a method is proposed for recovering the intrinsics by first estimating an incidence field from an image (using a recently introduced neural network architecture, NewCRFs) and then applying RANSAC. An extensive evaluation shows that the proposed method achieves superior performance to recent methods on numerous benchmarks.  Additionally, several example applications are demonstrated.\n\nSTRENGTHS:\n- Addresses an important and interesting problem. Single image camera calibration \"in the wild\" has received a lot of attention lately and is a fundamental vision task. Knowledge of camera intrinsics is essential for numerous applications.\n\n- The proposed approach has technical novelty. In particular, shows how the concept of an incidence field (backprojected rays in 3D space) are related to camera intrinsics, then introduces a method to estimate them (method of [62] plus RANSAC). As far as I know, this has not been done before. Often in this space, a novel take on how to represent something leads to downstream performance benefits.\n\n- The method is extremely simple, which I consider a positive. \n\n- The evaluation is extensive, achieving superior results to recent methods on several benchmarks.\n\n- Numerous interesting applications presented, such as detecting image resizing and cropping.\n\n- Code provided and will be released with data and models.\n\nWEAKNESSES:\n- The evaluation should have descriptions of the metrics and how they are calculated (found in supplemental).\n\n- The quality of the writing is poor in certain aspects. For example, L14 in the abstract: \"With the estimated incidence field, a robust RANSAC algorithm recovers intrinsic.\" Ultimately, the manuscript just needs a solid editing pass top-to-bottom.\n\n- The quality of the figures, though parseable, could be improved from an aesthetic sense.\n\nQUESTIONS:\nMy initial rating is a weak accept. The proposed approach is simple, explained well, has novelty, and the results are compelling. I think this will be of interest to the community.\n\nSuggestions:\n\n- Full editing pass to address minor language issues. Improve aesthetics of figures (e.g., Figure 2, left).\n\n- Implementation details should be in the main document instead of supplemental.\n\n- L233 should reference Table 3. Text should introduce the dataset.\n\nLIMITATIONS:\nLimitations minorly touched on in the manuscript.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces incidence field, a per-pixel representation for single image camera intrinsics calibration. The incidence field is defined as a 2D vector, pointing to the principal point of the image, normalized by the focal length. It is invariant to cropping and resizing. The paper suggests using a neural network to regress the incidence field given an input image. Finally, RANSAC is used to get the 4DoF intrinsics (fx, fy, cx, cy).\n\nSTRENGTHS:\nThe paper connects the incidence fields with surface normals and depthmaps. Thus, making it a reasonable representation that can be predicted by a network such as NewCRF, which is originally designed for depth estimation. \n\nThe incidence field formulation is simple yet powerful. It has desirable properties such as invariant to arbitrary cropping and resizing, as shown in S3.3; and can be derived from surface normals and depth, as shown in S3.2. \n\nIncidence fields are independent of camera extrinsic, contrasting Perspective Fields [26], which requires camera roll and pitch for formulation. This feature enables incidence fields to be trained on extensive data where the camera is calibrated but the pose is unknown.\n\nThe paper demonstrates SOTA performance on various datasets, both qualitatively and quantitatively.\n\nWEAKNESSES:\n- The paper claims in its contribution that “Our method meaks no assumption for the to-be-calibrated image”, while in fact, it presumes no distortion in the image (L118). \n\n- Table 1 needs revision, as it inaccurately describes the baselines and the comparisons are not apple-to-apple. Contrary to the claims made, [24, 26] do not assume Manhattan data during training and can be trained on arbitrary scenes, such as natural scenes. Furthermore, it is unfair to state that your method requires no assumption on the training data when comparing it with others [24, 31, 26], since they also predict camera roll and pitch while your method does not. \n\n- There are missing details in Table 2. It is unclear what the unit for the error metric is, and which dataset [26] is trained on. If [26] is solely trained on GSV data, then it implies a zero-shot scenario for [26] on all the listed datasets in Table 2. If camera extrinsics are provided, you should be able to train [26] on the same training set as your method, ensuring a fair comparison.\n\n- Including a baseline that predicts intrinsics from depth and surface normal predictions (as described in Sec 3.2) would be beneficial. This would empirically validate the derivation and give an idea of the method’s accuracy in Sec 3.2. Furthermore, it would support the argument in L150 that “Minimal solver in Eq. (9) can lead to a poor solution”, therefore further justifying the contribution of the method in Sec 3.4. \n\n- Typo in L218 and L233: There is no [26] in Tab 4. It should be Tab 3 instead. \n\n- In Table 5 and Sec 4.4, the baseline method is not clarified.\n\nQUESTIONS:\nSee weakness.\n\nLIMITATIONS:\nIn real application, whether to apply the assumption still waits for human input.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAn in-the-wild monocular camera calibration method is proposed. It allows to estimate the focal lengths $f_x$ and $f_y$ as well as the optical center $b_x$, $b_y$ without any additional information such as a checkerboard or the Manhattan world assumption. The proposed method consists in employing a neural network to predict the incidence field, followed by a RANSAC. Several experiments are performed to demonstrate the ability of the proposed method to perform in-the-wild calibration.\n\nSTRENGTHS:\n1. The proposed approach is simple and efficient.\n2. Training a network to predict the incidence field seems straightforward (the authors took a depth prediction network architecture and simply changed the last layer and the loss).\n\nWEAKNESSES:\n1. The idea of using a sota depth prediction network to predict the incidence field (section 3.4) seems new, but this is a rather weak contribution. If I am not mistaken, it essentially consists in taking the network from a github page, change the last layer and the cost function and retrain the network on the same data. This contribution seems weak for a conference like NeurIPS.\n2. The idea of using the incidence field to estimate the calibration parameters (section 3.5) seems new, but the derivation is trivial: here the minimal solver is the solution of a trivial linear system (it takes a few minutes to derive, compared to Grubner-basis-based minimal solvers that we encounter in essential matrix estimation). In terms of contribution, I believe this is not strong enough for NeurIPS. Btw, I believe eq.17 and 18 do not correspond to a RANSAC since you simply quantize the focal length space and test each value. \n3. I believe section 3.2 is not a contribution, is it ? It is said that this calibration method will not produce good results but I could not find any experiment using it. I suggest to include this experiment.\n\nQUESTIONS:\nAfter carefully reading the paper, I recommend to reject it. I believe the contributions are not sufficient for a conference like NeurIPS and not interesting for the broader NeurIPS community. I suggest to submit the paper to another conference like 3DV or CVPR.\n\nLIMITATIONS:\n.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a novel learning based approach for 4 DOF camera intrinsics calibration from a single image in the wild. They propose to use a neural network to predict the incidence rays for each pixel in an image from which the camera intrinsics can be recovered. The authors motivate the prediction of the incidence rays as an alternative for predicting the depth and normals, as the computation of the intrinsics from these signals involves derivatives it is very succesible to noise. The evaluation of the authors shows great potential of this method to generalize to images in the wild and show potential applications like uncalibrated 2-view pose estimation and image transformation detection.\n\nSTRENGTHS:\nThe idea of leveraging the relationship betwen the scene depth and normals and the incidence rays is very interesting.\nThe explanations are great and the neccesary derivations are all there. There are comparisons to both monocular methods using geometry and methods using known objects.\n\nWEAKNESSES:\nMost of the tables (2, 4 & 6) with the quantitative results do not provide a metric unit and while the results are compared to existing methods, the metrics should be introduced properly. For 4.4 and table 5 the description and or reference of the baseline is missing.\n\nQUESTIONS:\nPlease add the metrics to the tables, introduce them and provide a reference for the definition.\nAdd the description of the baseline in 4.4.\nHow does the performance depend on the image content and the sampling of pixels? E.g. what samples are usually providing the best hypotheses and what would intuitively be a minimal image content that would allows the model to predict the incident rays? The same as for depth and normal prediction?\n\nLIMITATIONS:\nSimilar to depth and normal prediction there are probably cases where the image content is not sufficient to predict incidence rays. The authors did not discuss this case as far as I can see.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-08"}