PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
Authors: Sunghwan Hong
Jaewoo Jung
Heeseong Shin
Jisang Han
Jiaolong Yang
Chong Luo
Seungryong Kim
Venue: ICML 2025 poster (2025)
Keywords: Pose-free
novel view synthesis
3D reconstruction
Primary area: applications->computer_vision
TL;DR: A feed-forward model to estimate 3D Gaussians for novel view synthesis

Abstract: We consider the problem of novel view synthesis from unposed images in a single feed-forward. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS, where we further extend it to offer a practical solution that relaxes common assumptions such as dense image views, accurate camera poses, and substantial image overlaps. We achieve this through identifying and addressing unique challenges arising from the use of pixel-aligned 3DGS: misaligned 3D Gaussians across different views induce noisy or sparse gradients that destabilize training and hinder convergence, especially when above assumptions are not met. To mitigate this, we employ pre-trained monocular depth estimation and visual correspondence models to achieve coarse alignments of 3D Gaussians. We then introduce lightweight, learnable modules to refine depth and pose estimates from the coarse alignments, improving the quality of 3D reconstruction and novel view synthesis. Furthermore, the refined estimates are leveraged to estimate geometry confidence scores, which assess the reliability of 3D Gaussian centers and condition the prediction of Gaussian parameters accordingly. Extensive evaluations on large-scale real-world datasets demonstrate that PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. We will make the code and weights publicly available.

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
Sunghwan Hong"
Jaewoo Jung
Heeseong Shin
Jisang Han Jiaolong Yang" Chong Luo
Seungryong Kim
Abstract
2024), and substantial image overlaps (Yu et al., 2021; Jo- hari et al.. 2022), which limit their practical applicability. In real-world scenarios, casually captured images contain sparse and distant viewpoints, and lack precise camera poses, making it impractical assume densely captured views with accurate camera poses.
We tackle the problem of view synthesis from sparse, unposed images in single feed-forward pass. Our method builds on 3DGS and relaxes common requirements such as dense views, accu- rate camera poses or depth, and large image over- laps. However, the main challenge arises from the parametrization of pixel-aligned 3D Gaussians, as their misalignments inevitably yield noisy sparse gradients that destabilize training. To address this, we leverage pretrained monocular depth estimation and visual correspondence net- works for coarse alignment, then refine depth and pose via lightweight learnable modules. We fur- ther estimate geometry confidence scores, driven by aggregated monocular and multi-viee view depth, to assess the reliability of 3D Gaussian centers and condition the prediction of Gaussian parame- ters accordingly. Extensive experiments on large- scale real-world datasets confirm that PF3plat achieves state-of-the-an performance across all benchmarks, with ablation studies validating our design choices.
To address some of these limitations, recent efforts (Yu et al., 2021; Johari et al., 2022: Chen et al., 2021; Yang et al., 2023) have introduced generalized view synthesis frame- works capable of performing single feed-forward novel view synthesis from sparse images with minimal overlaps (Du et al., 2023; Xu et al., 2023a). Among these methods, par- ticularly those utilizing 3DGS (Charatan et al., 2023; Chen et al., 2024), have demonstrated remarkable rendering speed and efficiency, alongside impressive reconstruction and view synthesis quality, highlighting the potential of 3D Gaussian- based representations. However, they still depend on ac- curate camera poses, which are challenging to acquire in sparse settings, thereby restricting their practical use.
More recently, pose-free generalized view synthesis frame- works (Chen & Lee, 2023; Fan et al., 2023; Jiang al., 2023; Smith et al., 2023; Hong et al., 2024) have been intro- duced to decouple view synthesis from camera poses. Given a set of unposed images, these frameworks aim to jointly learn radiance fields and 3D geometry without relying on ad- ditional data, such as ground camera pose. The learned radiance fields and geometry can then be inferred through infer-
1. Introduction
In recent years, 3D reconstruction and view synthesis have garnered significant attention, particularly with the emer- gence of NeRF (Mildenhall et al., 2021) and 3DGS (Kerbl al., 2023). These advancements have enabled high-quality 3DD reconstruction and view synthesis. However, many isting methods rely on stringent assumptions, such as dense image views (Yu et al., 2024; Barron et al., 2021; 2022), accurate camera poses (Charatan et al., 2023; Chen et al.,
trained neural networks, enabling single feed-forwer ence. While these pioneering efforts enhance practicality, their performance remains unsatisfactory and their slow ren- dering speeds (Chen & Lee, 2023; Fan et al., 2023; Jiang et al., 2023; Smith et al., 2023; Hong et al., 2024) remain unresolved.
In this work, we propose PF3plat (Pose-Free Feed-Forward 3D Gaussian Splatting), novel framework for fast and pho- torealistic view synthesis from imposed images in a single pass. Our approach leverages the efficiency and high-quality reconstruction capabilities of pixel-aligned 3DGS (Charatan et al., 2023; Szymanowiez et al., 2024), while relaxing common assumptions such as dense image views, accurate camera poses, scene-specific optimization
Eqqual contribution 'KAIST AI "Microsoft Research Asia. Correspondence to: Jiaolong Yang <jiaoyan @ microsoft.com Chong Luo <cluo@miertt caaio@@micosso..omm.... Seungryong Kim <seungry- ong.kim kim kaist.ac.kr>..
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
and substantial image overlaps. However, a unique chal- lenge emerges in the parametrization of pixel-aligned 3DGS. In scenarios lacking the aforementioned assumptions, inac- curacies in localizing 3D Gaussian centers can lead to noisy or sparse gradients, destabilizing training and hindering convergence. Moreover, unlike scene-specific optimization approaches (Fu et al., 2023; Fan et al., 2024), which can iteratively rectify errors at inference time, typical learning- based feed-forward frameworks are unable to benefit from such iterative refinements.
sequent research (Jeong al., 2021; Wang et al., 2021; Bian et al., 2023; Truong et al., 2023b) has expanded upon this foundation. Notably, the use of 3D Gaussians as dy- namic scene representations has led to significant advance- ments: Fu et al. (2023) progressively enlarges 3D Gaussians by learning transformations between consecutive frames, SplaTAM et al., 2024) utilizes RGB-D sequences and silhouette masks to jointly update Gaussian parameters and camera poses, and InstantSplat (Fan et al., 2024) opti- mizes 3D Gaussians rapidly for scene reconstruction and view synthesis. Among these, methods like DBARF (Chen & Lee, 2023), FlowCAM (Smith et al., 2023), CoPoN- eRF (Hong et al., 2023), and GGRt (Li et al., 2024) aim to determine camera pose and radiance fields in a single pass. Concurrently, Splatt3R (Smart al., 2024) and NoPoSplat (Ye et al., 2024) leverage pre-trained 3D reconstruction models (Leroy et al., 2024) to relax cer- tain assumptions. However, these methods either target two-view scenarios or rely on additional supervision (e.g.. ground-truth depth and pose) during training.
To mitigate these issues, we leverage pre-trained monocular depth estimation (Piccinelli et al., 2024) and visual corre- spondence (Lindenberger et al., 2023) models to achieve a coarse alignment of 3D Gaussians to promote a stable learn- ing process. Subsequentl, we introduce learnable modules designed to refine the depth and pose estimates from the coarse alignment to enhance the quality of 3D reconstruction and view synthesis. These modules are geometry-ave -aware and lightweight, since we leverage features from the depth net- work and avoid direct fine-tuning. These refined depth and pose estimates are then used to implement geometry-ave confidence scores to assess the reliability of 3D Gaussian centers, conditioning the prediction of Gaussian parameters such opacity, covariance, and color.
Monocular Depth and Correspondence Estimation Monocular depth estimation and visual correspondence es- timation are fundamental computer vision tasks that have been extensively researched over several decades. Recent advancements (Yin et al., 2023; Piccinelli al., 2024; Ke et al., 2024; Yang al., 2024) in monocular depth estimation have greatly matured these models, thereby expanding the capabilities of numerous 3D vision applications. Likewise, visual correspondence estimation has undergone remarkable progress since the emergence of deep neural networks. Mod- ern approaches have optimized each step, ranging from 2D descriptor extraction (Yi et al., 2016; DeTone et al., 2018) and 3D descriptor extraction (Yew & Lee, 2018; Choy et al., 2019), to sparse and dense matching (Hong & Kim, 2021;
Our extensive evaluations on large-scale real-world indoor and outdoor datasets (Liu et al., 2021; Zhou et al., 2018; 8; Ling et al., 2024) demonstrate that PF3plat sets a new state- of-the-art across all benchmarks. Comprehensive ablation studies validate our design choices, confirming that our framework provides fast and high-performance solution for pose-free generalizable novel view synthesis. We sum- marize our contributions below:
We propose PF3plat, a feed-forward network that re- construets 3D scenes, parameterized by 3D Gaussians, from sparse, unposed views without requiring ground- truth depth or pose at either training or inference phase.
Cho et al., 2021; 2022; Hong et al., 2022a;b; 2023; Sun et al., 2021; Edstedt et al., 2024) and outlier filtering (Barath et al.,
We propose a two-stage pipeline, coarse and fine align- ment, to address the unique challenges of pixel-aligned 3D Gaussian parameterization and further enhance the reconstruction and view synthesis quality, respectively.
2019; Wei et al., 2023), surpassing traditional methods in many scenarios.
Using large-scale real-world indoor and outdoor dat- sets, we show that our method outperforms existing works terms of rendered image quality and the cam- era pose accuracy.
Learning-based 3D Reconstruction. Meanwhile, learning-based 3D reconstruction methods (Wang et al., 2023; Leroy et al., 2024; Wang et al., 2024) have garnered significant attention for their performance in sparse-view reconstruction, forming robust foundation for tasks such as camera pose estimation and tracking. In contrast, our approach simultancously tackles 3D reconstruction and view synthesis without requiring lengthy radiance field optimization at inference, enabling more direct and efficient inference.
2. Related Work
Generalizable View Synthesis from Unposed Imagery. Several innovative efforts have addressed the joint learn- ing of camera pose and radiance fields within Nese-based frameworks. Starting with BARF (Lin et al., 2021), sub-
2

---

PF3plat: Pose-Free Feed Beed-Forward 3D Gaussian Splatting for Novel View Synthesis
Frazen
Unproject with T)
E2n-20
Project P
Monocular Depth Module
Depth Refinement
D
Gaussian Predictor
Inital Depti
Retined Depth
Pose Salver
P
Confidence
S Conficience
G
LA E.
Du AA
Matching Module
M Conespendence
Refinement
Relimed Prose
Predict
Predict Gj
Egp
Σ
(AC Hy(0
Coarse Alignment Module
Fine Alignment Module
Ipa (b) Illustration of Training Losses
(a) Overall Framework
Figure/ 1. Overall architecture and loss of the proposed method. (a) Given of unposed images and their camera intrinsics, method aligns the 3D Gaussians using a coarse-to to foneett-oeeeeeeee strategy. (b) In addition to photometric loss, we enforce 3D Gaussian consistency by ensuring they are placed on the same object surface through 2D-3D and 3D-3D consistency losses.
3. Method
nature of this representation. Specifically, inaccuracies in localizations of 3D Gaussian centers makes highly vulner- able to noisy and sparse graidents, which cannot be easily compensated.
3.1. Problem Formulation
Our objective is to reconstruct a 3D scene from a set of N un- posed images {4.)/= with 1; E IR #x x3 and correspond- ing camera intrinsic K;, to synthesize photo-realistic im- ages 1, from novel viewpoints in single feed-forward pass. Note that, in line with existing pose-free view synthesis methods (Fu al., 2023; Ye et al., 2024; Hong et al., 2024; Chen & Lee, 2023; Smith et al., 2023), we assume camera intrinsic parameters are available, as they are generally avail- able from modern devices (Amoll et al., 2022). To render, we output the depth maps D, R #x for each image I,, along with their corresponding camera poses P, R3x4 consisting of rotation matrix R, R3x3 and translation vector 4; E R3x1, Additionally, we compute set of pixel- aligned 3D Gaussians denoted as G = {Hi oj, E, a } Here, (p) E RR33 indicates the 3D Gaussian center derived from the depth (p),, camera pose P,, and camera intrinsic K,, where p E R.H represents each pixel. Finally, the opacity is represented by (p) 1), (p) E g3x3 is the covariance matrix, and the color is encoded using spher-
Such misalignments and sprase gradients can either cause severe performance degradation or disrupt the learning pro- cess. This issue particularly exacerbated when wide- baseline images are given as input or the absence of ground- truth pose depth prevents alignments of 3D Gaussians. Without effectively addressing these challenges, we find the problem becomes nearly intractable, we demonstrate in Tab. 4. A possible solution to mitigate this issue is to em- poloy iterative scene-specific optimization steps (Fu et al., 2023) or to assume ground-truth camera poses or depth as a guidance for stable learning process (Ye al., 2024). However, these solutions are incompatible with our goal of achieving single feed-forward process with training solely from unposed images. Therefore, overcoming these limitations requires a novel strategy that can handle depth and pose ambiguities while maintaining efficiency in a feed- forward manner.
ical harmonics (p) E R3(4+1), where L is the order of the spherical harmonics.
To this end, we propose to provide coarse alignment of 3D Gaussians. We employ off-the-shelf models (Piccinelli et al., 2024; Lindenberger al., 2023) to estimate initial depths D; and camera poses P: for our images I, while other variants can also be leveraged, as we show in supple- mentary material. Specifically, given depth maps D; and sets of correspondences Mij and their confidence values Cjj acquired from each pairwise combinations of images, e.g. (Ji, I5), where i, refer to image indices, use robust solver (Fischler & Bolles, 1981; Li et al., 2012) to estimate the relative poses Pıj between image pairs. Inte- grating these components, we provide the necessary coarse alignment to promote stabilizing the learning process and serve as a strong foundation for further enhancements.
3.2. PF3plat: Pose-Free Ferd-Forward 3D Gaussian Splatting
3.2.1. COARSE ALIGNMENT OF 3D GAUSSTANS
Despite numerous advantages such as speed, efficiency, and high-quality reconstruction and view synthesis (Charatan et al., 2023; Chen et al., 2024), pixel-aligned 3D Gaus- sians poses certain challenges. Unlike previous methods for generalized novel view synthesis that utilize implicit rep- resentations (Chen & Lee, 2023; Smith et al., 2023; Hong al., 2024) and benefit from the interpolation capabilities of neural networks, our approach is challenged by the explicit
3

---

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
3.2.2. MULTI- CONSISTENT DEPTH ESTIMATION
ken is then transformed into 6D rotations (Zhou et al., 2019) and translations, which are added to the initial camera poses to estimate the rotation and translation offsets. Formally, these are defined as:
While pre-trained monocular depth models (Wang et al., 2023; Leroy et al., 2024; Piccinelli et al., 2024; Yin et al., 2023) can offer powerful 3D geometry priors, inherent limi- tations of these models, namely, inconsistent scales among predictions, still remain unaddressed. This requires further adjustments to ensure multiview consistency across predic- tions. To overcome this challenge, we aim to refine the predicted depths and camera poses obtained from coarse alignment in fully learnable and differentiable manner.
PCLS - Tpose (FFr, PCTS: r] + Epoa), AR. Al = Prot (Pots); Prrans (Pets),
(2)
Re, f Rr + A RR At,
where AR, At are the pose offsets, and Epos is positional embedding.
Our refinement module includes pixel-wise depth offise estimation that uses the feature maps F; from the depth net- work (Piccinelli et al., 2024) as the sole input and processes them through a series of self-attention operations, making lightweight and geometry-aware (Xu et al., 2023b). The process is defined as:
3.2.4. 3D GAUSSIAN PARAMTER PREDICTIONS
Multi- and Guidance Cost Volume Construction and Aggregation. Using the refined pose and the monoc- ular depth estimates, P, and D., we assess the quality of the predictions to obtain confidence scores, to assist predict- ing 3D Gaussian parameters. To achieve this, we construct both a conventional multi-view stereo cost volume and a guidance cost volume derived from D;,
Ad,- ( (F)),, = + Ad,, (1)
where (m) is a linear projection, Taepth is a deep Trans- former architecture and Λ6; is the pixel-wise depth off- set. This extension promotes consistency across views and enhances performance without relying on explicit cross- attention. Instead, it leverages supervision signals derived from pixel-aligned 3D Gaussians that connect the informa- tion across views, and leverage them for the novel view synthesis task (Zhou et al., 2017) and our loss functions, which are detailed in Section 3.3. Additionally, we avoid fine-tuning the entire depth network, thereby reducing com- putational costs and mitigating the risk of catastrophic for- getting.
Specifically, given the current pose estimates P., we build multi-view stereo cost volume C.oulti R follow- ing the plane-sweeping approach (Yao et al., 2018; Chen et al., 2021: 2024; An et al., 2024). For each of the K depth candidates within specified near and far ranges along the epipolar lines, we compute matching scores using cosine similarity (Cho et al., 2022; 2021; Hong et al., 2024). Sub- sequently, to guide the depth localization along the epipo- lar lines, we construct a guidance cost volume Cruide (Li et al., 2023), where each spatial location is represented by a one-hot vector indicating the depth candidate closest to the monocular depth estimate. We finally obtain an aggregated cost volume CAAS by feeding C.malti and Cguide to series of cross- aesseattentoon layers to update the multi-view cost volume guided by a guidance cost volume. We define the process as following:
epipo-
3.2.3. CAMERA POSE REFINEMENT
Here, we further refine camera poses to enhance reconstruc- tion and view synthesis quality. Initially, we replace the estimated relative poses P.j with newly computed camera poses Pij derived from following the similar process in coarse alignment and using the previously obtained refined depths D,.. We then introduce learnable camera pose refine- ment module that estimates rotation and translation offsets. To streamline this process, we first utilize fully differen- tiable transformation synchronization operation that takes P. and Cij as inputs. Using power iterations (El Banani et al., 2023), this operation efficiently recovers the absolute poses P. prior to the refinement module.
C.E.K = Tagg (Cag (Cmmulti , CRuide)
where T(-) is deep transformer architecture that computes cross-attention between inputs.
Geometry-aware Confidence Estimation. Using the ag- gregated cost volume C we apply softmax function (Xu et al., 2022: Hong et al., 2023) along the K dimension to ob- tain a matching distribution. We then extract the maximum value from this distribution to derive a confidence score, Sgea, which assesses the quality of the estimated camera pose and depth. Formally, this is defined as:
Next, we convert the absolute poses P. into Plücker coordi- nates (Sitzmann et al., 2021), defined as r (d, oxd) E Æ where d represents the camera direction and denotes the camera origin. These coordinates, along with the feature maps F. E Ig hxx and a pose token POLS E Rd, are in- put into a series of self- and approach, we designate P. as the reference world space and update only the other pose estimates. The resulting pose to-
Sg00
max vofumax(C,.R))) (k). (1........
(4)
layers. In our
These confidence scores assess the reliability of the pre- dicted 3D Gaussian centers, where high confidence indicates
4

---

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
accurate localization and low confidence suggests potential inaccuracies due noise misalignment. To condition the prediction of Gaussian parameters, such opacity, covari- ance, and color, we incorporate Sgeo as additional input.
Intuitively, this differs from LyD-3D in that, unlike the previous function, which considers the alignment of Gaus- sian centers from only one side when dealing with pair- wise correspondences, the regularization term symmetri- cally enforces consistency from both sides. Specifically, while the multi-view consistent surface loss projects the Gaussian center from one view to another using the esti- mated depth and camera pose, e.g., from source target, the regularization term jointly minimizes the distances be- tween all corresponding Gaussian centers across multiple views. By considering both directions in pairwise corre- spondences, this approach promotes a more coherent and robust estimation of the object's 's surface, reducing the influ- ence of outliers and improving convergence during training. This additional regularization can be formally defined as: LaD-aD 3D = (pp-)eM [lee(p) - p;(g))le
3D Gaussian Parameters. Finally, using the inputs [T, Dr, Pi, J J. we compute the opacity o; through small convolutional layers, derive the covariances from the esti- mated rotations and scales, and obtain the color from the estimated SH coefficients. A key idea of our approach is that 8000 enables supervision signals to flow from the Gaus- sian parameters back the depth and pose estimates. This feedback loop enhances the accuracy of both depth and pose estimations, resulting in more consistent and reliable 3D reconstructions.
3.3. Loss Function
Reconstruction Loss. We combine photometric loss, de- fined as the L2 loss between the rendered and target im- ages, as well as SSIM (Wang et al., 2004) loss LSSIM and LPIPS (Zhang et al., 2018) loss LiePs to form our recon- struction loss Limg.
Final Objective Function. Combining the three loss functions, we define our final objective function: Limg + CyD + -DDD-aD 3D. where we set ADD-3D = 0.05.
4. Experiments
2D-3D Consistency Loss. We identify that provided good coarse alignments, RGB loss is sufficient, as similarly ob- served in (Ye et al., 2024), but with larger baselines, the training process starts to destabilize. Moreover, one remain- ing issue with learning solely from the photometric loss is that the gradients are mainly derived from pixel intensity differences, which suffer in textureless regions. To remedy these, we enforce that corresponding points in the set of images ( N lie on the same object surface, drawing from principles of multi-view geometry (Hartley & Zisserman, 2003).
4.1. Implementation Details
We compute attentions using Flash Attention (Dao et al., 2022), and for the Gaussian rasterizer, we follow the method described in (Charatan et al., 2023). Our model is trained on 4 NVIDIA A100 GPU for 50,000 iterations using the Adam optimizer (Kingma, 2014), with a learning rate set to 8 x 10-1 and a batch size of 9 per each GPU, which takes approximately two days. For training on the RealEstate 10K and ACID datasets, we gradually increase the frame distance between In and 12 training progresses, initially setting the frame distance to 15 and gradually increasing it to 75. For the DL3DV dataset, we start with frame distance of 5 and increase it to 10. The target view is randomly sampled within this range. The code and pretrained weights will be made publicly available.
Formally, using the estimated depths Î), the camera poses P. and the correspondence sets M, we can define a geo- metric consistency loss that penalizes deviations from the multi-view geometric constraints. For each correspondence (p. 9) E Mij between images I; and I,. we compute the 3D point from the pixel p and its estimated depth (p((p) using the camera intrinsics. We then transform this to the coordi- nate frame of I; using the relative pose P.j and project it back onto the image plane to obtain the predicted correspon- dence p. This is defined as L2D-3D = E (0 g), where (p-) denotes huber loss.
4.2. Experimental Setting
Datasets. We train and evaluate our method on three large- scale datasets: RealEstate 10K (Zhou et al., 2018), a collec- tion of both indoor and outdoor scenes; ACID (Liu et al., 2021), a dataset focusing on outdoor coastal scenes; and DL3DV (Ling et al., 2024), which includes diverse real- world indoor and outdoor environments. For RealEstate 10K, due to some unavailable videos on YouTube, we use a sub- set of the full dataset, comprising a training set of 21 21,618 scenes and a test set of 7,200 scenes. For ACID, we train on 10,935 scenes and evaluate on 1,893 scenes. Lastly, for DL3DV, we train on 10,510 different scenes and evaluate on the standard benchmark set of 140 scenes for testing (Ling et al., 2024).
3D-3D Consistency Loss. While the multi-view consis- tent surface loss directly connects each pair of correspond- ing Gaussians and their centers, guiding the model towards the object's surface, we find that relying solely on this loss can lead to suboptimal convergence, especially in regions with sparse correspondences. To further stabilize and en- hance the learning process, we introduce an additional regu- larization term that minimizes the discrepancies among the centers of the corresponding Gaussians.
5

---

PF3plat: Pose-Free Feed Beed-rorward 3D Gaussian Splatting for Novel View Synthesis
Tdephh
Jose
x6
transformer
connforme
transformer encoder
Recader
Canuformer
mranformer Poodeer
AR
F
CLs
E us
P
CLS
MLP
At
Fiider anbed.
R,t
Geometry-aware Confidence Estimation
cost volume
colpolar
black
residual
Beconde
traosformet
downsample
black
residual
residual
decoder
transforme:
2x
downsample
C
C
crOSs attemtion
CaBs
$880
one-hot
Caulde
Figure 2. Proposed refinement and confidence estimation modules. In our Fine Alignment module, we refine depth and pose to improve 3D reconstruction and view synthesis quality, alongside estimating confidence assess the reliahility of predicted 3D Gaussian centers.
ACID
REIOK
(a)
(b) 12
(c) DBARF
(d) FlowCAM
(e) CoPoNeRF
(f) Ours
(g) GT
Figure 3. Qualitative Comparison on RealEstate- and ACID, Given two context views (a) and (b), compare novel view rendering results.
Baselines. Following (Hong et al., 2024), we evaluate our method on two tasks: novel-view synthesis and cam- era pose estimation. For novel view synthesis, we com- pare our approach against established generalized NeRF and 3DGS variants, including PixelNeRF (Yu al., 2021), (Du al., 2023), PixelSplat (Charatan al., 2023), and MVSPlat (Chen et al., 2024). It is important note that these methods assume GT camera poses during training and inference, so we include them only for reference. Our pri- mary comparisons focus on existing pose-free generalized novel view synthesis methods, such as DBARF (Chen & Lee, 2023), FlowCAM (Smith et al., 2023), and CoPoN- cRF (Hong et al., 2024).
compare with direct pose regression methods such as 8VIT (Rockwell et al., 2022), RelPose (Zhang et al., 2022). Our main comparisons include pose-free view synthesis approaches, including DBARF (Chen & Lee, 2023), Flow- CAM (Smith et al., 2023), and CoPoNeRF (Hong al., 2024).
Evaluation Protocol For evaluation, we follow the pro- tocol outlined by (Hong et al., 2024) using unposed triplet images (4,, I2, It), with the set divided into small, mid- dle, and large based the extent of overlap between 11 and Iz. We also provide multi-view evaluation in Tab. 5c. We provide further details in the supplementary materials.
For camera pose estimation, correspondence-based pose estimation methods (e.g.. SfM), including SP+SG (DeTooee et al., 2018; Sarlin et al., 2020), PDC-Net+ (Truong et al., 2023a), DUS13R (Wang et al., 2023), and MASt3R (Leroy et al., 2024). Due to absence of GT depth in our datasets, we leverage their pre-trained weights and direct comparison is avoided. Additionally,
evaluate against
4.3. Experimental Results
RealEstate- -00 & ACID. Tab. summarizes the perfor- mance for the novel view synthesis task, while Tab.2 reports the results for pose estimation. From the results in Tab. 1, our method significantly outperforms previous pose-free generalizable methods (Chen Lee, 2023; Smith al.,
6

---

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
Table Novel View Synthesis Performance on RealEstate- and ACID, Gray entries indicate methods that use ground truth camera poses during evaluation and are not directly comparable.
RealEsttate- 16K
Avg
Small
Medium
Large
Pose-Free Method
PSNR
LPIPS
SSIM
MSE
PSNR
LPIPS
SSIM
MSE
PSNR
LPIPS
SSIM
MSE
PSNK
LPIPS
SSIM
MSE
PiseiNoRF
14,438
0:577
0.047
13.126
0.639
0.055
13,099
0.583
0.42
15.445
0.479
0.031
21.833
0.294
0.736
0.011
18.733
0.578
0.661
0.018
22.553
0.263
0.764
0.088
26.199
0.182
0.836
0.004
PisclSylat
24.788
0.176
0.820
0.005
21.222
0.246
0.342
0.013
26.105
0.862
0.135
0.004
29.545
0.916
0,892
0.003
MVS3taan
25,054
0.157
01827
0.008
21,029
0.226
0.747
0.013
26,369
0.116
0.874
1.004
30.516
0.074
0.926
0,002
DEARF
14,789
0.490
0.570
0.033
13,453
0.563
0.522
0.045
15.201
0.487
0.560
0.090
16.615
0.380
0.648
0.022
FlowCAM
18:342
0.597
0.023
15.45
0.570
0.034
18.461
0.502
22.455
0.207
0.267
CoPoNeRP
19,536
0.398
0.638
0.016
17.153
0:459
0.577
0.025
19.965
0.343
0.645
0.013
22.542
0.250
0.724
0.088
Ours
23,589
0.181
0.782
0.010
19.998
0.344
0.700
0.015
24.073
0.155
0.819
0.006
28.834
0.098
0.889
0.003
ACID
Apg
Small
Medium
Large
Posp-Freee
Methed
PSNR
LPIPS
SSIM
MSD
PSNR
LPIPS
SSIM
MSB
PSNR
LPIPS
SSIM
MSB
PSNR
LPPPS
SSIM
MSC
PiseNNoRF
17.190
0.527
0.49%
0.029
16.996
0.538
0.487
0.030
17.328
0.534
0.501
0.029
17.229
0.522
0.500
0.028
at
25.482
0.304
0.769
0.005
25.553
0.301
0.005
25.094
0.303
0.769
0.005
25.338
0:307
0.763
0.005
PiselSplat
28.336
0.157
0.004
28.142
0.164
0.004
28.650
0.148
0.846
0.003
28,306
0.157
0.833
0.004
MV-Splat
28.252
0.157
0.829
0.004
28.085
0.164
0.820
0.004
28.571
0.143
0.813
0.003
28.205
0.155
0.825
0.004
DRARF
14.187
0.452
0.537
0.38
14 14.306
0.503
0.541
0.087
14.253
0.457
0.538
0.018
14.086
0441
0.534
0.039
FlowCAM
20.116
0.477
0.585
0.116
20.153
0.475
0.594
0.015
20.158
0.176
0.565
0.015
20.075
0.478
0.580
0,016
CaPeNeRR
22.440
0.323
0.649
0.100
22.322
0.358
0.649
0.010
22.407
0.352
0.648
0.009
32.539
0.351
0.649
0.006
0,006
Table 2. Pose Estimation Performance on RealEstate-10K and ACID. Gray entries indicate methods that were not trained on the same dataset due missing ground-truth data (e.g.. depth or correspondences), making them not directly comparable to ours. In other words, we cannot train our approach on their dataset, nor can they train theirs on ours. *; We also include a MASt3R variant that omits iterative pose optimization (roughly taking 10s) and instead relies on PuP solver to maintain consistency with other baselines.
Rralfxtan--ok tok
Small
Medium
Lorge
Task
Additional Info.
Method
Rosutin
Trandation
Rotation
Trandlrtin
Bosation
Tromlation
Rocation
Transiticn Mod!".)
Axg
Meal" ()
Aval-ia
Modl"l) ()
Argi"1)
Mad"..) ()
Medd '40
Asge (1)
Sied"11
Aug
Meal") ()
Aust" 1)
Madl") ())
Arre"1)
SP+-5G
5.606
1.501
5.008
9.793
12.55
4.63%
0.962
9:200
1.415
0.847
21.42
7,200
3.343
3,400
3.753
6.667
0.981
5.447
Deyth
0.751
MASIRR
3,390
14455
5,045
14.252
5,472
1.361
1.576
42.355
29,590
Poss Extimation
Ploce
ENA
Relltsue
6.385
1230
0.31
3.845
12.00
12.00
4.300
91.50
4.542
0351
34255
4.217
12.1
2442
7214
BK72
X
DHAR
11.14
5.585
61.10
107.5
17.63
13.22
176.1
140.4
7.565
4.700
75.41
3.455
1.997
90.00
13.06
Phne-Free View Syatheis
x
PAMCAM
7.466
4.051
50.66
46 25
13.85
5.778
87.12
55.25
4.154
3345
42:29
41.99
2.349
1.524
34,47
27.79
Pase
CuPwNeRF
2,60
1,799
12.33
7.55
5.673
2,551
11.05
5.344
2,133
1.855
10.19
5.709
1.529
0.591
15.514
1,907
Durs
1.766
0.897
9.874
4.628
3.335
1,800
7.121
4.005
1.287
1.336
9.311
4.36
1.138
0.499
15
5.778
AE30
Small
Mesiun
Tiok
Adiitional Info.
Methed
Reaaiin
Trandarivn
Riration
Tranbiiion
Retatios
Trandation
Rotaton
Tronnlation
Mok".) -)
Aug"()
Mrck1]
Axg"1)
Mcd'() ()
Arg"
Med(")
Aug!
Mad("))
Ange":
Mork.))
Aug"
Ang"1)
Mrd"1)
SP-SG
1.203
20.302
6.278
10:923
2.797
22:214
7.535
3.225
1.306
15.455
5.425
0.345
22.015
PDC-Nes-
1.042
48.409
25:158
2.530
0.579
15.664
4.215
2.378
14:940
4,303
0.636
Depth
Pinr Nntimation
Powe
RelPose
2.567
33,235
3751
4.753
15221
283355
4.32
5.001
90335
0./25
2003
10.200
$7.359
x
BHARE
4.611
3.431
71,711
06,822
8.731
5.305
95. 1:1190
90.000
4.834
1.685
73.324
77.291
2.355
0.59
$4.533
34,839
Pom-FFre Syothesis
x
FosCAM
9101
6.499
95.405
98:133
8.665
6.655
92.130
55 55.545
8.725
5.589
95.444
$7.905
9.36
6.805
97.50 07.992
$8:359
Powe
3.283
2,134
22.809
14.502
3.546
1.129
23,899
11:289
2.573
1.189
21.401
10,056
3.455
1.129
22:035
10.566
Oer
2.00
1,133
20.319
9.366
3.551
11994
33
8,880
1,009
6.788
$7.884
4,887
3.112
1.36
19.867
9.887
2023; Hong et al., 2024), achieving a 4.0 dB improvement to CoPoNeRF in PSNR, demonstrating superior reconstruction quality and robustness. We also provide qualitative compar- ison in Fig. 3. Additionally, our approach also demonstrates superior pose estimation performance on both datasets, even surpassing (Hong et al., 2024) that trains its network with GT camera poses; however, we observe that compared to MASt3R, we slightly fall behind on ACID dataset. This dis- crepancy may be attributed to the larger scale of scenes, such as coastal landscapes and sky views, or dynamic scenes in ACID which complicates our refinement process and poses challenges for our depth network in estimating the metric depth of the scene.
dominantly includes indoor environments, whereas ACID
features numerous dynamic scenes. To comprehensively evaluate our method across a broader spectrum of real- world scenarios, we further assess it on the recently released DL3DV dataset (Ling et al., 2024). The results are summa- rized in Table 3. From these results, we observe that our method outperforms CoPoNeRF (Hong et al., 2024) by over dB in large-overlap scenarios and by dB in ovvrlao scenarios, highlighting the superior accuracy and robustness of our approach in handling diverse and complex environ- ments. This highlights the effectiveness of our method in managing varied scene and object types, reinforcing its ap- plicability for practical view synthesis tasks.
DL3DV. While RealEstate- 10K and ACID encompass a
variety of indoor and outdoor scenes, RealE 10K pre-
7

---

PF3plat: Pose-Free
3D Gaussian Splatting for Novel View Synthesis
Table Novel View Synthesis and Pose Estimation Performance on DL3DV. include PixelSplat and MVSplat for reference only.
DLAD
Smail
Large
Pose-Free
Method
PSNR
LPIPS
SSIM
Avg.
Rotation
Med.
Avg.
Translation
Med.
PSNR
LPIPS
SSIM
Avg.
Rocation
Med.
Avg.
Translation
Med.
x
PixelSplat
19.427
0.342
0.582
22. 22.889
0.193
0.734
x
MVSPlat
20.849
0.230
0.680
24.211
0.147
0.796
/
CaPoNoRI
15,509
0,563
0.396
13.121
6.721
44,645
30,269
17,586
0.467
0.469
5.609
2,905
17.974
12:445
v
Ours
19,822
0.248
0.651
9,998
22.668
0.198
0.723
3.448
1,598
9.338
6.177
4.5. Analysis and More Results
Table 4. Component ablations on RealEstate
Comparison to scene-specific optimization methods. In this study, we also compare our method with In- stantSplat (Fan et al., 2024) and CF-3DGS (Fu et al., 2023), as summarized in Tab. 5a. Our approach already surpasses InstantSplat, method that adopts similar 2-stage approach as ours, but instead of feed-forward inference, it iteratively optimizes the 3D Gaussian parameters. This results high- lights the effectiveness of our refinement modules and our design. The performance gap widens further when we adopt similar test-time optimization (TTO) strategy. By using our predictions as initialization, TTO takes significantly less time than InstantSplat, demonstrating high practicality. Fi- nally, CF-3DGS struggles to find accurate camera poses and suffers in rendering quality, likely because its design does not adequately handle wide-baseline images.
Componculs
PSNR
SSBM
LPPPS
Rotatins
Tranlation
Ang.
Mec.
Ang
Med
18)
Bucine
30.40
0.094
01.51
2.7.6
0.430
10.041
1,36
(1)
2M/00
0.782
0.181
1.756
0.897
V.474
4.004
(1)
Depi Rehneemen
23.012
0.755
0.311
2.543
1.132
V.811
4.93
(10
Pose Reiinement
21.623
0.741
0.219
2,310
1.233
11.599
6,544
(IV)
Ceometiy Confidence
21,443
0.741
0.223
2.228
1.001
11,322
5.90
(V)
Corres Network
NA
N5A
N/A
NAA
NOA
NA
(V))
Moto Depth Netwerk
16.132
0.511
0.403
6.990
5.329
21.325
14,32
(44)
Full FT Depth Network
A/A
N/A
N/A
NA
N/A
NA
N/A
(1:10
ScaleSSiit Tuning Depti Network
NA
N/A
NA
N0
N/A
NA
MA
(1-1
Ttl Consis Lois
19:001
0.644
0,402
5.66]
2059
18.322
10.331
(1-1))
Remiariioiii Lows
21.352
0.733
0.231
4.555
2012
12:388
9.998
(1-V)
(-IV) Tá. Cunes, Joss
NAA
N/A
NIA
NA
N/A
NA
NIA
4.4. Ablation Study
In this ablation study, we aim to investigate the effectiveness of each component of our method. We first define a baseline model, which combines our depth and pose estimates from coarse alignments with MVSplat (Chen et al., 2024) for 3D Gaussian parameter prediction. We also explore both full fine-tuning and partial fine-tuning strategies for the depth network. Additionally, we report the results of ablation studies on our loss functions. The results are summarized in Tab. 4.
Inference speed comparisons. We conduct a compre- hensive inference speed comparison between our method and competing approaches using varying numbers of input images, specifically N E 2, 6, 12. For each scenario, we evaluate the time required render 1, 3, and views. The results, summarized in Tab. 5b, show that our approach is generally faster than existing methods. However, for N = 12, our inference speed is slower than that of DBARF, as our method involves estimating camera poses via ro- bust solver for every pairwise combination. Despite this overhead, our approach gains a significant advantage as the number of rendered views increases, due the efficient ren- dering capabilities of 3DGS once the scene has been fully reconstructed. Finally, we provide the inference time of each of our components: overall inference time, UniDepth processing time, and decoder time. Given 2, 6, 12 views and to render a single target view, it takes 0.251, 0,832, 1.535 seconds for UniDepth inference, while rendering takes ap- proximately consistent 0.00247 seconds.
From the results, we find that significant improvement from (0). This improvement is further supported by the compar- isons from (I) to (1V) and from (I-III) to (I-IV), which show performance degradation as each component is re- moved. While we observe that (0) surpasses our final model in terms of the median rotation and translation angular differ- ence, this aligns with a commonly observed trend between learning-bacc based and robust solver-based approaches (Rock- well et al., 2024). Classical methods tend to achieve higher precision, whereas Ceaming-based approaches generally of- fer greater robustness.
We also demonstrate that without pre-trained weights for the depth and correspondence networks, the training either fails or achieves significantly lower performance. Similar observations are made in (I-I), (I-II), and (I-V), where we identify that directly tuning the depth network or training only with photometric losses leads to failure in the training process. The former issue may arise from overfitting, common problem when directly manipulating foundation models. With only the photometric loss, we observe that after certain iterations, as the baseline becomes wider, the training loss quickly diverges.
Extending to N-views. In practical scenarios, more than two views (N > 2) are commonly used. Moreover, un- like (Ye et al., 2024), our method naturally supports exten- sion to N- without needing to train network again. Here, we demonstrate that our method can process multiple views and render 4t. We input N views into our network to obtain P. D., and (Hi, o. E;,c;) Following a similar approach to (Chen & Lee, 2023), we select the top-k nearby views using P. and render f to compare with the ground
8

---

PF3plat: Pose-Free Feed-Forwate 3D Gaussian Splatting for Novel View Synthesis
Table 5. More Analysis and Results.
(a) Performance and speed comparisons on RealEstaat- 10K against per-seene optimization methods.
(b) Speed comparisons between pose-free generalizable
view synthesis models. Times are measured in seconds.
Meliod
PSNR
LPIPS
("])
Truss (1)
Time
Mahool
Views
Vlews
12
Aug.
Avgg
Med.
5
5lew
InstanSpport
23,079
0.777
0.182
2,693
0.852
11,365
3,094
DBARF
1.456
4.560
8.137
3,561
7.268
13,780
4,009
10.491
17.50
CF-3DGS
14,024
0.455
0.450
13.278
3.456
106 397
106.337
BOoAAM
4.030
10.13
23.715
34,000
23.44
48.55
Ous
TTO
24,689
23.589
0.782
0.798
0.167
0.181
1,622
1.756
0.871
0.897
8,998
9.474
4.311
4.623
0.390
CoPoNeRR
17.29
0.392
33.78
0.394
54.52
NA
NOA
NIA
57729
NOA
(c) RealEstateIOK 6, 12 input views.
(d) Cross-dataset Evaluation.
Mathod
6 views
viess
PSNR
SSIM
1.MP5
ATE
PSNR
SSIM
LPIS
ATE
Med
Mac
Mac.
DEARF
23,91662
0.7837
0.2226
00101166
24.1798
0.790
0,2186
0,0457777
FAO
Oses
246660
27.0384
0.825
8.5788
0.1158
02232
0,0022022
0,8010045
25.2200
28.1334
IL9934
0.8406
0,2165
0,8688
0.0022655
0,0084228
12a
328
130
6312
6000
0228
1211
12381
1280
truth target view image. For this evaluation, we compare our method with those of (Chen & Lee, 2023) and (Smith et al., 2023), since the method by (Hong et al., 2024) can only take two input views. We also report the Absolute Tra- jectory Error (ATE). The results are summarized Tab. 5c. From these results, we find that our method achieves signifi- cantly better performance than the others, highlighting our capability extend to multiple N views. We also provide qualitative results and videos for N-view experiments, in supplementary material.
(IITP) grant funded by the Korea government (MSIT) (RS- 2019-11190075, RS-2024-00509279, RS-2025-I12 12068, RS- RSS-022-00457888 and the Culture, Sports, and Tourism R&D Program through the Korea Cre- ative Content Agency grant funded by the Ministry of Cul- ture, Sports and Tourism 005-2024-0,345025,, RS-2024- 00333068), and National Research Foundation of Korea (RS-2024-00346557).
Impact Statement
Doss--Daasst Evaluation. To demonstrate the general- ization capability, conduct cross-dataset evaluation and compare against (Hong et al., 2024). Specifically, we evalu- ate performance on RealFittat 10K and DL3DD, using each dataset for training in cross-dataset setting. The results, summarized in Tab. 5d, show that our method achieves a PSNR of over 20 dB for both datasets, significantly out- performing (Hong et al., 2024). This indicates that, even under auloof-distribution conditions, our method produces high-quality renderings, highlighting its robustness and ef- fectiveness in zero-shot capability.
This paper presents work whose goal is to advance the field of Machine Learning, which focuses in practical novel view synthesis. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
5. Conclusion
In this paper, we have introduced learning- based framework that tackles pose-free novel view synthesis with 3DGS, en- abling efficient, fast and photorealistic view synthesis from unposed images. Our framework, PFSplat, is built on foun- dation models to overcome inherent limitations of 3DGS. We have demonstrated that PF3plat is capable of training and inference solely from unposed images, even in scenarios where only a handful of images with minimal overlaps are given. Furthermore, PF3plat surpasses all existing methods on real-world large-scale datasets, establishing new state-of- the-art performance.
Acknowledgement
This research was supported by Institute of Information & communications Technology Planning & Evaluation
9

---

PF3plat: Pose-Free Feed-Forwate 3D Gaussian Splatting for Novel View Synthesis
References
Chen, Y., Xu, H., Zheng, C., Zhuang, B., Pollefeys. M., Geiger, A., Cham, T.-J., and Cai, J. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. arXiv preprint arXiv:2403. 14627, 2024.
An. H.. Kim, J., Park. S., Jung, J., Han, J., Hong, S., and Kim, S. Cross-view completion models are zero-shot correspondence estimators. arXiv preprint arXiv:44/2.0007, 2024.
Cho, S., Hong, S., Jeon, S., Lee, Y., Sohn, K., and Kim, S. Cats: Cost aggregation transformers for visual corre- spondence. Advances in Neural Information Processing Systems, 34:9011-9024 2021.
Amold, E., Wynn, J., Vicente, S., Garcia-Hermande G., Monszpart, A., Prisacariu, V., Turmukhambetor, D., and Brachmann, E. Map-free visual relocalization: Metric pose relative to a single image. In European Conference on Computer Vision, pp. 690-708. Springer, 2022.
Cho, S., Hong. S., and Kim, S. Cats++: Boosting cost aggregation with convolutions and transformers. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 2022.
Barath, D., Matas, J., and Noskova, J. Magsac: marginaliz- ing sample consensus. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10197-10205, 2019.
Choy, C., Park, J., and Koltun, V. Fully convolutional geometric features. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 8958- 8966, 2019.
Barron, J. T. Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., and Srinivasan, P. P. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5855-5864, 2021.
Dao, T., Fu, D. Y., Ermon, S., Rudra, A., and Ré, C. FlashAt- tention: Fast and memory- eeffire-efficient exact attention with 10-awareness. In Advances Neural Information Pro- cessing Systems (NeurIPS), 2022.
Barron, J. T., Mildenhall, B.. Verbin, D.. Srinivasan, P. P., and Hedman, P. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. -470-5479, 2022.
DeTonee D., Malisiewicz, T., and Rabinovich, A. Super- point: Self-supervised interest point detection and de- scription. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 224-236, 2018.
Bian, W., Wang, Z., Li, K., Bian, J.-W., and Prisacariu, A. Nope-nerf: Optimising neural radiance field with no pose prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4160- 4169, 2023.
Du, Y., Smith, C., Tewari, A., and Sitzmann, V. Learning to render novel views from wide-baseline stereo pairs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4970-4980 2023.
Edstedt, J., Sun, Q., Bökman, G., Wadenbäck, M., and Felsberg, M. Roma: Robust dense feature matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19790-19800, 2024.
Bochkovskii, A., Delaunoy, A., Germain, H., Santos, M., Zhou, Y., Richter, S. R., and Koltun, Depth pro: Sharp monocular metric depth in less than second. arXiv 2024.
preprint ar arXiv:2410.0200
Banani, M., Rocco, I., Novotny, D., Vedaldi, A., Neverova, N., Johnson, J., and Graham, B. Self- supervised correspondence estimation via multiview reg- istration. In Proceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision (WACV), pp. 12 1216-1225, January 2023.
Charatan, D., Li, S., Tagliasacchi, A., and Sitzmann, V. pixelsplat: 3d gaussian splats from image pairs for seal- able generalizable 3d reconstruction. arXiv preprint arXiv:2312.12337, 2023.
Chen, A., Xu, Z., Zhao, F., Zhang, X., Xiang, F., Yu, J., and Su, H. Mysnerf: Fast generalizable radiance field reconstruction from multi-view stereo. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14133,14133, 2021.
Fan, Z., Pan, P., Wang, P., Jiang, Y., Jiang, H., Xu, D., Zhu, Z., Wang, D., and Wang, Z. Drag view: Generaliz- able novel view synthesis with unposed imagery. arXiv preprint, 2023.
Fan, Z., Cong, W., Wen, K., Wang, K., Zhang, J., Ding, X., Xu, D., Ivanovic, B., Pavone, M., Pavlakos, G., et al. Instantsplat: Unbounded sparse-view pose-free gaussian splatting in 40 seconds. arXiv preprint arXiv:2403.20309 2024.
Chen, Y. and Lee, G. H. Dbarf: Deep auddle-adjusting generalizable neural radiance fields. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24 34, 2023.
10

---

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
Fischler, M. A. and Bolles, R. C. Random sample consensus:
Keetha, N., Karhade, J., Jatavallabhula, K. M., Yang, G., Scherer, S., Ramanan, D., and Luiten, J. Splatam: Splat, track & map 3d gaussians for dense rgb-d slam. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Puttern Recognition, 2024.
paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-395, 1981.
Fu, Y., Liu, S., Kulkarni, A., Kautz, J., Efros, A. A., and Wang. X. Colmap-free 3d gaussian splatting. 2023.
Kerbl, B., Kopanas, G., Leimkühler, T., and Drettakis, G. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions Graphics, 42(4), July 2023. http:: htwp:/ppppoaaaiiiiaaaaaaaaaaaaaa fr/ fungraph/
Hartley, R. and Zisserman, A. Multiple view geometry in computer vision. Cambridge university press, 2003.
Hong, S. and Kim, S. Deep matching prior: Test-time optimization for dense correspondence. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 9907-9917, 2021.
Kingma, D. Adam: A method for stochastic optimization. arXiv preprint urXiv: 2014.
Leroy, V., Cabon, Y., and Revaud, J. Grounding im- age matching in 3d with mast3r. arXiv preprint WNiv:240060997566 2024.
Hong, S., Cho, S., Nam, J., Lin, S., and Kim, S. Cost aggregation with 4d convolutional swin transformer for few-shot segmentation. In European Conference Com- puter Vision, pp. 106-126. Springer, 2022a.
Li, H., Gao, Y., Wu, C., Zhang, D., Dai, Y., Zhao, C., Feng, H., Ding, E., Wang, J., and Han, Ggrt: Towards generalizable 3d gaussians without pose priors in real- time. 2024.
Hong, S., Nam, J., Cho, S., Hong, S., Jeon, S., Min, D., and Kim, S. Neural matching fields: Implicit representation of matching fields for visual correspondence. Advances in Neural Information Processing Systems, 35:13512- 13526, 2022b.
Li, R., Gong, D., Yin, W., Chen, H., Zhu, Y., Wang, K., Chen, X., Sun, J., and Zhang, Y. Learning to fuse monoc- ular and multi-vies cues for multi frame depth estimation in dynamic scenes. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 21539-21549 2023.
Hong, S., Jung, J., Shin, H., Yang. J., Kim, S., and Luo, C. Unifying correspondence, pose and nerf for pose-free novel view synthesis from stereo pairs. arXiv preprint arXiv:2312.07246. 2023.
Li, S., Xu, C., and Xie, M. A robust o (n) solution to the perspective-n-poigs problem. IEEE transactions on pattern analysis and machine intelligence, 34(7): 1450, 2012.
Hong, S., Cho, S., Kim, S., and Lin, S. Unifying feature and cost aggregation with transformers for semantic and visual correspondence. arXiv preprint 1rXiv:240.111120 2024.
Lin, C.-H., Ma, W.-C., Torralba, A., and Lucey, S. Barf: Bundle-adjusting neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5741-5751, 2021.
Jeong, Y., Ahn, S., Choy, C., Anandkumar, A., Cho, M., and Park, J. Self-calibrating neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 5846-5854, October 2021.
Lindenberger, P., Sarlin, P.-E., and Pollefeys, M. Lightglue: Local feature matching at light speed. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17627-17638, 2023.
Jiang H., Jiang, Z., Zhao, Y., and Huang, Q. Leap: Liber- ate sparse-view 3d modeling from camera poses. arXiv preprint arXiv:2310.014/1 2023.
Ling, L., Sheng, Y., Tu, Z., Zhao, W., Xin, C., Wan, K., Yu, I.., Guo, Q., Yu, 7., Lu, Y., et al. D13dv-10k: A large- scale scene dataset for deep learning-basee 3d vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22160-22169, 2024.
Johari, M. M., Lepoittevin, Y., and Fleuret, F. Geonerf: Generalizing nerf with geometry priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13375,1877,, 2022.
Ke, B., Obukhov, A., Huang, S., Metzger, N., Daudt, R. C.,, and Schindler, K. Repurposing diffusion-based image generators for monocular depth estimation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9492-9502, 2024.
Liu, A., Tucker, R., Jampani, V., Makadia, A., Snavely, N., and Kanazawa, A. Infinite nature: Perpetual view generation of natural scenes from a single image. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1
2021.

---

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
Melekhov, I., Ylioinas, J., Kannala, J., and Rahtu, E. Rela- tive camera pose estimation using convolutional neural networks. In Advanced Concepts for Intelligent Vision Systems: 18th International Conference, ACIVS 2017, Antwerp, Belgium, September 18-21, 2017, Proceedings 18, pp. 675-687. Springer, 2017.
Szymanowiez, S., Rupprecht, C., and Vedaldi, A. Splatter image: Ultra-fast single-view 3d reconstruction. Con- ference Computer Vision and Pattern Recognition (CVPR), 2024.
Truong, P., Danelljan, M., Timofte, R., and Van Gool, L. Pdc-net+: Enhanced probabílistic dense correspondence network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023a.
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R. Nerf: Representing scenes as neural radiance fields for view synthesis. Communica- tions of the ACM, 65(1):99~106, 2021.
Truong. P., Rakotosaona, M.-J., Manhardt, F., and Tombari, F. Sparf: Neural radiance fields from sparse and noisy poses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4190- 4200, 2023b.
Nistér, D. An efficient solution to the five-point relative pose problem. IEEE transactions on pattern analysis and machine intelligence, 26(6):756-770, 2004.
Piccinelli, L., Yang, Y.-H., Sakaridis, C., Segu, M., Li, S., Van Gool, L., and Yu, F. Unidepth: Universal monocular metric depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 10106-10116, 2024.
Wang. J., Karaev, N., Rupprecht, C., and Novotny, D. Vg- gsfm: Visual geometry grounded deep structure from motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21686- 21697, 2024.
Rockwell, C., Johnson, J., and Fouhey, D. F. The 8-point algorithm as an inductive bias for relative pose prediction by vits. In 2022 International Conference on 3D Vision (3DV), pp. 1-11. IEEE, 2022.
Wang. S., Leroy, V., Cabon, Y., Chidlovskii, B., and Revaud, J. Dust3r: Geometric 3d vision made easy. arXiv preprint 2rXiv:2312.14133 2023.
Wang. Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, P. Image quality assessment: from error visibility struc- tural similarity. IEEE transactions on image processing, (3(4):600-612 2004.
Rockwell, C., Kulkarni, N., Jin, L., Park, J. J., Johnson, J., and Fouhey, D. F. Far: Flexible accurate and robust 6dof relative camera pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19854-19864, 2024.
Wang. Z., Wu, S., Xie, W., Chen, M., and Prisacariu, V. A. Nerf-: Neural radiance fields without known camera parameters. arXiv preprint arXiv:2102.07064, 2021.
Sarlin, P.-E., DeTone, D., Malisiewiez, T., and Rabinovich, A. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 4938-4947, 2020.
Wei, T., Patel, Y., Shekhovtsov, A., Matas, J., and Barath, D. Generalized differentiable ransac. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 17649-17660, 2023.
Sitzmann, V., Rezchikov, S., Freeman, B., Tenenbaum, J., and Durand, F. Light field networks: Neural scene rep- resentations with single-evaluation rendering. Advances in Neural Information Processing Systems, 34:19313- 19325, 2021.
Xu, H., Zhang, J., Cai, J., Rezatofighi, H., and Tao, D. Gmflow: Learning optical flow via global matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8121-8130, 2022.
Smart, B., Zheng, C., Laina, I., and Prisacariu, V. A. Splatt3r: Zero-shot gaussian splatting from uncalibarated image pairs. arXiv preprint arXiv:2408. 13912, 2024.
Xu, H., Chen, A., Chen, Y., Sakaridis, C., Zhang, Y., Polle- feys, M., Geiger, A., and Yu, F. Murf: Multi-baseline radiance fields. arXiv preprint 3rXiv:2312.04566 2023a.
Smith, C., Du, Y., Tewari, A., and Sitzmann, V. Flowcam: Training generalizable 3d radiance fields without cam- era poses via pixel-aligned scene flow. arXiv preprint arXiv:2306.00110, 2023.
Xu, M., Zhang, Z., Wei, F., Hu, H., and Bai, X. Side adapter network for voccbbocyy semantic segmentation. In Proceedings of the IEEF/CVV Conference on Computer Vision and Pattern Recognition, pp.
2023b.
Sun, J., Shen, Z., Wang, Y., Bao, H., and Zhou, X. Loftr: Detector-free free local feature matching with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8922-8931, 2021.
Yang, H., Hong, L., Li, A., Hu, T., Li, Z., Lee, G. H., and Wang, L. Contranerf: Generalizable neural radiance fields for synthetic-to-ree novel view synthesis via contrastive learning. In Proceedings of the IEEE/CVF Conference
12

---

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
Computer Vision and Pattern Recognition, pp. 16508- 16517, 2023.
Zhou, T., Tucker, R., Flyan, J., Fyffe, G., and Snavely, N. Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint 2018.
Yang, L., Kang, B., Huang, Z., Xu, X., Feng, J., and Zhao, H. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 10371-10381, 2024.
Zhou, Y., Barnes, C., Jingwan, L., Jimei, Y., and Hao, L. On the continuity of rotation representations in neural networks. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
Yao, Y. Luo, Z., Li, S., Fang, T., and Quan, L. Mvsnet: Depth inference for unstructured multi-view stereo. In Proceedings of the European conference on computer vision (ECCV), pp. 767-783, 2018.
Ye, B., Liu, S., Xu, H., Li, X., Pollefeys, M., Yang, M.-H., and Peng, No pose, no problem: Surprisingly simple 3d gaussian splats from sparse unposed images. arXiv preprint 410. 24207, 2024.
Yew, Z. and Lee, G. H. 3dfeat-net: Weakly supervised local 3d features for point cloud registration. In Pro- ceedings of the European conference on computer vision (ECCV), pp. 607-623, 2018.
Yi, K. M., Trulls, E., Lepetit, V. and Fua, P. Lift: Learned invariant feature transform. In Computer Vision- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part 14, pp. 467-483. Springer, 2016.
Yin, W., Zhang, C., Chen, H., Cai, Z., Yu, G., Wang, K., Chen, X., and Shen, C. Metric3d: Towards zero-shot metric 3d prediction from single image. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 9043-9053, 2023.
Yu, A., Ye, V., Tancik, M., and Kanazawa, A. pixelnerf: Neural radiance fields from one or few images. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pp. 4578,4577, 2021.
Yu. Z., Chen, A., Huang, B., Sattler, T., and Geiger, A. Mip- splatting: Alias-free 3d gaussian splatting. Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 19447-19456, 2024.
Zhang, Y., Ramanan, D., and Tulsiani, S. Relpose: Pre- dicting probabilistic relative rotation for single objects in the wild. In European Conference on Computer Vision, pp. 992-611. Springer, 2022.
Zhang, R., Isola, P., Efros, A. A., Shechtman, F.,, and Wang, O. The unreasonable effectiveness of deep features as perceptual metric. In CVPR, 2018.
Zhou, T., Brown, M., Snavely, N., and Lowe, D. G. Unsu- pervised learning depth and ego-motion from video. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1851-1858 2017.
13

---

PF3plat: Pose-Free feed-Forward... 3D Gaussian Splatting for Novel View Synthesis
A. Appendix
Method
PSNR
SSIM
LPIPS
Rot.
(*.)
Trass ("4)
Arg.
Mod.
Avg.
Medl
A.1. Training Details
MASI3R
2.555
0.751
9,775
2,830
RoMa
2,470
0.391
8,047
1,601
We train MVSplat (Chen et al., 2024) and CoPoNeRF (Hong et al., 2024) using our data loaders, similarly increasing the distance between context views during training. as explained in Sec 4.1. Specifically; we train MVSplat for 200,000 iterations using a batch size of 8 on single A6000 GPU. All the hyperparameters are set to the authors' default setting. For CoPoNeRF, we train it for 50,000 iterations using 8 A6000 GPUs with effective batch size of 64, following the authors' original implementations and hyperparameters. Finally, for InstantSplat (Fan et al., 2024), we train and evaluate single A6000 GPU with batch size of 1 by following the official code", and the hyperparameters were set according to the default settings provided by the authors.
Ours
23.389
0.782
0.181
1.756
0.897
9,474
4.628
Deptharro Fine Align. (Ours)
21.882
0.201
0.289
2.778
11,322
5.44
RoMa Fine Aligu. (Ours)
24.412
0.799
0.167
2.152
0.501
7,544
3.233
Table 6. Different Strategies for Coarse Alignment.
Pro (Bochkovskii et al., 2024) for coarse alignment. The results, presented in Tab. 6, reveal that using more robust and accurate correspondences leads to noticeable improvements, as these correspondences enhance our loss computation. However, we observed that integrating DepthPro results in significant performance degradation. This issue may stem from the feature maps used by our fine alignment modules, which could potentially be mitigated through further engi- neering refinements.
A.2. Implementation Details
We provide a detailed illustration in Fig. 2 for our fine alignment and confidence estimation modules.
A.5. More Qualitative Results
Fig. 4, Fig. 5 and Fig. 6 present more novel view rendering results of different methods. Fig. 7 shows qualitative re- sults on pose estimation in N-view experiments, compared against FlowCAM (Smith et al., 2023). Finally, we provide 6-view and 12-view qualitative results in Fig. and Fig. 9, while videos can be found in the attached file.
A.3. Evaluation Details
For the evaluation on RealEstate- 10K and ACID, we follow the protocol outlined by (Hong et al., 2024), where evalua- tion is conducted using unposed triplet images (/1, I2, It). The test set is divided into three groups, small, middle, and large, based on the extent of overlap between 11 and Ig. This allows the model's performance to be assessed under varying levels of difficulty, reflecting different real-world scenarios. For the relatively new DL3DV dataset, we intro- duce new evaluation protocol. For each scene, we select two context images, 11 and 12, by skipping frames at in- tervals of 5 and 10, creating two groups per scene, each representing small and large overlap cases. We then ran- domly select three target images from the sequence between the context images.
A.6. Limitations and Future Work
As our model currently lacks a mechanism to handle dy- namic scenes, it is unable to accurately capture scene dynam- ies or perform view extrapolation. Additionally, our model's s performance is contingent on the quality of the coarse align- ments, which rely on the accuracy of the depth and corre- spondence models. In cases where either of these models fails, our approach may not function optimally. However, because our refinement modules are lightweight, simple, and model-agnostic, incorporating more advanced methods for coarse alignment could further enhance performance. Moreover, while existing pose-free view synthesis meth- ods (Chen & Lee, 2023; Smith et al., 2023; Hong al., 2024; Ye et al., 2024) share limitation that requires camera intrinsic parameters, exploring joint optimization of both camera extrinsic and intrinsic is a promising direction. Fi- nally, our method is not designed for dynamic scenes, but we believe exploring -erward dynamic scene recon- struction is promising direction.
For evaluation metrics, sures, PSNR, SSIM, LPIPS, and MSE, for novel view synthesis. For camera pose estimation, we compute the geodesic rotation error and angular difference in transla- tion, as commonly done in classical methods (Nistér, 2004; Melekhov et al., 2017). Our statistical analysis reports both the average and median errors, with the median providing robustness against outliers.
standard image quality mea-
A.4. Additional Experiment
For future work, we plan to train our model on diverse large- scale datasets. Since our approach relies exclusively on supervision signals from RGB images, it is straightforward to scale up the training data. We also aim to extend our method to handle 4D objects, ultimately enabling the model- ing of 4D scenes, which would be beneficial for applications such as egocentric vision and roboties.
Different Strategy for Coarse Alignment. In this addi- tional experiment, we analyze the impact of different coarse alignment strategies. Specifically, we replace LightGlue with RoMa (Edstedt et al., 2024) and UniDepth with Depth-
14

---

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
(a) 1
(b) 12
(c) DBARF
(d) FlowCAM
(c) CoPoNeRF
(f) Ours
(g) GT
Figure Qualitative results on RealEstate-100 dataset. Given two context views (a) and (b), we compare novel view rendering results.
15

---

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
(a) 4
(b) 12
(c) DBARF
(d) FlowCAM
(c) CoPoNeRF
(f) Ours
(g) GT
Figure Qualitative results on ACID dataset. Given two context views (a) (b), we compare novel view rendering results.
16

---

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
(a) 4
(b) lz
(e) CoPoNeRF
(f) Ours
(g) GT
Figure Qualitative results on DL3DV dataset. Given two context views (a) and (b), we compare novel view rendering results.
17

---

PF3plat: Pose-Free Feed-Forwait 3D Gaussian Splatting for Novel View Synthesis
Catmmiion
Feee Extematioo
Tooe Extimatoon
Foom
Cas
Ediiwiion
Extirvation
Faoe Estination
CowCam
Coccam
Mlse Eodi
Puse Graaiin
Yise Ealination
Ca
on
Figure 7. Pose estimation qualitative results.
18

---

PF3plat: Pose-Free 3D Gaussian Splatting for Novel View Synthesis
BEGC
GUSE
Mele
Mll
Mile
Context Views
Ground Truth
Context Views
Ground Truth
Ours
Ours
Figure 8, Qualitative results on RealEstate-10K with 6 input views.
19

---

PF3plat: Pose-Free Feed-Forwaie 3D Gaussian Splatting for Novel View Synthesis
H
Context
Ground Truth
Context Views
Ground
Ours
Views
Truth
Figure 9, Qualitative results on RealEstate-10K with 12 input views.
20


Reviews:
[{"review_id": "OadmDrfmE6", "reviewer": "Reviewer_TAPU", "summary": "This work presents a framework for novel view synthesis (NVS) from unposed images in a single feed-forward pass. It estimates depth and pose from unposed images using a combination of pre-trained monocular depth estimation and visual correspondence models. It outperforms prior pose-free methods like DBARF and CoPoNeRF in NVS quality and pose estimation.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The paper states that correspondence models like LightGlue struggle in low-texture regions. It is unclear how PF3plat can improve in these areas.\n\nWe enhance performance in low-texture regions through two complementary approaches:\nAs explained in line 233, our proposed 2D-3D consistency loss encourages corresponding points to lie on the same surface, drawing from principles of multi-view geometry. The effectiveness is shown in ***Table 4 (I-III)***. \n\nSecond, the flexibility of the design choice at coarse alignment stage allows us to use other methods beyond LightGlue. For example, replacing LightGlue with networks like RoMA has led to noticeable improvements, as demonstrated in ***Table 6***.\n\n> Lack of qualitative results for pose estimation.\n\nPlease refer to ***Figure 7***, which provides a visual comparison for pose estimation. Additionally, we have included the baseline pose estimates in this figure to clearly demonstrate the effectiveness of our fine alignment module. Please refer to the following anonymous link: https://anonymous.4open.science/r/7048/7048_camvis.pdf \n\n> However, the method still requires GT camera intrinsics, which may not always be available in real-world scenarios. \n\nCamera intrinsics are commonly required in previous pose-free methods, such as DBARF, CoPoNeRF and NoPoSplat, as  discussed in Sec. A.6. While this is a common limitation, for this rebuttal, we have conducted an additional experiment using camera intrinsics from UniDepth prediction:\n\n| Method                 | PSNR    | SSIM   | LPIPS |\n|------------------------|---------|--------|-------|\n| CoPoNeRF               | 19.536  |  0.638 | 0.398 |\n| Ours w/o GT intrinsics | 22.688  | 0.733  | 0.201 |\n| Ours                   | 23.589  | 0.782  | 0.181 |\n\nFrom the results, we highlight that even without GT intrinsics, ours outperforms CoPoNeRF and performs on par with the model with intrinsics. This suggests a potentially feasible approach to alleviate the limitation.\n\n> Mast3R for pose estimation + MVSplat for rendering.\n\nWe opted not to use MVSplat with MASt3R because ***MASt3R's runtime of around 10 seconds conflicts with our goal of achieving a fast, feed-forward process***. Despite our superior performance in in RealEstate10K, thanks to the relatively larger-scale outdoor training of MASt3R, it is true that MASt3R performs slightly better than ours in pose estimation in ACID. Nevertheless, this is compensated when compared to MAST3R* in Tab. 2. We also highlight that with more advanced correspondence network, RoMa, ours further broadens the gap, as shown in Tab. 6. \n\nNonetheless, for this rebuttal, we provide an additional comparison below: \n|                           |         |       |       |       |\n|---------------------------|---------|-------|-------|-------|\n| Method                    | PSNR    | SSIM  | LPIPS | Time  |\n| (0) Coarse Pose + MVSplat | 20.140  | 0.694 | 0.281 | 0.264 |\n| Mast3R + MVSplat          | 21.712  | 0.721 | 0.254 | 11    |\n| Mast3R* + MVSplat         | 21.167  | 0.702 | 0.268 | 0.642 |\n| Ours                      | 23.589  | 0.782 | 0.181 | 0.390 |\n\nFrom the results, we find that ours outperforms the baselines, thanks to our fine alignment modules and confidence estimation that contribute to further improvement. \n\n> ACID dataset results indicate weaknesses in large-scale outdoor scenes\n\nWe acknowledge that our performance in large-scale outdoor scenes (dynamic coastline environments), is not optimal, ***as discussed in Sec. A.6***. We attribute these challenges primarily to UniDepth's training dataset, which, unlike DUSt3R or MASt3R, was built on a smaller and less diverse collection of outdoor scenes. A straightforward approach would be to leverage depth models that are trained on large-scale outdoor scenes or advanced correspondence models, to yield more accurate initial 3D Gaussian locations.\n\n> The explanation of how the refined poses are computed is unclear \n\nIn Section 3.2.3, the refined pose is computed using three distinct inputs: Plücker Coordinates,  Feature Maps and Pose Token. Each of these inputs is processed through a series of attention layers, which help propagate information about previous camera estimates, multi-view geometry, and the current camera state. After fusing this information, the pose token is passed through a simple MLP that predicts residual rotation and translation parameters. These residuals are then added to the coarse camera parameters to yield the final refined pose.\n\n> Cost Volume Construction and Aggregation, how does this differ from prior multi-view stereo cost volumes?\n\nOur approach differs from traditional MVS cost volumes by introducing a guidance cost volume. Since the cost volume built from our estimated camera poses is inherently noisy, we supplement it with a guidance cost volume derived from a monocular depth estimate. By aggregating these two volumes, we construct a fina cost volume better tailored for a pose-free setting.", "claims_and_evidence": "The claims in the paper are generally supported by experimental results, ablation studies, and comparisons with prior work. \n1) The method shows clear improvements over prior pose-free methods like DBARF and CoPoNeRF in PSNR, SSIM, and LPIPS metrics on RealEstate-10K and ACID.\n2) The ablation studies in Table 4 demonstrate a clear drop in performance when removing depth refinement, pose refinement, or geometry-aware confidence scores. They effectively validate the effect of these components.\n3) The inference time of 0.39s per view (Table 5a) is significantly faster than InstantSplat (53s), making PF3plat more practical.\n\nHowever, some claims could be better substantiated with additional experiments.\na) \"Our method improves robustness in regions with low texture or significant viewpoint changes.\" The paper states that correspondence models like LightGlue struggle in low-texture regions. It is unclear how PF3plat can improve in these areas.\nb) Lack of qualitative results for pose estimation. While the paper presents quantitative results (e.g., rotation and translation errors), it does not provide visual examples of estimated vs. ground truth poses, which would offer clearer insight into the actual performance of the pose refinement module.", "methods_and_evaluation_criteria": "The benchmarks make sense for the problem. The paper tackles a well-motivated problem: most existing 3DGS methods rely on accurate camera poses, which are difficult to obtain in casual image capture scenarios. PF3plat removes this dependency, making it more practical for real-world applications. \n\nHowever, the method still requires ground-truth (GT) camera intrinsics, which may not always be available in real-world scenarios. This contradicts the claim of being fully pose-free since intrinsic parameters are a part of the camera model.", "theoretical_claims": "The paper does not appear to contain formal theoretical proofs—it is primarily an experimental and algorithmic contribution focused on pose-free novel view synthesis using 3DGS. Since there are no complex theoretical derivations or proofs, there are no major mathematical errors to verify.", "experimental_designs_or_analyses": "Yes, I reviewed the soundness and validity of the experimental design and analysis. Overall the experimental setup is well-structured. \nHowever, the paper does not compare PF3plat to a simple pipeline using Mast3R for pose estimation + MVSplat for rendering. Since Mast3R outperforms PF3plat in pose estimation, it is unclear whether a Mast3R + MVSplat baseline would yield better results than PF3plat.\nWithout this comparison, it’s hard to judge whether PF3plat’s pose-free approach is necessary or if it just introduces more error.", "relation_to_broader_scientific_literature": "PF3plat extends prior works (e.g. PixelSplat, MVSplat) by:\nRemoving the need for ground-truth camera poses, which prior 3DGS methods required.\nIntroducing coarse-to-fine pose estimation using monocular depth and correspondence networks, making pose-free 3DGS feasible.\nUsing confidence-aware refinement to stabilize Gaussian placement, which was a known issue in pixel-aligned 3DGS.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n1) The coarse alignment module creatively uses monocular depth estimation + feature matching to initialize 3D Gaussian positions. The method is fully feed-forward, making it more scalable and applicable to real-world settings.\n2) The paper introduces confidence-aware refinement, ensuring that unreliable depth/pose estimates are given lower weight when estimating 3D Gaussians. This helps stabilize training and improves robustness.\n\nWeakness:\na) ACID dataset results indicate weaknesses in large-scale outdoor scenes.\nb) PF3plat claims robustness in low-texture regions, but this is unclear.", "other_comments_or_suggestions": "1. Overall the paper is well-written and easy-to-follow. There are some typos, e.g. \"fast speed\".\n2. Equation (2) does not clearly define E_pos. Equation (3) should use consistent notation for confidence scores.\n3. The caption in Fig.1 should briefly explain the input/output of each module.", "questions_for_authors": "1. The explanation of how the refined poses are computed is unclear (Sec. 3.2.3).\n2. In Sec 3.2.4, about Cost Volume Construction and Aggregation, how does this differ from prior multi-view stereo (MVS) cost volumes?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UsSTFf8mp6", "reviewer": "Reviewer_EyLp", "summary": "The paper presents a novel feed-forward method for 3D reconstruction and view synthesis from sparse, unposed images, eliminating the need for ground-truth depth or pose at both training and inference. The method builds on pixel-aligned 3D Gaussian Splatting (3DGS) but addresses the challenge of Gaussian center misalignment, which can destabilize training.\nTo tackle this, the method first leverages off-the-shelf monocular depth estimation and image correspondence models to infer coarse depth and camera pose estimates. It then refines these estimates through a multi-view refinement process using learned modules, improving reconstruction quality and stability. Finally, the method computes geometry-aware confidence scores to assess the reliability of Gaussian centers, which condition the prediction of opacity, covariance, and color.\nUsing large-scale real-world indoor and outdoor datasets, the paper demonstrates that this method outperforms existing approaches in both rendered image quality and inferred camera pose accuracy, setting a new state-of-the-art in pose-free generalizable novel view synthesis.\n\n## update after rebuttal\nAfter reading the rebuttal, I think the paper makes a valuable contribution to pose-free novel view synthesis. The comparison to NoPoSplat is reasonable—it’s a concurrent paper, and the authors added results in the rebuttal showing that their method is competitive, faster, and works in more general settings. They also compared to MASt3R + MVSplat and showed better results and faster runtime.\nI still recommend weak accept, and I believe the paper is above the bar.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Given that NoPoSplat was accepted to ICLR 2025 only few days before the ICML submission deadline, its omission is understandable. However, incorporating it as a baseline in future revisions would further strengthen the claims of state-of-the-art performance.\n\nWe appreciate the reviewer’s valuable suggestion. As noted, NoPoSplat is a concurrent work released on arXiv less than three months ago and accepted only three days before our submission. Moreover, as indicated in Table 2, fair comparison is challenging since other works (CoPoNeRF and NoPoSplat) use additional supervisory data (GT Camera pose) during training. In contrast, our approach can be learned without such data, clearly differentiating our approach to them. Nonetheless, we agree that including NoPoSplat as a baseline along with detailed comparison would further enhance our work.  Below includes the comparison:\n\n| Methods | Pose Supervision | PSNR $\\uparrow$ | LPIPS $\\downarrow$ | SSIM $\\uparrow$  |\n| --- | :---: | --- | --- | --- |\n| CoPoNeRF | $\\checkmark$ | 19.536 | 0.398 | 0.638 |\n| NoPoSplat | $\\checkmark$ | 26.820 | 0.126 | 0.879 |\n| DBARF | $\\times$ | 14.789 | 0.490 | 0.570 |\n| FlowCam | $\\times$ | 18.242 | 0.597 | 0.455 |\n| Ours | $\\times$ | 23.589 | 0.181 | 0.782 |\n\n\nWhile NoPoSplat performs well, as indicated in the table, we stress that our method clearly differentiates in the setting that the 3D geometry data, such as GT camera pose, is not utilized during training. Moreover, we show in the following that our method can be naturally extended to N-view setting, where NoPoSplat struggles.\n\n| Methods (6 views) | Pose Supervision | PSNR $\\uparrow$ | LPIPS $\\downarrow$ | SSIM $\\uparrow$  |\n| --- | :---: | --- | --- | --- |\n| NoPoSplat | $\\checkmark$ | 18.007 | 0.384 | 0.584 |\n| Ours | $\\times$ | 27.028 | 0.116 | 0.879 |\n\n| Methods (12 views) | Pose Supervision | PSNR $\\uparrow$ | LPIPS $\\downarrow$ | SSIM $\\uparrow$  |\n| --- | :---: | --- | --- | --- |\n| NoPoSplat | $\\checkmark$ | 17.625 | 0.399 | 0.583 |\n| Ours | $\\times$ | 28.133 | 0.099 | 0.993 |\n\nTo summarize, compared to NoPoSplat, our approach offers key advantages compared to NoPoSplat: it is a conceptual contribution proposing a general pipeline for pose-free view synthesis that addresses 3D Gaussian misalignment using off-the-shelf depth and correspondence networks and fine alignment modules; it naturally extends to N-view inference without retraining; and it is designed for scenarios without 3D geometry data (GT camera pose) during training. We will additionally include this discussion in related works.\n\n> Did you consider or experiment with other parameterization strategies or approaches to mitigate the impact of inaccuracies in 3D Gaussian center localization on noisy and sparse gradients? \n\nWe appreciate the reviewer’s suggestion. While we have not yet experimented with alternative strategies like PixelSplat, we agree that mitigating inaccuracies in 3D Gaussian center localization is promising. Future work could explore leveraging NeRF representations, as in RADSplat, or formulating a probabilistic approach similar to PixelSplat. We will include a discussion of these alternatives in the supplementary material.\n\n> Could you provide further analysis on why inaccuracies in 3D Gaussian center localization result in noisy and sparse gradients? Specifically, what factors contribute to this phenomenon? Including citations that discuss this phenomenon would also improve credibility.\n\nWe thank the reviewer for the valuable suggestion. In our framework, we claim that inaccuracies in 3D Gaussian center localization lead to noisy and sparse gradients because only the Gaussians within the rasterization window receive gradients. When centers are mislocalized, few Gaussians get effective updates, resulting in weak guidance, especially when initialized far from the optimal positions [1]. For this, we stressed the importance of coarse alignment. Moreover, providing photometric only showed limited effectiveness, in which we proposed 2D-3D consistency and regularization losses for addressing this.  Additionally, [2] and NoPoSplat highlight that when the network is initialized without 3D priors and reliable 3D geometry data during early training (e.g., GT pointmaps) to compensate for such priors is absent, convergence is difficult (Early training stage for FreeSplatter and CroCo initialization for NoPoSplat). We will include these discussions and citations in the final version of the paper.\n\n\n[1] Jung, J., Han, J., An, H., Kang, J., Park, S. and Kim, S., 2024. Relaxing accurate initialization constraint for 3d gaussian splatting. arXiv preprint arXiv:2403.09413.\n\n[2] Xu, Jiale, Shenghua Gao, and Ying Shan. \"FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstruction.\" arXiv preprint arXiv:2412.09573 (2024).", "claims_and_evidence": "The authors’ claim of state-of-the-art pose-free generalizable novel view synthesis is supported by extensive quantitative and qualitative analysis. See more in “Evaluation Criteria.”\nTheir claim that improved depth and camera pose estimates enhance pixel-aligned 3D Gaussian Splatting is backed by their ablation analysis, which demonstrates that refining scene estimates contributes to better view synthesis quality.\nAdditionally, their experimental results suggest that proper initialization is critical for stable training, as training without it leads to convergence issues. However, a deeper theoretical discussion on why inaccuracies in 3D Gaussian center localization cause noisy and sparse gradients—and whether alternative initialization strategies could mitigate this—would further strengthen their argument (see \"Theoretical Claims\").", "methods_and_evaluation_criteria": "The authors evaluate their method on two tasks: novel-view synthesis and camera pose estimation. Their model is trained and tested on three large-scale datasets: RealEstate10K (Zhou et al., 2018), ACID (Liu et al., 2021), and DL3DV (Lang et al., 2024).\n\nFor novel-view synthesis, they use standard image quality metrics, including PSNR, SSIM, LPIPS, and MSE. Camera pose estimation is assessed using the geodesic rotation error and angular difference in translation. This evaluation protocol follows a pose-free sparse view reconstruction method (Hong et al.) and has been adopted by other works in the field.\n\nWhile the authors acknowledge NoPoSplat (Ye et al., 2024), a concurrent work achieving state-of-the-art results in pose-free generalizable novel-view synthesis from two views, they do not include it as a baseline for comparison. Given that NoPoSplat was accepted to ICLR 2025 only 50 days before the submission deadline, its omission is understandable. However, incorporating it as a baseline in future revisions would further strengthen the claims of state-of-the-art performance.", "theoretical_claims": "The authors do not provide any formal proofs. However, their hypothesis that inaccuracies in the localization of 3D Gaussian centers lead to noisy and sparse gradients, which cannot be easily compensated for, is supported by empirical evidence. Specifically, their results show that training without proper initialization leads to almost intractable convergence, reinforcing this claim.\nWhile their findings provide some support, a more thorough analysis of the underlying causes could further strengthen their argument. Including citations that discuss this phenomenon would also improve credibility. Additionally, if training without initialization leads to intractable results, it would be valuable to explore whether alternative initialization strategies—beyond depth-based alignment—could be viable.\nPrior work, such as PixelSplat, demonstrated that parameterizing Gaussian positions implicitly via dense probability distributions can mitigate local minima issues when optimizing primitive parameters through gradient descent. This suggests a potential complementary approach that the authors could explore. Integrating implicit parameterization into their method might further improve robustness.\nIn addition, the loss functions are mathematically well-defined.", "experimental_designs_or_analyses": "The experiments are well-structured and include strong ablations on different model components.", "relation_to_broader_scientific_literature": "The paper presents a novel feed-forward method for 3D reconstruction and view synthesis from sparse, unposed images, situating itself within the existing research. These methods open the door to applicability in real-world settings, where casually captured images contain sparse and distant viewpoints, and lack precise camera poses.", "essential_references_not_discussed": "I did not notice any missing citations. However, I recommend including works that further analyze the sensitivity of 3D Gaussian Splatting (3DGS) synthesis quality to the initial locations of the 3D Gaussian centers, which would provide valuable context and support on the matter.", "other_strengths_and_weaknesses": "The claim and demonstration that increasing metric depth and camera pose inference to a sufficiently high level is enough to infer the locations of Gaussian centers for high-quality view synthesis is original and presents an intriguing direction for future work.\n\nIn addition, multiview refinement of monocular predictions is an interesting direction on its own, with potential of surpassing sparse view depth estimators. \n\nMost sections of the paper are written clearly. The authors include and describe all relevant literature and related works effectively. The evaluation protocol follows established standards in the field and is well-described.\n\n\nHowever, the description of the method could be clearer. It takes some time to understand that the Gaussian centers are derived directly from the depth of the pixels, rather than from a learned module. A more refined overview of the method, along with a revision of Figure 1 to better explain how and when each Gaussian parameter is estimated, could improve clarity.", "other_comments_or_suggestions": "In Introduction (line 27, right column) unnecessary repetition of citation.\n\nPF3plat is sometimes written as “PFsplat”.", "questions_for_authors": "What is the performance of NoPoSplat in comparison to your approach?\n\nDid you consider or experiment with other parameterization strategies or approaches to mitigate the impact of inaccuracies in 3D Gaussian center localization on noisy and sparse gradients?\n\nCould you provide further analysis on why inaccuracies in 3D Gaussian center localization result in noisy and sparse gradients? Specifically, what factors contribute to this phenomenon?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AOGT1J3mG9", "reviewer": "Reviewer_K81V", "summary": "The paper introduces PF3plat to address the problem pose-free feed-forward view synthesis using 3D Gaussians parametrization. First, the method leverages pretrained monocular depth estimator and visual correspondence models to get coarse depth and pose. Then learnable refinement modules are proposed to refine the depth and pose, which are conditioned on estimated confidence using cost volumes. The method further utilizes the 2D-3D and 3D-3D consistent loss to regularize the geometry. It achieves sota performance in pose-free synthesis methods.\n\n## update after rebuttal\nThe contribution of work mainly relies on the refinement module and new losses, which I argree is incremental rather than substantial. I also encourage the authors to include more analysis/qualitative experiments of the ablation studies to demonstrate their proposed method better. Given the good empirical performance, I decided to maintain my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> FreeSplatter not discussed in the paper.\n\nWe thank the reviewer for highlighting FreeSplatter[1]. FreeSplatter addresses the same task using an LRM-based architecture that directly maps images to 3D Gaussians. To tackle coarse alignment, it employs staged training with early supervision from 3D geometry data (ground-truth pointmaps constructed using pose and depth), which is essential for convergence. This supports our claim that misaligned 3D Gaussians hinder learning, and contrasts with our approach, which incorporates a dedicated coarse alignment module. We will include this discussion in the related works section.\n\n> The contribution mainly relies on the refinement modules. While it is effective, much of the work builds on existing works or ideas.\n\nWhile our approach leverages existing methods for coarse alignment (while recognized as creative by reviewer TAPU), our main technical contributions lie in the novel fine-alignment modules. In addition to our findings to identify a unique challenge in pose-free feed-forward 3DGS, we address the challenge by proposing new losses, fine alignment modules, and a confidence estimation module. ***As demonstrated in Table 4, each component significantly contributes to performance improvements***, underscoring the technical innovation of our work.\n\n> NoPoSplat [4] is acknowledged but not evaluated and compared.\n\nAs mentioned by the reviewer EyLp, ***NoPoSplat is a concurrent work released on arXiv less than three months ago and accepted only three days before our submission.*** Moreover, as shown in Table 2, fair comparison is challenging since other works (CoPoNeRF and NoPoSplat) use additional supervisory data (GT camera pose) during training. Nonetheless, we agree that including NoPoSplat as a baseline would further enhance our work.  Below includes the comparison:\n\n| Methods | Pose Supervision | PSNR $\\uparrow$ | LPIPS $\\downarrow$ | SSIM $\\uparrow$  |\n| --- | :---: | --- | --- | --- |\n| CoPoNeRF | $\\checkmark$ | 19.536 | 0.398 | 0.638 |\n| NoPoSplat | $\\checkmark$ | 26.820 | 0.126 | 0.879 |\n| DBARF | $\\times$ | 14.789 | 0.490 | 0.570 |\n| FlowCam | $\\times$ | 18.242 | 0.597 | 0.455 |\n| Ours | $\\times$ | 23.589 | 0.181 | 0.782 |\n\nWhile NoPoSplat performs well, as indicated in the table, we stress that our method clearly differentiates as they use GT camera pose during training. Moreover, below shows our method can be naturally extended to n-view setting, where NoPoSplat struggles.\n\n| Methods (6 views) | Pose Supervision | PSNR $\\uparrow$ | LPIPS $\\downarrow$ | SSIM $\\uparrow$  |\n| --- | :---: | --- | --- | --- |\n| NoPoSplat | $\\checkmark$ | 18.007 | 0.384 | 0.584 |\n| Ours | $\\times$ | 27.028 | 0.116 | 0.879 |\n\n| Methods (12 views) | Pose Supervision | PSNR $\\uparrow$ | LPIPS $\\downarrow$ | SSIM $\\uparrow$  |\n| --- | :---: | --- | --- | --- |\n| NoPoSplat | $\\checkmark$ | 17.625 | 0.399 | 0.583 |\n| Ours | $\\times$ | 28.133 | 0.099 | 0.993 |\n\n> More qualitative analysis for the ablation study. \n\nWe thank the reviewer for the valuable suggestion. Although it has been challenging to identify samples that clearly distinguish all the variants in Table 4 during rebuttal period, we agree that including such examples would further strengthen our work. We will incorporate additional qualitative results in the updated version of the paper.\n\n> In Table 4, Please explain (I-II) Scale/Shift Tuning Depth Network in the paper.\n\nVariant (I-II) in Table 4 refers to the approach where we fine-tune the scale and shift parameters used to convert relative depth estimates into UniDepth predictions. We observed that directly fine-tuning the UniDepth parameters resulted in unstable training, indicating that a more sophisticated training strategy may be required.\n\n> Around line 211 and 212, does the inputs to 3D Gaussian parameter prediction also include the refined pose? I assume that gradient can also be flowed from the refined depth input.\n\nYes, that's correct. The refined pose is used to construct the standard MVS cost volume, which is then aggregated with the guidance cost volume. Consequently, gradients do flow from the refined depth input as well, contributing to the overall optimization process.\n\n> Intuition of confidence estimation and what exactly does it help with the reconstruction overall?\n\nThis confidence score is used to modulate the regression of 3D Gaussian parameters, such as opacities and covariances, ensuring that more reliable matches have a greater influence on the reconstruction. Empirically, this approach improves both rendering quality and pose estimation accuracy, leading to an overall improved reconstruction, as shown in the Tab. 4.\n\n> How did you determine the lambda for 3D-3D loss?\n\nWe set the lambda for the 3D-3D loss to 0.05 by balancing its scale with the other losses in our framework. This value was chosen empirically, and a more extensive hyperparameter search might yield performance improvements.", "claims_and_evidence": "Claims are supported by qualitative and quantative results of novel view synthesis and pose estimation on RealEstate10K, ACID, and DL3DV datasets.", "methods_and_evaluation_criteria": "Utilizing pretrained models and refining them to predict 3D Gaussians makes sense. Evaluation includes both indoor and outdoor real-world datasets, following pixelSplat[1] , MVSplat[2], and NoPoSplat[3]. \n\n[1].Charatan, D., Li, S., Tagliasacchi, A., and Sitzmann, V. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. arXiv:2312.12337, 2023.arXiv preprint\n[2].Chen, Y., Xu, H., Zheng, C., Zhuang, B., Pollefeys, M., Geiger, A., Cham, T.-J., and Cai, J. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. arXiv preprint arXiv:2403.14627, 2024.\n[3].Ye, B., Liu, S., Xu, H., Li, X., Pollefeys, M., Yang, M.-H., and Peng, S. No pose, no problem: Surprisingly simple 3d gaussian splats from sparse unposed images. arXiv preprint arXiv:2410.24207, 2024.", "theoretical_claims": "In the last equation in equation 2, t misses subscript i. The architecture of T_agg is not specified.", "experimental_designs_or_analyses": "1. Experimental designs on unposed triplet, with the test set divided into small, middle, and large based on overlap between input views follows . \n2. Separating the pose-free and pose-required methods in results is resonable. \n3. Experiments and analyses in comparison to scene-optimization approach, speed, and cross-domain results are valid.", "relation_to_broader_scientific_literature": "The key contributions of the paper relate to how to leverage pretrained models and refine their estimations to predict 3D structures. Certain design choices, e.g. network architecture, cost volumes, training losses contribute the performance. The pose-free setting can be applied to in-the-wild scenarios and general 3D reconstruction. Feed-forward manner relates to the efficiency and speed of such methods.", "essential_references_not_discussed": "Another pose-free feed-forward Gaussian Splatting paper FreeSplatter[1] not discussed in the paper.\n\n[1] Xu, Jiale, Shenghua Gao, and Ying Shan. \"FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstruction.\" arXiv preprint arXiv:2412.09573 (2024).", "other_strengths_and_weaknesses": "Strengths\n1. The task is relevant and important in the field, and the paper is overall well-written and easy to follow.\n2. The proposed coarse and refinement pipeline together with other design choices mentioned above is effective and outperform other pose-free methods. \n\nWeaknesses\n1. The contribution mainly relies on the refinement modules. While it is effective, much of the work builds on existing works or ideas, such as Unidepth[1], LightGlue[2], cost volumes[3].\n2. NoPoSplat [4] is acknowledged but not evaluated and compared.\n3. More qualitative analysis is expected for the ablation study.\n\n[1]. Piccinelli, L., Yang, Y.-H., Sakaridis, C., Segu, M., Li, S., Van Gool, L., and Yu, F. Unidepth: Universal monoc- ular metric depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10106–10116, 2024\n[2]. Lindenberger, P., Sarlin, P.-E., and Pollefeys, M. Lightglue: Local feature matching at light speed. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17627–17638, 2023.\n[3]. Chen, Y., Xu, H., Zheng, C., Zhuang, B., Pollefeys, M., Geiger, A., Cham, T.-J., and Cai, J. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. arXiv preprint arXiv:2403.14627, 2024.\n[4]. Ye, B., Liu, S., Xu, H., Li, X., Pollefeys, M., Yang, M.-H., and Peng, S. No pose, no problem: Surprisingly simple 3d gaussian splats from sparse unposed images. arXiv preprint arXiv:2410.24207, 2024.", "other_comments_or_suggestions": "1. In Table 4, I assume (I-III) and (I-IV) are 2D-3D and 3D-3D consistent loss. The naming are somewhat confusing. These losses are essential to achieve good quality based on the results. In the current writing, however, they are not introduced in the earlier part of the paper. \n2. In Table 4, Please explain (I-II) Scale/Shift Tuning Depth Network in the paper.\n3. There is a typo \"sprase\" in line 154.", "questions_for_authors": "1. Around line 211 and 212, does the inputs to 3D Gaussian parameter prediction also include the refined pose? I assume that gradient can also be flowed from the refined depth input. Is this the case?\n2. Followed by question 1, could you explain more about the intuitation of confidence estimation and what exactly does it help with the reconstruction overall?\n3. How did you determine the lambda for 3D-3D loss?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CMd9ZauaPF", "reviewer": "Reviewer_iLyu", "summary": "This paper proposes a novel 3D Gaussian Splatting prediction model based on sparse views. Starting from a coarse initialization using off-the-shelf depth and correspondence models, the proposed fine alignment module predicts the correct scale for the depth map and camera poses, and finally, Gaussian heads predict the 3DGS parameters. The results demonstrate improved performance over baseline methods.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Questions about “feed-forward” approach, because there is a module for a test-time iterative optimization for initial pose.\n\nWe wish to refer to reviewer k81v’s appreciation that ***“feed-forward” in our context emphasizes efficiency and speed*** (See comparison to InstantSplat).  Although our coarse initialization stage employs iterative optimization, specifically, PnP+RANSAC, which is recognized as fast and efficient that takes only milliseconds (as also noted in NoPoSplat), the subsequent fine alignment module is fully feed-forward and represents a novel contribution. This fine alignment refines the coarse pose to correct 3D Gaussian misalignments, leading to significant performance gains (see Tab. 4).  Leveraging off-the-shelf models for coarse alignment allowed us to focus on this key challenge (Our novelty in this aspect of coarse alignment is also recognized by the reviewer TAPU and EyLp),  and we believe that our framework is novel and addresses a very important task in computer vision, which are appreciated by reviewer K81V and EyLp. We hope that this minor terminology point should not impact the overall evaluation of our work.\n\n> The authors assume that camera intrinsics are generally available, which is not always the case for generic video analysis tasks.\n\nWhile we acknowledge that there are some corner cases where intrinsics might not be available, ***this limitation is common to many pose-free feed-forward NVS methods, as discussed in Sec. A.6.***  Nonetheless, when using intrinsic estimates from UniDepth at inference, our method still outperforms CoPoNeRF as shown below. We wish to stress that in practical applications, intrinsics are generally available, and even in cases where they are not, our approach maintains competitive performance relative to other methods facing the same limitation.\n\n\n| Method                 | PSNR    | SSIM   | LPIPS |\n|------------------------|---------|--------|-------|\n| CoPoNeRF               | 19.536  |  0.638 | 0.398 |\n| Ours w/o GT intrinsics | 22.688  | 0.733  | 0.201 |\n| Ours                   | 23.589  | 0.782  | 0.181 |\n\nFrom the results, we highlight that even without GT intrinsics, ours outperforms CoPoNeRF and also performs on par with ours with GT intrinsics. This suggests a potentially feasible approach to alleviate the limitation. \n\n> The study overlooks some state-of-the-art baselines, such as NoPoSplat and MASt3R-SfM, both of which had publicly available code well before the ICML submission deadline.\n\nWe thank the reviewer for highlighting NoPoSplat and MASt3R-SfM. As noted by reviewer EyLp and the reviewer guideline explicitly states, ***NoPoSplat is a concurrent work—released on arXiv less than three months before our ICML submission and accepted by ICLR three days prior—so we are not obliged to compare against it***, though it is acknowledged in the related works (Nonetheless, we provide additional comparison below, as the reviewer kindly suggests). Moreover, MASt3R-SfM is  an arXiv paper and, while it shares similar objectives in 3D reconstruction, it addresses a different task than our pose-free feed-forward NVS (its sparse view variants (MASt3R) are already compared with our method in Tab. 2). \n\n Below includes the comparison:\n\n| Methods | Pose Supervision | PSNR $\\uparrow$ | LPIPS $\\downarrow$ | SSIM $\\uparrow$  |\n| --- | :---: | --- | --- | --- |\n| CoPoNeRF | $\\checkmark$ | 19.536 | 0.398 | 0.638 |\n| NoPoSplat | $\\checkmark$ | 26.820 | 0.126 | 0.879 |\n| DBARF | $\\times$ | 14.789 | 0.490 | 0.570 |\n| FlowCam | $\\times$ | 18.242 | 0.597 | 0.455 |\n| Ours | $\\times$ | 23.589 | 0.181 | 0.782 |\n\nWhile NoPoSplat performs well, as indicated in the table, we stress that our method clearly differentiates in the setting that the 3D geometry data, such as GT camera pose, is not utilized during training. Moreover, we show in the following that our method can be naturally extended to N-view setting, where NoPoSplat struggles.\n\n| Methods (6 views) | Pose Supervision | PSNR $\\uparrow$ | LPIPS $\\downarrow$ | SSIM $\\uparrow$  |\n| --- | :---: | --- | --- | --- |\n| NoPoSplat | $\\checkmark$ | 18.007 | 0.384 | 0.584 |\n| Ours | $\\times$ | 27.028 | 0.116 | 0.879 |\n\n| Methods (12 views) | Pose Supervision | PSNR $\\uparrow$ | LPIPS $\\downarrow$ | SSIM $\\uparrow$  |\n| --- | :---: | --- | --- | --- |\n| NoPoSplat | $\\checkmark$ | 17.625 | 0.399 | 0.583 |\n| Ours | $\\times$ | 28.133 | 0.099 | 0.993 |\n\nTo summarize, compared to NoPoSplat, our approach offers key advantages compared to NoPoSplat: it is a conceptual contribution proposing a general pipeline for pose-free view synthesis that addresses 3D Gaussian misalignment using off-the-shelf depth and correspondence networks and fine alignment modules; it naturally extends to N-view inference without retraining; and it is designed for scenarios without 3D geometry data (GT camera pose) during training. We will additionally include this discussion in related works.", "claims_and_evidence": "There are several concerns regarding the claim:\n- I do not think the proposed method qualifies as a “feed-forward” approach, because the coarse initialization stage employs a test-time iterative optimization for initial pose estimation. A feed-forward model typically involves directly regressing scene parameters in an end-to-end manner, but the proposed method is more like an engineered system that combines several off-the-shelf pre-trained models and iterative optimization modules, raising questions about its elegance and novelty.\n- The authors assume that camera intrinsics are generally available, which is not always the case for generic video analysis tasks.", "methods_and_evaluation_criteria": "The benchmark datasets are reasonable; however, the study overlooks some state-of-the-art baselines, such as NoPoSplat and MASt3R-SfM, both of which had publicly available code well before the ICML submission deadline.", "theoretical_claims": "Checked.", "experimental_designs_or_analyses": "Checked.", "relation_to_broader_scientific_literature": "As noted in the \"Methods and Evaluation Criteria\" section, some important baselines are missing.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "While the method demonstrates a practical system design and solid performance on benchmark datasets, it appears to be a composite system that lacks significant scientific insight.", "other_comments_or_suggestions": "No", "questions_for_authors": "Already commented questions and concerns.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R7K48l5yiS", "reviewer": "Reviewer_Sfrw", "summary": "This paper introduces PF3Plat, a novel two-stage framework for novel view synthesis from sparse and unposed images. In the first stage, it leverages pre-trained depth estimation and visual correspondence models to achieve a coarse alignment of 3D Gaussians. In the second stage, it refines depth and pose estimation using geometry-aware modules.\n\n## update after rebuttal\n\nI remain positive due to the impressive experimental results.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Why not use MVSplat with MASt3D, given that this method falls behind MASt3R in camera pose estimation?\n\nWe opted not to use MVSplat with MASt3R because MASt3R's runtime of around 10 seconds conflicts with our goal of achieving a fast, feed-forward process.  Moreover, our experiments include a variant, MASt3R*, which employs PnP+RANSAC for pose estimation similar to our approach, yet MASt3R* performs worse than both our baseline and final model, as shown in Tab. 2. We believe these reasons justify our design choice for the baseline. \n\nAdditionally, we wish to highlight that we have shown comparisons that outperform MASt3R. We kindly refer the reviewer to ***Tab. 6***, where RoMa outperforms MASt3R, and our variant that uses RoMa enables further improvements in both image and pose estimation quality. Through this experiment, we highlight the flexibility of model choice for coarse alignment stage and a potential for performance improvements. \n\n> This paper assumes known camera intrinsics, whereas DUSt3r does not have this requirement. \nWe thank the reviewer for highlighting this point. As discussed in Sec. A.6, we acknowledge that assuming known camera intrinsics is a limitation common to many NVS methods. Nevertheless, we conducted an additional experiment using UniDepth's intrinsic predictions instead of ground truth, and the results indicate that our method still performs well without GT intrinsics.\n\n\n| Method                 | PSNR    | SSIM   | LPIPS |\n|------------------------|---------|--------|-------|\n| CoPoNeRF               | 19.536  |  0.638 | 0.398 |\n| Ours w/o GT intrinsics | 22.688  | 0.733  | 0.201 |\n| Ours                   | 23.589  | 0.782  | 0.181 |\n\nFrom the results, we highlight that even without GT intrinsics, ours  outperforms CoPoNeRF and also performs on par with ours with GT intrinsics. This suggests a potentially feasible approach to alleviate the limitation. \n\nMoreover, our primary focus is on generalized novel view synthesis from unposed images—where pose, depth, and correspondence estimation serve as intermediate tasks to achieve high-quality novel view rendering. In contrast to SfM-based methods such as DUSt3R, VGGSfM, and SP+SG, our approach differs in terms of data, objectives, and training setups. Notably, methods like DUSt3R and MASt3R require additional steps (e.g., training a radiance field via NeRF or 3D Gaussian Splatting), incur long optimization times per scene.\n\n> The improvement over InstantSplat is limited.\nWe thank the reviewer for the comment. As shown in Tab. 5, our method outperforms InstantSplat by a large margin, and our inference speed is 100x faster, demonstrating a significant practical advantage. Moreover, when a more advanced correspondence network is adopted or test-time optimization is performed (like InstantSplat) for coarse alignment, training process or inference-phase optimization, our method further broadens the gap, as shown below:\n\n| Method       | PSNR   | SSIM  | LPIPS | Rot Avg | Rot Med | Trans Avg | Trans Med | Time  |\n|--------------|--------|-------|-------|---------|---------|-----------|-----------|-------|\n| InstantSplat | 23.079 | 0.777 | 0.182 | 2.693   | 0.882   | 11.866    | 3.094     | 53    |\n| Ours         | 23.589 | 0.782 | 0.181 | 1.756   | 0.897   | 9.474     | 4.628     | 0.390 |\n| Ours+TTO     | 24.689 | 0.798 | 0.167 | 1.662   | 0.871   | 8.998     | 4.311     | 24    |\n| Ours + RoMa  | 24.412 | 0.799 | 0.167 | 2.152   | 0.501   | 7.544     | 3.233     | 0.523 |", "claims_and_evidence": "The claims are well-supported by extensive experimental results.", "methods_and_evaluation_criteria": "The paper presents a highly practical solution for novel view synthesis from unposed images.", "theoretical_claims": "To the best of my knowledge, the theoretical claims appear to be correct.", "experimental_designs_or_analyses": "In Table 4, the baseline model is MVSplat with pose estimation from Stage 1. Why not use MVSplat with MASt3D, given that this method falls behind MASt3R in camera pose estimation?", "relation_to_broader_scientific_literature": "This paper is related to novel view synthesis, pose estimation, and 3D reconstruction.", "essential_references_not_discussed": "To the best of my knowledge, none are missing.", "other_strengths_and_weaknesses": "This paper assumes known camera intrinsics, whereas DUSt3D does not have this requirement.\n\nFor camera pose estimation, this method underperforms compared to MASt3R, which does not require additional training on those datasets.\n\nThe improvement over InstantSplat is limited.", "other_comments_or_suggestions": "None.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper initially received mixed scores (2,2,3,3,3) and was on the fence. The authors submitted a rebuttal which did not help clarify the main concerns of the reviewers leaning towards rejection, in particular around the novelty of the work and the limited quality of the results provided. At the same time, one other reviewer confirmed the support for the acceptance based on the performance demonstrated by the method. Two other reviewers did not provide further feedback, despite the reminders from the AC. Taken all comments and material into account, the current AC recommendation is to weak accept this work. This decision has been also reviewed and agreed upon by the SAC.
Decision: Accept (poster)