Puzzles: Unbounded Video-Depth Augmentation for Scalable End-to-End 3D Reconstruction
Authors: Jiahao Ma
Lei Wang
Miaomiao Liu
David Ahmedt-Aristizabal
Chuong Nguyen
Venue: NeurIPS 2025 poster ()
Keywords: 3D reconstruction
data augmentation
Primary area: deep_learning
TL;DR: We propose Puzzles, a controllable augmentation that simulates high-quality posed video-depth data from a single image or clip, boosting 3D reconstruction performance with much less training data.

Abstract: Multi-view 3D reconstruction remains a core challenge in computer vision. Recent methods, such as DUSt3R and its successors, directly regress pointmaps from image pairs without relying on known scene geometry or camera parameters. However, the performance of these models is constrained by the diversity and scale of available training data. In this work, we introduce Puzzles, a data augmentation strategy that synthesizes an unbounded volume of high-quality, posed video-depth data from just a single image or video clip. By simulating diverse camera trajectories and realistic scene geometry through targeted image transformations, Puzzles significantly enhances data variety. Extensive experiments show that integrating Puzzles into existing video‑based 3D reconstruction pipelines consistently boosts performance, all without modifying the underlying network architecture. Notably, models trained on only 10% of the original data, augmented with Puzzles, achieve accuracy comparable to those trained on the full dataset.

Puzzles: Unbounded Video-Depth Augmentation for Scalable End-to-End 3D Reconstruction
Jiahao Ma Lei Wang" Miaomiao Liu', David Ahmedt-Aristizaaal" Chuong Nguyen 1. AAustrllian National University, Grifffith University, 'Data61/CSIRE
(jiahao. ma, miaomiaa. @anu. edu. au, 1. wang48grirfita edu, au (david. ahmedtaristizabbe chuong .gguyen @data61...... csiro. roroooad .u
Abstract
Multi-view 3D reconstruction remains core challenge in computer vision. Recent methods, such as DUSt3R and its successors, directly regress pointmaps from image pairs without relying known scene geometry camera parameters. However, the performance of these models is constrained by the diversity and scale of available training data. In this work, introduce Puzzles, data augmentation strategy that synthesizes an unbounded volume of high-quality, posed video-depth data from just single image video clip. By simulating diverse camera trajectories and realistic scene geometry through targeted image transformations, Puzzles enhances data variety. Extensive experiments show that integrating Puzzles into existing video-based 3D reconstruction pipelines consistently boosts performance, all without modifying the underlying network architecture. Notably, models trained on only 10% of the original data, augmented with Puzzles, still achieve accuracy comparable to those trained on the full dataset. website] [Code]
Introduction
Dense 3D reconstruction is a fundamental problem in computer vision, aimed at recovering detailed scene geometry from images or videos. Traditional approaches, based on Simultaneous Localization (SIM) [4], and Multi View Sterco (MVS) [5-7],
and Mapping (SLAM) [1-3], Structure-from-Moute typically follow multi-stage pipeline involving camera pose estimation, correspondence matching, and dense depth inference. While effective in controlled environments, these methods often strug- gle with scalability, computational efficiency, and robustness to viewpoint changes, limiting their applicability in large-scale, real-world scenarios.
limitations of monocular dense SLAMM. DUS13R [8], for instance, introduced an end-to-end pipeline that directly regresses pointmaps from image pairs. However, requires costly global alignment for multi-view reconstruction. Follow- up methods have improved on efficiency and scalability: Spann3R [9] replaces optimization- based alignment with incremental pointmap fusion shared coordinate frame using spatial memory; SLAM3R [10] reconstructs local geometry within a sliding window and integrates it incrementally into global map: Fast3R [11] scales further, processing thousands of frames per forward pass without the need for alignment. Despite their advances, the 3R-series methods depend heavily on large volumes of high- qualiiulity posed video clips with ground -tnnh depth. Unfortunately, such datasets [1]11] remain limited in diversity and scale, ultimately constraining model generalization.
Recent advances in leasning-based two-view geometry address
To address these challenges, introduce Puzzles, a data augmentation framework for 3D reconstruc- tion that increases data diversity while reducing dependency on large-scale video datasets. Rather than relying redundant video frames (Figure 1.A), Puzzles extracts and amplifies the geometric and photometric cues already presented in a single image (Figure I). Inspired by the making of jigsaw-puzzle, Puzzles partitions an image into ordered, overlapping sequence of patches. The
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

A. Real video clip
Puzzles augmentation
8.2
Ground-truhh comera trajectory
Symtheris camera Ingjectory
Figure I: Puzzles: A new data augmentation framework for training feedforward 3D reconstruction deep models. A. Existing vidco-based 3D reconstruction datasets often contain redundant frames due to fixed camera trajectories after recording (A.1). B. Puzzles increases view diversity by generating novel viewpoints from single image (B.1) and simulates realistic camera trajectories (B.2) to support 3D reconstruction training.
ordering encodes pseudo-temporal structure, while overlapping regions ensures spatial consistency, to- gether simulating the geometric characteristics of real video sequences. As illustrated in Figure 1.B.2, these patches are further augmented to simulate realistic camera rotations and translations, producing diverse, like clips with synthetic depth and pose. We refer to this process Image-to-Clips,
While Image-to-Clips enables fine-grained local reconstruction, single image does not provide sufficient spatial coverage for large-scale scenes, To overcome this limitation, we propose Clips-to- Clips, an extension that operates on video clips by selecting strategic keyframes to generate multiple diverse sub-clips. This balances spatial coverage with augmentation diversity and mitigates the continuity issues caused by random frame sampling in prior works [9, 10].
Extensive experiments show that integrating Puzzles into existing video-based 3R-series pipelines consistently improves reconstruction quality under the same training data budget. Remarkably, models trained on just 10% of the dataset, when augmented with Puzzles, match or exceed the performance of models trained on the full dataset. Notably, Puzzles requires no changes to network architectures, making it drop-in, plug-and-plas solution. Our contributions are:
We propose Puzzles, comprising Image-to- souge-to-Clips and Clips-to- to-Clips, which generates realistic and diverse video-like clips with estimated depth and pose from single images or video clips via ordered, overlapping patch-based augmentation.
ii We show that integrating Puzzles into video-based 3R-series pipelines consistently improves performance, even under reduced data budget, with models trained on just 10% of the data achieving comparable or better results than full-data baselines.
ii.. Puzzles is plug-and-play augmentation method that can be applied any learning-basse based dense 3D reconstruction pipeline, without modifications to the underlying architecture.
2 Related Works
End-to-end dense 3D reconstruction. DUSt3R [8] presents the first fully end-to-end framework for dense 3D reconstruction without requiring explicit camera calibration. Building on this, subsequent works have adapted the same paradigm to tasks, e.g., single-view reconstruction [15], feature matching [16), novel-view synthesis [17-19], camera pose estimation [20], and dynamic scene reconstruction [21-25], highlighting the general urility of dense point prediction. However, extending DUSt3R to video reconstruction remains both computationally expensive and error-prone. To address this, Spann3R [9] processes frames sequentially using sliding window and external memory, while

---

A. RGBD image
B. Generati pulches
C. Calibration
D. Augmeciation
Posed
images
Depths
Figure 2: Puzzles: Image-to-Clips. (A) Starting from a single RGB-D image, we (B) partition it into ordered, overlapping patches, (C) simulate diverse viewpoints by calibrating virtual camera poses, and (D) generate augmented, posed images with aligned depth maps for use in 3D reconstruction.
Slam3R [10] incrementally registers overlapping clips. Fast3R [11] further improves scalability by reconstructing hundreds of unordered views a single forward pass. Despite architectural advances, these 3R-series works [8, 16, 26, 9-11, 27, 28] require vast amounts of posed video depth data or image pairs extracted from resources that we found still limited both diversity and quantity. Unlike prior work focused solely model design, adopt data-centric strategy: augmenting real data to synthesize rich, varied training samples that substantially improve model performance.
Data augmentation for deep learning, Data augmentation is essential in deep learning, particularly when labeled data is scarce. Accordingly, has been widely adopted across numerous computer vision tasks. In image classification, methods such as image erasing [29-31] typically remove sub- regions to improve invariance, while image mixing techniques [32-35] blend multiple images or patches into one. For object detection, region-level transformations coupled with bounding-bow adjust- ments [36] improve localization accuracy. In segmentation, ClassMix [37] synthesizes new examples by exchanging class-specific regions across unlabeled images. For visual tracking. MASA [38] and BIV [39] simulate pseudo frame pairs by augmenting static images to emulate motion. Despite these advances, data augmentation for dense 3D reconstruction remains underexplored. To address this gap. we introduce Puzzles, a method that synthesizes full video sequences with corresponding camera poses and depth maps from a single image or short clip. Unlike prior methods [38-40] that focus on generating frame pairs, Puzzles create temporally coherent video streams that offer rich 2D viewpoint variation while preserving geometric consistency in 3D.
3 Method
Given a monocular video consisting of sequence of RGB frames (T" E HxWs3 ) static scene, the goal is to reconstruct dense pointmap (X" E #RWW33N represented the coordinate system of the initial frame. Our focus on improving reconstruction quality through data augmentation, without modifying the network architecture training objectives, to isolate the effect of the augmentation itself. To this end, we propose two strategies for augmenting input video sequences: (i) Image-to-Clipp (Section 3) synthesizes video clips with simulated geom- etry and camera motion from a single image. (ii) Clips-to- Clips-to-Clips (Section 3.2): extends augmentation entire videos by generating diverse sub-clips that improve spatial coverage.
In=1 capturing JR=I where each X" is
3.1 Image-to-Clips
key property of video the spatial overlap between consecutive frames, which induces multi-view consistency, crucial element in 3D reconstruction. We use this principle synthesize pseudo-video sequences from a single RGB-D image. Below, we detail the Image-to- -l-pp method (see Figure 2).
Patch generation. To simulate video-like continuity, we extract an ordered set of overlapping patches from a single RGB-D frame. As shown Figure 2.B ("Generate patches"), each patch overlaps with at least one previous frame, mimicking temporal coherence. All generated patches share identical

---

Figure 3: Example of Image-to- Co-pp.. Given single input image (left of each block), the proposed Image-to-Clips method samples ordered, overlapping patches (colored boxes) and assigns simulated 6-DoF camera trajectories (frustums) to generate view-consistent video clips. Examples across human-centric illustrations, indoor environments, and outdoor landscapes demonstrate the method's ability to synthesize diverse and realistic training sequences from single frame.
resolution. Patch size determines the field of view: smaller patches mimic close-up views rich in local detail, while larger ones simulate wide-angle views, replicating zoom-in/out effects. We control overlap using bounding box IoU and dynamically vary the overlap ratio, starting high to promote consistency, then decreasing it to increase diversity and training difficulty. Whether ground-truth predicted depth is used, we observe similar benefits (Figure 2.A uses predicted depth).
Patch calibration. We estimate the camera intrinsics and extrinsics for each patch, allowing for consistent geometry and downstream use. There are two calibration strategies. Varying intrinsics, fixed extrinsics: modify the full-image intrinsics per patch for fast rendering, but cannot simulate camera motion. 2. Fixed intrinsics, varying extrinsics: use constant intrinsics to emulate camera motion (Figure 3), although extrinsic estimation incurs errors. We employ a hybrid of the two approaches, with the second method being applied more frequently during training. Details follow.
Varying intrinsics, fixed extrinsics. In this case, Tm = 4s 0). Let the full-image intrinsics be Kínl E IR3x3, and the depth map be D E HxW We define the corresponding 3D point map
Xn Kial [uDu, DDn,, Da.v] where (u,u) (1..... W) (1,..... H) pixel coordinates.
From each image, extract M E Zz1 rectangular patches (Bm) Im=1, where mm juft, 2f" ut" T are the pixel coordinates of the top-left and bottom-right corners of the m-th patch. Each patch is resized to a fixed resolution W" x H'. The intrinsics of each patch Km patch derived from Kinl via scaling:
Ja sm
(Cr
W
H'
patch =
fg
27 3
(Cy
y
8
2
y
U/
(2)

---

Fixed intrinsics, varying extrinsics. Let consistent intrinsics per patch: Km patch (/= o /2 to fy recover the extrinsic parameters Tw2 R t= for each patch m. we first crop the dense point map to obtain patch-specific 3D point set X E IR xW'x3, We then estimate the camera pose by (PnP) problem using RANSAC [41, 42], which provides robustness where X; R is 3D point
solving a Perspective-n-Pe to outliers. We define set of 3D-2D correspondences (x,, in world coordinates (from the depth map), and 2; E R is its corresponding 2D location in the image patch. To estimate the rotation R Th (O(3) and translation t" E R minimize the total reprojection error over a set of inliers I S {I,...,N} N..
This is formulated as:
RT tm arg min R.
x;
(K patch (RX;+ + t))ll?
(3)
AEI
The projection function π RR3 I222 is defined as: x( (x 2 2 T This process yields the optimal camera-to-worde transformation for each patch, enabling accurate alignment between 2D image observations and 3D geometry.
Camera motion augmentation. After recovering geometric and camera parameters for each patch, we obtain all the necessary information to support standard 3D reconstruction and pose estimation tasks. However, this setup has a key limitation: simply cropping image patches mostly simulates translational motion, such as lateral or forward shifts, without introducing any realistic rotational dynamics (see Figure 2.C). Experimental results in supplementary materials show that combining both rotational and translational motion improves performance.
Centroid- based camera rotation. To simulate more realistic and diverse camera motion, we apply controlled random rotation around the centroid of the 3D point cloud within each patch. Let the original camera-to world pose be T
T T and define the centroid of the 3D points R
0
as Cp = N ΣN == X; We sample a random rotation angle from a mixture of a uniform distribution Gaussian noise, ~ (0 maxx + (0 o²), ). and random unit axis, n ~ (0, 7), normalized such that | 1. Using Rodrigues' formula [43], construct the corresponding rotation matrix
Rn,0 E (3) as:
R cos 0 I sin 0 [n]x + (1 cos 0) mn
where in| is the skew-symmetric matrix of n. The resulting 4x4 homogeneous transformation about centroid Cp is:
Trot = Rn, Cp
-RRR
(5)
We update the camera pose by applying this rotation, and compute the new world-to-cameraa extrinsic matrix
w2c = (Trot T
(6)
This augmentation introduces realistic viewpoint variation while maintaining the scene focus, enabling free-viewpoint rotation beyond what patch cropping can achieve, shown in Figure 2.
Valid view checks. Random rotation can result invalid camera views that negatively impact training. We therefore apply two geometric validation tests to ensure the quality of the rotated view: front-surface coverage test and view-frustun inclusion test.
Front Front-surface coverage test. This test evaluates whether the camera observes a sufficient amount of front-facing geometry. For each 3D point with position pt and outward- unit normal ni, we compute the dot product between the normal and the viewing direction from the camera center, given by c. RR'm tm The front-surface coverage score is defined as:
FrontCov
1 N
Σ
(Ce p:)
> cos Ovalid)
Cc pill2
where Ovalid a predefined angular threshold, 1(. (()) is the indicator function, N the number of 3D points, p. E R is the 3D position of point and B is the corresponding unit normal
5

---

serpine
Key frenes seeeice Irogeto dlpe
B. Oveelippin rutri
Figure Puzzles: Srips-to-Clips. (A) We begin by uniformly sampling frames from a video. (B) pair-wise overlap matrix is computed to measure frame redundancy, with overlap visualized in purple and overlap ratios annotated in red. (C) Low-redundancy keyframes are then selected, and diverse sub-clips are synthesized from them using the Image-to-Clin method.
vector. The metric FrontCove E [0, 1] quamtifies the fraction of points that are front-ffcing with respect to the camera. A score of FrontCov 1 indicates all points are front-facing, while FrontCov implies that none of the points face the camera.
ii View-frustum coverage test. This test ensures that the projected 3D points lie within the visible image frame. Each 3D point P; projected into the image plane using the updated camera extrinsics and the patch-specific intrinsic matrix Km patch
f/ 1
putch (R" "P'
P. + ) = =1(0000 S u, < W', 0. 0
< H', Du,,, > 0), (8)
where Si is a binary indicator that equals if the projected point falls within the image bounds and has a positive depth value. We define the image coverage score as: ImgCov w' Σ, Ji. which measures the fraction of the projected points that are valid and visible in the image.
During augmentation, we generate finite set of candidate camera poses and select the one that maximizes both FrontCov and ImgCov. Finally, we render the point clouds using Open3D [44] with the augmented camera parameters to generate high-quality posed video-depth sequences (see bottom row of Figure 2). If camera rotation reveals occluded areas, we fill them with white; too many occlusions occur, we discard that patch's 's rotation. The main text focuses on camera motion augmentation, while additional 2D augmentations are detailed the supplementary material.
3.2 Clips-to-Clips
To extend coverage beyond individual images, we generalize our augmentation strategy from Image- to-Clips to Clips-to-Clips. Video clips often exhibit significant redundancy (see Figure 1.A), and prior methods [9, 10] typically sample frames at fixed or random intervals, overlooking the spatial relationships between them (details in supplementary material). This can result in pointmaps with little no overlap between views, leading to geometrically inconsistent clips that hinder training and reduce accuracy. Our goal is to select key frames that both preserve scene continuity and provide sufficient visual coverage. We then apply the Image-to-Clipp augmentation on these selected frames.
Overlapping matrix. Given the camera parameters and depth maps, we compute geometric overlap between video frames using an overlapping matrix O E R xN," where N is the total number of frames. Each element O,y [0, quantifies the fraction of overlapping geometry between frames i and j. A value of Oz = 0 means no overlap, while O:j = 1 indicates full overlap. As shown in Figure 4.B, this matrix provides a principled basis for selecting a minimal yet representative set of key frames that maximize coverage while minimizing redundancy. To compute Oij. we treat each frame v as a source view and project pixels into a reference view v". Let x" be the homogeneous coordinates of pixel in the source frame, and D*(x) its depth. We compute the reprojected coordinates - and the corresponding depth x" in the reference frame. We then compute the reprojected depth with the reference depth map D=-rr at the same pixel location. The

---

overlap score is defined as:
Σ 1(D 1(1D->(x**)))
() - <T),
O₂ = 1&
(9)
mEV.
where V2 is the set of valid source pixels (D₂ > 0) and is threshold that accounts for scene- dependent depth variance.
Key-frame selection. To ensure comprehensive scene coverage with minimal redundancy, we extract representative frames from the overlap matrix O using a three-step procedure (see Figure 5): i. Validity filtering: Retain only frames whose maximum overlap with any other frame exceeds a threshold ny: V/ = { maxiąi Oij > n). For example, the gray frame in Figure 5(A), with an overlap 0.1, is discarded.
ii Longest cover set: Among valid frames, select a seed frame S that has the largest number of neighbors with sufficient overlap: S = arg max,ev, [ E V Oi 2 r}/ In Figure 5(B), the purple frame selected over the pink on as covers three neighbors instead two.
iii. Redundancy pruning: From this set, keep frame j only it is not highly redundant with any earlier selected frame: P = {j E S Yk < j, max(O5k. Ouÿ) < p}. For instance, the pink frame excluded in Figure 5(C) due to strong bidirectional overlap with an earlier frame.
The final keyframe set P removes isolated frames, identifies the largest group of mutually overlapping views, and eliminates redundant se- lections As shown in Figure 4, the original clip contains repeated views of both the front and back of the building. Our method selects two representative frames: front (Fz) and back (Fi), as keyframes, and applies Image-to-Clipp aug- mentation to each, resulting more diverse and informative training clips.
1.Bd Enzes
0.7
0.6
0.9
0.1
0.8
0.R
0.7
0L0
0.8
Longed corver set
Phaaa rafaslncy
0.6
Figure 5: Keyframe selection. Colors repre- sent overlapping groups; numbers indicate pair- wise overlap scores O.j. We discard low-overlap frames, keep the highest-coverage group, and re- move redundant ones to form the final keyframes.
Plug-and-play augmentation. Our Clips-to-
Clips is modular and easily integrates into most modern 3D reconstruction frameworks. It accepts standard video clips and outputs augmented, geometry-awan training samples. The key-frame selection process is internal and does not require altering the original data loading or sampling logic.
4 Experiments
4.1
Setup
Datasets and metries. We train on a blended corpus comprising ScanNet-v2 [12], ARKitScenes [45], a selected Habitat subset [46], and the in- initheewild/ojjeet dataset BlendedMVS [47], totaling approxi- mately 14 million images. For evaluations in Sections 4.2 and 4.3, we draw uniform subsamples of varying size to study the scaling behavior of Puzzles. To assess the impact of our augmentation, we evaluate on three unseen datasets: 7Scenes [48], NRGBD [49] and DTU [50] using Accuracy (Acc) and Completion (Comp) metrics from [9]. Predicted dense point maps are compared to back-projected ground-truth depth, excluding invalid and background points.
Baselines. We adopt three representative video-based 3R-series methods, Spann3R [9]. SLAM3R [10], and Fast3R [11], basclines. Since each was originally trained on a distinct dataset, we preserve their published training protocols and architectures, and retrain them on a unified dataset both with and without our Puzzles data augmentation. As a result, the reported metrics may deviate from those reported in the original papers; our objective is not to replicate prior results, but to investigate how each model responds to changes in data distribution. Using Spann3R as an example, we follow the official training setup and train for 120 epochs on the same dataset with the same 14M training samples. The only change the application of Puzzles augmentation. Puzzles is a plug-and-pl module applied on-the-fly through the dataloader, without requiring any pre-processing or modification of the training pipeline. Training was conducted on eight H100 GPUs for 120 epochs. The total training time approximately 40 hours with Puzzles, compared to 36 hours without. We
7

---

NRGBD
Method
w/ Puzzles
Data
Acc
Comp
Acc.J.
Comp
Acc
Cemp 1
Va
(4)
Value
4.1%
Valie
& (S)
Value
&1N)
Value
&
Value
full
0.0388
0,0253
0.0696
0,0315
6.2432
3.1259
Spann3R
1,30
0.0339
26
0.02248
+1.
0,0753
-9-78
0.0341
-5.30
4.9832 +20.18
2,1122
-19.47
1011
0.0330
+14.94
+11.46
0.0644
+5.00
0.0291
+7.91
+15.90
2.5113
-19.66
000412
0.0735
FaStRR [11)
1/10
0.0-02
-2.30
30
0.0272
+1. 09
0,0772
-5.11
0.0295
-2.78
3.7174
413.47
1,811
+6.41
Full
0.0342
+16.99 99
0.0239
+13.09
0.0684
+5.94
0.0259
3,5912
+16.41
1,7379
+15.96
Full
0.0291
0.0245
0.0481
0.0292
4 4.3820
2.4794
SLAM3R (10)
1710
0.0289
-0.
+3.26
0.0493
-2.49
0.0313
-7 -9.19
417.89 85
2,0891
-15. 60
Fo11 D.0264
-9.27
+11.02
0.0439
+3.73
3,6497
+16.
-16.12
Table I: Quantitative comparison on 7Scenes [48], NRGBD [49] and DTU [50]. For each metric report both the value and the relative improvement (4) achieved by incorporating Puzzles. All methods are evaluated on long video sequences without bundle adjustment refinement; note that Spann3R operates in online mode.
Cipe-roCCpp
ClpssteCClps
longetoCCpps
Figure 6: Comparison Clips-to-Clips and Clage-to-Clips.. A. Object-level reconstruction using 10 input views (#Views = 10), where both methods perform comparably. B. Scene-level reconstruction with 75 imput views (# (#VVeew) Views 75), where Image-to-Clips exhibits drift and unstable predictions, resulting in the discard of low-confidence outputs.
observed slightly slower convergence when using Puzzles, which is expected due to the increased diversity and complexity of the training data. While longer training could yield further improvements, use the same configuration across experiments to ensure fair comparison.
Implementation details. To generate N patches from single image, the first patch is generated randomly. Each subsequent patch is then created to ensure it overlaps with at least one of the previously generated patches, based on IoU calculations. This overlap constraint helps maintain spatial continuity and supports consistency across views. The patch size is determined dynamically, based on the image resolution and the desired number patches per image. In addition, each newly generated patch is required to overlap with at least one of the previously generated patches, but not necessarily with consecutive frames. This design choice accommodates scenarios with rapid camera motion, where adjacent frames may have little not overlap.
For Clipstto-Clips, we employ cut-off threshold 7) 0.1, minimum visibility threshold 0.2, and redundancy threshold 0.7. In Image-to-Clips, we constrain the maximum angle of incidence valid 100% and sample rotation angles uniformly from [30", 90%). Additionally, we apply random affine perturbation to the pointmaps: rotations up to 445. translations up to 20% of the
bounding box size, and scaling factors uniformly sampled from |0.8, 1.0]. The official source code for our method is available on GitHub: https:/ //g thub. com/ Ji a aaaoooo Ma ppzz/uzzzzzu es-code
4.2 Evaluation
Scene-level reconstruction. On scene-level 7Scenes and NRGBD benchmarks, integrating Puzzles consistently improves accuracy across all 3R-series methods. Fast3R shows the largest gains on 7Scenes with 17.0% reduction in Acc and 13. 1% in Comp, while Spann3R and SLAM3R also notable improvements. On NRGBD, gains are smaller but stable (6 10%). Notably, all models trained with just 1/10 of the training data match surpass their full-data baselines 7Scenes, though not on NRGBD. This discrepancy likely stems from NRGBD's larger and more complex scenes, which more prone to drift and demand greater data diversity. These results demonstrate that Puzzles enhances generalization and robustness, especially large-scale reconstruction, without requiring architecture modifications.
Object-level reconstruction. On the object-level DTU benchmark, Puzzles yields consistently large improvements across all methods. SLAM3R benefits the most, with 16.71% reduction in accuracy

---

error and a 16.12% reduction in completion error. Spann3R and Fast3R also improve substantially. each showing gains of 16 20% in accuracy. Notably, our training data differs from that used in [9-11], as we do not incorporate a dedicated object-level dataset [14]-only small portion of BlendedMVS is included. Despite this, Puzzles effectively generates diverse and detailed local data, resulting in strong performance gains on DTU.
4.3 Analysis
Camera pose distribution after puzzles. We analyze the differ- ence in camera pose distribution across the dataset used in the main paper.
Statistic
roll
pitch (*) yaw (*)
Rotation μ
89.6
0.04
Statistie
f, (m)
f, (m)
f. (m)
As shown in Table 2, Puzzles follow symmetric zero-mean aug- mentation that leads to no obvious changes in mean. The Puzzles augmentation introduces pronounced variance increases across all pose axes. the original dataset, broader axes (yaw, tz, tyy exhibit only modest variance gains, whereas previously narrow axes (pitch, roll, (2) expand substantially.
Translation P
-1.38
-1.39
2.45
Table 2: Mean camera poses (p). Zero-mean augmentations in Puz- zles preserve pose symmetry.
After augmentation, as shown in Table 3, most rotation axes fall within the 50% 110% range, and translations cluster around 20 ~ 35 m. This yields a more balanced and comprehensive pose distribution, exposing the model to a wider range of viewpoints. The enlarged standard deviations (σ) quantitatively capture the degree to which Puzzles expands the pose space, thereby enhancing gen- eralization to rare viewpoints, improving robustness atypical camera motions, and ultimately boosting model performance.
Axis
Deng
Paew
Increase
Rotation X (roll)
15.9"
54.3"
+2428
Rotation (pitch)
52.2
+85538
Rotution Z (yaw)
93.7
107.2"
+148
Translation io
31.1
35.6m
+14.48
Trunslation by
30.9
35.4m
+14.65
Translation d.
10.2m
20.1 m
+96.5%
Table 3: Camera-pose standard deviation (a) before and after augmentation.
Impact of white filling. The Image-to- Clips process inevitably introduces occluded re- gions due to camera motion. These regions are filled with white color and excluded from su- pervision during optimization. We explore the impact of the occluded regions. Specifically, we experimented with the inpainting method LAMA [51] to fill the occluded regions, while excluding the inpainted areas from the loss computation. However, shown in Table 4, our ex- periments conducted on eight NVIDIA H100 GPUs indicate that inpainting offers no significant performance improvement, while increasing the training time by approximately 32.5%. Moreover, existing inpainting approaches often introduce noticeable artifacts when filling large missing areas, which can contaminate valid image regions. Therefore, we chose to discard this strategy in our final pipeline.
Setting
Acc
Comp
Time J [h]
Spann3R (w/o, Inpainting)
0,0330 0.0224
40
Spann3R (w/ Inpainting) (())
0.0332 0.0221
-0.6 +1.3
+32. ((1338)
Table 4: Effect of inpainting: increased training time with negligible accuracy change.
Component effect analysis. As shown in Ta- ble 5. We further explore the ablation study to illustrate the impact of each component. The ex- periments show that Clips-to-Clips (C2C) bring the most significant performance gains, while Image-to-Clips (I2C) with augmentation (w/. Aug.) provides smaller but still positive im- provement. In scene-level reconstruction, I2C with augmentation helps simulate camera mo- tion, allowing the network to capture local de- tails. In contrast, C2C improves robustness and supports large-scale reconstruction by capturing global context.
Config.
Components
7Scenes
12C w/o. Aug. 12C w.. Aug. C2C
Ace |
4(%)
Baseline
0.1278
+ Aug. + C2C
0.0916
-28.3
0.0330
-64.0
Table 5: Impact Image-to-Clips (I2C), Clips- to- to-Clips (C2C), and augmentation (Aug.) compo- nents on 7Scenes.
Image-to- Clips vs. Clips-to-Clipp, Image-to-Clips generates dense, posed video-depth sequences inherently offers only local spatial coverage. Clips-to-Clips generalizes this idea by supplying significantly broader scene-level coverage. As illustrated in Figure 6.A, and the DTU results in

---

0.10
6.0
B
0.07
6.0
depth
Image-o-Clips
Puzzles
GT. depth
5.5
Clips-to-Clips
5,5
0.06
Accuracy
Accuracy
Accuracy
Accuracy
0.05
0.04
4.5
0.04-
0.02
4.0
0.03
4,0
7Scenes
DTU
Iwege-teC0p
DTU
Dataset
Comparison on 7Scenes [48] (scene-level) and DTU [50] (object- level). B. Model accuracy with Puzzles at varying data fractions. C. Image-to-Clipss single-image augmentation: model accuracy comparing predicted vs. ground-truth depth. All experiments use Spann3R as the baseline, and accuracy is reported using the same metric throughout (lower is better). The results in sub-figure B. are based on Clips-to-Clipp while C. corresponds to Image-to-Co topppps....
Figure 7: Ablation study.
Figure 7.A.2, both methods attain comparable accuracy for object-level reconstruction. However, in full scene reconstruction, as shown in Figure 6.B and 7Scenes results Figure 7.A.1, Clips-to-Clipp produce better and more stable results than Image-to-Clips. This is because Clips-to-Clipp supplies the network with additional scene-level supervision that Image-to-Clipp lacks. This broader coverage mitigates drift and stabilizes the inference, especially when test sequences contain more frames than seen during training.
Model accuracy across increasing dataset fractions. The experiment illustrated in the middle of Figure 7.B, compares Puzzles augmentation (black) to the baseline without augmentation (blue), over data fractions ranging from 1/20 to full. Lower values indicate better performance. Puzzles consistently yields gains at all fractions, with the largest relative gain at 1 //20 (over 15% improve- ment). Even with full data, Puzzles reduces error from 0.0388 to 0.0330 (see Table 1), demonstrating improved generalization. Notably, models trained with Puzzles on just 1/10 of the data match or exceed the performance of full-data baselines. Beyond 1/10, Puzzles consistently outperforms training on the entire dataset without augmentation.
Robustness of Image-to-Clips to predicted ground-truth depth. We evaluate the sensitivity of Image-to- Songe-to-Clips augmentation to the quality of depth input using the DTU benchmark, as shown in Figure 7.C. Two training settings contrasted: depth from an off-the-shelf monocular estimator [15] (solid blue bar), and another using the dataset's ground-truth depth (white bar). The model trained with predicted depth achieves accuracy of roughly 4.9, marginally outperforming the 5.1 result with ground-truth depth. This small difference. well within experimental variance, demonstrates that Puzzles is robust to depth quality and remains effective even without precise supervision, enabling the method to scale to large, unlabeled image collections without sacrificing reconstruction quality, reinforcing its "unbounded" scalability.
where augmented clips are paired with predicted
Trends and challenges. Video-based 3R-series methods show strong potential for dense 3D recon- struction, but they tend to suffer from drift when scaling large scenes with more views than during training. One mitigation strategy [52] combines learned front-end pointmap prediction with global optimization back-end. A more radical alternative [9, 11, 10] adopts fully data-driven approach, requiring a scalable architecture and vast scene-level datasets. In contrast, we pursue data-centric solution: our current augmentation is scene-specifie, but future work could explore stitching clips across scenes to synthesize unbounded, large-scale video-depth data.
5 Conclusion
We presented Puzzles, plug-and-play auttettgmmnttoionn framework that transforms single images or short clips into high-quality, unbounded video-depth sequences through ordered, overlapping patch synthesis. When integrated into existing 3R-series pipelines, Puzzles consistently boosts reconstruction accuracy Remarkably, models trained with just 10% of the original data, augmented with Puzzles, can match or exceed the performance of full-data baselines, demonstrating the power of data-centric solution scalable 3D reconstruction.
10

---

Acknowledgments and Disclosure of Funding
We sincerely thank the anonymous reviewers for their invaluable insights and constructive feedback, which have greatly contributed to improving work.
This work is supported by the CSIRO Data6l PhD Scholarship.
References
[1] Carlos Campos, Richard Elvira, Juan J Gómez Rodríguez, José MM Montiel, and Juan D Tardós. Orb-slam3: An accurate open-source library for visual, visual-inertial, and multimap slam. IEEE Iransactions on robotics, 376))1877-1189, 2021.
[2] Jakob Engel, Thomas Schöps, and Daniel Cremers. Lsd-slam: Large-scale direct monocular slam. In European conference on computer vision, pages 834- 849. Springer, 2014.
[3] Georg Klein and David Murray. Parallel tracking and mapping on camera phone. In 2009 8th IEEE International Symposium on Mixed and Augmented Reality, pages 83-86. IEEE, 2009.
[4] Johannes L Schonberger and Jan-Michael Frahm. Structure- frree-rro--eeeee motion revisited. In Proceed- ings of the IEEE conference computer vision and pattern recognition, pages 4104 4113, 2016.
from-motion
[5] Silvano Galliani, Katrin Lasinger, and Konrad Schindler. Massively parallel multiview stereopsis by surface normal diffusion. In Proceedings of the IEEE international conference on computer vision, pages 873 0.-88,12200 2015.
[6] Yasutaka Furukawa, Carlos Hernández, et al. Multi-view stereo: A tutorial. Foundations and trends@ in Computer Graphics and Vision, 91-2):1 2015.
[7] Johannes L Schönberger, Enliang Zheng, Machael Frahm, and Marc Pollefeys. Pixelwise view selection for unstructured multi-view stereo. In Computer Vision Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 1 11-14, 2016, Proceedings, Purt III 14, pages 501-518. Springer, 2016.
[8] Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 20697-20709, 2024.
[9] Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory. arXiv preprint arXiv:2408. 16061, 2024.
[10] Yuzheng Liu, Siyan Dong, Shuzhe Wang, Yanchao Yang, Qingnan Fan, and Baoquan Chen. Slam3r: Real-time dense scene reconstruction from monocular rgb videos. arXiv preprint arXiv:2412.09401,, 2024.
[11] Jianing Yang. Alexander Sax, Kevin Liang. Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. arXiv preprinl arXiv:2501.13922, 2025.
[12] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: kichly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828-5839, 2017.
[13] Chandan Yeshwanth, Yueh-Cheng Liu, Matthias NieBner, and Angela Dai. Scannet++: A high- fidelity dataset of 3d indoor scenes. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pages 12-22, 2023.
[14] Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the JEFF/CVF international conference on computer vision, pages 10901-10911, 2021.

---

[15] Ruicheng Wang, Sicheng Xu. Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open- open-domain images with optimal training supervision, arXiv preprint UrXiv:2410.19) 2024.
[16] Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. In European Conference on Computer Vision, pages 71-91. Springer, 2024.
[17] Brandon Smart, Chuanxia Zheng, Iro Laina, and Victor Adrian Prisacariu. Splatt3r: Zero-shot gaussian splatting from uncalibrated image pairs. arXiv preprint 24t.:i::40..39999 2024.
(18) Botao Ye, Sifei Liu, Haofei Xu, Xueting Li, Mare Pollefeys, Ming-Hsuan Yang, and Songyou Peng. No pose, no problem: Surprisingly simple 3d gaussian splats from sparse unposed images. arXiv preprint arXiv:2410.242077 2024.
[19] Jiahao Ma, Miaomiao Liu, David Aristt-arisi and Chuong Nguyen. Hashpoint: Accelerated point searching and sampling for neural rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4462-4472, June 2024.
[20] Siyan Dong, Shuzhe Wang, Shaohui Liu, Lulu Cai, Qingnan Fan, Juho Kannala, and Yanchao Yang. Reloc3r. Large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization. arXiv preprint arXiv:2412.08376, 2024.
[21] Xin Fei, Wenzhao Zheng, Yueqi Duan, Wei Zhan, Masayoshi Tomizuka, Kurt Keutzer, and Jiwen Lu. Driv3r: Learning dense 4d reconstruction for autonomous driving, arXiv preprint wrXiv:2412.06777 2024.
[22] Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for estimating geometry in the presence of motion. arXiv preprint arXiv:2410.03825, 2024.
[23] Jiahao Ma, Jinguang Tong, Shan Wang, Zicheng Duan, and Chuong Nguyen. Voxelized 3d feature aggregation for multiview detection. In 2024 International Conference Digital Image Computing:
Techniques and Applications (DICTA), pages 166-173. IEEE, 2024.
[24] Jiahao Ma, Tianyu Wang, Miaomiao Liu, David Ahiteta-ristizaball, and Chuong Nguyen. Dehm: Depth-consistent human modeling for multiview detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7731-7740, October 2025.
[25] Jiahao Ma, Zicheng Duan, Liang Zheng, and Chuong Nguyen. Multiview detection with cardboard human modeling. In Proceedings of the Asian Conference on Computer Vision (ACCV), pages 870.886, December 2024.
[26] Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. arXiv preprint arXiv:2501. 12387, 2025.
[27] Jianyuan Wang. Minghao Chen, Nikita Karaev, Andrea Vedaldi. Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. arXiv preprint arXiv:2503.1165) 2025.
[28] Shangzhan Zhang, Jianyuan Wang, Yinghao Xu, Nan Xue, Christian Rupprecht, Xiaowei Zhou, Yujun Shen, and Gordon Wetzstein. Flare: Feed-forward geometry, appearance and camera estimation from uncalibrated sparse views. arXiv preprini 13Xiv:2502.12/38, 2025.
[29] Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 13001-13008. 2020.
[30] Krishna Kumar Singh, Hao Yu, Aron Sarmasi, Gautam Pradeep, and Yong Jae Lee. Hide-and- seek: A data augmentation technique for weakly-supervisee localization and beyond. arXiv preprint arXiv:
2018
[31] Pengguang Chen, Shu Liu, Hengshuang Zhao, Xingquan Wang, and Jiaya Jia, Gridmask data augmentation. arXiv preprint arXiv:2001.04086, 2020.
12

---

[32] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017.
[33] Dan Hendrycks, Norman Mu, Ekin Cubuk, Barret Zoph. Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprini arXiv: 19/2.0771, 2019.
[34] Terrance Vreei and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv: 04552, 2017.
[35] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6023-6032, 2019.
[36] Barret Zoph, Ekin D Cubuk, Golnaz Ghiasi, Tsung- Yi Lin, Jonathon Shlens, and Quoe V Le. Learning data augmentation strategies for object detection. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVII 16, pages 566-583. Springer, 2020.
[37] Viktor Olsson, Wilhelm Tranheden, Juliano Pinto, and Lennart Svensson. Classmix: Segmentation-based data augmentation for semi-supervised learning. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1369-1378, 2021.
[38] Siyuan Li, Lei Ke, Martin Danelljan, Luigi Piccinelli, Mattia Segu, Luc Van Gool, and Fisher Yu. Matching anything by segmenting anything. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18963-18973, 2024.
[39] Sanghyun Woo, Kwanyong Park, Scoung Wug Oh, In So Kweon, and Joon-Young images and videos: simple learning framework for large vocabulary video object detection. In European Conference on Computer Vision, pages 238 258..58. Springer, 2022.
Young Lee. Bridging
[40] Xingyi He, Hao Yu, Sida Peng, Dongli Tan, Zehong Shen, Hujun Bao, and Xiaowei Zhou. Matchanything: Universal cross-modality image matching with large-scale pre-training. In Arxiv, 2025.
[41] MA FISCHLER AND. Random sample consensus: paradigm for model fitting with applica- tions to image analysis and automated cartography. Commun. ACM, 24(6):381-395, 1981.
[42] Richard Hartley and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge university press, 2003.
[43] Wikipedia contributors. Rodrigues' rotation formula formula-Wikipediia Wikipedia, the free encyclope- en wwkipedia. httppp/ww...cooottt.t..///ttt//e rttpeesttt__rttttiooo____eeeeoo formula. 2025. Accessed: 2025-05-04.
dia. https: //en
[44] Qian- Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3D: A modern library for 3D data processing. 18Xiv:1801.088 098477 2018
[45] Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe. Daniel Kurz, Arik Schwartz, et al. Arkitscenes: A diverse real- world dataset for 3d indoor scene understanding using mobile rgb-d data. arXiv preprint arXiv:2111.08897 2021.
[46] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub. Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEF/CVF international conference on computer vision, pages 9339-9347, 2019.
[47] Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 1790-1799 2020.
13

---

[48] Jamie Shotton. Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene coordinate regression forests for camera relocalization in rgb-d images. In Proceedings of the IEEE conference computer vision and puttern recognition, pages 1930-2937, 2013.
[49] Dejan Azinović, Ricardo Martin-Brualla, Dan B Goldman, Matthias Nießner, and Justus Thies. Neural rgb-d surface reconstruction. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 6290-6301, 2022.
[50] Henrik Aanæs, Rasmus Ramsbøl Jensen, George Vogiatzis, Engin Tola, and Anders Bjorholm Dahl. Large-scale data for multiple-view stereopsis. International Journal of Computer Vision, 120:153-168, 2016.
[51] Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka. Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. In Proceedings of the IEEE/CVE winter conference on applications of computer vision, pages 2149-2159, 2022.
[52] Riku Murai, Eric Dexheimer, and Andrew Davison. Mast3r-slam: Real-time dense slam with
3d reconstruction priors. arXiv preprint arXiv:2412. 2024.
14

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, we accurately claim our paper's contributions and scope in the abstract and introduction section. These claims are demonstrated in our methodology and experiments sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
ii Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We include the limitation in the supplementary materials.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
iii. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [NA]
This paper does not include theoretical results and only includes experimental results.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
iv. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We include all the information needed to reproduce the main experimental results in the method section, experiment section, and supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We will release all data and code after the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https:/ /ese:://niii.eee cc/
public/ guides/ CodeSubmii sssisssioon sswonpolicyyyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: á ps. cc/ cc/publi c/gui des / CCodeooouuaaa ss onPol (cy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
vi. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: We include the experimental details the main paper and the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
vii. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: This paper does not report error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
viii.
Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We include the computer resources used for this work in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
ix. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: /twuriuuuup..ereee/eee cc
Answer: [Yes]
Justification: This work conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provide the trend and potential challenged in the main text.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
xi. Safeguards
Question: Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
This paper does not have a high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
xii. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have cited the original owners of all assets that used in this paper properly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papersw rswi thoode. com / da sets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
19

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
xiii. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing and human subjects.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
xiv.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing and human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
xv. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

xvi. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The LLM only involves the paper writing, editing and formatting.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer our LLM policy (https: / ////rrrripppp cc/ Conferences / 2025/ LLM) for what should or should not described.
21

---

A Additional Examples of Clips-to-Clips
Example of Clips-to- Shiss-to-Clips. Top: Consecutive frames from the original training clip. Middle: Selected Keyframes with corresponding patch selections. Bottom: Synthetic video clips generated from keyframes using the Image-to-Clips process.
Figure 8:
We present additional examples of the Clips-to-Clips process in Figure 8, as supplement to Figure in the main text. Top row: These are consecutive frames extracted from raw training video. The sequence exhibits high temporal redundancy, neighboring frames share nearly identical viewpoints and contribute minimal new geometric information. Middle row: Keyframe selection. We highlight the patches that maximize scene coverage using black bounding boxes and indices. These selected keyframes ensure continuous, dual-sided visibility of the target structure, capturing the most informative perspectives within the clip. Bottom Augmented clip via Image-to Clips. Starting from the selected keyframes, the image-to-Clipp augmentation synthesizes new video sequences with greater baseline shifts, camera rotations, and scale variations. Despite these transformations, inter-frame overlap is preserved, ensuring geometric consistency. The resulting clips are significantly richer in structure and offer more diverse visual cues compared to the original sequence.
B Additional Experiments and Evaluations
B.1
Joint 2D-3D Data Augmentation
In addition to simulating camera motion, extend the standard PyTorch data augmentation pipeline to operate jointly on images, point clouds, and overlap masks, incorporating both affine and per- apply RandomA
augmentation with rotation angles sampled from the range [-45 45%), translations up to 20% of the image dimensions, and scaling factors between 0.8 and 1.0. We also adopt RandomPersper www.ccomeeereeeiiiveee... transformation with distortion scale of 0.1.
spective transformations. Specifically,
To maintain geometric plausibility and prevent excessive warping that could degrade 3D reconstruc- tion quality, the RandomP e erspect ivve transformation is applied with a low probability. The visual effeets of these augmentations shown in the bottom rows of Figure 8.
Importantly, these augmentations are applied only to paired images and their corresponding point clouds. We do not currently update the associated camera parameters to reflect the applied transformations, an omission that represents a potential direction for future work, particularly for enhancing geometric consistency in training.
22

---

Original sampling
odi
O
0.00%
0.13%
0.12%
3. B.B8
b.odk
e oodk
Overlapping matrix
Keyframes selection
Figure 9: From random sampling to coverage-awan key-frame selection. Top: arbitrary sampling yields spatial discontinuities. Middle: overlap matrix shows pair-wise coverage percentages. Bottom: key-frame selection (green boxes) picks frames that maximise coverage and continuity.
Method
Puzzles
Data
Acc +
Comp
Value
4 (%)
Value
4 (8)
1/10
0.0542
0.0325
Spann3R
1/10
0.0389
+28.2 2
0.0248
+23.7
Full
0.0388
0.0253
full
0.0330
+14.3
0.0224
+11.4
Table 6: Quantitative comparison on 7Scenes. For each metric, both the raw value and the relative improvement (A) achieved by incorporating Puzzles are reported. The checkmark indicates configu- rations trained with Puzzles.
23

---

B.2 Impact of Puzzles under Limited Data
We have supplemented the evaluation with an experiment where only 10% of the training data is used without Puzzles augmentation. The improvement shown in the Table 6 are based on the comparison under the same data size. Compared to the baseline using 10% of the data, applying Puzzles augmentation significantly improves reconstruction quality. Notably, under this limited data regime, the ACC and Comp errors are reduced by 28.2% and 23.7%, respectively. This demonstrates that Puzzles greatly improves data efficiency. While the original setting relies large-scale data to achieve diversity, Puzzles maximizes the utility of available data by generating more informative and varied training samples.
B.3 Keyframe Selection for Spatial Continuity
Existing video-based 3R-series methods often sample clips by selecting frames at arbitrary intervals. This random sampling strategy can result in spatially discontinuous sequences, where frames lack sufficient overlap for effective 3D reconstruction.
Figure 9 illustrates how our keyframe selection strategy addresses this issue. In the top row ("original sampling"), five frames are randomly selected from a training clip. However, the first two frames share almost common field of view with the remaining three, leading poor geometric continuity, The middle grid shows the pairwise geometric overlap between these frames: red percentages and purple marks indicate high overlap between the first two frames (approximately 80%), near-zero overlap between those and the final three, and moderate overlap among the last three (up to ~ 38%).
Using the overlapping matrix, our method identifies and discards the spatially disconnected first two frames, retaining only the final three as representative keyframes. These are shown in the botton row. The green rectangles indicate the image regions that will be cropped and warped during the subsequent Image-to- to-Cpppp augmentation. This ensures that the generated clip maintains sufficient scene coverage and smooth inter-frame continuity, both critical for learning effective 3D representations.
B.4 Translation and Rotation Augmentation
As shown in Figure 10, camera translation- only augmentation (TransAug) involves simple cropping-base -based patch generation (Figure 2.B, "generate patches"), while Trans+RolAug extends this by applying centroid- based camera rotation around each generated patch (the full "Augmen- tation" pipeline in Figure 2.D). As shown Fig- ure 10, translation-only augmentation (TransAug) involves generating image patches through sim- ple cropping (see main text Figure 2.B, "generate patches"). In contrast, translation piliatiiinn-lssotttttooo augmentation (Trans+RotAug) extends this ap- proach by applying centroid-bart based camera rota- tions around each patch, corresponding the full "Augmentation" pipeline shown in Figure 2.D of main text.
0.87
Transtug Trans-RotAug
0.05
0.84
0.02
Reconstruction accuracy (lower Acc. is better) for translation-only augmen- tation (TransAug) versus combined translation and rotation augmentation (Trans+RotAug)
Figure 10:
Quantitatively, combining translation and rotation yields the lowest mean reconstruction error of 0.0330, representing an improvement of approximately 35% over translation-only augmentation (TransAug. 0.0508). This substantial gain highlights the benefit of jointly perturbing both camera position and orientation. Not only does this increase the diversity of geometric viewpoints training data, but it also leads more robust and generalizable 3D reconstruction performance.
B.5 Impact of Puzzles on 3D Reconstruction
Figure presents a comparison reconstruction results from three methods, SLAM3R, Fast3R, and Spann3R, both before (middle) and after (right) applying Puzzles. In all cases, applying Puzzles leads to a notable improvement in reconstruction quality by enhancing completeness and reducing noise and holes. Specifically, SLAM3R benefits from the filling of boundary gaps, Fast3R recovers
24

---

SLAMER
SLAMER w: Puzziess
Fist3R
FastRR wi Puzzles
Spamn3R
Spann3R wi Puzcies
3R
Fast3 w/ Puzzles
Spanad1R
Spann3R Puzdes
Figure 11: Reconstruction results without (left) and with (right) the proposed Puzzles augmentation.
Setting
Acc 4
Comp
w/o. Aug. Cropping + Jittering + Flipping Puzzles
0.0395
0.0261
0.0388
0.0253
0.0330
0.0224
Table 7: Comparison different augmentation strategies.
finer details with fewer artifacts, and Spann3R more accurately reconstructs missing structures while producing smoother surfaces. Please refer the project website for a detailed visual comparison.
B.6 Comparing Puzzles with Traditional Augmentation
As show in Table 7, we compared the performance of Spann3R on the 7Scenes dataset under three settings: (I) no augmentation (w/o. Aug.), (2) traditional augmentation (cropping jittering + flipping), and (3) our proposed Puzzles. In the official implementation, traditional augmentation includes cropping, jittering, and flipping, which enhance appearance diversity at the image level. In contrast, Puzzles augments data from a spatial perspective by simulating camera motion and introducing diverse viewpoints. Our results show that Puzzles leads to more effective generalization, outperforming both the no-augmentation and traditional augmentation settings.
C Limitations and Potential Future Work
While Puzzles demonstrates strong performance gains in static scene reconstruction under the standard pinhole camera model, several limitations remain that present exciting avenues for future exploration:
Support for diverse camera models. Currently, Puzzles operates under the assumption of pinhole camera model. Extending support to alternative camera types, such as fisheye, panoramic, or catadioptric lenses, would enable augmentation for broader range of real-world capture devices and use cases, especially in robotics and immersive applications.
ii. Dynamic scene augmentation. The current framework assumes scene rigidity and does not account for object or human motion within frame. Future work could explore strategies for augmenting dynamic scenes, for example, by learning synthesize plausible motion trajectories or segmenting and animating scene components independently.
Cross-scene composition and stitching. Puzzles presently operates on single scenes or short clips. An extension to stitch patches or sub-clips across multiple distinct scenes could simulate transitions between rooms or outdoor environments, enriching training diversity for large-scale 3D reconstruction tasks. This could also help bridge domain gaps between datasets.
25

---

iv. Learning-guided augmentation. Currently, Puzzles applies geometric transformations via hand- designed procedures. Incorporating learning- baaiing-based augmentation policies. where the system automatically optimizes transformations to maximize reconstruction utility, could further improve performance and generalization.
Physics-aware realism. Enhancing the photometric and geometric fidelity of augmentations (e.g., simulating lighting changes, motion blur, or depth-dependent occlusions) could bring synthetic data even closer real-world conditions, leading more robust models challenging environments.
These directions suggest that Puzzles can evolve beyond a data augmentation strategy into a more general framework for synthetic 3D scene generation and learning. We view this work foundation toward that broader goal.
D Broader Impacts
The development of Puzzles introduces both promising opportunities and important considerations for the broader computer vision and 3D reconstruction communities.
Positive impacts. By significantly reducing the dependency large-scale posed video-depth datasets, Puzzles democratizes access to high-quality 3D reconstruction tools. Researchers and practitioners in snource-consttraied settings, such as those lacking access expensive sensors or extensive capture setups, can now generate effective training data from minimal inputs. This may accelerate progress in areas like robotics, AR/VR, digital heritage preservation, and autonomous navigation, particularly in domains where data collection is difficult or expensive (e.g., remote environments, cultural landmarks, or historical archives). The plug-am sang--nd-paay nature of Puzzles also encourages wider adoption across various architectures, facilitating rapid experimentation and innovation in data-efficient 3D learning.
Risks and ethical considerations. As with many advancements in generative data augmentation, the increased realism and diversity of synthetic training samples also raise potential concerns. For example, improved 3D reconstruction capabilities could be misused for surveillance, privacy-invasive applications, or the creation of deepfakes. While Puzzles itself does not introduce adversarial manipulation, lowers the barrier to creating convincing 3D models from limited imagery, which could be repurposed in harmful ways.
To mitigate these risks, encourage responsible deployment, including transparency around the of synthetic data, and advocate for integration with ethical guidelines when applying this method in sensitive domains. We also recommend further exploration of watermarking or provenance tracking techniques for synthetic data generated via Puzzles.
Environmental considerations. By enabling more data-efficient training pipelines, Puzzles may reduce the computational cost and environmental footprint associated with large-scale 3D model training. Training models only 10% of the original dataset, while achieving similar performance, can substantially cut energy usage, aligning with broader goals sustainable Al research.
26


Reviews:
[{"review_id": "4coNwnesqX", "reviewer": "Reviewer_tQex", "summary": "The paper proposes a data augmentation scheme that can generate novel video clips of posed RGBD images, given a single RGBD image, or a posed video of such images.\nThis kind of data generation is motivated by the need of current end-to-end dense 3D resonstruction methods in large amounts of labeled data. Such data is relatively scarce and dedicated augmentation schemes are not available. \nThe solution is based on sampling a sequence of overlapping patches, within a given input image, which are later augmented in both 2D and 3D to create a synthetic sequence of images that simulate coherent and challenging camera motion in the scene. \nThe benefits of the method are demonstrated by showing accross the board improvements of the several relevant '3R' schemes after training with the additional generated data.", "questions": "1) Please explain in more detail how the augmentation scheme is incorporated at training.\n2) Regarding image-to-clip: \n    - How are the patches actually selected? \n    - In patch calibration: \n         a. Why do you allow changing either intrinsics or extrinsics, but not both at the same time? \n         b. When using fixed-intrinsics-varying-extrinsics, Doesn't it incur only change in translation? (You say it doesn't create *realistic* rotation dynamics)\n         c. Why is RanSAC needed in PnP? What are the outliers?\n    - In camera motion augmentation, there are the valid view coverage checks, but it seems that you do no enforce overlap between consecutive frames. For example, in bottom row of Figure 3, samples 2 and 3 seem to have no overlap.\n    - You create occluded regions, due to camera motion, which you color white and do provide depth. What is the extent of this and does it have impact on the methods that are trained? Should you perhaps have considered image/depth inpainting?\n3) Regarding clip-to-clip: You decide to subsample the clip images and then apply image-to-clip on each. Wouldn't you get more general solutions by treating the whole sequence as an input and sampling a sequence of patches from different frames? (i.e. two overlapping patches are more realistic if not sampled from the same frame, but say from consecutive frames)\n4) Table 1: Are your results using clip-to-clip or image-to-clip? (And same question regarding Figure 7 B and C).", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n-------------\n1) I find the main bottom-line results very convincing. Especially Table 1, that reports for the combination of 3 different leading methods (Spann3R, Fast3R and SLAM3R) over 3 main datasest (7Scenes, NRGBD and DTU), showing consistent improvements when adopting Puzzles (10%-20% gains in accuracy and completion metrics) and comparable results when using (augmenting) only 10% of the original data.\n2) The idea of working on a sequence of crops in order to simulate a video sequence is very appealing. It can be used both to generate a huge number of clips, with great controll on variety and diffuculty of the generated data, in terms of overlap camera pose dynamics and scene coverage.\n3) The extension from single image input to using clips is challenging, but the suggested procedure is able to extract longer and more complex clips from data that contains lots of overlap. In some sense, it is able to extract good summarizations of this highly redundant data.\n\nWeaknesses:\n-----------------\n1) The background motivation is a rather limited and it is not entirely clear what the target function is. I would expect to see a more detailed discussion regarding how the mentioned '3R' methods process data - What the requirements are, and What of those is missing in current datasets. \n2) This would clarify the design choices of the augmentation method. Currenly, there is a general notion of the need to produce more data, with very vague requirements like being multi-view and temporally consistent, improving spatial coverage, none which are defined or quantified in any way. Therefore, it is hard to tell how much the specific design choices are optimal. Typically, when proposing a new data-set or even such an augmentation scheme, it is beneficial to present statistics of the data, before and after, to show how the new data is more diverse, complete, balanced, challenging, etc'.\n3) Desing choices are very much heuristic. I wonder whether it would be possible to make them in a more general way, trying to optimize over some specific functional that captures the desired properties of the clip.  \n4) Experimentation - It's not clear to me at which extent the scheme is used in the training. In terms of the amount of generated data, whether it is used instead of along with the original data, whether it is done on the fly or in pre-processing and how it effects the training dynamics (e.g. number of iterations, convergence).", "was_revised": true, "final_justification": "I want to thank the authors for their comprehensive rebuttal in response to my review. Most of my concerns were properly addressed — especially those regarding clarifications and technical details. I would suggest updating the manuscript accordingly.\n\nMy cautious rating was mainly based on the impression that the solution is rather heuristic and lacks a more direct formulation of what video‑based 3D reconstruction pipelines actually require from their training data. Such an understanding, which is still lacking, could enable more rigorous solutions to this problem.\n\nNevertheless, I acknowledge that this situation is common to most works in this area. However, even if the general goals are to obtain more data, greater diversity, larger motion, etc., these should be better quantified and compared — before and after the augmentations. An important addition in this direction is the table introduced in response to Reviewer SXr5, which analyzes camera pose distributions.\n\nGiven this understanding (and the authors’ efforts to improve their experimentation in this area), as well as the fact that the method is highly practical with clear benefits in usage, I have raised my score and would not object to accepting this paper.", "rebuttal": "## Weakness:\n#### **Q1**. Lack of 1) background motivation, 2) clear target function and 3) detailed data processing.\n* **Answer**: 1) Data-driven feed-forward 3D reconstruction methods, such as VGGT, have shown great promise, with data quantity and quality of training data being key to their success. Existing feed-forward methods primarily rely on 2D data augmentation techniques such as cropping, clipping, and jittering, without incorporating any 3D-aware augmentation. In contrast, our method enhances viewpoint diversity by explicitly controlling camera motion, leading to a more diverse and balanced data distribution in 3D space. 2-3) The task involves taking multiple images (extracted from a video) as input, without requiring any camera parameters, and outputs a pointmap (point cloud) for each image. All pointmaps are expressed in the coordinate frame of the first image. The training data consists of video clips along with their corresponding pointmap. \n\n---\n\n####  **Q2**. Hard to tell how much the specific design choices are optimal, which lacks clear definition and quantitative analysis. \n* **Answer**: Directly evaluating the effectiveness of data augmentation is inherently challenging. Instead, we provide both an analysis of the augmented data distribution and its impact on  feature representations. On the ScanNet++ dataset, we observe that the original camera pose distribution is dense along the Yaw and Pitch axes but sparse along the Roll axis, due to the nature of indoor scanning trajectories. After applying Puzzles, the distribution becomes more balanced across all three axes. At the feature level, we extract representations using Spann3R models trained with and without Puzzles. Features obtained with Puzzles exhibit a more uniform distribution, indicating better coverage of the representation space. While we are unable to include figures in the rebuttal due to space constraints, we will provide detailed visualizations and expanded discussion in the main paper.\n\n---\n\n####  **Q3**. Design choices are heuristic, need a general way.\n* **Answer**: While our method involves heuristic components, similar to many existing data augmentation techniques, it is designed to be general and broadly applicable. Puzzles functions as a **plug-and-play** module that can be integrated with any feed-forward 3D reconstruction model, whether it operates on image pair or sequences. It requires **no modification to the network architecture** and can be seamlessly incorporated at the data loading stage to enhance viewpoint diversity.\nFurthermore, our method aligns well with current training paradigms. Many reconstruction models adopt **local-to-global training strategies**, where the early warm-up stages involve high data overlap, and later stages reduce overlap to promote broader scene coverage. Puzzles naturally supports this approach: its **Image-to-Clips module allows control over patch overlap**, allowing flexible adaptation of augmentation strength across training phases.\n\n---\n\n####  **Q4**. Lack of experimentation details including training scheme, training data, on the fly or pre-processing, training time, iteration and convergence. \n* **Answer**: Using Spann3R as an example, we follow the official training setup with the same 14M training samples (ScanNet-v2, ARKitScenes, Habitat subset, and BlendedMVS). The only change is the application of Puzzles augmentation. Puzzles is a plug-and-play module applied **on-the-fly** through the dataloader, without requiring any pre-processing or modification of the training pipeline. Training was conducted on 8×H100 GPUs for 120 epochs. The total training time was approximately 40 hours with Puzzles, compared to 36 hours without. We observed slightly slower convergence when using Puzzles, which is expected due to the increased diversity and complexity of the training data. While longer training could yield further improvements, we use the same configuration across experiments to ensure fair comparison.\n\n---\n\n## Questions\n####  **Q1**. More details about augmentation\n* **Answer**: Our augmentation method is designed to be easily integrated into any feed-forward 3D reconstruction training pipeline. For each training batch, augmentation is applied at the video clip level. Given a clip with N images and their corresponding point clouds, we first select key representative frames using an overlap-based clip-to-clip matching matrix. Then, we apply image-to-clip augmentation to synthesize new training samples. The augmented data retains the same format as the original—each clip contains of a sequence of images along with their corresponding pointmap. This consistency ensures compatibility with existing architectures and training workflows.\n\n---\n\n### 2. Image-to-Clips\n####  **Q2.1**. How are the patches actually selected?\n* **Answer**: To generate N patches from a single image, the first patch is generated randomly. Each subsequent patch is then created to ensure it overlaps with **at least one of the previously generated patches**, based on IoU calculations. This overlap constraint helps maintain spatial continuity and supports consistency across views. The patch size is determined dynamically, based on the image resolution and the desired number of patches per image.\n\n---\n\n#### **Q2.2**. In patch calibration: a. Why do you allow changing either intrinsics or extrinsics, but not both at the same time? b. When using fixed-intrinsics-varying-extrinsics, Doesn't it incur only change in translation? (You say it doesn't create realistic rotation dynamics) c. Why is RanSAC needed in PnP? What are the outliers? \n* **Answer**: \n* **a**: Varying both intrinsics and extrinsics simultaneously is theoretically possible, but it causes unnecessary computational overhead. Given that our training data consists only of images and their corresponding pointmap—without access to ground-truth camera parameters—we prioritize simplicity and robustness. A lightweight calibration strategy suffices to generate diverse and effective image–pointmap pairs.\n* **b**: Without augmentation, generating patches only simulates camera translation. As shown in Fig. 2C, all cameras views remain largely forward-facing, limiting rotational diversity. In contrast, our augmentation introduces varying extrinsics, which results in more realistic and diverse camera orientation, as illustrated in Fig. 2D.\n* **c**: The training data includes real-world samples collected via sensors and manually calibrated, which inevitably introduces noise and outliers. Using RANSAC PnP helps mitigate the effect of these errors. It helps filter out erroneous correspondences between 2D image features and 3D points that could otherwise degrade pose estimation.\n\n---\n\n#### **Q2.3**. No enforce overlap between consecutive frames.\n* **Answer**: Yes, in our augmentation strategy, each newly generated patch is required to overlap with at least one of the previously generated patches, but not necessarily with consecutive frames. This design choice accommodates scenarios with rapid camera motion, where adjacent frames may have little or not overlap.\n\n---\n\n#### **Q2.4**. The impact of filling occluded regions with white colour. \n\n| Setting                         | ACC ↓     | Comp ↓    | Training Time (Hours) |\n|---------------------------------|-----------|-----------|------------------------|\n| Spann3R (w/o. Inpainting - Default) | 0.0330    | 0.0224    | 40                     |\n| Spann3R (w/. Inpainting)        | 0.0332 (−0.6%) | 0.0221 (+1.3%) | 53 (**+32.5%**)            |\n| Δ Change                        | +0.0002   | −0.0003   | **+13 h**                  |\n|\n* **Answer**:\nWe also observed this issue. We applied LAMA [1] to inpaint occluded regions and excluded the inpainted areas from supervision. However, our experiments conducted on 8×H100 GPUs show that inpainting does not provide a significant performance improvement, while increasing training time by 32.5%. Additionally, existing inpainting methods tend to introduce artifacts when filling large missing regions, contaminating the valid data. Therefore, we decided to discard this approach.\n\nReference:[1] Resolution-robust Large Mask Inpainting with Fourier Convolutions \n\n---\n\n#### **Q3**. Clips-to-Clips. Render a new sequence on point cloud during training. \n* **Answer**: Pre-processing Hypersim data using this method is time-consuming—it takes about 3 seconds per clip and around 9 hours for the entire dataset. Therefore, re-rendering scenes on the fly during training is currently not feasible. That said, we agree it is a promising direction, especially for large-scale scene reconstruction (as mentioned in the supplementary), where stitching multiple scenes and generating expansive training data could offer significant benefits.\n\n---\n\n#### **Q4**. Table 1: Are your results using clip-to-clip or image-to-clip? (And same question regarding Figure 7 B and C).\n* **Answer**: Thanks for pointing out. The results in Table 1 and Figure 7 B are based on the clip-to-clip setting, while Figure 7C corresponds to the Image-to-Clip setting."}, {"review_id": "upBFlSZgog", "reviewer": "Reviewer_2KDP", "summary": "This paper introduces a data augmentation framework for 3D reconstruction that generates high-quality multi-view video-depth sequences from a single image or short video clip. It includes two components called Image-to-Clips and Clips-to-Clips, which enhance local and global view diversity, respectively. Experiments show that it can improve reconstruction accuracy even when models are trained on only 10 percent of the original dataset. Additionally, the method is plug and play and does not require changes to existing network architectures.", "questions": "Does the use of PnP on small, local depth patches introduce noisy training data due to inaccurate pose estimates? Since image features within a patch are often similar, pose estimation can become unstable. Does such noise propagate into the final 3D reconstruction or affect model performance?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1) The paper introduces a novel two-part data augmentation pipeline that directly addresses the data scarcity problem in 3D reconstruction. The idea is simple yet well-motivated, and the subsequent methodology is theoretically sound and technically solid.\n\n2) It requires no changes to network architectures and can be seamlessly integrated into various state-of-the-art 3D reconstruction pipelines such as Spann3R, SLAM3R, and Fast3R.\n\n3) The paper evaluates the method across multiple benchmarks (7Scenes, NRGBD, DTU) and models, under various data regimes, demonstrating consistent and robust improvements.\n\nWeaknesses\n\n1) The augmentation pipeline consists of several components,such as different patch calibration strategies, camera rotation augmentation, and view validation check, but their individual contributions are not isolated. An ablation study would clarify which components are most impactful and whether all are necessary.\n\n2) While the paper highlights that Puzzles enables models trained on only 10% of the data to match or exceed full-data performance, it does not include a baseline where the same 10% is used without Puzzles.", "was_revised": true, "final_justification": "After reading the rebuttal and discussions, I find that most of my concerns have been adequately addressed. I will therefore keep my original score.", "rebuttal": "## Weakness\n####  **Q1**. An ablation study clarify which component are more important\n\n| I2C w/o. aug. | I2C w/. aug. | C2C | Performance ACC ↓ (7 Scenes) |\n|:-------------:|:------------:|:---:|:-----------------------------:|\n| ✓             |              |     | 0.1278                        |\n| ✓             | ✓            |     | 0.0916                        |\n| ✓             | ✓            | ✓   | **0.0330**                        |\n|\n* **Answer:**\nWe further explore the ablation study of Figure 7.A to illustrate the impact of each component. The experiments show that Clips-to-Clips (C2C) bring the most significant performance gains, while Image-to-Clips (I2C) with augmentation (w/. aug.) provides a smaller but still positive improvement. In scene-level reconstruction, I2C with augmentation helps simulate camera motion, enabling the network to capture local details. In contrast, C2C improves robustness and supports large-scale reconstruction by capturing global context. \n\n---\n\n#### **Q2**.  Experiment include 10% data without Puzzles is required. \n| Method               | Data      | ACC ↓         | Comp ↓         |\n|----------------------|-----------|----------------|----------------|\n| Spann3R w/o. Puzzles | 1/10      | 0.0542         | 0.0325         |\n| Spann3R w/. Puzzles  | 1/10      | 0.0389 (+28.2%)| 0.0248 (+23.7%)|\n|\n| Spann3R w/o. Puzzles | full      | 0.0388         | 0.0253         |\n| Spann3R w/. Puzzles  | full      | 0.0330 (+14.9%)| 0.0224 (+11.4%)|\nQuantitative comparison on 7Scenes.\n* **Answer:**\nWe have supplemented the evaluation with an experiment where only 10% of the training  data is used without Puzzles augmentation. The improvement shown in the above table are based on the comparison under the same data size. Compared to the baseline using 10% of the data, applying Puzzles augmentation significantly improves reconstruction quality. Notably, under this limited data regime, the ACC and Comp errors are reduced by 28.2% and 23.7%, respectively. This demonstrates that Puzzles greatly improves data efficiency. While the original setting relies on large-scale data to achieve diversity, Puzzles maximizes the utility of available data by generating more informative and varied training samples.\n\n---\n\n## Questions\n####  **Q1**. Does the inaccurate pose estimation have an impact of the model performance.\n* **Answer:**\nWe acknowledge this issue, the calibration process is not perfect and can introduce pose inaccuracies. For augmented patches that rely on calibration, we re-render the image using the updated camera intrinsics and extrinsics to align it with the existing pointmap. This step helps correct mismatches between the image and pointmap caused by imperfect calibration, thereby mitigating the impact of inaccurate pose estimation on model performance."}, {"review_id": "AhyLfpuOT5", "reviewer": "Reviewer_7DC6", "summary": "This paper introduces a new data augmentation approach for DUSt3R-based multiview 3D reconstruction methods. Given a single view or short clips of a static scene with RGBD data, this augmentation strategy samples new views from the data via patched cropping and camera view pose augmentation to obtain a series of new views with different intrinsics or extrinsics. As 3R-based methods predict point maps aligned to the first frame coordinates, the point maps sampled in new views are naturally aligned and obtained from the original frame.\n\nThis data augmentation strategy partially addresses the constraints faced by 3R-based methods that require large-scale data and diverse viewpoints. It efficiently expands limited training data to much larger scales while providing more diverse camera intrinsics and extrinsics. The method achieves competitive performance compared to the same approach trained on only 10% of the annotated data.\n\nThe paper describes the data augmentation for image-to-clips data augmentation strategies with 2 options: fixed extrinsics augment intrinsics, and fixed intrinsics augment extrinsics. All via simple homogeneous coordinates augmentation, transformation and optimizations via linear transformations and optimizations with reprojection errors given GT 3D scene points.\n\nAs for clips-to-clips augmentations, an overlapping threshold filtering algorithm is applied to select less redundant keyframes. The same augmentation strategies as image-to-clips are then followed for the keyframes.\n\nExperiments demonstrate that the augmentation pipeline consistently improves the performance of Spann3R, Fast3R, and SLAM3R in scene and object multiview reconstruction tasks without requiring any network architecture modifications. Furthermore, this augmentation approach achieves competitive performance compared to the original models trained on the full dataset while using only a fraction of the training data. These results suggest that the augmentation strategy can effectively address data-limited training scenarios.", "questions": "1. Is there a statistical analysis of the occlusion areas after applying the augmentation to quantify the extent of these artifacts?\n2. How does this augmentation approach compare to traditional computer vision augmentation techniques (cropping, rotation, resizing, color jittering, etc.)?\n3. How do the training time and convergence rate compare between experiments with and without data augmentation? Are the experiments conducted with the same number of epochs?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n1. The data augmentation pipeline is simple, efficient yet effective.\n2. Addresses the data scarcity problem by expanding limited training data to much larger scales without requiring additional annotations.\n3. The approach is data-centric imporves perfromance on prior work without requiring any modifications to the underlying network architecture.\n4. Achieves competitive results compared to models trained on full datasets while using only a fraction of the training data (10% mentioned).\n\n### Weaknesses\n\n1. New views with different intrinsics and extrinsics will introduce artifacts in invisible areas, but may have less effect on the performance of reconstruction tasks.\n2. No comparison of this data augmentation to experiments with no data augmentation at all, more traditional data augmentation techniques (cropping, jittering, image rotation, etc.), and this data augmentation to provide a more comprehensive understanding of how important the perspective projection augmentation is for the reconstruction task.\n3. It is unclear whether the model performance comparison with and without data augmentation uses the same number of training epochs, and the impact of data augmentation on training efficiency and convergence rate.", "was_revised": true, "final_justification": "The authors have addressed my concerns regarding image occlusion introduced by the data augmentation pipeline and its impact on the reconstruction task. They also provided helpful comparisons to other augmentation strategies and clarified details related to baseline comparisons. I believe the proposed simple yet effective improvements in data augmentation are important and could play a significant role in training data-centric multiview reconstruction models, especially when annotations are costly. Therefore, it is an accept for me.", "rebuttal": "## Weakness and Questions\n#### **Q1**. New views with different intrinsics and extrinsics will introduce artifacts in invisible areas, but may have less effect on the performance of reconstruction tasks.\n| Setting                         | ACC ↓     | Comp ↓    | Training Time (Hours) |\n|---------------------------------|-----------|-----------|------------------------|\n| Spann3R (w/o. Inpainting - Default) | 0.0330    | 0.0224    | 40                     |\n| Spann3R (w/. Inpainting)        | 0.0332 (−0.6%) | 0.0221 (+1.3%) | 53 (**+32.5%**)            |\n| Δ Change                        | +0.0002   | −0.0003   | **+13 h**                  |\n|\n* **Answer**:\nDuring the method design stage, we observed that camera augmentation can introduce occlusion-induced holes in the projected views, especially in newly visible areas. To address this, we experimented with inpainting to fill these regions. However, our results showed no significant performance improvement, while training time increased by 32.5%. This suggests that such artifacts have limited impact on reconstruction performance, and the cost of mitigating them outweighs the benefits.\n\n---\n\n#### **Q2**. No comparison of this data augmentation to experiments with no data augmentation at all, more traditional data augmentation techniques (cropping, jittering, image rotation, etc.), and this data augmentation to provide a more comprehensive understanding of how important the perspective projection augmentation is for the reconstruction task.\n| Setting                        | ACC ↓  | Comp ↓  |\n|--------------------------------|--------|---------|\n| No augmentation                | 0.0395 | 0.0261  |\n| Cropping + Jittering + Flipping| 0.0388 | 0.0253  |\n| Puzzles                        | **0.0330** | **0.0224**  |\n|\n* **Answer**:\nWe compared the performance of Spann3R on the 7Scenes dataset under three settings: (1) no augmentation, (2) traditional augmentation (cropping + jittering + flipping), and (3) our proposed Puzzles. In the official implementation, traditional augmentation includes cropping, jittering, and flipping, which enhance appearance diversity at the image level. In contrast, Puzzles augments data from a spatial perspective by simulating camera motion and introducing diverse viewpoints.\nOur results show that Puzzles leads to more effective generalization, outperforming both the no-augmentation and traditional augmentation settings.\n\n---\n\n#### **Q3**.It is unclear whether the model performance comparison with and without data augmentation uses the same number of training epochs, and the impact of data augmentation on training efficiency and convergence rate. \n* **Answer**:\nUsing Spann3R as an example, we follow the official training setup and train for 120 epochs on the same dataset. On 8×H100 GPUs, training takes 40 hours with Puzzles and 36 hours without. We observe slower convergence when using Puzzles, likely due to increased data diversity and complexity it introduces. While longer training could improve performance, we use the same settings for fair comparison."}, {"review_id": "Faqm4bspkq", "reviewer": "Reviewer_SXr5", "summary": "This paper introduces Puzzles, a data augmentation strategy designed to enhance the performance of multi-view 3D reconstruction by synthesizing video-depth sequences from single images or short video clips. Puzzles simulates realistic scene geometry and diverse camera trajectories through targeted patch-based transformations, significantly enriching the diversity of training data. Furthermore, Puzzles is a plug-and-play framework that can be seamlessly integrated into existing 3D reconstruction pipelines without requiring modifications to network architectures. Experimental results demonstrate its effectiveness, showing that models trained on just 10% of the original data augmented with Puzzles can achieve or even surpass the accuracy of full-data baselines. This data-centric approach provides a scalable solution for improving existing 3D reconstruction models.", "questions": "See weakness", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. This paper is well-written and easy to follow.\n\n2. The results of the proposed method are promising. It effectively scales the original dataset during training, enabling more efficient use of existing data.\n\n3. The proposed data augmentation method is flexible and can be easily applied to different models.\n\nWeaknesses\n\n1. When using monocular depth maps from MoGe, how is depth consistency ensured between video frames or keyframes? Inconsistent depth could result in poor supervision labels.\n\n2. I am concerned about whether the proposed method can be applied to videos with lower resolutions, e.g., 512x384. The resized patches may be significantly noisier, which could harm training performance.\n\n3. Keyframe selection is commonly used in video-based tasks. It is unclear whether the proposed keyframe selection method performs better than those used in existing approaches, e.g., NICE-SLAM.\n\n4. How efficient is the proposed method for data augmentation? It would be helpful to include an analysis of the time consumption for each step.\n\n5. I noticed that the performance of the selected baselines is much worse than that of recent SOTA models such as VGGT. It would be interesting to explore whether the proposed method is effective when applied to more recent SOTA models.\n\n6. More patch sampling strategies should be discussed and compared in the ablation study. For example, uniformly sampling patches with a fixed overlap ratio or randomly sampling patches, even without overlapping, could be evaluated.", "was_revised": true, "final_justification": "The proposed data augmentation method, Puzzles, effectively improves the performance of *3R methods with even 10% data. Puzzles is flexible and can be easily applied to different models. Results in the rebuttal addressed most of my concerns. The analysis of camera pose distribution during training also shows a deep understanding of the proposed method making it more insightful.", "rebuttal": "## Weakness and Questions\n#### **Q1**. When using monocular depth maps from MoGe, how is depth consistency ensured between video frames or keyframes? Inconsistent depth could result in poor supervision labels.\n\n* **Answer**: \nIn the Image-to-Clip experiment, we test two variants (as shown in Fig. 7C): one uses GT depth, and the other uses predicted depth from MoGe to extract geometry from a single image, which is then used to generate clips. In the Clips-to-Clips setting, to ensure depth consistency, we only use GT depth for augmentation and do not include predicted depth.\n\n---\n\n#### **Q2**. I am concerned about whether the proposed method can be applied to videos with lower resolutions, e.g., 512x384. The resized patches may be significantly noisier, which could harm training performance.\n\n* **Answer**: \nYes, applying Puzzles to low-resolution images (e.g., 512×384) may degrade patch quality, which could impact training performance. In our experiments, we use commonly available resolutions such as 1280×720 and 720×1080, which are sufficient to generate high-quality patches and clips. Examples of the resulting outputs can be seen in Fig. 3, demonstrating the visual fidelity preserved at these resolutions.\n\n---\n\n#### **Q3**. Keyframe selection is commonly used in video-based tasks. It is unclear whether the proposed keyframe selection method performs better than those used in existing approaches, e.g., NICE-SLAM. \n\n* **Answer**: \nThank you for the suggestion. We will include a comparison and discussion of SLAM-based keyframe selection methods (e.g., NICE-SLAM) in the main paper. While both approaches involve keyframe selection, their objectives and mechanisms differ fundamentally. In SLAM systems like NICE-SLAM, keyframes are selected to guide online optimization—typically focusing on frames that contribute to pose and geometry refinement through reprojection. In contrast, our method operates on pre-calibrated training data, selecting representative frames based on inter-frame similarity to remove redundancy and ensure viewpoint diversity during data augmentation. Due to these fundamental differences, it is difficult to integrate the two approaches for a fair comparison.\n\n---\n\n#### **Q4**. How efficient is the proposed method for data augmentation? It would be helpful to include an analysis of the time consumption for each step.\n* **Answer**:\nUsing Spann3R as an example, we follow the official training setup and train for 120 epochs on the same dataset. On 8×H100 GPUs, training takes 40 hours with Puzzles and 36 hours without. While integrating Puzzles requires longer training time, the additional 10% is a reasonable and acceptable cost given the performance gains it brings.\n\n---\n\n#### **Q5**. I noticed that the performance of the selected baselines is much worse than that of recent SOTA models such as VGGT. It would be interesting to explore whether the proposed method is effective when applied to more recent SOTA models.\n| Method            | ACC ↓          | Comp ↓         |\n|-------------------|----------------|----------------|\n| VGGT w/o. Puzzles | 0.0178         | 0.0118         |\n| VGGT w/. Puzzles  | 0.0148 (+16.8%)| 0.0101 (+14.4%)|\nQuantitative comparison on 7Scenes.\t\n* **Answer**:\nVGGT was released in May, around the same time as the NeurIPS deadline (May 15), making it a concurrent work. Due to time and data constraints, we fine-tuned VGGT using the settings from our paper to compare its performance with and without Puzzles augmentation. As shown in the table, on the 7Scenes dataset, Puzzles brings notable improvements—16.8% in ACC and 14.4% in Comp. Like the methods in our paper, Puzzles is compatible with any existing feed-forward 3D reconstruction framework, requiring no changes to the network or training strategy.\n\n---\n\n#### **Q6**. More patch sampling strategies should be discussed and compared in the ablation study. For example, uniformly sampling patches with a fixed overlap ratio or randomly sampling patches, even without overlapping, could be evaluated.\n| Setting (Baseline: Spann3R)               | ACC ↓  | Comp ↓  |\n|-------------------------------------------|--------|---------|\n| w/o. Puzzles                              | 0.0388 | 0.0253  |\n| w/. Puzzles – Uniformly sampling          | 0.0378 | 0.0248  |\n| w/. Puzzles – Randomly sampling (Default) | 0.0330 | 0.0224  |\n|\n* **Answer**:\nWe compared different patch sampling strategies in Puzzles. Using a uniform patch sampling method—where each image is evenly sampled with fixed-size patches and overlap—we found it still outperforms the no-Puzzles baseline. However, our proposed random patch sampling strategy achieves the best results. By generating patches of varying sizes, it better simulates diverse camera motions and captures more scene details, leading to improved model performance."}]

Author rebuttal:

Meta-review: This paper received unanimously positive reviews after the rebuttal (1 accept and 3 borderline accepts).

All of the reviewers recognize the core contribution of this paper, the proposed data augmentation scheme, being simple, well-motivated, and effective. It can serve as a plug-play module into Dust3R variants without modifications of the model architecture and training objectives.  The strong empirical results are also recognized by the reviewers.

There were concerns raised after the initial reviewing stage. Notably, Reviewer tQex questioned the heuristic nature of the design choices and requested more details on how the augmentation scheme is used in the training. Reviewer 2KDP pointed out the need for a baseline using 10% of the data without Puzzles and requested a more detailed ablation study. Reviewer 7DC6 asked for a comparison against traditional data augmentation techniques, while Reviewer SXr5 raised questions about the method's efficiency and applicability to newer SOTA models. The authors addressed these comments in the rebuttal, including providing new experimental results of ablation studies and integrating the proposed data augmentation scheme into VGGT.

Although minor concerns still remain, mainly about the heuristic nature of the proposed data augmentation scheme and lack of systematic understanding of its effectiveness, the AC found that the proposed approach is solid, tackles an important problem, and may inspire future research in this direction.

The authors are highly recommended to incorporate the results reported in the rebuttal into the final revision.
Decision: Accept (poster)