Title: Visual Sync: Multi‑Camera Synchronization via Cross‑View Object Motion
Venue: NeurIPS 2025
Accepted: yes
Keywords: camera synchronization

video motion analysis

scene understanding
Abstract: Today, people can easily record memorable moments, ranging from concerts, sports events, lectures, family gatherings, and birthday parties with multiple consumer cameras. However, synchronizing these cross‑camera streams remains challenging. Existing methods assume controlled settings, specific targets, manual correction, or costly hardware. 
We present VisualSync, an optimization framework based on multi‑view dynamics that aligns unposed, unsynchronized videos at millisecond accuracy. Our key insight is that any moving 3D point, when co‑visible in two cameras, obeys epipolar constraints once properly synchronized. To exploit this, VisualSync leverages off‑the‑shelf 3D reconstruction, feature matching, and dense tracking to extract tracklets, relative poses, and cross‑view correspondences. It then jointly minimizes the epipolar error to estimate each camera’s time offset. Experiments on four diverse, challenging datasets show that VisualSync outperforms baseline methods, achieving an average synchronization error below 130 ms.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents VisualSync, a training-free framework for synchronizing unsynchronized videos captured from multiple cameras. It formulates synchronization as an epipolar-based optimization problem and leverages pretrained visual foundation models to construct a robust three-stage optimization pipeline. Experimental results demonstrate superior performance across four diverse datasets, consistently outperforming strong baselines including SyncNeRF, Uni4D, and Mast3R.\n\nSTRENGTHS AND WEAKNESSES:\nPaper Strengths\n1. Novel Insight.  The idea of leveraging epipolar constraints to achieve video synchronization is new to me.\n2. Complete Systematic Design. A training-free, well-designed three-stage inference pipeline, including visual cue extraction, estimating pairwise offsets, and global offset estimation, is proposed. \n3. Extensive Experiments. The method is evaluated on four diverse datasets (CMU Panoptic, EgoHumans, UDBD, and 3D-POP) with detailed ablations. \n\n\nWeaknesses\n1. Heavy reliance on external models:\nThe proposed pipeline depends on several pretrained models, including GPT, Grounded-SAM, DEVA, CoTracker3, Mast3R, and VGGT, to extract visual cues. This makes the system overly reliant on the robustness of these components and also leads to a too complex, inefficient pipeline. The performance is particularly sensitive to VGGT’s pose estimation quality, which is not always reliable, especially under challenging viewpoints or motion blur.\n\n2. Computational cost and scalability:\nAs noted in Lines 248–250, components like CoTracker3 and Mast3R are already computationally expensive. Combined with the method’s O(N²) pairwise synchronization strategy, this severely limits scalability, especially for latency-sensitive or large-scale applications. Furthermore, the paper lacks a runtime or efficiency comparison with existing baselines or competitors, which is important for assessing practical usability.\n\n3. Lack of failure case analysis:\nThe paper briefly mentions potential failure cases (e.g., fast camera motion, small dynamic objects), but does not provide qualitative examples or a detailed discussion. Including visual or diagnostic results for such scenarios would help clarify the method’s limitations and robustness boundaries.\n\nQUESTIONS:\nPlease see the weaknesses section, especially the heavy reliance on the other models and the efficiency.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-28"}

comments: SUMMARY:
The paper presents a novel framework for synchronizing unsynchronized multi-camera videos capturing dynamic scenes. It addresses a common challenge in everyday scenarios—such as sports events, concerts, or family gatherings—where videos from multiple handheld devices lack time alignment. By leveraging cross-view object motion, the method recovers globally consistent temporal offsets with millisecond accuracy, enabling applications like 4D scene reconstruction and novel-view synthesis.

STRENGTHS AND WEAKNESSES:
Strengths:
1. By integrating advanced models (e.g., CoTracker3 for trajectory tracking and MAST3R for cross-view matching), the method avoids training from scratch, thereby improving efficiency (Section 3.3, Stage 0).
2. The method significantly outperforms baselines on Egohumans (δ_med=46.6ms) and CMU Panoptic (δ_med=41.5ms), particularly in complex scenarios.

Weaknesses:

The core idea — that synchronized frames of dynamic points should obey epipolar constraints is simple, which seems like more of an effective integration of recent tools (e.g., VGGT, CoTracker, MAST3R). 

1. The method assumes a fixed frame rate, but in real-world videos, inconsistent frame rates may limit its applicability (Limitations). It is recommended to extend the approach to a variable frame rate model.
2. It is recommended to add comparisons of resource efficiency between methods.

3. Expand the comparison to include PoseSync[59] (human pose priors) and audio synchronization[46], strengthening the validation of the 'assumption-free' approach.

4. There shall be many camera synchronization methods in the literature, including traditional and deep learning methods. The study and comparison seem not enough.

QUESTIONS:
See in Weaknesses.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a 3D geometry-based method for synchronizing multi-view videos of the same scene that have different starting recording times. It leverages off-the-shelf techniques to acquire camera poses, cross-view correspondences, and per-view 2D point tracks. The authors develop an optimization method that uses epipolar constraints to optimize the time offset for each video. The paper evaluates its method on various multi-view datasets, demonstrating its outperformance over baseline methods. However, the proposed method has several limitations: it cannot handle videos with different FPS, assumes ideal camera poses, and its optimization can be hindered by outlier correspondence matches. Overall, despite addressing a practical problem, the proposed solution still has significant limitations.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- This paper tackles a practical problem relevant to real-world multi-view video applications.\n- The authors perform a thorough evaluation across diverse multi-view datasets and include comprehensive ablation studies on the algorithm's design.\n- The overall presentation is clear and well-structured.\n\n**Weaknesses**\n- High sensitivity to input quality: The method's performance heavily depends on the accuracy of upstream components, specifically camera pose estimations, 2D point tracks, and cross-view correspondences. In Table 3, utilizing ideal ground truth data for poses and correspondences yields significantly higher accuracy (e.g., 94.8 vs. 40.2 in the metric AUC@100). A major limitation is the method's inability to refine or mitigate errors from imperfect estimations and outliers in these inputs, which inherently degrades the final synchronization results.\n- Insufficient quantitative performance: the quantitative evaluation reveals significant synchronization errors, averaging around 100ms. This margin remains substantial (e.g., ~3 frames at 30 FPS) and raises concerns about the method's practical utility for precise synchronization.\n- Limited frame rate handling: The method presented in the main paper assumes all input videos share an identical frame rate (FPS). While the supplementary material briefly demonstrates an extension for different frame rates, this crucial aspect lacks a thorough experimental evaluation.\n- Minor writing issue:\n  - Table 2 caption. \"we show mean and median error\", while this table shows AUC instead of errors.\n\nQUESTIONS:\n- The optimization operates on 2D point tracks at discrete timestamps. While a global offset estimation may approximate a continuous-time solution, it's unclear if this leads to sub-optimal results. If the input videos have low framerates (i.e, the point track observations become more discrete), will the optimization results significantly degrade?\n\n- In the experiments, many videos have stationary cameras rather than moving cameras. Would the stationary camera be a better anchor for synchronization? Would the method perform worse with a set of videos where all cameras are in motion?\n\n- The paper uses a neural radience field method (K-Planes) as a proxy to demonstrate the effectiveness of synchronization. However, K-Planes is not a state-of-the-art method. Is there any reason not to use more recent methods or explicit representation like dynamic 3D Gaussian splatting?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper presents a method for time-synchronizing multiple cameras by observing moving 3D points and ensuring they follow the epipolar constraints.   It achieves an error below 130ms.   It uses the following set of tools: camera parameters via VGGT, dense correspondences with CoTracker3, cross-view matches with MAST3R, and dynamic objects with DEVA.  The loss function, the Sampson error, is related to the squared Euclidean distance from a point to its corresponding epipolar line.  

The key challenge is that each of these measurements is noisy, and the total loss function across all camera pairs and trajectories is highly nonlinear.  The method employs an exhaustive search of time alignment to find the optimal time alignment and calibration between two cameras. 
This is followed by iteratively reweighted least squares for the global optimization.

STRENGTHS AND WEAKNESSES:
The algorithm is more effective than all existing strong baseline methods, including both geometric and learning-based approaches.   On the challenging Egohuman dataset, it reduced the time synchronization error by a factor of 4. 

It also listed the weakness of the paper itself:
a) it depends on accurate camera poses;
b) it fails when there is rapid motion or large viewpoint changes
c) it does not handle frame rate variations.

QUESTIONS:
None. 

The authors have listed their weaknesses.  Addressing them is beyond the scope of this paper.   However, it is unclear whether NeurIPS is the proper venue for this work.

LIMITATIONS:
none.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-04