# End-to-End Online Video Stitching and Stabilization Method Based on Unsupervised Deep Learning
## 2 Methodology
### Stitching Trajectory Estimation  
The stitching trajectory estimation module aims to determine the relative motion between frames captured by different cameras at the same timestamp, enabling precise stitching of video frames across cameras. This module first estimates the initial motion by computing the local motion field (meshflow) between the reference and target frames at the same moment. Subsequently, the previously obtained camera trajectory information is integrated to refine the local motion, generating the final stitching trajectory. The computation process is illustrated in Figure 2.  
In the first stage, due to the potentially limited overlapping regions between frames from different cameras, direct local feature matching often fails to provide stable motion estimation. To address this, our method adopts a network structure similar to the camera trajectory estimation module, while incorporating a context-aware module at a larger feature scale to enhance long-range matching capability between images. Specifically, for the same timestamp \(t\), let \(I_{t,ref}\) and \(I_{t,tar}\) denote the reference and target frames, respectively. The local motion field in the reference frame coordinate system can be expressed as  
Figure 2: Computation of meshflow between the reference and target frames.  
\[MV_{i,j}^{t}=stitch(I_{t,ref},I_{t,tar};i,j) \tag{9}\]  
where \(MV_{i,j}^{t}\) represents the motion vector at the \(i\)-th row and \(j\)-th column of the mesh grid. This design captures long-range correlations between images, mitigating the issue of motion information loss caused by insufficient overlap between frames.  
During stitching trajectory generation, the reference frame's motion trajectory has already been computed via the camera trajectory estimation module and serves as a global baseline. The motion of the target frame, however, is determined by combining its temporal continuity with the local motion information obtained from the stitch meshflow in the spatial domain. Defining \(MV_{i,j}^{ref,temporal}\) and \(MV_{i,j}^{ref,spatial}\) as the local motion vectors obtained from temporal sequence analysis and spatial matching, respectively, the overall motion of the target frame in the reference frame coordinate system can be expressed as  
\[MV_{i,j}^{ref}=MV_{i,j}^{ref,temporal}+MV_{i,j}^{ref,spatial} \tag{10}\]  
where \(MV_{i,j}^{ref,temporal}\) captures the temporal motion trend of the target frame, while \(MV_{i,j}^{ref,spatial}\) supplements the spatial motion information that may be difficult to capture due to insufficient frame overlap.  
Following the approach used for constructing camera trajectories, the stitching trajectory of the target frame is computed based on the camera trajectory \(C_{target}(t)\) of the reference frame and the estimated homography matrix \(H_{stitch}(t)\) for stitching, as follows:  
\[C_{stitch}(t)=C_{target}(t)\cdot H_{stitch}(t) \tag{11}\]  
where \(C_{stitch}(t)\) represents the position of the target frame at timestamp \(t\) in the reference frame coordinate system. This formulation ensures that the final stitching trajectory not only inherits the global motion information of the reference frame but also incorporates fine-grained corrections from local motion estimation, leading to more precise stitching results.  
The proposed stitching trajectory estimation method enhances long-range image matching by introducing contextual correlation mechanisms at a large feature scale. Additionally, by integrating both temporal and spatial motion information, this approach accurately determines the target frame's position in the reference coordinate system. It demonstrates high robustness against challenges such as insufficient overlap regions and motion discontinuities, laying a solid foundation for subsequent video stitching and stabilization.