# End-to-End Online Video Stitching and Stabilization Method Based on Unsupervised Deep Learning
## 2 Methodology  
### Camera Trajectory Estimation  
To address motion estimation issues in handheld videos caused by jitter and dynamic foreground interference, this paper proposes an unsupervised end-to-end method based on grid motion fields. This method directly estimates local motion information from consecutive frames and constructs the overall camera trajectory. Unlike traditional feature point matching-based approaches, this method leverages deep neural networks to model motion at the grid level, enabling finer capture of both local and global motion while enhancing robustness against non-ideal motion disturbances. The overall network architecture is illustrated in Figure 1.  
Figure 1 presents the overall architecture of the proposed unsupervised end-to-end video stabilization and stitching network, which is composed of three key modules: camera trajectory estimation, stitching trajectory estimation, and trajectory smoothing. The camera trajectory estimation module analyzes the sparse motion field (meshflow) between adjacent frames to estimate the global transformation induced by camera motion. This provides a foundational motion representation for downstream processing. The stitching trajectory estimation module refines alignment across overlapping views by optimizing homographies within a temporal window, minimizing photometric inconsistency. To ensure temporal coherence and reduce cumulative drift, the trajectory smoothing module introduces a relative coordinate-based Markov smoothing strategy, which incrementally models motion  
Figure 1: Overall network architecture. and applies historical constraints. Together, these modules enable real-time processing with high stability and stitching accuracy in an unsupervised learning framework. To begin with, consider two consecutive frames, \(I_{t}\) and \(I_{t+1}\). This study defines the function \(Stab(\cdot)\) to directly estimate the motion vectors of grid nodes between the two frames:  
\[MV_{i,j}^{t,t+1}=Stab(I_{t},I_{t+1};i,j) \tag{1}\]  
where \(i\) and \(j\) represent the row and column indices of the grid, respectively. Compared to pixel-level motion estimation, grid-level estimation provides a more stable representation of the overall motion trend within local regions, thereby mitigating the interference caused by dynamic foreground objects. To further enhance the accuracy and robustness of motion estimation, we follow the approach of [PERSON] et al. [13] and design a multi-scale and content-aware feature extraction module, which consists of the following four components:  
1. Multi-scale feature extraction: A feature extractor,\(f(\cdot)\), is employed to extract multi-scale feature maps from the input image \(I_{\beta}\): \[F_{\beta 1},F_{\beta 2},F_{\beta 3},F_{\beta 4},F_{\beta 5}=f(I_{\beta})\] (2)  
The first two scales are utilized for coarse alignment, while the remaining three scales capture fine details and contextual information, thereby providing rich semantic support for subsequent motion estimation. 2. Context-aware feature extraction: To integrate contextual information across adjacent frames, multi-scale contextual features are extracted from \(F_{\beta 3}\), \(F_{\beta 4}\), and \(F_{\beta 5}\) using the context-aware feature extractor \(ccl(\cdot)\): \[F_{ab1},F_{ab2},F_{ab3}=ccl(F_{\beta 3},F_{\beta 4},F_{\beta 5})\] (3)  
These features help compensate for information loss that may occur when using single-scale features in dynamic scenes, thereby improving the overall accuracy of motion estimation. 3. Inlier mask prediction: Since dynamic foregrounds and noisy regions may negatively impact homography estimation, we introduce a mask predictor \(m(\cdot)\) to generate an inlier probability mask \(M_{\beta}\): \[M_{\beta}=m(I_{\beta})\] (4)  
This mask is then applied to the corresponding scale feature maps, producing weighted features:  
\[G_{\beta k}=F_{\beta k}\odot M_{\beta},k\in\{1,2\} \tag{5}\]  
where \(\odot\) denotes the element-wise multiplication operation.