# End-to-End Online Video Stitching and Stabilization Method Based on Unsupervised Deep Learning
## 2 Methodology
### Loss Function
#### 2.4.1 Loss for Trajectory Estimation and Stitching Motion Estimation  
Both the camera trajectory estimation and stitching motion estimation modules adopt an image registration strategy. The trajectory estimation module employs a feature mask to filter out dynamic regions, which is directly applied in the stitching motion estimation module. To mitigate the interference introduced by dynamic regions, a triplet loss is introduced to optimize the homography estimation. This loss aims to ensure that the masked and warped image pairs become more similar while constraining the inconsistency between the forward and backward homography matrices. The loss function is formulated as follows:  
\[\text{min}L_{triplet}=L_{n}(I_{a}^{\prime},l_{b})+L_{n}(I_{b}^{\prime},l_{a})- \xi L_{n}(I_{a},l_{b})+\mu\|H_{ab}H_{ba}-I\|^{2} \tag{16}\]  
where \(la^{\prime}\) and \(lb^{\prime}\) represent the images after inlier mask weighting and warping, while \(Hab\) and \(Hba\) denote the forward and inverse homography matrices between images \(la\) and \(lb\). The hyperparameters \(\xi\) and \(\mu\) are used to balance the relative importance of different terms.  
Given the significant viewpoint variations in stitching scenarios, relying solely on the triplet loss may lead to slow convergence or even divergence. To enhance robustness in low-overlap regions, optical flow information is incorporated during training. Specifically, the optical flow loss is formulated based on the discrepancy between the actual optical flow \(OF\) obtained from PWC-Net and the feature flow \(FF\) estimated from network-based feature matching:  
\[L_{flow}=\frac{M_{\beta}\times F_{\beta}\times\left\|OF-FF\right\|^{2}+\lambda \times(1-M_{\beta})\times F_{\beta}\times\left\|OF-FF\right\|^{2}}{pixel} \tag{17}\]  
where \(M_{\beta}\) represents the inlier probability mask (ranging from 0 to 1), and \(F_{\beta}\) denotes feature similarity. The hyperparameter \(\lambda\) controls the contribution of the mask to the loss.  
To ensure spatial continuity of the grid motion vectors computed via the function \(stitch(I_{t,ref},I_{t,tar};i,j)\), a continuity loss function \(L_{\text{cont}}\) is introduced. This loss constrains the motion vector differences between adjacent grid points in both vertical and horizontal directions, preserving the smoothness of the motion field. It is formally defined as follows:  
\[L_{cont}=L_{x}+L_{y} \tag{18}\]  
Combining the above components, the total loss function for trajectory estimation and stitching motion estimation is expressed as  
\[L_{total}=L_{triplet}+\lambda L_{flow}+\beta L_{cont} \tag{19}\]where the hyperparameters \(\lambda\) and \(\beta\) balance the impact of each component. The weight \(\lambda\) is selected so that the flow consistency term effectively suppresses inter-frame jitter without dominating the triplet alignment term; if \(\lambda\) is too small, motion artifacts persist, whereas if \(\lambda\) is too large, fine alignment deteriorates. Likewise, \(\beta\) is chosen to enforce long-range temporal coherence while preserving local stitching accuracy: a small \(\beta\) permits cumulative drift over extended sequences, while a large \(\beta\) over-smooths local alignment. In practice, \(\lambda\) and \(\beta\) are adjusted to equilibrate the gradient magnitudes of their respective loss components, and their values are confirmed through validation set metrics that jointly measure alignment fidelity and stability.  
To facilitate early convergence while ensuring fine alignment in later stages, the parameter \(\lambda\) decays with the training epoch \(t\) as follows:  
\[\lambda(t)=\lambda_{0}\exp(-\gamma t) \tag{20}\]  
where \(\lambda_{0}\) is the initial value, and \(\gamma\) is the decay factor.