# End-to-End Online Video Stitching and Stabilization Method Based on Unsupervised Deep Learning
## 1 Introduction  
With the widespread adoption of smartphones, sensor resolutions have steadily improved. However, due to the limitations of built-in consumer-grade lenses, the field of view (FoV) of cameras remains restricted [1]. Video stitching technology combines multiple video sequences from moving cameras into a single frame with a wider FoV and higher resolution, thereby enhancing the user's immersive visual experience. This technology has been widely applied in video surveillance [2], drone inspections [3], and autonomous driving [4]. However, low-quality or failed video stitching not only compromises the visual experience but also reduces the accuracy of downstream video analysis tasks such as object detection and tracking [5, 6, 7]. In handheld camera video stitching, common distortions can be categorized into two main types: natural shaking (Shaky), caused by camera motion, and visual inconsistency (Jitters), arising from insufficient consideration of temporal continuity during frame stitching. Traditional solutions often rely on additional hardware to impose constraints on camera motion for video stabilization, but these methods are costly and inconvenient. Incontrast, algorithm-based video stitching and stabilization methods offer greater flexibility and cost-effectiveness. Conventional video stabilization techniques typically involve three steps: camera trajectory estimation, trajectory smoothing, and motion compensation. Camera trajectory estimation relies on feature point extraction and tracking, approximating camera motion through homography matrices or motion vectors [8; 9; 10]. However, in scenes with sparse textures, severe occlusions, or intense motion, these methods often suffer from feature matching failures and cumulative errors, affecting stabilization performance [11]. Moreover, reliance on handcrafted features limits robustness under varying illumination and scene conditions, and the iterative nature of RANSAC-based outlier removal can lead to increased computational latency. These shortcomings make it challenging for traditional pipelines to maintain real-time performance and high-quality stabilization in complex, dynamic environments. Image and video stitching techniques have been extensively studied in computer vision and multimedia fields, with homography estimation being the core challenge. Traditional methods typically rely on feature point detection algorithms such as SIFT, SURF, and ORB for matching, followed by robust estimation algorithms like RANSAC to eliminate outliers and compute the transformation matrix. Additionally, flexible transformation models such as grid-based transformations and thin-plate splines (TPSs) have been introduced to mitigate parallax errors while incorporating shape-preserving constraints to improve stitching quality. Deep learning-based approaches have also emerged, using end-to-end networks to directly regress homography transformation parameters, demonstrating better adaptability and robustness in complex scenes. Nevertheless, handcrafted feature-based stitching still exhibits poor handling of low-texture regions and repetitive patterns, leading to visible seam artifacts. Even deep learning regressors can fail in extreme parallax or illumination changes due to insufficient training diversity, and the lack of explicit geometric constraints may result in nonphysical warping artifacts. For trajectory smoothing, early methods primarily employed signal processing techniques such as low-pass filtering, Kalman filtering, and polynomial interpolation to suppress camera shake and optimize trajectory smoothness. More recently, deep learning-based approaches, including StabNet, optical flow-based optimization, and adversarial learning networks, have further enhanced video stability, allowing trajectory smoothing to adapt more effectively to complex scenarios. However, signal processing-based filters often introduce temporal lag or over-smoothing that distorts rapid camera movements.