# Rectification for Stitched Images with Deformable Meshes and Residual Networks
###### Abstract  
Image stitching is an important method for digital image processing, which is often prone to the problem of the irregularity of stitched images after stitching. And the traditional image cropping or complementation methods usually lead to a large number of information loss. Therefore, this paper proposes an image rectification method based on deformable mesh and residual network. The method aims to minimize the information loss at the edges of the spliced image and the information loss inside the image. Specifically, the method can select the most suitable mesh shape for residual network regression according to different images. Its loss function includes global loss and local loss, aiming to minimize the loss of image information within the grid and global target. The method in this paper not only greatly reduces the information loss caused by irregular shapes after image stitching, but also adapts to different images with various rigid structures. Meanwhile, its validation on the DIR-D dataset shows that the method outperforms the state-of-the-art methods in image rectification.  
image rectangular; deformable mesh; width residual network; global loss function +
Footnote †: journal:selection of the deformable mesh shape is based on the judgment of the current image's rigid structure by a convolutional neural network, offering three options: triangle, rectangle, and regular hexagon. Once the mesh shape is determined, the prediction network generates an initial predicted mesh based on the input irregular image and its mask matrix. The training process employs a width residual network to predict the initial mesh by the content-aware processing of irregularly stitched images. Subsequently, the input irregular image, predicted initial mesh, and predefined target mesh are collectively input into the width residual neural network for rectification regression. The loss function of the width residual network comprises local and global parts. The local loss function controls the deformation loss of targets within the mesh, while the global-related loss function helps avoid global information loss during the deformation process. Finally, the image rectangular restoration is achieved through continuous iterative regression using residual neural networks (Figure 1).  
In response to the current problems of a large loss of edge information and severe deformation of internal details in image rectangles, this paper proposes a method based on variable grids and residual neural networks. The highlights of this study include the following:  
* This approach utilizes a deformable mesh as the initial mesh, allowing for more versatile directional movement during mesh restoration and achieving better correction effects for targets within irregular images.
* The introduction of local and global-related loss functions significantly mitigates the drawbacks of traditional methods that focus only on partial regions, enhancing the overall coherence during the deformation recovery process and preserving content more effectively after image rectification.
* In the experiment of randomly parallelizing 300 irregular images on the public dataset DRI-D [21], SSIM and PSNR reached 0.7234 and 22.65, respectively, achieving a relatively accurate level.  
Figure 1: Comparison of rectification methods for irregular image stitching.