# Rectification for Stitched Images with Deformable Meshes and Residual Networks
## 3 Materials and Methods
### Network Architecture  
The network architecture proposed in RIS-DMRN consists of two components (Figure 3): the rigid target recognition network and the width residual regression network [41]. The input comprises irregularly stitched images and their stitching mask matrix. The input image is initially processed by a simple recognition convolutional neural network to detect the quantity and orientation of rigid structures. Based on the detection results, the most suitable mesh shape is chosen for rectification. For instance, if there are predominantly horizontal or vertical rigid structures in the image, a preference is given to selecting a rectangular mesh. In the case of a higher prevalence of curved surfaces or curved structures,a triangular mesh is chosen. If the quantities of vertical rigid structures and curved structures are comparable, a hexagonal mesh is selected for image rectification.  
The main structure of the rigid structure detection in this paper is a convolutional neural network (Figure 4), with input images resized to a unified 448 \(\times\) 448. The CNN consists of six convolutional blocks, each composed of various combinations of 3 \(\times\) 3 convolutional kernels, 1 \(\times\) 1 convolutional kernels, and 2 \(\times\) 2 max-pooling layers with a stride of 2. After extracting image features into a 1000-dimensional 7 \(\times\) 7 feature vector, a 1000-dimensional vector is generated through average pooling. This vector is then input into Softmax for rigid structure detection. The network also incorporates normalization and dropout operations, although they are not explicitly shown in the diagram.  
Figure 4: Neural networks for target detection in rigid structures.  
Figure 3: Schematic structure of width residual neural network.  
Once the mesh shape is determined, the irregularly stitched image and its mask matrix are fed into the Wide Residual Neural Network (Wide ResNet) for the rectification process. The choice of Wide ResNet as the recovery prediction network for image rectification is motivated by its ability to enhance feature dimensions in each residual block through increased channel numbers. This augmentation enables the network to capture richer feature representations, playing a crucial role in the recovery of content items after mesh transformation and minimizing the loss in rectified content. Moreover, due to significant internal variations within the meshes during the rectification process, some meshes experience a gradual decrease in gradients during the backward propagation of model training, leading to convergence challenges. The introduction of wider residual blocks in the Wide ResNet facilitates easier gradient flow, mitigating the issue of gradient vanishing during training [42, 43].  
The architecture of the Wide ResNet employed in RIS-DMRN consists of four residual convolutional blocks followed by an average pooling layer (Table 1). In the network, \"k\" represents the multiplier for the convolutional kernels in the original module, \"N\" indicates the number of residual modules in that layer, and \"B (3,3)\" signifies each residual module consisting of two 3 \(\times\) 3 convolutional layers. After feature extraction through the residual network, a simple fully convolutional structure is utilized as a mesh motion regressor to predict the horizontal and vertical movements of each vertex based on the regular mesh, facilitating the output of the rectified image.