# Cross-Modal Image Registration via Rasterized Parameter Prediction for Object Tracking
## 3 Methodology
### Image Transformation  
A homography matrix describes the transformation relations between the moving image \(I_{m}\) and reference image \(I_{r}\). Here, we use \(y_{i}\) to represent the homogeneous coordinates  
Figure 2: The process of registration parameter prediction; the matching response of dual-way features is fed into the full-connected layers to predict the offsets of the four corner points between the moving image and reference image. Then, obtain the feature point pair is obtained to resolve the respective homography matrices.  
of one sampling point in the moving image \(I_{m}\); after transformation, this is represented as \(x_{i}=Hy_{i}\) in the coordinate system of the reference image \(I_{r}\). Based on this transformation relation, we can use bilinear sampling and sampling point transformation for image registration between the moving image \(I_{m}\) and reference image \(I_{r}\) to obtain a transformation image \(I_{w}\), which has a grayscale space that is coherent with \(I_{m}\) and pixel coordinates aligned with \(I_{r}\).  
Specifically, we first generate a sampling point matrix \(X_{w}\) according to the resolution of the transformation image \(I_{w}\in\mathbb{R}^{3\times HW}\), in which each column vector represents the homogeneous coordinates of each pixel in \(I_{w}\). The corresponding pixel coordinates of the sampling point for transformation image \(I_{w}\) in moving image \(I_{m}\) are represented as  
\[X_{w\to m}=MX_{w}, \tag{2}\]  
where \(M=H^{-1}\) is the inverse matrix of homography matrix \(H\), indicating the transformation relation from the coordinate of \(I_{r}\) to the coordinate of \(I_{m}\). We note that the sampling point transformation is performed in each raster region using its own homography matrix in accordance with the rasterized local homography matrix estimation method of our registration parameter prediction network. In this case, the final sampled transformation image has serious boundary discontinuity problems when the homography matrices of adjacent raters are quite different, as shown by the example in Figure 2(a).  
In this paper, we propose a position-dependent weight matrix for the homography matrix constraint in each raster region. The weight matrix is used to enhance the transformation effect of the homography matrix on the pixels inside the raster area while equalizing the effect on the raster junctions. Specifically, a vector \(w\) is generated in accordance with the ratio of the weights' full action region length to the raster action length \(a\) and the ratio of the transition action region \(b\), where the weight of full action region is 1 and the weights of the transition regions on the left and right sides are linearly distributed from 1 to 0.  
Next, we take an outer product on the vector \(w\) to obtain the weight matrix of a single raster, then translate it according to each raster's center point to obtain the final weight matrix \(W\). The complete sampling point transformation can be expressed as follows:  
\[X_{w\to m} =\sum_{Z}(M_{Z}\odot W_{Z})X_{W} \tag{3}\] \[=\sum_{Z}M_{Z}(X_{W}\odot W_{Z}). \tag{4}\]  
After sampling point transformation, we can obtain the transformation image \(I_{W}\) by performing a bilinear sampling in accordance with the grayscale value of the \(X_{w\to m}\) to moving image \(I_{m}\). Figure 2(b) illustrates the raster region weights, and the weighted transformed image is shown in Figure 2(c).  
Figure 3: Illustration of weighted raster image transformation to address boundary discontinuity problems, showing (**a**) example image with discontinuous boundary issue, (**b**) the raster region weights, and (**c**) the weighted transformed image.