ShiftMorph: A Fast and Robust Convolutional Neural Network for 3D Deformable Medical Image Registration

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-17 13:52:30

Abstract: Deformable image registration (DIR) is crucial for many medical image applications. In recent years, learning-based methods utilizing the convolutional neural network (CNN) or the Transformer have demonstrated their superiority in image registration, dominating a new era for DIR. However, very few of these methods can satisfy the demands of real-time applications due to the high spatial resolution of 3D volumes and the high complexity of 3D operators. To tackle this, we propose losslessly downsampling by shifting the strided convolution. A grouping strategy is then used to reduce redundant computations and support self-consistency learning. As an inherent regularizer of the network design, self-consistency learning improves the deformation quality and enables halving the proposed network after training. Furthermore, the proposed shifted connection converts the decoding operations into a lower-dimensional space, significantly reducing decoding overhead. Extensive experimental results on medical image registration demonstrate that our method is competitive with state-of-the-art methods in terms of registration performance, and additionally, it achieves over $3\times$ the speed of most of them.

Main paper: 
 introduction: 
 Image registration is commonly used in various biomedical applications, such as surgical guidance, histological imaging, and neurosurgery. This technique is often used as a preliminary step, particularly in applications that involve multiple misplaced images. For instance, registration is highly valued in surgical navigation systems for its potential to reproduce high-quality intraoperative images [4, 15, 26] . Image registration usually involves two stages, i.e., affine image registration (AIR) and deformable image registration (DIR). AIR aligns view perspectives through scaling, translation, and rotation in rigid scenarios [7, 11, 34, 43] . Nevertheless, DIR has to be conducted to align anatomical structures accurately because of continuous deformations of soft tissues and organs within the human body. Mathematically, DIR can be reduced to optimizing the deformation field that specifies a warping destination for each voxel. Previous studies have devised numerous non-rigid deformation models grounded in diverse theoretical frameworks. Thereinto, traditional machine learning methods [3, 41, 47, 49] iteratively optimize deformation fields based on hand-crafted energy functions. These techniques typically require numerous iterations to achieve desirable registration outcomes, resulting in low computational efficiencies.In recent years, the boom of deep learning has opened up new avenues for image registration. Many learning-based techniques [2, 9, 11, 20, 33, 50] have now been introduced to enhance registration quality. However, most existing methods still can not fulfill the requirements of real-time scenarios, especially when handling 3D medical images. Despite GPU support, the inference time often remains impractical for time-critical applications, primarily due to the high resolution of 3D volumes and the high computational complexity of 3D operators. Seriously, the computational overhead explodes significantly as the resolution scales up. The computational modules at the top resolution level occupy the most inference time. Therefore, downsampling has to be used to reduce the image scale for some high-complexity models [29, 44, 53] . Some works [22, 46, 49] instead restrict the representation space of deformation fields. However, they cost the price of reduced image information or model capacity, as shown in Fig. 1 .Naively downsampling can cause a notable degradation in registration performance. The non-invertible downsampling process inevitably loses partial image details. To tackle this, we introduce the particular property of the strided convolution, i.e., the outcome sequence varies with the convolutional starting point. Hence, shifting the input sequence leads to distinct encoding results, thus complementing the missing information caused by scaling down. We then propose grouping the features according to their shifting behaviors, offering a basis for self-consistency learning. When merged within groups, the shifted encoding features can effectively remove redundant computations and improve robustness. Eventually, the grouping strategy and self-consistency learning allow for the trained network to be pruned by half after training without compromising the registration performance. Additionally, shifted connections restrict the decoding process in lower-dimensional spatial space, further accelerating the proposed network.To summarize, the network proposed in this study is referred to as ShiftMorph, and the major contributions of this paper are as follows:• We propose a lossless embedding module to reduce the spatial resolution and preserve image information simultaneously. We then present the companion modules, including feature grouping, group merging, and shifted connection, for fast and robust 3D image registration. where 𝐼 𝑓 is the fixed target image; L sim (•), as a similarity metric, penalizes the distance between the registered image pair; Reg(•) imposes extra constraints based on a priori knowledge of the deformation. Due to the physical nature of deformation, many methods [5, 10] employ the diffusion energy and bending energy to regularize the first and second derivatives of displacement fields, promoting smooth deformation. 
diffeomorphic deformation: 
 Image registration is often utilized to facilitate information fusion for multiple images [21, 45] , highlighting the importance of preserving topological structures. Thus, DIR often necessitates a delicate balance between registration precision and diffeomorphism. Welldesigned regularization can somewhat avoid destroying topological structures by voxel folding, but there is usually no guarantee of diffeomorphism. To tackle this, the Lie group has to be introduced to ensure diffeomorphic deformation. Let deformation be the continuous motion within a time interval, i.e., 𝑡 ∈ [0, 1]. 𝜙 𝑡 represents the deformation field at a given time 𝑡. Assume that the velocity field, denoted by 𝑣, is stationary, which holds 𝑣𝑑𝑡 = 𝑑𝜙 𝑡 . In particular, 𝜙 𝑡 =0 = id is an identity map. Then, the exponential map of the velocity field is defined by:EQUATIONwhich splits the deformation into 𝑛 equivalent minimal motions over 𝑡 𝑛 . Thus, 𝜙 𝑡 belongs to a Lie group that holds 𝜙 𝑡 = exp(𝑣𝑡) and 𝜙 𝑡 1 +𝑡 2 = exp(𝑣𝑡 1 ) • exp(𝑣𝑡 2 ). As long as each sub-deformation is sufficiently small, the integrated deformation can be ensured to be diffeomorphic. Ultimately, 𝜙 1 can be derived recursively through𝜙 𝑡 = 𝜙 𝑡 /2 • 𝜙 𝑡 /2, which is the widely applied squaring and scaling (SS) skill [1, 9, 32] .Learning-based methods have been driving the advancement of DIR for years due to their capability of generalizing knowledge from vast amounts of data. These methods can be trained efficiently in endto-end manners with appropriate loss functions. Currently, there are two main categories of registration networks according to their computational modules, i.e., CNN-based methods and Transformerbased methods. Both of the two types of registration networks commonly adopt UNet-like architectures [35] to produce deformation fields by incorporating a final registration head [2, 6, 37, 46] . Some studies [20, 28, 29, 39] propose refining the deformation field progressively, which is beneficial for addressing large deformations. Jia et al. [22] utilize the band-limited Fourier domain to represent displacement fields in a low-dimensional space. Kim et al. [24] propose employing cycle consistency to enhance topology preservation. Zhou et al. [52] and Vray et al. [42] utilize the technique of knowledge distillation to improve network efficiency and registration performance. The networks proposed by Chen et al. [5, 6, 38, 48, 53] introduce the widely applied self-attention mechanism to enlarge reception fields, achieving state-of-the-art registration performance. TransMatch [8] performs feature matching between the misplaced image pair with a modified cross-attention mechanism. Most of the Transformer-based methods adopt the attention technique of Swin-Transformer [27] due to the high resolution of 3D medical volumes. 
methodology: 
 Although the existing methods have significantly improved registration performance, their computational complexities remain infeasible to deploy in real-time medical devices. Based on our observation that most computational latencies stem from the upper encoders and decoders, the main idea of this study is to sink the computation of the network down to a lower resolution while still maintaining high registration performance. The overall architecture and the proposed modules are illustrated in Fig. 2 .A 1D sequence can be cut in half through the 1D convolution with a stride of 2, commonly used as a learnable downsampling approach. However, this process inevitably incurs image degradation and information loss, which is not invertible, making it impossible to recover the original information completely. Essentially, the strided convolution skips parts of the sliding positions, thus obtaining a shorter scanning path. The missing sliding positions can be directly recovered through shifting. As illustrated in Fig. 3 , two different paths are derived by shifting the starting point, leading to distinct kernel coverages and convolutional outputs. Hence, the 1D stride-2 convolution can produce two different downsampling sequences, effectively filling in the missing details caused by resolution reduction. When extended to the 3D convolution, eight shifting options are available on the cubic grid. Based on this, a lossless downsampling operator is obtained by shifting the input images and acquiring multiple encoding sequences through the stride-2 convolution. We utilize this approach to construct the shifted embedding, which serves as the kernel module of the proposed network. As shown in Fig. 2b , the input image pair is concatenated and shifted along three directions to produce eight data volumes. The stride-2 convolution is then used to reduce the resolution to 1 2 . Nonetheless, this module actually performs pseudo-downsampling that grafts the spatial dimension onto the batch size. The complexity of network computation remains unchanged. Thus, we propose the complementary techniques of feature grouping and group merging to achieve efficient registration.As commonly known, the local intensities of a natural image are highly correlated, which limits the encoding differences of the eight shifts. In the decoding stage, i.e., the right part of Fig. 2a , which is comparatively costlier than encoding, reserving all these feature tensors is unnecessary. A better deal is to perform mean estimation as it can markedly reduce computational overhead and boost the feature representations. Therefore, we divide the eight feature tensors into two groups according to their convolutional starting points. Fig. 2b displays that every four features with the same color are grouped together. Precisely, four shifting destinations of one group are positioned near the starting point, whereas others are far from it. The shifting step is minor, no greater than 1. Consequently, the two group-wise mean estimates exhibit a high level of consistency, thereby providing a basis for conducting self-consistency learning. In addition, averaging significantly reduces feature variance, demonstrating strong robustness.During decoding, we initially perform averaging within groups to reduce the eight feature tensors to only two, thereby reducing the decoding overhead to 1 4 . Even though the encoding stream costs the same as before, the proposed shifting and grouping strategies significantly accelerate our network, in view of the fact that decoders of a UNet-like network are usually more expensive than encoders.The typical implementation of the decoder with a skip connection can be formulated by:EQUATIONwhere ⊕ and 𝑊 𝑇 dec represent concatenating and decoding, respectively; 𝑓 𝑟 enc and 𝑓 𝑟 dec denote the encoding and decoding outcomes at the resolution scale of 𝑟 . The skip connection concatenates the decoding features with the encoding features at the same resolution level, which is time-consuming due to the concurrent expansion of channels and spatial resolution. We propose shifting the skip connection down to obtain an even cheaper decoder, i.e., the shifted connection in Fig. 2d . The shifted connection densely connects the decoding features with the downsampled features from the upper encoder, as expressed by:EQUATIONwhere 𝑊 down represents downsampling. As a result, feature concatenations are ahead of upsampling; the decoder operates in a lower-dimensional space, further reducing the computational complexity. Besides, this design is of lower rank than the classical skip connection, which is beneficial for filtering out noise and redundant features [14, 23] .Due to the grouping strategy, the final registration head of Shift-Morph can produce two deformation fields for each image pair, as depicted in Fig. 2a . These two fields, denoted by 𝜙 and φ, represent the deformation of an approximately identical voxel grid. In view of this, a reasonable assumption is made that 𝜙 and φ are highly consistent. We then reverse the registration direction of the second group, thus obtaining φ -1 . To promote natural deformations, an extra regularizer is introduced to approximate the identity map by composing 𝜙 and φ -1 , which is expressed by:EQUATIONThis regularization term, namely self-consistency learning, aids in model training and enhances the quality of deformation fields. In addition, the restricted difference between 𝜙 and φ offers a potential for removing one feature group after training.In the training stage, both groups are needed for computing 𝜙 and φ. Alternatively, either 𝜙 or φ can be used for testing due to selfconsistency learning, which implies that we can discard one group naively after training. This paper only reserves the group near the starting point and uses 𝜙 to perform registration in testing cases.Hence, the overall computational burden of ShiftMorph can be reduced by half, achieving a 2× speedup on top of the complete implementation of ShiftMorph.A plain CNN tends to be weak in capturing long-distance correlations. Without special designs, these networks are prone to failure in the case of large deformation. This topic is beyond the focus of this paper. As an alternative, we start with 𝜙 (0) = id and recurrently warp the moving image multiple times as follows:EQUATION𝜃 denotes the learnable parameters of ShiftMorph.As presented in Eq. ( 1), the deformation loss used in this paper consists of two terms based on the image similarity and the smoothness of the deformation field. Image Similarity Metric. Mean square error (MSE) is a common metric for quantifying the similarity between the model output and the ground truth in generative tasks, such as denoising and super-resolution [30, 40, 51] . However, MSE is sensitive to the voxel intensity. The image pairs in registration tasks often hold weak voxel-wise consistency, thus limiting the effectiveness of MSE in training registration networks. Instead, normalized crosscorrelation (NCC) is not sensitive to the voxel intensity and is preferable for measuring texture and structure similarities. NCC can be formulated as:EQUATIONwhere Cov(•) and Var(•) respectively estimate the covariance and variance of the input sequence. Local NCC (LNCC), which estimates the covariance and variance within a local patch, is a more common choice when dealing with image data. Smoothness Regularization. The smoothness of deformation is encouraged by its physical nature. Let 𝑢 represent the displacement field of a 3D image that holds 𝜙 = id+𝑢 and ∇ 𝑖 𝜙 = 1+∇ 𝑖 𝑢, ∀𝑖 ∈ {𝑥, 𝑦, 𝑧}. ∇ 𝑥 , ∇ 𝑦 , and ∇ 𝑧 are the Laplacian operators estimating the spatial gradients along three directions. We utilize the diffusion energy to penalize the spatial gradients of the output displacement S h if tM o r p h × 3 S h if tM o r p h × 3 field, as formulated by:EQUATIONKeypoint Correspondence Supervision. Keypoint correspondences can be used as the ground truth to supervise deformation estimation. In this case, target registration error (TRE) is employed for supervised training. Consider two point clouds with 𝑘 matched point pairs of the fixed and moving image, denoted by𝑃 𝑓 = {𝑥 1 , 𝑥 2 , • • • , 𝑥 𝑘 } and 𝑃 𝑚 = {𝑦 1 , 𝑦 2 , • • • , 𝑦 𝑘 }.The TRE loss is then calculated as:EQUATIONIncorporating the aforementioned self-consistency, the combined loss function of this paper is as follows:EQUATION) 
experiments 4.1 experimental settings: 
 Baseline Methods. We evaluate two Transformer-based methods and five CNN-based methods, including TransMorph [5] , Trans-Match [8] , VoxelMorph [2] , VoxelMorph++ [16] , LapIRN [33] , PCNet [29] , and FourierNet [22] , in comparison with ShiftMorph. Thereinto, PCNet, TransMorph, and TransMatch are the current state-ofthe-art learning-based methods. VoxelMorph is a classical method and has been widely applied. VoxelMorph++ with spatial search for keypoint displacements is an improved version for supervised learning with keypoint correspondences. The original implementation of PCNet is based on TensorFlow. We reimplement it under the PyTorch framework to ensure a fair comparison. All the other methods are trained using the published source codes. Implementation Details. Three versions of the proposed method are implemented in this paper, including ShiftMorph, ShiftMorphdiff, and ShiftMorph×3. ShiftMorph-diff employs the squaring and scaling skill to improve the diffeomorphism of the resultant deformation field through 7 iterations. ShiftMorph×3 recurrently warp the moving image three times and output 𝜙 (3) following Eq. ( 6). Parameter Settings. The proposed method and the comparison baselines are trained under the same experimental settings for a fair comparison. We set 𝛼 = 1 for supervised training if correspondence labels are available; otherwise, TRE loss is not involved. We follow a common setting of 𝜆 = 1 without hyperparameter fine-tuning for all the comparing methods. 𝛾 = 0.01 is used only for ShiftMorph with self-consistency learning. The Adam optimizer [25] with a learning rate of 0.0001 is utilized for training. The learning rate is adjusted by a cosine annealing schedule with a maximum epoch of 300. All the experiments are conducted using an Intel Xeon Silver 4314 CPU and a 24G NVIDIA Geforce RTX 3090 graphics card.Evaluation Metric. The Dice similarity coefficient (DSC) and Hausdorff distance (HD) are utilized to measure the registration accuracy of each anatomical structure. DSC calculates the degree of overlap of segmentation labels after deformation. The HD score measures the shape distance between the warped image and the target image from being isometric. We use the 95th percentile HD to mask out outliers, namely HD95. Topological preservation should also be emphasized to protect the original information of the moving image. To this end, the Jacobian Determinant (JacDet) is utilized to examine whether a voxel is folded by warping. ∇ 𝑥 𝜙 (𝑝), ∇ 𝑦 𝜙 (𝑝), and ∇ 𝑧 𝜙 (𝑝) represent 3 directional vectors determining the 𝑝-th minimal cube in 𝜙. The determinant, i.e., |∇ 𝑥 𝜙 (𝑝), ∇ 𝑦 𝜙 (𝑝), ∇ 𝑧 𝜙 (𝑝)|, calculates the volume of this cube. A non-positive volume value indicates that the corresponding voxel is flipped over or folded after deforming, losing its original topological structure. Thus, the degree of topology destruction is measured using the percentage of voxels with a non-positive JacDet.In this section, we evaluate the registration performance in interpatient and atlas-based brain magnetic resonance imaging (MRI) registration tasks.For inter-patient registration, we utilize the OASIS dataset [19, 31] acquired from the 2021 Learn2Reg challenge [18] , which consists of 451 brain T2-weighted MRI images. The Learn2Reg official split contains 394, 19, and 38 images for training, validation, and testing. The segmentation labels of testing images are not available. Thus, we conduct the testing stage on the validation set instead.For atlas-based registration, we choose the IXI dataset1 , a commonly chosen benchmark. In this experiment, the brain atlas image is used as the source image, whereas patient images act as the target images. The IXI dataset consists of 414 brain MRI images that are T1-weighted. The original MRI scans are preprocessed using FreeSurfer [13] through a stream that includes skull stripping, spatial normalization, labeling, and others. We adopt the same split used by TransMorph [5] , which contains 403 images for training, 58 for validation, 115 for testing, and one atlas image. Each image undergoes voxel-wise labeling for 30 anatomical structures.The first part of Tab. 1 displays the runtime performance of registering images sized 160 × 192 × 224 from the two datasets. The Transformer-based methods own large amounts of parameters and multiply-accumulate (MAC) operations, whereas CNN-based methods tend to be faster and contain fewer parameters. Most of the comparing methods incur high latencies exceeding 0.1s, leading to a low throughput even on a high-performance GPU device. The results given by FourierNet seem appealing, but this method roughly masks out most high-frequency features, causing significant accuracy degradation. In comparison, ShiftMorph demonstrates significant superiority in both the computational overhead and the inference speed, achieving over 3× the speeds of most comparing methods. Even if recurrently warping the moving image three times for high accuracy, ShiftMorph×3 can still achieve a better speed than most on a GPU device. However, warping as a highly parallel operator is slow when using a serial CPU processor. The right parts of Tab. 1 present the averaged evaluation results on these two registration tasks. As for topological preservation, ShiftMorph-diff is the most effective in this experiment. ShiftMorph folds 0.7786% of voxels on the OASIS dataset; this value can be lowered to 0.0231% by ShiftMorphdiff. As for the IXI dataset, the folded voxel proportion is reduced from 1.4993% to 0.048%. In contrast, PCNet, equivalently armed with the SS skill, folds more voxels than our method and has significantly higher computational complexity. FourierNet also shows good runtime performance and topological preservation, but its registration precision is relatively weak. If not using the SS and band-limit skills, i.e., excluding ShiftMorph-diff, PCNet, and FourierNet, ShiftMorph keeps better diffeomorphism than other methods, which benefits from self-consistency learning and merging the shifted features.We then use the Lung250M-4B dataset [12] , a combined lung computed tomography (CT) image registration benchmark, to evaluate our method in a supervised large deformation scenario. This dataset contains 124 pairs of expiratory and inspiratory lung CT scans from different patients. The official split selects 97, 17, and 10 paired scans for training, validation, and testing. For the CT scans in the training set, keypoint correspondences are automatically generated by the corrfield method2 [36] to support supervised training. As for the validation and testing set, manual landmark annotations are provided to determine registration accuracy. Lung CT image registration is still challenging for learning-based methods. Following the pipeline in [36] , we conduct instance optimization to refine the network outputs through 50 iterations of minimizing the dissimilarity of MIND features [17] and a Lapalace regularization. We next utilize Gaussian noise to simulate image degradation and conduct robustness testing on the OASIS dataset. Fig. 5 illustrates the warped images and deformation fields produced by corresponding methods. The baseline methods, except FourierNet, suffer from significant performance degradation, resulting in seriously folded deformation fields and abnormal warping results. In contrast, Shift-Morph demonstrates the best deformation results and preserves topological structures commendably, exhibiting smooth and continuous surface shapes in the warped slices. Fig. 7 Figure 7 : The impacts of noise on registration performance regarding the Dice score. Our method exhibits good robustness to Gaussian noise.We conduct a brief ablation study on the proposed modules and assess the effect of self-consistency learning. We evaluate the registration performance of the corresponding model variants on the OASIS dataset. Results are summarized in Tab. 3 and Tab. 4. By default, we apply the shifted embedding in combination with the grouping strategy to save space. The first item of Tab. 3 represents a naive UNet using the skip connection without downsampling. Downsampling with trilinear interpolation significantly improves network speed; however, it comes at the high price of a loss of registration accuracy. Comparingly, shifted embedding armed with the grouping strategy can reduce computational overhead and improve registration performance. Besides, the voxel folding is mitigated by imposing consistency among neighbor voxels with self-consistency learning. The shifted connection also contributes to a notable improvement in network performance, achieving 1.6× the speed of a naive UNet.Ultimately, ShiftMorph achieves 3.6× the speed, better registration results, and better diffeomorphism.As evidenced by Tab. 4, performing self-consistency learning is beneficial for improving the quality of output deformation fields. In most cases, except the red one, self-consistency leads to better evaluation scores for three versions of ShiftMorph. However, this improvement is relatively small when combined with the SS skill, which can also be viewed as particular regularization. Applying too much regularization may cause underfitting.  
conclusion: 
 In this paper, we have investigated the common issue that 3D deformable registration networks frequently suffer from low inference efficiency. We have revisited the widely adopted network architecture of the UNet style for image registration, demonstrating that the major computational burden of these networks stems from the operations at the top level. To achieve fast and high-quality registration, we have proposed downsampling using the strided convolution and compensating for information loss by shifting. We have presented a grouping strategy to merge features group-wisely and reduce redundant computations. The proposed shifted connection contributes to a further acceleration and improvement of registration. Self-consistency learning, derived from the grouping strategy, promotes the quality of deformation fields and supports pruning half of the network. Experimental results demonstrate that the proposed method performs competitively with current state-of-the-art methods, however, at 3× the speed of most.  
 
 Appendix: https://brain-development.org/ixi-dataset/https://grand-challenge.org/algorithms/corrfield/ 


Decision: Although two of three reviewers tend to accept this paper, and the rebuttal letter has addressed most concerns from the two reviewers. However, it does not address the concerns from the remaining reviewer, specially about the novelty.

Reviews: review: paper_summary: paper_summary: 1. This paper proposes a lossless embedding module to reduce the spatial resolution and preserve image information simultaneously.
2. This paper proposes self-consistency learning to train the network and restrict the deformation difference between groups.
3. Experimental results show good performance in both runtime and registration performance.
strength_weakness: strength_weakness: 1. The motivation of this paper is clear, it aims to give an efficient solution for deformable image registration.
2. The proposed methods for shifting are new, along with the merging and self-consistency learning design.
3. The comparison results are promising, and the experiments are relatively complete.
limitations: limitations: 1. Relevance to multimedia is not clearly stated. I think the paper about medical image registration may be more suitable for MICCAI or ISBI conferences.
2. Some reference papers are missing. The proposed ShiftMorph relies on the novel operation of the input image, and I think the author should discuss the related paper "Cross-Resolution Distillation for Efficient 3D Medical Image Registration" in the Related Work section. Although numerical comparison is unnecessary due to different technical routes, a discussion is needed.
3. The specific network structure is not clear, including the number of conv blocks, channels and so on. By the way, why do we need to use instance normalization? 
4. Visualization is not convincing. First, the differences in segmentation masks between different methods are not obvious. Second, the warped moving image seems to be distorted. Usually, such distortions are caused by unsuitable regularization, can you explain this problem?
5. The ablation experiments about regularization weights are missing. As mentioned above, the regularization weights are important, so the related experiments are demanded.
6. Though the performance compared with baseline methods is promising, the improvement of efficiency should be clarified in detail. In other words, I think the author should conduct one efficiency comparison experiment, focusing on the proposed operations and maintaining the same parts including conv blocks/channels as other baseline methods, aiming to show the superiority of the key contribution.
7. When applied with the diff-version, I wonder how to operate the self-consistency learning, can you give the specific designs?
paper_summary: paper_summary: This paper propose losslessly downsampling by shifting the strided convolution for 3D Deformable Medical Image Registration. Besides, a grouping strategy is then used to reduce redundant computations and support self-consistency learning. Experiments prove the effectiveness of the proposed method.
strength_weakness: strength_weakness: The article is well organized and elaborated, and the experiments are relatively rich.
limitations: limitations: 1) Although the manuscript proposes a ShiftMorph network, it is not innovative enough compared with existing 3D deformable image registration methods;
2) There are many hyperparameters in Equation(10), making the parameter tuning process complex and time-consuming.
paper_summary: paper_summary: This paper is about image registration using CNN. The main method is lossless downsampling by shifting the strided convolution. A grouping strategy is then used to reduce redundant computations by using the average of pixels in a local group to represent the region and support self-consistency learning. The process of taking the average of pixels in a local group has the advantage of reducing variance of features. Self-consistency learning is used to improve the quality of deformation. The final results are small network and fast decoding.
strength_weakness: strength_weakness: Main strengths include 1D stride-2 convolution to produce two downsampling sequences. Other strengths include the use of shifted connection as a way to reduce inference time. The overall idea of training the network to produce two similar deformation fields \phi and \tilde{\phi}^{-1} and \phi*\tilde{\phi}^{-1} are approximately the inverse of each other and then only one is needed for registration seems novel.
limitations: limitations: Limitations include some hard-to-understand sentences. For example, “they cost the price”, probably the authors meant to say “they pay the price” or “they cost”.

In Equation (8), it seems L_Grad is not normalized by the number of pixels in an image, but in Equation (9), L_TRE is normalized by the number of matched point pairs. Will it create a bias toward one loss or the other, depending on how many matched point pairs there are?
It seems some explanation about LNCC is needed. How is a local patch selected? How big is it? 

Furthermore, how are the keypoints selected to compute TRE loss for the MRI? In other words, how are the point clouds computed? 

How to determine the weight factors alpha, lambda, gamma in Equation (10)? The authors used alpha=1, lambda=1, and gamma = 0.01, are those choices applicable to other types of image? 

It seems that \gamma (Table 4) has little effect on the performance of the proposed method. Then why is this loss term included in Equation (10)?

If not considering time of computation, will better results be obtained if one warp the moving image more than three times? That is, will people obtain better registration results by ShiftMorph*N, where N> 3?