Multi-Objective Learning for Deformable Image Registration

Authors: monika  grewal
henrike  westerveld
peter a n bosman
tanja  alderliesten
Venue: MIDL_io_2024_Conference
Published: 2024-01-30 21:26:52

Abstract: Deformable image registration (DIR) involves optimization of multiple conflicting objectives, however, not many existing DIR algorithms are multi-objective (MO). Further, while there has been progress in the design of deep learning algorithms for DIR, there is no work in the direction of MO DIR using deep learning. In this paper, we fill this gap by combining a recently proposed approach for MO training of neural networks with a well-known deep neural network for DIR and create a deep learning based MO DIR approach. We evaluate the proposed approach for DIR of pelvic magnetic resonance imaging (MRI) scans. We experimentally demonstrate that the proposed MO DIR approach -- providing multiple DIR outputs for each patient that each correspond to a different trade-off between the objectives -- has additional desirable properties from a clinical use point-of-view as compared to providing a single DIR output. The experiments also show that the proposed MO DIR approach provides a better spread of DIR outputs across the entire trade-off front than simply training multiple neural networks with weights for each objective sampled from a grid of possible values.

Main paper: 
 introduction: 
 Deformable image registration (DIR) refers to the task of finding a non-linear transformation that aligns two images. The non-linear transformation is characterized by a deformation vector field (DVF), that maps each location in the target image (also referred to as fixed or reference image) to a location in the source image (also referred to as moving image). The source image is then warped by resampling from the mapped locations. Some of the potential applications of DIR in medical imaging are dose accumulation in radiation treatment, contour propagation, tumor growth tracking, and creating a digital atlas (Mohammadi et al., 2019; Rigaud et al., 2019; Zhao et al., 2022; Salehi et al., 2022) .DIR involves optimization of a parameterized DVF to maximize the similarity between two images. However, optimizing only for maximizing image similarity may yield a highly irregular or sometimes physically implausible DVF due to model overfitting. Therefore, an additional objective penalizing irregularity in the DVF is often used, which inherently conflicts with the objective of maximizing image similarity (Li and Fan, 2018; Balakrishnan et al., 2019; De Vos et al., 2019) . Further, an additional guidance objective (either maximizing the similarity between organ contours or minimizing the distance between corresponding landmarks) is often utilized in challenging DIR problems (Balakrishnan et al., 2019; Hering et al., 2021) . Intuitively, improvement in the additional guidance objective should always lead to improvement in the image similarity objective. However, in practice, the additional guidance objective may still conflict with the image similarity objective. This is often caused when the optimization gets overfitted to the regions where additional guidance is provided, deteriorating performance in other image regions (Balakrishnan et al., 2019) . Another cause for conflict between the image similarity objective with the additional guidance can be the uncertainty in the additional guidance, which, in turn, could be caused by either inter/intra-observer variance in case of manual annotation or modeling error in case of automatic generation of additional guidance. Therefore, DIR is essentially a multi-objective (MO) problem (Deb et al., 2016) , which involves two or more conflicting objectives. This implies that fundamentally an MO approach is appropriate for DIR, where multiple DIR outputs corresponding to a diverse range of trade-offs between the conflicting objectives are provided to the clinicians to a posteriori choose the best solution. Although the notion of DIR being multi-objective is well accepted and discussed, not many DIR approaches have been developed with this perspective. Alderliesten et al. (2015) provided a proof-of-concept study for MO DIR of 2D images. Pirpinia et al. (2017) used an evolutionary algorithm to tune the corresponding weights of different objectives for each 3D breast MRI pair and run single objective DIR multiple times. Nakane et al. (2022) formulated DIR as MO problem by partitioning the template image into several overlapping regions. Andreadis et al. (2023) presented the first integral approach to MO DIR that could be used for 3D volumetric scans using an MO optimization algorithm.With the advent of deep learning in the past few years, multiple deep learning based DIR approaches have been proposed (Balakrishnan et al., 2019; de Vos et al., 2017; Li and Fan, 2018; Li et al., 2022; Salehi et al., 2022; Rigaud et al., 2019) , which provide the possibility to predict the DVF for an entire volumetric scan within seconds. However, to the best of our knowledge, there is no work done in the direction of MO DIR using deep learning. In this paper, we fill this gap and provide a novel approach for MO DIR using deep learning. To this end, we employed a well-known deep neural network for DIR, VoxelMorph (Balakrishnan et al., 2019) , and combined it with a recently proposed technique for training neural networks multi-objectively (Deist et al., 2023) . Our main contributions are the following:• We develop a deep learning based approach for MO DIR so that multiple DIR outputs corresponding to different trade-offs between multiple objectives can be presented to the clinical experts for a posteriori decision-making (Hwang and Masud, 2012 ). • We demonstrate MO DIR for a challenging real-world registration task: DIR of female pelvic magnetic resonance imaging (MRI) scans and highlight its potential benefits. 
approach: 
 We first provide a brief background on the concepts of MO optimization that we apply to deep learning based DIR. MO optimization refers to minimizing1 a vector of n objectives simultaneously. The goal is to find a set (often referred to as 'approximation set') of p solutions that are both close to as well as diversely-spread along the Pareto front -the set of all Pareto optimal solutions in objective space. A solution is Pareto optimal if none of the objectives can be improved without a simultaneous detriment in performance in at least one of the other objectives (Van Veldhuizen and Lamont, 2000) .Our deep learning based MO DIR implementation consists of a DIR network within the MO learning framework proposed in Deist et al. (2023) . We selected VoxelMorph (Balakrishnan et al., 2019) for DIR because it is a well-known neural network for DIR. VoxelMorph uses an encoder-decoder style neural network for predicting a DVF, which is a basis for many deep learning based DIR approaches proposed afterwards. We selected the MO learning framework proposed in Deist et al. (2023) for two reasons: a) it achieves MO training of neural networks through hypervolume (HV) maximization -a process that inherently ensures Pareto optimality2 and diversity between the solutions, b) it is the only MO approach that allows training neural networks multi-objectively without a priori knowledge of the exact preference between different objectives. It should be noted that the latter is crucial in the task of DIR. This is because earlier literature suggests that the exact preference between different objectives may be different between different image pairs, which may only be known a posteriori after inspecting multiple solutions (Pirpinia et al., 2017) .In this paper, we aim to minimize p loss vectors (corresponding to p solutions or DIR outputs in the approximation set), each comprising of three losses: L ImageSimilarity , L DV F Smoothness , and L SegSimilarity . Here, for L ImageSimilarity , we used normalized crosscorrelation loss. L DV F Smoothness is the squared sum of spatial gradients of the predicted DVF in all directions, and L SegSimilarity is the Dice loss between the fixed image's organ mask and the moving image's organ mask warped by the predicted DVF (refer to Balakrishnan et al. (2019) for details). In the original formulation of MO learning in Deist et al. (2023) , p neural networks are required corresponding to p solutions in the approximation set. Due to the memory intensive nature of training a 3D DIR network, this poses a challenge due to limited GPU memory. To tackle this, we modified the original implementation by sharing the weights of the encoder between p DIR networks as shown in Figure 1 . The DIR network predicts p DIR outputs (DVFs). This is followed by calculation of p loss vectors, which are used in the MO learning framework. The parameters of the DIR network are updated using a dynamic loss formulation, that, for each DIR output is defined as:EQUATION) Where, the weights w i 1 , w i 2 , w i 3 are calculated in each iteration using HV maximization described in Deist et al. (2023) . This ensures that at the end of the training the DIR outputs (that are used to calculate the p loss vectors) are close to, and diversely distributed along the Pareto front of the three objectives.MO DIR as described above can be understood as training p DIR networks simultaneously, each with different weights for the loss terms, and the weights being selected automatically such that the HV is maximal. That said, MO DIR is fundamentally different from the traditional single DIR following hyperparameter search for the loss weights. In the traditional set up, the selection of a weight (which translate to a trade-off on the approximation front) for each loss is done a priori based on quantitative comparison of a single aggregated (on a validation set) performance metric. Whereas in MO DIR, the selection is done a posteriori by clinical experts based on qualitative evaluation of multiple criteria specific to each patient.We retrospectively used data from cervical cancer patients who received brachytherapy treatment at Leiden University Medical Center (LUMC), The Netherlands. We received 136 MRI scan pairs (along with associated contours generated for clinical use of four organs at risk: bladder, bowel bag, rectum, and sigmoid) corresponding to two fractions of brachytherapy treatment in anonymized form after approval from the medical ethics committee. The original resolution of the MRI scans was 0.5 mm × 0.5 mm × 4 mm. We resampled the MRI scans to isotropic voxel spacing of 1 mm × 1 mm × 1 mm because the convolution kernels, downsampling, and upsampling operations in VoxelMorph are symmetric. We used randomly cropped patches of size 192 × 192 × 32 as an input to the neural network. We separated the scans at patient level based on their chronological order of acquisition into train and validation (126 scan pairs), and test (10 scan pairs) splits. On the test scans, a radiation therapy technologist annotated 23 anatomical landmarks (details in Appendix B), which were selected by a radiation oncologist on the basis of their importance in brachytherapy treatment for cervical cancer patients. The placement of landmarks was cross-checked by another radiation oncologist. 
experiments and results: 
 We implemented3 our proposed approach using Python and PyTorch. The training hyperparameters were: number of solutions p = 27, initialization = Kaiming He, optimizer = Adam, learning rate (lr) = 1e -4 , number of training iterations = 20K, reference point for HV calculation = (1, 1, 1) (details in Appendix C). For each experimental setting, we trained 5 models, each corresponding to a different data split. We report their performance on the test set without model selection. To assess the DIR performance, we calculated target registration errors (TREs) of the 23 manually annotated landmarks by transforming the landmarks in the target image with the predicted DVF and calculating the Euclidean distance with the corresponding landmarks in the source image. We also calculated the percentage of voxels with a negative determinant of the spatial Jacobian of the DVF, as an indication of folding in the transformation.Contrary to traditional DIR, in MO DIR, the decision maker (in our case a clinical expert) is provided with multiple DIR solutions spread across a range of trade-offs between conflicting objectives. This is demonstrated in Figure 2 (a) . The figure shows that there are multiple possible ways to align the two images. In DIR, the solutions at the extremes of the approximation set are likely not interesting because they might be overfitted to a single objective and consequently may yield sub-optimal performance in other objectives. For example, the solution highlighted in the red frame (Output 1) corresponds to minimum L ImageSimilarity , but maximum L DV F Smoothness causing a lot of folding in the DVF. Similarly, the solution highlighted in brown (Output 5) corresponds to no deformation at all. To assist the a posteriori decision-making, such uninteresting solutions can be filtered out by setting acceptance thresholds on each objective. The region of interest in the objective (loss) space where all the acceptance criteria are met, could be considered the preferred region. In Figure 2 (a), we show this region with arbitrarily selected acceptance thresholds (L ImageSimilarity < 0.55, L DV F Smoothness < 0.1, and L SegSimilarity < 0.025).Within a preferred region of interest, one solution cannot be selected over another based on quantitative comparison of performance metrics as demonstrated in Figure 2 . The solution highlighted in green (Output 2) has minimum folding in the DVF, magenta (Output 3) has minimum mean TRE of landmarks, and blue (Output 4) has maximum Dice similarity between organ masks while other metrics are worse. While Output 2 and Output 3 have less folding in the DVF and smaller mean TRE between landmarks, the warped bladder contours (dashed cyan color) considerably deviate from the target bladder contours (solid cyan color) as compared to Output 4. This is due to MO training of the DIR neural network, which ensures that the obtained DIR solutions are all (close to) Pareto optimal i.e., no solution is better than another in any objective without a simultaneous detriment on other objectives. In such a scenario, the most appropriate DIR output can only be selected after visual inspection of the DIR outputs in the preferred region of interest and considering other clinical criteria. For example, the visual inspection of the DVF from Output 4 may reveal that the folding occurs in regions not relevant for brachytherapy treatment. Further, the alignment of the bladder may be more important than the alignment of some landmarks in other regions. Therefore, a clinical expert may prefer Output 4 over Output 3 despite a larger mismatch between landmarks and more folding in the DVF in this test scan pair. Whereas, in another test scan pair, the characteristics of the DVF may be different and the clinical preference may be reversed. Moreover, it is already known from previous research that the weights, which translate to a given trade-off between objectives on the approximation front and the quantitative value of the performance metrics are different in different scan pairs (Pirpinia et al., 2017) . This means that the preferred region of interest corresponds to different solutions in the approximation sets from different scan pairs.Because multiple solutions are provided with MO DIR that are spread in objective space, the clinical expert can navigate through these solutions and select an appropriate trade-off based on the underlying clinical scenario. In contrast, with traditional single DIR, only one of these solutions is provided to the clinical expert. Therefore, the opportunity to evaluate other possibilities and make an informed decision specifically tuned to each patient is lost.In the case of single DIR, a DIR network is trained multiple times with different weight combinations for each loss function following a certain strategy. The weights yielding the best aggregated performance on a validation set are used for final training. In MO DIR, multiple neural networks (in our case a single DIR network with multiple decoders) are trained. Therefore, the training overhead of MO DIR in terms of runtime is similar to that of single DIR. However, in MO DIR, the training is done in parallel, requiring more memory. In our implementation, training for p = 27 required ∼39 GB and ∼32 GB without and with a shared encoder, respectively, as compared to ∼3.5 GB required for training a single DIR network. In the proposed MO DIR, we used HV maximization to dynamically find the weights for each loss term such that the differently weighted loss training of different neural network heads yields their outputs diversely spread across the approximation front. It may be speculated that a similar diversity of outputs can be trivially obtained by training the different neural networks with uniformly distributed weights for different losses. Such an approach is called 'linear scalarization '. Deist et al. (2023) , in their paper, compared linear scalarization with HV maximization for different shapes of Pareto fronts. The authors observed that the translation of the weights to a location on the front is dependent on the shape of the Pareto front, and is as such non-trivial. To investigate this in the case of MO DIR, we compared the proposed HV maximization based MO DIR approach with linear scalarization based MO DIR. To simulate the MO DIR set up with linear scalarization, we trained the different heads of our MO DIR neural network with weights corresponding to diversely distributed points in a grid. We used 27 grid points by enumerating over all the possible combinations for w 1 ∈ {0, 0.5, 1}, w 2 ∈ {0, 0.1, 0.5, 1}, and w 3 ∈ {0, 0.5, 1} and omitting redundant (e.g., {0, 0.5, 0.5} and {0.5, 0.5, 0.5}). It should be noted that this process of selecting linear scalarization weights is already slightly better than naive linear scalarization.The approximation sets obtained from linear scalarization vs HV maximization based MO DIR are shown in Figure 3 . It is apparent upon visual inspection of the figure that even though the weights used for linear scalarization were diversely distributed, still the obtained solutions are clustered along two edges of the expected triangle-like approximation front. There is a void of solutions in the center region of the expected triangle-like approximation front. This observation corroborates the results in (Deist et al., 2023) -the diverse spread of solutions across the approximation front cannot be obtained trivially through linear scalarization -in the case of DIR as well. In contrast, visual inspection of the solutions in the approximation set obtained using HV maximization based MO DIR, shows a rough triangle-like shape with diversely distributed points in the center as well. This is because HV maximization ensures not only proximity to the Pareto front but also diversity across the approximation front. 
conclusions and discussion: 
 We propose the first deep learning approach for MO DIR, which provides multiple DIR solutions diversely spread across the trade-off front between conflicting objectives. With such an approach, clinicians can evaluate multiple DIR solutions that are of potential interest and select the preferred one according to patient-specific and/or treatment-specific clinical criteria. While the prospect of clinicians having to review multiple DIR solutions may seem burdening, in a previous study using a dedicated user interface to navigate MO DIR solutions obtained through optimization (as opposed to deep learning as in this paper), clinicians were positive, considering the use of MO DIR to be insightful (Pirpinia et al., 2018) . We also demonstrated that a diverse spread of solutions across the approximation front such as obtained by the proposed MO DIR approach can not be trivially obtained by linear scalarization with diversely distributed weights. Although the potential utility of deep learning based MO DIR is evident from experimental results, the presented work is still only a proof-of-concept. Some of the limitations, open questions, and possible future research directions are as follows:• HV maximization provides a straightforward way to distribute the solutions diversely on the approximation front without requiring any manual tuning. In future work, it would be interesting to investigate the use of the weighted HV (Zitzler et al., 2007) metric in MO DIR to steer the solutions to a desired region (if such a region can be defined clearly a priori). It is also important to investigate which part of the approximation front is more desired by involving clinicians as a posteriori decision-makers. • In Figures 2 and 3 , the solutions seem more clustered in the region where L ImageSimilarity and L SegSimilarity are large and L DV F Smoothness is small. This could be because solutions in this region of the front are easy to obtain due to no or little deformation, or because of the corresponding shape of and local density along the Pareto front. It is known that setting the reference point differently can impact this (Ishibuchi et al., 2018) (also see Appendix C). It is interesting to investigate this further in the future. • In our proof-of-principle, we made certain choices e.g., number of objectives, number of solutions in the approximation set, type of additional guidance, type of neural network for DIR, in an effort to create a baseline deep learning based MO DIR approach. That said, the current approach leaves multiple improvement possibilities open in order to realize the complete potential of the MO perspective for DIR. For example, it can be improved by using a more sophisticated neural network for DIR, multi-resolution registration, constraints on tissue types, and diffeomorphism. All of these aspects are independent from the general idea and framework proposed in this paper. • The presented MO DIR work provides more insights than traditional single DIR approaches by showcasing the trade-offs between different objectives and how these trade-offs differ between scan pairs. However, the objectives are still average values per pair of scans. Practically, the DIR performance will likely not be uniform across the entire scan. Additionally, it is possible that clinically a solution in the vicinity of a provided discrete solution on the approximation front is more desired. It is therefore essential to research in the direction of intuitively visualizing the DVFs and navigating across (and in the local neighborhood of) different solutions. also report the associated folding in the DVF of the selected DIR solution. Although it is difficult to derive any clinical conclusions without inspecting the underlying DVFs, it can be observed that both linear scalarization and HV based MO DIR find quantitatively similar trade-offs between the best TRE values and associated DVF folding. This is not entirely surprising, given that the underlying DL architecture for DIR is the same for both methods.One might notice a trend of higher TRE values and lower image folding in the selected solutions from HV maximization based MO DIR. However, it is important to realize that the training approach may play a role in this and that training for MO DIR and linear scalarization proceeds differently. Training neural networks with HV maximization is more complex as compared to using fixed weights as in the case with linear scalarization. This is because of the dynamically changing gradients for each network head as a consequence of the HV maximization goal. Therefore, if the exact weights corresponding to the desired trade-off between each objective are known a priori, linear scalarization may yield non-dominated solutions faster. For a fair comparison, we trained the networks in both the linear scalarization and the MO DIR approach with the same number of iterations. It may be possible that this was not the saturation point for both procedures. Ideally, upon saturation, we would expect both linear scalarization and HV maximization to obtain solutions with the same proximity to the Pareto front. However, obtaining the same diversity of solutions (for a given p) along the front is not guaranteed for linear scalarization. As demonstrated in section 3.2, this is because the translation from scalarization weights to a well distributed set of solutions along the approximation front is not trivial. Therefore, achieving a diverse spread of solution through linear scalarization would require trying many more combinations. On the other hand, with the HV maximization based MO DIR approach, it can be achieved in a single go.  
 
 Appendix: In this paper, we assume minimization as objectives correspond to losses in deep learning.If HV is maximal, all the solutions are Pareto optimal.The implementation is available at https://github.com/monikagrewal/DL-MODIR/tree/public. 


Decision: Reviewers’ questions were mainly about details on experiments and were well addressed in the authors’ feedback. Reviewers acknowledged that the paper had been improved during the discussion, and their questions were answered.

Reviews: review: paper_summary: paper_summary: The authors proposed to combine a multi-objective training approach with deep learning-based deformable image registration. Evaluated on a MRI dataset, the proposed method outperforms single registration network and grid search of possible loss weights.
main_review: main_review: I appreciate that the authors addressed my concerns in the previous cycle, and the paper is in a richer context now. However, I am still a bit concerned about whether the manual selection in the provided solution set will lead to better outcomes in various complex clinical settings. A clinician reader study reflecting the preferences will be of great interest in future work. Also, if the "better similarity" or "better smoothness" can be achieved through a tuneable prompt to further automate the framework. Thus I am increasing the rating to borderline accept but will agree with the majority
strength_weakness: strength_weakness: The comparison of MO DIR with single DIR was only through the spread of the solution set, but wasn't through an implementation with another single DIR.

Though the multiple neural networks share the same encoder, there might still be significant memory consumption increase to train p decoders (e.g. 27 decoders with the hyperparameter they selected). A comparison on this will be of interest.
questions: questions: As in the previous sections, it will mainly be a comparison with single instance benchmark and the memory/time consumption.
paper_summary: paper_summary: The authors adapted a deep learning method specifically for multi-objective deformable image registration to medical imaging setting. The authors tested the MO DIR framework with pelvic MRI images annotated for radiation oncology treatment for cervical cancers. The authors compared output of MO DIR vs that of single DIR.
main_review: main_review: I appreciate the authors' update on the paper that address most of my questions and suggestions, and their detailed reply to other reviewers as well. I think the updates address some of my technical questions and I think the authors could improve on novelty of their technical approach in future research work.
strength_weakness: strength_weakness: How MRI scans are sampled and how the images are separated into train/validation/test may impact results. This may warrant additional discussion and/or experiments in addition to what is being said in the paper.

For the results section, why presentation of different solutions matters in this application scenario seems unclear and it seems unclear why MO DIR is better than single DIR.
questions: questions: How MRI scans are sampled and how the images are separated into train/validation/test may impact results. This may warrant additional discussion and/or experiments in addition to what is being said in the paper.

For the results section, why presentation of different solutions matters in this application scenario seems unclear and it seems unclear why MO DIR is better than single DIR.
paper_summary: paper_summary: This paper introduces the application of multi-objective optimization for unsupervised image registration, tackling the issue of conflicting objectives encountered during the training of registration networks like Voxelmorph and its variants. Traditionally, these methods strive to maximize visual similarity while ensuring the displacement field remains smooth and invertible. Additional objectives might include matching labels or keypoints, where optimizing for one objective could lead to suboptimal results for others. The authors propose utilizing hypervolume maximization to generate a set of solutions that represent a balanced trade-off among all objectives. This is achieved by modifying the standard Voxelmorph architecture with a shared encoder and separate decoders for each solution. The optimization of each solution targets a weighted sum of intensity similarity loss, label similarity loss, and smoothness loss, with the weights themselves being the focus of hypervolume maximization. The paper evaluates this approach by optimizing across 27 possible outcomes and compares the merits of this method against traditional hyperparameter optimization.
main_review: main_review: Thank you very much for revising the manuscript and addressing my questions. The paper has noticably improved. However, in my opinion, the paper lacks sufficient novelty so I will remain with my previous decision of 'Borderline'.
strength_weakness: strength_weakness: The paper falls short in detailing the hypervolume optimization process, which is central to the novelty of the proposed method. The description of the network architecture, while necessary, does not contribute significant novelty and overshadows the more critical aspects of the optimization process. A deeper dive into the optimization challenges, including memory demands and computational complexities, would have been valuable.
questions: questions: Could you clarify if an evolutionary algorithm is utilized for the hypervolume maximization?
How does the quality of the "best" solution compare to the "worst," and are there strategies in place to eliminate less promising solutions to reduce the manual review burden of potential candidates?
paper_summary: paper_summary: This paper proposes to combine an approach for multi-objective training of neural networks with image registration. This can be understood as a new way of finding an optimal set of hyper-parameters weighting the different loss terms during the registration.

The authors evaluate their proposed method on 10 MRI scans of the pelvis which includes segmentation of four organs (bladder, bowel bag, rectum, and sigmoid). Furthermore, 23 landmarks per scan were annotated.

The results suggest that the presented MO approach gives comparable results to the baseline but seem not yet to work as hoped.
main_review: main_review: After reviewing the rebuttal, I have decided to adjust my rating to a weak accept. The quality of the paper has been enhanced through the authors' efforts to discuss and address my concerns. Given the improvements and the intriguing nature of the work, I believe this paper merits further discussion at the MIDL conference.
strength_weakness: strength_weakness: -	The paper is missing related work in the field of automatic hyper-parameter search for image registration
o	https://arxiv.org/pdf/2101.01035.pdf
o	https://arxiv.org/pdf/2106.12673.pdf
o	Several conventional methods
-	The experimental settings are not clearly reported which makes it hard for the reader to judge the contributions.
-	The TRE does not improve much during the registration suggesting that for both settings (grid search and MO), the registration does not work well (average before 5.588mm, after grid search: 5.164mm, after MO: 5.274mm. The proposed method is even worse than the baseline method. At least if I haven’t done a mistake calculating this, because the authors do not provide the average or median over all cases which makes it even harder for the reader to compare the two methods.
-	 Furthermore, the authors report the “Minimum mean TRE of 23 anatomical landmarks in mm” over 5 different models. This does not make sense since the minimum can be achieved from different methods. (see detailed comments)
-	The authors claim a few things which are not supported by the presented experiments. 
-	The discussion is quite limited and should be extended on limitations and failure cases.
questions: questions: - Please improve the description of the experiments such that it is clear.
- Please rework the presentation of the results and provide not the minimum over 5 models but the average and explain how they are chosen. 
- Please add the average of over all 10 cases. Currently the grid-search method achieves better results which are not clearly visible to the reader because first the average has to be computed. 
- Discuss the results in a fair and open way. What can the reader still learn from this paper? What are limitations?

Comments: title: Thanks for the discussion. We made adaptations in the paper and address the points in comment.
Thanks for your feedback. Below we provide clarifications for better understanding. We also adapted the paper for clarity of presentation.
Thanks for your feedback. We have addressed the comments point-by-point and added more details in the paper.
Thanks for your feedback. We have addressed the comments point-by-point and added more details in the paper.
Thanks for your feedback. We have addressed the comments point-by-point and added more details in the paper.
General response to reviewers
comment: Thank you very much for revising the manuscript and addressing my questions. I have no further questions at this time.
We thank the reviewer for their feedback regarding improving the manuscript. Following the reviewer's suggestion, we have added a paragraph in the paper (please refer to page 4, paragraph 2), which provides more details on MO DIR and how MO learning distinguishes from hyperparameter search.

**Regarding MO learning adapting to criteria beyond the initial training objectives**:

The reviewer’s suggestion appears to point in the direction of MO DIR in an adaptive or active learning manner. We have no particular solutions for this at this time. However, we can speculate that an iterative learning process, wherein, the decision makers provide feedback on what additional objective might be of interest after evaluating the results from first MO DIR training, might be possible. Consequently, a new DIR network can be trained using MO learning with the additional objective suggested by the decision-makers.
I extend my gratitude to the authors for their comprehensive response, which has provided a clearer understanding of the paper's objectives. While I recognize the challenges posed by the page limit constraints, I recommend that the authors contemplate integrating these detailed explanations into the manuscript itself. Enhancing the clarity of the paper's objectives in this manner would greatly facilitate readers' comprehension and improve the paper's accessibility.

Still, I would like to inquire about the authors' perspective on a particular aspect of the study: In the multi-objective (MO) training process, specific objectives are employed to construct the Pareto front, guiding the solutions predominantly toward these goals. The authors mention that MO learning allows clinicians to make an "optimal" choice based on additional clinical criteria not included in the direct image registration (DIR) process. My question concerns the extent to which the solutions generated are independent of the predefined objectives.

To illustrate, if the training only utilizes segmentation masks and aims at objectives like plausibility and alignment with these masks, it's natural to expect that the MO solutions primarily address these factors. However, a clinician might be interested in aligning particular landmarks or structures not considered during the training phase. While it seems that the MO approach can partially accommodate such needs, I believe it would be valuable to explore how the method performs in adapting to criteria beyond the initial training objectives in greater depth.
> *Comparing MO learning with hyperparameter optimization*

We thank the reviewer for elaborating on their perspective. We agree that for some cases, there are clear similarities between hyperparameter search and MO learning. For example, we agree with the reviewer’s sentence `This is not a search for a single optimal parameter set, but rather an exploration of a spectrum of viable alternatives from which clinicians can choose, tailored to their specific requirements.` And the explanation afterward.

That said, we see the sentence `One of the advantages of MO learning, in my view, is its ability to make an "optimal" selection and thus avoid the limitations that might result from a less careful selection of weights.` differently. We would rather prefer to say that `MO learning enables the clinicians to make an “optimal” selection based on other clinical criteria that were not part of DIR.` In other words, in MO learning, the decision-maker is the clinical expert , who makes a final selection a posteriori using qualitative metrics. On the other hand, in hyperparameter search, selection is done a priori based on quantitative comparison of a single aggregated (on a validation set) performance metric.

> *how have you demonstrated that “that the MO DIR setting is more efficient than a grid search for weights for each objective in terms of spread of solutions across the approximation front.” ?*

We would refer the reviewer to section 3.2 and Figure 3 of the updated paper. We have adapted the text for more clarity of presentation.

> *inclusion of the average TRE for all 10 cases in table 1*

We have adapted Table 1 according to the suggestion. However, we would like to point out that the key motivation behind MO DIR is to provide a DIR solution separately tuned to each patient-specific criterion. With this perspective, an average for all 10 cases is not necessarily the most informative. Moreover, since the underlying registration model is the same for the linear scalarization and the HV maximization based MO DIR approach, it is not necessarily to be expected that the final TRE will be much different. 

Because TRE values are generally of interest for DIR, we presented the TRE table as a sanity check. With the TRE table in appendix D and experiment in section 3.2, we want to demonstrate that nothing changes in terms of TRE values by doing MO DIR using HV maximization instead of using linear scalarization. The only thing that changes is that we obtain diversity in solutions across the front in a more straightforward manner, which with linear scalarization would require trying many more combinations.

--------------------------------------------------------------

We apologize that we missed addressing the following two points from the detailed comments section in the original review. We address them in the updated version.

> *It may also be that the choice of weights (only 3 possibilities, one of which is 0) is not a good choice, which leads to this happening.*

We want to point out that the number of possibilities were chosen such that the total number of combinations after omitting redundant combinations is equivalent to $p$ (27) used for HV based MO DIR for fair comparison. $p=27$ was chosen keeping in mind that these many points would give a sufficient impression of an approximation of a trade-off front for 3 objectives. Further, choosing one of the possibilities for a weight as 0 is essential to know the extremes of the approximation set.

> *Are the different loss terms normalized to [0,1] or in which range are those? Does it make sense to have them equaly weighted?*

The $L_{ImageSimilarity}$ and $L_{SegSimilarity}$ are bound to [0, 1]. The lower bound on $L_{DVFSmoothness}$ is 0. In our experiments, we observed that the upper range of $L_{DVFSmoothness}$ was ~0.4. 

We performed some preliminary experiments on normalizing the loss terms with a motivation to bias the spread of solutions across the approximation front. We do not have any conclusive results to share from those experiments. The main reason is that the translation of normalization to a point on the approximation front is affected by the shape of the Pareto front and optimization landscape. It will be interesting to investigate this further in the future.

We interpret that by *equally weighting the losses*, the reviewer intends to say *giving equal importance to each loss*. Please correct, if otherwise. We have discussed this in the first bullet point in section 4. We also speculate that likely one objective (or a specific region of the approximation front) may be more important than others. We believe that with the use of **weighted HV** (https://link.springer.com/chapter/10.1007/978-3-540-70928-2_64), this can be achieved in the most intuitive way. It will be interesting to investigate this in the future.
Thank you for the insightful explanations and the improvements made in the rebuttal.

- I appreciate the clarification regarding the motivation behind MO Learning. I acknowledge the perspective presented, but would like to offer an alternative view where MO Learning could also be understood as a sophisticated form of hyperparameter optimization. This is not a search for a single optimal parameter set, but rather an exploration of a spectrum of viable alternatives from which clinicians can choose, tailored to their specific requirements. The core of my argument lies in the alignment of objectives with the different components of the loss function, each of which can be weighted differently. This flexibility allows for the training of different models that emphasize different aspects such as image similarity or the importance of segmentation masks. By presenting these different models to physicians, they are able to choose the configuration they find most effective. One of the advantages of MO learning, in my view, is its ability to make an "optimal" selection and thus avoid the limitations that might result from a less careful selection of weights. 
With this in mind, I would argue that MO Learning can be understood as an innovative approach to hyperparameter selection.

   I would be grateful for your thoughts on this perspective. In particular, I am curious about your reasons why you see this view as potentially flawed. Please note that in the original review I said that MO can be **understood** as a hyperparametric method, not that it is one :)

- *It is unfortunate that our main message was not clear from the presented experiments. We have modified the presentation of results based on feedback from other reviewers. We are happy to make further changes if the reviewer could be more specific.*
The points are mentioned in detailed comments of the original review.

- For example, how have you demonstrated that
“that the MO DIR setting is more efficient than a grid search for weights for each objective in terms of spread of solutions across the approximation front.” ? 

   It's conceivable that I may have misinterpreted the experimental design or findings, but the evidence supporting this assertion wasn't immediately apparent to me. Could you please clarify this?

- Lastly, I would kindly request the inclusion of the average TRE for all 10 cases in table 1 to provide a more direct and comprehensible comparison with the grid search method's outcomes.
We would like to point out what appears to be a misunderstanding of the paper by the reviewer. The key idea of the paper is to provide an approach to perform deep learning based DIR multi-objectively. The proposed approach is *NOT a way of finding an optimal set of hyper-parameters weighting the different loss terms*. In fact, the premise of the paper is: a) DIR is inherently multi-objective, b) *there is no optimal set of hyper-parameters weighting the different loss terms* that is optimal for each test scans pair (https://pubmed.ncbi.nlm.nih.gov/28436922/).

We made following changes in the paper to provide better understanding and avoid confusion.

- We understand that the MO approach is new to machine/deep learning. Therefore, we have provided a brief background on the MO concepts in section 2 paragraph 1 of the paper. We also added more references.

- We realized that the use of the term `grid search` could have caused confusion that the paper is related to hyperparameter search. To avoid confusion, we have changed it to `linear scalarization`, which is more commonly used in MO literature.

- The main aim of MO DIR is to provide multiple solutions, each corresponding to a different trade-off between objectives, which translates to trade-offs between performance metrics. We argue, and demonstrate how TRE or any other single performance metric is not the full story. There is always a trade-off to make, and the trade-off can only be made after visual inspection of several possible trade-offs, hence the need for multiple outputs instead of one. That said, we did not aim to achieve a better TRE. In fact, we nowhere claim a better performance in terms of improving TRE values. Neither do we create a doubt that the reported TRE values cannot be obtained by traditional methods. It is furthermore important to realize that the maximum possible performance that can be obtained (in terms of TRE) is bound by the underlying DIR DL architecture (VoxelMorph), which is the same for both MO DIR settings. To avoid dilution of attention towards TRE, we have shifted the TRE table to appendix D and focused more on the key message of our work (i.e., benefits of performing DIR multi-objectively) in the main paper. In the appendix, we provide more details regarding reporting of TRE values.

- We made changes in Figures 4 and 5 following the reviewer's suggestions.


### Response to other points

>*The authors claim a few things which are not supported by the presented experiments.*

It is unfortunate that our main message was not clear from the presented experiments. We have modified the presentation of results based on feedback from other reviewers. We are happy to make further changes if the reviewer could be more specific. 

>The discussion is quite limited and should be extended on limitations and failure cases.

We agree that the presented discussion in the paper does not cover all the nuances of the field. As stated in the paper, the idea of MO is new to the field of DL (and DL-based DIR) and the presented paper is the first proof-of-principle. Out of 8 pages, one page is dedicated to the discussion section, in which we feel we addressed the most important topics.

>*... Often it is not obvious which displacement field is optimal, especially not to a clinician who does not understand the technical background.*

The reviewer is right to point out that in clinical setup, the metrics like TRE can not be provided, only the information on the objectives considered for learning such as image similarity and deformation magnitude (and foldings) can be provided. We argue that the selection of the appropriate solution is not possible by quantitative comparison, the decision maker does not require technical knowledge, but domain expertise on what matters more in a clinical scenario. For example, does alignment of bladder matter more or alignment of a femur head. The clinicians know the answer to such questions best.

>*What does it mean that an algorithm is multi-objective?...*

In this sentence, we intend to say that not many existing DIR algorithms provide multiple DIR outputs diversely spread across the trade-off front between the conflicting objectives.

> *What can the reader still learn from this paper? What are limitations?* 

The main take-home message from the paper are: 

a) as was already known, MO DIR provides a possibility to a posteriori select a suitable trade-off between the conflicting objectives, specifically tuned to each test case scenario, but now we can also achieve this through deep learning.

b) in the (deep) learning setting, MO DIR can not be achieved trivially through linear scalarization with uniformly distributed weights.

We have discussed limitations of our work in section 4 as much as the page limit allowed in the paper.
## Weaknesses

>*The paper falls short in detailing the hypervolume optimization process, which is central to the novelty of the proposed method.*

We would like to point out that we did not develop the hypervolume optimization process as a part of this work. We use our previously developed approach ([Deist et. al,](https://dl.acm.org/doi/abs/10.1007/978-3-031-27250-9_8)) to train a DIR neural network in this paper. The main novelty of our paper comes from presenting a proof-of-principle approach to achieve MO DIR using deep learning. We refer the interested reader to Deist et.al, for details on hypervolume optimization.

> *A deeper dive into the optimization challenges, including memory demands and computational complexities, would have been valuable.*

We thank the reviewer for their feedback. We have added section 3.1.1 on comparing the memory and computational overhead of our MO DIR approach compared to the single DIR setting.

## Detailed Comments
> *The paper would benefit from a more detailed explanation of the hypervolume maximization process, including its implementation and integration with the modified network architecture.*

We would like to point out again that a detailed explanation of hypervolume maximisation is beyond the scope of this paper. We refer the reviewer to the original work on the use of hypervolume maximization for training neural networks ([Deist et. al,](https://dl.acm.org/doi/abs/10.1007/978-3-031-27250-9_8)) for a detailed understanding. In section 2, paragraph 3, we explain the integration of HV maximization based training with a DIR neural network, which is the main contribution of this paper.

## Questions To Address In The Rebuttal

>*Could you clarify if an evolutionary algorithm is utilized for the hypervolume maximization?*

We did not use an evolutionary algorithm for hypervolume maximization. We used a gradient based approach, in which, the hypervolume maximization of a set of neural networks can be reformulated as training the neural networks with a dynamically weighted loss. The dynamic weights are the gradients of hypervolume in loss space. We refer to the original article ([Deist et. al,](https://dl.acm.org/doi/abs/10.1007/978-3-031-27250-9_8))  for further details.

>*How does the quality of the "best" solution compare to the "worst,".*

There are no “best” or “worst” solutions that can be decided quantitatively. All the solutions are, in principle, (close to) Pareto optimal, which means no solution is better than another solution in one objective without a simultaneous detriment in other objectives.
In the paper, we say “best” solution with respect to a specific criterion i.e., TRE, percent folding, or matching of segmentation masks. We have updated the experiment section of our paper. We hope that it presents the results more clearly than before.

> *are there strategies in place to eliminate less promising solutions to reduce the manual review burden of potential candidates?*

We do not have such a method in our paper. However, we can potentially a posteriori eliminate a large section of the approximation front corresponding to solutions at the extremes i.e., the solutions corresponding to very high DVFSmoothness loss, or ImageSimilarity loss, as they are extreme and therefore unlikely to be of interest. We have demonstrated this in Figure 2 of the updated paper.
> *How MRI scans are sampled and how the images are separated into train/validation/test may impact results.*

We thank the reviewer for pointing out this shortcoming in the paper presentation. The reviewer is right to speculate that MRI sampling may impact the results because CNN kernels and down/up-sampling techniques are all isotropic. To avoid any bias between processing of transverse vs in-plane direction, we resampled the MRI scans to isotropic voxel spacing. We have added this information in section 2.1 of the updated paper.

Further, the separation of train, validation, and test scans was done as follows. First, the test scans were separated by sorting the scans in chronological order and taking the latest scans as test scans. The remaining scans were split into 5 folds by random sampling. This was followed by training 5 models, each utilising 4 splits for training and the 5th split for early stopping. The mean and standard deviation of the performance of these models on the test data is reported without doing model selection. We have added this information in section 3, first paragraph of the updated paper.

> *For the results section, why presentation of different solutions matters in this application scenario seems unclear and it seems unclear why MO DIR is better than single DIR.*

It is unfortunate that the presented results fall short of conveying the key message to the reader. We have updated the experiments section, and modified Figure 2 for a clearer presentation. Briefly summarized, we want to demonstrate that in DIR different competing objectives are of interest. Especially if the DIR task is challenging, the conflict between the objectives is large and many different solutions (deformation vector fields) exist that are equally good according to these criteria and adhering to a multi-objective notion of not being able to a priori weight the importance of these objectives. For instance, a deformation that achieves a better image similarity objective may have unrealistic deformations. The appropriate selection can only be made through visual inspection of potential DIR solutions by a clinical expert, who can also take into account other clinical criteria that are not part of DIR. Moreover, what is the preferred solution also depends on what you want to use the DIR output for. In single DIR, because only one DIR output corresponding to a pre-selected trade-off is provided to the clinician, the opportunity to make patient specific choices is lost.
## Concerns in Weaknesses

> *The comparison of MO DIR with single DIR wasn't through an implementation with another single DIR.*

We would like to remark that MO DIR with $p$ solutions is equivalent to presenting outputs from $p$ single DIR (each corresponding to a different trade-off between objectives) at once. Therefore, we believe that comparison with a single DIR instance would be redundant as each of the $p$ outputs already represents the output from a single DIR run.

A follow up question may arise as to why we didn't just train a DIR network $p$ times, each time with a different weight for each objective. The answer is that the weight given to an objective during training, does not translate trivially to a trade-off on the front after training. It depends on the shape of the front ([Deist et. al,](https://dl.acm.org/doi/abs/10.1007/978-3-031-27250-9_8)), which is not known for DIR or most of other MO applications a priori. We experimentally investigate this in our paper in section 3.2.

Further, we have updated Figure 2 and section 3.1 of the paper to present the comparison of MO DIR with single DIR more clearly. 

> *there might still be significant memory consumption*

We thank the reviewer for their feedback. We have added a paragraph (section 3.1.1)  to the paper to compare the memory overhead of training a MO DIR approach as compared to a single DIR approach. Briefly, training the proposed MO DIR with p=27 requires ~32GB GPU RAM as compared to ~3.5GB for training a single DIR, which is significantly more but in our opinion well within the limits of modern GPUs used for training deep neural networks.

## Detailed Comments

> *After getting the solution set, the final registration result will be manually selected by the decision maker. Will this make the method less automated and requiring additional intervention?*

Yes, from this perspective, MO DIR is indeed a less automated approach than single DIR, which is intentional. The key motivation of MO DIR is to have better potential of adoption in challenging DIR scenarios where the trade-off between the conflicting objectives can not be known a priori. Moreover, the desired trade-off between conflicting objectives is different between patients and may be dependent on clinical criteria not considered during DIR. In such DIR scenarios, the above mentioned reasons necessitate a posteriori evaluation of multiple DIR solutions and manual intervention for decision-making. We have laid out this motivation in the introduction section of the paper. Further, the task of DIR in the pelvic region, which has been selected for experiments in this paper, is an example of such a challenging DIR scenario.

We would also refer to a previous study that provided a dedicated user interface to navigate through the resulting solutions from MO DIR (https://pubmed.ncbi.nlm.nih.gov/30840735/). In this study the clinicians considered MO DIR insightful. We have added discussion on this in section 4, paragraph 1 of the updated paper.
We thank the reviewers for taking the time to review our work and provide feedback. We also thank the reviewers for noticing the novelty and appreciating the innovativeness of the approach, and presentation of the paper. We have made adaptations to our paper to address the points raised by reviewers. Below we provide a point-by-point response to the reviewers’ feedback.