Title: One-to-Multiple: A Progressive Style Transfer Unsupervised Domain-Adaptive Framework for Kidney Tumor Segmentation
Venue: NeurIPS 2024
Accepted: yes
Keywords: Unsupervised Domain Adaptation

Multi-sequence MRI Segmentation

One-to-Multiple

Progressive Style Transfer
Abstract: In multi-sequence Magnetic Resonance Imaging (MRI), the accurate segmentation of the kidney and tumor based on traditional supervised methods typically necessitates detailed annotation for each sequence, which is both time-consuming and labor-intensive. Unsupervised Domain Adaptation (UDA) methods can effectively mitigate inter-domain differences by aligning cross-modal features, thereby reducing the annotation burden. However, most existing UDA methods are limited to one-to-one domain adaptation, which tends to be inefficient and resource-intensive when faced with multi-target domain transfer tasks. To address this challenge, we propose a novel and efficient One-to-Multiple Progressive Style Transfer Unsupervised Domain-Adaptive (PSTUDA) framework for kidney and tumor segmentation in multi-sequence MRI. Specifically, we develop a multi-level style dictionary to explicitly store the style information of each target domain at various stages, which alleviates the burden of a single generator in a multi-target transfer task and enables effective decoupling of content and style. Concurrently, we employ multiple cascading style fusion modules that utilize point-wise instance normalization to progressively recombine content and style features, which enhances cross-modal alignment and structural consistency. Experiments conducted on the private MSKT and public KiTS19 datasets demonstrate the superiority of the proposed PSTUDA over comparative methods in multi-sequence kidney and tumor segmentation. The average Dice Similarity Coefficients are increased by at least 1.8% and 3.9%, respectively. Impressively, our PSTUDA not only significantly reduces the floating-point computation by approximately 72% but also reduces the number of model parameters by about 50%, bringing higher efficiency and feasibility to practical clinical applications.

One-to-Multiple: A Progressive Style Transfer Unsupervised Domain-Adaptive Framework for Kidney Tumor Segmentation

Abstract

In multi-sequence Magnetic Resonance Imaging (MRI), the accurate segmentation of the kidney and tumor based on traditional supervised methods typically necessitates detailed annotation for each sequence, which is both time-consuming and labor-intensive. Unsupervised Domain Adaptation (UDA) methods can effectively mitigate inter-domain differences by aligning cross-modal features, thereby reducing the annotation burden. However, most existing UDA methods are limited to one-to-one domain adaptation, which tends to be inefficient and resourceintensive when faced with multi-target domain transfer tasks. To address this challenge, we propose a novel and efficient One-to-Multiple P rogressive S tyle T ransfer U nsupervised D omainA daptive (PSTUDA) framework for kidney and tumor segmentation in multi-sequence MRI. Specifically, we develop a multi-level style dictionary to explicitly store the style information of each target domain at various stages, which alleviates the burden of a single generator in a multi-target transfer task and enables effective decoupling of content and style. Concurrently, we employ multiple cascading style fusion modules that utilize point-wise instance normalization to progressively recombine content and style features, which enhances cross-modal alignment and structural consistency. Experiments conducted on the private MSKT and public KiTS19 datasets demonstrate the superiority of the proposed PSTUDA over comparative methods in multi-sequence kidney and tumor segmentation. The average Dice Similarity Coefficients are increased by at least 1.8% and 3.9%, respectively. Impressively, our PSTUDA not only significantly reduces the floating-point computation by approximately 72% but also reduces the number of model parameters by about 50%, bringing higher efficiency and feasibility to practical clinical applications. 2

1 Introduction

Kidney tumor segmentation is a crucial task in medical image analysis, playing an essential role in the diagnosis, staging, and treatment of kidney cancer [1, 2, 3]. Previous studies have primarily focused on computed tomography (CT) images [4, 5, 6, 7, 8, 9], but in recent years, magnetic resonance imaging (MRI) has emerged as a safer alternative due to its non-radiative nature [10, 11, 12]. Compared to CT,

MRI offers superior contrast for characterizing kidney tissue and tumors, enabling a clearer distinction between normal and pathological tissues. Furthermore, the multi-parameter imaging capability of MRI generates diverse sequences [13, 14, 15] that enhances the comprehensive description of pathological features. These advantages have established MRI as the preferred modality for clinicians in diagnosing kidney disease, particularly kidney tumors, and have facilitated the widespread use of multi-sequence MRI in clinical practice for kidney and tumor segmentation.

In existing studies, traditional supervised segmentation methods typically rely on a large amount of annotated data, and performing fine-grained annotations for a single sequence is already a timeconsuming and labor-intensive task, let alone for every sequence. Unsupervised domain adaptation (UDA) methods [16, 17, 18, 19, 20] have been proposed as a promising solution to address these challenges. Current UDA methods mainly focus on one-to-one domain adaptation, involving a single source domain and a target domain. Although these methods perform well in certain scenarios, they lack scalability, meaning that for multi-target domain tasks, training paired generators for each sourcetarget domain pair results in significant computational costs and resource consumption. Furthermore, one-to-one domain adaptation methods can only learn fixed mappings between two domains, thereby failing to capture the potential commonalities and connections among multiple sequences.

As an initial attempt to apply multi-domain adaptation technology to medical imaging, Xu et al. [21] proposed OMUDA for abdominal organ segmentation using multi-sequence MRI. However, the architecture of OMUDA is derived from StarGAN v2 [22], which was originally designed for style transfer in natural images and has not been specifically tailored and optimized for the needs of domain adaptation in medical imaging. Although it shows improvement in resolving domain confusion across multiple domains, this limitation leads to suboptimal generated images with regards to organ structural consistency and detail preservation. Consequently, the task of one-to-multiple domain adaptation in medical imaging remains a significant challenge, necessitating more refined and specialized approaches for further advancement.

In this paper, we propose a One-to-Multiple P rogressive S tyle T ransfer U nsupervised D omainA daptive (PSTUDA) framework, which explicitly stores multi-level style features from different domains in designated multi-level style dictionaries, thus alleviating the burden on the generator and achieving the decoupling of content features from style features. Our PSTUDA employs multiple cascaded style fusion modules to recombine multi-level style features from different domains with content features layer by layer using P oint-wise I nstance N ormalization (PIN), thereby ensuring that the generated images across multiple target domains have high-quality style and structure.

The main contributions of our work are summarised as follows:

We explore a novel and efficient One-to-Multiple P rogressive S tyle T ransfer U nsupervised D omainA daptive (PSTUDA) framework, which is capable of simultaneously transferring a single annotated source domain to multiple unannotated target domains, significantly reducing the need for tedious domain adaptation work for each target domain.

We introduce a multi-level style dictionary that stores style information for each domain at different stages of style transfer, alleviating the burden on the generator to perform multiple tasks and effectively decoupling content features from style features.

We propose a progressive style transfer paradigm and a P oint-wise I nstance N ormalization (PIN) method. The former comprises multiple cascading style fusion modules, each recombining content features with corresponding style features through PIN, thereby achieving better cross-modal alignment and structural consistency.

We construct a multi-sequence kidney tumor MRI dataset called MSKT to facilitate research on kidney tumor analysis. Quantitative and qualitative results on the MSKT and the public dataset KiTS19 show that our PSTUDA framework outperforms the state-of-the-art methods and significantly improves segmentation performance and training efficiency.

2 Related Work

2.1 Kidney Tumor Segmentation

Computer-aided diagnostic methods for kidney tumor segmentation play a crucial role in clinical practice [23, 24, 25, 26, 27]. The significant variability in the size, shape, and location of kidneys and

tumors presents a considerable challenge to accurate segmentation. Yu et al. [28] developed CrossbarNet to capture global and local features of kidney tumors through crossbar patches and focused on difficult-to-segment regions through a cascade training strategy. Myronenko et al. [29] designed a dedicated boundary branch supervised by an edge-aware loss term to enhance the consideration of organ and tumor edge information. These approaches are centered on improving the ability of the model to recognize the complex morphology of kidney tumors. To further advance this effort, Pandey et al. [30] integrated active contouring with 3D-U-Net to achieve precise delineation of kidney tumor shapes, showcasing the potential of merging deep learning with traditional image processing techniques. However, these approaches typically rely on fully supervised learning with extensive pixel-level annotations. To address this limitation, researchers are exploring alternative solutions in the fields of semi-supervised, self-supervised, and unsupervised learning. Ma et al. [31] introduced an Affinity Network that learns from limited data using k-nearest neighbors attention pooling layers. Similarly, Ciga et al. [32] and Faust et al. [33] developed methods that enhance feature learning and guide tumor analysis through self-supervised and unsupervised techniques, respectively. By reducing the reliance on annotated data, these methods not only lower costs but also improve the model's generalization capabilities.

2.2 Unsupervised Domain Adaptation

UDA is one of the important methods for addressing domain difference. It aims to transfer a model from an annotated source domain to an unannotated target domain. Existing works have mostly focused on one-to-one domain adaptation [34, 35, 36, 37, 38, 39], yielding impressive outcomes. For example, CycleGAN [34] used cycle consistency constraints to transform unpaired images from one domain to another. CyCADA [40] enforced cycle consistency by combining methods of image space alignment and latent representation space alignment. MUNIT [41] decomposed image representations into content and style codes, enabling multimodal image translation. For medical image domain adaptation, SIFA [35] achieved domain alignment from both image and feature perspectives, enabling the segmentation network to adapt to the unannotated target domain. Thereafter, DEPL [39] further improved the segmentation accuracy by employing multi-source domain extension and selective pseudo-labelling strategies.

However, these one-to-one domain adaptation methods lack scalability when handling multiple domain transfer tasks, as they can only learn the relationships between two different domains at a time. StarGAN [42] performed image-to-image translation across multiple domains using a single model, and then the improved version, StarGAN v2 [22] further enhanced diversity in generated images by introducing style codes specific to each domain. Additionally, Sharma and Hamarneh [43] proposed a multi-modal generative adversarial network that leveraged redundant information from available sequences to synthesize missing MRI pulse sequences in patient scans. Gholami et al. [44] developed an information-theoretic approach that aimed to find shared latent spaces for domain adaptation across multiple target domains. It should be noted that these methods predominantly focus on the diversity of the generated images, while paying less attention to the structural consistency before and after image translation.

2.3 Normalization of Image Translation

In UDA tasks, image normalization is a key step that can help models to learn and transfer features effectively [45, 46, 47, 48, 49, 50]. Instance Normalization (IN) [51] normalizes the features of each sample to improve the quality and realism of generated images. Adaptive Instance Normalization (AdaIN) [52] dynamically adjusts the relationship between style and content features of input images to enable a rapid transformation across arbitrary styles. Batch-Instance Normalization (BIN) [53] explicitly normalizes unnecessary style variations in images while preserving useful styles. AdaAttN [54] introduces adaptive attention normalization to optimize the effects of arbitrary neural style transfer. GramLIN [55] continuously measures the proximity of the current stylized output to the target style to achieve progressive stain transfer. In medical image domain adaptation tasks, it is crucial to maintain structural consistency during image translation, in addition to changing image styles. Therefore, we propose a novel normalization method called PIN, which progressively fuses content and style features at each pixel by considering image details and local style differences. The goal is to ensure well-structured images after transfer.

C

ye

Progressive StyleTransfer

ProgressiveStyleTransfer

V1

V2

V3

V4

Vn-1

Disc

V1

V2

V3

Style Fusion Moudle

Style Fusion Moudle

Style Fusion Moudle

Style Fusion Moudle

Style Fusion Moudle

Style Fusion Moudle

Style Fusion Moudle

Style Fusion Moudle

Encoder

Decoder

Encoder

Decoder

X'src

Xrecon

Generator

Generator

StyleFusion Mloudle

StyleFusionMoudle

H

Conv

H

PIN

0000

Relu

①→

H←

Conv

H←

PIN

Relu

④→H

W

C

W

C

Style Feature V

W

C'

Style Feature D

W

Feature Map X

Feature Map X

↑

Feature Map X,

Feature Map X"

↑

Weighted Feature

Style Dictionary 1

Conv

Style Dictionary n

Upsample

Conv

。

.

o

.

00.0

Reshape

.

.

.

.

00.

Reshape

:Skip Comnection

S2xN

.

.

Nx1

$²x1

00000

b0000

SxS

O

S²xN

.

Nx1

S²x1

+

SxS

PIN

• : Omitted Style Transfer

Point-wise Instance

Nomalization

S:Length of Style Encoding

Domain Label L

Target Style Encoding V1

Domain Label L

Target Style Encoding V

N : Number of Domains

(b)Illustration of theprogressivestyletransferprocess.

3 Method

3.1 Overview of the proposed One-to-Multiple Framework

Formally, for a one-to-multiple domain adaptation task, we have one source domain S and multiple target domains T i . The source domain S is annotated, denoted as S = { ( x j , y j ) } N s j =1 , while the target domains are unannotated, denoted as T i = { x T i j } N T i j =1 . Each domain is assigned a domain label, for example, L = [0 , 1 , ..., N ] representing the source domain 0, target domain 1, etc. , up to target domain N . Figure 1(a) illustrates our PSTUDA framework, which mainly consists of a shared generator (including an encoder, a decoder, and multiple style fusion modules) and a multiscale discriminator. X src , X trgs , and X recon represent the input image from the source domain, the generated pseudo-target domain images corresponding to the input image, and the image reconstructed back to the source domain from the pseudo-target domain images, respectively. Our task involves two stages. The goal of the first stage is to train a single generator G such that given a source domain image X src ∈ S and any target domain label l i , it can generate a pseudo-target domain image X trg _ i corresponding to the image X src , i.e. , G ( X src , l i ) ⇒ X trg _ i . The objective of the second stage is to utilize the generated pseudo-target domain images X trgs and source domain annotation y to train a segmentation network to achieve accurate segmentation of the kidney and tumor.

3.2 Multi-level Style Dictionary

Our design is inspired by the StarGAN v2 multi-domain image generation framework, which employs a Mapping Network and a Style Encoder to obtain style encodings. The Mapping Network transforms random Gaussian noise into diverse style encodings, while the Style Encoder extracts style encodings from reference images. These techniques are essential for transforming natural image styles and enhancing artistic variety. However, in medical image domain adaptation, they encounter challenges in extracting comprehensive style features. Style encodings from a single image may not capture the full range of styles within a domain, and deriving styles from Gaussian noise, while introducing

diversity, can result in instability. In medical image analysis, such instability could potentially compromise feature recognition and segmentation accuracy.

To address these challenges, we propose to define a set of learnable multi-level style dictionaries for each domain, where the style information at each level closely corresponds to the respective phase of style fusion. This multi-level structure allows for starting with basic style features and exploring more complex layers of style, thereby providing a progressively refined path for the style fusion process. During the generative adversarial process, the continuous updating and learning of the style dictionary can adapt to the ever-changing style demands. The early-stage stylized features will provide feedback for subsequent style feature learning, thus guiding them to update in a dynamic and targeted manner. By learning style information from the whole domain, the style dictionary becomes more representative, which effectively overcomes the limitation of extracting style encoding from a single image. Moreover, the iterative updates of style dictionaries enhance stability and reduce uncertainty in extracting style encodings from random noise. Storing style encoding information in a multi-level dictionary helps alleviate burden on the generator and achieves effective decoupling of content features and style features. This allows it to focus on capturing domain-invariant features such as structure and shape. In this way, the multi-levels style dictionary refines complex styles within target domains at each level, ensuring that each can timely reflect latest progress in style transfer process more precise and coordinated.

3.3 Progressive Style Transfer Paradigm

In PSTUDA, multiple cascaded style fusion modules and decoders constitute the core components of progressive style transfer. Figure 1(b) illustrates the main processes of the first and last (decoding phase) style transfer stages, with similar style transfer processes in the intermediate stages. In the first style transfer stage, the style fusion module requires two inputs: the content feature X 1 ∈ R H × W × C obtained by downsampling from the encoder, and the first-level style encoding V 1 ∈ R S × S × 1 from the target domain, matched with the current style fusion module. We utilize the domain label L multiplied by the first style dictionary to select the target style encoding V 1 . First, the content feature X 1 passes through a convolutional layer to obtain the content feature ˆ X 1 , and the target style encoding V 1 undergoes two consecutive 1 × 1 convolutions for channel transformation to obtain the style feature ˆ V 1 ∈ R H × W × C .

After obtaining the content feature ˆ X 1 and style feature ˆ V 1 , we propose a novel style fusion normalization method, Point-wise Instance Normalization (PIN), to effectively combine the two for more subtle and accurate pixel-level style transfer. The PIN can be defined as:

where Norm ( · ) denotes the Instance Normalization of the content feature ˆ X l ∈ R H × W × C at layer l . The parameters γ chw and β chw are scaling and shifting parameters specific to the target domain. These parameters are derived by applying ConvBlock and Chunk operations to the style feature ˆ V l ∈ R H × W × C and are adjusted to match the spatial dimensions of the content feature ˆ X l , thus enabling independent style transfer at each pixel. By rescaling the feature map using these parameters, style information specific to the target domain is integrated into the feature map for style transfer. PIN provides unique style statistics for each spatial point of the content feature, allowing for customized style fusion based on different regions and features of the image content. This facilitates finer local style variations and richer style details, making it particularly suitable for applications requiring precise style adjustments, such as medical image segmentation.

In the final stage of style transfer, the decoder takes the stylized content feature X n ∈ R H ′ × W ′ × C ′ from the previous layer and the last-level style encoding V n ∈ R S × S × 1 as input. To spatially match the content feature X n , the style encoding V n first undergoes upsampling through a deconvolutional layer for scale transformation, followed by two consecutive 1 × 1 convolutions for channel transformation, resulting in the style feature ˆ V n ∈ R H ′ × W ′ × C ′ . The other style fusion operations are similar to those in the first stage. There are two considerations for performing style transfer during the decoding phase. Firstly, the decoding phase is the process of image reconstruction, where integrating

CxHxW

Res Block x 4

ReLU+Conv1x1

N×H/16×W/16

DomainLabel L

1×H/16×W/16

R/F

Downsam

C×H/2×W/2

Res Block x 4

ReLU+Conv1x1

N×H/32×W/32

Domain Label L

1×H/32×W/32

R/F

ReLU+Conv1x1

Domain Label L

/MxHx

Res Block x 4

N×H/64×W/64

1×H/64×W/64

R/F

C x H/8 × W/8

Res Blockx4

ReLU+Conv1x1

N×H/128×W/128

Domain Label L

1×H/128×W/128

R/F

style features can effectively incorporate high-level style information from the target domain into the image. This helps to better express the target style while preserving content information, especially in terms of details and textures. Secondly, as the decoder is responsible for upsampling, it has the opportunity to refine and restore image details while enlarging the feature maps. Introducing style transfer at this stage ensures that these details not only conform to the content structure but also match the characteristics of the target style, thus achieving effective fusion of content and style at different scales.

3.4 Generator and Discriminator Architecture

Generator As previously mentioned, the generated pseudo-target domain images will be used in conjunction with the corresponding segmentation annotations from the source domain to train the subsequent segmentation networks. Therefore, it is crucial to maintain structural consistency during the image translation process for the success of downstream applications. The generator in StarGAN v2, due to its multiple downsampling steps, unfortunately leads to a loss of spatial information, which poses challenges in maintaining structural consistency during image translation. In light of CycleGAN's outstanding performance in one-to-one medical image translation, we adopt the approach of OMUDA by integrating the generator architecture of CycleGAN into our encoder and decoder. To facilitate a single generator to handle one-to-multiple transfer tasks, we replace the IN layers in the decoder with PIN layers. The complete generator comprises an encoder, a decoder, and a series of cascaded style fusion modules. Among them, the encoder, equipped with two downsampling layers and multiple IN layers, is tasked with extracting domain-invariant features from the input image of the source domain. The cascaded style fusion modules are responsible for integrating the style features of the target domain with the domain-invariant features of the source domain. The decoder is composed of two upsampling layers embedded with PIN and is responsible for style transfer and image reconstruction during the decoding phase.

Discriminator Inspired by Wang et al.'s work [56], we enhance the original discriminator architecture of StarGAN v2 with a multi-scale mechanism. As depicted in Fig. 2, the discriminator is composed of multiple independent discrimination branches, each containing four residual blocks [57], with each residual block having N output branches serving N specific target domains. The input to the discriminator includes not only the original image but also images processed through different scales of downsampling. The multi-branch output of the discriminator first multiplies with the target domain label L to select the discrimination output for the corresponding domain before proceeding to authenticity determination. By evaluating the generated images at various scales, the multi-scale discriminator can more comprehensively judge the realism of the images, thereby encouraging the generator to produce images with higher quality. More detailed information on the generator and discriminator architectures can be found in Appendix B.1.

3.5 Loss Function

For convenience, we give the following symbol definition. x s ∈ X denotes a source domain image, with its corresponding source domain label l s ∈ L . l t ∈ L represents any target domain label. G and D stand for the generator and discriminator, respectively. The loss functions in the generator include adversarial loss L adv , cycle consistency loss L cyc , and identity loss L idt [58]. The loss in the discriminator includes adversarial loss L adv . Among them, the adversarial loss is defined as

which encourages the generator to generate images that are indistinguishable from the target domain images by the discriminator. The cycle consistency loss is computed as

which ensures that an image translated from the source domain to the target domain can be translated back to the original image, thus preserving the structural consistency of the image throughout the translation process. The identity loss can be defined as

which is used to preserve the content of the source domain images when the generator is applied to them. The complete training loss can be summarized as follows:

where λ cyc and λ idt denote the weights for the corresponding loss terms and they are set to 10 and 1 in our experiments, respectively.

4 Experiments

4.1 Dataset

In this study, we utilize two datasets for the performance evaluation of the method. The first one is a private dataset named MSKT, which comprises 104 cases, each including four sequences: T1c, FS T2W, T2W, and DWI. The second is the publicly available KiTS19 [59] dataset. We conduct our first set of comparative experiments using the MSKT dataset. Specifically, the annotated T1c data is served as the source domain, while the unannotated FS T2W, T2W, and DWI data are considered as the target domain. For our second set of comparative experiments, we combine KiTS19 with the MSKT dataset. In this case, all training data from KiTS19 are used as the source domain, and all four sequences of the MSKT are considered as the target domain. We ensure that there is no overlap between the source domain, target domain, and test set to prevent any potential information leakage. More details about the dataset and evaluation metrics are provided in Appendix A.

4.2 Comparative Study

To comprehensively evaluate the performance of the proposed method, we compare PSTUDA with five other state-of-the-art UDA methods on the MSKT and KiTS19 datasets: CycleGAN [34], MUNIT [41], SIFA [35], DEPL [39], and StarGAN v2 [22]. Except for SIFA and DEPL, the others are limited to cross-domain image transfer. Therefore, we train a dedicated U-Net [60] from scratch based on the pseudo-target domain images generated by these methods for multiple sequences. For a fair comparison, all hyperparameters in the U-Net remain consistent. Due to the one-to-one image transfer characteristics of SIFA, DEPL, CycleGAN, and MUNIT, we train a separate model for each sequence using these methods to handle multi-sequence MR image transformations.

Due to the unpaired nature of multi-sequence MR images, it is challenging to quantitatively assess the style effects and structural consistency of the generated images. Experience suggests that if anatomical structures are distorted during the image transfer process, the pseudo-target domain images generated will not correspond with the annotations of the source domain, thereby impairing segmentation performance. Moreover, if there is a significant difference between the style of the generated images and the real target domain images, this discrepancy in data distribution may also detrimentally affect the segmentation results. Hence, the quality of the generated images is indirectly indicated by the segmentation results.

Results on MSKT Table 1 presents the Dice Similarity Coefficient ( DSC ) and 95% Hausdorff Distance ( HD 95 ) results for all methods across different MR sequences. The average DSC indicate that our PSTUDA outperforms other methods in segmentation performance on all MRI sequences. We also observe that PSTUDA significantly surpasses MUNIT, SIFA, DEPL, and StarGAN v2 from both DSC and HD 95 . Although SIFA and DEPL are more efficient end-to-end domain adaptation segmentation methods, they do not perform well in our task. The images generated by these methods exhibit structural distortions and lack naturalness (4 th and 5 th columns in Fig. 3). Particularly, DEPL

SourceT1c

CycleGAN

MUNIT

SIFA

DEPL

StarGANv2

PSTUDA

T1c

↑

FST2W

T1c

↑

T2W

T1c

↑

DWI

shows severe structural distortion of kidneys and tumors in the T2W sequence, which has greater inter-domain differences. MUNIT and StarGAN v2, while excelling in the diversity of generated images, fall short in maintaining cross-domain structural consistency, leading to distorted structures in the generated images (3 rd and 6 th columns in Fig. 3). The performance of CycleGAN is close to that of PSTUDA on FS T2W and DWI sequences, but it underperforms on the T2W sequence, as the translated images lose many important structural details, resulting in poor image alignment. In contrast, our PSTUDA prioritizes structural consistency in the cross-domain image translation process by separating content from style and progressively recombining content and style features in a point-wise manner. This approach produces images with higher fidelity, as demonstrated in the last column of Fig. 3, thereby significantly enhancing segmentation performance.

Results on KiTS19 and MSKT This experiment utilizes CT images from KiTS19 as the source domain and four MRI sequences from MSKT as the target domain. Given the different imaging technologies employed by CT and MRI, the domain difference between them is significantly greater than that between the various MRI sequences. As shown in Table 2, most comparative methods show a reduction in kidney and tumor segmentation performance on FS T2W, T2W, and DWI when compared to the first set of experiments. Although the average DSC of CycleGAN on the DWI is higher than that of PSTUDA (by about 1.9%), its performance is lower than that of PSTUDA by approximately 7.5% and 9.1% on FS T2W and T2W, respectively. Our PSTUDA almost achieves the

best average DSC results on all sequences. Compared to the first set of experiments, the performance of CycleGAN has decreased by about 6.8% and 4.4% on FS T2W and T2W, respectively, while the performance of the proposed PSTUDA has decreased by 0.5% and increased by 1.2% on these two sequences. These results underscore PSTUDA's exceptional robustness in tasks with larger domain differences. As shown in Fig. 4 in Appendix C, our PSTUDA outperforms other comparative methods in maintaining structural consistency in translated images. Moreover, thanks to the one-to-multiple features, our method also shows a significant advantage in training efficiency, which will be detailed in the subsequent section. In summary, the highlight of PSTUDA lies in its dual improvement of segmentation performance and training efficiency.

Training efficiency In this section, we evaluate the performance of PSTUDA in comparison with CycleGAN, MUNIT, and StarGAN v2 in terms of model parameters and FLOPs. As shown in Table 9 in Appendix D, the FLOPs for CycleGAN and MUNIT represent the cumulative results of the image translation tasks for the three domains, i.e. , T1c → FS T2W, T1c → T2W, and T1c → DWI. For the transfer of T1c to the other three MR sequences using CycleGAN, MUNIT, and StarGAN v2, the parameters that need to be optimized are 2, 3.3, and 1.9 times that of PSTUDA, respectively. Regarding FLOPs, PSTUDA demonstrates a reduction of 72% and 79% compared to CycleGAN and MUNIT, respectively, but shows a 17% increase compared to StarGAN v2. The increased computational demand is mainly due to the PIN module in PSTUDA, which requires per-pixel fusion of style and content features. It is worth noting that the training efficiency of PSTUDA becomes more advantageous when more target domains are included in a one-to-multiple domain adaptation task. Overall, PSTUDA substantially decreases both model parameters and FLOPs while maintaining high segmentation accuracy, implying lower memory requirements and faster inference speed. In summary, these results highlight the significant advantages of PSTUDA in enhancing the efficiency and feasibility of clinical applications.

4.3 Ablation Study

Effectiveness of PST and MS_D To investigate the effectiveness of the proposed Progressive Style Transfer (PST) paradigm, we conduct ablation studies on it with the Multi-Scale Discriminator (MS_D) on the MSKT dataset. As shown in Table 3 , the introduction of PST significantly improves the DSC and HD 95 metrics compared to the Baseline. This reflects the ability of the PST to enhance the realism of translated images while maintaining structural consistency. Furthermore, the multiscale discriminator assesses the pseudo-target domain images at different scales, thereby further enhancing the capabilities of the generator.

Effectiveness of M_SD and PIN We also conduct ablation studies on the internal sub-modules within PST, namely the Multi-level Style Dictionary (M_SD) and the Point-wise Instance Normalization (PIN). As illustrated in Table 4 , replacing the Mapping Network and Style Encoder in the

Baseline with the M_SD results in a significant performance improvement. This can be attributed to the stability and representativeness of the learned style encodings. Moreover, the integration of the M_SD with PIN fully considers the details and local style variations of images, thereby yielding exceptional results.

Ablation of PIN and other normalization methods We compare PIN with AdaIN [52] and BIN [53], and as shown in Table 5, PIN outperforms the others. We attribute this superior performance to PIN's ability to account for local style differences, which is particularly advantageous for finegrained segmentation tasks ( e.g. , kidney tumors, as abnormal pathological tissues, exhibit significant style differences). This capability enables PSTUDA to generate synthetic images that align more closely with the data distribution of the target domain.

5 Conclusion

In this work, we propose a novel and efficient One-to-Multiple PSTUDA framework that utilizes a multi-level style dictionary to decouple and store style information. By employing multiple cascaded style fusion modules, our framework progressively recombines content and style features, thereby achieving superior cross-modal alignment and consistency of medical tissue structures. Experimental validation on both a private dataset and a public dataset demonstrates the significant advantages of our method in improving training efficiency for one-to-multiple domain adaptation tasks and enhancing the accuracy of multi-sequence kidney tumor segmentation.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 62272404, 62076007, and 62372170, in part by the Natural Science Foundation of Hunan Province of China under Grants 2022JJ30571 and 2023JJ40638, and in part by the Research Foundation of Education Department of Hunan Province of China under Grant 23A0146.

References

Dinggang Shen, Guorong Wu, and Heung-Il Suk. Deep learning in medical image analysis. Annual Review of Biomedical Engineering , 19:221-248, 2017.

Huiyan Jiang, Zhaoshuo Diao, and Yu-Dong Yao. Deep learning techniques for tumor segmentation: a review. The Journal of Supercomputing , 78(2):1807-1851, 2022.

Matteo Ferro, Felice Crocetto, Biagio Barone, Francesco Del Giudice, Martina Maggi, Giuseppe Lucarelli, Gian Maria Busetto, Riccardo Autorino, Michele Marchioni, Francesco Cantiello, et al. Artificial intelligence and radiomics in evaluation of kidney lesions: a comprehensive literature review. Therapeutic Advances in Urology , 15:17562872231164803, 2023.

Guanyu Yang, Jinjin Gu, Yang Chen, Wangyan Liu, Lijun Tang, Huazhong Shu, and Christine Toumoulin. Automatic kidney segmentation in CT images based on multi-atlas image registration. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society , pages 5538-5541. IEEE, 2014.

HeiShun Yu, Jonathan Scalera, Maria Khalid, Anne-Sophie Touret, Nicolas Bloch, Baojun Li, Muhammad M Qureshi, Jorge A Soto, and Stephan W Anderson. Texture analysis as a radiomic marker for differentiating renal tumors. Abdominal Radiology , 42:2470-2478, 2017.

Xue-Ying Sun, Qiu-Xia Feng, Xun Xu, Jing Zhang, Fei-Peng Zhu, Yan-Hao Yang, and YuDong Zhang. Radiologic-radiomic machine learning models for differentiation of benign and malignant solid renal masses: comparison with expert-level radiologists. American Journal of Roentgenology , 214(1):W44-W54, 2020.

Nicholas Heller, Sean McSweeney, Matthew Thomas Peterson, Sarah Peterson, Jack Rickman, Bethany Stai, Resha Tejpaul, Makinna Oestreich, Paul Blake, Joel Rosenberg, et al. An international challenge to use artificial intelligence to define the state-of-the-art in kidney and kidney tumor segmentation in ct imaging. Journal of Clinical Oncology , 38(Suppl):626, 2020.

Nicholas Heller, Fabian Isensee, Klaus H Maier-Hein, Xiaoshuai Hou, Chunmei Xie, Fengyi Li, Yang Nan, Guangrui Mu, Zhiyong Lin, Miofei Han, et al. The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 challenge. Medical Image Analysis , 67:101821, 2021.

Jinhao Li, Huying Li, Yuan Zhang, Zhiqiang Wang, Sheng Zhu, Xuanya Li, Kai Hu, and Xieping Gao. MCNet: A multi-level context-aware network for the segmentation of adrenal gland in CT images. Neural Networks , 170:136-148, 2024.

Lawrence P Panych and Bruno Madore. The physics of MRI safety. Journal of Magnetic Resonance Imaging , 47(1):28-43, 2018.

Subhashis Banerjee, Sushmita Mitra, Francesco Masulli, and Stefano Rovetta. Deep radiomics for brain tumor detection and classification from multi-sequence MRI. arXiv preprint arXiv:1903.09240 , 2019.

Xinyu Zhou, Xuanya Li, Kai Hu, Yuan Zhang, Zhineng Chen, and Xieping Gao. ERV-Net: An efficient 3d residual neural network for brain tumor segmentation. Expert Systems with Applications , 170:114566, 2021.

JJ Nikken and GP Krestin. MRI of the kidney-state of the art. European Radiology , 17:27802793, 2007.

Fabian Isensee, Marianne Schell, Irada Pflueger, Gianluca Brugnara, David Bonekamp, Ulf Neuberger, Antje Wick, Heinz-Peter Schlemmer, Sabine Heiland, Wolfgang Wick, et al. Automated brain extraction of multisequence MRI using artificial neural networks. Human Brain Mapping , 40(17):4952-4964, 2019.

Daniela Said, Stefanie J Hectors, Eric Wilck, Ally Rosen, Daniel Stocker, Octavia Bane, Alp Tuna Beksaç, Sara Lewis, Ketan Badani, and Bachir Taouli. Characterization of solid renal neoplasms using MRI-based quantitative radiomics features. Abdominal Radiology , 45:2840-2850, 2020.

Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning , pages 1180-1189. PMLR, 2015.

Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G Hauptmann. Contrastive adaptation network for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4893-4902, 2019.

Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(4):4396-4415, 2022.

Pengfei Wei, Lingdong Kong, Xinghua Qu, Yi Ren, Zhiqiang Xu, Jing Jiang, and Xiang Yin. Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspective. Advances in Neural Information Processing Systems , 36, 2024.

Zhongqi Yue, Qianru Sun, and Hanwang Zhang. Make the U in UDA Matter: Invariant Consistency Learning for Unsupervised Domain Adaptation. Advances in Neural Information Processing Systems , 36, 2024.

Xiaowei Xu, Yinan Chen, Jianghao Wu, Jiangshan Lu, Yuxiang Ye, Yechong Huang, Xin Dou, Kang Li, Guotai Wang, Shaoting Zhang, et al. A novel one-to-multiple unsupervised domain adaptation framework for abdominal organ segmentation. Medical Image Analysis , 88:102873, 2023.

Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8188-8197, 2020.

Xiaoshuai Hou, Chunmei Xie, Fengyi Li, Jiaping Wang, Chuanfeng Lv, Guotong Xie, and Yang Nan. A triple-stage self-guided network for kidney tumor segmentation. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) , pages 341-344. IEEE, 2020.

Mingjun Ma, Haiying Xia, Yumei Tan, Haisheng Li, and Shuxiang Song. HT-Net: hierarchical context-attention transformer network for medical ct image segmentation. Applied Intelligence , pages 1-14, 2022.

Luana Batista da Cruz, Domingos Alves Dias Júnior, João Otávio Bandeira Diniz, Aristófanes Corrêa Silva, João Dallyson Sousa de Almeida, Anselmo Cardoso de Paiva, and Marcelo Gattass. Kidney tumor segmentation from computed tomography images using deeplabv3+ 2.5 d model. Expert Systems with Applications , 192:116270, 2022.

Xiuzhen Xie, Lei Li, Sheng Lian, Shaohao Chen, and Zhiming Luo. SERU: A cascaded SEResNeXT U-Net for kidney and tumor segmentation. Concurrency and Computation: Practice and Experience , 32(14):e5738, 2020.

Yasmeen George. A coarse-to-fine 3D U-Net network for semantic segmentation of kidney CT scans. In International Challenge on Kidney and Kidney Tumor Segmentation , pages 137-142. Springer, 2021.

Qian Yu, Yinghuan Shi, Jinquan Sun, Yang Gao, Jianbing Zhu, and Yakang Dai. Crossbar-net: A novel convolutional neural network for kidney tumor segmentation in ct images. IEEE Transactions on Image Processing , 28(8):4060-4074, 2019.

Andriy Myronenko and Ali Hatamizadeh. 3d kidneys and kidney tumor semantic segmentation using boundary-aware networks. arXiv preprint arXiv:1909.06684 , 2019.

Mohit Pandey and Abhishek Gupta. Tumorous kidney segmentation in abdominal CT images using active contour and 3D-UNet. Irish Journal of Medical Science (1971-) , 192(3):1401-1409, 2023.

Tianle Ma and Aidong Zhang. Affinitynet: semi-supervised few-shot learning for disease type prediction. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 1069-1076, 2019.

Ozan Ciga, Tony Xu, and Anne Louise Martel. Self supervised contrastive learning for digital histopathology. Machine Learning with Applications , 7:100198, 2022.

Kevin Faust, Adil Roohi, Alberto J Leon, Emeline Leroux, Anglin Dent, Andrew J Evans, Trevor J Pugh, Sangeetha N Kalimuthu, Ugljesa Djuric, and Phedias Diamandis. Unsupervised resolution of histomorphologic heterogeneity in renal cell carcinoma using a brain tumoreducated neural network. JCO Clinical Cancer Informatics , 4:811-821, 2020.

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In Proceedings of the IEEE International Conference on Computer Vision , pages 2223-2232, 2017.

Cheng Chen, Qi Dou, Hao Chen, Jing Qin, and Pheng Ann Heng. Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation. IEEE Transactions on Medical Imaging , 39(7):2494-2505, 2020.

Chenhao Pei, Fuping Wu, Liqin Huang, and Xiahai Zhuang. Disentangle domain features for cross-modality cardiac image segmentation. Medical Image Analysis , 71:102078, 2021.

Devavrat Tomar, Manana Lortkipanidze, Guillaume Vray, Behzad Bozorgtabar, and JeanPhilippe Thiran. Self-attentive spatial adaptive normalization for cross-modality domain adaptation. IEEE Transactions on Medical Imaging , 40(10):2926-2938, 2021.

Qingsong Xie, Yuexiang Li, Nanjun He, Munan Ning, Kai Ma, Guoxing Wang, Yong Lian, and Yefeng Zheng. Unsupervised domain adaptation for medical image segmentation by disentanglement learning and self-training. IEEE Transactions on Medical Imaging , 43(1):4-14, 2022.

Xiaokang Liu, Zhiqiang Wang, Kai Hu, and Xieping Gao. Pseudo Multi-Source Domain Extension and Selective Pseudo-Labeling for Unsupervised Domain Adaptive Medical Image Segmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 1-5. IEEE, 2023.

Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In International Conference on Machine Learning , pages 1989-1998. Pmlr, 2018.

Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-toimage translation. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 172-189, 2018.

Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 8789-8797, 2018.

Anmol Sharma and Ghassan Hamarneh. Missing MRI pulse sequence synthesis using multimodal generative adversarial network. IEEE Transactions on Medical Imaging , 39(4):11701183, 2019.

Behnam Gholami, Pritish Sahu, Ognjen Rudovic, Konstantinos Bousmalis, and Vladimir Pavlovic. Unsupervised multi-target domain adaptation: An information theoretic approach. IEEE Transactions on Image Processing , 29:3993-4002, 2020.

Yoonsik Kim, Jae Woong Soh, Gu Yong Park, and Nam Ik Cho. Transfer learning from synthetic to real-noise denoising with adaptive instance normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3482-3492, 2020.

Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, and Chengpeng Chen. Hinet: Half instance normalization network for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 182-192, 2021.

Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450 , 2016.

Jingjing Xu, Xu Sun, Zhiyuan Zhang, Guangxiang Zhao, and Junyang Lin. Understanding and improving layer normalization. Advances in Neural Information Processing Systems , 32, 2019.

Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tieyan Liu. On layer normalization in the transformer architecture. In International Conference on Machine Learning , pages 10524-10533. PMLR, 2020.

Junho Kim, Minjae Kim, Hyeonwoo Kang, and Kwanghee Lee. U-gat-it: Unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation. arXiv preprint arXiv:1907.10830 , 2019.

Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 , 2016.

Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision , pages 1501-1510, 2017.

Hyeonseob Nam and Hyo-Eun Kim. Batch-instance normalization for adaptively style-invariant neural networks. Advances in Neural Information Processing Systems , 31, 2018.

Songhua Liu, Tianwei Lin, Dongliang He, Fu Li, Meiling Wang, Xin Li, Zhengxing Sun, Qian Li, and Errui Ding. Adaattn: Revisit attention mechanism in arbitrary neural style transfer. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 6649-6658, 2021.

Xianchao Guan, Yifeng Wang, Yiyang Lin, Xi Li, and Yongbing Zhang. Unsupervised MultiDomain Progressive Stain Transfer Guided by Style Encoding Dictionary. IEEE Transactions on Image Processing , 2024.

Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High-resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 8798-8807, 2018.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 770-778, 2016.

Yaniv Taigman, Adam Polyak, and Lior Wolf. Unsupervised cross-domain image generation. arXiv preprint arXiv:1611.02200 , 2016.

Nicholas Heller, Niranjan Sathianathen, Arveen Kalapara, Edward Walczak, Keenan Moore, Heather Kaluzniak, Joel Rosenberg, Paul Blake, Zachary Rengel, Makinna Oestreich, et al. The kits19 challenge data: 300 kidney tumor cases with clinical context, ct semantic segmentations, and surgical outcomes. arXiv preprint arXiv:1904.00445 , 2019.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 , pages 234-241. Springer, 2015.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

A Emre Kavur, N Sinem Gezer, Mustafa Barı¸ s, Sinem Aslan, Pierre-Henri Conze, Vladimir Groza, Duc Duy Pham, Soumick Chatterjee, Philipp Ernst, Sava¸ s Özkan, et al. CHAOS challenge-combined (CT-MR) healthy abdominal organ segmentation. Medical Image Analysis , 69:101950, 2021.

Appendix

In the appendix of this paper, Section A provides a detailed introduction to the experimental datasets and evaluation metrics. Subsequently, Section B offers the specific implementation details of the experiments. Section C presents a visual comparison of the experimental results from different methods on the KiTS19 and MSKT datasets. Section D shows a comparison of training efficiency among different methods. Sections E and F present two sets of supplementary comparative experiments and two sets of supplementary ablation experiments, respectively. Finally, Section G discusses the potential limitations and the broad impacts of our method.

A Details of the Experimental Datasets and Evaluation Metrics

MSKT The MSKT dataset contains 104 cases of multi-sequence kidney and kidney tumor MRI data. Each case includes four MRI sequences: T1c, FS T2W, T2W, and DWI. The data are stored in NRRD format and scanned in axial view. The MR images have a resolution of 512 × 512 × N , where N represents the number of slices, ranging from 11 to 96. The slice thickness varies from 2 to 6 millimeters. Annotations were created by experienced clinicians, with normal kidney tissue regions labeled as 1, kidney tumor regions as 2, and other areas labeled as 0, considered as the background. In the first set of comparative experiments (Section 4.2), we randomly divide all cases into source domain training sets, target domain training sets, and target domain test sets in a 4.5 : 4.5 : 1 ratio. The annotated T1c data, comprising 46 cases, serve as the source domain, while the unannotated FS T2W, T2W, and DWI data, each comprising 46 cases, constitute the target domain. The remaining 12 cases are used as the test set. This division ensures that there is no intersection between the source domain, target domain, and test set to prevent information leakage. The number of samples and slices included in each sequence is detailed in Table 6.

To protect patient privacy, all data were anonymized during collection and processing, removing any information that could identify the patients. The use of this dataset has been reviewed and approved by the Institutional Review Board (IRB) to ensure that the research complies with ethical standards and safeguards participant rights. We strictly adhere to relevant laws and regulations regarding data use and protection to ensure the legal and compliant use of the data.

KiTS19 The KiTS19 dataset, established for the 2019 kidney and kidney tumor segmentation challenge, comprises multi-phase CT scans along with corresponding segmentation annotations for 300 patients. A total of 210 cases are designated for model training, and the remaining 90 are used to evaluate model performance. The CT images have a resolution of 512 × 512 × N , where N ranges from 29 to 1,059 slices. For our second set of comparative experiments (Section 4.2), we combine the KiTS19 dataset with the MSKT dataset. Specifically, as the KiTS19 challenge initially provides only the training set, we use all cases from this set as our source domain training set, which includes 45,424 CT slices, with 16,336 slices containing targets. All four sequences of the MSKT are considered as the target domain, and we merge the 92 cases identified as source and target domains in Table 6 to form the target domain training set, while the test set remains unchanged. Please note that in all of our experiments, the slices used for training all contain the kidney or tumor targets, while the slices used for testing are all the slices in each case.

Evaluation metrics Following [21], the Dice Similarity Coefficient ( DSC ) and the 95% Hausdorff Distance ( HD 95 ) are utilized for quantitative comparisons. It is important to note that if the predicted results and the true annotations do not contain the same categories, directly calculating HD 95 may lead to errors. To address this issue, we set the HD 95 value to 100 in cases of potential calculation errors, indicating no overlap between the predicted results and the true annotations.

B Implementation Details

All experiments are conducted on an NVIDIA A800 GPU utilizing the PyTorch framework. CT and MR data are uniformly processed to a resolution of 256 × 256. In the one-to-multiple domain adaptation stage, the number of domains is set to 4 and 5 for the two sets of experiments, respectively (including source and target domains). The length of the multi-level style dictionary is set to 4,096. The number of style fusion modules in the generator is 4. The weights of adversarial loss, cycle

consistency loss, and identity loss are set to 1, 10, and 1, respectively. Network optimization employs the Adam optimizer [61] with an initial learning rate of 1e-4, and the batch size is 8. Other hyperparameters follow the configuration in StarGAN v2. In the segmentation stage, U-Net is chosen as the network architecture, paired with the Adam optimizer starting with a learning rate of 1e-4. The batch size is set to 16, with a total training duration of 50 epochs. In the above setup, it takes about 18 hours for our method to fully execute once.

B.1 Network Architecture

Generator As shown in Table 7, the generator within PSTUDA comprises an encoder, four style fusion modules, and a decoder. The encoder includes three sets of convolutional blocks and four Resnet blocks, while the decoder is composed of two upsampling layers with transposed convolution operations and one convolutional block. Instance Normalization is applied during the encoding phase, and Point-wise Instance Normalization is utilized within the style fusion modules and the decoder.

Discriminator As shown in Fig. 2 in Section 3.4, we utilize four independent and identical discrimination branches in the multi-scale discriminator. The detailed architecture of the first discrimination branch, which takes the original size image as input, is presented in Table 8.

B.2 Implementation of Comparative Methods

CycleGAN This method is derived from the GitHub repository: https://github.com/junyanz/pytorchCycleGAN-and-pix2pix. In our experiments, ResNet_9block and PatchGAN with 3 convolutional layers are selected as the generator and discriminator for CycleGAN, respectively. The weights for adversarial loss (MSE), cycle consistency loss (L1), and domain-invariant perceptual loss (L1) are set to 1, 10, and 5, respectively. Other hyperparameters are consistent with the open-source code. During the segmentation phase, all hyperparameters for U-Net are the same as those in our PSTUDA.

MUNIT This method is from the GitHub repository: https://github.com/NVlabs/MUNIT. The architecture of the MUNIT network remains the same as the open-source code. The weights for adversarial loss (MSE), image reconstruction loss (L1), style reconstruction loss (L1), and content reconstruction loss (L1) are set to 1, 10, 1, and 1, respectively. In the segmentation stage, all U-Net hyperparameters are identical to those in our PSTUDA.

SIFA This method is available from the GitHub repository: https://github.com/JianghaoWu/SIFApytorch. The official version of this method is implemented in TensorFlow, available at https://github.com/cchen-cc/SIFA. In OMUDA, they have re-implemented SIFA in PyTorch, and we have used the PyTorch version of SIFA to ensure all methods use the same framework for a fair comparison. All settings, hyperparameters, and losses used in our experiments are the same as those in the open-source code.

DEPL This method is based on our own implementation.

StarGAN v2 This method is from the GitHub repository: https://github.com/clovaai/stargan-v2. In our experiments, the network architecture is consistent with the official code. The weights for R1 regression loss, adversarial loss (CE), cycle consistency loss (L1), style reconstruction loss (L1), and diversity-sensitive loss (L1) are all set to 1. High-pass filtering is not used in this comparative experiment. During the segmentation phase, all U-Net hyperparameters are the same as those in PSTUDA.

C Visualization of the KiTS19 and MSKT Experiments

In this section, we present the visualized results of the second set of comparative experiments conducted on the KiT19 and MSKT datasets, and these visualizations are discussed in Section 4.2.

Source CT

CycleGAN

MUNIT

SIFA

DEPL

StarGANv2

PSTUDA

CT

↑

T1c

CT

↑

FS T2W

CT

↑

T2W

55

CT

↑

DWI

D Training Efficiency

This section displays the comparative outcomes in terms of model parameters and FLOPs among different cross-domain image translation methods. As illustrated in Table 9, we compare PSTUDA with CycleGAN, MUNIT, and StarGAN v2, because these methods do not involve a segmentation stage. It is worth note that the style dictionary itself does not contribute to the computation of FLOPs. All models are trained using FP32 precision. The detailed analysis of these comparative results is provided in Section 4.2.

G : Generator, D : Discriminator, NM : Mapping Network, ES : Style Encoder, DS : Style Dictionary.

E Supplementary Comparative Experiments

To further validate the generalization capability of PSTUDA, we conduct bidirectional cross-modal validation experiments on a publicly available abdominal multi-organ dataset [62] and perform reverse validation experiments from MR to CT on the MSKT and KiTS19 datasets. The results, as shown in Tables 10 and 11, indicate that our method significantly outperforms StarGAN v2 (baseline) in both experimental groups.

F Supplementary Ablation Experiments

The multi-level style dictionary and style fusion module are the core components of PSTUDA, and these two modules include the following hyperparameters: the depth of the style dictionary, the number of levels in the style dictionary, and the number of style fusion modules. It is important to note that the number of levels in the style dictionary is equal to the number of style fusion modules. To investigate the sensitivity of model performance to various hyperparameters, we conduct extensive ablation studies on the depth of the multi-level style dictionary (M_SD) and the number of style fusion modules (SFM). As presented in Tables 12 and 13, under the original settings (dictionary

depth of 4,096 and module number of 4), PSTUDA performs optimally in most experiments. Notably, the segmentation performance on the T2W sequence improves when the dictionary depth is increased to 16,384.

G Limitations and Broader Impacts

Limitations Our work presents a domain adaptation method for multi-target domain image translation. However, the field of one-to-multiple domain adaptation is not extensively explored, and our

study may have some limitations that should be acknowledged. Firstly, the tasks we focus on are confined to domain adaptation of multi-sequence MR images and the segmentation of kidneys and tumors as presented in this work. In future research, we aim to broaden our scope to include domain adaptation for other medical and natural images, as well as other downstream tasks such as object detection. Secondly, compared to the ideal one-to-multiple model, our current model inherently operates within a broader multi-domain to multi-domain adaptation framework. Therefore, in our future studies, we aspire to design a more specialized one-to-multiple model, which is expected to further improve model performance and reduce model complexity.

Broader impacts In fact, the impact of one-to-multiple domain adaptation technology is profound, especially when facing numerous target domains and limited annotated data. Our PSTUDA framework can efficiently extend an annotated source domain to adapt to multiple unannotated new domains, significantly reducing the time and resource investment required for multi-domain transfer tasks. This capability is crucial in complex medical data environments, as it not only greatly alleviates the workload of medical professionals but also holds promise for improving the diagnosis and treatment processes for patients. Overall, our research enhances the flexibility and efficiency of domain adaptation technology, which advances the field of machine learning and brings innovative solutions to critical industries such as healthcare.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our main claims stated in the abstract and introduction accurately reflect the contributions and scope of the paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of our work in Appendix G.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We introduce our method in detail in Section 3.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We disclose all the information needed to reproduce the main experimental results of the paper in Appendix B.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: We open-source the code and a portion of the data.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide detailed information about the experimental setup in Appendix B. Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Error bars are not reported because they are too computationally expensive for one-to-one domain adaptive methods.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide detailed information about the compute resources needed for the experiments in Appendices B and D.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in our paper conforms, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss the potential broader impacts of our work in Appendix G.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our work does not pose such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have cited all the relevant assets used in our paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We used our private dataset in our work, which has good documentation.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [Yes]

Justification: We provide an explanation about this issue in Appendix A.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [Yes]

Justification: Our private dataset has been reviewed and approved by the IRB, and we provide an explanation about this issue in Appendix A.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes the One-to-Multiple Progressive Style Transfer Unsupervised Domain-Adaptive (PSTUDA) framework for kidney and tumor segmentation in multi-sequence MRI, addressing inefficiencies in existing one-to-one UDA methods. PSTUDA features a multi-level style dictionary and multiple cascading style fusion modules using point-wise instance normalization to recombine content and style features progressively. Tested on private (MSKT) and public (KiTS19) datasets, PSTUDA showed 1.8% and 3.9% improvements in Dice Similarity Coefficient for kidney and tumor segmentation, respectively, and achieved significant reductions in model parameters and computational costs. The framework outperformed state-of-the-art UDA methods, enhancing both segmentation performance and training efficiency.\n\nSTRENGTHS:\nThis paper demonstrates several strengths across various dimensions. \n\nIn terms of originality, PSTUDA introduces a relatively interesting approach by employing cascaded style fusion modules and Point-wise Instance Normalization (PIN) to achieve excellent cross-modal alignment and structural consistency. This combo allows for more precise and progressive recombination of content and style features, addressing limitations in existing one-to-one UDA methods and extending the applicability of domain adaptation techniques in medical imaging.\n\nThe strong review of related works situates PSTUDA within the broader context of domain adaptation and medical image segmentation.\n\nThe paper is well-written and clear, providing detailed explanations of the architecture and methodologies. It emphasizes structural consistency, which is critical for medical image analysis. The comprehensive writing extends to the appendices, offering an appropriate level of detail on datasets, which aids in reproducibility and understanding\n\nPSTUDA’s experimental design is robust, featuring both comparative and ablation studies as well as appropriate metrics and analysis. \n\nThis paper is original, detailed, clear, and demonstrates reasonable improvements in segmentation with significant improvements in floating point operations and model size.\n\nWEAKNESSES:\n- The reported improvements in Dice Similarity Coefficient (DSC) and 95% Hausdorff Distance (HD95), while notable, are relatively modest (1.8% and 3.9%, respectively). \n\n- The framework's scalability to handle a larger number of target domains in real-world scenarios remains untested.\n\n- The generalizability of this approach beyond simple kidney/tumour segmentation into multi-class intra-organ segmentation or even multi-organ segmentation remains to be seen.\n\nQUESTIONS:\n- I would agree with the concept of the multiple cascading modules and Point-wise Instance Normalization but am surprised it doesn't benefit the overall segmentation results further. Can the authors provide their insights into why there isn't a more dramatic boost? What future work do they envision to achieve more substantial performance boosts?\n\n-  paper demonstrates the effectiveness of PSTUDA on kidney and tumor segmentation in multi-sequence MRI. Have the authors tested the framework on other medical imaging tasks or modalities, such as CT or ultrasound? What would expected results and challenges be in other modalities?\n\n- Despite the reduction in model parameters and FLOPs, PSTUDA still appears resource-intensive. How feasible is the deployment of PSTUDA in real-world clinical settings with limited computational infrastructure?\n\n- Are there any plans to release code, pre-trained models, or detailed implementation guides? How can the research community best leverage PSTUDA in their own work?\n\nLIMITATIONS:\nLimitations and Broader Impacts are appropriately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a novel and efficient One-to-Multiple Progressive Style Transfer Unsupervised Domain-Adaptive (PSTUDA) framework to address the UDA task for MRI sequences. Specifically, they developed a multi-level style dictionary that explicitly stores style information for each target domain at different stages, reducing the burden on a single generator in multi-target transfer tasks and effectively decoupling content and style. Additionally, multiple cascaded style fusion modules are employed, utilizing point-wise instance normalization to progressively recombine content and style features, thereby enhancing cross-modal alignment and structural consistency. Experiments conducted on both the private MSKT and public KiTS19 datasets demonstrate the effectiveness of this approach.\n\nSTRENGTHS:\n1.\tA multi-level style dictionary was developed, which reduces the burden on a single generator in multi-target transfer tasks and effectively decouples content and style.\n2.\tMultiple cascaded style fusion modules are employed to progressively recombine content and style features, thereby enhancing cross-modal alignment and structural consistency.\n3.\tThe organization of the paper's structure is excellent.\n\nWEAKNESSES:\n1.\tThe open-source code is not explicitly provided.\n2.\tThe explanation of the One-to-Multiple Framework is somewhat rough and needs more details.\n3.\tPSTUDA needs to be validated bidirectionally on cross-modal data, such as MRI and CT.\n\nQUESTIONS:\n1.\tReading content from the multi-level style dictionary, does it incur additional overhead? If a new unknown domain is encountered, the learning of the multi-level style dictionary requires a warm-up (there is additional overhead the first time a new domain is encountered).\n2.\tWhat is puzzling is that Table 6 shows the multi-level style dictionary only takes up 0.1M; does this include tensors?\n3.\tIs there any difference between the Generator and Discriminator Architecture in Section 3.4 and previous work?\n4.\tThe theory of achieving independent style transfer for each pixel using Point-wise Instance Normalization (PIN) requires further validation. While it may be effective, the explanation of its application in style transfer is not sufficiently thorough. Additionally, although the targets in medical image segmentation are more precise, this does not necessarily imply that style adjustments need to be more precise. Further explanation is needed.\n\nLIMITATIONS:\nThe authors have addressed the limitations of the current work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a one-to-multiple progressive style transfer unsupervised domain adaptation framework designed for kidney and tumor segmentation. \n\nIt aims to mitigate the challenges of annotation burden and domain differences by employing a multi-level style dictionary and cascading style fusion modules. \n\nIt demonstrates significant improvements in segmentation performance and efficiency on the MSKT and KiTS19 datasets.\n\nSTRENGTHS:\nx. The introduction of a multi-level style dictionary and point-wise instance normalization (PIN) for progressive style transfer that effectively decouples content and style features.\n\nx. It significantly reduces the floating-point computation by approximately 72% and the number of model parameters by about 50%, highlighting its efficiency and feasibility for practical clinical applications.\n\nWEAKNESSES:\nx. **Concern about novelty.**\n\nFirst, the authors extended the UDA setting to 'one-to-multiple'. I am not sure about its practical or clinical relevance here compared to continued UDA or UDA on evolving domains. \nSecond, there seems to be limited novelty compared to the architecture of OMUDA derived from StarGAN v2. \n\nx. **Concern about the effectiveness of PIN**: \nPoint-wise Instance Normalization is a central component of the framework, but its effectiveness compared to other normalization techniques is not thoroughly evaluated. There might be scenarios where the PIN is less effective, and alternative normalization methods like Adaptive Instance Normalization or Batch-Instance Normalization might perform better. \n\nx. **Concern on analysis.**\nThe multi-level style dictionary and the cascading style fusion modules. Understanding the contribution of each part would help in refining the model. The sensitivity of the model’s performance to various hyperparameters, such as the temperature parameter (\\tao), the number of style fusion modules, and the depth of the style dictionary, is not extensively studied.\n\nQUESTIONS:\nPlease see my comments above.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-14"}