Dual-Branch Fusion with Style Modulation for Cross-Domain Few-Shot Semantic Segmentation

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-09 10:07:59

Abstract: Cross-Domain Few-Shot Semantic Segmentation (CD-FSS) aims to achieve pixel-level segmentation of novel categories across various domains by transferring knowledge from the source domain leveraging limited samples. 
The main challenge in CD-FSS is bridging the inter-domain gap and addressing the scarcity of labeled samples in the target domain to enhance both generalization and discriminative abilities. 
Current methods usually resort to additional networks and complex strategy to embrace domain variability, which inevitably increases the training costs. 
This paper proposes a Dual-Branch Fusion with Style Modulation (DFSM) method to tackle this issues.
We specifically deploy a parameter-free Grouped Style Modulation (GSM) layer that captures and adjusts a wide spectrum of potential feature distribution changes, thus improving the model's solution efficiency. 
Additionally, to overcome data limitations and enhance adaptability in the target domain, we develope a Dual-Branch Fusion (DBF) strategy which achieves accurate pixel-level prediction results by combining predicted probability maps through weighted fusion, thereby enhancing the discriminative ability of the model.
We evaluate the proposed method on multiple widely-used benchmark datasets, including FSS-1000, ISIC, Chest X-Ray, and Deepglobe, and demonstrate superior performance compared to state-of-the-art methods in CD-FSS tasks.

Main paper: 
 introduction: 
 Deep learning-powered semantic segmentation methods achieve outstanding performance, which heavily relies on a substantial amount of pixel-wise labeled data. However, this reliance can restrict their ability to learn effectively from a few samples in complex scenarios and to generalize to new categories. Few-shot semantic segmentation (FSS) is proposed to train a model with excellent capability for both generalization and adaptability when confronted with previously unseen classes. The core of this task is to rapidly learn new concepts and effectively segment images from new categories using minimal annotated data. Despite the significant advancements in the FSS task, the current methods primarily focus on the intra-domain scenario, where the training and testing data belong to the same domain. However, this assumption often falls short of real-world requirements, where the differences between the training and testing data arise due to variations originating from diverse sources or changes over time. As a result, the training sets become outdated, which can significantly affect the performance of the FSS task. CD-FSS has attracted interest as a solution to circumvent these restrictions. It enables the utilization of data from different domains and label spaces to be employed as source and target data, respectively.As shown in Figure 1 , CD-FSS tasks involve training a model on a source domain with abundant pixel-level annotated data like PASCAL VOC [9] and applying it to a target domain with a completely different data distribution and categories, which has only a limited number of labeled samples like a skin lesion dataset [6] . Therefore, the primary challenges are the disparities in data distribution between the source and target domains, as well as the scarcity of labeled samples in the target domain. The former results in limited inter-domain correlation, thereby reducing the effectiveness of knowledge transfer. The latter leads to inadequate feature extraction within the domain, which in turn weakens the learning performance of class-specific semantic knowledge. As described, our primary focus is on addressing two key issues: 1) How to minimize the visual differences between the training and testing domains to improve the model's generalizability; 2) How to predict unknown categories in the face of limited samples, thus enhancing the model's discriminative ability.Recently, many methods for CD-FSS tasks have been studied, which can be divided into two main categories. The first category of works [3, 14] is focused on the objective of training a model with robust generalization on the source domain while lacking access to target domain data. These methods enhance the model's generalizability by constructing complex parameter modules. However, the substantial domain gap between the source and target domains makes it difficult to achieve effective model transfer, leading to unsatisfactory results. Therefore, to achieve better segmentation outcomes, Lei et al. [17] introduce target domain data during the fine-tuning stage to learn domain-specific features, i.e., domain adaptation. Considering the limited labeled data in the target domain, we advocate for this direction.In order to tackle the aforementioned problems, we propose a novel CD-FSS approach named Dual-Branch Fusion with Style Modulation (DFSM) to enhance the model's generalizability and improve its discriminative capabilities in the target domain. Specially, to handle the first challenge, we introduce a Grouped Style Modulation (GSM) layer to generate diverse domain styles, thereby enhancing the model's generalization capabilities across different domains. Inspired by [13, 19] , we significantly alter the domain style by adjusting the statistical properties of deep network features, specifically by transforming the mean and variance of features to those of the target domain. However, during the training process, the target domain remains unknown, making it challenging to identify an appropriate target domain style. Existing methods attempt to simulate the target style by swapping styles of source domain data or using learnable style parameters, but these approaches increase the complexity of the model and extend the training duration.To address these challenges, we adopt a parameter-free approach, capturing the diversity of the source domain from an implicit semantic enhancement perspective within the feature space, thus broadening the coverage for unknown target domains. Based on statistical analysis of source domain data, we construct latent style estimates to capture the potential directions and magnitudes of style changes between domains. In addition, we leverage channel grouping strategies to extend the spectrum of possible style representations.To address the second challenge, we devise a Dual-Branch Fusion (DBF) strategy to enhance the model's discriminatory ability and domain adaptability through efficient utilization of available data and knowledge transfer. On the one hand, considering the limitation of sample quantity in the target domain, we employ data augmentation techniques to expand the sample space, thus increasing the diversity and robustness of model fine-tuning. By treating augmented support samples as pseudo-support samples and original support samples as pseudo-query samples, we provide additional training signals for fine-tuning, further improving the model's performance in the target domain. On the other hand, to enhance the model's discriminative power in specific domains, we propose not merely updating the parameters of the pre-trained model to adapt to the new domain but instead construct a dualbranch model to augment decision support information, delivering precise pixel-level predictions. By adaptively integrating predicted probability maps via weighted fusion, our model not only resolves the issue of scarce samples in the target domain but also enhances the model's ability to discriminate.The primary contributions can be described as follows:• We propose a Dual-Branch Fusion with Style Modulation (DFSM) method for CD-FSS that effectively bridges the domain gap, thereby improving both generalization and discrimination in specific domains.• We introduce a parameter-free Grouped Style Modulation (GSM) layer to generate diverse domain styles aimed at enhancing the model's generalizability. Our latent style estimation effectively captures the appropriate range of potential feature distribution variations. • We develope a Dual-Branch Fusion (DBF) strategy to elevate the model's discriminative power and domain adaptability in the target domain by expanding the sample space and adaptively integrating predicted probability maps via weighted fusion.• We conducte extensive experiments on four widely used benchmark datasets. The results indicate that our method significantly improves upon the baseline models and outperforms the state-of-the-art CD-FSS algorithm. 
related work 2.1 cross-domain semantic segmentation: 
 The current researchs on cross-domain semantic segmentation can be categorized into two groups: domain adaptation (DA) and domain generalization (DG). In the context of domain adaptation tasks, a model that has been trained effectively on the source domain must undergo fine-tuning using test data prior to its deployment in the target domain. A variety of [5, 15, 26, 33] The metric mechanism plays a vital role in the study of few-shot semantic segmentation, based on the meta-learning paradigm. Previous research can be classified into two main categories: prototypebased approaches and parameterized-based methods, regardless of whether the metric approach involves trainable parameters or not. Prototype-based methods [4, 18, 22] extract representative features or prototypes from a limited set of training samples and classify new samples to be segmented by calculating their distance relationships with the prototypes of each category, achieving precise pixel-level segmentation. Parameterized-based methods [16, 23, 25, 32] typically consist of encoders, feature processors, and decoders. During training, such methods dynamically adjust the network's parameter structure to learn the similarities between samples adaptively. However, these methods solely focus on learning performance within analogous data settings. In practical applications, they often face inconsistent data distributions across various domains. Finite data resources constrain these methods, making it difficult to effectively enhance generalization capabilities in new domains. We incorporate a GSM after each layer of the feature extractor to generate diverse feature representations, enhancing the model's generalizability. It is worth noting that GSM does not introduce any learnable parameters, thereby reducing the computational complexity of the model. 2)Fine-tuning and inference phase. We establish a dual-stream branch to learn effective feature representations with each branch is fine-tuned independently. Subsequently, during the inference stage, we perform a weighted fusion of the predictions from the dual streams.Segmentation.Compared with cross-domain semantic segmentation and FSS, the training set and testing set in CD-FSS exhibit differences in data distribution and no label distribution overlap. To alleviate interdomain discrepancies, Wang et al. [28] develope a meta-memory that collects statistical characteristics of instances from the source domain, thereby reducing the domain gap between the source and target domains. They also adopt contrastive learning to regulate the relationships among prototypes of different novel categories. Lei et al. [17] propose a model called PATNet, which utilizes a pyramid module to convert the features acquired from the source domain into features that are invariant across different domains. However, mapping features across domains may only sometimes be accurate, which can result in misaligning well-aligned support-query pairs in the original space when transferred to the new space. To address this issue, Huang et al. [14] propose the RestNet based on PAT-Net, which utilizes residual connections to bridge the gap between the original space and the latent space. However, these methods require additional parameters. Our approach learns diversified domain styles through a parameter-free GSM layer to reduce model complexity. 
method 3.1 preliminaries: 
 In the Cross-Domain Few-Shot Semantic Segmentation task, there exists a source domain 𝐷 𝑠 = (𝑋 𝑠 , 𝑌 𝑠 ) and a target domain 𝐷 𝑡 = (𝑋 𝑡 , 𝑌 𝑡 ), where 𝑋, 𝑌 denote the data distributions and label spaces respectively. There is a significant domain shift between the source and target domains, and their label spaces have no intersection, i.e., To prepare for training the CD-FSS model, start by selecting 𝐶 classes at random from the corresponding dataset. Then, pick 𝐾 samples for each selected class to create a support set. Additionally, choose 𝑁 𝑞𝑢𝑒𝑟 𝑦 samples from each class to form a query set. It is crucial to guarantee that there is no intersection between the support and query sets within the same situation. The CD-FSS model is trained on the training set 𝐷 𝑡𝑟𝑎𝑖𝑛 of the source domain during the training process. The model is not allowed to access any data from the target domain at this stage. Finally, the well-trained model is evaluated on the test set 𝐷 𝑡𝑒𝑠𝑡 of the target domain in testing phase.𝑋 𝑠 ∩ 𝑋 𝑡 = ∅, 𝑌 𝑠 ∩ 𝑌 𝑡 = ∅.Design of the framework. The overall framework of the proposed method is illustrated in Figure 2 , using a feature extractor with the first three layers of ResNet-50 [12] as the backbone network to generate the initial visual feature for each sample. To explore potential domain styles, a group style modulation layer is strategically positioned after each layer. We then use the decoder from SSPNet [10] that refines and fuses prototypes from support and query samples to make accurate predictions on the query sample.Two-stage Methods As shown in Figure 2 , our method consists of two learning stages. In the first stage, the model is trained on the source domain. In the second stage, the model is fine-tuned on the target domain. Specially, the model parameters obtained during the training phase will serve as the initial parameters for the dual-branch model during the fine-tuning stage. Our objective is to fine-tune the dual branches independently to adapt to the specific domain's style, and then fuse the predictions of both branches during the inference stage.Effective simulation of diverse style information within the training domain is crucial to enhance generalization capability. In this work, we propose a Grouped Style Modulation (GSM) to predict potential style variations between domains, thereby enhancing the model's robustness across different testing tasks. Existing methods often generate diverse styles through deep learning modules or random strategies, increasing computational costs with parameter-based learning and potentially producing meaningless styles that degrade segmentation performance. Inspired by previous works [13, 19] , we introduce a GSM that capture potential style estimates from small batches of data, providing each feature channel with an appropriate and meaningful range of variation. This approach does not compromise model training but enables the simulation of different potential shifts.Statistics Calculation. The statistics of features, such as mean and standard deviation, can represent the style of an image. Therefore, our initial step involves acquiring the feature statistics from both support and query samples. For each iteration, the intermediate feature 𝐹 𝑙 ∈ R 𝐵×𝐶 ×𝐻 ×𝑊 is extracted by the feature extractor for each sample from current batch of training data B = {(𝐼 𝑠 , 𝐼 𝑞 )} 𝐵 𝑖=1 , where 𝐼 𝑠 ∈ R 3×𝐻 𝑠 ×𝑊 𝑠 and 𝐼 𝑞 ∈ R 3×𝐻 𝑞 ×𝑊 𝑞 represent support set and query set respectively. Then we compute the channel-wise feature statistic including mean 𝜇 (•) and standard deviation 𝜎 (•) of given feature via:EQUATIONEQUATIONwhere 𝐵, 𝐶, 𝐻,𝑊 represent batch size, channel, height and weigth of 𝐹 𝑙 , respectively. 𝜖 is set to 1e- Latent Style Estimation. Assuming that the distribution of the target feature statistics follows a multivariate Gaussian distribution, the center of this distribution can be considered as the original feature statistics for each feature, while the standard deviation describes the range of different potential offsets. To obtain an appropriate and meaningful range of variation, we characterizes latent style estimation from a semantic enhancement perspective. There exist multiple distinct semantic directions within the deep feature space, and by translating features along specific directions and magnitudes, semantic transformations of the features can be realized. The variance among features encapsulates inherent semantic information. Consequently, we utilizes the variance of feature statistics across all batches of samples to construct latent style estimation.EQUATIONEQUATIONwhere, 𝑙 𝜇 , 𝑙 𝜎 represent latent style estimation of domain mean and variance. It is worth noting that the magnitude of latent style estimation plays a critical role in representing potential feature semantic information, providing potential transformations that may occur within style embeddings.Grouped Style Modulation. Once latent style estimations are obtained, corresponding multivariate Gaussian distributions can be constructed to synthesize new domain statistics, i.e., mean 𝜇 𝑡 ∼ 𝑁 (𝜇 𝑙 , 𝑙 𝜇 ) and variance 𝜎 𝑡 ∼ 𝑁 (𝜎 𝑙 , 𝑙 𝜎 ), as shown in Figure 3 . We sample from the Gaussian distribution and select semantic transformation directions in a random manner for enhancement.𝜇 𝑡 = 𝜇 𝑙 + 𝜖 𝜇 𝑙 ∑︁ 𝜇 (5) 𝜎 𝑡 = 𝜎 𝑙 + 𝜖 𝜎 𝑙 ∑︁ 𝜎 (6)where 𝜖 𝜇 and 𝜖 𝜎 follows the standard normal distribution N (0, 1).Then we utilize affine transformations to transform the style (𝜇, 𝜎) of the intermediate layer features 𝐹 𝑙 in the model into a novel style (𝜇 𝑡 , 𝜎 𝑡 ) that is randomly sampled:EQUATIONIn practical applications, we group the features of this batch by the channel dimension and perform affine transformations on each group separately. This module can be integrated at various positions within the network. By setting hyperparameters 𝑝 to control the probability of applying the model, it enhances the model's ability to learn stable features and robustness, thereby improving its generalization capability. Note that this module only operates during training and fine-tuning and can be discarded during testing. It is worth noting that this module does not introduce any learnable parameters, which to some extent reduces the computational complexity of the model. By introducing GSM, we can capture the variability of feature distributions, aiding the model in learning a more diverse range of domain styles. Compared to traditional methods, this strategy, which does not require additional complex calculations, not only improves the model's generalization but also reduces the difficulty and computational cost of model training.To further enhance the model's discriminative ability in specific domains, we employ a dual-branch fusion strategy to fine-tune the model, aiming to ensure that it adapts to the data distribution of the target domain.Dual-Branch Joint Modeling. In the target domain, support samples and query samples representing the same category may exhibit different appearances. The limited number of support samples restricts the model's ability to learn about the current category, thereby reducing its predictive performance on the query samples. Introducing a Group Style Modulation (GSM) layer during the finetuning phase can enhance the model's generalizability. However, due to the limitations of few samples in CD-FSS, the model may produce ambiguous predictions in some complex areas, leading to reduced discriminative power. Under limited data conditions, we employ an auxiliary model to augment decision support information, thereby enhancing the model's discriminatory ability.Therefore, during the fine-tuning and testing phase, we design two parallel branches to enhance the model's discriminative ability for the current task. Specifically, the first branch consists of two components: a feature extractor and a decoder formed by a prototypical network. Similar to the training phase, we use ResNet-50 as the feature extractor and add a GSM layer after each residual block. The second branch is based on the first but with the GSM layer removed.Weighted Fusion. As shown in Figure 4 , the branch with GSM captures excessive class-irrelevant information, the branch without GSM loses some class-relevant information, while the fusion method achieves accurate pixel-level prediction results. In this work, we further utilize the predictive differences between the two branches as an uncertainty estimation. Specifically, during the inference stage, after obtaining predicted probability maps from both branches for a query sample, we adopt a weighted fusion approach to integrate the prediction results. Formally, for a given query sample 𝐹 𝑞 ∈ R 𝐵×𝐶 ×𝐻 ×𝑊 and the predicted mask 𝑀 𝑞 ∈ R 𝐵×2×𝐻 ×𝑊 , we first perform mask average pooling to obtain the foreground prototype 𝐹 𝑃 𝑞 ∈ R 𝐵×𝐶 and background prototype 𝐵𝑃 𝑞 ∈ R 𝐵×𝐶 . Subsequently, cosine similarity is used to compute the similarity between the foreground and background.EQUATIONBy applying Equation 8to the prediction results from the two branches, we obtain two similarity scores 𝑆 1 , 𝑆 2 . Finally, the weight of the first branch can be obtained by:𝛼 = 1 1 + 𝑒 -𝑆 1 +𝑆 2 2 (9)The fusion of the two branches enhances the overall model's ability to distinguish between different categories within the same domain.Weights sharing. In the fine-tuning and inference stages of each scenario, the two branches share the parameters of the trained model as initial parameters. Additionally, except for the last residual block which requires parameter updates during fine-tuning, the two branches share the same weights. On one hand, sharing weights can reduce the computational complexity during the fine-tuning phase. On the other hand, the knowledge learned by the model in the source domain can be equally transferred to both branches, guiding them to learn quickly.We employ a two-stage training approach, where the GSM is used during the training phase to simulate potential domain styles, thereby enhancing the model's adaptability to the target domain. In the fine-tuning phase, data from the target domain is introduced for domain-specific adaptation, allowing the model to conform to the feature distribution of the target domain more accurately.Training. During the training, the fused prototype obtained via a self-matching strategy yields the corresponding prediction results 𝑀 𝑞 𝑜𝑢𝑡 . To facilitate training, new predictions 𝑀 𝑠 𝑠𝑢𝑝𝑝 and 𝑀 𝑞 𝑠𝑒𝑙 𝑓 are derived by predicting the support samples with the support prototype and the query samples with the self-matching prototype, respectively. Subsequently, the cross-entropy loss between the prediction results and the ground truth is calculated. Ultimately, we EQUATIONwhere 𝛼 1 , 𝛼 2 and 𝛼 3 are the balance weights.Finetune. In each episode, we randomly select different query images while constructing augmented samples for the remaining support set, treating it as a pseudo-support set and the original support set as a pseudo-query set. The parameters of the training model are loaded simultaneously for the two branches, and the fine-tuning processes are independent of each other. After training each episode, the support set is immediately used to predict the query set. It is important to note that training gradients at this stage does not involve updates. 
experiment 4.1 experimental setup: 
 Datasets. Following previous research, we train our model using the PASCAL [9] dataset as the source domain and evaluate it on target domains including FSS-1000 [20] , Deepglobe [7] , ISIC2018 [6] , and Chest X-ray [2] . The PASCAL dataset is composed of PASCAL VOC 2012 [9] with annotations from the SDS dataset [11] . FSS-1000 [20] consists of 1000 categories split into training, validation, and test sets with 520, 240, and 240 categories, respectively. Each category contains ten images. For evaluation, we selected 2400 supportquery pairs from the test set. Deepglobe [7] features satellite images annotated at the pixel level across seven categories. To reduce individual image sizes and expand the test image pool, each image is split into six sections. By removing single-class and unknown area images, we have 5666 images for evaluation. ISIC2018 [6] is comprised of skin lesion images, with each image precisely containing one primary lesion. The dataset includes a total of 2596 images. Chest X-ray [2] contains X-ray images for tuberculosis analysis, collected from 58 cases with tuberculosis manifestations and 80 normal cases, totaling 566 images. Set support-query pair 𝑇 𝑓 = (𝑆 𝑝𝑠𝑒𝑢𝑑𝑜 , 𝑆)Fine-tune the model and update the model parameters 𝑀 1 𝜃 1 using 𝑇 𝑓 = (𝑆 𝑝𝑠𝑒𝑢𝑑𝑜 , 𝑆) 6: end while 7: while not done do Implementation Details In this study, we adopt the same experimental setup as PATNet [17] . All samples from the PASCAL-5 𝑖 dataset are combined to form the training set, and only one round of experimentation is conducted during the training phase. In the testing phase, to ensure a fair performance comparison with existing methods, we utilize the prevalent evaluation methods currently in use. We will assess all different types of test datasets using the same model, namely, the model trained on the PASCAL-5 𝑖 dataset. We employ a pre-trained ResNet-50 on ImageNet as our feature extractor, utilizing features generated by conv4x. Following the methodology of previous work [17] , during the training and fine-tuning phases, we set the spatial dimensions of both support and query images at 400×400. During training, we comprehensively train the model using an SGD optimizer with an initial learning rate of 1e-3, where all parameters of the feature extractor are updated.Quantitative Results. The algorithm proposed in this paper has been thoroughly compared with the current state-of-the-art methods for cross-domain few-shot semantic segmentation. Specific experimental results are presented in Table 1 . All methods are trained on the PASCAL dataset and tested on the FSS, ISIC, Chest X-ray, and Deepglobe datasets, respectively. Bold indicates the best performance among all methods. The results demonstrate that our method achieves the best performance in three out of the four datasets and lead in average metrics. Compared to the second best method PMENet [3] , our approach improved accuracy by 7.91% and 11.05% under the 1-shot and 5-shot settings, respectively. These exceptional cross-domain accuracies indicate that our method can effectively generalize to unknown testing domains. As the number of support images increased from one to five, the model performance showed a robust improvement trend. Particularly on the Deepglobe and ISIC datasets, our method exhibited significantly more progress with the additional supervisory information, with performance gains reaching as high as 11.7% and 7.75%, respectively. These results thoroughly validate the efficient learning capability and outstanding generalization performance of our approach.Qualitative Results. Figure 5 illustrates the visualized prediction outcomes of our method on the four datasets (sequentially from top to bottom: FSS, ISIC, Chest X-ray, and Deepglobe datasets). It is noteworthy that, prior to making predictions on the test set, the model had not been exposed to the same categories within the training dataset. The qualitative results implie that with merely 1/5th of a support sample providing categorical target information, the model is still capable of identifying and accurately segmenting the complete foreground object in query images. For instance, the second row of Figure 5 displays the prediction results on skin lesion images. Despite the training data originating from natural images, which significantly differ in style from skin lesion images, the approach employed in this study still manages to closely predict the segmentation boundaries of the primary lesions in the query images.We conducted a series of ablation experiments on the modules proposed in our model. The 1-shot and 5-shot metrics mentioned in the table below represent average values. Specifically, to ensure fairness, evaluations are performed using five different random seeds for each test dataset, and the average of these five trials is taken as the result for that dataset.Effectiveness of Specific Modules. Table 2 shows the impact of different modules on enhancing model performance. We adaptively improve the few-shot semantic segmentation method SSPNet [10] , enabling it to effectively handle cross-domain few-shot semantic segmentation tasks. The enhanced SSPNet is then used as a baseline and 5-shot settings. In this work, we achieve local style modulation by grouping channels, thereby enhancing the diversity of features.To unveil the impact of channel group size, we conduct ablation studies on the size of the grouped channels. As demonstrated in the table 3, the model achieves the best results when the channel size is 16. Additionally, compared to the model without channel grouping, there is a performance improvement of 1.04%, which underscores the effectiveness of this method. Moreover, we systematically investigate the potential impact of noise ratio on model performance, where a higher noise ratio corresponds to the generation of a richer variety of domain styles. The results displayed in Figure 6 reveal a trend: as the noise ratio incrementally increases, the model's overall performance shows an upward trajectory. Furthermore, we utilize t-SNE to visualize the feature distribution differences between datasets, as shown in Figure 7 . The pink dots are features extracted from the training domain dataset PASCAL, while the orange dots represent features extracted from the test domain dataset Deepglobe, using the feature extractor. The left image depicts the feature distribution between the two datasets under the baseline model. It is evident that there is a significant domain discrepancy between the two datasets. However, after employing the GSM proposed in this study, the distribution between the two datasets shiftes from a clustered aggregation to a radial, point-like spread, significantly reducing the distribution gap, as shown in the right image. This illustrates the effectiveness of the GSM.Effectiveness of Dual-Branch Fusion. To verify the effectiveness of DBWF, wo conduct some comparative experiments as shown in Table 4 . In the experiment named w/o GSM, we remove the operation of GSM. We discover that each branch exhibits distinct performance advantages on different datasets. Therefore, we further consider combining the two aforementioned methods. The method called Mean integrates the two methods equally and achieves a performance gain, which demonstrates the necessity of complementarity in prediction results. Moreover, utilizing weighted fusion can help the model achieve better outcomes.  
 
 Appendix:  


Decision: Most reviewers appreciate the innovative combination of techniques proposed in this work. However, one reviewer expresses concerns about the justification of the proposed architectural choices and the demonstration of runtime and memory efficiency. This reviewer expected the authors to provide a comprehensive table presenting quantitative evaluations across various ablation studies and comparisons to state-of-the-art methods.
After reviewing the comments from all four reviewers, I suggest accepting this work. The authors are encouraged to refine the manuscript before submitting the final version.

Reviews: review: paper_summary: paper_summary: The paper proposes a Dual-Branch Fusion with Style Modulation (DFSM) method for Cross-Domain Few-Shot Semantic Segmentation (CD-FSS). The DFSM method includes a Grouped Style Modulation (GSM) layer to adapt to feature distribution changes across domains and a Dual-Branch Fusion (DBF) strategy to improve pixel-level prediction accuracy. The approach is designed to be parameter-free, enhancing generalization and discrimination abilities with limited labeled samples.
strength_weakness: strength_weakness: 1. a lot of comparison experiments of other domain adaptation work.
2. Includes results of visualization of the features, visualizing the contribution of the proposed module
3. The problem are defined clearly 
4. Many visual results are given in the main paper and the supplementary materials to visualize the model performance
limitations: limitations: - Limited Contribution: it seems to be an incremental work of previous work[17] which combine the work [17] and a statical distribution shift module.

 - In section2.1, you need to clarify why you solve this problem by few shot domain adaptation instead of using domain generalization. According to your clarification, in the practical applications scenario, it seems more likely that the information about the target domain may not be available, and it seems that domain generalization is a more robust generalization scheme. It would be better if you could illustrate more advantage of DA compared to DG.

 - As you mentioned in section 2.3, there are also works that use learning methods to bring the target domain closer to the source domain in latent space. It would be beneficial to include comparisons with such methods in your experiments to demonstrate that your method does not degrade performance while maintaining model complexity. Since based on your mission scenario settings, you are not extremely constrained by computational resources.

 -  Line398-401 has some elaboration errors and formatting errors

 - The example you provided in Fig. 5 is somewhat simplistic, which may not fully capture the migration performance of your method in the context of semantic segmentation in remote sensing images. It would be beneficial to examine results on more granular datasets such as SpaceNet, loveDA.
paper_summary: paper_summary: This work aims to resolve Cross-Domain Few-Shot Semantic Segmentation task, which mainly faces two challenges: 1) domain gap between source and target domains, 2) limited number of samples in the target samples. To address these two issues, the author proposed an efficient parameter-free Grouped Style Modulation (GSM) layer that generates various styles for intermediate features by utilizing their statistical property within a mini-batch. Then, to tackle the second challenge, they proposed to augment samples from support set as pseudo-support samples and true support-set samples as pseudo-query samples.
strength_weakness: strength_weakness: 1. The primary motivation is reasonable and it is well-addressed by appropriate and interesting modules.
2. Extensive experiments demonstrated the effectiveness of the proposed methods, showcasing the promising performance gains.
3. The presentation of the paper is highly qualified.
limitations: limitations: 1. Compared with [19], the novelty of GSM is limited, although applying it for style modulation makes sense. Please carefully compare the proposed GSM with [19].
2. The motivation for Dual-Branch Fusion (DBF) strategy is unclear. The author claimed that “the branch with GSM captures excessive class-irrelevant information, the branch without GSM loses some class-relevant information, while the fusion method achieves accurate pixel-level prediction results”, which makes me confused. Why does GSM capture class-irrelevant information? and why does w/o GSM also lose class-relevant information? Isn’t class-relevant important for semantic segmentation? Then how can each branch help each other? 
3. The details of augmentation are missing. How can it ensure the augmentation is meaningful? which may significantly affect performance.

[19] Xiaotong Li, Yongxing Dai, Yixiao Ge, Jun Liu, Ying Shan, and LINGYU DUAN. 2022. Uncertainty Modeling for Out-of-Distribution Generalization. In International Conference on Learning Representations.
paper_summary: paper_summary: The submitted paper, "Dual-Branch Fusion with Style Modulation for Cross-Domain Few-Shot Semantic Segmentation," addresses the critical challenge of cross-domain few-shot semantic segmentation (CD-FSS), which is highly significant given the practical limitations in labeled data across diverse domains.
strength_weakness: strength_weakness: The paper introduces a Dual-Branch Fusion with Style Modulation (DFSM) that uniquely combines style modulation and dual-branch fusion to tackle domain discrepancies and limited data issues.
1. Grouped Style Modulation (GSM): This parameter-free approach dynamically adjusts feature distributions to enhance model generalizability across domains.
2. Dual-Branch Fusion (DBF): The strategy integrates two branches of modeling to refine the discriminative power of the network, allowing for more accurate and robust segmentation.
3. Comprehensive Evaluation: Extensive experiments on multiple benchmarks such as FSS-1000, ISIC, Chest X-Ray, and Deepglobe, where it outperforms existing methods, showcasing the practical effectiveness and adaptability of the proposed method. Especially in Figure 7, the distribution between the two domains is shown clearly. However, some test tables are described not clearly such as whether the task was 1-shot or 5-shot, and the distribution is still a barrier across different domains, not well integrating into similar distributions.
4. High performance: Can you explain why the model has improved a lot on the ChestX data set, but the improvement on other data sets is not obvious? What is the reason for this?
limitations: limitations: 1. Lack of Theoretical Analysis: The paper primarily focuses on empirical results without substantial theoretical backing for the effectiveness of the proposed methods. Especially for the parameter-free approach: GSM, please use some quantitative methods to explain the reduction in the number of parameters.
2. GSM Methodology: It is not clear how the styles are selected and how their optimal relevance to the target domain is determined.
3. Impact of DBF Complexity: The dual-branch architecture increases the model complexity, which might not be justifiable unless a clear benefit is shown over simpler methods. Besides, the two-stage methods, in the fine-tuning stage, the target is used on Dual-branch fusion, whether the target data set are used for model fine-tuning? How many target data sets are used for fine-tuning? Is this unfair to CD-FSS tasks?
4. Generalization over Datasets: While performance improvements are noted, the generalization capability over more diverse or unseen datasets is not discussed, which is important in the CD-FSS field.
5. Parameter Tuning and Scalability: Details on the sensitivity of the model to hyperparameter settings and its scalability to larger datasets are lacking.
6. Lack of novelty: what is the main difference between your work and paper: 'Yang, Y., Chen, Q., & Liu, Q. (2023). A dual-channel network for cross-domain one-shot semantic segmentation via adversarial learning. Knowledge-Based Systems, 275, 110698.' ?
7. Lack of detail comparison with state-of-the-art methods, such as 'Huang, X., Zhu, C., & Chen, W. (2023). Restnet: Boosting cross-domain few-shot segmentation with residual transformation network. arXiv preprint arXiv:2308.13469.'.