Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper proposes a meta-learning framework for open compound domain adaptation. The method uses k-mean clustering to estimate the domain of target samples and employs the image translation method to augment datasets. Then the augmented images are used for the meta-learning framework. The method outperforms baselines on Cityscapes and C-Driving. 
main_review: 
1. While the paper improves performance, the novelty is somewhat limited since the paper combines the existing image translation method for DA and meta-learning framework. 
2. The paper is hard to read and explanations are not compact. Especially, notation is not very clear to understand. 
3. The paper should explore the effect of k-means clustering. If there is no k-mean clustering (i.e., all target images are treated as single target domain), how well does the method work? At the same time, it would be better to provide accuracy of SOTA single target DA baselines (e.g., [20]) in Table 2.
4. No sensitivity analysis, no visualization of generated images, and no theoretical analysis.
5. How many runs are averaged in the experiments? (standard deviations?)
6. GTA5 is the only source domain in the experiments. Why don't use the Synthia dataset? In addition why there is a big gap between source-only models?


-- POST REBUTTAL -- 

I appreciate the effort for the response, but my concern about the experiment still remains. The experiments are performed only on one dataset as a source domain. I encourage authors to explore other datasets to verify the effectiveness of the method. In addition, other backbones (e.g., ResNet) should be explored. I also agree with the reviewer fSWY about concerns on the novelty. Therefore, I keep my score as weak reject.

limitations_and_societal_impact: Limitations are well addressed.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work introduces a meta-learning framework, based on MAML, for single-target DA (STDA) and open compound DA (OCDA) in semantic segmentation. Similar to Li et al. [22], the idea is to learn the initial condition (model's parameters) that is more favorable for target adaptation. The proposed framework makes use of an existing image-2-image model to translate source images into, what are called here, latent images; this is to have target-like annotated data to guide meta-optimization.

Each iteration of the proposed algorithm is composed of two consecutive steps: (1) meta-training operating on source and latent data to find the favorable initial condition for the next step and (2) meta-testing to perform target adaptation starting from the learned initial condition. 

The meta-training optimization, similar to MAML, involves inner and outer loops. In case of STDA, the inner loop performs UDA optimization steps on labeled source and unlabeled latent samples; the outer loop, based on the optimized weights of the inner loops, meta-optimizes the network to minimize segmentation loss computed using labeled latent data. In case of OCDA with k target sub-domains, k-mean clustering is first done to assign sub-domain pseudo-labels to target images. This information is used in the image-2-image operation to translate source data into sub-domain-like data. Different to STDA, there is an additional adversarial objective in the outer loop.

To evaluate performance, this work consider two DA benchmarks: GTA5->Cityscapes (STDA) and Cityscapes->C-Driving (OCDA). Reported results show improvement of the proposed framework as compared to previous approaches.
main_review: This is the first work adopting meta-learning for domain adaptation semantic segmentation. Improvement over state-of-the-art methods demonstrates the merit of the proposed framework. The paper presentation is quite clear and easy to follow. In general this is a decent work and I'm leaning toward the positive side. There are though still a few questions and concerns that I hope to receive satisfactory answers.

- What are the exact values of I,J and N (Algos 1 and 2) used in the experiments?

- In the OCDA experiment, clustering is done on features coming from which model? Using domain labels, may the authors provide the clustering performance? How does bad clustering performance affect the final results?

- The authors took the segmentation and discriminator models in [6] but instead of using the proposed GAN-like adversarial training in [6], the authors opted to GRL. What is the reason of this choice?
limitations_and_societal_impact: The authors discussed the limitations of their work. Potential negative societal impact was not provided.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper proposes a meta-learning-based UDA framework for semantic segmentation, which adopts meta-knowledge learned from source-latent adaptation to teach latent-target adaptation gradually. 
main_review: This paper is well organized and the method is explained in details. But there are stills some limitations in this work:
 1. The idea is not novel enough. On the one hand, meta-learning strategy has been applied to UDA for classification. On the other hand, meta-knowledge learning looks very similar to pre-training and fine-tuning strategy, both of whom aim to alleviate the difficulty of direct transfer from source to target. The only difference is that this method adopts an iterative strategy to update the model. 

2. The method is somewhat complicated, which incorporates transfer learning and adversarial learning. This will increase the complexity of model and impedes reproducibility. In addition, the author build many adaptation pairs, such as"source to latent", "latent to target", and "latent to latent"pairs, which makes the model too difficult to train.

 3. The experimental results are not convincing. The authors claim that this paper achieves SOTA on several benchmarks, but some works are missed in this paper, e.g., Self-supervised Augmentation Consistency for Adapting Semantic Segmentation (CVPR 2021), which achieves better performance. In addition, the authors only use VGG as backbone, while recent works adopt resnet101. I think the authors would better compare with these new methods with the same setting, such as
 (1) Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021).
 (2)Category anchor-guided unsupervised domain adaptation for semantic segmentation (NeurIPS2019)

 4. Since this model is suitable for open compound adaptation, it would be better to compare this method to other open compound adaptation methods, such as references [21]. [23].

limitations_and_societal_impact: Yes, they have addressed the limitation and potential negative societal impact.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper works on the domain adaptation task for semantic segmentation with either one target domain or multiple target domains. In order to solve this problem, the author proposed a novel meta-learning framework to explore the better initialization for the adaptive model and achieve a clear improvement compared to other SOTA methods. 
main_review: Overall speaking, the paper is good. It is easy to read and the proposed method is novel. The experiments were well designed and further complemented by the supplementary material. My main concerns about the submission are listed as follows.

1. I wonder whether the author used any self-training methods in the paper to achieve the results shown in Table 1 of the main paper and Table 2 in the supplementary material. I think most of the baselines in both tables are achieved via self-training kind of techniques by generating pseudo-labels in various ways. If the author does not use self-training, I will be astonished by the performance and hope to see what the results will be if self-training is further added. If the author used self-training, I would like to see the comparison to the baselines without using any self-training technique  to further evaluate the achivement given by the proposed method.

2. The mechasim of meta-learning used for domain adaptation is not very clear to me. In classicial meta-learning,  the meta-training often contains seveval datasets and let the network learn how to adapt to novel dataset. Thus during meta-testing phase, the model can fit to the test dataset in just a few iterations. However, in the paper the meta-training only contains a single dataset pair for the single target domain adaptataion case. So I think the model is not fully trained such that it knows how to fit to a novel dataset. This problem can be solved by training the model with multiple dataset pairs which is missing in the paper. I hope the author can further show enough explanation and it will be great if the author can provide more experimental results for the case I mentioned above.

3. The adaptation performance for ResNet-101 is more ciritcal and the author might need to consider to move the results in supplementary material to the main paper.


final comments after rebuttal

After carefully reading the paper and discussing it with the authors, my concern is well addressed. Although other reviewers question the novelty of this paper about using meta-learning, meta-learning is easier to be used for domain generalization or multi-source domain adaptation problems, where there are multiple domains. I think domain adaptation is a different case and generating a latent domain to help build the meta-learning setting is still novel. Thus I raise my rating and suggest accepting this paper.
limitations_and_societal_impact: Yes
review_title: null

Meta-review: After the discussion period this paper received mixed scores with one recommendation for rejection, one for acceptance, one leaning reject and one leaning accept. There was disagreement about the novelty of the proposed approach. The key novelty claimed by the authors is using a translated source image as a domain unto itself and performing meta learning to learn an adaptable model from source $\rightarrow$ latent such that further adaptation from latent $\rightarrow$ target should be more effective. All reviewers agreed that the image translation portion (i.e., generation of latent domain) was based on prior work. The debate then centered around whether the introduction of a meta learning objective using this translated image was sufficient contribution as well as whether that idea proved effective empirically. After considering all reviewer comments, discussion, author rebuttal and examining the paper, it seems that although prior work has used meta learning for domain generalization the specific use in this paper for the latent domain together with the ablation study in Table 3 showing the value of using the meta objective instead of just adapting directly from L$\rightarrow$T implies that this design decision is useful and novel. It does however seem that the study into the open compound target together with using existing clustering from [21] is less of a contribution and I would encourage the authors to revise the text to clarify their key contribution. Further, as promised please move the ResNet experiments to the main paper. 
Decision: Accept (Poster)