From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Accept: poster
metareview:_summary,_strengths_and_weaknesses: The submission introduces a cross-domain few-shot learning approach called cross-level distillation (CLD). It operates in four distinct phases:

1. A teacher model is pre-trained on the base classes using the usual cross-entropy loss.
2. A student network is trained on unlabeled target data using a combination of contrastive self-supervised learning and cross-level distillation from the teacher network. The knowledge distillation loss gradually shifts from focusing on the teacher network activations to focusing on the activations of previous student networks.
3. The student network's classification layer is then fine-tuned on the labeled support set using "feature denoising", i.e., exponentiating embedding values with the largest magnitude and zeroing out the others.
4. The student network is evaluated on the query set.

The proposed approach is evaluated on BSCD-FSC and compared against several recent baselines, including STARTUP and Dynamic-Distillation, and is claimed to achieve state-of-the-art performance.

Reviewers note the proposed approach's strong performance, the thorough ablation studies, and the submission's writing clarity. The main concerns surfaced by reviewers are:

- The assumption regarding access to unlabeled target data is not well justified (am4M). The authors respond by citing defect detection as an example of a practical application where this assumption holds. Reviewer am4M contends that this is one example, and that a more complete justification would be needed to make the assumption convincing. Given the existence of previously published works (STARTUP, Dynamic-Distillation) making a similar assumption, I think we can safely assume that the research community is convinced that it makes sense at least in some circumstances, and I side with the authors.
- The proposed approach contains many moving parts requiring careful hyperparameter optimization (8cJ8). The authors explain that the hyperparameters are chosen using EuroSAT and shared across other evaluation settings, which shows that a good choice of hyperparameters transfers across settings. In general, approaches with many moving parts face a larger burden of proof, but in this instance all reviewers agree that the ablation studies are comprehensive. I side with the authors on this point.
- The dual use of a teacher network and an old student network introduces additional computational complexity (am4M, 8cJ8). The authors respond by comparing the computational complexity of CLD using a model-averaging with that of CLD using an old student. Reviewer am4M thinks this is missing their point: in previous work either a teacher network or a model-averaged old student is used, and not both. I side with reviewers on this point, although I don't think this is a critical flaw, since the computational complexity remains low enough for CLD to be applicable in practice. I would encourage the authors to incorporate this point when discussing the drawbacks of their proposed approach.
- The proposed approach feels more like a collection of engineering innovations than a new idea which brings a fresh perspective (am4M, yRW4). Moreover, the aspects of the proposed approach which appear to offer only modest performance improvements (am4M). The authors list differences between CLD and BYOT, and they point to results in Table A and Table 1 to support their claim of a significant performance improvement. Reviewer am4M is unconvinced. They feel the differences between CLD and BYOT mostly boil down to engineering decisions. They also do not feel that the authors address their concerns regarding BYOL+CLD+FD specifically, since it does not outperform Dynamic-Distillation significantly and underperforms SimCLR. I agree with the reviewers that the submission is thin on new insights and that its contributions are mostly empirical.

Opinions are split on acceptance. On one hand, the submission is clear and the proposed approach is effective and well ablated. On the other hand, there is concern about the proposed approach's longer-term impact potential. Ultimately, while I agree with those concerns, I think the submission makes enough of an empirical contribution to be of interest to the community and to warrant acceptance.
justification_for_why_not_higher_score: There is concern about the proposed approach's longer-term impact potential.
justification_for_why_not_lower_score: The submission is clear and the proposed approach is effective and well ablated. It makes enough of an empirical contribution to be of interest to the community and to warrant acceptance.
summary_of_AC-reviewer_meeting: Reviewer am4M remained concerned that the submission's technical contributions were too thin. Reviewer yRW4 still felt positive about the paper, but agreed with the technical contribution concern and did not necessarily wish to champion the paper.
note_from_PC: if the above contains the word "oral" or "spotlight" please see: "oral" presentation means -> notable-top-5% and "spotlight" means -> notable-top-25%. As stated in our emails, we are disassociating presentation type from AC recommendations

From: ICLR.cc/2023/Conference/Paper233/Authors
title: Response to Reviewer yRW4
comment: We appreciate your positive review and encouragement. Our code will be released publicly for testing and reproduction. We believe that our findings are useful for the field of cross-domain few-shot classification research.

From: ICLR.cc/2023/Conference/Paper233/Authors
title: Response to Reviewer 8cJ8
comment: Thank you for your positive review as well as insightful suggestions. The replies are summarized below.

**Q1: There are too many hyperparameters. Is it convenient to tune them for new applications?**

**A1**: In our experiments, we only tune the parameters on EuroSAT (5-way 1-shot) and adopt these hyperparameters to other datasets and tasks. Even if these hyperparameters are not deliberately adjusted on other datasets, the results on them are still competitive. Considering the performance and the memory requirement, we always choose $\tau=1$ for the old student in cross-level distillation (CLD). Table 4 in our paper shows that CLD is not sensitive to $\tau$. And CLD can help naive SimCLR promote the performance from 75.10% to 79.38%.

In terms of feature denoising (FD), we conduct experiments with different $h$ and $\beta$ in Table C. The range between the largest and the smallest accuracies in Table C is 0.96%, which indicates the robustness of FD with different $h$ and $\beta$.

|              | $h=48$ | $h=64$ | $h=96$ | $h=128$ |
| ------------ | ------------ | ------------ | ------ | ------- |
| $\beta=0.1$  |  81.96  | 82.35  | 82.26  | 81.63   |
| $\beta=0.2$  |  82.10  | 82.49  | 82.46  | 81.86   |
| $ \beta=0.3$ |  82.10  | 82.57  | 82.52  | 82.11   |
| $\beta=0.4$  |  82.04  | 82.52  | 82.59  | 82.25   |
| $\beta=0.5$  |  81.83  | 82.46  | 82.55  | 82.34   |

Table C. Accuracies (%) of 5-way 1-shot experiments on EuroSAT with different $h$ and $\beta$.

Overall, our CLD and FD mainly introduce three hyperparameters, $\tau$, $h$, and $\beta$. All these parameters are not sensitive in the large ranges and can be easily tuned on a new application.

**Q2: How much extra cost will the old student introduce?**

**A2**: We introduce the old student (OS) mechanism for assembling the historic information and reducing the teacher network’s bias. And its necessity is shown in Table 4 of our paper. Besides, moving average (MA) may achieve a similar goal by weighting the average of the teacher network and the historic network, which is designed by Dynamic-Distillation. The detailed comparison is given below.

**In terms of the additional complexity**, there are only about 4.9M parameters in the old student. Compared to MA, which does not introduce extra memory, CLD with MA occupies 2505M memory, while CLD with OS occupies 2525M memory, which are almost identical. When training on one P100 GPU with 2% of target data, it takes 39 minutes to train CLD with OS, and 33 minutes to train CLD with MA. When 20% of target data is used, it takes 4 hours and 40 minutes to train CLD with OS and 4 hours to train CLD with MA.

**In terms of the performance**, we empirically find that the old student mechanism usually performs better than MA. We conduct experiments that uses MA instead of the old student to assemble the historic knowledge. The results are reported in Table D. $\Delta$ means the performance gap between the two methods. Then the old student can often perform significantly better than MA, especially when the target data is less during the training phase.

| **Target data**                                              | **2%** | **5%** | **10%** | **15%** | **20%** |
| ------------------------------------------------------------ | ------ | ------ | ------- | ------- | ------- |
| **CLD (with MA)**                                            | 64.47  | 69.71  | 73.59   | 75.61   | 76.97   |
| **CLD (with old student)**                                   | 69.04  | 74.42  | 77.04   | 77.89   | 79.38   |
| $\Delta$ | 4.57   | 4.71   | 3.45    | 2.28    | 2.41    |

Table D. Accuracies (%) of 5-way 1-shot experiments on EuroSAT with different proportions of target data.

Considering the performance improvement brought by the old student, both the small extra memory and time consumption are acceptable. It is also notable that the old student does not introduce any additional complexity in the testing phase.


We hope we have addressed all your concerns about our work. Any further comments and questions are welcome.

From: ICLR.cc/2023/Conference/Paper233/Authors
title: Response to Reviewer am4M [1/2]
comment: We would like to sincerely thank you for the positive comments, valuable suggestions, and inspiring criticism. We believe that all the comments have been carefully addressed. The point-by-point responses to them are given below.

**Q1: Is the assumption of accessing to a few of unlabeled target data practical? Whether 20% of target domain data is small enough?**

**A1**: In fact, some industrial scenarios have similar problems, such as the defect detection task. In this task, a few of unlabeled images of malfunctioning machinery are provided for training a recognition system. In this case, our work can be used efficiently.

Besides, the reason that we set 20% of target data in the training stage is to fairly compare with STARTUP and Dynamic-Distillation. Now, we conduct some additional experiments in Table A, which indicate our model can still work well with less available target data. The proposed cross-level distillation (CLD) and feature denoising (FD) (69.66%) with 2% of target data outperforms STARTUP (65.24%). Moreover, CLD+FD (75.74%) with 5% of target data outperforms Dynamic-Distillation (73.14%). Note that STARTUP (65.24%) and Dynamic-Distillation (73.14%) are trained with **20%** of target data.

| **Target data**           | **2%** | **5%** | **10%** | **15%** | **20%** |
| ------------------------- | ------ | ------ | ------- | ------- | ------- |
| **STARTUP**               | **-**  | **-**  | **-**   | **-**   | 65.24   |
| **Dynamic-Distillation**  | **-**  | **-**  | **-**   | **-**   | 73.14   |
| **SimCLR**                | 51.93  | 64.86  | 72.70   | 72.91   | 75.10   |
| **SimCLR+CLD  (ours)**    | 69.04  | 74.42  | 77.04   | 77.89   | 79.38   |
| **SimCLR+CLD+FD  (ours)** | 69.66  | 75.74  | 79.35   | 80.74   | 82.52   |

Table A. Accuracies (%) of 5-way 1-shot experiments on EuroSAT with different proportions of target data.

**Q2: What are the differences between the old student mechanism and the moving average (MA)? And, how much is the computational cost of the old student in the training stage?**

**A2**: The MA weights the average of the networks’ parameters. In contrast, the old student (OS) mechanism weights the average of the features. To understand the benefits of OS more easily, we provide a case. Assume that the teacher network and the historic student network only have one convolution-ReLU layer. The weights of the teacher and the historic student are $ \mathbf{w}_t$ and $ \mathbf{w}_s$, respectively. The fusion coefficient is $\alpha$ and the input is $\mathbf{x}$. In MA, the fused target feature $f_M$ for teaching the current student network is $f_M = ReLU(\alpha \mathbf{w}_t^\top \mathbf{x} + (1-\alpha)\mathbf{w}_s^\top \mathbf{x})$. That for OS $f_O$ is $f_O = \alpha ReLU( \mathbf{w}_t^\top \mathbf{x}) + (1-\alpha)ReLU( \mathbf{w}_s^\top \mathbf{x}).$ In $f_M$, when some values in $\mathbf{w}_t^\top \mathbf{x}$ are negative but the values in the corresponding positions of $\mathbf{w}_s^\top \mathbf{x}$ are positive and large, the ReLU activation function will keep those negative elements in $\mathbf{w}_t^\top \mathbf{x}$. However, this situation is undesirable because the teacher network should have removed the negative elements by the ReLU function. In this case, the negative elements in $\mathbf{w}_t^\top \mathbf{x}$ are redundant and should not be kept. So is the historic student network. In contrast, in $f_O$, $\mathbf{w}_t^\top \mathbf{x}$ and $\mathbf{w}_s^\top \mathbf{x}$ are both activated by ReLU, where the above undesirable situation does not happen. Therefore, the old student mechanism can filter the undesirable elements better.

We also empirically find that the old student mechanism usually performs better than MA. We conduct experiments that uses MA instead of the old student to assemble the historic knowledge. The results are reported in Table B. $\Delta$ means the performance gap between the two methods. Then the old student can often perform significantly better than MA, especially when the target data is less during the training phase.

| **Target data**                                              | **2%** | **5%** | **10%** | **15%** | **20%** |
| ------------------------------------------------------------ | ------ | ------ | ------- | ------- | ------- |
| **CLD (with MA)**                                            | 64.47  | 69.71  | 73.59   | 75.61   | 76.97   |
| **CLD (with old student)**                                   | 69.04  | 74.42  | 77.04   | 77.89   | 79.38   |
| $\Delta$ | 4.57   | 4.71   | 3.45    | 2.28    | 2.41    |

Table B. Accuracies (%) of 5-way 1-shot experiments on EuroSAT with different proportions of target data.


  

(Because of the character limitation, please continue to read **Response to Reviewer am4M [2/2]**)

From: ICLR.cc/2023/Conference/Paper233/Authors
title: Response to Reviewer am4M [2/2]
comment: **A2**: In terms of the concern of additional complexity, there are only about 4.9M parameters in the old student. CLD with MA occupies 2505M memory, and CLD with old student occupies 2525M memory, which are almost identical. When training on one P100 GPU with 2% of target data, it takes 39 minutes to train CLD with old student, and 33minutes to train CLD with MA. When 20% of target data is available, it takes 4 hours and 40 minutes to train CLD with old student and 4 hours to train CLD with MA. Considering the performance improvement brought by the old student, we think both extra memory and time consumption are acceptable. It is notable that the old student does not introduce any additional complexity in the testing phase. 


**Q3: What are the differences between CLD and BYOT@ICCV2019?**

**A3**: Our CLD is similar to BYOT@ICCV2019 only in the distillation structure. The differences are described below.

**The distilled contents are different**. BYOT distills the logits, namely pseudo labels, of the deep layers to the shallow layers. CLD distills the features in the deep layers of the teacher network and the old student network directly. The logits may lose some information, because they are projected from the originally continuous features to a lower-dimensional subspace. The original features contain more accurate information than logits.

**The training scheme is different**. BYOT is in a supervised scheme and requires the true labels of target data to guide the pseudo-label generation. In contrast, CLD is trained in a self-supervised manner on the unlabeled target data and does not require true labels.

**The training effect is different**. BYOT cannot assemble the historic information because it uses the current student network to distill itself. In contrast, CLD can assemble all the historic information with the support of the old student.


**Q4: Concerns about the performance**

**A4**: For SimCLR, please refer to Table A in A1. Our CLD+FD (69.66% accuracy with 2% of target data) outperforms STARTUP (65.24% accuracy with 20% of target data). CLD+FD (75.74% accuracy with 5% of target data) beats Dynamic-Distillation (73.14% accuracy with 20% of target data). In both cases, naive SimCLR shows ordinary results, which are 51.93% and 64.86% accuracies with 2% and 5% of target data, respectively. In Table 1 of our paper, CLD+FD outperforms SimCLR at a low extra cost with performance gains of 7.42% (EuroSAT, 1-shot), 3.99% (ISIC, 1-shot), 3.45% (ISIC, 5-shot), and 2.72% (EuroSAT, 5-shot). 

For BYOL, in Table 1 of our paper, the results of BYOL+CLD+FD on CropDisease, ISIC, and ChestX can beat the previous SOTA Dynamic-Distillation. On EuroSAT, CLD+FD can help BYOL to increase the accuracies by 8.4% and 6.06% on 1-shot and 5-shot, respectively. Therefore, our CLD and FD are also helpful for BYOL.
 
We hope our answers address all your concerns and look forward to discussing with you further.

From: ICLR.cc/2023/Conference/Paper233/Reviewer_am4M
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper proposes to tackle the problem of cross-domain few-shot classification, where such problem scenario especially considers the few-shot learning under the case of having (large) domain-shift between base and target domains. In particular, with being motivated to address the large domain gap, this paper follows the prior works (i.e. STARTUP@ICLR2021 and Dynamic-Distillation@NeuraIPS2021) to include a specific assumption of having access to a small portion of unlabelled images in the target domain during the training stage. 

Moreover, the proposed method of this paper also follows the similar train of thoughts as the prior works: 1) having a teacher model trained on base dataset; 2) having self-supervised contrastive learning on the unlabeled target images; 3) the student for the target domain tries to learn from the teacher model by using the knowledge distillation (KD) technique, while the knowledge distillation strategy gradually changes from taking "the teacher trained on base dataset" as the main source of knowledge (similar to STARTUP) in the early iterations to taking "the moving average of the updated teachers on previous iterations" as the main source of knowledge (similar to Dynamic-Distillation) in the later iterations. 

In addition to three aforementioned points which are similar to prior works, the proposed method in this paper has another two key designs: 4) the knowledge distillation implementation is cross-level (i.e. encouraging the shallow layers of student to learn from the deeper layers of teacher), in which the idea is similar to BYOT@ICCV2019 (the slight difference is that BYOT operates self-distillation); 5) due to the phenomenon stemmed from the self-supervised loss and discovered by previous works where the final feature vector for classification has a small number of strongly activated elements while the others are close to zero, the proposed method introduces the "feature denoising" technique to truncate the elements with small magnitudes to zero during the fine-tuning stage (i.e. updating the model to the target domain by using the given target-domain few-shots with labels). 

The proposed method is experimentally evaluated on the BSCD-FSL benchmark and the comparison with respect to several baselines (including ATA and CI which tackle the cross-domain few-shot classification problem without access to target domain data during training, self-supervised methods, STARTUP, and Dynamic-Distillation) is made, where the proposed method is shown to provide superior performance. 
strength_and_weaknesses: Strength:
+ The proposed method is able to outperform different types of baselines (including ATA and CI which tackle the cross-domain few-shot classification problem without access to target domain data during training, self-supervised methods, STARTUP, and Dynamic-Distillation, where the latter two are under the same problem scenario as the proposed method) in a challenging benchmark.
+ The detailed investigations/ablation studies are performed to validate the model designs and hyper-parameter settings in the proposed method (e.g. the model structure for cross-level knowledge distillation, the hyperparameter $\tau$ for building the old student, ablation study for feature denoising, the hyperparameter $\lambda$ to balance self-supervised loss and knowledge distillation loss for training student model, and the hyperparameter $h$ for determining the number of truncation in feature denoising operation).
+ The overall organization is clear and the writing is easy to follow.  

Weaknesses:
- Though I can understand that the large domain gap between base dataset and target domain is quite challenging to be tackled, but the assumption of having access to a portion of target domain data (even it is unlabeled) seems to be quite impractical and hard to envision its usage in real-world scenarios. Moreover, even there exists two prior works (i.e. STARTUP and Dynamic-Distillation) addressing the same problem setting as the proposed method, the authors should still provide sufficient motivations as well as the description on the potential usage of such problem setting in their paper to make the whole story self-contained and persuasive. 
(Besides, in this paper they set to have 20% of target domain data available during training, in which such amount is not "a small portion" in my perspective).
- In addition to the teacher model trained on base dataset and kept fixed during knowledge distillation, the authors introduce another model, named as "old student" (which in implementation is the student model in the previous learning iteration), in order to keep the knowledge of historical students during the learning of student model. Though such design seems to provide benefit to the overall model performance, having such additional model actually increase the computational cost for the overall training procedure. Moreover, the proposed method adopts the linear combination of such old student model and the teacher model as the actual source of knowledge to execute the knowledge distillation, in which such old student model acts similarly to the moving average of historical students (i.e. an ensemble) if we expand the overall iterative procedure (thus being closely related to Dynamic-Distillation@NeuraIPS2021), hence the linear combination of the old student and the teacher becomes an "ensemble of ensemble" or analogously a straightforward integration over the distillation strategies of STARTUP and Dynamic-Distillation. Overall, such the distillation strategy of the proposed method seems to be incremental from the methodology perspective. 
- The cross-level distillation mechanism is closely related to the well-known BYOT@ICCV2019 work (where the shallow layers try to learn the knowledge from the deeper layers), in which the only difference exists on that BYOT is doing self-distillation while the proposed method applies the same idea across two networks. The cross-level distillation proposed in this paper thus seems to be incremental as well. 
- Though the experimental results show that the proposed method provide superior performance, there are some concerns as detailed below: 1) we see that the proposed method using BYOL as its self-supervised loss (denoted as BYOL+CLD+FD (ours) in Table.1) does not present significant performance improvement with respect to Dynamic-Distillation (especially the loss function in Dynamic-Distillation is closely related to BYOL); 2) the baseline SimCLR (self-supervised learning applied on unlabeled target domain data) already provides strong performance (e.g. better than BYOL+CLD+FD (ours) and even competitive with SimCLR+CLD+FD (ours) which also adopts SimCLR in its loss functions). From these two observations, the contribution of the proposed method becomes skeptical (in which the proposed method adopts many different designs into its framework but in results only achieves sightly better, comparable, or even worse performance with respect to the baselines). 
clarity,_quality,_novelty_and_reproducibility: Overall, the clarity of the paper is satisfactory (well organization and good paper writing) but the novelty is incremental (taking the aforementioned weaknesses into consideration, in which the proposed distillation strategy can be simply treated as a dynamic combination over STARTUP and Dynamic-Distillation and the proposed cross-level distillation mechanism is also identical to the well-known BYOT). Moreover, as the problem setting of having access to a portion of target domain data is not well motivated and seems to be impractical, together with the insignificant performance improvement compared to the baselines, the quality of the paper becomes also problematic. 
summary_of_the_review: Due to the limited/incremental novelty, potentially skeptical improvement with respect to baselines, as well as not-well-motivated and impractical problem setting (please refer to weaknesses for more details), I would suggest to reject this paper from ICLR2023 but is willing to change my rating if my concerns can be well addressed.  
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper233/Reviewer_8cJ8
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: In this paper, the authors propose a cross-level distillation method for cross-domain few-shot learning. The are four stages in the pipeline. A pre-trained model is trained on the base classes data, then a student network is trained with cross-level distillation by mimicking the teacher network. A feature denoising step is further added to reduce overfitting of the features. The authors conduct extensive experiments on the BSCD-FSL benchmark to show the effectiveness of the proposed approach. 
strength_and_weaknesses: Strength: 

1. The writing of the paper is clear and the organization is also well structured.
2. The idea of cross-level distillation is also interesting and novel. 
3. The authors also conducted extensive ablations to show the benefits of the proposed approach. 

Weaknesses:

1. There are too many hyperparameters in the method. It may be difficult to tune the parameters for new applications.
2. The introduction of the old student network may introduce additional overhead.

The authors may have some discussions on how to choose the hyperparameters and compare the computational complexity of the method with the baselines. 
clarity,_quality,_novelty_and_reproducibility: Clarity: overall, the paper is clearly written and easy to understand. Although there should be more motivations for the design choices of cross-level distillation. 

Quality: the quality of the paper is good.

Novelty: the idea of cross-level distillation is novel for cross-domain few-shot learning. 

Reproducibility: the paper seems reproducible. 
summary_of_the_review: The paper proposes a novel method for cross-domain few-shot learning and extensive experiments show the effectiveness of the method. The main drawbacks of the method are the 1) introduction of excessive hyperparameters and 2) the computation complexity. 

Post-rebuttal:

I have read the authors' rebuttal. It partially addresses my concerns. I tend to keep my rating but do not have an inclination to support the paper if other reviewers have concerns. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper233/Reviewer_yRW4
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper tackles the problem of cross-domain few-shot learning, where the base dataset and the target domain are far away from each other. It builds upon past distillation-based frameworks, where unlabeled data from the target domain is used to distill a source teacher, pretrained on the base dataset. It introduces multiple modifications to this basic approach, including using distillation of multiple layers, a way of feature selection on the target problem, and an approach for avoiding inheriting the teacher's biases.

strength_and_weaknesses: Strengths:
1. The problem of cross-domain few-shot learning is a practically significant problem.
2. The results obtained by the proposed method are much more accurate than prior state-of-the-art.
3. The paper has thorough ablations for its many contributions.

Weaknesses:
1. The contributions seem like engineering innovations rather than new insights. However, each contribution has well thought-out motivations, so I am less worried.
clarity,_quality,_novelty_and_reproducibility: Generally I find the paper well written. The code will be released, and that should make the approach reproducible. The insights though not particularly novel are still well motivated and worth knowing for the community.
summary_of_the_review: Despite my short review, I find the contributions useful, the problem significant, and the result impressive. I am in favor of acceptance.
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 8: accept, good paper