Adversarial Reweighting for Partial Domain Adaptation

Abstract: Partial domain adaptation (PDA) has gained much attention due to its practical setting. The current PDA methods usually adapt the feature extractor by aligning the target and reweighted source domain distributions. In this paper, we experimentally find that the feature adaptation by the reweighted distribution alignment in some state-of-the-art PDA methods is not robust to the ``noisy'' weights of source domain data, leading to negative domain transfer on some challenging benchmarks. To tackle the challenge of negative domain transfer, we propose a novel Adversarial Reweighting (AR) approach that adversarially learns the weights of source domain data to align the source and target domain distributions, and the transferable deep recognition network is learned on the reweighted source domain data. Based on this idea, we propose a training algorithm that alternately updates the parameters of the network and optimizes the weights of source domain data. Extensive experiments show that our method achieves state-of-the-art results on the benchmarks of ImageNet-Caltech, Office-Home, VisDA-2017, and DomainNet. Ablation studies also confirm the effectiveness of our approach.


Reviews and author responses:

 Thanks for the comments again. We promise to include all those discussions in the paper.

 Thanks for the comments again. We promise to include all those discussions in the paper.

 Thank you for your comments.

Most of my concerns are solved in the response.
I also noticed that the responses to the other reviewers are also fruitful to understand this work. I hope that all those discussions will be included in the final version of the paper. I'm keeping my original rating.

Thank you.

 I have read the response and other reviews, while the response indeed addressed major parts of my concerns, I think the authors have given enough results to demonstrate the effectiveness of the proposed model. Therefore I change my previous decision, and raise my rating from "5: Marginally below the acceptance threshold" to "6: Marginally above the acceptance threshold" since the response makes the paper closer to acceptance.

This paper proposed a training algorithm that alternately updates the parameters of network and optimizes the weights of source domain data. A novel Adversarial Reweighting (AR) approach is proposed to adversarially learn the weights of source domain data in order to align the source and target domain distributions.  - Figure 1 shows that the different reweighted distribution alignments are not robust to noisy weights, but experiments lack verification of the robustness of the proposed method. 
- Table 1 is not clear, and a clearer caption about domain adaptation setting should be presented.
- Figure 1 caption S->R and C->P are not clear.
- The motivation and contributions are not clear.  Introduction and Conclusion both present that the reweighted distribution alignment is not robust to the “noisy” weights of source domain data and may hurt the performance of learning in target domain. But there are no discussions and experiments about robustness of the proposed method.
- There are some typos. 
 Adversarial training may bring more calculations, and adversarial learning may be instability during training, it will be better if there are extra discussions about these points. 

 We thank the reviewer for the comment.

**Q1**: Motivation and contributions.

1) We experimentally find that when the source data weights contain "noise", some state-of-the-art reweighted distribution alignment losses (e.g., the reweighted adversarial training losses in PADA [4] and BAA [22], and the reweighted MMD loss [21]) for PDA are not robust to the weight noise. On VisDA and DomainNet datasets with larger diversity and domain gaps, additionally utilizing the reweighted distribution alignment losses of PADA, BAA, and MMD degrades the performance of the baseline method, which uses only source classification loss and entropy loss on the target domain.

2) We further propose a novel adversarial reweighting approach for PDA without using the reweighted distribution alignment losses. We learn to reweight the importance of source domain data for reducing the domain shift, measured by the Wasserstein distance. We then train the transferable model for target domain by the reweighted source classification loss that is shown to be more robust to weight noise (please refer to Q2). Due to the computational issue of the Wasserstein distance, we use the approximation of its dual form, deducing the adversarial reweighting model. We also propose an iterative training algorithm that alternately updates the network parameters and learns the source weights by adversarially training the discriminator and solving cone programming. Extensive experiments on five benchmarks confirmed the effectiveness of the proposed approach.

**Q2**: Verification of the robustness of AR to weight noise.

To verify the robustness of AR w.r.t. weight noise, we conduct simulation experiments for our method under different noise levels. Specifically, we first obtain the source data weights $w_i, i=1,2,..,n_s$, through our adversarial reweighting model. For the noise level $p$, we simulate the weight of the $i$-th sample that belongs to source-only classes by $p*w_i/\sum_{j=1}^{n_s}I(y^s_j\in \mathcal{Y}^s\backslash\mathcal{Y}^t)w_j$, for $i$ such that $y^s_i\in \mathcal{Y}^s\backslash\mathcal{Y}^t$ (i.e., the $i$-th source sample that belongs to source-only classes is assigned noisy weight in proportional to the weight learned by our adversarial reweighting model). Similarly, the $i$-th sample that belongs to source-shared classes is assigned noisy weight by $(1-p)*w_i/\sum_{j=1}^{n_s}I(y^s_j\in\mathcal{Y}^t)w_j$, for $i$ such that $y^s_i\in \mathcal{Y}^t$. The results in the tasks S$\rightarrow$R on VisDA-2017 and C$\rightarrow$P on DomainNet are reported in Tables r3-1 and r3-2.

Table r3-1. Results for AR with varying noise levels in the task S$\rightarrow$R on VisDA-2017.

|noise level|0.0|0.1|0.2|0.3|0.4|
|:-|:-:|:-:|:-:|:-:|:-:|
|Baseline|84.72|84.72|84.72|84.72|84.72|
|AR|89.76|88.01|88.74|86.02 |85.83 |


Table r3-2. Results for AR with varying noise levels in the task C$\rightarrow$P on DomainNet.

|noise level|0.0|0.1|0.2|0.3|0.4|
|:--|:--:|:--:|:--:|:--:|:--:|
|Baseline|49.40|49.40|49.40|49.40|49.40|
|AR|56.86|55.42|53.97|52.17|51.45|

Since the Baseline does not use reweighting, its results are the same under different noise levels. We observe that our AR consistently outperforms Baseline when the noise level ranges from 0.0 to 0.4. We will include these results as figures in our main paper.

**Q3**: Caption of Table 1.

Table 1 compares  the results of different reweighted distribution alignment losses (PADA, BAA, and MMD) for partial domain adaptation on five benchmark datasets. "w/o alignment" denotes the approach that minimizes the source classification loss and entropy loss for target domain data (target entropy loss). "PADA"/"BAA"/"MMD" denote the approaches that additionally minimize the reweighted distribution alignment losses of PADA/BAA/MMD, compared with "w/o alignment".  The results of "PADA"/"BAA"/"MMD" are lower than that of "w/o alignment" on VisDA-2017 and DomainNet datasets, indicating that these reweighted distribution alignment losses lead to negative domain transfer on the two datasets.
The caption of Table 1 will be "Comparison of results of different reweighted distribution alignment losses (PADA, BAA, and MMD) for partial domain adaptation on five benchmark datasets".

**Q4**: Caption of Figure 1.

In Fig. 1, S$\rightarrow$R is a partial domain adaptation task on VisDA-2017 dataset. In this task, the Synthetic domain is set to source, and the data of the first six classes in the Real domain are set to target. C$\rightarrow$P is a partial domain adaptation task on the DomainNet dataset. The Clipart domain and partial Painting domain (the first 40 classes among all 126 classes) are set to source and target domains, respectively. The details are in section 4. 
We will revise the caption of Fig. 1 in the main paper as "The results for reweighted distribution alignment losses with generated source data weights with varying noise levels in the tasks of (a) Synthetic (S) $\rightarrow$ Real (R) on VisDA-2017 and (b) Clipart (C) $\rightarrow$ Painting (P) on DomainNet".


**Q5**: Stability of the training process.

In our training algorithm, we alternately update the network parameters and learn the source data weights.  The source data weights  are learned by adversarially training the discriminator and solving cone programming. As an iterative algorithm, we found that our training process is stable in all experiments. As examples, we plotted the curves of test errors on the target domain in Fig. 3(c) in the main paper, and the training losses in Fig. 1(a), 1(d), and 1(g) in Supp. E in the supplementary material. Please refer to them. We found that both the test errors and the training losses stably decrease and converge.


 We thank the reviewer for the comments. We will carefully revise the paper accordingly.

**Q1** and **Q2**: More realistic simulation experiments to study the influence of weight noise on reweighted source classification loss and reweighted distribution alignment loss.

To make the simulation experiments more realistic, we assign the noise to each source-only class in proportion to its weight predicted by the classifier. If the weight predicted by the classifier is $c^i, i\in \mathcal{Y}^s$, for the noise level $p$, the simulated weight for each source-only class is $p*c^i/\sum_{k\in \mathcal{Y}^s\backslash\mathcal{Y}^t}c^k, i\in \mathcal{Y}^s\backslash\mathcal{Y}^t$.
Similarly, the simulated weight for each source-shared class is $(1-p)*c^i/\sum_{k\in \mathcal{Y}^t}c^k, i\in \mathcal{Y}^t$. To study the influence of weight noise on the classification loss, we also use the simulated weights to reweight the importance of source data in the classification loss. We conduct the experiments on VisDA-2017 and DomainNet datasets, which have larger domain gaps and may be more realistic than some of the other datasets (e.g. Office-31). The results in the tasks S$\rightarrow$R on VisDA-2017 and C$\rightarrow$P on DomainNet are reported in Tables r2-1 and r2-2, respectively.

Table r2-1. Results for varying noise levels in the task S$\rightarrow$R on VisDA-2017 dataset.


|noise level|0.0|0.1|0.2|0.3|0.4|
|:--|:--:|:--:|:--:|:--:|:--:|
|w/o Alignment|82.32|82.32|82.32|82.32|82.32|
|w/o Alignment and w/ class-rew|86.24|85.46|85.46|84.85|83.74|
|PADA (w/ class-rew)|89.29|78.43|68.85| 62.08|57.27|
|BAA (w/ class-rew)|86.32|78.82|71.94|69.47|58.97|
|MMD (w/ class-rew)|85.33|82.29|75.16|67.79|55.77|

Table r2-2. Results for varying noise levels in the task C$\rightarrow$P on DomainNet dataset.

|noise level|0.0|0.1|0.2|0.3|0.4|
|:--|:--:|:--:|:--:|:--:|:--:|
|w/o Alignment|50.14|50.14|50.14|50.14|50.14|
|w/o Alignment and w/ class-rew|55.26|54.94|52.61|52.97|50.72| 
|PADA (w/ class-rew)|57.66|50.82|49.30|46.39|42.76| 
|BAA (w/ class-rew)|59.97|54.17|52.79|47.38|42.96| 
|MMD (w/ class-rew)|57.03|51.52|48.07|44.73|40.22| 

In Tables r2-1 and r2-2, "w/o Alignment" denotes the approach that minimizes the source classification loss and entropy loss for target domain data (target entropy loss). "w/o Alignment and w/ class-rew" denotes the approach that minimizes the reweighted source classification loss and target entropy loss. "PADA (w/ class-rew)"/"BAA (w/ class-rew)"/"MMD (w/ class-rew)" denote the approaches that minimize the reweighted source classification loss, target entropy loss, and the reweighted alignment losses of PADA/BAA/MMD.  The results for the reweighted alignment losses (PADA, BAA, and MMD) decrease with the noise level increases, and become lower than the results of "w/o Alignment and w/ class-rew" and "w/o Alignment" when the noise level is larger than 0.3. 
It is also seen that when the noise level ranges from 0 to 0.4, the approach "w/o Alignment and w/ class-rew" consistently outperforms the approach "w/o Alignment" (comparing the 2-th and 3-th lines). This indicates that adding weight to the source classification loss may be more robust w.r.t. weight noise than adding weight to alignment losses of PADA/BAA/MMD.  These results will be included in the main paper as figures to replace Fig. 1 and the corresponding analysis for Fig. 1 will be revised accordingly.


**Q3**: Why our feature extractor may satisfy the requirement that the source-shared-class data is closer to the target data than the source-only-classes, in learned feature space.

In our method, besides the source classification loss, we also minimize the entropy loss on the target domain (see Eq. (1)). As stated in line 175, the entropy loss is used only to update the feature extractor (instead of both the classifier and feature extractor). Our classifier is the spherical logistic regression layer that outputs the cosine similarity of the target features and source prototypes [13]. With lower entropy, the target feature may move towards some of the source prototypes to make its classification probability more certain. Therefore, the learned target features may be not distant from the source features. Meanwhile, minimizing the entropy may increase the discrimination of extracted features on the target domain.


**Q4**: Ablation on JS-divergence and MMD.

We conduct ablation studies for JS-divergence and MMD to learn the weights in our framework (denoted as AR (w/ JS) and AR (w/ MMD), respectively), on Office-Home dataset. The results are reported in Table r2-3.

Table r2-3. Ablation study for JS-divergence and MMD.

|Method|Ar$\rightarrow$Cl|Ar$\rightarrow$Pr|Ar$\rightarrow$Rw|Cl$\rightarrow$Ar|Cl$\rightarrow$Pr|Cl$\rightarrow$Rw|Pr$\rightarrow$Ar|Pr$\rightarrow$Cl|Pr$\rightarrow$Rw|Rw$\rightarrow$Ar|Rw$\rightarrow$Cl|Rw$\rightarrow$Pr|Avg|
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|AR (w/ JS)| 61.37|80.50|89.73|74.12|73.83|80.89|76.58|61.31|86.69|__80.53__|63.28|84.65|76.12 
|AR (w/ MMD)|64.54|80.56|86.80|70.79|__74.42__|81.26|75.12|59.97|87.49|77.59|62.99|85.08|75.55 
|AR|__67.40__|__85.32__|__90.00__|__77.32__|70.59|__85.15__|__78.97__|__64.78__|__89.51__|80.44|__66.21__|__86.44__|__78.29__|


We can see that our AR using the Wasserstein distance outperforms AR (w/ JS) and AR (w/ MMD). When the supports of source and target data distributions are disjoint, the Wasserstein distance may be more suitable to measure their distance than JS-divergence [1]. This may account for the improvement of performance achieved by AR over AR (w/ JS). The MMD with widely used kernels may be unable to capture very complex distances in high dimensional spaces [1, R1], which may make it less effective than the Wasserstein distance in our framework. The results will be included in the main paper. 


**Q5**: Ablation on learning one weight for each class.

We conduct experiments for learning one weight for each class in our framework (denoted as AR (w/ OWEC)) on Office-Home dataset. Table r2-4 reported the results.

Table r2-4. Ablation study for learning one weight for each class.

|Method|Ar$\rightarrow$Cl|Ar$\rightarrow$Pr|Ar$\rightarrow$Rw|Cl$\rightarrow$Ar|Cl$\rightarrow$Pr|Cl$\rightarrow$Rw|Pr$\rightarrow$Ar|Pr$\rightarrow$Cl|Pr$\rightarrow$Rw|Rw$\rightarrow$Ar|Rw$\rightarrow$Cl|Rw$\rightarrow$Pr|Avg|
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|AR (w/ OWEC)| 64.02|85.15|86.36|72.26|60.39|79.91|77.78|64.42|88.40|__82.09__|65.73|86.16|76.06|
|AR|__67.40__|__85.32__|__90.00__|__77.32__|__70.59__|__85.15__|__78.97__|__64.78__|__89.51__|80.44|__66.21__|__86.44__|__78.29__|

In Table r2-4, AR with learning the weight for each sample significantly outperforms AR (w/ OWEC) that learns one weight for each class. 
If the weight is learned for each sample, the importance of samples in the same class could be different. It is possible to assign higher weights to samples closer to the target domain, even in the same source class. The model trained in this case may be more transferable, because samples (even in the source-shared classes) less relevant to the target domain become less important.
This result will be also included in the main paper.


[R1] Reddi et al., On the high dimensional power of a linear-time two sample test under mean-shift alternatives. AISTATS, 2015.

 We thank the reviewer for the comments and suggestions. We will revise our paper accordingly.

**Q1**: Convergence of the objective and weight in Eq. (4). 

As empirical evidence, we plotted the objective function and relative difference of weight (reflecting the change of weight) during the iterative training process in Supp. E in the supplementary material. Please refer to Fig. 1(b), 1(e), and 1(h) in Supp. E for the objective functions (approximated distances) and Fig. 1(c), 1(f), and 1(i) in Supp. E for the relative differences of the weight. We can see that after several iterations, the objectives only fluctuate slightly and relative differences of weight have small non-zero values, which may be because the feature extractor is optimized by SGD using approximated gradients over mini-batch. We will study the theoretical analysis of the convergence of our approach in future work.

**Q2**: Visualization of learned weights.

We have visualized the learned weights in Fig. 4(a) and 4(b) in the main paper. Please zoom in and view them. We will make them clearer in the revised paper. In both Fig. 4(a) and Fig. 4(b), the weighted source (blue) and target (brown) features are plotted in the figure on the right, where the clearer the blue is, the greater the weight of the source points. Intuitively, the source samples distant from target data are assigned with small weights. We also report the average weight of each class for the task of S$\rightarrow$R on VisDA-2017 dataset, in Table r1-1.

Table r1-1. Average weight of each class in S$\rightarrow$R on VisDA-2017.

|Class ID|0|1|2|3|4|5|6|7|8|9|10|11|
|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|
|Weight|0.28|1.47|0.61|3.05|2.22|0.03|0.0|0.0|0.0|0.0|0.02|0.10|

The first six classes are source-shared classes (0-5). In general, the average weights of the source-shared classes (except  class 5) are larger than the source-only classes (6-11).  We will also include the histograms of these average weights of each class in the main paper.

**Q3**: Comparison with BA$^3$US [22].

We will report the results of BA$^3$US in Tables 2, 3, and 4 in the main paper. The results of BA$^3$US and our AR are compared in Table r1-2.

Table r1-2. Results of BA$^3$US and AR.

|Method |Office-31|Office-Home|ImageNet-Caltech|VisDA-2017|DomainNet|
|:----|:----:|:----:|:----:|:----:|:----:|
|BA$^3$US|__97.81__|75.98|83.68|68.71*|60.63*|
|AR (ours)|96.93|__78.29__|__84.69__|__83.62__|__65.47__|

*Since the original paper of BA$^3$US does not report the results on VisDA-2017 and DomainNet datasets, we run its official code  to obtain the results on VisDA-2017 and DomainNet datasets. We can see that on Office-Home, ImageNet-Caltech, VisDA-2017, and DomainNet datasets, our AR significantly outperforms BA$^3$US, especially on VisDA-2017 and DomainNet with larger domain gaps compared with other datasets. On Office-31, AR obtains comparable results. We think this maybe because, on Office-31, the prediction of the classifier on the target domain could be reliable (since the accuracy is around 98\%) and in such a case, the reweighted distribution alignment in BA$^3$US may lead to positive domain transfer.

**Q4**: Combining AR and other DA techniques.

We combine our AR with the Adaptive Uncertainty Suppression (AUS) loss [22] and label smoothing (LS), which are techniques used in state-of-the-art methods of BA$3$US [22] and SHOT [R1], respectively. 
The results for AR+AUS and AR+LS on Offcie-Home dataset are in Table r1-3. 

Table r1-3. Results of AR+AUS and AR+LS and their corresponding baselines on Office-Home dataset.

|Method|Ar$\rightarrow$Cl|Ar$\rightarrow$Pr|Ar$\rightarrow$Rw|Cl$\rightarrow$Ar|Cl$\rightarrow$Pr|Cl$\rightarrow$Rw|Pr$\rightarrow$Ar|Pr$\rightarrow$Cl|Pr$\rightarrow$Rw|Rw$\rightarrow$Ar|Rw$\rightarrow$Cl|Rw$\rightarrow$Pr|Avg|
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|Baseline+AUS|63.34|81.12|86.14|74.01|76.53|79.79|77.69|62.57|86.42|78.33|62.69|84.38|76.08|
|AR+AUS|68.24|85.60|90.61|75.91|77.54|81.89|81.73|66.39|89.01|83.65|65.61|86.95|79.43| 
|Baseline+LS|62.99|83.59|87.30|74.20|73.05|81.67|79.25|63.46|87.85|78.97|64.54|84.76|76.80|
|AR+LS|65.67|87.36|89.62|79.25|75.01|86.97|80.81|65.79|90.61|80.81|65.25|86.12|79.44|

From Table r1-3, it can be observed that combining AR and AUS/SL consistently improved the results of their corresponding baselines. The achieved results on Office-Home are state-of-the-art. We will include these results in the main paper. SHOT [R1] will be also referred to in Table 2.

[R1] Liang et al., Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation, ICML 2020.


The paper introduces the adversarial reweighting (AR) technique for estimating weights in the source domain. Compared to the previous methods on partial domain adaptation, the proposed method seems robust on the effect of negative domain transfer during the adaptation. Authors demonstrate the effectiveness of AR in several partial domain adaptation benchmarks.  [Originality] The proposed method, adversarial reweighting, combines reweighting scheme and adversarial learning, which is a brand-new idea, although there have been several methods for estimating weight for the samples.

[Quality] The proposed method is technically sound and plausible. But, the analysis on the learning behavior is not illustrated in the paper. I will give my questions on the limitation section.

[Clarity] There are some curious things in the experiments about state-of-the-art. First, why did you report the performance of BAA [22] instead of BA$^3$US [22] in table 2, 3, and 4. Second, SHOT [R1] is not referred to in table 2, which achieves 78.3 % in Office-Home with partial domain adaptation protocol.

[Significance] As I know, the method is well-designed and the formulation is quite aligned with the partial domain adaptation task. The experiment results are also promising because it consistently improves the baseline over several datasets.

[R1] Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation, ICML 20 [Analysis on the proposed method]
First, is there any problem with convergence of the objective in eq 4 and its weight, because adversarial reweighting scheme is implemented on the concept of adversarial learning.
Second, could you visualize or plot the learned weights using histograms or etc? Is there evidence related to negative domain transfer?

[Benchmark] As stated in the clarity section, currently the proposed method is not state-of-the-art performance. Can AR be combined with the other state-of-the-art or comparable method rather than used baseline? If it consistently improves the new baseline, this can reinforce the effectiveness of AR.

The paper aims to solve the challenge of 'noise' in the source domain weight for partial domain adaptation.

The paper proposes an adversarial reweighting approach to adversarially learn to reweight the source domain data for aligning the distributions of the source and target domains.

The paper conduct experiments on five benchmark datasets: Office-31, ImageNet-Caltech, Office-Home, VisDA-2017, and DomainNet and show that the proposed method outperforms the prior partial domain adaptation methods.   Strength:

Partial domain adaptation is an important problem setting given the prevalence of large-scale datasets for different applications.

The paper conduct experiments on mainstream datasets with various domain gaps and sizes.


Weakness:

The paper tries to give an empirical study on why current partial domain adaptation methods on aligning reweighted distribution fail. However, within the empirical study, the authors simulate the situation of noise as all the source-only classes share the same weight. This may not be the case for real datasets. Could the authors do the empirical study based on the realistic datasets and trying to vary the level of noise more realistically?

The authors only study the influence of the weight noise on the distribution alignment but fail to show the influence of the weight noise on the source classifier. Such an unfair study of weight noise cannot ensure that adding weight on classifier is the best choice.

The adversarial learning process highly relies on the quality of the feature extractor. You can imagine that if the feature extractor map the input data randomly, then the features z for all the source and target data are random. Then for any weight assignment, the W-distance will be very big. So there is no way to learn the weight assignment. The strong assumption for the proposed method is that in the latent feature space, the distance between the source shared class is closer to the target data than the source only classes. Could the authors explain why the feature space learned by source classification loss satisfy this requirement?

The paper selects the W-distance as the method to measure the distribution distance but no ablation is done for this choice. There is no guarantee that whether the JS-divergence (adversarial networks) or the MMD distance will perform better. 

The authors adopt one weight for each source data but prior works usually use one weight for each source class because if a class is a source only class, the ground-truth weight for all the data for this class should be 0, which are the same. So we just need to assign one weight to each source class instead of each sample. Assigning one weight to each sample increases the number of parameters. No obvious social impact.

Meta-review: This paper tackles the problem of partial domain adaptation. After considering the reviews, author rebuttal, and discussions the reviewers remained split on their recommendations with one accept, one borderline accept, and one borderline reject. 2HKg found that the work contributed a “brand-new idea” combining re-weighting and adversarial learning and KSRV increased their recommendation from borderline reject to borderline accept post-rebuttal citing that the authors had “given enough results to demonstrate the effectiveness of the proposed model.” The AC has reviewed the weaknesses brought up by 3wdy and determined that the responses and additional experiments authors provided in the rebuttal are sufficient. The authors have in general provided many additional experiments and clarifying thoughts during the rebuttal and need to be sure to include these in the final paper.

Decision: accept