Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: 
The paper proposed a transfer learning setting where the target domain varies/evolves over time and the source domain is considered static. The paper uses C-divergence to measure label-dependent domain discrepancy between source/previous target domain and the current target domain and provided a theoretical bound. The paper also used supervised VAE for CONTE algorithm and included C-divergence as a part of the objective function.

The paper address an interesting and an important practical problem on non-stationary target distribution. The proposed (semi-supervised) divergence measure and it’s relation to the previously proposed (unsupervised) divergence measures such as (A-distance and disc distance) is well-motivated. Please consider the following questions to help me understand the problem. 

### Major Questions:

* The reason for the label dependency is motivated as explained but I believe that the usage of adversarial semi-supervised VAE was not properly motivated and related it the C-divergence.  

* Sec 5.2 needs a little clarification for us readers to understand it clearly.  For instance, How is minimizing Eq 6 minimizes Continuous Error bound in Theorem 5.1. The first two terms in Eq 6 is related to the bound but Why is the ELBO term needed and how it is related to the bound. Eq 7-9 follows directly from the Kingma et al.’s Semi-supervised VAE so no explanation is needed.

* Please help me to understand how to get Eq 4 from Lemme 4.2. I couldn’t find the related steps in the Suppl. On a related question, it seems that, in Eq 11 or the one above … if the F-classifier doesn’t distinguish between the domains (across the time or between source and target domains), ie., if F-classifier always say  that the example is from one of the two domains, then one of the two terms in the bracket will be 0 and the other will be 1 and it will make the distance measure to 0. 

* I believe the problem setting is very closely related to the Continual learning setting,  addition. To the Catastrophic forgetting, most recent work in Continual learning focuses on Negative transfer (such as Gradient Episodic Memory). The paper will be stronger if the key comparison to baselines in Continual learning is added.

### Minor Questions:

* Notation issue in Eq.9 for the unlabeled U(x) to estimate the log-likelihood of the marginal distribution of the features. ‘y’ shouldn’t be in the U function.

* Overloaded use of notation F in Eq. 11 and in Suppl. 

* In many practical applications, for instance product reviews, if a new version of the product released periodically with new sets of features. Even though, the review data is  not only from the same target distribution but sample the different regions of this same target distribution. Can the current distance measure and  the CONTE algorithm be adapted to this problem setting.

* Related question to the one above: Can we automatically learn \mu from the data? If there is transfer from Ti to Tt+1 but not from Tt to Tt+1, will the proposed method can be adapted to this problem setting. Does the information from Ti is retained in Tt from previous learning cycle, such that, it will be used by Tt+1 task?

i have read the author(s)' comments and I have updated the ratings based on their replies. Thanks for very extensive clarification. Adding these comments in the final revision would significantly improve the quality of the paper.
review_title: Interesting problem on Non-stationary Transfer Learning 

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:

This paper studies how to transfer the information in the static source domain to the time-evolving target domain. This paper proposes a domain discrepancy measure and an algorithm for continuous transfer learning. The results seem to be interesting and the problem this paper studies is important. However, the domain rate in the main results and algorithm could be easily generalized which can make the results more broadly applicable. Moreover, it needs more clarification about the motivation of using the C-divergence measure in the time-evolving target domain.

Major comments:

1. Domain decay rate: There is a domain decay rate in all the main results and the algorithm. From the proof of all the main results, the rate $\mu^{t-j}/\bar{\mu}$ can be easily generalized to any arbitrary weight $\omega_j$ with the constraint $\sum_{j = 0}^t \omega_j = 1$ and $\omega_j \geq 0$. Hence, there is no assumption about $\mu$ in the loss function. Allowing general $\omega_j$ is more appealing for many reasons. 

First, this paper indexes the source domain by $T_0$. The CONTE algorithm assigns similar weights to the source domain ($\mu^t$) and target domain at time 1 ($\mu^{t-1}$), but a different weight to the target domain at time $t$ ($\mu^0$) (when $\mu \neq 1$). This algorithm seems to implicitly assume the target domain at time 1 is more similar to the source domain than the target domain at time $t$, which seems unnatural and requires further justification. However, using  $\omega_j$ does not have this issue. 

Second, it is possible that the time-evolving domain has a cyclic or seasonality pattern. Even in the experiment setup in Section 6, $\sin(\theta) = \sin(\frac{i\cdot \pi}{t})$ is cyclic. It may make more sense to assign a larger weight to the time period whose domain is more similar to the current domain. 

2. Potential improvement of the algorithm: Following the previous point, the CONTE algorithm could potentially be improved. Since the domain discrepancy measure $d_c(\cdot, \cdot)$ can be estimated, we could assign the weight $\omega_j$ inverse proportional to $d_c(\cdot, \cdot)$ and the error could be potentially reduced. 

3. C-divergence measure: It seems the C-divergence measure does not involve the time concept and is generally applicable to transfer learning tasks beyond continuous transfer learning? Then why is the C-divergence measure particularly of interest in the context of continuous transfer learning and why is it more preferred to use the C-divergence measure rather than the conventional measures, such as A-divergence in this setting? Some discussions on this point and numerical comparisons will be helpful. 
review_title: Interesting question and results, but easily generalizable

final_score: rating: 6: Marginally above acceptance threshold
review_content: 
(1) This article is more detailed in its theoretical analysis, and the C-divergence has advantages over other divergence from the theoretical analysis. 

(2) This article proposes a new continuous transfer learning setting. It theoretically analyzes the upper bound of the target domain error, and utilizes the C-divergence to measure the joint distribution discrepancy in the two domains.

(3) Some parts of the presentation needs to be improved, such as the presentation of negative transfer in section 4.3. How to use other methods such as DAN to this new setting? It should be described in more details in section 6. How to set the target domain label in section 6?

(4) Although the results outperforms some other methods, the experiments are not sufficient. In experiments, the authors only do experiments in some easy task in DA, and no experiments are shown for the effect of the hyper-parameter mu.
Detailed Evaluation:
 
(5) About formula (10), why does the cross-entropy of T_{t+1} calculate t+1 times?

review_title: This paper focuses on the continuous transfer learning setting with a time evolving target domain. A major challenge is that the relationship between the source domain and the target domain changes over time. This paper proposes a new C-divergence to measure the discrepancy between domains. It could be utilized to instantiate a tighter error upper bound in the continuous transfer learning setting. The clarity of this paper still needs some improvement. And experimental results are insufficient.

Meta-review: The paper proposes transfer learning where the target domain data is evolving along time.  They use both labeled and unlabeled data to learn domain and time-invariant features based on a discrepancy measure they introduce.  Their proposed algorithm uses VAE to learn such features.  Reviewers have mixed response, although the author feedback did help.  
The main limitation with the paper is that it does not seem to be aware of the very extensive literature on continuous domain adaptation.  The related work only discusses papers on transfer learning, multi-source domain adaptation, and continuous learning.  But ignores papers on continuous domain adaptation which are much more related to this paper.  The most recent of these that appeared in ICML 2020 also attempts to learn time invariant features using adversarial methods.  Unfortunately, the reviewers seem to be also unaware of this literature:

1.  Continuously Indexed Domain Adaptation,  Hao Wang, Hao He, Dina Katabi, ICML 2020
2.  Active Adversarial Domain Adaptation 
3.  Continuous Domain Adaptation using Optimal Transport
4. Learning to Adapt to Evolving Domains - NeurIPS 2020
5.  Judy Hoffman, Trevor Darrell, and Kate Saenko. Continuous manifold based adaptation for evolving visual
domains. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 867–874,
2014.
6 Massimiliano Mancini, Samuel Rota Bulo, Barbara Caputo, and Elisa Ricci. Adagraph: Unifying predictive and
continuous domain adaptation through graphs. In Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pages 6568–6577, 2019.

7 Atsutoshi Kumagai and Tomoharu Iwata. Learning future classifiers without additional data. In Thirtieth AAAI
Conference on Artificial Intelligence, 2016.


Decision: Reject