Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: Further comments after the rebuttal

First, I would thank the authors for the great efforts in trying to address my comments. I should say that many of my previous concerns have been clarified. Again, I like the theoretical part. Nonetheless, I agreed with some other reviewers that the experiments seemed not to fully convince me.  Particularly to myself, the authors may want to show some analysis on how their method could truly stand out by even a toy example, which may further enhance the paper. I tend to keep my original rating  after reading all the rebuttal messages in the whole thread.

====================
This paper proposed a f-domain adversarial learning framework (f-DAL) using the complete family of f-divergences as domain discrepancy measurements. The proposed method extended the seminal works of Ben-David et al. (2007; 2010a;b); Mansour et al. (2009) that provided generalization bounds for UDA based on a special type of f-divergence. To enable the complete family of f-divergences to measure the domain distribution discrepancy, the variational characterization of f-divergences is leveraged to estimates f-divergences from samples by turning the estimation problem into variational optimization. Furthermore, a new type of discrepancy was used to compare two marginal distribution and its corresponding generalization bound was tailored for a general class of f-divergence, which can mitigate two limitations of the seminal works. In addition, the optimal solution of f-DAL is a Stackelberg equilibrium, which allows f-DAL to incorporate the latest optimizers from the game-optimization literature, such as Aggressive Extra-Gradient (AExG).

The key strength of this paper is, it enables a generalized version of f-divergences that can be used for adversarial domain adaptation. This is valuable for UDA algorithms in many application areas. As verified in the experiments, the proposed method achieved comparable results in Office-31 and Office Home dataset.

The paper was well organized and written. However, there are some major concerns as follows:
(1)	Although the authors provided theoretical insight for their method, the performance improvements are not very significant. 

(2)	The ablation study needs to be further conducted. It is necessary to eliminate the Aggressive Extra-Gradient (AExG) and other learning strategies, such as spectral normalization (SN) and GRL warm-up strategy when compared with state-of-the-art methods. More specifically, the authors employed an integrated model that makes the comparison experiments seemingly unfair especially in the case of using MDD as the baseline. Although achieving the Stackelberg equilibria is a good characteristic of f-DAL, AExG may also increase the performance. Furthermore, the spectral normalization (SN) and GRL warm-up strategy may also potentially improve the performance. However, these techniques were seemingly not applied in other comparison methods in Table 2 and Table 3.

(3)	The experiments may not answer the general question ‘(2) Is there a better universal notion of f-divergence that achieves significant performance gains across different datasets?’. In Figure 3, it is hard to say that there is a consistent increment tendency for a specific f-divergence across different transfer tasks. More importantly, it is better to demonstrate if choosing different f-divergence can lead to consistent performance increase on more diverse datasets or applications. 

(4)	For a typical UDA theory method, it is better to show results on the VisDA-2017 dataset.

(5)	The authors offered some hints to generalize the proposed method to a multi-class scenario. It is better to provide theoretical insight/details.

(6)	It may enhance the paper if more illustrative or even toy examples can be conducted to further show clearly the advantages of the proposed method. Again, the improvement may not be very obvious as observed in the experiments though it is theoretically interesting. 

review_title: A paper with a good theory but  weak experimental evaluation 

final_score: rating: 5: Marginally below acceptance threshold
review_content: ###############

Summary: This paper proposes a new generalization bound for domain adaptation based on f-divergences. Accordingly, a new algorithmic framework is also derived.

###############

Pros:

Some theoretical results are new – to the best of my knowledge, there are no existing works establishing learning bounds for DA with f-divergences.
One benefit of f-divergence is that it can offer a very general framework that accommodates various discrepancy measures for DA. The authors empirically evaluate different choices of divergences over several benchmarks.
The paper is well-written and easy to follow.

Cons:

My main concern is the theoretical contribution and motivation of this work. Given a large variety of divergence measures (e.g., H-divergence, JS-divergence, Wasserstein distance, MDD…), this paper does not give me any new theoretical insights compared to previous results. From an algorithmic perspective, extending JS divergence to more general f-divergences has already studied in GAN training [1].

In addition, most of the theoretical analysis follows standard steps of existing works. For example, the high-level idea of Definition 3 follows the notion of Disparity Discrepancy in [2] and is also similar to the notion of source-guided discrepancy in [3]. The proofs of Lemma 1, Lemma 2, Theorem 2, and Theorem 3 are either straightforward or follow standard techniques in DA (e.g., [2,4,5]).

The empirical results in Table 3 indicate that the improvement of f-DAL is marginal compared to other algorithms (e.g., MDD). Furthermore, the improvement may even come from AExG, rather than f-DAL itself. I would speculate that f-DAL can even be outperformed by other baseline algorithms without AExG.

################

Overall: While I appreciate the f-divergences generalization bounds derived in this paper, I didn’t get any new perspective after reading the paper. Every point of the theoretical results in this paper seems familiar to me.

[1] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems 29, pp. 271–279. Curran Associates, Inc., 2016.

[2] Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. Bridging theory and algorithm for domain adaptation. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 7404–7413, Long Beach, California, USA, 09–15 Jun 2019.

[3] Kuroki, S., Charonenphakdee, N., Bao, H., Honda, J., Sato, I., and Sugiyama, M. Unsupervised domain adaptation based on source-guided discrepancy. In AAAI Conference on Artificial Intelligence (AAAI), 2019.

[4] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79(1-2):151–175, 2010.

[5] Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representations for domain adaptation. In International Conference on Machine Learning, pp. 7523-7532, 2019.



review_title: the theoretical results are not so novel as claimed in the paper 

final_score: rating: 4: Ok but not good enough - rejection
review_content: --------After rebuttal (updated)---------------

Thanks for your detailed response. I would like to apologize for my late feedback.
Since your rebuttal is long without proper organization (5 pages without properly using markdown), I may miss several points. I directly commented my feedback on my original review, `by the marked text`

Several of my concerns have been surely fixed and the understanding of the proposed approach is much more clear. However, it is still difficult to change my score because of the following reasons:
1. *Current paper still requires careful polishing for facilitating reading.*  e.g. The authors claimed several times they will update the paper with pseudo-code, it is still missing after discussion. The paper and the rebuttal are still dense, which make the reader
difficult to understand.

2. *The diverse datasets (such as NLP, digits experiment in the original DANN paper) and additional in-depth empirical analysis (not only accuracy) are indeed necessary.* 
I think you do not need to claim a significantly better performance with SOTA. The detailed empirical analysis is more important. 

I hope my additional comments and feedback can help you improve your paper. 

 ------------------------------------------ 



Summary: 

This paper proposed a hypothesis based f-divergence domain adaptation theory and algorithm. They found previous popular divergences such as H-divergence and MDD can be viewed as the special cases of f-divergence. Finally, they validated their practical benefits on office-31 and office-home dataset.

------------------------------------------------------

Overall review 

Pros:

[1] New DA theory on f-divergence.

[2] The proof is technically sound.

Cons:

[1] The significance of the paper (theoretical and practical) is rather unclear.

[2] The experimental results are weak. 

[3] Some details need better justifications and discussions.

Based on these, I recommend a rejection but encourage a major revision for resubmission.

-----------------------------------------------------------

Detailed explanations

[A] Significance. (theoretical and practical aspects)

Theoretical aspects

[1] Using f-divergence in DA is not new. For instance, [1][2][3][4] have already discussed DA by using $\beta$ (Rényi)-divergence, $\chi$ divergence. I do not understand why such a **unified** f-divergence does not include these divergences. I guess because the theory requires Lipschitz f-divergence, but this is too limited. A theory on general f-divergence (not only Lipschitz $\phi^{\star}$) is highly expected.

`Partially fixed. Missing point: it is possible to prove f-divergence only with f(1)=0 and f is convex? (the original definition of f-divergence)`


[2] The theoretical assumptions are too restrictive for practice. I just list some of them:

f-divergence is Lipschitz, 
$\ell\in[0,1]$ and strong triangle inequality,
deterministic label function setting.

In contrast, previous work [2,3] has proposed strong theoretical guarantees for the cross-entropy loss and **stochastic data generation process** $P(y|x)$ instead of $f(x)$. Given this paper is in the same nature as f-divergence. More general settings are **highly** expected.

`Not fixed. My concern is that the previous approach has proved stochastic settings in some f-divergence. This should be properly discussed and addressed.`


[3] The contribution of the optimization part is unclear. If this part is the theoretical contribution, an optimization convergence bound should be provided.

`Partially fixed. The term *rigorous* is confusing. You did not provide a bound how to define it is rigorous.`


[4] The theoretical assumptions on representation learning are strong. I am not sure how these are realistic in the office-31 and office-home dataset. The conditional distribution is not clearly defined. More discussions on this part are highly expected.

`Not fixed. I am still not sure how these are realistic. The conditional distribution is referred as a labeling function on $x$ f(x)? or $z$ f(z), or probability distribution P(y|x) or P(y|z). This is quite important in the representation learning approach.`


[5] Theorem 3 seems too coarse in deep learning. (Since this paper claims they have a strong theoretical contribution in deep learning. I think this ought to be addressed).

`Fixed`


[6] The KL-divergence is **not tight** if we use dual terms of f-divergence. A Donsker-Varadhan Theorem based theoretical analysis on KL divergence is expected.


  `Partially fixed. The Wasserstein distance makes me more confused about introducing f-divergence. Since JS, TV has such problems, why not Wasserstein distance. It is always better.`


Practical aspects.

[7] From a practical aspect, f-divergence adversarial training is not new.

Based on the f-gans paper, we can practically easily replace JS divergence with any-other f-divergence without any technical difficulty. For example, [5] derives some DA algorithms on general f-divergence. From this perspective, the new empirical insights are rather limited.

`Fixed. Thanks for your additional figures. I understand your new practice.`


[8]  The extra gradient approach presented in the experiments is rather unclear and inspired by existing optimization papers. I think a clear and complete discussion is expected. The current version seems like a plug-in approach.

`Not fixed. Maybe the rebuttal is too dense. I can not get your point`


[9] The whole empirical parts are presented in a dense mode, I suggest some parts can be safely moved to the appendix. 

`Partially fixed. It is better but the dense wrap figure makes it still hard to read.`


[10] It is Ok not to provide the code, but a detailed algorithm description or protocol ought to be provided. This is particularly important when this paper aims at proposing several new ideas.

`Not fixed. This is really important. Author claimed they will write a description but i have not seen even in the appendix`


[B] Empirical results and analysis

[1] From all the results. The empirical gain is too limited. Besides, the compared baselines are limited or not recent SOTA. (The newest results only come from ICML 2019). 

`Not fixed. If you claim your approach is SOTA, at least a statistical test should be added to show it is indeed significant.`


[2] In office-home. I do not know why the std values are not reported in this case. The other 7 tasks are missing in the paper and appendix. The number of baselines is significantly fewer than office-31. 

`Partially fixed. STD still not reported and still fewer baselines. The author claimed previous papers did not report these..But do you think it is a good thing not reporting variance? Particularly in office-31 you reported std, which is really odd.`


[3] The standard digits datasets are not evaluated. Since the Digits dataset is trained from scratch and different from the pre-training approach. Testing on these is expected.

`It seems to be ignored..I think this is essential to validate the theory.`


[4] When testing KL-divergence, the dual term of KL-divergence is not tight. I think a Donsker-Varadhan Theorem based practice should be tested. 

`Fixed.`


[C] Other details

The only analysis of this paper is the numerical accuracy of office-home/31 and toy data. A deeper analysis of why the proposed f-divergence is better than the previous is quite lacking. I can not feel the strong motivation of why preferring f-divergence. 
I suggest to put additional analysis such as T-SNE, the optimal value on **real-data** (such as p_t(z)/p_s(z)), the convergence behavior, the evolution of f-divergence, ablation study.

`Not fixed. The only indicator in this paper is numerical accuracy. I still do not know what makes f-divergence better.`


--------------------------------------------
Suggestions 

I suggest a major revision on the improved theory and empirical analysis (not simply accuracy) on the benefits of f-divergence.


Ref:

[1] Multiple source adaptation and the Rényi divergence. UAI 2009

[2] A new PAC-Bayesian perspective on domain adaptation. ICML 2016

[3] Algorithms and theory for multiple-source adaptation. NeurIPS 2018

[4] Revisiting (\epsilon, \gamma, \tau)-similarity learning for domain adaptation. NeurIPS 2018

[5] Domain adaptation with asymmetrically-relaxed distribution alignment. ICML 2019

review_title: Review #1

final_score: rating: 5: Marginally below acceptance threshold
review_content: Update after reading authors' response.

The authors didn't address my question on the statistical significance of their results compared with baselines.
The authors can address this question by "perform a double-sided student-t test to verify whether the performance difference between your method and the baselines are statistically significant."

I suggest the authors to perform experiments to compare with SOTA baselines in the entire field of domain adaptation, instead of just comparing with a subset of algorithms that are based on adversarial training. After all, you want to see how your work stands in the entire field, rather than limited to a sub-field. 

------------------------

This paper studies unsupervised domain adaptation. The authors derive a generalization bound that utilizes a new measure of discrepancy between distributions based on a variational characterization of f-divergences. They develop an algorithm for domain-adversarial learning for the family of f-divergences. 

Overall, I recommend to reject this paper, due to the following major concerns: 1) writing is not self-contained; 2) important baselines are missing; 3) insufficient justification of the advantage of the proposed method; 4) experimental results are not strong. 

My major concerns of this paper include:
1. The writing needs to be significantly improved, especially the experiment section. For example,  it is very difficult to find out what those baselines in Table 2 refer to. The paper needs to be self-contained. Directing the readers to a third paper for important details such as experimental settings and results is not proper.

2. Some state-of-the-art baselines are not compared with. For example, Contrastive Adaptation Network for Unsupervised Domain Adaptation. Guoliang Kang, Lu Jiang, Yi Yang, Alexander G Hauptmann. CVPR 2019.

3. In the generalization bound in Theorem 2, the authors utilize a new discrepancy to compare the two marginal distributions. The authors didn't articulate the advantage of this new discrepancy over existing ones. Why is it significant so that we need to study it?

4. In table 5, are the results significantly different? It would be nice to perform a double-sided student-t test to verify whether the performance difference between your method and the baselines are statistically significant.

5. Values of hyperparameters are missing, making this paper difficult to reproduce. If space is an issue, the authors can put the hyperparameters in the appendix.

6. The related works section in the appendix needs to significantly improved. There are a lot of works in the space of adversarial domain adaptation and in the broader space of general domain adaptation methods. The authors need to give a more comprehensive review. Again, the review needs to be self-contained in this paper instead of directing the readers to read other review papers. And the review needs to summarize the high-level key ideas, contributions, limitations of those papers instead of focusing on detailed math.

However, this paper does have a few strong points.

1. The theoretical analysis in Section 3 is sound. I read the proofs, which appear to be correct.

2. Section 3 and 4 are well-organized and easy to follow.

Other comments
1. What does ADAA in Table 2 refer to?

2. It's better to use an algorithm box to outline the optimization algorithm in Section 4.1.

3. The paragraph title "Experimental results" in Section 5 is confusing. This paragraph is more like an introduction of this section instead of one presenting results. 
review_title: Experiments and writing need to be significantly improved.

Meta-review: This paper presents a novel theoretical analysis for unsupervised domain adaptation based on f-divergences. The reviews unanimously pointed out the interest and the quality of the theoretical part. However, some limitations in the experiments, presentation and the significance of the result have been raised. The authors provided a rebuttal that addresses some concerns.
However, the reviewers agree that the experimental part still requires some extension to fully support the claim of the paper, as well as some writing improvement.
The paper was evaluated to be not ready for ICLR, thus I recommend rejection.

Decision: Reject