Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The setting in the paper is the classic unsupervised domain adaptation problem, where we are given a labeled sample from a source distribution and an unlabeled sample from a target distribution. The goal is to minimize the risk on the target distribution. Theoretical results led to a breakthrough in practice - the Domain Adversarial Learning architecture (Ganin et al., 2016). 

The paper suggests looking at the paper from a game theory perspective. This is natural, as the objective is to minimize the loss on the source distribution while maximizing the distinction between the distributions. 
The optimal solutions of the game are characterized by the local NE.

Motivated by the results from game theory, the authors suggest replacing Gradient Descent (due to its limitation in this optimization problem) with other optimizers - ODE (ordinary differential equation) solvers.
main_review: Strengths :
The game-theoretical formulation is natural for this important problem.

The experiments seem to support the asymptotic convergence guarantees of the optimizer.

The empirical results look somewhat significant.

The practical significance of this work reveals that using high-order solvers instead of Gradient Descent (GD) in this setting leads to better results.

Weakness:
The technical novelty is only marginally novel and quite straightforward.



summary_of_the_review: My vote is -  marginally above the acceptance threshold due to the aforementioned reasons.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors exhibit a strong link between game theory and domain-adversarial training. They show the optimal point in the latter is a Nash equilibrium of a three players game. From this perspective, the authors show that standard approaches, like gradient descent, cannot work in this setting as the method is known to be divergent in such a case. Instead, they propose to use Runge-Kutta methods (for example) to discretize the ODE, which gives insights for novel algorithms with better convergence guarantees.
main_review: (I am not a specialist in domain adversarial learning. Therefore I am not entirely sure about the novelty and impact of the work in this field).

The paper is very well written. The introduction of both fields of domain adversarial learning (sec 2) and game theory (sec 3) is done properly, with an adequate study of the related work. Moreover, the authors did a good job convincing about the complexity of the three-players game setting.

From this perspective, using known tools from game theory / variationally inequalities / numerical analysis for ODE discretization, the authors show necessary and sufficient conditions for the existence of local Nash equilibrium (prop. 1 and 2), showed that a modified dynamic of the gradient flow (equation (8) ) ensure global convergence of the Euler discretization, and developed an algorithm (RK-2, generalization of extra-gradient, equation (9)).

Finally, they discussed several approaches with solid numerical experiments on various problems.

In terms of novelty, there are no "new" theoretical results, properly speaking. The novelty here is the derivation of the domain adversarial learning into a game, which is not straightforward but simplifies its analysis. Thanks to this new perspective, I believe this could open new doors and research direction in this field. For this reason, and because the paper is particularly well written, I recommend the paper to be accepted.


*** Post rebuttal

I have read the authors' answer and decided to keep my score. 
summary_of_the_review: The authors presented a link between game theory and domain-adversarial training. In similar topic, such as GAN, such link opened new research perspectives and impacted positively the area. I believe this could be also the case here.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper analyzes adversarial domain learning (DAL) from a game-theoretical perspective, where the optimal condition is defined as obtaining the local Nash equilibrium. From this view, the authors show that the standard optimization method in DAL can violate the asymptotic guarantees of the gradient-play dynamics, thus requiring careful tuning and small learning rates. Based on these analyses, this paper proposed to replace the existing optimization method with higher-order ordinary differential equation solvers. Both theoretical and experimental results show that the latter ODE method is more stable and allows for higher learning rates, leading to noticeable improvements in transfer performance and the number of training iterations. 
main_review: 1. The authors claim that "The gradient field of Equation (2) and the game's vector field (see Section 3.2) are equivalent, making
the original interpretation of DAL and our three-player formulation equivalent." Why does field equivalence induce formulation equivalence? What is the exact definition of "equivalence" here?

2. The layout of the paper needs to be improved. For example, Example  2 should be an independent paragraph in the paper but not a "window" embedded in the other section. Some figures and tables have the same issue. 

summary_of_the_review: This paper is well-written and easy to follow. The theoretical contribution is solid, and the experimental studies show the superiority of the proposed method on the domain transfer task. As a consequence, I think the quality of this paper is marginally above the acceptance threshold.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This manuscript considers the adversarial domain adaptation training problem, specifically the gradient reversal method, from the perspective of game theory. The authors show that gradient-based optimizers without an upper bound on the learning rate violate asymptotic convergence guarantees to local NEs. The authors further show that these constraints can be lifted by higher order ODE solvers. In the experimental part, the authors evaluate their method i.e. Runge-Kutta ODE solvers of order 2 and 4 with different general and also game optimized gradient-based optimizers on a MNIST/USPS digits dataset. Furthermore, they show hyperparameter robustness of their method and finally, the method is tested on more complex image and NLP datasets and compared to current SOTA methods. Overall better results are achieved.

main_review: Strengths:

* Well written and understandable
* Theoretical sound in both problem description and solution
* Analyzing domain adaptation learning from a game perspective and analyzing the stability of the optimizer with the gradient reversal layer model is novel 
* Relevant related work is considered and compared
* The experimental part is sufficient and supports the theoretical claims

Weaknesses:
* The theoretical analysis is limited to full batch training while in practice as well in the experiments mini-batches are used. However, the authors mentioned it in the paper.
* As with gradient-based methods, this method could also converge to a non-Nash equilibrium as outlined in [1], however this is likely a rare case and not unique to this proposal

[1] Mazumdar et al. On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games. https://arxiv.org/abs/1901.00838

summary_of_the_review: The proposed method is theoretical founded and the experimental part supports the claims and the analysis is novel. I vote for accept, good paper.
review_title: null

Meta-review: Summary (from reviewer uzT5): This paper analyzes adversarial domain learning (DAL) from a game-theoretical perspective, where the optimal condition is defined as obtaining the local Nash equilibrium. From this view, the authors show that the standard optimization method in DAL can violate the asymptotic guarantees of the gradient-play dynamics, thus requiring careful tuning and small learning rates. Based on these analyses, this paper proposed to replace the existing optimization method with higher-order ordinary differential equation solvers. Both theoretical and experimental results show that the latter ODE method is more stable and allows for higher learning rates, leading to noticeable improvements in transfer performance and the number of training iterations.

All reviewers appreciated the contributions of this paper and recommended acceptance. While the methods themselves are not novel, the game perspective applied to this problem appears to be and the use of higher-order solves yield interesting theoretical and empirical improvements.

== Additional comments ==

1) For the comparison vs. game optimization algorithms (Figure 3), it would be nice to normalize the x-axis so that one "epoch" yields comparable computational cost among the different methods (as RK4 and RK2 is much more expensive than EG or GD per mini-batch). Given that EG had such bad performance there, it would not change the conclusions; but the current scaling is still quite misleading. Same comments for Figure 2.

2) Note that modern approaches for stochastic extragradient recommend to use different step-sizes for the extrapolation step and the update step (see e.g. Hsieh et al. NeurIPS 2020 "Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scaling") I suspect that much bigger step-sizes could be used in this case while maintaining convergence, and this version should be added to Figure 3.

3) In "Related Work | Two-Player Zero-Sum Games" -> note that Gidel et al. 2019a provided all their convergence theory and methods for stochastic variational inequalities and thus it also applies to three-player games, unlike seems to be implied by this paragraph. In particular, all the algorithms they investigated (Extra-Adam amongst others) could also be applied to DAL. While I can see that the specifics of the objective in DAL might be different than for GAN optimization, it would be worthwhile to acknowledge these alternative approaches more clearly, and I encourage the DAL community to investigate their performance more exhaustively for DAL than what was done in this paper.
Decision: Accept (Poster)