Discrepancy-Optimal Meta-Learning for Domain Generalization

Abstract: This work attempts to tackle the problem of domain generalization (DG) via learning to reduce domain shift with an episodic training procedure. In particular, we measure the domain shift with $\mathcal{Y}$-discrepancy and learn to optimize $\mathcal{Y}$-discrepancy between the unseen target domain and source domains only using source-domain samples. Theoretically, we give a PAC-style generalization bound for discrepancy-optimal meta-learning and further make comparisons with other DG bounds including ERM and domain-invariant learning. The theoretical analyses show that there is a tradeoff between classification performance and computational complexity for discrepancy-optimal meta-learning. The theoretical results also shed light on a bilevel optimization algorithm for DG. Empirically, we evaluate the algorithm with DomainBed and achieves state-of-the-art results on two DG benchmarks.

Reviews and author responses:

The paper investigates a meta-learning approach to domain generalisation. Proposed theoretical contributions include a generalisation bound for domain generalisation that depends on the $\mathcal{Y}$-discrepancy between the source domains available during training, the training error on these source domains, and some measures of model complexity. An algorithm inspired by this theory is proposed, and the experimental results show that is can achieve good performance. In general, I think the paper is reasonably well-written and quite easy to follow, from a clarity point of view. The experimental results are also somewhat promising. The performance on PACS is pretty good, and the DomainNet performance is very good. Some additional experiments show that the proposed algorithm does have an influence over the $\mathcal{Y}$-discrepancy. However, I do have some fairly significant concerns about both the theoretical contributions and the motivation for the algorithm, resulting in large doubts over why the method actually achieves good performance on DomainNet.

Theory:
* In Lemma 1: where does the $\mathcal{B}\delta$ term come from when bounding (II) in the proof for Lemma 1? Should this not be a term something like $\mathcal{O}(\sqrt{\frac{log(\delta)}{N}})$, arising from Hoeffding's inequality, or simiar?
* In Lemma 1: (III) cannot be bounded in this way because the elements in the meta-sample are not iid, according to the definition at the end of Section 3.1.
* In Lemma 2: the meta-sample is not iid, so (II) cannot be bounded in this way.
* Because of the issues with Lemmas 1 and 2, it seems like Theorem 1 is not true. I suspect the fix will cause the proposed bound to scale similarly to ERM.

Algorithm:
* I do not understand the rationale for mixing meta-learning with domain generalisation. Meta-learning, as formalised in Maurer (2005), involves building a different model for each task (or domain), and the goal of the meta-learning is to find the smallest hypothesis class that achieves good performance on novel tasks sampled from the same evironment. In contrast, the goal of domain generalisation is to build a *single* model that will generalise well to other domains sampled from the environment.
* Why is bi-level optimisation needed? The bound (if we ignore issues with the proof) suggests good performance can be obtained on novel domains if one minimises the error averaged accross source domains, plus another domain discrepancy term. Bi-level optimisation problems arise when one must optimise an objective subject to a constraint, where the constraint is itself defined in terms of another optimisation problem. The bound does not suggest such a structure. One way I could see bi-level optimisation being required is to compute the $disc_\mathcal{Y}$ terms, because they have a supremum, but this is not what the proposed method does.

Other questions and comments:
* Why have a probabilistic number of source domains?
* Note that Jaakkola is not an author of the Maurer (2005) paper—this seems to be a problem with the bibtex entry in Google Scholar.
* Why does the meta-sample contain target domain data? In DG one does not have access to target domain data. Or are the source domains arbitrarily split into two different sets labelled S and T?
* Are all pairs in the meta-sample from the same environment, or is each pair from a different environment?
* Minor comment: the notation for $disc_{\mathcal{Y}}$ (Eq 4) is a bit strange. It looks like $A^f(\hat{\mathcal{S}})$ is a function being applied to $\mathbb{S}$ and $\mathbb{T}$, but it seems more natural (to me) to consider $\mathbb{S}$ and $\mathbb{T}$ as a second and third parameter for $disc_{\mathcal{Y}}$: i.e., $disc_{\mathcal{Y}}(A^f(\hat{\mathcal{S}}), \mathbb{S}, \mathbb{T})$.
* How is the Mann-Whitney test performed here? This test is designed to compare two methods at a time, and it seems unlikely that 4 measures (e.g., in the case of PACS) is enough to get statistically significant results. Contains some new ideas for mixing meta-learning and domain generalisation, but there are issues with motivation and theoretical correctness.

Domain generalization (DG) deals with a setting where one or several different but related domains are given, and the goal is to learn a model that can generalize to an unseen test domain. The existing DG methods cover "domain-invariant representation learning" and "meta-learning". This paper attempts to train a discrepancy optimal meta-learner, gaining training experience and minimizing the domain discrepancy between any unseen target domain and the source domains. The proposed method has strong theoretical findings and shows the efficiency of the discrepancy-optimal meat-learner, in terms of PAC-generalization bound, compared to the ERM and domain-invariant learning. From the computational point, the authors provide results on two DG benchmarks using a bilevel optimization algorithm. 
- The paper is well-written and the approach is mostly well presented. The proposed method has strong theoretical findings and shows the efficiency of the discrepancy-optimal meat-learner compared to the ERM and domain-invariant learning. 

- To let the theoretical findings in Section 4.1 more readable, it would be better to discuss the results and highlight the complexity order term. Additionally, it is worth to discuss scenarios of the order of $n, M$, the number of training data, and source domains. Intuitively, I think that the fact $M \ll n$ is more likely to happen in the practice.

#### Typo:
In Lemmas 1 and 2, the constant $B$ is not explicit. 
 The paper is well-written and the approach is mostly well presented. The proposed method has strong theoretical findings and shows the efficiency of the discrepancy-optimal meat-learner compared to the ERM and domain-invariant learning.

This paper studies the problem of domain generalization with a meta-learning approach. It proposes domain generalization bounds based on a Y-discrepancy measure already used in the literature. A meta-learning algorithm is proposed under the form of a bi-level optimization strategy. An experimental evaluation on two datasets is proposed. Strength
The paper proposes a novel domain generalization bound using a discrepancy based approach. The bound is used to derive an algorithm. 
An interesting point is that the algorithm is theoretically founded.
As far as I know, the domain generalization problem is not that studied yet and this paper provides an interesting contribution to bridge this gap.


Weaknesses
As far as I understand the paper, the bounds require to see pairs with respect to all the combinations of the source domains. This is certainly useful to obtain better bounds, but this seems then costly and actually I am not completely convinced that this very useful in practice: we rather would like to able to generalize with fewer pairs. This point is not really discussed by authors.
The informativeness of the bound could be better discussed, we do not know large are the constants and it would be interesting if the authors could give settings where the bounds are not loose.
The convergence of the algorithm is not studied.
I do not really understand the term "discrepancy-optimal" because, unless I am mistaken, no optimality has been proven yet
The experimental evaluation is a bit limited: 2 datasets.
The writing requires a more rigorous attention, the domain generalization problem is for example not clearly presented.


Equation (2), I wonder if it would not be better to reverse the order of the elements in the expectation: you draw a number of domains first, then the domains and then the samples. The paper presents a novel domain generalization bound using a discrepancy measure, for a problem rather understudied. An algorithm optimizing this bound is proposed.
The theoretical result is certainly interesting but seems loose and a setting that appears to be heavy (the need to consider all combinations of source domains). It is not clear if the bound is informative. The experimental evaluation is a bit limited.

This paper incorporated Y-discrepancy and meta-learning for domain generalization. Concretely, they derived a novel theoretical upper bound in domain generalization via adopting Y-discrepancy and upper bound the Y-discrepancy through a bi-level optimization. The empirical results clearly suggest a significant improvement. 
 ---------------------------------------------------------
Review Summary:

I spend roughly 6 hours carefully reading the paper. As far as I understand, this paper has the following benefits:

- As far as I know, this is the first paper that adopts the Y-discrepancy in domain generalization, the theoretical results are relatively novel;
- They further upper bound Y-discrepancy as a bi-level objective, which is novel and insightful. 
- The empirical results are significantly better than previous meta-baselines.
- The writing is mathematically rigorous.

Simultaneously, I have found a couple of important concerns in the paper:

- The concerns in the Theoretical results:  part of the proof is seemingly incorrect, unclear presentation, and incorrect discussion with previous theoretical work.
- The Concerns in the algorithmic design. This reviewer has multiple important concerns and questions in the derived algorithm.
Other concerns are in the clarity part.

Based on the importance of technique issues. I would like to recommend a weak rejection.

------------------------------
Detailed Reviews:

- About the Theoretical results

1. There always exist several unclear math definitions that make it hard to follow the paper. For instance, in Eq (5), what is $B$ and which factor will influence it? On page 3, what is the clear definition of meta-target samples $\hat{T}$ ?   And I am also not sure about the benefits or insights of introducing $N$ and $P(N)$ in the theoretical definition.

2. On page 13, The proof of proposition (1), why do you omit the order of $N$? I do think keeping the original form will be much better. Besides, discussion on the metric Entropy or covering number (def 3) w.r.t. Hypothesis complexity will be beneficial for the non-experts in learning theory.

3. In the proof of Lemma 1 (page 15), Upper bound of term 2, I am not clear how $B\delta$ is derived. 

4. **(Important)** In the proof of Lemma 2 (page 16), The term $E_{\hat{T},T}|\epsilon_{T}(h) - \hat{\epsilon}_{T}(h)|$ is vacuously upper bound or incorrectly proved. First, I am not clear how it is related to $B\delta$ (since the author did not clearly define B). Moreover, intuitively, this term aims to bound the gap between **future unseen** empirical samples and its population part. Since in the domain generalization we never see the future test samples, the meta-learning-based domain generalization essentially adopts the source (or observed) dataset as a surrogate of the future unseen test samples. However, the limitation is clear, if the source/environment number is limited, such a gap will surely be large. Although I have not tried to prove some formal results, I speculate this term is related to $\sqrt(1/N_m)$ rather than a vacuous constant $B$. 

  Based on this argument, the convergence behavior is essentially identical with ERM with \sqrt(1/N_m) + \sqrt(1/n), where the convergence can not be improved in the worst-case. But it is not surprising due to the problem set-up in DG and Domain Bed indeed empirically validates this. Therefore, the whole analysis on Page 5 (Tab.1 and Sec 4.2) is incorrect. The convergence rate has not improved in theory. From a heuristic perspective, when the source environment's numbers are limited, any kind of episodic training could not improve the understanding (convergence rate) of the ground-truth generating environment. But this does not indicate the impossibility in the empirical improvement since it is a worst-case bound.  


- Concerns on the proposed algorithm:

5. As far as I understand, Corollary 1 is correct. The result is logic. My main concern is the definition of Def 2, where the bi-level objective is presented in a rather confused manner. For instance,  in Eq(8) $\phi^{\star}$ is essentially an implicit function of \phi, which should be clarified in the text. In Eq(10), the initialization operation is quite weird. (Sorry I have not seen such a math notation before.) I would like to suggest the author carefully revise Eq (8-10). 

6. Besides, \alpha in Eq (8-10) is not defined. How do we decide this? By the uniform coefficient?

7. **(Important)** I could not understand why we could split the hypothesis into the representation \phi and classifier h, which seems unnecessarily in the Y-discrepancy. Why not regarding them jointly and updating through a parameter transfer manner (like MAML). One reason I guess lies in Alg 3, the estimation of Y-discrepancy, which naturally adopts the classifier to conduct the min-max training. But I still think separately introducing a critic function is sufficient. 

8. The updating $\theta$ in Algo 3 seems not to differentiate \thelta through the whole gradient path? (e.g. Line 4-5). Since the algorithm adopted multiple levels of inner- and outer- loop. It is generally quite difficult to understand which part is differentiating or not. 

9. **(Important)** Another question is the motivation of adopting the meta-test procedure (also 2). In the derived theoretical results or the bi-level objective, it seems Algorithm 1 is sufficient. A bi-level optimization that efficiently solves the Y-discrepancy upper bound. I feel like it is a bit hard to understand the motivation or theoretical support, which sounds like a simple plug-in idea from meta-learning. 

- Other suggestions for improving the paper.

10.  I would like to suggest the author revise the paper by providing clear intuitions of the proposed approach. For instance, the Y-discrepancy exactly encourages the same level of prediction risk of different domains. (But the author adopts bi-level to solve the problem)  From this perspective, it is better to compare the robust optimization-based approach. 

11. Besides, the concept of environment is essentially not adopted in the practice. In fact, the proposed bi-level objective depends on the convex hull of observed and unseen test domains, which is still viewed as a similarity-based approach. I do think the notion of a generation environment does not properly reflect the core idea in the paper, where the author can revise it for a better illustration. 

12. The paper can be written in a more accessible way. E.g, Fig 1 and Fig 2 can be improved to better reflect your motivation. (The current figures did not help too much for a better understanding.)





 (This part is adjusted from the Main review)

As far as I understand, this paper has the following benefits:

- As far as I know, this is the first paper that adopts the Y-discrepancy in domain generalization, the theoretical results are relatively novel;
- They further upper bound Y-discrepancy as a bi-level objective, which is novel and insightful. 
- The empirical results are significantly better than previous meta-baselines. The writing is mathematically rigorous.

Simultaneously, I have found a couple of important concerns in the paper:

- The concerns in the Theoretical results:  part of the proof is seemingly incorrect, unclear presentation, and incorrect discussion with previous theoretical work.
- The Concerns in the algorithmic design. This reviewer has multiple important concerns and questions about the derived algorithm.
- Other concerns in the clarity part.

Based on the importance of technique issues. I would like to recommend a weak rejection.


Meta-review: The reviewers had a number of concerns which remain since the authors did not provide any response nor they have updated the paper.  Hopefully, once the paper is improved in terms of clarification (significance and correctness of the theoretical results and the technical approach approach), it will be ready for publication in one of the ML venues.

Decision: Reject