Domain Generalization without Excess Empirical Risk
Authors: Ozan Sener
Vladlen Koltun
Venue: NeurIPS 2022 Accept (2022)
Keywords: Domain Generalization
Penalty-Based Methods
Optimization
Rate-Distortion

Abstract: Given data from diverse sets of distinct distributions, domain generalization aims to learn models that generalize to unseen distributions. A common approach is designing a data-driven surrogate penalty to capture generalization and minimize the empirical risk jointly with the penalty. We argue that a significant failure mode of this recipe is an excess risk due to an erroneous penalty or hardness in joint optimization. We present an approach that eliminates this problem. Instead of jointly minimizing empirical risk with the penalty, we minimize the penalty under the constraint of optimality of the empirical risk. This change guarantees that the domain generalization penalty cannot impair optimization of the empirical risk, \ie, in-distribution performance. To solve the proposed optimization problem, we demonstrate an exciting connection to rate-distortion theory and utilize its tools to design an efficient method. Our approach can be applied to any penalty-based domain generalization method, and we demonstrate its effectiveness by applying it to three examplar methods from the literature, showing significant improvements.

Domain Generalization without Excess Empirical Risk
Ozan Sener Apple
Vladlen Koltun Apple
Abstract
Given data from diverse sets of distinct distributions, domain generalization aims to learn models that generalize to unseen distributions. A common approach is designing data-driven surrogate penalty capture generalization and minimize the empirical risk jointly with the penalty. We argue that a significant failure mode of this recipe is excess risk due to erroneous penalty hardness in joint optimization. We present an approach that eliminates this problem. Instead of jointly minimizing empirical risk with the penalty, we minimize the penalty under the constraint of optimality of the empirical risk. This change guarantees that the domain generalization penalty cannot impair optimization of the empirical risk, i.c.. in-distribution performance. To solve the proposed optimization problem, we demonstrate an exciting connection to rate-distortion theory and utilize its tools to design an efficient method. Our approach can be applied to any penalty-based domain generalization method, and we demonstrate its effectiveness by applying it to three examplar methods from the literature, showing significant improvements.
1 Introduction
Empirical risk minimization (ERM) is the workhorse machine learning. In this setting, a learner collects set of data, assumed to be independent and identically sampled from particular distribution of interest. An empirical risk over these samples later minimized, hoping that the resulting model will generalize future unseen data from the same distribution. Although ERM ubiquitous, is brittle to minor shifts in the distribution, and the resulting models suffer a significant performance drop when evaluated beyond their training distributions. Examples of such distribution shifts include different hospitals for medical imaging [49, camera types for perception models (8) and time periods for natural language models (0). The problem of learning models which generalize under such distribution shifts is studied in the literature under the rubric of domain generalization.
Given collection of domains (data distributions), domain generalization aims find models that generalize to unseen domains. Although the domain generalization literature rich, many model-free methods follow common recipe. A surrogate penalty that encapsulates generalization is designed and joint optimization problem of minimizing the empirical risk with the surrogate penalty is solved. Typical such penalties invariance optimal predictor [5], and uniformity of risk 355 Although these approaches are technically sound,
adversarial domain classifier [17 42]. invariance of an
their effectiveness has been challenged by recent empirical benchmarks 23 18
Joint optimization of the empirical risk and penalty is a sound method since the error on the unseen distribution can be bounded as the combination of error the training distributions (empirical risk) and the gap between performance on training and unseen distributions (generalization gap). Although domain generalization literature focuses mainly controlling the generalization gap, hypothesize that this joint optimization can increase empirical risk, effectively hurting performance on both training and unseen distributions. In other words, the lacking empirical performance of existing algorithms might be because they fail to optimize the empirical risk (i.e., in-distribution performance) even though they learn control generalize gap. Consider the extensive evaluation conducted on the WILDS benchmark 23 ERM yields higher out-of-distribution performance than dedicated domain
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

generalization methods on most problems. In contrast, when we look at the generalization gap (i.c., the difference between in-distribution and out-of-distribution performance), domain generalization methods have smaller gap than ERM on most problems. This supports our hypothesis that existing algorithms might be creating too much excess empirical risk. Thus the motivating question of our work: Can we perform joint optimization without causing any excess empirical risk?
Solving joint optimization without sacrificing empirical risk is not generally possible unless surrogate penalty term and empirical risk align perfectly. Instead, we formulate an optimization problem with the explicit constraint of no excess empirieal risk. In other words, we propose to minimize the surrogate penalty only if it is not hurting the empirical risk. Formally, instead of joint minimization of empirical risk and surrogate penalty, we minimize the surrogate penalty under the constraint of optimality of the empirical risk.
To design an algorithm for the proposed optimization problem, we start with searching for formal definition of the optimality of the empirical risk. Since the applications of interest are non-convex, this can only be local optimality. Hence, the constraint we consider is convergence to a stationary point of the empirical risk. We show that biased gradient descent converges to a stationary point as long as the bias is bounded and decreases through optimization. Using this result, we design iterative algorithm that seeks the update minimizing the surrogate penalty under the constraint of bounded deviation from the empirical gradient. This construction closely related to the rate- dittooistootion problem from information theory [13], and we draw on this vast literature to design an algorithm. Specifically, we propose Blahut- Arimoto-style method
to optimize the proposed objective.
Our method only considers the optimization aspect of the domain generalization problem and can be applied to any benalty-based model with minor modifications. We perform our primary empirical evaluation on the CORAL model due to its simplicity [42). We evaluate on the large-scale domain generalization problems in the WILDS benchmark of real distribution shifts 23 Our evaluation using the image, formal language, natural language, and graph modalities suggests that our method significantly improves performance. To further validate the generality of our method, we apply it to the FISH model [39 and VRex (25] showing similar gains on WILDS [3 and DomainBed [18).
2 Method
2.1
Formulation and the Preliminaries
In this section, we introduce our notation and present a unified treatment of existing penalty based domain generalization methods.
We are interested in a learning problem in the space of X x V, where X' is the space of inputs and y is the space of labels. In domain generalization, collection of different domains म = {..,...., TM as well as an independent and identical (iid.) dataset for each domain are given as {0), y3} ~ po, for domain j with N, samples. Consider the union of these datasets as the training set of domain generalization in the form of tuples {x, e, y such that x is the input, e is the domain id, and is the label. We further consider parametric function family h(:;0): X + y and loss function -(:-)): y x y + R.. Throughout our discussion, we denote random variables with capital letters (e.g. X, E,Y) Y) and their realizations with lower-case letters (e.g. 2,, e,
We are specifically interested in penalty based domain generalization methods where empirical risk and domain generalization penalty are jointly minimized. Each model defines a penalty following different set of assumptions. For example, penalty is defined in terms of second-order statistics for CORAL 42] and in terms of sub-optimality of the domain-specific classifier for IRM (5) Since we are interested in the optimization of these models, not their construction, we treat Penalty as a generic entity and consider the optimization problem defined by these methods. These methods aim to solve the following joint optimization problem:
min 0
C(0: (0' ( + Penalty((; eet.ety'
where L(0; (xr, e", y²) )')) is the empirical risk defined as L()) M Σ No E 1(b()):)) 0), wz). Although, the penalty term is typically weighted, weighting can be considered as part of its definition.
2

---

2.2 Optimizing Domain Generalization Objectives
The optimization problem defined in | is a sound proxy in order to optimize the risk for the unseen domains. Consider an unseen distribution T., its corresponding population risk can be bounded using the following straightforward decomposition;
Eg.yrer ((,((:;0, <
Eu.y (),(2;;),,)) + Ex.grvr. [1(h(2;)) 0), v)1 - Ex (),
Risk on Unseen Domains
Risk un Seen Domains
Generalization Error
Following this decomposition, the optimization problem in ( is sensible since the empiri- cal risk (C(0; {xe et, yy))) approximates the risk on seen domains and a the penalty term OPenalty(() (at, ,e, y'))) approximates the generalization emror. However, this formulation has a failure point. Consider well-designed penalty term which successfully approximates the general- ization gap. Clearly the joint optimization will decrease the generalization error. However, it can effectively increase the risk on seen domains due to the difficulties in optimization process. Moreover, the reduction in generalization error is lower than the unintended excess risk on seen domains, the final performance on unseen domains can be worse than not doing any domain generalization. We hypothesize that this behavior might explain the recently demonstrated behavior from 23 18).
As motivating empirical finding, consider the benchmarking results from [23 which are enclosed in Appendix D Take CORAL 42 as a representative method When we look at out-of-distribution performance, ERM outperforms CORAL on 7 out of 9 problems. However, when we look at the generalization gap (i.e., the difference between in-distribution and out-of out-of-distribution performance), CORAL has lower gap than ERM on 7 out of 9 problems. Although the models learned with CORAL have lower generalization error, the loss of in-distribution performance is so significant that the resulting models do not perform well when tested ofito--distribution..
In order to partially address this issue, we propose constraint of no excess risk. Instead of jointly minimizing the empirical risk and the penalty, we minimize the penalty under the constraint of the optimality of the empirical risk. More formally, we consider the following optimization problem:
min Penalty(0;; {a e. v 8
(3)
st. 0 E arg min C())
Remarks on the reformulation: i) This reformulation eases the aforementioned issue as it guarantees no excess risk over the seen domains. Since the optimality of the empirical risk is directly a constraint, this formulation would not harm in-distribution performance. ii) This reformulation is not sensible performance seen domains and unseen domains are inherently contradictory. For example, if there is strong nuisance information in the seen domains, any model not using this nuisance information will fail to optimize the empirical risk, effectively making excess risk requirement for generalization. However, for the benchmarks we are interested in, which involve supervised learning from multiple datasets, this is not the case universal model solving all datasets (seen and unseen) exists. In other words, we address only a subset of domain generalization problems where seen and unseen domains do not compete. iii) Since the models of solutions to empirical risk minimization is quite large, likely including many domain-invariant and semsin-sensitive solutions. Hence, this problem is still non-trivial from the numerical optimization perspective. iv) Finally, one can mistakenly think that our reformulation and the original joint optimization would find the same solution due the Lagrangian function. However, this is incorrect as the Lagrangian function requires optimization of the Lagrangian multiplier, which is intractable for large-scale deep learning and typically replaced with fixed pyrae--asameters. Moreover, we propose to directly solve 3) without resorting to Lagrangian function.
are significantly overparameterized, the space
In the rest of this section, first discuss in Section 2.3 how to convert the constraint in (3) into practical form for first-order optimization. Then in Section [2.4 we discuss how (3) is closely related
to rate-distortion theory, demonstrating unexpected connection. Finally in Section 2.5 we utilize
tools from rate-distortion theory to numerically solve (3..
This choice is not binding: similar conclusions hold for other methods.
3

---

2.3 Stationarity of the ERM as a Constraint
The overarching objective of 0 E arg ming ((0) is rather too ambitious. Since the family of loss functions we are interested in is non- the best we can hope is convergence to a stationary point. Hence, we convert ( into
min Penalty(0; "e,ee'
(4)
[V,[(0)] Sc
While solving the problem in (4, we constrain ourselves to iterative methods to be compatible with stochastic gradient descent (SGD), which is commonly used to train deep neural networks. Hence, we would like to have an iterative procedure 0++1 0e r/G which solves (4) in convergence. Before we proceed, we note that when G is the stochastic gradient (i.e., E[G*] = (oL()),, convergence to stationary point of the empirical risk can be shown under mild conditions. However, we are also interested in minimizing the penalty. Hence, we need to allow updates that are different than the gradients while still supporting eventual convergence.
Convergence of SGD with biased gradients has previously been analyzed by Sener & Koltun 37.. Hu al. [22), and Ajalloeian & Stich [I]. The common theme is the necessity for the bias to decrease over time. This requirement is explicit in [37] and implicit in [22] and [.. In the implicit case, bias is assumed be linearly bounded by the gradient norm, later shown decrease.
We adapt the analysis of SGD with bias from Sener & Koltun (37] and show that as long as the norm of the bias is bounded and this bound decreases during training faster than the rate 1/V7, it convergences to a stationary point of a smooth and non-convex risk function. We state the convergence result in the following proposition and defer the proof to Appendix A
Proposition 2.1. Apply stochastic updates 0*+1 0* n/C² for ' steps. Assume i) R()) is a non- convex, L-Lipschitz, and j-ssmoth function bounded by A: ii) Updates are bounded in expectation, < V; iii) Bias of the updates is decreasing, ||E[G") V'( 1|2 S P/Vi. Then,
E|||G* [
1 T
E VI(
S ( LLD
AuV + LD)
I
V
T
In this proposition, we assume that the risk function is non-convex, L-Lipschitz, and yu-smooth 2 and that the stochastic updates are bounded. These are standard assumptions and shown to be valid for common deep learning architectural choices [19]. Finally, the third assumption is on the bias decreasing with the rate of 1/vi, and we explicitly handle it in our solver, which we discuss below.
With Proposition 2.1 we operationalize the definition of a satisfactory iterative update. As long we can bound the difference between the updates and the gradient of the empirical risk, we can guarantee convergence to a stationary point. Hence, our iterative approach to domain generalization starts with an initialization 8 and applies the iteration gt+1 Hr n with G being the solution of the following optimization problem:
min P(G))
Eg [Penalty(O' + G (f M
(5)
D
st. Eo [IIG* VoC(O)]l]] < Vi
The constraint of our final formulation in (5) guarantees the assumption (iii) from Proposition 2.1 Hence, applying the updates G", which are the solution of (5). would satisfactorily minimize the empirical risk. In the meantime, we seek the updates minimizing the penalty among these satisfactory ones. Since our approach utilizes satisfactory updates for empirical risk instead of the first-order optimal ones, following Herbert Simon's 's characterization of satisficing 44), we call this approach Satisficing Domain Generalization (SDG). Finally, the minimization is over the distribution of the updates instead of the update itself. Although this unconventional, is intentional and crucial for making the problem compatible with rate-distortion theory, which we utilize heavily in our solver.
L- L--iischh,,, mmoshhhhh: (R(0-) RR(02) I 0I 0 |VR VR(O?)) yu|0² 0² vo 0².

---

Eg: |enalty + G"; Gi; (at (wi,e' .et v/1
ERM(No Distortion,
Feasible Set of Updates
Our Solution
Penalty Minimizing Updates seg Panalty(f G% (..6
Satisficing Updates
D'
Eg
|IG - VoL(0)]la
Figure 1: The
function R(D) a non-increasing and convex function of D that describes the of feasible updates. Our analysis of SGD with bias implies that the E|IIG* - (0*))l2 < D' describes the set of updates that eventually minimize the empirical risk. The penalty minimizing updates lie well beyond the region that solves ERM. Our formula- tion chooses the update that minimize the penalty while eventually solving the ERM problem.
2.4 Understanding Satisficing Domain Generalization
In this section, we provide observations on the proposed formulation 5 by utilizing tools from rate-distortion theory. Consider the function R(D) as the solution of (5) for a given constant factor D. The problem (5) is simply minimizing the penalty with the constraint of the limited distortion in updates. This form is strikingly similar the distortto function, where the (compression) rate optimized with the constraint of acceptable degradation in decoding. If the penalty were the mutual information between the true gradient and G it would have been exactly equivalent. Hence, we can further utilize tools from rate-distortion theory to better understand (5). We now state few simple facts derived from rate-distortion theory. (Proofs follow their rate-distortion counterparts directly and are provided in Appendix (
Extreme cases: When D = D/vI= 0, B[G* VC(('). Hence, at the end of training, we perform ERM updates. When the distortion is unbounded (D' = oo), updates directly minimize the penalty, Moreover, our proposed formulation smoothly goes from updates minimizing penalty ERM updates during training.
Improvement guarantee: R(D) is non-increasing and convex function of D. Hence, R(0) 2 R(D'). other words, SDG decreases the penalty when compared with ERM.
Geometric picture: The satisficing requirement implies a certain structure over the plane where the r-axis is the distortion and the y-axis is the penalty. We visualize this geometry and the interpretation of our method in Figure I
2.5 Solving Satisficing Domain Generalization
We present numerical algorithm to solve (5). We first give an iterative method similar to Blahut- Arimoto (BA) 411 with intractable complexity. We later make simplifying assumptions to design tractable method.
We propose to regularize the objective with the mutual information between gradients (G') and the domain-ids (F)) This regularization is sensible would like gradients carry little information about the domain-ids. The final numerical optimization problem we tackle is
min Eg [Penalty(0* + G fa+ e r (IG; (G))
(6)
st. Eo ||| VoL(A)]l2 < Vt
5

---

we keep track of the average of The information regularization is not only desired but also crucial as it enables us to apply technical derivation similar to Blahut-Arimoto
To solve (6). We only consider the discrete and finite probability mass functions where Go E {G k hence, we only need to solve for the point masses Pc; p(G² -.)- We first define auxiliary variables Patje - p(G = C = e), which are conditioned on domain-ids. By this decomposition, we can solve for each domain conditional probability Patle separately and later
marginalize them via Pa; 1/M E. Pojle- We further replace Eg [|] V (,((0) D/vT with stronger constraint of uniform bounds over domains as EGI/E=ellle ||| G² Vo - P/V and apply a similar derivation as Blahut [1]] and Arimoto [4] We defer the complete derivation to the Appendix C.1 and state the update steps directly.
Consider the notation d(G, e) | (0) and Pen(G) = Penalty(0' + Gifaa', 3 We initialize the conditional probabilities uniformly as payle 1/K and apply the iterations:
Domain Specific Gradients: Paaye- = pla exp - (pen(G))) + Sd( e))
PoLie
1++1
Normalization: Poglc Σx ot
1+1
Marginalization: Pc.
1+1 1/21 Ee
Pajle
1+1
Here Be is a parameter that inversely depends on Dt. These iterations converge to a solution where all the conditional distributions are equal.
Applying these updates to the space of gradients is intractable since the space of G is high- dimensional. We propose two simplifications to make the application of BA tractable: i) we solve for each parameter independently, and ii) we only estimate the sign and copy the magnitude from the ERM. Specifically, we solve the BA problem using the discrete set (G'), E {(G = (VL),. (Gt), -VV,), for (G'), p. which is the update for the pz parameter. We use uniform initializa- tion: p" p(( = p
= 0.5. We present a more detailed discussion in Appendix & with complete derivation and give pseudocode in Algorithm I After the BA iterations are completed, we sample the update G ~ p(G²) and apply it to the model.
To understand the proposed simplifications, we consider their implications for the rest of the pipeline. We feed the resulting estimated gradients to first-order numerical optimizers. Moreover, we utilize the first-order approximation of the penalty function in our algorithms in Appendix & Hence, the rest of the pipeline is first order, not utilizing higher-order relationships between coordinates, justifying the first simplification. The second simplification is similar to SignSGD (10] like methods, where only the sign of the gradient is used without its magnitude for efficient communication between worker nodes in multi-node optimization. Sign of the gradient suffices for convergence in non-convex problems both theoretically and empirically.
3 Experimental Results
3.1 Implementation details.
We implement the proposed algorithm in PyTorch. Following the analysis in Proposition 2.1
D' = D//%/ Moreover, 3 is inversely related to D. We choose 8r (8/vTTy/t for epoch t from t = 1 tot = T. We treat 3o as hyperparameter and search in 8 E 00., 1.0] together with all other hyperparameters. Instead of treating (weight of the information regularization) as hyper-parameter we present simple heuristic set it. Specifically, since the relevant term e))] (InvP(G)) + Bd(GG e)) throughout training and set it as y at each instant. In all of our experiments, we used 25 BA iterations. Since using all domains at each batch is not feasible, we first sample Bn domains, then sample B examples per domain (named as group sampler in WILDS 23). We search all hyperparameters with random sampling with the constraint that all methods have the same budget for the hyperparameter search. Specifically, we use 20 random hyperparameter choices. We also
in the iterative update is exp - (InvP(G)) + Bd((C)

---

Table 1:
test performance in the WILDS benchmarks 23
We report average
over seeds for OGB-MoIPCBA, FMoW, Amazon, and py 150, 5 folds for PavertyMap, and 10 seeds for Camelyon17. We report standard deviations uncertainty.
IWILDCAM
CAMELYON17
OGRMOL.PCBA
FMoW
AMAZON
PY150
PAVERTYMAP
MACRO-F1
Avo ACC
AP
WORSTR ACC
10IE ACC
METHODČLASS
WSTPEARSONE
CORAL
32.8
59.5 7.7
0.5
31.0 0.4
52.9 0.8
65.9 0.1
0.4120006
+AND
8.2
70.8 +: 6.7
10.3 0.1
28.0 1.0
51.5 0.7
66.7 + 0.6
0.44=00.03
+SAND
33.9 1.4
64.7
23.
33.3
2 0.2
53.1
66.9 0.2
0.44::0.08
+SDG
34.6 1.2
75.2
4.4
25.8 0.1
33.9 0.9
53.7 - 0.6
68.3 0.1
0.49:00.05
FISH
1.80
74.7 + 7.1
0.3
34.6 1.2
0.8
0.1
0.35+40.03
+SDG
32.9 +: 0.2
76.2 8.4
26.1 + 0.5
34.6 + 0.3
53.5 +: 0.6
66.5 + 0.2
0.46++0.03
ERM
31.0
70.3 6.4
27.2 :: 0.3
32.8 0.5
53.8 + 0.8
67.9 -: 0.1
0.45=00.66
Table 2: In-distribution test performance in the WILDS benchmarks 23
We report average
over seeds for OGB-MoIPCBA, FMoW, Amazon, and py 5 folds for Poverty Map, and 10 seeds for Camelyon17. We report standard deviations as uncertainty.
IWILDCAM
CAMELYON17 OGUMOLPCHA
FMow
AMAZON
PY150
POVERTYMAP
MACRO-F1
AvG AcC
AP
WORSTR Acc
10 -TLE AcC
METHODCLASS
WSTPEARSONI
95.4
55.0
55.1
0.59-10.03
+AND
1:0.
96.6 0.1
10.6.:0..
48.9
0.2
53.8 0.5
73.4 0.2
0.57:10.01
+SAND
55.4
56.3
0.9
+SDG
48.1 1.0
95.5.1:3 3.8
28.3 = 0.1
56.0 - 3.1
56.6 -1 0.5
75.4 + 0.1
0.60:10.02
ERM
47.0 1.4
5.2
27.8 0.1
58.3 +: 0.9
57.3 - 0.1
75.4 + 0.4
0.57+00.07
perform early stopping and choose the best epoch using validation domain performance. Moreover, to ensure a fair comparison, all methods are run for the same amount of wall-clock time. Since our method is slower, perform fewer epochs than other methods. We list all chosen hyperparameters in Appendix F
3.2 Evaluation on WILDS
Datasets Our main evaluation using the WILDS benchmark 23 WILDS is designed to evaluate domain generalization and subpopulation shift in realistic problems. Among these problems, we use seven problems that are either pure domain generalization problems or a combination of domain generalization and subpopulation shift. These problems cover wide range, including outdoor images (iWildCam), medical images (Camelyon), satellite images (FMoW, PovertyMap), natural language (Amazon). formal language (py150), and graph-structures data (OGB-MoIPCBA with the number of domains from to 120K. The summary of the benchmarks is in Appendix
Baselines We apply our proposed optimizer (SDG) CORAL and denote CORAL+SDG. We compare
with i) CORAL: the standard implementation of CORAL which is jointly minimizing the loss and the penalty, ii) CORAL+AND-MEL MNN-MMKK: Applying the WND-MASKIA CORAL minimization, iii) CORAL+S MALL+AAN--MAS minimization. It is important to note that AND-MASK 30) and SAND-MASK have not been applied in conjunction with CORAL before. Still, we compare with them since they also directly modify the optimization procedure. We compute the gradient of the empirical risk plus coral penalty for each domain separately and utilize their and-mask and sand-mask as corresponding updates.
to Applying the SAND-MASK 38 to CORAL
3.2.1 Main Results
We summarize the out-of- dif-iistrinttion results in Table and in in-distribution results in Table We also compute the generalization gap (difference between in-distribution and distr--iistiitttio performance) and tabulate it in Table 3 When evaluated on out-of out-of-distribe distriibution performance, our method improves CORAL in all settings. The improvement is significant for Camelyon17 with more than 15% and OGB-MoIPCRA with more than 7%. Moreover, although the CORAL outperforms ERM with standard optimizer only for out benchmarks, combined with our optimizer, CORAL outperforms ERM for out of benchmarks. Our method closes the gap for Camelyon17, FMoW,
7

---

Amazon, Py150, and Mave The only setting CORAL fails to outperform ERM when combined with our optimizer is OGBMoIPCBA. We believe one reason for this is the additional computational complexity of our method. For a fair comparison, our method performs significantly fewer epochs than ERM. When compared with AND-mask and SAND-mask, our method outperforms them in all benchmarks.
When evaluated for in-distribution performance in Table 2 our method improves upon CORAL in all bench- marks. This is rather expected as our approach explicitly ensures the optimality of empirical risk. When the generalization gap (difference between in-distribution and out-of- distribution) evaluated in Table our method has similar generaliza- tion gap with CORAL in average (our method has smaller gap for 3 bench-
Table 3: Generalization Gap (difference between in- distribution and out-distribution performance) the WILDS benchmarks [23] We denote the results which generalize better than ERM with Green color and worse than ERM with Red color.
IWILDC.
CAM
MoL
FMoW
AMAZON
PV150
Pov.MAP
CORAL
10.7
35.9
0.5
24
2.2
4.7
0.15
+AND
0.3
20.9
+SAND
13.9
33.2
4.6
22.1
3.2
7.3
0.15
+SDG
13.5
20.3
2.5
22.1
2.9
7.1
0.11
ERM
16
22.9
0.6
25.5
3.5
7.5
0.12
marks, similar gap within the stan- dard deviation for 2, and a larger gap for other 2). Hence, our proposed optimizer improves empirical risk without hurting the generalization ability on out of datasets, Even when our method hurts generalization, the improvement the empirical risk is so large the resulting out-of-distribution performance improves.
3.3 Additional Empirical Analysis
In addition to main study of applying method to CORAL in WILDS benchmark, here we test the generality of our method for penalty function choice (i.e. generality beyond CORAL) as well as application (i.e. generality beyond WILDS).
Applicability beyond CORAL We test generality of our method beyond CORAL. Specifically, we apply method to FISH (39 model in WILDS benchmark [23] and report in Table Application to FISH 339 requires modifications as it is not directly penalty based and we explain them in Appendix Results on FISH (39 suggest that method improves FISH on 5 out of benchmarks. Moreover, the improvement for iWildCam and PovertyMap is significant, more than 9%.
Applicability beyond WILDS We test generality of our method beyond WILDS benchmark. Specifically, we apply our method to FISH (39. VRex [25). and CORAL 42 model in Do- mainBed [18] benchmark. We directly follow the official process. We tabulate the results in Table [4] The results suggest that our method consistently improves the performance of all tested methods.
the benchmark code shared by DomainBed (18] and
Consistent improvement for different domain generalization models on different benchmarks over 2 experimental suites conclusively demonstrates the generality of the proposed method.
Table 4: Out-of-distribution test performance of VRex 25 in the DomainBed benchmarks We average over 3 seeds and report standard deviations uncertainty.
ALGORITIM
C-MNIST
R-MNIST
VLCS
PACS
OFFicEHomE
TERRAINCOG.
DOMAINNET
AvG
FISH
51.6 0.1
98.0 10.0
77.8 1 0.3
85.5 1: 0.3
68.6 1 68.4 0.5
4
42.7
67.1
+DDG
5..9
98.0 10.0
79.9 1 0.5
87.3 4: 0.3
48.4
42.6
68.1
CORAL
0.1
98.0
78.8 0.6
0.3
68.7 0.3
47.6
4..5
67.5
+SDG
98.2 0.1 0.2
79.7 + 0.3
86.9
68.7 + 0.7
49.9 + 1.2
4..4
68.2
VREX
0.1
9..9
78.3
66.4 0.6
46.4
3..6
65.6
+SDG
52.7 + 0.1
98.1 0.2
78.9
86.4 0.3
68.2 0.5
48.1
41.8 + 0.4
67.7
ERM
0.1
98.0 +0.0
77.5 0.4
85.5
66.5 0.3
46.1 1.8
40.9
66.6

---

3.3.1 Impact on training time.
Although our method effectively learns generalizable models, has a rather significant impact on computa- tional complexity. Consider an input batch of B x BD datapoints over Bd domains. Direct optimization only needs the gradient for the average risk, while our method requires a gradient for each domain. Hence, method needs BD backward passes, whereas direct optimization needs only a single pass. Effectively, our training time is up to BD times slower. It is impor- tant to note that our method has very limited impact on training time when the original algorithm computes gradient per domain. For example, utilizing our method with IRM would result in small change in training time.
Table 5: Wall-clock time (in minutes) for training single epoch for CORAL and CORAL+SDG method on an RTX 3090 GPU, Since our method needs gradients per- domain, it needs number of domains in the batch times backward pass during training. This is limitation of our method.
CORAL
CORAL+SDG
iWildCam
57
91
Camelyon17
19
72
MCB-MoIPCBA
22
73
FMoW
9
41
Amazon
100
227
To empirically evaluate the computational impact, we compute the average time per epoch per dataset and re- port it for our method and ERM in Table[5] Our method increases the training time significantly. Although this limitation can be easied using some approximation techniques 336 we leave the potential computa- tional improvements future work. Moreover, it is important to clarify that we decrease the number of epochs for our method to ensure all numbers are fair in terms of used compute.
Py150
186
420
PovertyMap
4
4 Related Work
Domain generalization. Domain generalization addresses settings where the training and test data come from different distributions. violating the common independent and identically distributed (iid.) assumption. The training data comes from multiple distinct data distributions (called domains). The
typical assumption to tackle domain generalization is covariate shift (9), where only the data distribu- tion changes between domains and the conditional distribution of label given data stays the same. We specifically interested in renalty/nvaarance based domain domain generalization. However, the approaches for domain gencralization are more general and include domain conditioning (28) self-supervised pre-training 33, domain-specific decomposition 32 and data augmentation 44
We treat these works as orthogonal to us. One approach for invariance based domain generalization is distributionally robust optimization. Here robust form of risk is optimized instead of the average risk (35] 45 [6]. The robustness is typically against a predefined amount of distributional shift around the training data, hoping that it covers the test distribution. Similarly, Krueger et al. (25) enforce uniformity of risk functions to guarantee robustness following a causality-inspired approach. Another common approach is adversarial feature learning. Here representation learning problem is posed such that an adversary cannot detect the domain information from the learned features. Adversarial feature learning is achieved via game-theoretir min-max formulations [17 26 distributional dis-
tances such as MMD (2]. and divergence of second -order statistics of features 42). Causality inspired
methods is another thrust where the problem is posed in terms of finding features following the causal direction. The out-of out-of-distribee distrribbttoonn generalization problem considered in causality-based based methods addresses a larger class of problems beyond domain generalization since they are applicable beyond covariate shift. An early application of causality to cottof-diisributioo generalization was for linear models 31, later extended to the non-linear case [5). Final category which we discuss gradient alignment. Here gradient-based learning is directly addressed and algorithms to align gradients of risk functions from different domains are developed. This class of methods is the most similar to our case. Koyama& & Yamaguchi (24) enforce gradient similarity by minimizing the variance of the gradients and Shi et al. [39] minimize the cosine distance between gradients. Similarly, [50) finds domain transferrable solutions using meta-learning. Parascandolo et al. [30] combine these ideas with causality- based motivation and propose a method that directly manipulates gradients. Among these methods, ANDMask [0) and SANDMask (38] are the most relevant to since they the only ones that explicitly modify gradients and use sign information.
Rate-distortion approaches. Rate-distortion theory addresses the problem of lossy compression within information theory [3]. Among approaches that are numerical and directly applicable to any distribution, the Blahut-Arimote algorithm is a common choice [ Although there are

---

improved variants [А 48 271, we find the original algorithm simple and effective. To the best of our knowledge, our work is the first application of the Blahut-Arimoto algorithm to the problem of domain generalization. The only application of Siahut-Arimond within machine learning that we are aware of the BLAST algorithm (6). which uses the pnpormation-thhoreee analysis of Thompson sampling [34] for better sample efficiency. Ideas from rate-distortion theory have recently been applied to the problem of learning invariant representations by Zhao et al. [51] to understand the underlying tradeoff's of transfer learning theoretically. Different from [5I], we analyze convergence properties addition to the cooormttionnthhoretii motivation. We also propose a practical method for transfer learning. A closely related problem rate-distortion is the information bottleneck [3),, where the mutual information between representation and input data is minimized while keeping the mutual information between representation and label maximal. Although these methods have partially been applied to domain adaptation and generalization setting [14] 41 13 [2], they typically require stochastic neural networks and additional modeling assumptions; hence, they are not directly applicable to our case. To the best of our knowledge, these methods have not yet been applied to large-scal acage-scale problems on the scale that from these approaches as our method stems from an analysis of gradient descent and an explicit formulation of satisficing gradient descent.
interested in. Our approach is significantly different
5 Conclusion
We considered penalty- based domain generalization and reformulated it to find updates minimizing the surrogate penalty under the constraint that applying these updates eventually converges to the stationary point of the empirical risk. We developed an optimizer that is applicable to existing domain generalization models. Our extensive empirical analysis on the WILDS and DomainBed benchmark validates the efficacy of our method for CORAL, FISH and VRex models.
major drawback of our method is the additional computational complexity as needs gradients per domain. A possible future direction is applying approximation techniques to reduce this complexity. Moreover, we provided an exciting connection between rate-distortion theory and domain generaliza- tion. Another interesting future direction is utilizing ideas from the rate-distortion theory literature to better understand and improve domain generalization.
References
[1] A. Ajalloeian and S. U. Stich. On the convergence of SGD with biased gradients. TeXXi:2008.00051 2021.
[2] K. Akuzawa, Y. Iwasawa, and Y. Matsuo. Adversarial invariant feature learning with accuracy constraint for domain generalization. In ECML PKDD, 2019.
[3] Albuquerque, N. Naik, J. Li. N. Keskar, and R. Socher. Improving -uitof-disttiuution generalization via multi-task self-supervised pretraining. arXiv preprint:2003. 13525, 2020.
[4] S. Arimoto. An algorithm for computing the capacity of arbitrary discrete memoryless channels. IEEE Trans. Inf. Theory, 18(1), 1972.
[5] M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz. Invariant risk minimization. 17Xiv:1907.02883, 2019.
[6] D. Arumugam and B. V. Roy. Deciding what to learn: A rate-distortion approach. 1xXiv22001.06197 2021.
[7] P. Bándi, O. Geessink, Q. Manson, M. V. Dijk. M. Balkenhol, M. Hermsen, B. E. Bejnordi, B. Lee, K. Paeng, A. Zhong. Q. Li. G. Zanjani, S. Zinger, K. Fukuta, D. Komura, V. Ovtcharov, S. Cheng, S. Zeng. J. Thagaard, A. B. Dahl, H. Lin, H. Chen, L. Jacobsson, M. Hedlund, M. Çetin, Halici. H. Jackson, R. Chen, Both, Franke, H. Küssters- Vandevelle, W. Vreuls, P. Bult, B. van Ginneken, J. van der Laak, and G. Litjens. From detection of individual metastases to classification of lymph node status at the patient level: The CAMELYON17 challenge. IEEE Trans. Medical Imaging, 38(2):550 -50, 2019.
[8] S. Beery, G. V. Horn, and P. Perona. Recognition in terra incognita. In 2018
10

---

[9] S. Ben--avvd, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan. A theory of learning from different domains. Machine Learning, 79(1-2):1111775, 2010.
[10] J. Bernstein, Y. Wang, K. Azizzadenesheli, and A. Anandkumar. SIGNSGD: compressed optimisation for non-convex problems. In J. Dy and A. Krause, editors, JCML, 2018.
[11] R. E. Blahut. Computation of channel capacity and rate-distortion functions. IEEE Trans. Inf. Theory, 18(4), 1972.
[12] G. Christie, N. Fendley, J. Wilson, and R. Mukherjee. Functional map of the world. In CVPR, 2018.
[13] T. M. Cover and J. A. Thomas. Elements of information theory (2. ed.). Wiley, 2006.
[14] Y. Du, Xu, H. Xiong, Q. Qiu, X. Zhen, C. G. M. Snoek, and L. Shao. Learning learn with variational information bottleneck for domain generalization. In ECCV, 2020.
[15] Y. Dubois, D. Kiela, D. Schwab, and R. Vedantam. Learning optimal representations with the decodable information bottleneck. In NeurlPS 2020, 2020.
[16] C. Duchi and Namkoong. Learning models with uniform performance via distributionally robust optimization. arXiv:1911.087. 2018.
[17] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. S. Lempitsky. Domain-adversarial training of neural networks. JMLR, 17:59:1-59:35, 2016.
[18] I. Gulrajani and D. Lopez-Paz. In search of lost domain generalization. In ICLR, 2021.
[19] He and D. Tao. Recent advances in deep learning theory. arXiv:20/2.10991 2020.
[20] H. Hu, O. Sener, F. Sha, and V. Koltun. Drinking from firehose: Continual learning with web-scale natural language. arXiv:2007.09335. 2020.
[21] W. Hu, M. Fey, M. Zitnik, Y. Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovee. Open graph benchmark: Datasets for machine learning on graphs. In NeurlPS, 2020.
[22] X. Hu, P. L. A., A. György, and C. Szepesvári. (Bandit) convex optimization with biased noisy gradient oracles. In AISTATS, 2016.
[23] P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Ya- sunaga, R. L. Phillips. S. Beery, J. Leskovec, A. Kundaje, E. Pierson, Levine, C. Finn, and
P. Liang. WILDS: A benchmark of wn-the-wild distribution shifts. arXiv:20/2.07421, 2020.
[24] M. Koyama and S. Yamaguchi. When is invariance useful in an out-of-distribution generalizatiion problem? 1rXiv:2008.00885 2021.
[25] D. Krueger, E. Caballero, J. Jacobsen, A. Zhang, Binas, R. L. Priol, and A. C. Courville. Outto---istii generalization via risk extrapolation (REx), 20XY::0003.00688, 2020.
[26] M. Long. Z. Cao, J. Wang, and M. Jordan. Conditional adversarial domain adaptation. In NeurIPS, 2018.
[27] G. Matz and P. Duhamel. Information geometric formulation and interpretation of accelerated Blahut-Arimoto-tyae algorithms. In IEEE Information Theory Workshop, 2004.
[28] Monteiro, X. G. Serra, J. Feng. V. Dumoulin, and D. Lec. Domain conditional predictors for domain adaptation. CoRR, abs/2106. 138999 2021.
[29] Ni, J. Li, and McAuley. Justifying recommendations using distantly-labeled reviews and fine- sinengrained aspects. In EMNLP, 2019.
[30] G. Parascandolo, A. Neitz, A. Orvieto, Gresele, and Schölkopf. Learning explanations that are hard to vary. 2XX0:20099000229 2020.

---

[31] J. Peters, P. Bühlmann, and N. Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society: Series B, 18(5):947-1012, 2016.
[32] V. Piratla, P. Netrapalli, and S. Sarawagi. Efficient domain generalization via common-specifer low-rank decomposition. In ICML, 2020.
[33] V. Raychev, P. Bielik, and M. T. Vechev. Probabilistic model for code with decision trees. In 0OPŠLA, 2016.
[34] D. Russo and B. V. Roy. Satisficing in time-sensitive bandit learning. arXiv: 2018.
[35] S. Sagawa, P. W. Koh, T. B. Hashimoto, and P. Liang. Distributionally robust neural net- works for group shifts: On the importance of regularization for worst-case generalization. arXiv: 1ete:9911// 1.08731, 2019.
[36] O. Sener and V. Koltun. Multi-task learning as multi-objectivv optimization. In NewrIPS, 2018.
[37] O. Sener and Koltun. Learning to guide random search. In ICLR, 2020.
[38] S. Shahtalebi, J. Gagnon-Audet, T. Laleh, M. Faramarzi, K. Ahuja, and I. Rish. Sand-mask: An enhanced gradient masking strategy for the discovery of invariances in domain generalization. CoRR, abs/2106. 02266, 2021.
[39] Y. Shi, J. Seely, P. H. S. Torr, N. Siddharth, A. Hannun, N. Usunier, and G. Synnaeve. Gradient matching for domain generalization. arXiv:2104.000 (0999777 2021.
[40] H. A. Simon. Rational choice and the structure of the environment. Psychological Review, 63(2), Mar. 1956.
[41] Y. Song, L.. Yu, Cao, Zhou, Shen, Shao, W. Zhang, and Yu. Improving unsupervised domain adaptation with variational information bottleneck. In ECAI, 2020.
[42] B. Sun and K. Saenko. Deep CORAL: Correlation alignment for deep domain adaptation. In ECCV, 2016.
[43] N. Tishby, F. C. N. Pereira, and W. Bialek. The information bottleneck method. CoRR, physics/0004057, 2000.
[44] R. Volpi, H. Namkoong, O. Sener, C. Duchi, V. Murino, and S. Savarese. Generalizing unseen domains via adversarial data augmentation. In NewrlPS, 2018.
[45] R. Volpi, H. Namkoong, O. Sener, C. Duchi, V. Murino, and S. Savarese. Generalizing to unseen domains via adversarial data augmentation. In NeurIPS, 2018.
[46] P. O. Vontobel, A. Kavcic, D. Arnold, and H. Loeliger. A generalization of the Blahut-Arimoto algorithm to finite-state channels. IEEE Trans. Inf. Theory, 54(5), 2008
[47] C. Yeh, A. Perez, A. Driscoll, G. Azzari, Z. Tang, D. Lobell, S. Ermon, and M. Burke. Using publicly available satellite imagery and deep learning to understand economic well-being in Africa. Nature Communications, 11(1), 5 2020.
[48] Y. Yu. Squeezing the Himotoo-Blhhu algorithm for faster convergence. IEEE Trans. Inf. Theory, 36(7):3149-33 4157, 2010.
[49] J. R. Zech, M. A. Badgeley, M. Liu, A. B. Costa, J. J. Titano, and E. K. Oermann. Variable generalization performance a deep learning model to detect pneumonia chest radiographs: A cross- scossnncctional study. PLoS Med, 15(11), 2018.
[50] M. Zhang, H. Marklund, A. Gupta, S. Levine, and C. Finn. Adaptive risk minimization: A meta-learning approach for tackling group shift. arXiv:2007.0293/ 2020.
[51] H. Zhao, C. Dan, B. Aragam, T. S. Jaakkola, G. J. Gordon, and P. Ravikumar. Fundamental limits and tradeoffs in invariant representation learning. arXiv:2012.10713, 2020.
[52] H. Zhao, R. T. des Combes, K. Zhang, and G. Gordon. On learning invariant representations for domain adaptation. In /CML, 2019.
12


Reviews:
[{"review_id": "sTOy5k15Mbn", "reviewer": "Reviewer_zhHr", "summary": "This paper argues that the joint optimization of the empirical risk and the penalty in penalty-based DG methods can increase empirical risk, which hurts performance on both training and unseen distributions. Based on this hypothesis, this paper proposes to minimize the penalty under the constraint of optimality of the empirical risk and presents solutions by connecting to rate-distortion theory. Experiments show that the proposed optimizer (SDG) effectively improves the OOD test performance.", "questions": "My main concerns are about the evaluation of experiments. Please see weaknesses for more details.", "limitations": "The authors addressed the limitations of their work. I didn't see potential negative societal impact of this work.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths (originality, quality, clarity and significance):\n1. As far as I know, the originality is good.\n2. The proposed method is theoretically and technically sound.\n3. This paper is well organized.\n4. The obversion of the generalization gap and in-distribution performance in the WILDS benchmarks is interesting. The proposed optimizer (SDG) significantly improves the performances of CORAL, FISH, and VREx on several datasets.\n\nWeaknesses:\n1. The evaluation of the proposed optimizer (SDG) is a little weak. Experiments with more penalty-based DG methods will make the effectiveness of SDG stronger.\n2. The statement that the proposed optimizer improves empirical risk without hurting the generalization ability as desired in lines 184 – 287 is unconvincing. In Table 2, SDG increases the generalization gap on four datasets (IWILDCAM., OGBMOLPCBA, AMAZON, and PY150) and decreases the generalization gap on only three datasets (CAMELYON17, FMoW, and POVERTYMAP).\n3. In lines 309 – 311, the explanation that the performance of the FISH for FMoW is already strong is far-fetched with the worst accuracy of 34.6.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "kG9xZ5ZPBW", "reviewer": "Reviewer_Uqg1", "summary": "This work argues that penalty-based DG methods often perform poorly in practice as, due to the hardness of joint optimization, they fail to minimize the empirical risk (i.e. in-distribution performance). To address this, the authors: (1) assume that excess risk is not required for generalization, i.e. that in-domain performance need not be sacrificed for out-of-domain performance; and (2) eliminate excess empirical risk by reformulating penalty-based DG such that the penalty is minimized under the constraint of optimal empirical risk. Then, drawing on connections to rate-distortion theory, a method is proposed to solve the proposed optimization problem. Experiments on the WILDS and DomainBed benchmark datasets illustrate the effectiveness of this method.", "questions": "- See above.\n- In Eq. 6, adding in another separate penalty/regularization seemed strange to me -- can't this just be absorbed into the abstract $\\texttt{Penalty}(\\\\cdot)$ term? Was this addition purely for the theory/derivations? Does it make a difference empirically (e.g. via an ablation study)?", "limitations": "Yes, the authors discuss the computational complexity of their method as a limitation in 3.2.2 and in the conclusion.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- *Clarity of writing*\n    - This paper was a joy to read. Lucid writing, easy to follow, well organised. \n    - Sections 2.2--2.4 were particularly impressive.\n- *Novel perspective and interesting solution*\n    - Blaming excess risk for the failures of penalty-based DG methods seems new, as does formulating a DG problem in which the constraint pertains to the empirical risk rather than the penalty (e.g. as in IRM).\n    - While I disagree with the assumptions (see below), I think the perspective, solution and results are valuable to the community.\n- *Nice theory and connections to rate-distortion*\n\n**Weaknesses**\n- *The reported experimental results seem hand-picked*\n    - CORAL and ERM are reported on 6 WILDS datasets, along with their generalization gaps.\n    - FISH is reported on 3 WILDS datasets.\n    - VREx is reported on DomainBed.\n    - The ad-hoc reporting makes me question the results -- why not report all methods on all datasets? In particular, I would question why:\n      - ERM is left out of DomainBed\n      - CORAL and FISH are left out of DomainBed\n      - FISH is only reported on a 3 of 6 WILDS datasets -- this seems to be due to the hyperparameter ranges of FISH being designed/reported for only 3 or 4 datasets, but the results could still be reported to see if SDG improves FISH.\n    - Happy to increase my score if this is addressed.\n- *The central assumption---in-domain performance need not be sacrificed for out-of-domain performance---is often violated*\n    - This essentially assumes no spurious correlations in the (possibly few) training domains---a central motivation for DG methods in the first place, with many works reporting real-world failure modes due to spurious correlations (e.g. [1]).\n    - The authors justify this by claiming that:\n      - \"for the benchmarks we are interested in, which involve supervised learning from multiple datasets, this is not the case as a universal model solving all datasets (seen and unseen) exists\" -- here I would question both the existence of such a universal model and its inference from a small subset of the domains.\n      - \"Since the models we use are significantly overparameterized, the space of solutions to empirical risk minimization is quite large, likely including many domain-invariant and domain-sensitive solutions.\" -- while I agree that the set of minimal-risk solutions is large, I would question how often domain-invariant solutions lie in this set.\n\n\n[1] Geirhos, R., Jacobsen, J. H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020). Shortcut learning in deep neural networks. *Nature Machine Intelligence*, 2(11), 665-673.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "jo9aYBEfLwE", "reviewer": "Reviewer_AMTZ", "summary": "This paper argues that domain generalization methods that jointly minimize the empirical risk with a penalty term fail to decrease the empirical risk and hurt the performance both on the source and unseen target domains. Therefore, it proposes an optimization algorithm that minimizes the penalty under the constraint of optimality of the empirical risk.", "questions": "- Should be \\hat{L} in Eq. (4), (5), and (6) be L since L is already the empirical risk defined in Line 87?\n- Why is the penalty minimized at \\theta^t + G^t in Eq. (5) and (6)? Why is it not \\theta^t - \\eta * G^t?\n- It would be better to point out in the paper instead of the appendix that the constraint is replaced with a stronger constraint of uniform bounds over domains (Line 602, Appendix).", "limitations": "See weaknesses.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed algorithm in this paper provides a new idea for manipulating the gradients during the training for penalty-based domain generalization methods.\n- Overall, the paper is well written and organized.\n\nWeaknesses:\n- As the authors point out, the proposed method can only solve domain generalization problems where a universal model solving all domains (seen and unseen) exists, which could be a weakness when applied to real-world applications.\n- The proposed simplification for the space of gradients for updating looks over-simplified. It would be better to provide some justifications for the choice of gradient space in the paper or consider more sophisticated choices.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept