Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper illustrates the desired IRM solution $f_{\text{IRM}}$ does not belong to the Pareto front of the IRMv1/IRMs loss included MOO problem in the environment (Kamath et al., 2021) described. This paper includes the VERx objective by (Krueger et al., 2021) into the MOO problem (IRMX) and shows in Proposition 1 that IRMX enjoy the same set of the invariant predictors as the original IRM in the ideal case. This paper illustrates PAIR recovers the causal invariance (Arjovsky et al., 2019) better than IRMv2 and VREx in Figure 4. This paper proposes to solve the IRMX problem by a MGDA-based optimizer with preference-awareness (Mahapatra & Rajan, 2020) and a 2-stage descent and balance scheme (Gulrajani & Lopez-Paz, 2021). Specifically, in the descent step, the PAIR-o algorithm minimizes the ERM loss. In the balance step, the algorithm follows the EPO search algorithm by (Mahapatra & Rajan, 2020). This paper also proposes a model selection method based on the highest PAIR score on the validation set. To relax the computational burden, this paper proposes only using the gradients of the classifier $w$ and choose needs an empirically large enough preference to the OOD objective, $p_{ood} = (1,1e10, 1e12)$. This paper also consider solving IRMX using linear weighting scheme but it achieves suboptimal generalization vs PAIR. The ablation study shows IRMX and PAIR are picking opposite trade-off between VREx and IRMv1 (Fig 5 (c)). Experiments using DOMAINBED evaluation protocol demonstrate PAIR-s can help worst case performance. 
strength_and_weaknesses: Strength:
1. The motivation of studying the IRMX objective is well discussed. Firstly, this paper has state-of-art literature on OOD generalization well discussed. Early examples (e.g. Figure 2 and Figure 4) help to understand the drawbacks of IRM variants and the effectiveness of PAIR. Discussions in Appendix B also validates the necessity of keeping IRMv1 in the objective. 
2. The proposed model selection criterion PAIR-s achieves consistent worst environment improvement therefore it seems to be a good baseline for follow-up research. I also appreciate the detailed experimental setup.

Weaknesses:
1. I think the experimental setup between IRMX and PAIR-o is not fair enough. Most importantly, I think the one to one correspondence between $p_{ood}$ and linear weighting scheme can be established due to same number of objectives. If this is true, why they have different tuning range? IRMX is from 1 to 1e6 while PAIR is from 1e8 to 1e12 as specified in Fig 5 (d).  Furthermore, IRMX seems can not set different weights between IRMv1 and VREx while PAIR-o can. Is this intended? If IRMX is allowed to tuned as PAIR, I wonder if loss landscapes between these two will match each other in Fig 5(c). Right now they are almost opposite. 
2. I am not quite sure about two context reasons provided at Page 5 on why PAIR-o is preferable than IRMX. For (i), how exactly linear weighting scheme can not reach non-convex Pareto front? For (ii), why is $p_{ood}$ considered as a preference between ERM and OOD but linear weighting scheme is not considered a preference?
3. I think the proposed advantage "PAIR-o can avoid exhaustive tuning" can be over claimed. I do not see clear reason the tuning over preference is less heavy than other algorithms such as linear weighting scheme. 
4. I think Theorem 4.1/D.4 is misleading. Firstly, the formal Theorem D.4 actually describes the ERM generalization between empirical $\hat{f}^\epsilon$ and population $f^\epsilon$ specified by gap $\delta$. It shouldn't be claimed as "PAIR yield an $\epsilon$-approximated solution of $f_{ood}$" In Theorem 4.1. Secondly, the context before this theorem states "ideal preference is usually unknown" but I doubt the theorem can provide any guidance on choosing p. In particular, it tells us that p should have an upper bound depending on $\delta$ - more accurate solution (i.e. small $\delta$) needs smaller $p$. Is this contradicting the choice $p_{max} = 1e12$ in the experiment?
5. The scalability seems to be a honest issue. While I do not want to emphasize this one too much, I encourage the author to look for alternatives. For example, stochastic algorithms seem an option for IRMX with linear weight scheme.
clarity,_quality,_novelty_and_reproducibility: Clarity and quality wise is good. This paper does a great job on explaining ideas. Plenty empirical evaluations and context discussions are given. 
Originality wise can be improved. While I appreciate the engineering dedication, most of the techniques are not novel, from objective, to training, to evaluation, to model selection, as described in Summary Of The Paper. The biggest contribution seems to be Eq (7) with well rounded reason.
summary_of_the_review: While I appreciate the engineering dedication and detailed discussion, my biggest concern is misleading/over-claimed statement, unfair comparison and originality.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors provides many instances of optimization dilemma in ERM vs OOD from
the MOO perspective, and attributed the failures of OOD optimization to the compromised robustness
of relaxed OOD objectives and the unreliable optimization scheme.
strength_and_weaknesses: The paper provides theoretical instances of tradeoffs between optimizing for ERM vs OOD objectives from a Pareto optimality lens and illustrates their empirical techniques with some theoretical justifications. They also provide better alternative optimization algorithms to perform this optimization than the main baselines.

The main gains are derived from two observations: the first being that using OOD vs ERM objectives "can even eliminate the desired invariant solution from the Pareto front". This is very interesting and they provide a theoretical example in Figure 1, although the model is not explained very clearly. More importantly, it is unclear why the invariance property is even a desired property given that selecting for such solutions will provide an inferior solution in terms of OOD/ERM. If the invariance property is indeed necessary, it is then unclear why it should not be incorporated as an alternative objective. A clear explanation of the inadequacies of the current approach should be elucidated further.

The second gain is from "using the traditional linear weighting scheme that linearly reweights the ERM
and OOD objectives, cannot reach the solution if it lies in the non-convex part of the front" and they propose an alternative method of multiobjective optimization that finds a "best" point on the Pareto frontier. It would be good to explore if optimizing with  non-linear scalarizations (https://arxiv.org/abs/2006.04655) that can provably explore the Pareto frontier could help. Furthermore, the proposed method seems to only find 1 "best" point in the Pareto frontier, which almost degrades to single-objective optimization since there is no utilization of multiple Pareto optimal points (for ensembling for example). It becomes unclear why a MOO approach is needed, other than to perhaps inspire the right "weight" tuning? 

In the experiments section, there seems to be no clear "tradeoff" between algorithm objectives that were considered theoretically. Rather it seemed like the different objective were actually the quality of optimization on different environments. Why are these not used as objectives instead? Furthermore, it seems like PAIR-O calls a packaged optimizer (EPO solver), so it does not seem the algorithm was novel.  
clarity,_quality,_novelty_and_reproducibility: Clarity: A bit lacking in the clarify for the theoretical justifications for why invariance is an important principle and why their optimization finds the "best" point out of all Pareto fronts.

Quality/Novelty: The novelty was generally good and the plots are well-done and captions do a good job of explaining some new concepts. The quality of the paper was limited since the main contribution of the paper was to demonstrate the superiority of considering MOO in ERM vs OOD objectives. Throughout the paper, it was not made clear why more objectives should not be considered.
summary_of_the_review: The paper provided an interesting theoretical understanding of OOD losses and the invariance principle, as well as an advocation for using MOO to balance conflicting objectives. However, the lack of novelty in MOO and clarity in the necessity of their MO framework make this paper a weak reject.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This submission analyzes and extends previous invariant risk minimization (IRM) formulations to a multi-objective optimization scheme, named as PAreto Invariant Risk minimization (PAIR) for finding models generalizing well and with good OOD performance. The authors motivated the work by analyzing how IRM may fail to achieve good generalizability and OOD performances and providing practical and theoretical failures of the IRM variants (relaxations). They discussed the importance of the trade-offs between ERM and OOD objectives and choosing desired solutions in Pareto optimal frontier and  developed PAIR-o, as an optimizer for OOD generalization and PAIR-s for model selection by leveraging the previous multi-objective optimization (MOO) solver EPO from the literature. Experimental results on several OOD benchmarks demonstrated the efficacy of the proposed PAIR framework. 



strength_and_weaknesses: 
The elaborated discussion on the practical and theoretical failures of the IRM variants (with relaxations IRMs, IRMX, VREX) is clear and motivating the proposed development. Illustrations, proof-of-concept as well as empirical benchmark results have also demonstrated the potential of the PAIR framework. 

Due to the limited review time, this reviewer may have misunderstood some points but here are some potential issues which the authors may consider addressing. 

1. Mathematical notations may need to be revised to be consistent. For example, the operators changed from $\circ$ in equation (1) to $\cdot$ in equations (2) and (3). Some notations were not defined throughout the submission. Probably due to the page limit, however, Section 4.1 is the part where the details of the methods were described but since the algorithm is also in the Appendix, maybe better description for the methods can help improve readability of the submission. 

2. PAIR considers a composite OOD loss in the MOO setting. What other OOD objective combinations can be made,  which will lead to better performance? Can we make a structured analysis of which objectives should be combined in MOO?

3. It appears that the theoretical analysis for classification was mostly on binary classification problems. While empirical experiments on benchmarks are with multi-class classification problems. The authors may want to state clearly about the generalizability of the theoretical results. 

4. The discussion in Appendix C.3 may also be mentioned in the main paper to clarify where to use PAIR-o and PAIR-s. Can optimization and model selection ideas be combined? Will it be useful in any settings or provide superior performance?

5. The empirical results, in particularly in Tables 2 and 3, did not always show consistent improvements and the improvements are not with a huge margin, if any. More discussions on the observed trends may also help better understand possible gaps. 



clarity,_quality,_novelty_and_reproducibility: The authors analyzed potential issues of previous invariant risk minimization (IRM) formulations and their variants and proposed a multi-objective optimization scheme, named as PAreto Invariant Risk minimization (PAIR) to help improve generalizability and OOD performances. 

Probably due to the page limit and significant amount of theoretical and empirical results to report, the authors may want to consider better organizing the presentation to clearly describe the contributions. Math notations as well as many acronyms may need to be clearly defined throughout the submission too. 
summary_of_the_review: This submission analyzes and extends previous invariant risk minimization (IRM) formulations to a multi-objective optimization scheme, named as PAreto Invariant Risk minimization (PAIR) for finding models generalizing well and good OOD performance, with both solid theoretical and empirical results. The authors leveraged the existing MOO solver for PAIR and it may be more interesting if new solution algorithms can be derived. The presentation may need to be significantly improved as detailed in the above concerns.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper motivates the need for Pareto optimal solutions from the fact that most domain generalization methods can be seen as regularized ERM problems that struggle to outperform ERM.
The paper suggests that this is due to two key problems. One is the approximation of the original OOD objective (often encoded as a regularization). The other is due to using linear penalty methods for approximating a Pareto optimal solution, which is known to only work if the frontier is convex.
Thus, the paper proposes to formulate domain generalization methods as multi-objective optimization problems with ERM as one objective and the OOD objectives as additional objectives.
Then, multiple gradient descent method for multi-objective optimization is applied.
The empirical results demonstrate the benefits of the proposed approach.

strength_and_weaknesses: *Strengths*
- The paper motivates its work via two key problems with domain generalization methods, most of which can be seen as regularized versions of ERM training. Namely, that the surrogate objectives may be inappropriate and that the regularized solutions do not find Pareto optimal solutions.

- The examples and illustrations are well-constructed and instructive for understanding some of the key issues.

- The results demonstrate the promise of using the proposed PAIR-o and PAIR-s schemes for better domain generalization. Particularly, PAIR-s can be applied non-IRM DG methods.

*Weaknesses*
- (Minor weakness) It seems odd that this paper focuses on IRM, as IRM has been often shown to be suboptimal and not necessarily SOTA.  It seems that the overall framework of Pareto optimization might be better applied to more SOTA domain generalization methods.

- (Minor weakness) The novelty It seems that the problems with domain generalization and MOO methods are parts of prior work. Is the main novelty in synthesizing these two areas? I do think it's useful but it seemed the "key problems" (i.e., poor IRM approximations and pareto efficiency) and the MOO method were not novel by themselves. Perhaps this is just a problem of distinguishing the paper's contributions and prior work.

clarity,_quality,_novelty_and_reproducibility: - The paper is well-written and flows logically. See weaknesses above about novelty.

summary_of_the_review: Overall, the perspective of multi-objective optimization for domain generalization is a fresh take on domain generalization and provides practical performance benefits. The clarity of writing and illustrations are very well done and will aid in the works impact. The work could be improved by clarifying its contributions with respect to prior work in IRM variants and MOO. Also, it would be interesting to apply this more generally to other OOD methods.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper is proposing a multi-objective optimization perspective on invariant risk minimization (IRM). The paper starts with demonstration of a problem with optimization of IRMs and later describes this is due to the fact that converting the Bilevel problem into a single objective via Lagrangian results in optimization difficulties. Moreover, authors provide a multi-objective approach to this difficulty and suggest utilizing MOO solvers. The results of this approach is promising and authors provide further theoretical justifications. The paper was reviewed by 4 experts and the summary of the points are as follows:

- Positive Aspects:
  - The paper is very well written and toy problems motivate it well.

- Negative Aspects:
  - Empirical results are not strong enough
  - Mathematical formalism can be improved with more consistent notation and better explanation of the theoretical results
  - MOO approach is not novel

Since the reviews were diverging, I also read the paper in detail and checked all technical details. First of all, I believe MOO approach being not novel is irrelevant as the paper is not about MOO it is about applying MOO to IRM. Hence, I ignore this comment. I agree with the remaining aspects. Moreover, it is also important to note that this paper connects two communities MOO and Domain Generalization/Causality which are somewhat related but never had a strong connection. I believe conceptual novelty of this connection could be very impactful and overwhelm the somewhat limited empirical improvements. I see this as a beginning of such a connection and it can very well be impactful. Overall, I recommend acceptance.
justification_for_why_not_higher_score: The paper is definitely interesting and novel. However, it has very limited applicability. I believe it is important for the specific sub community to hear about but not for larger ICLR community. Hence, I see no reason for spotlight or oral designation.
justification_for_why_not_lower_score: The paper is addressing an important problem which has not been considered before. Moreover, it is written clearly and results are promising. I believe it will be impactful to the generalization/causality community and will connect them with multi-objective optimization community. It clearly has enough merit to be published and shared.
Decision: Accept: poster