Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose to study different distributionally robust optimization problems in the same form of a composite optimization problem. They propose a variance reduction type of gradient based algorithm to solve the composite optimization problem and prove the convergence rate in both (strongly) convex and nonconvex settings. 


main_review: The paper is well written and the method is well motivated. The theoretical analysis seems to be incremental and is very similar to those in the literature such as Fang et al. and Zhang & Xiao. 

Can you explain the intuition behind Assumption 3? Why is there a gradient lower bound imposed?

What is the choice of \gamma in Corollary 4.2 that makes the results match that in Theorem 4.1?

In the experimental results, the variance seems to be very high. How many repetitions are conducted to obtain the results?

Why did you only compare with one baseline algorithm in the experiments? Although the algorithm structure and the relaxation might be different, it will be good to see the empirical comparison of the proposed algorithm with other algorithms on distributionally robust optimization such as those in Table 1. 

summary_of_the_review: My recommendation is mainly based on the contribution of this work in theoretical results compared with existing work.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper targets on solving distributionally robust optimization (DRO) that considering   distribution shifts in the data. In this paper, they show that  how different variants of DRO are simply instances of a finite-sum composite optimization for which they provide scalable methods by utilizing variance reduction algorithm. They also provide empirical results that demonstrate the effectiveness of our proposed algorithm with respect to the prior art in order to learn robust models from very large datasets. 
main_review: Strengths 
1. clearly written and provide comprehensive comparison among relative works
2. The summarization of different variants of DRO are simply instances of a finite-sum composite optimization is interesting

Weaknesses:
1. While the summarization  is interesting, all of them seems already justified by other literatures. 
2. The theoretically improvement of the proposed algorithm seems not that significant. 
summary_of_the_review: The paper is clearly written and provide comprehensive comparison among relative works.  The summarization of different variants of DRO are simply instances of a finite-sum composite optimization is interesting. However all of them  seems already justified by other literatures. The proposed variance reduced algorithm mainly targets on solving a  finite-sum composite optimization. However, the object function involves a new ingredient  in which they also employ the variance reduction technique to solve it. As compared to the previous variance reduction work in composite optimization, the dealing with the new ingredient   may not be that difficult to handle. Therefore, I tend to reject this paper at this time.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper views different variants of DRO are simply instances of a finite-sum composite optimization, from which efficient optimization algorithms were proposed. The convergence analysis was established for strongly-convex and non-convex settings.  The effectiveness of the proposed algorithm are well demonstrated in experiments. 
main_review: The paper provided a novel view of DRO as composite optimization with theoretical guarantees.  The mini-batch constraint sampling for handling heavily-constrained optimization problem is also interesting.  The experiments on the application to fairness are comprehensive and convincing.  

The main weakness is that the convergence analysis heavily depends on the existing work (Zhang and Xiao, 2019b).  The only difference is the extra empirical risk term.  It is not clear that what is the main challenges in this slightly modified setting. 

Other comment:  Is optimally strongly convex condition the same as PL condition?   It seems that no reference about this condition is mentioned and no specific examples are given. 


**The authors have addressed my questions and the technical differences from (Zhang and Xiao, 2019b)
summary_of_the_review: In summary, the paper seems to be very interesting.  It regards DRO as a composite optimization problem and proposed a a novel mini-batch constraint sampling for handling heavily-constrained optimization problems. The efficiency of the proposed algorithm is sufficiently validated by experiments. The originality and novelty of the proof techniques are very limited.
review_title: null

Meta-review: The main concerns from the reviewers is the novelty of the algorithm and analysis from the CIVR algorithm of Zhang and Xiao (2019b). The author rebuttal clarified the main contributions as reformulation of DRO as composite finite-sum optimization, solving heavily constrained optimization problems through composite optimization, and extension to distributed algorithms. They indeed lead to meaningful contributions to the important topic of DRO and open new avenues for structured constrained optimization problems. The paper is written very clearly and the empirical results on realistic problems are much appreciated.
Decision: Accept (Poster)