Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper show the equivalence of Optimal Transport Distributional Robust Optimization and Tikhonov regularization under martingale constraints. Inspired by this observation, the paper further introduce some new regularization techniques, both for linear regressions and black-box models such as neural networks.  
strengths_and_weaknesses: **Strength**: 

1. The presentation in this paper is good, all the proofs in Appendix are carefully written. 

2. The proposed regularization method works better than the old OT-DRO method, and the theoretical support is solid.

**Weakness**: I think this is a strong paper with no major weakness. 

questions: 1. How do you get the last equality of the proof of Proposition 3.2 in the main text? Can you explain this step clearly? The Holder's inequality provides that 
$$ (\beta^{T} \Delta)^2 \leq |\Delta|_{M}^{2}  |\beta|_{M^{-1}}^{2}, $$ 
with the equality occurs if $\Delta = M^{-1} \beta$ (up to a constants). After that, we need to construct a $\pi$ (or a sequence of $\pi$) to attain this equality and satisfy all the imposed constraints. Although the other longer proof in the Appendix is correct, I am still confused about this short proof in the main text. 

2. The fact that we can choose the cost function in OT to be adjusted by a positive definite matrix $M$ is nice. But in practice, how do we choose $M$ rather than identity matrix $I$? It would be helpful to have some sentences to explain it around line 90-93. 
limitations: The paper is well written and has no major limitations.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes  a connection between Tikhonov regularization and optimal transport map with exact martingale constraints. This finding provides an explanation and guidance to the existing regularization in DRO problem. The authors also introduced a meaningful implementation to solve the problem at hand.
strengths_and_weaknesses: Strengths: The paper  appears to be theoretically sound, firstly establishing a connection between regularization and martingale constraints, thereby building a potential basis for adversarial learning.
Weaknesses: The notation in the paper is rather  hard to track, e.g., L_\beta, l(f_\beta), and f_\beta are three different things. The experiments are rather thin. It  would be more useful  to show some comparisons with existing regularization approaches. Please also see questions below

questions: a)	Theorem 3.7 requires a convex quadratic function and linear mapping. I think these two constraints are quite strict in the adversarial learning problem, how can you guarantee these?  Can one relax these two conditions if so how? 
b)	In the first experiment of section 5.2, the  intuitive explanation of the results is not very enlightening if not meaningless to reader. Is there a  statistical measure to estimate the ensuing  results?


limitations: The model should also be tested on a more complex dataset thereby provide  a deeper understanding on the effectiveness and improvement of the model.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors focus on the problem of distributional robust optimization, which consists in training a model, assuming the learned distribution lies in a ball around the empirical dataset (w.r.t. to some metric). They consider the Wasserstein distance as such metric, and propose to add another martingale constraint on the true distribution. It is motivated by imposing a provably higher dispersion on the learned distribution. Inspired by a previous work proving the equivalence between distributionaly robust problem in some setting and a regularized MSE minimization problem, they prove an analog result on their formulation.  The martingale constraint is penalized using a Mahalanobis norm. They propose an subgradient algorithm to estimate the model. They provide experiments on synthetic and MNIST data.
strengths_and_weaknesses: The contributions of this work are interesting. The equivalence with a regularized MSE problem is interesting. However, I am not an expert of distributionally robust optimization, so I cannot precisely assess the novelty of the authors contributions. I provide below some remarks.

1) The derivation of the duality is a bit fast in the paper. In particular I am not sure how the martingale constraint is handled. It seems that in the proof you assume the adversary distribution is discrete (similar to the empirical distribution), so that the martingale constraint can be handled, and to obtain a duality result as for Theorem 2.3. Is it indeed the case ? If so, isn't it a restriction to assume it is discrete ? The adversary distribution could have a continuous density for instance.

2) There seems to be a typo at line 147.

3) Concerning Proposition 3.6, I wonder how behaves this result when taking asymptotics. For instance if \epsilon goes to infinity, we should retrieve Proposition 2.2, but it does not seem obvious to retrieve the square root of the MSE. Could you discuss why we should retrieve such result ?

4) Could you provide the definition of adversarial RMSE in the main body, so that we are sure what is plotted in figure 1 ?

5) At line 242 you say that "without loss of generality" we can assume that M=I. However I see no argument to justify it. Is it a simplification assumption, or can one prove the loss and gradients is the same for any covariance  M ?

6) It would be interesting to track time in the numerical experiments. The penalty seems much more complicated, thus do the benefits of the performance outweigh the extra computation complexity of the model ?
questions: Could you please provide an answer to the above remarks (1), (3), (4), (5) and (6) ?
limitations: The authors did not adress societal impact, but I think it is limited for this work.
review_title: null

Meta-review: metareview: This work focuses on robust stochastic optimization (under a Wasserstein constraint), and shows the efficiency of Tikhonov regularization for this problem.

There has been a lively and constructive discussion between authors and reviewers, and ultimately all agree that this work should be accepted, and so do I. 
confidence: Certain
Decision: Accept