Paper:
Deep metric learning (DML) has received much attention in deep learning due to its wide applications in computer vision. Previous studies have focused on designing complicated losses and hard example mining methods, which are mostly heuristic and lack of theoretical understanding. In this paper, we cast DML as a simple pairwise binary classification problem that classifies a pair of examples as similar or dissimilar. It identifies the most critical issue in this problemimbalanced data pairs. To tackle this issue, we propose a simple and effective framework to sample pairs in a batch of data for updating the model. The key to this framework is to define a robust loss for all pairs over a mini-batch of data, which is formulated by distributionally robust optimization. The flexibility in constructing the uncertainty decision set of the dual variable allows us to recover state-of-the-art complicated losses and also to induce novel variants. Empirical studies on several benchmark data sets demonstrate that our simple and effective method outperforms the state-of-the-art results.

Review:
This paper casts deep metric learning (DML) as a pairwise binary classification problem such that pairs of examples need to be classified as similar or dissimilar. The authors propose an objective function that computes a weighted sum over the pairwise losses in a mini-batch. The weight vector is selected to maximize the objective from a decision set encoding constraints. This formulation is called the distributionally robust optimization (DRO) framework.
The authors argue that the DRO framework is theoretically justified by showing how certain decision sets result in existing machine learning loss functions. This portion of the paper seemed hand-wavy. It is not clear what is the purpose of including the theorem from Namkoon & Duchi. It would be more clear in my view to just make the short point that a certain decision set recovers the DRO with f-divergence as would be expected. The claims with regard to learning theory are over-stated in the paper.
The authors proposed three variants of the general framework. They include a top-K formulation, a variance-regularized version, and a top-K version using a balance between positive and negative examples. The DRO framework and the variants are the main contributions in terms of methodology in this paper. It is also shown that the framework generalizes more complicated recently proposed losses.
The experiments demonstrate the DRO framework consistently outperforms state of the art deep metric learning methods on benchmark datasets by small margins. There is also a computational speed advantage that is shown.
Overall, this paper shows that the ideas from distributionally robust optimization work well in deep metric learning. In particular, the paper shows that by combining the DRO framework with simple loss functions, performance comparable with complicated loss functions can be obtained. This aspect, along with the generality are the main strong suits. That being said, I do not see this paper to be that significant of a contribution. The main idea in the paper seems like a rather direct application of the DRO modeling framework and it does not provide too significant of improvement over the MS loss.  The paper was not written super clearly and was too long. Reviewers were instructed to apply a higher standard to papers in excess of 8 pages and this paper would have been presented more effectively if it was shorter. For these reasons, I recommended a weak reject.