Paper:
The minimization of loss functions is the heart and soul of Machine Learning. In this paper, we propose an off-the-shelf optimization approach that can seamlessly minimize virtually any non-differentiable and non-decomposable loss function (e.g. Miss-classification Rate, AUC, F1, Jaccard Index, Mathew Correlation Coefficient, etc.). Our strategy learns smooth relaxation versions of the true losses by approximating them through a surrogate neural network. The proposed loss networks are set-wise models which are invariant to the order of mini-batch instances. Ultimately, the surrogate losses are learned jointly with the prediction model via bilevel optimization. Empirical results on multiple datasets with diverse real-life loss functions compared with state-of-the-art baselines demonstrate the efficiency of learning surrogate losses.

Review:
In this paper authors propose to jointly optimize a critic, that estimates some non-differentiable objective (or an objective with intractable derivatives and/or derivatives that are 0 almost everywhere). Authors conduct numerous experiments, and show improvements over some sota methods, and maybe more importantly - provide a unified way of achieving these across multiple metrics. Paper is well written, very easy to understand and in reviewers opinion, a nice, simple story worth sharing.
Major concerns:
The theoretical result provided is just a citation of an existing proof, rather than a new contribution. More importantly however, the setup considered strongly violates the assumptions of the theorem, for example assumption (b) does not hold for the neural networks, as there are continuum many global minima (imagine having a unit in a neural net, that has a huge negative bias, which causes it to always be "turned off" by relu, then any weight on top of it will have the same final loss value; or simply notice that relu(a*x)=a*relu(x) if x>0, and so you can always "push" the norm of weights from one layer to the next one without affecting the outcome etc.), and so argmin is not a singleton, but the set of exactly same power as the whole parameter space (since for each k there exists a bijection from R^k to R, and R is of continuum power). This property is not a mild assumption, but rather a critical element guaranteeing convergence of such systems, with powerful universal approximators, one cannot hope for such strong convergence results without actually analysing the approximators family. Reviewer strongly believes this should be either removed completely, or just briefly mentioned, rather than made a strong statement in the paper. It is an empirical work, and stands strong on its own rights, there is no need to add theorems with assumptions that are never satisfied in the proposed scenario.
Results provided use somewhat non-standard datasets for deep learning, and as such it is hard to asses statistical significance of the differences reported; while the test datasets sizes are big enough to trust error to 1e-3 level, they are heavily imbalanced and relatively low dimensional problems, which have proven to be hard for neural network many times in the past, consequently I would expect to see confidence intervals or stds of each result, at least for Table 3. The problem becomes even more severe for metrics such as F1 which do not decompose additively and so can be very sensitive to the (false) positive rates values - reported improvements might disappear once these are introduced, but even if one does not outperform hand-crafted proxies for specific objectives, having a unified method that is on-part with those in black-box scenario is a good result (I would argue that even if all the results are slightly worse, it is still worth publishing).
I find it a bit disappointing, that authors did not try to analyse trained models, it would be invaluable to see what kind of aggregation g and h came up with, that is well aligned with losses such as F1 or JAC.
Minor concerns:
The work resembles closely methods of error critic learning - where one uses an update direction coming from a model, that regresses towards the loss itself, or the gradient of the loss (is available), see:
- Neural Network Design for J Function Approximation in Dynamic Programming (Pang and Werbos paper from '98)
- Sobolev Training of Neural Networks (from NIPS; more precisely "Critic" baseline, which has the same functional form as the loss presented in the paper, if applied at the very top of the network only)
It might be worth discussing in the paper.
Overall I recommend weak acceptance, and encourage authors to address some of the concerns raised above.
**Update**
Based on discussion, corrections and new additions, I recommend acceptance of the above work.