Title: Generating Informative Samples for Risk-Averse Fine-Tuning of Downstream Tasks
Venue: NeurIPS 2025
Accepted: yes
Keywords: Generative models

data augmentation

score-based generative models

risk

importance sampling

wireless communications

applications
Abstract: Risk-averse modeling is critical in safety-sensitive and high-stakes applications. Conditional Value-at-Risk (CVaR) quantifies such risk by measuring the expected loss in the tail of the loss distribution, and minimizing it provides a principled framework for training robust models. However, direct CVaR minimization remains challenging due to the difficulty of accurately estimating rare, high-loss events—particularly at extreme quantiles. In this work, we propose a novel training framework that synthesizes informative samples for CVaR optimization using score-based generative models. Specifically, we guide a diffusion-based generative model to sample from a reweighted distribution that emphasizes inputs likely to incur high loss under a pretrained reference model. These samples are then incorporated via a loss-weighted importance sampling scheme to reduce noise in stochastic optimization. We establish convergence guarantees and show that the synthesized, high-loss-emphasized dataset substantially contributes to the noise reduction. Empirically, we validate the effectiveness of our approach across multiple settings, including a real-world wireless channel compression task, where our method achieves significant improvements over standard risk minimization strategies.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes to use generative models to sample high risk samples for efficient distributionally robust learning. The authors design an appropriate algorithm that takes advantage of guided generative models to do some form of \"online importance sampling\". The authors present convergence guarantees of their approach. They illustrate the relevance of their approach on synthetic and real datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n- The idea is original, interesting, and can open fruitful avenues both in theory and in practice.\n- The empirical illustrations are well crafted. \n\n**Weaknesses**:\n- The distribution of the losses depends not only on the inputs but also on the predictor. So generating adverse samples from the distribution of the losses at the initial point, does not necessarily reflect the adequate tail distribution of the losses at the predictor we want to use. Consider taking $\\theta_0 = 0$, would the corresponding guided samples be relevant? Probably not. If $\\theta_0$ was the optimum of the ERM, then the guided samples would probably be relevant but this highlights the problem: the distribution of losses one wants the tails of depend on the predictor. Said differently, distributionally robust learning is fundamentally a min-max problem, i.e., best predictor over worst possible distribution around training distribution. What the authors have is essentially a single \"max\" step i.e., a single step towards getting a \"hard\" distribution from the initial point, but this should be iterated. An effective algorithm would probably consist in iterating the current proposal several times. Weirdly the authors argue in favor of this \"fixed importance sampling distribution\", which I don't understand. An ideal way to frame the contribution will probably to fully write the original problem in terms of distributions, see Preliminaries in [1] for example. \n- The key part of the algorithm (and where it changes from usual distributionally robust optimization (DRO)) is in having access to low variance unbiased estimates of the loss and the subgradients of the CVAR objective in $\\alpha$, $\\theta$ (so the MC step in the algorithm). The authors do not show that key point. Their argument goes through proving the convergence of the algorithm by some means and with some bounds from which they argue what should be the choice of initial distribution. This circumvolved path hides whether the claimed objective of the authors \"getting low variance oracles for optimization\" is reached or not.\n- CVAR is not the only risk-sensitive measure that could be used. Spectral risk measures like the extremile [2] partially avoid the problem of CVARs thresholding (see also [3] for optimization of spectral risk measures). That said adequate estimation of tails are always at the core of distributionally robust learning even if some other risk measure than the CVAR is used.\n- The overall idea hinges upon the assumption that the generative model is sufficiently good to generate high risk data points when such high risk data points are not present in the data. In other words it hinges upon the generalization capabilities of the generative model. It would be best to formalize this, i.e., what generalization capabilities are assumed on the generative model for the overall approach to work. Once again formalizing the problem as in [1] could help.\n- Assumptions on Theorem 1 are rather stringent (assuming bounded loss for risk-sensitive applications is always a bit counter intuitive and having other research works making the same assumptions is not a good reason for this). That said, as long as the theoretical results are meaningful to understand (i) the relevance of the approach, (ii) how to further refine e.g. parameters, that's fine. As said above, it would be best for the results to show e.g. lower variance of the stochastic estimates.\n- Eq; 8: the bound on square gradient norms does not give a bound on the variance. So the whole argument does not seem sound.\n- While the illustrations are very well done, the choice of competing algorithms is not very well justified. Namely, the given contribution could be seen as some form of importance sampling (though not refined over iterates). It would be interesting to compare to such approaches.  For example, [1] provided for example analyses of DRO in online learning setup that would be relevant here. \n\n[1] Levy, Daniel, et al. \"Large-scale methods for distributionally robust optimization.\" Advances in Neural Information Processing Systems 33 (2020): 8847-8860.\n[2] Daouia, Abdelaati, Irene Gijbels, and Gilles Stupfler. \"Extremile regression.\" Journal of the American Statistical Association 117.539 (2022): 1579-1586.\n[3] Mehta, Ronak, et al. \"Distributionally Robust Optimization with Bias and Variance Reduction.\" The Twelfth International Conference on Learning Representations.\n\nQUESTIONS:\n- See first weakness point. Can the authors explain why the samples are guided using only the initial point? Why aren't samples refined with new predictors? Please consider reading some of the references given in that part. I sincerely believe that there was a slight misunderstanding on what distribution really matters, but please correct me if I'm wrong. Maybe what the authors missed to highlight is that their procedure should only be used after a first round of empirical risk minimization? \n- Can the authors prove the reduced variance they hope their method can bring (on both losses and subgradients)?\n- Can the authors explain how they chose the other methods they compared to?\n- Can the authors try experiments using extremile rather than the CVAR?\n\nLIMITATIONS:\nSee weakness part.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper develops an interesting approach to using guided generative models to produce synthetic data samples for more robust optimization of the conditional value-at-risk (CVaR). Training for good CVaR is challenging, especially when the target risk quantile is quite high. At these extreme quantiles the events that we would like to produce low-loss predictions for are very rare, and therefore relying on standard datasets to measure and optimize empirical estimates of the CVaR leads to very high variance. This work leverages generative models with guidance to (1) sample from a distribution $q(x)$ that is tilted towards high-loss samples, i.e., $q(x) \\propto \\phi(\\ell(\\theta_0; x))p(x)$, and then (2) use importance weighting on samples from $q$ to minimize a MC estimate of the CVaR. The idea takes advantage of existing ideas in generative modeling and importance weighted optimization, but is elegantly designed and well-motivated.\n\nSTRENGTHS AND WEAKNESSES:\nAs described above, I think that the main idea in the paper makes sense, is well described, and forms a nice connection between recent advances in generative modeling with guidance and data augmentation techniques for risk-averse optimization. The convergence analysis and the connection to importance weighting reducing the variance of the stochastic gradient update provide further support and justification for this approach (which are aligned with standard results in the literature). That said, the empirical results are a bit weak (as I believe the authors already somewhat acknowledged in their discussion of limitations and future directions). Specifically, the paper evaluates on one synthetic task and just one real data task on wireless communications. It would be more compelling to see results on tasks where generating high quality samples may be more challenging --- such as in image generation or text sequences (e.g., to name a few suggestions: optimizing CVaR for image classification in ImageNet, radiology report generation on MIMIC-CXR, or reward modeling / preference learning on language model responses).\n\nQUESTIONS:\n- I'm curious about the effect that the method has on the standard risk. In Table 2 we start to see ERM improve over RAMIS for lower values of $\\beta$. While its expected that methods that optimize the CVaR will not also optimize the standard empirical risk, it would be nice to show these results in the table as well --- even if just to understand the tradeoffs.\n\n- I understand that the fixed importance sampling distribution is computationally efficient, but am curious about how sensitive this is to the choice of the initial model $\\theta_0$. For example, if $\\theta_0$ is uniformly bad, it will not be informative. Is there any other reason _not_ to iterative update the reference model used for loss guidance?\n\nMinor comments:\n- L256 / L291 / Table 1 acronym confusion: Do you use the stochastic subgradient method, the stochastic prox-linear method, or the SPL+ method of the cited Meng & Gower? (And if the latter, please specify a different acronym that SGM, as SGM is already widely used in the paper).\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper provides a new approach for optimizing conditional value-at-risk (CVaR) over distributions generated from diffusion-based generative models. The primary challenge of optimizing CVaR is typically due to being able to efficiently sample high loss samples in order to reduce the variability of CVaR objective. To address the challenge, the paper's approach proposes an algorithm for reweighting the sampling distribution of the generative diffusion models and theoretically analyze the convergence of their approach. They conclude with numerical experiments that compare their approach with benchmark approaches on the same sampling budget and show their approach better optimizes CVaR.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The paper provides a straightforward and effective method for improving the efficiency of generating accurate estimates of CVaR. The approach highlighted by Algorithm 1 and corresponding descriptions for each line were clear and easy to follow. \n2. The paper's theoretical analysis seems correct and provides good motivation for why their approach is effective and what inputs affect the convergence rate. The provide remarks are useful in highlighting challenges in their approach and heuristic solutions for addressing the challenges. \n3. The work feels relevant and important given recent works in generative models. Given the complexity of such models and their outputs, it makes sense that obtaining sufficient samples to optimize metrics like CVaR would be costly. Moreover, given the prevalance of these models, it makes sense that practitioners would be interested in training models that optimize the CVaR of outcomes that leverage these models. \n4. The numerical section while simple highlights intuition for why the proposed approach is effective. Leveraging real-world data also highlights the practical effectiveness of their method.\n\nWeaknesses\n1. The paper does not compare or comment on their theoretical analysis with past work so it's hard to fully understand the significance of their analysis. For example, is the theoretical result similar to past works that work with a fixed dataset or monte-carlo simulation. Similarly, it's unclear if the other methods used in the numerical section achieve similar rates or how it could be substantively different. \n2. The numerics section's synthetic experiment provides limited evaluation as the authors could also include results that study the convergence rate of their approach compared to benchmarks. They could also compare different choices of $\\phi$.\n\nQUESTIONS:\n1. Can any comparison be made about the theoretical convergence rate between the proposed method and other benchmark methods? For example, how does the rate presented in this paper compare to the rates SGM, DORO, etc.? Similarly, how does the analysis differ from previous works that also use importance sampling for non-generative models?\n\n2. It's not clear to me how to select $\\phi$, are there empirical results that compare different choices of $\\phi$?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-10"}