Title: Amortized Active Generation of Pareto Sets
Venue: NeurIPS 2025
Accepted: yes
Keywords: multi-objective optimization

multi-objective generation

black-box optimization

guided generation

multi-objective Bayesian optimization

discrete optimization

variational inference

reinforce

active generation

online black-box optimization
Abstract: We introduce active generation of Pareto sets (A-GPS), a new framework for online discrete black-box multi-objective optimization (MOO). A-GPS learns a generative model of the Pareto set that supports a-posteriori conditioning on user preferences. The method employs a class probability estimator (CPE) to predict non-dominance relations and to condition the generative model toward high-performing regions of the search space. We also show that this non-dominance CPE implicitly estimates the probability of hypervolume improvement (PHVI). To incorporate subjective trade-offs, A-GPS introduces _preference direction vectors_ that encode user-specified preferences in objective space. At each iteration, the model is updated using both Pareto membership and alignment with these preference directions, producing an amortized generative model capable of sampling across the Pareto front without retraining. The result is a simple yet powerful approach
that achieves high-quality Pareto set approximations, avoids explicit hypervolume computation, and flexibly captures user preferences. Empirical results on synthetic benchmarks and protein design tasks demonstrate strong sample efficiency and effective preference incorporation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a method for generating Pareto fronts for black-box multi-objective optimisation tasks. They achieve this by using a generative model that creates samples conditioned on being on the Pareto set, and on alignment with preference vectors set by the user.\n\nThe modelling is based on active generation of sets guided by a class probability estimator. Where an estimator is first fit to model the distribution $p(z = 1 | x)$, and then a generative model is created to approximate $p(x | z)$. In this work, user preferences are further incorporated by considering alignment with preference vectors $u$. This results in the need to first approximate three distributions: $p(z | x, u)$ and $p(a | x, u)$ which are modelled using CPEs, and $p(u | z)$ which is modelled with  a von Mises-Fischer distribution and MLE. All the distributions together are then used to fit the ELBO of the generative model using an off-policy gradient estimator with importance weights.\n\nThe model is then shown to work well in a variety of real-world benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- Quality: The paper is technically sounds, uses well known techniques and justifies the methods.\n\n- Clarity: The paper is well written, with good figures, equations, and tables. Most of the concepts get across well and can be understood easily.\n\n- Significance: Pareto front generation is of great importance in many scientific disciplines, and this paper proposes a good solution to the problem.\n\n- Originality: The general ideas of the paper rely on previous work of Steinberg et al. (2025) (ICLR), however, there is enough extensions and challenges introduced.\n\nWeaknesses:\n\n- Quality: I don't think the training of alignment vectors is well justified, indeed intuitively if $x_1$ and $x_2$ are close in space, the alignment should reflect stronger than if they were apart. However, by training on the permutations only there is no difference between the two scenarios. This is particularly concerning when we see in the experiment section, that the spread of points conditions on specific preferences can still be wide. Furthermore, the fact that pointing a preference vector ever so slightly outside the Pareto front results in poor suggestions from the algorithm is concerning, and is probably related to this fact.\n\nFurthermore, in the results, it appears that some of the methods have been implemented poorly, e.g. why is CbAS-TFM always outperformed by Random search.\n\n- Clarity: How to set the preference vectors remains a little unclear. I think a clearer example of how a practitioner would go about choosing the vectors and how this would affect the Pareto front would improve the paper. Additionally, I find it confusing about what the paper means by \"amortised\", since indeed there is re-training of the generative model at each iteration of the optimisation loop which results in a large expensive procedure between iterations -- the only amortisation that I understood related to changing preference vectors however I think this makes the paper premise and title misleading.\n\nOverall, I think the paper could be improved to address the alignment issue, which is both the main strength of the method but appears to have many pitfalls. However, it is mostly technically sound and addresses an important area, so it is in the borderline.\n\nMinor comments:\n- In Figure 3, we see the lines change for Random between the first plot and the rest, making it difficult to read\n- In Eq. (17) should the D_KL lie outside the inner expectation?\n\nQUESTIONS:\n- Generally see my \"weaknesses\" section.\n\n- Is there a way a user can set a preference over the spread of the generation in the model?\n\n- Why do you think that your method which does not calculate hyper-volume improvement, beats methods that do go through the explicit calculations and try to maximise it?\n\nLIMITATIONS:\nNo, the authors claim to discuss limitations in the related work section, however, these are not spelled out clearly. What the limitations are should be written more clearly.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Amortized Conditional Pareto Estimation (A-CPE), a conditional generative modeling framework for online multi-objective optimization (MOO) that enables preference specification at inference time. The authors develop an autoregressive variational distribution $q_{\\phi}(x∣u)$ (motivated by their sequence design tasks) conditioned on preference vectors $u$, which are sampled from an empirical approximation of $p(u∣z)$to amortize over $u$. Pareto-optimality and preference-alignment are modeled using discriminative classifiers $p(z∣x,u)$ and $p(a∣x,u)$, which are integrated into the ELBO objective. To accelerate optimization, the authors employ importance-weighted sampling for gradient estimation and only resample new trajectories when the likelihood ratio under significantly changes. The method is applied online per BO iteration and demonstrates competitive performance to baselines like qEHVI and qNEHVI on bi-objective benchmarks, including both continuous input and discrete sequence design tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- Introduces a flexible generative method for conditional Pareto set sampling that allows user preference specification at inference time without retraining.\n- The use of an autoregressive sequence model is appropriate for discrete design problems and mitigates the difficulty of searching over combinatorial spaces.\n- Avoids explicit hypervolume computation, potentially improving scalability.\n- The paper is well-written. \n\nWeakness\n- **Key concern** on $q(u|z)$ and random sampling step in Algorithm 1 (lines 11–12) relies on previously observed Pareto-optimal points. In settings where user preferences are not specified and the initial data has poor Pareto Frontier coverage, this empirical distribution will not support sufficient exploration of the Pareto front. This should be either addressed explicitly or the scope of the method should be reframed as targeting preference-aware MOO, rather than general MOO.\n- Time and computational cost profiling is missing. A breakdown of time per component (CPE training, generative sampling, ELBO optimization) would help assess scalability.\n- While the paper claims to avoid costly hypervolume (HV) computation, in bi-objective settings HV is computationally cheap. An empirical demonstration in higher-dimensional MOO (e.g., 4–5 objectives) would strengthen the claim of improved efficiency over HV-based methods.\n\nI would be happy to reconsider my score if the author could clarify my key concern and the question below.\n\nQUESTIONS:\n**On the benefit of a generative model**: In low-dimensional continuous input spaces (e.g., Sec. 6.1), what is the practical advantage of using a generative model over directly optimizing $p(z=1|x,u)p(u|z=1)$ via gradient-based or heuristic methods? A controlled comparison against direct optimization would clarify the end-to-end utility of the generative model.\n\nLIMITATIONS:\n- As noted above, the reliance on an empirical $p(u|z)$ may limit exploration unless preferences are explicitly provided. This should be acknowledged as a limitation or mitigated.\n- The scalability of the method on the number of objectives remains unclear. In particular, amortizing over preference vectors  becomes increasingly difficult as the number of objectives grows, a clear elaboration of its scalability boundary, or a dicussion of future work or a limitation claim should be done on this.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new framework for online multi-objective optimization called active generation of Pareto sets. The method builds upon ideas introduced in variational search distributions, with the goal of extending to multi-objective optimization. The method learns a generative model and uses a class probability estimator, where class is Pareto optimality, to condition this model along with user preferences. This model also does not require expensive hypervolume calculation, which is common in other multiobjective optimization techniques. The authors detail the methodology used and show the results of their method compared to other multi-objective optimization algorithms on a few test functions. These results show strong performance over the test functions presented and flexibility with regard to user preferences and type of generative model.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper includes good insight into the ability to control user preferences and flexibility within a generative model-based optimization scheme. The results shown in Figure 2 illustrate the ability to condition on user preferences while also matching or beating the hyper-volume performance of baselines. This is particularly helpful given a trained model can be amortized, as users can input different preferences afterwards given a single trained model. This provides an advantage over other methods such as VSD, which do not support this type of amortization. Additionally, the experiments show comparable results from using a generative model as opposed to a GP-based method, which is a promising aspect of the paper.\n\nThe experiments shown are relatively limited and would be more convincing with more diversity and quantity of test functions. For example, the DTLZ and ZDT test suites contain further problems with varying pareto frontiers. Additionally, the paper would be improved with clearer discussion of its limitations. The authors mention the limitation of hyperparameter tuning, but it is not clear to what extent this is done for the included results. The results shown require selecting the best model, either transformer or LSTM, for individual test problems, suggesting the robustness of the method could be improved, as the results vary across test functions with the same setup. Therefore, if a single generative model is used with A-GPS, the results may be worse than baselines. As a whole, the paper would be improved by showing how these parameters and test setups were chosen to provide fair comparison to other methods.\n\nQUESTIONS:\nEach evaluation appears to have detailed setups for both the baselines and proposed method, which differ across test functions. For example, the model sometimes uses an LSTM and sometimes uses a transformer. However, the performance varies widely across test functions. What is the justification for these decisions and do they provide fair comparison against the baselines? How would you use A-GPS to approach a new problem apriori, including deciding which model to use?\n\nCan you provide further test results? The results shown are fairly limited and a wider range of evaluations would provide more convincing evidence of A-GPS performance. It is mentioned that some test functions have large variations between runs and we do see high standard deviations in the results. Showing further results from other DTLZ and ZDT test problems would be more convincing and help show the true performance of A-GPS.\n\nLIMITATIONS:\nThe authors mention that specifying hyper-parameters can have a significant impact on performance, but could further expand on how these results change and other possible limitations the methodology has. It appears the selection of these hyperparameters has a significant impact on performance and may be highly tuned for the presented results.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel approach to multi-objective optimization (MOO) by directly modeling the Pareto front, rather than relying on traditional model-based methods. This strategy avoids common pitfalls associated with hypervolume-based approaches, offering a more stable and efficient alternative that also works seamlessly for discrete problems. A key strength of the method is its ability to incorporate user preferences into the exploration of the Pareto front without requiring retraining of the Pareto classifier. Empirical results demonstrate strong performance across a representative set of baselines, highlighting the method’s practical effectiveness.\n\nSTRENGTHS AND WEAKNESSES:\nI really enjoyed reading this paper. As someone with expertise in Bayesian optimization but less familiarity with multi-objective generation (MOG) approaches, I found the paper informative and very accessible. The concept of active generation is clearly defined, and the overall presentation is strong. However, I found Figure 1(a) a bit hard to interpret and would have appreciated a more detailed caption—this figure plays a central role in conveying the method’s intuition. In contrast, Figure 1(b) added relatively little. While I don’t feel qualified to assess the significance of the contribution relative to prior MOG methods, I do think the work is well-executed and worthwhile. Thank you for a thoughtful and well-written paper.\n\nMy main concerns lie in the experimental section. Since the method lacks strong theoretical justification, a rigorous empirical evaluation is critical for a venue like NeurIPS. First, I would have liked to see an ablation that justifies the use of the preference alignment component described in lines 148–153—this was the one part of the method I found difficult to follow, and further evidence of its impact would be valuable. Additional ablations (e.g., on the number of samples used to approximate Equation 18) would also strengthen the paper. More seriously, a lot of implementation details for the baselines (mentioned on line 230) are missing entirely (e.g. how are acquisition functions optimized, choices of reference points e.t.c). In Bayesian optimization and MOO, these details are crucial, as implementation nuances can heavily affect performance. Without this information, it’s difficult to assess the validity of the performance comparisons. I also found it concerning that Figure 2 is based on only five repetitions—this raises questions about the statistical robustness of the results. That said, Sections 6.2 and 6.3 were compelling, and I appreciated the inclusion of more challenging, real-world examples to demonstrate the method’s applicability.\n\nQUESTIONS:\nPlease can you provide a more detailed explanation for Fig 1(a) ?\nPlease can you provide detailed info about the baseline methods you ran in BOTORCH?\nPlease can you comment on why you used a sigmoid transform for the domains of the synthetic test problems? Did you then use Botorch in the unbounded space? This feels like it will impair the standard MOBO approaches.\n\nLIMITATIONS:\nNo", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}