Title: Bayesian Optimization with Preference Exploration using a Monotonic Neural Network Ensemble
Venue: NeurIPS 2025
Accepted: yes
Keywords: Bayesian Optimization

Preference Learning

Multi-objective Optimization

Neural Network
Abstract: Many real-world black-box optimization problems have multiple conflicting objectives. Rather than attempting to approximate the entire set of Pareto-optimal solutions, interactive preference learning, i.e., optimization with a decision maker in the loop, allows to focus the search on the most relevant subset.  However, few previous studies have exploited the fact that  utility functions are usually monotonic. In this paper, we address the Bayesian Optimization with Preference Exploration (BOPE) problem and propose using a neural network ensemble as a utility surrogate model. This approach naturally integrates monotonicity and allows to learn the decision maker's preferences from pairwise comparisons. Our experiments demonstrate that the proposed method outperforms state-of-the-art approaches and exhibits robustness to noise in utility evaluations. An ablation study highlights the critical role of monotonicity in enhancing performance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces BOPE-MoNNE (Bayesian Optimization with Preference Exploration via Monotonic Neural Network Ensemble), a novel multi-objective optimization method that learns decision-makers’ preferences by leveraging the monotonicity of underlying utility functions. A key advantage of the proposed approach is its ability to incorporate utility monotonicity through neural network design—an often-overlooked yet critical aspect of preference modeling. Extensive experiments demonstrate the efficiency of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The proposed method uses neural networks as the utility surrogate model and incorporates the monotonicity of utility functions.\n2. This paper proposes one modification acquisition function IEUBO for MoNNEs, experiments demonstrate that IEUBO generally outperforms EUBO in the presence of utility noise.\n3 . The experimental results are comprehensive and verify the effectiveness of the proposed method.\n\nWeaknesses:\nWhile the paper demonstrates strong performance on synthetic benchmarks, the evaluation does not include real-world applications. A discussion of how the method might adapt to such settings—or a case study on real-world data—would strengthen the practical relevance of the contributions.\n\nQUESTIONS:\n1. Regarding lines 161–162: While constraining preference queries to only previously observed output pairs enhances realism, could this restriction hinder the efficiency of preference learning by limiting exploration?\n2. The ablation study section is completely moved to the appendix and somewhat limit. I would encourage the authors to include it in the main text, because the ablation study provides important insights into the contribution of key components of the proposed method.\n3. MoNNE requires training an ensemble of 8 networks, which is more expensive than GP-based methods. Is the performance gain worth the computational cost?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of mulit-objective Bayesian optimization with preference exploration. It proposes a Monte-Carlo based method to solve the problem, using surrogate model of monotonic neural network ensemble. Experimental results indicate that it can achieve superior performance as compared to other baseline methods.\n\nSTRENGTHS AND WEAKNESSES:\nQuality: Overall the proposed method is solid with convincing experimental result. But there are some key concerns I feel the paper needs to address.  \nClarity: The paper is mostly well-written except some minor typo and font issues.\nSignificance: The paper combines several existing techniques. The siginificance of the contribution is medium to me. \nOriginality: As far as I know, this work is original.\n\nQUESTIONS:\n1. What is the benefit of doing separate modelling for g and f? What if we just run preferential BO over the input to preference composite function? What is the intuition why the proposed method outperform PBO in Fig. 3？\n2. Can the method be extended to the popular Bradley-Terry preference model?\n3. How is the NN initialized? I expect this initialization method determines the prior of the model. (Similar to the lengthscale of the GP model.) \n4. In Sec. 4.1, \"Firstly, neural networks could be interpreted as nonparametric when their number of parameters is large enough, providing the desired flexibility.\" I feel this claim is a bit too strong.\n5. What do you specifically mean by \"scale invariance in preference learning\"?\n\nLIMITATIONS:\n1. NN usually requires more data to train. While BO setting has quite small dataset (especially in the initial steps). Can the authors comment on this issue?\n2. The survey would be more complete with more recent works on preferential BO, since similar techniques can be easily transferred to multi-objective setting with theoretical guarantees.\n3. The justification for only using $y$ in the historical data set for comparison seems weak to me. Indeed, if the preference elicitation is cheap, we can even build a preference model for $g$ offline. \n4. Some typos and font issues:\n- Line 28, \"it may\" ---> \"it may be\"\n- Fig. 3, the font for xticks and yticks look small.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces BOPE-MoNNE, a method for Bayesian Optimization with Preference Exploration (BOPE) that learns a decision maker’s preferences from pairwise comparisons. BOPE-MoNNE models the utility function with a monotonic neural network ensemble, which enforces monotonicity through exponential weight transformations and trains on pairwise data. They show neural network ensembles provide better-calibrated uncertainty estimates than Bayesian neural networks, which struggle with scale invariance in pairwise preference learning. The method adapts standard acquisition functions. Across five benchmark problems, BOPE-MoNNE consistently outperforms previous methods, demonstrating that incorporating monotonicity and ensemble-based uncertainty estimation significantly improves performance.\n\nSTRENGTHS AND WEAKNESSES:\nThe method shows strong performance compared to baselines on the test functions presented. Additionally, their contributions include adding monotonicity to the utility model, which could better mimic real-world scenarios than other methods that do not enforce this condition. There are extension ablations provided in the appendix detailed network choices and comparisons to other ways of modeling utility.\n\nHowever, the overall evaluation is fairly limited and more robust evaluation on a larger variety of benchmarks would be more convincing. This includes scaling to more dimensions or more complex utility functions, as well as a greater quantity of evaluations.\n\nQUESTIONS:\nHow does the method perform over a larger variety of utility functions and benchmarks? \n\nAre there cases where BOPE-MoNNE fails/performs worse than baselines?\n\nLIMITATIONS:\nThe authors could further address the limitations of their work. They mention the additional computational overhead but do not discuss any other possible limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn interactive multi-objective optimization, the utility functions are usually monotonic. The paper considers this fact and introduces a new method BOPE-MoNNE. BOPE-MoNNE uses neural network ensemble model with positive weight constraints to ensure monotonicity while modeling the mean and variance. The model is trained on pairwise preferences queried from the decision maker (DM). Experimental results on several established low-dimensional benchmark problems demonstrate that BOPE-MoNNE outperforms all baselines, and achieves performance close to the ideal case where the true utility is known.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- The proposed method effectively incorporates the monotonicity property of utility functions, which is often overlooked in previous work.\n- Experimental results show that the proposed approach consistently achieves lower simple regret compared to existing methods.\n- The paper is clearly written and easy to follow. \n\nWeaknesses\n\n- The method requires a preference query from the DM at every iteration, which could become impractical or inefficient for complex or real-world scenarios where querying is expensive.\n- Experiments are limited to a few standard low-dimensional tasks. It is unknown whether the method performs well on more challenging tasks with higher dimensions.\n- Although the authors state that code is available in an anonymous GitHub repository, I was unable to find the link. The lack of accessible code may hinder reproducibility of the results.\n\nQUESTIONS:\nCould you provide visualizations the Pareto front of different methods? This could offer additional insight into the performance and trade-offs.\n\nLIMITATIONS:\nThe main limitations are the potential inefficiency due to frequent DM queries and the unclear scalability of the method to higher-dimensional or more challenging scenarios.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}