Title: Neural Evolution Strategy for Black-box Pareto Set Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: evolution strategy

multi-objective optimization

black-box optimization

pareto set learning
Abstract: Multi-objective optimization problems (MOPs) are prevalent in numerous real-world applications. Recently, Pareto Set Learning (PSL) has emerged as a powerful paradigm for solving MOPs. PSL can produce a neural network for modeling the set of all Pareto optimal solutions. However, applying PSL to black-box objectives, particularly those exhibiting non-separability, high dimensionality, and/or other complex properties, remains very challenging. To address this issue, we propose leveraging evolution strategies (ESs), a class of specialized black-box optimization algorithms, within the PSL paradigm. Traditional ESs capture the complex dimensional dependencies less efficiently, which can significantly hinder their performance in PSL. To tackle this issue, we suggest encapsulating the dependencies within a neural network, which is then trained using a novel gradient estimation method. The proposed method, termed Neural-ES, is evaluated using a bespoke benchmark suite for black-box PSL. Experimental comparisons with other methods demonstrate the efficiency of Neural-ES, underscoring its ability to learn the Pareto sets of challenging black-box MOPs.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author proposes a new method called Neural-ES, which integrates evolutionary strategies into the paradigm of Pareto set learning. The key idea is to use neural networks to model and capture the complex dimensional dependencies in high-dimensional design variables, rather than relying on the traditional evolution strategy covariance matrix.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper proposed a novel method for solving the Black-box Pareto Set Learning problem.\n\n2. The experimental setting is perfect, and a large number of numerical test results prove the effectiveness of the method.\n\nWeaknesses:\n1. I checked the provided three refs on Line 230. These papers do not seem to provide a clear convergence rate of the ES method. The author should provide more discussion on the theoretical convergence properties of this method. Can it directly link to the convergence analysis of the standard ES method? Can the authors add a discussion with \"Black-box Optimizer with Implicit Natural Gradient\" about the connection between methods and convergence?\n\n2. I still have concerns about the Pareto properties of the solved MOO. It seems the original problem is solving $F(x)$. But the proposed method is finding the gradient of $J(\\theta)$, is the solution of optimizing $J(\\theta)$ the solution of minimizing $F(x)$. In the convex case, I think the ES method works, but in the non-convex case, the Pareto-stationay point of the distribution approximated objective and original objective seems not equal. (See \"Adaptive Stochastic Gradient Algorithm for Black-box Multi-Objective Learning\")\n\n3. Is there any practical application scenario here, about Pareto Set Learning in the black box scenario? We have lots of Pareto optimization methods, such as PaMaL, LORPMAN, COSMOS, PHN, why do we need it (for the black-box case). In many numerical cases, I think the advantage of black-box optimization is it can solve non-smooth or non-continuous, or complex functions, but ES-based method seems do not have a theoretical guarantee on those problems.\n\nQUESTIONS:\nSee Weaknesses part.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Neural-ES, a black-box optimization algorithm for Pareto set learning in high-dimensional multi-objective problems. The method replaces explicit full covariance matrices with a neural network that outputs a low-rank covariance structure in an autoregressive sampling framework. The authors demonstrate the algorithm on the proposed BBO-PSL benchmark suite and a real-world trajectory planning problem, showing competitive or superior performance compared to state-of-the-art MOEAs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1. The paper clearly articulates why existing methods fail to scale in high-dimensional multi-objective settings. The problem is well-motivated and practically relevant. \n\n2. The mathematical derivations, including the gradient estimation and the natural gradient correction, are rigorous and clearly presented. \n\n3. The authors designed the BBO-PSL benchmark, which covers a diverse set of challenging properties (nonseparability, scalability, known Pareto sets). \n\n\nWeaknesses:\n\n 1. The hyperparameters are fixed across experiments, without a sensitivity analysis. \n\n2. The paper does not include sufficient ablation experiments. This makes it harder to attribute performance gains to specific components. \n\n3. No runtime measurements are reported. It is unclear how much additional time the autoregressive sampling introduces compared to standard ES variants. This makes it hard to assess the method’s practical efficiency.\n\n 4. Most algorithmic components (low-rank approximation, natural gradient) are adaptations of prior work. The main novelty is in combining autoregressive sampling with ES.\n\nQUESTIONS:\n1. Could the authors clarify why the results for EPSL-R1 are not reported in Figure 6(d)? \n\n2. Could the authors provide more details on how the BBO-PSL benchmark suite was designed and constructed?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new method for black-box multiobjective optimization based on Pareto Set Learning (PSL). The approach relies on a Evolution Strategy (ES) scheme to conduct the optimization for specific objectives preference (trade-off), which search for specific preference is fed back to PSL for updating the general PSL set model. The PSL set model provides the sampling distribution used by the ES for the given preference. Results show significantly better performances that several other multi-objective optimization techniques applied to black box problem, including approaches based on PSL.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- A quite straightforward and sound proposal to extend PSL to black box multi-objective optimization.\n- Detailed presentation of the method, with enough details that should allow to reproduce the method from the paper rather easily.\n- The paper is quite well-written and easy to follow.\n\nWeaknesses:\n- Black box optimization usually requires more potential solutions (samples) to be evaluated for achieving good enough solutions compared to cases where analytical equations of the objectives are available. Except for Fig. 3, which shows convergence over generations in a specific case, this is not considered in much details in the paper, while being an important aspect to consider, in my opinion.\n- The experiments are not always easy to follow and the figures presented are not that obvious to understand. For instance, from Fig. 6 (a)-(c), I am not sure about conclusions to have.\n- Likewise, the practical application in Sec. 5.3 is not super convincing, the results in Fig. 7 showing simply that the proposed Neural-ES approach is skewed toward low f2 values, while EPSL-R1 is the opposite. Many of the results presentated appear more visual/qualitative than anything.\n\nQUESTIONS:\n- Can you provide a more detailled assessment of the overall efficiency of the proposed Neural-ES approach regarding the number of samples to evaluate for converging to good set compared to other approaches?\n- I think there is a notion of coverage over the Pareto front is important for the current problem, but this is not much explored, metrics used are IGD and hypervolume. Can you explain how these metrics relate to the fact that the Pareto front is well covered (or not) from a given optimization run?\n\nLIMITATIONS:\nThere is not much direct impact on society coming from the current work, so no discussion is required here. The Limitation paragraph in the Conclusion (sec. 6) consider constraint in multi-objective optimization as a future work is quite out-of-scope and seems to be a general problem with PSL. I am not sure it is much relevant to mention this in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper proposes a NES-based optimization method to solve high-dimensional black-box multi-objective optimization problems (MOPs) via Pareto Set Learning (PSL). PSL maps user preferences to Pareto-optimal solutions using a set model (a neural network). Neural-ES partitions high-dimensional variables into segments, models dependencies via conditional Gaussian distributions parameterized by a neural network, and uses a novel natural gradient estimator for training. Notably, the method scales PSL to over 1000 dimensions.

STRENGTHS AND WEAKNESSES:
Strengths:

1. The proposed method integrates ES with neural networks to capture complex dependencies in high-dimensional spaces, addressing a critical gap in black-box PSL.

2. The proposed method demonstrated scalability to 1024-Dim problems, which significantly beyond prior PSL methods.

3. The authors leverage natural gradient estimation and Fisher information for stable convergence, with proofs in supplementary materials.

Weaknesses

1. Neural-ES requires training a neural network and running ES evaluations. The wall-clock time vs. traditional ES/MOEAs is unreported, raising concerns about practicality.

2. Complexity claims of the proposed method focus on asymptotics but ignore constants (e.g., network inference costs).

3. BBO-PSL functions are synthetic. Real-world high-dimensional MOPs (e.g., neural architecture search) remain unvalidated.

4. The method assumes unconstrained MOPs, but real-world problems often include constraints. The limitation is acknowledged but unaddressed.

QUESTIONS:
Please refer to 'weaknesses' in Strengths And Weaknesses.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03