Bayesian Evolutionary Optimization for Heterogeneously Expensive Multi-objective Problems
Creators: name: WANG, XILU
Publisher: name: University of Surrey
Year: 2022
Subjects: Multi-objective optimization
evolutionary algorithm
Bayesian optimization
heterogeneous evaluation

description: Various multi-objective optimization algorithms have been proposed with a common
description: assumption that the evaluation of each objective function takes the same period of
description: time. Little attention has been paid to more general and realistic optimization scenarios where different objectives are evaluated by different computer simulations or
description: physical experiments with different time complexities (latencies) and only a very limited number of function evaluations is allowed for the slow objective. Such problems
description: are known as heterogeneously expensive multi-objective optimization problems (HEMOPs). Accordingly, this thesis aims to present several pieces of work for handling
description: HE-MOPs.<br>
description: <br>
description: To begin with, we provide a clear description of HE-MOPs. Then, some necessary
description: preliminaries are introduced, including Bayesian optimization, Gaussian processes, acquisition functions and a representative multi-objective evolutionary algorithm. In particular, we focus on multi-objective Bayesian optimization, and provide a brief review
description: of the existing work. As we turn to transfer learning techniques to solve HE-MOPs, an
description: overview of Bayesian optimization with transfer learning is given.<br>
description: <br>
description: After that, we focus on bi-objective problems with different evaluation times, and
description: present a surrogate assisted evolutionary algorithm along with a parameter-based transfer learning scheme (T-SAEA). While the surrogate for the cheap objective can be
description: updated on sufficient training data, the surrogate for the expensive one is updated by
description: either the training data set or a transfer learning approach. To find out the transferable knowledge, a filter-based feature selection algorithm is used to capture the pivotal
description: features of each objective, and then use the common important features as a carrier for
description: knowledge transfer between the cheap and expensive objectives. Then, the corresponding parameters in the surrogate models are adaptively shared to enhance the quality of
description: the surrogate models. The experimental results demonstrate that the proposed algorithm outperforms the compared algorithms on the bi-objective optimization problems
description: whose objectives have a large difference in computational complexities<br>
description: <br>
description: Moreover, we have investigated the instance based transfer learning scheme for surrogateassisted evolutionary algorithms (SAEAs) to solve bi-objective optimization where
description: objectives have non-uniform evaluation times. In the first scheme, a co-surrogate is
description: adopted to model the functional relationship between the fast and slow objective functions and a transferable instance selection method is introduced to acquire useful knowledge from the search process of the fast objective. In the second scheme, the training
description: data is augmented for the surrogate for the slow objective function by transferring
description: knowledge from the fast one. Specifically, a hybrid domain adaptation method aligning
description: the second-order statistics and marginal distributions across domains is introduced to
description: generate promising samples in the decision space according to the search experience of
description: the fast one. A Gaussian process model based co-training method is adopted to predict
description: the value of the slow objective and those having a high confidence level are selected as
description: the augmented synthetic training data, thereby enhancing the approximation quality
description: of the surrogate of the slow objective. <br>
description: <br>
description: Finally, as only a few studies have been reported to address HE-MOPs, most of which
description: focus on bi-objective problems, we aim to deal with HE-MOPs having more than two
description: black-box and heterogeneous objectives. To this end, we develop a multi-objective
description: Bayesian evolutionary optimization approach to HE-MOPs by exploiting the different
description: data sets on the cheap and expensive objectives in HE-MOPs to alleviate the search
description: bias caused by the heterogeneous evaluation costs for evaluating different objectives.
description: To make the best use of two different training data sets, one with solutions evaluated on
description: all objectives and the other with those only evaluated on the fast objectives, two separate Gaussian process models are constructed. In addition, a new acquisition function
description: that mitigates search bias towards the fast objectives is suggested, thereby achieving a balance between convergence and diversity. We demonstrate the effectiveness of
description: the proposed algorithm by testing it on widely used multi-/many-objective benchmark
description: problems whose objectives are assumed to be heterogeneously expensive.<br>