AutoOED: Automated Optimal Experimental Design Platform with Data- and Time-Efficient Multi-Objective Optimization

1 INTRODUCTION . Optimal Experimental Design ( OED ) problems in science and engineering often require satisfying several conflicting objectives simultaneously . These problems aim to solve a multi-objective optimization system and discover a set of optimal solutions , called Pareto optimal . Furthermore , the objectives are typically black-box functions whose evaluations are time-consuming and costly ( e.g. , measuring real experiments or running expensive numerical simulations ) . Thus , the budget that determines the number of experiments can be heavily constrained . Hence , an efficient strategy for guiding the experimental design towards Pareto optimal solutions is necessary . Recent advances in machine learning have facilitated optimization of various design problems , including chemical design ( Griffiths & Hernández-Lobato , 2017 ) , material design ( Zhang et al. , 2020 ) , resource allocation ( Wu et al. , 2013 ) , environmental monitoring ( Marchant & Ramos , 2012 ) , recommender systems ( Chapelle & Li , 2011 ) and robotics ( Martinez-Cantin et al. , 2009 ) . A machine learning concept that enables automatic guidance of the design process is Bayesian optimization ( Shahriari et al. , 2016 ) . This concept is extensively studied in the machine learning community from a theoretical aspect and in the single-objective case . However , its practical applications in multi-objective problems are still not widely explored due to the lack of easy-to-use and open-source software . In this paper , we present AutoOED1 , an open-source platform for efficiently optimizing multiobjective problems with a restricted budget of experiments . The key features of AutoOED include : • Data-efficient experimentation : AutoOED employs state-of-the-art MOBO strategies that rapidly advances the Pareto front with a small set of evaluated experiments . • Time-efficient experimentation : AutoOED supports both synchronous and asynchronous batch optimization to accelerate the optimization . We propose a novel and robust asynchronous optimization strategy named Believer-Penalizer ( BP ) , which is instrumental when multiple workers run experiments in parallel , but their evaluations drastically vary in time . 1Code , screenshots , detailed documentation and tutorials can be found at https : //sites.google . com/view/autooed . • Intuitive GUI : An easy-to-use graphical user interface ( GUI ) is provided to directly visualize and guide the optimization progress and facilitate the operation for users with little or no experience with coding , optimization , or machine learning . • Modular structure : A highly modular Python codebase enables easy extensions and replacements of MOBO algorithm components . AutoOED can serve as a testbed for machine learning researchers to easily develop and evaluate their own MOBO algorithms . • Automation of experimental design : The platform is designed for straightforward integration into a fully automated experimental design pipeline as long as the experiment evaluations ( either in simulation or physical ) can be controlled via computer programs . 2 RELATED WORK . Bayesian optimal experimental design Optimal experimental design ( OED ) is the process of designing the sequence of experiments to maximize specific objectives in a data- or time-efficient manner . Therefore , Bayesian optimization ( BO ) ( Shahriari et al. , 2016 ) is usually applied to find optimal solutions with a minimal number of evaluations . Essentially , BO relies on surrogate models like the Gaussian process ( GP ) to accurately model the experimental process and proposes new experimental designs based on defined acquisition functions that trade-off between exploration and exploitation . Popular choices of the acquisition functions include Expected Improvement ( EI ) ( Močkus , 1975 ) , Upper Confidence Bound ( UCB ) ( Srinivas et al. , 2010 ) , Thompson Sampling ( TS ) ( Thompson , 1933 ) . Bayesian OED has found success in a wide range of applications ( Greenhill et al. , 2020 ) and is the main methodology of AutoOED . To further speed up when evaluations can be carried out in parallel , asynchronous BO approaches have been developed ( Ginsbourger et al. , 2010 ; Kandasamy et al. , 2018 ; Alvi et al. , 2019 ) . However , all of the previous literature focuses on single-objective BO rather than the multi-objective scenario . In this paper , we extend several single-objective asynchronous BO methods to multi-objective versions and propose a novel asynchronous method named Believer-Penalizer ( BP ) with the stablest performance on multi-objective benchmark problems . Multi-objective Bayesian optimization MOBO is developed to optimize for a set of Pareto optimal solutions while minimizing the number of experimental evaluations . Early approaches solve multi-objective problems by scalarizing them into single-objective ones using random weights ( Knowles , 2006 ) . Instead of scalarization , some acquisition functions are proposed to compute a single objective , e.g. , entropy-based or hypervolume-based ( Russo & Van Roy , 2014 ; Belakaria et al. , 2019 ; Emmerich & Klinkenberg , 2008 ; Daulton et al. , 2020a ) . Alternatively , MOBO can be solved by defining a separate acquisition function per objective , optimizing using cheap multi-objective solvers ( usually evolutionary algorithms like NSGA-II ( Deb et al. , 2002 ) ) and finally selecting one or a batch of designs to evaluate next ( Bradford et al. , 2018 ; Belakaria & Deshwal , 2020 ; Konakovic Lukovic et al. , 2020 ) . AutoOED implements many of them in a modular way and allows easily changing modules in an unified MOBO framework ( see Section 3.2 ) . Open-source Bayesian optimization platform There are many existing Python libraries for Bayesian optimization including Spearmint ( Snoek et al. , 2012 ) , HyperOpt ( Bergstra et al. , 2013 ) , GPyOpt ( authors , 2016 ) , GPflowOpt ( Knudde et al. , 2017 ) , Dragonfly ( Kandasamy et al. , 2020 ) , AX ( Bakshy et al. , 2018 ) , Optuna ( Akiba et al. , 2019 ) , HyperMapper ( Nardi et al. , 2019 ) , BoTorch ( Balandat et al. , 2020a ) , SMAC3 ( Lindauer et al. , 2021 ) and OpenBox ( Li et al. , 2021 ) . These Python libraries are designed for general applications and have different algorithmic features supported . The feature comparison between AutoOED and these libraries is shown in Table 1 and is further discussed in Section 5.2 . However , they are all targeted for experts in coding without an intuitive GUI . In contrast , there are also software platforms that provide intuitive user interface and visualization to specific domain experts but the platforms can not be used for other general applications , for example , Auto-QChem ( Shields et al. , 2021 ) for chemical synthesis and GeoBO ( Haan , 2021 ) for geoscience . Combining powerful Bayesian optimization algorithms and an intuitive GUI , AutoOED is designed to be a general optimization platform that can be easily used by anyone for applications in any field . 2The comparison is based on AutoOED ’ s core features . ” ∼ ” means the package only supports a single multiobjective algorithm rather than a modular multi-objective framework with several state-of-the-art algorithms . 3 DATA-EFFICIENT MULTI-OBJECTIVE OPTIMIZATION 3.1 PROBLEM FORMULATION ℝ ! ℝ '' ! ! ( ! ) Design Space Performance Space Pareto FrontPareto Set ℝ ! ℝ '' ! ! ( ! ) Design Space Performance Space ! ℝ ! ℝ '' ! ! ( ! ) Design Space Performance Space Pareto FrontPareto Set i e aret ro t esi pac Performance pace Optimal experiment design problems involving multiple conflicting objectives can be formulated as a multi-objective optimization on design parameters as data- and time-efficient as possible . More formally , we consider a optimization problem over a set of design variables X ⊂ Rd , called design space . The goal is to simultaneously minimize m ≥ 2 objective functions f1 , ... , fm : X → R. Representing the vector of all objectives as f ( x ) = ( f1 ( x ) , ... , fm ( x ) ) , the performance space is then an m-dimensional image f ( X ) ⊂ Rm . Conflicting objectives result in a set of optimal solutions rather than a single best solution . These optimal solutions are referred to as Pareto set Ps ⊆ X in the design space , and the corresponding images in performance space are Pareto front Pf = f ( Ps ) ⊂ Rm . To measure the quality of an approximated Pareto front , hypervolume ( Zitzler & Thiele , 1999 ) is the most commonly used metric in multi-objective optimization ( Riquelme et al. , 2015 ) . Let Pf be a Pareto front approximation in an m-dimensional performance space and given a reference point r ∈ Rm , the hypervolumeH ( Pf ) is defined asH ( Pf ) = ∫ Rm 1H ( Pf ) ( z ) dz , where H ( Pf ) = { z ∈ Z | ∃1 ≤ i ≤ |Pf | : r z Pf ( i ) } . is the relation operator of objective dominance and 1H ( Pf ) is a Dirac delta function that equals 1 if z ∈ H ( Pf ) and 0 otherwise . The higher the hypervolume , the better Pf approximates the true Pareto front . 3.2 MODULAR ALGORITHM FRAMEWORK . < latexit sha1_base64= '' Zz6iNM3NIi3avy+two8AsfT7ldM= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeClx4r2lpoQ9lsJ+3azSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCoreNUMWyxWMSqE1CNgktsGW4EdhKFNAoEPgTjm5n/8IRK81jem0mCfkSHkoecUWOlu7D/2C9X3Ko7B1klXk4qkKPZL3/1BjFLI5SGCap113MT42dUGc4ETku9VGNC2ZgOsWuppBFqP5ufOiVnVhmQMFa2pCFz9fdERiOtJ1FgOyNqRnrZm4n/ed3UhNd+xmWSGpRssShMBTExmf1NBlwhM2JiCWWK21sJG1FFmbHplGwI3vLLq6Rdq3oX1drtZaXeyOMowgmcwjl4cAV1aEATWsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wdJxI3R < /latexit > Multi-objective Bayesian optimization ( MOBO ) is a data-driven approach that attempts to learn the black-box objective functions f ( x ) from available data and find Pareto optimal solutions in an iterative and data-efficient manner . MOBO typically consists of four core modules : ( i ) an inexpensive surrogate model for the black-box objective functions ; ( ii ) an acquisition function that defines sampling from the surrogate model and trade-off between exploration and exploitation of the design space ; ( iii ) a cheap multi-objective solver to approximate the Pareto set and front ; ( iv ) a selection strategy that proposes a single or a batch of experiments to evaluate next . These four modules ( see Figure 1 ) are implemented as core and independent building blocks of the AutoOEDs , making it highly modularized and easy to develop new algorithms and modules . The whole pipeline starts from a given small dataset or a set of random evaluated samples , then it works iteratively by proposing new design samples and evaluating them until the stopping criterion is met . For each module in this framework , AutoOED supports following choices : • Surrogate model : Gaussian process , neural network ( multi-layer perceptron ) , Bayesian neural network ( DNGO ( Snoek et al. , 2015 ) ) • Acquisition function : Expected Improvement , Probability of Improvement , Upper Confidence Bound , Thompson Sampling , identity function • Multi-objective solver : NSGA-II , MOEA/D , ParetoFrontDiscovery ( Schulz et al. , 2018 ) • Selection : Hypervolume improvement , uncertainty , random , etc . • Stopping criterion : Time , number of evaluations , hypervolume convergence class TSEMO ( MOBO ) : ’ ’ ’ [ Bradford et al . 2018 ] ’ ’ ’ spec = { ’ surrogate ’ : ’ gp ’ , ’ acquisition ’ : ’ ts ’ , ’ solver ’ : ’ nsga2 ’ , ’ selection ’ : ’ hvi ’ , } class USEMO_EI ( MOBO ) : ’ ’ ’ [ Belakaria and Deshwal 2020 ] ’ ’ ’ spec = { ’ surrogate ’ : ’ gp ’ , ’ acquisition ’ : ’ ei ’ , ’ solver ’ : ’ nsga2 ’ , ’ selection ’ : ’ uncertainty ’ , } class DGEMO ( MOBO ) : ’ ’ ’ [ Lukovic et al . 2020 ] ’ ’ ’ spec = { ’ surrogate ’ : ’ gp ’ , ’ acquisition ’ : ’ identity ’ , ’ solver ’ : ’ discovery ’ , ’ selection ’ : ’ direct ’ , } Code Example 1 : Creating algorithms in AutoOED by simply specifying module combinations . Based on this framework , we implement several popular and state-of-the-art MOBO methods , including ParEGO ( Knowles , 2006 ) , MOEA/D-EGO ( Zhang et al. , 2009 ) , TSEMO ( Bradford et al. , 2018 ) , USeMO ( Belakaria & Deshwal , 2020 ) , DGEMO ( Konakovic Lukovic et al. , 2020 ) . DGEMO exhibits state-of-the-art performance for data-efficient , multi-objective problems with batch evaluations to the best of our knowledge . With necessary modules of the MOBO framework implemented , the algorithms can be easily composed by specifying the choice of each module and inheriting the base class MOBO , see Code Example 1 . Supported choices of each module can be found in our documentation . Users can select an algorithm from this library that best fits the characteristics of their physical system or optimization goals , or they can easily create new algorithms by specifying novel combinations of existing modules in just a few lines of code .

Summary: The paper presents a package for black-box optimization and is specifically designed for the optimization of experimental designs. To this end, the authors build upon multi-objective Bayesian Optimization which allows to obtain good points within a few function evaluations and also enables to obtain a Pareto-Front of non-dominated points. To have an efficient, asynchronous parallelization, the authors propose Believer-Penalizer for choosing the next point while others are still being evaluated. The main idea is to choose either Kriging Believer (KB) and Local Penalization (LP) based on a threshold on the uncertainty in the posterior distribution of the probabilistic surrogate model. In the experiments, the authors show that their package achieves often a fairly good performance.