Scalable Bayesian Optimization Accelerates Process Optimization of Penicillin Production
Abstract
While Bayesian Optimization (BO) has emerged as sample-efﬁcient optimization method for accelerating drug discovery, it has rarely been applied to the process optimization of pharmaceutical manufacturing, which traditionally has relied on human-intuition, along with trial-and-error and slow cycles of learning.
The combinatorial and hierarchical complexity of such process control also introduce challenges related to high-dimensional design spaces and requirements of larger scale observations, in which BO has typically scaled poorly.
In this paper, we use penicillin production as a case study to demonstrate the efﬁcacy of BO in accelerating the optimization of typical pharmaceutical manufacturing processes.
To overcome the challenges raised by high dimensionality, we apply a trust region
BO approach (TuRBO) for global optimization of penicillin yield and empirically show that it outperforms other BO and random baselines.
We also extend the study by leveraging BO in the context of multi-objective optimization, allowing us to further evaluate the trade-offs between penicillin yield, production time, and COemission as by-product.
Through quantifying the performance of BO across highdimensional and multi-objective drug production optimization processes, we hope to popularize application of BO in this ﬁeld, and encourage closer collaboration between machine learning and broader scientiﬁc communities.
Introduction
Process optimization of pharmaceutical manufacturing typically involves tuning many control and composition parameters to maximize yield.
When new manufacturing processes are tested, practitioners often rely on human intuition for optimization and would have to go through many rounds of trial-and-error to gain a preliminary understanding.
They would also optimize input features oneat-a-time resulting in the loss of opportunity in reaching higher yields through global optimization.
In addition, yield data from each set of input control parameter combinations usually takes days to obtain resulting in large time and resource budgets that prohibit further optimization.
Bayesian
Optimization (BO) would be an ideal optimization algorithm for drug production processes because it excels in global optimization of systems with expensive cost functions using fewer experiments and has applications ranging from hyperparameter tuning, molecule screening to materials optimization [1, 2, 3, 4, 5, 6, 7, 8, 9].
However, success of BO has been limited to low-dimensional problems that require smaller sample budgets, whereas the pharmaceutical manufacturing often require optimization in high-dimensions to include the impact of temperature, time, and pH on ﬁnal yield.
Global optimization in high-dimensional spaces is challenging for BO mostly due to the curse of dimensionality, where design space grows exponentially with n dim .
The number of local optima increases and global optimal becomes more elusive.
As size of design space becomes intractable with respect to available budget, there will exist regions that have fewer observations, resulting in larger posterior uncertainty and thus over-exploration [10].
In addition, the commonly used surrogate model
Gaussian Process (GP) [11] shows cubic time complexity O ( n 3 + n 2 · n dim ) with data size n , making the task of ﬁtting a heterogeneous black-box function with a global model computationally costly.
To overcome the high-dimensional optimization challenges and retain the sample-efﬁcient trait of
BO, we apply a trust region BO (TuRBO) for global optimization [10] and demonstrate its efﬁcacy for pharmaceutical manufacturing process optimization.
TuRBO simultaneously keeps a set of local optimizations campaigns through many local GP surrogate models, each ﬁt independently to its own local trust region, and utilizes a multi-armed bandit strategy at each learning cycle to allocate batch samples between each trust region.
We have chosen the fed-batch penicillin fermentation process [12] as a representative drug production process to benchmark the performance of TuRBO and BO algorithms.
We demonstrate the efﬁcacy of TuRBO by quantitatively comparing it against state-of-the-art BO algorithms such as Ensemble Bayesian Optimization (EBO) [13], regular BO, and other commonly used sampling strategies such as random search and Latin Hypercube Sampling (LHS).
Leveraging Expected Hypervolume Improvement (EHVI) [14] as acquisition function, we additionally perform multi-objective optimization via BO and demonstrate the evaluation of trade-offs between penicillin yield, time, and carbon dioxide (CO 2 ) by-product emission.
We offer open source implementation of the penicillin model and benchmarking code to support future development [15].
Penicillin Production
To demonstrate the efﬁcacy of BO in increasing yield for a representative drug production process, we adopt a fed-batch process for penicillin production model proposed by Birol et al [12] and show the closed-loop optimization framework for the system in Figure 1.
We realistically simulate the manufacturing process by taking the impact of temperature, pH into account due to the heat generation and acidity tendency during fermentation and microorganisms cell activity.
In response to global movements in tracking greenhouse gas emissions, we also recorded evolution of CO 2 , and would use it to evaluate additional trade-offs in such production processes.
Suggest next batch of experiments
[Equation]
[Figure]
𝑃 𝑡 𝐶𝑂 "
[Figure]
We assume excess oxygen supply [16] and run simulations under closed-loop control of assigned temperature, pH and open-loop addition of given glucose substrate.
At zero offset of P (penicillin concentration gL ), CO 2 (carbon dioxide concentration mmolL ) and t (reaction time hr )), the initial values of V (culture medium volume L ), X (biomass concentration gL ), T (temperature K ), S (glucose substrate concentration gL ), F (substrate feed rate Lhr ), s f (substrate feed concentration gL ) and H (H + concentration molL ) form the design space as seen in Figure 1 and is governed by:
[Equation]
where rest of the constants can be obtained from Table 2 of Birol et al [12].
To simulate real pharmaceutical manufacturing processes, the fermentation process is run open-loop for penicillin concentration P until P converges or the culture medium volume V exceed V max = 180 L .
Each process variable is updated at ∆ t = 0 .
5 hr intervals to account for the lag in off-line quantitative laboratory measurements often seen in bioprocesses.
Our penicillin production model serves not only as a valuable toolbox for benchmarking optimization algorithms but also as a representative example for drug production processes including a variety of process variables on system dynamics.
Bayesian Optimization for Complex Process Optimization
Sample-efﬁcient Bayesian optimization (BO) [3, 11, 2, 17, 18] aims to solve the problem of ﬁnding a global optimum (min or max) of an unknown objective function [Equation]
where X is a domain of interest in R n dim .
There has been previous work in high-dimensional BO [19, 13, 20, 21, 22, 23, 24], yet many have relied on exploiting additive structure for the objective function [21, 22, 23, 13] such as EBO, and mapping to lower dimensional spaces [24, 25], which still require training a large number of GPs that scale poorly with the number of evaluations in high-dimensions or deep generative models [26].
Evolutionary algorithms [27] are well designed for high-dimensional and multi-objective optimization, yet they require many more experimental budget than BO, making them not suitable for applications such as pharmaceutical manufacturing processes optimization that typically have costly experiments.
To scale BO for high-dimensional process optimization, we have chosen to apply TuRBO [10] to demonstrate the feasibility and efﬁcacy of BO.
TuRBO assembles the advantages of previous high-dimensional BO algorithms, notably replacing global model with multiple parallel and local models to avoid intractable time complexity and uneven posterior uncertainty, and combines it with the use of trust regions (TR) [28, 29].
For TuRBO- m algorithm, m hyperrectangle TRs are held simultaneously, and the size of TRs can be dynamically adjusted based on experimental observation results to avoid over-exploration.
The selection of experiments for subsequent batches are controlled by an implicit multi-armed bandit approach.
b samples per batch are allocated unevenly into each
TR, and at iteration t , the i th new observation will be allocated to the most promising TR l
if x ( t ) i ∈ arg min l (arg min x ∈ TR l f ( i ) l ) where f ( i ) l ∼ GP tl ( µ l ( x ) , k l ( x, x (cid:48) )) .
In addition, we use BO in a multi-objective setting to perform multi-objective Bayesian Optimization (MOBO).
Each objective will have an independent GP surrogate model, and hypervolume will be used as indicator for optimization success.
We adapt EHVI from Yang et al.
[14] as acquisition function, which accelerates convergence [30, 31] towards the true Pareto front in experimental studies and supports researchers in evaluating trade-offs between different objectives.
Results and Discussions
[Figure]
We show the quantitatively evaluated performance of TuRBO and other algorithmic baselines in
Figure 2.
Due to limited experimental budgets in pharmaceutical manufacturing, we would suggest practitioners to examine performance in the ﬁrst 200 evaluations.
We observe TuRBO algorithms considerably outperform EBO and regular BO, which all clearly show acceleration of process optimization over random sampling baselines.
The improvement of holding multiple local models over a global surrogate model can be observed by comparing the performance of TurBO to that of regular BO.
EBO uses an ensemble of additive GPs together with a batch acquisition function to scale
BO to high-dimensions.
The design space of penicillin production is only 7-dimensional, and thus does not clearly show the advantage of EBO over regular BO that utilizes GP as surrogate model and
EI as acquisition function.
We also observe that TuRBO with TR = 5 and TR = 2 are more effective than that of TR = 10.
Depending on the batch size, having too many trust regions would allocate too few samples across each, resulting in inaccurate local surrogate model and subpar optimization performance.
The number of TR should scale reasonably with available batch size per iteration.
[Figure]
In Figure 3, we show the pareto front and hypervolume improvement at each evaluation when performing multi-objective optimization using BO equipped with GP surrogate model and EHVI acquisition function.
Compared to the performance of random search, we clearly observe the efﬁcacy of multi-objective BO in converging towards pareto front and expanding hypervolume in the ﬁrstevaluations.
We see that experiments with highest penicillin yield are not necessarily the most optimal when other experiments can show high penicillin yield but take signiﬁcant less time to ferment and result in lower CO 2 emissions.
While the exact trade-off criteria and chosen optimal process control parameters will be decided by practitioners, we hope to demonstrate how BO can accelerate process optimization and offer comparison of trade-offs in a multi-objective optimization setting.
Conclusions
We demonstrated the efﬁcacy of BO algorithms on optimizing yield of a fed-batch penicillin production process, which serves as a representative case for many more pharmaceutical manufacturing processes.
We have applied trust region BO (TuRBO) to address the challenges of high-dimensional optimization for BO and observe its advantage over other BO algorithms and random baselines.
We observe that a suitable number of trusted regions corresponding to designated batch size most be chosen for best optimization efﬁcacy.
In addition, we show the efﬁcacy of BO in combination with multi-objective optimization, which allowed us to evaluate trade-offs between additional objectives such as reaction time, and CO 2 besides penicillin yield with fewer experiments.
Our observations indicate that variations of BO are suitable tools for optimization drug production processes in highdimensions.
We hope to share our insights with the ﬁeld, and popularize the use of BO and machine learning in future drug production processes.
Broader Impact
Successful benchmarks of scalable BO algorithms for process optimization of Penicillin production show us an example of how machine learning algorithms can help pharmaceutical and potentially broader industrial areas.
Break-through in multi-objective optimization algorithms in high dimensions will have direct impact on the efﬁciency of those processes.
Considering usually the high cost to validate proposed conditions, when combined with efﬁcient experimental technologies (automated labs for example), the optimization algorithms will bring huge economical values to those ﬁelds.
In addition, successful optimization algorithms can also be extended to broader areas, which share similar characteristics with the Penicillin production we discussed here, such as multiple constraints (e g. budget, EHS (Environment, Health, and Safety), etc.), and large number of control parameters.
We hope our work will not only motivate algorithm development and adoption across broader scientiﬁc communities, but also promote the integrated applications of algorithms and cutting edge experimental technologies in broader industrial scenarios