Bayesian Optimization for the Explorationof Reaction Conditions
Literature seminar
M2 Yu Irie
2023/9/28 (Thu)Optimization of chemical reaction
BrNOEtONC

NNO2MeH

NMePd catalyst
LigandBaseSolvent
Temperature
Reaction timeetc.

NNCO

NNO2NOEt

MeMe>105 configurations!!
In typical laboratory, time and materials are limited…
Ø Scour chemical literature for similar reactions
Ø ExperienceØ Mechanistic understanding…
2

Reaction optimization in machine learning!

# = %(!)

yBlackbox function

Ø Automated approaches to algorithm optimization
Ø Bayesian optimization
ü High-quality configurations
in fewer evaluations

3

Gaussian process

Model variance
Model mean

E. Brochu, V. M. Cora, N. de Freitas, https://doi.org/10.48550/arXiv.1012.2599

4

Acquisition function

Exploitation
vs.
Exploration

E. Brochu, V. M. Cora, N. de Freitas, https://doi.org/10.48550/arXiv.1012.2599

5

Bayesian optimization

ü Find the objectivefunction
ü Uncertainty is gradually
minimized
ü Where to observe next isdetermined
automatically

E. Brochu, V. M. Cora, N. de Freitas, https://doi.org/10.48550/arXiv.1012.2599

6

Bayesian optimization

Initial data
Surrogate model
Acquisition function
New experiments
updatel Application to diverse search spacesl Selection of multiple experiments in parallelØ Optimization of chemical processes
7Table of contents
1. BO as a tool for reaction optimization
1. Introduction
2. Model optimization
3. Performance benchmarking (BO vs. chemists) andApplications
2. Optimizing BO to find general reaction conditions
1. Introduction
2. Data-guided down-selection, experiments and modeloptimization
3. Optimization and quantification
3. Summary

8

Table of contents
1. BO as a tool for reaction optimization
1. Introduction
2. Model optimization
3. Performance benchmarking (BO vs. chemists) andApplications
2. Optimizing BO to find general reaction conditions
1. Introduction
2. Data-guided down-selection, experiments and modeloptimization
3. Optimization and quantification
3. Summary

9

Application to chemical reactions
Conventional
reaction optimization

Ø Acceleration of the
optimization of synthetic
reactions

Experiment

Designing thenext experiment

Experiment

Analysis

Designing thenext experiment

Analysis

Bayesian optimization (BO)B. J. Shields, J. Stevens, J. Li, M. Parasram, F. Damani, J. I. M. Alvarado, J. M.
Janey, R. P. Adams, A. G. Doyle, Nature 2021, 590, 89–96.

10Application to chemical reactions
Conventionaloptimization
1. reaction
Application
to typicalbatch chemistry
2. General-purpose
software platforms
Experiment
3. Systematiccomparisons
to the performance ofchemists
Designing thenext experiment

AnalysisØ Acceleration of the
optimization of synthetic
reactions

Experiment

Designing thenext experiment

Analysis

Bayesian optimization (BO)B. J. Shields, J. Stevens, J. Li, M. Parasram, F. Damani, J. I. M. Alvarado, J. M.
Janey, R. P. Adams, A. G. Doyle, Nature 2021, 590, 89–96.

11

Table of contents
1. BO as a tool for reaction optimization
1. Introduction
2. Model optimization
3. Performance benchmarking (BO vs. chemists) andApplications
2. Optimizing BO to find general reaction conditions
1. Introduction
2. Data-guided down-selection, experiments and modeloptimization
3. Optimization and quantification
3. Summary

12

Tuning of the algorithm components

initial data

optimizer

ü Good average
performanceü Outcomes with low
variance with respect
to the initial data

optimalconditions

Reaction optimizations withdifferent random initialstarting data

R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96. 13

Training data used to select BO parametersØ Published results of HTE (HighThroughput Experiments) of 6
reactions in total were used formodel optimization.

P. Richardson, N. W. Sach, et al. Science 2018, 359, 429–434.
R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96. 14

Training data used to select BO parameters

S. D. Dreher, A. G. Doyle, et al. Science 2018, 360, 186–190.
R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96. 15

Selection of descriptors

initial data

optimizerxMeN

optimalconditions

computer cannot
understandNMe

moleculardescriptor○In this report, Density Functional Theory Descriptors was the best.
Contains: global (e.g. LUMO energy and dipole moment)/
local (e.g. atomic NMR shift and charge for labeled atoms)
electronic and global steric (e.g. molar volume) descriptors etc.
R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96.

16

Optimization of an acquisition function
ü utilization of bothinformation anduncertainty to drive
optimization

Ø Pure exploration

YieldYield

Ø Pure exploitation

Reaction parameter

Reaction parameter•

Could become trappedin local maxima•

May achieve the best
global understanding

R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96.

17Balancing exploration and exploitation

Two-dimensional representation ofreaction 1

Explorer: Better fit, low yield
Exploiter: High yield

Model fit score

Balancing

Expected
improvement

R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96. 18

Table of contents
1. BO as a tool for reaction optimization
1. Introduction
2. Model optimization
3. Performance benchmarking (BO vs. chemists) andApplications
2. Optimizing BO to find general reaction conditions
1. Introduction
2. Data-guided down-selection, experiments and modeloptimization
3. Optimization and quantification
3. Summary19

Test in a new reaction space
Palladium-catalyzed C–H functionalization

Experiment

Experiment

vs.
Designing thenext experiment

Analysis

Designing thenext experiment

Data collection

Analysis

Bayesian
optimization (BO)R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96. 20

BO vs. human experts (Data collection)• Experiments that wereexpected to give asatisfactory distribution
across the larger search
space was selected.
• Collected experimental
results for the entire
search space via HTE.

R. P. Adams, A. G. Doyle, et al. Nature2021, 590, 89–96. 21

BO vs. human experts (Game results)

ML performance

human performance

Optimizer

Humans

Initial choices

randomsignificantly
better selection

Final results○ (the averageperformancewithin threebatches of fiveexperiments)△

R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96.

22

BO vs. human experts (Game results)•Best- and worst-case bounds for average human performanceØ Bayesian reaction optimization on average outperformed
human experts.
R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96. 23

Optimization of a Mitsunobu reaction

Ø From 180,000 possible configurations in total…
ü The optimizer quickly surpassed the benchmark result.
ü Three distinct sets of reaction conditions (99% yield)
were identified in only four rounds of ten experiments.
R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96. 24

Optimization of a deoxyfluorination reaction

Ø From 312,500 possible configurations in total…
ü The optimizer surpassed the benchmark result within
three rounds of five experiments.
ü Reaction conditions that produced TM in 69% yield wereidentified in ten rounds of experiments.
R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96.

25

Short summary

ML performance

human performance

Experiment

Designing thenext experiment

Analysisü An efficient and solid method tooptimize reaction conditions

Bayesianoptimization (BO)R. P. Adams, A. G. Doyle, et al. Nature 2021, 590, 89–96. 26

Table of contents
1. BO as a tool for reaction optimization
1. Introduction
2. Model optimization
3. Performance benchmarking (BO vs. chemists) andApplications
2. Optimizing BO to find general reaction conditions
1. Introduction
2. Data-guided down-selection, experiments and modeloptimization
3. Optimization and quantification
3. Summary27

Optimization of general reaction conditions

N. H. Angello, V. Rathore, W. Beker, A. Wołos, E. R. Jira, R. Roszak, T. C. Wu, C. M.
Schroeder, A. Aspuru-Guzik, B. A. Grzybowski, M. D. Burke, Science 2022, 378, 399–405. 28

Importance of general reaction conditions

Automatedsynthesis
methods forpeptides, nucleicacids, andpolysaccharides

Highly generalreaction conditions

Automated oligosaccharide synthesis
P. H. Seeberger, et al. Science 2001, 291, 1523–1527.

29

Reaction conditions for small organic synthesis
conditions

conditions’
10%conditions’Optimization…

conditions’90%

Substrate scope

To identify general conditions:
all possible combinations of substrates
× all possible combinations of reaction conditions
Ø Difficult to navigate via standard approaches
B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405. 30

(Hetero)aryl Suzuki-Miyaura cross-coupling
A promising method to synthesize heteroaryl molecular fragments:
SMC (Suzuki-Miyaura cross-coupling)

OMeN
BO
O

HNO
O

X

H
N

Pd, base,temperature
5:1 dioxane:waterO

Ø Finding general conditions•

This had been attempted,
but failed, to discover themby mining the extensive
chemical literature
M. D. Burke, B. A. Grzybowski, et al. J. Am. Chem. Soc. 2022, 144, 4819–4827.

31

Overview of the method
1. Data-guided matrix
down-selection torender the vast search
space tractable
2. Modified BO to
efficiently drive
prediction optimization
3. Robotic
experimentation toincrease throughput,precision, andreproducibility
Ø Succeeds in identifying general reaction conditions!
B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405. 32

Table of contents
1. BO as a tool for reaction optimization
1. Introduction
2. Model optimization
3. Performance benchmarking (BO vs. chemists) andApplications
2. Optimizing BO to find general reaction conditions
1. Introduction
2. Data-guided down-selection, experiments and model
optimization
3. Optimization and quantification
3. Summary33

Data-guided down-selection of substrates

Potential product spacecompared to all previously
reported heteroaryl
products

Substrate pairs thatmaximize mutual
dissimilarity of theresulting products

B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405. 34

Data-guided down-selection of conditionsMeN
BOOXOO•••

catalyst, ligand,base,
temperaturesolvent

extent of prior use
structural diversity
functional diversity
selected conditionscatalysts

Pd SPhos G4, Pd(PPh3)4,PdXPhosG4, Pd P(tBu)3 G4,
Pd PCy3 G4, Pd2(dba)3, and Pd(dppf)Cl2

bases

sodium carbonate and potassium phosphate

temperatures

60° C and 100°C

solvents

dioxane, toluene, and dimethylformamide,
all used in 5:1 mixture with water
B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405. 35

Robotic system for reaction performance

When each reaction was repeated
twice, the yields exhibited only ±2% deviation.
Ø One of the key advantages of automated experimentation
B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405. 36

Seeding experiments

Catalysts with• similar functions• poor performance
were eliminated.
528 reactions in total
B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405.

37

Uncertainty-minimizing ML for generality• Aim: to maximize the objective function f(c)C={c}: the set of possible reaction
conditions
S = {s}: a set of substrate pairsy(s,c): reaction yield• BO: each experiment performed immediately provides
information about the objective function• In this case: determination of f(c) for given conditions
requires experiments with every pair of substrates in the Sset

B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405.

38

Table of contents
1. BO as a tool for reaction optimization
1. Introduction
2. Model optimization
3. Performance benchmarking (BO vs. chemists) andApplications
2. Optimizing BO to find general reaction conditions
1. Introduction
2. Data-guided down-selection, experiments and modeloptimization
3. Optimization and quantification
3. Summary39

Advantages found for model-guided researchü Apparent efficiency compared to random samplingü Uniformly distributed yields over the range of possible
valuesB. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405.

40Yields of reactions the model requested

Exploration of good reactions in the second iteration
Ø Attention shifted toward the “negative examples”
Findings through the overall analysis:(i) Relatively good candidate solutions were identified early(ii) The model initially tried to look for better-yielding reactions(iii) More and more attention was dedicated to decreasing theuncertainty of its estimates as the “loop” progressed
B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405. 41

Quantifying generality

95% successful

B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405. 42

Comparison with the reported conditions

ML-discovered general reaction conditions performed
substantially better.

B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405. 43

Short summary

A nearly impossible challenge of finding general conditions
was overcome by ML-assisted approach.
B. A. Grzybowski, M. D. Burke, et al. Science 2022, 378, 399–405. 44

Table of contents
1. BO as a tool for reaction optimization
1. Introduction
2. Model optimization
3. Performance benchmarking (BO vs. chemists) andApplications
2. Optimizing BO to find general reaction conditions
1. Introduction
2. Data-guided down-selection, experiments and modeloptimization
3. Optimization and quantification
3. Summary45

Summary

Experiment

Designing thenext experiment

Analysis

Bayesian optimization (BO)
ü Optimization of the reaction
conditions using certain
substrates utilizing BOü Further modified ML-assisted
method to find generalconditions applicable tovarious substrates
46