Online Multi-Objective Optimization Using
Gaussian Processes
Ryan Roussel, Adi Hanuka, Auralee Edelen, Luis Hidalgo

Handing off Control/Tuning to the Machines
What do we want from a fully (or almost) autonomous system for controlling anaccelerator?
- Able to optimize towards multiple objectives at once
- Satisfy multiple operating constraints- Quickly find reasonable solutions in extremely high dimensional systems(> 30D, < 1 hr tuning time)- Handle parameter/observation noise and drift- Learn from past experiments and simulations

11/4/20+2

Photoinjector Optimization

For online photoinjector optimization we wish
to simultaneously:- Minimize emittances (3x)- Minimize bunch sizes (3x)- Minimize energy spread (1x)
7 objectives

Tuning knobs:- Solenoid strengths (2x)- RF Amplitudes (2x)- RF Phases (2x)6 input parameters
Edelen, Auralee, et al. PRAB 23.4 (2020): 044601.

11/4/203

Normal Accelerator Optimization

11/4/20

4

Multiple Objectives: Pareto Front

Li, Zheng, et al. Structural and Multidisciplinary Optimization 58.5 (2018): 1961-1979.

11/4/20

5Genetic Optimization of PF
NSGA-IIhttps://www.strong.io/blog/evolutionary-optimization

11/4/20175 generations,100 individuals

At 5 seconds per observation,
optimization time > 24 hrs

6Hybrid Genetic Optimization at SLAC
NSGA-II + Gaussian
processes to evaluate GA
created candidates

Song, Minghao, et al. NIMA (2020): 164273.

Iterated neural network
optimized by NSGA-II topropose ideal points

Large improvements in
optimization speedup, but bothmethods rely on batch
observations, suited best forparallel optimization

Edelen, Auralee, et al. PRAB 23.4 (2020): 044601.

11/4/20

7

Accelerator Optimization w/Surrogates

Can we do the same optimization with less observations?
Need an efficient method!

11/4/208

Accelerator Optimization w/Surrogates

11/4/209

Gaussian Processes as a Surrogate#"#! ""
𝑘 𝑥, 𝑥! = 𝜎𝑒 $%"11/4/2010

Gaussian Processes for Optimization

11/4/2011

𝑓(𝑥)Gaussian Processes for Optimization

Chen, Yutian, et al. "Bayesian optimization in alphago." arXiv preprint arXiv:1812.06855 (2018).

11/4/2012

Gaussian Processes for Optimization

Benefits:- Specify tradeoff between
exploration and exploitation- Inherently improves model accuracy inregions of interest- Enables parallel optimization strategies

11/4/2013

Gaussian Processes for Optimization

Adapt this for multi-objective problems
Benefits:- Specify tradeoff between
exploration and exploitation- Inherently improves model accuracy inregions of interest- Enables parallel optimization strategies

11/4/20

14Extension to Multiple Objective Optimization
We wish to find the set of Pareto-optimal
points in objective space by maximizing thecontained Pareto front hypervolume- Each objective has a GP surrogate model- Using the surrogates we calculate theExpected Hypervolume Improvement(EHVI) as a function of the input- Find a point that maximizes the EHVI anduse as our next measurement point

This allows us to find the Pareto front
w/ a small number of measurements unlike
genetic or swarm optimization methods

Yang, Kaifeng, et al. Swarm and evolutionary
computation 44 (2019): 945-95
11/4/20

15

Two Objective Example Optimization
𝑓! = 𝒙 − 1,1"𝑓" = 𝒙 − −1, −1

Red cross – observation points
11/4/20"Analytical Pareto front
16

Alternative Hypervolume Improvement Metrics
Large number of objectives -> too expensive for EHVI calculation ~ O(N^D)

Slow! >>

Fast! >>>>>>>>>

While, Lyndon, Lucas Bradstreet, and Luigi Barone. IEEE Transactions on Evolutionary Computation 16.1 (2011): 86-95.
11/4/20

17

Photoinjector Optimization

For online photoinjector optimization we wish
to simultaneously:- Minimize emittances (3x)- Minimize bunch sizes (3x)- Minimize energy spread (1x)
7 objectives

Tuning knobs:- Solenoid strengths (2x)- RF Amplitudes (2x)- RF Phases (2x)6 input parameters
Edelen, Auralee, et al. PRAB 23.4 (2020): 044601.

11/4/20

18

Simulated Photoinjector Optimization
OPAL simulation of AWA photoinjector
6 mins on HPC cluster

NN surrogate of AWA photoinjector< 1 sec on a laptop

Edelen, Auralee, et al. PRAB 23.4 (2020): 044601.
11/4/20

19

Simulated Photoinjector Optimization- 10 optimization runs
- 20 initial points each
- Pk hypervolume ~ 90 in < 500 steps (NSGA-II ~ 17.5k) factor of 35x speedup, tuned in < 45 mins!

11/4/2020Adding Objective Preferences
Preferences: only calculate HV
inside objective region

11/4/2021Adding Constraints
Constraints: model probability that aconstraint is satisfied

Region ok

11/4/20

Region not ok22

Adding Preferences vs. Constraints
Control

Preference: dE < 0.52 MeV

Constraint: dE < 0.52 MeV

11/4/20

23

Smooth (Localized) Exploration

11/4/20

24

Smooth (Localized) Exploration

11/4/20

25

Smooth (Localized) Exploration

11/4/20

26

Smooth (Localized) Exploration

11/4/20

27

Smooth Exploration (AWA)No modification

Localized acquisition function

11/4/2028

Gaussian Processes for Accelerator Optimization

This work

11/4/2029

Gaussian Processes for Accelerator OptimizationImprove predictive accuracy
Expand capabilities- multi-fidelity simulation results - Include time dependent drift- neural network engine
and noise- manifold GP
Future work
11/4/20

Improve optimization speed- seeded swarm optimization- hierarchical Bayesian optimization

30

Conclusion

Experimental demonstration coming soon!

11/4/2031