DeepBO: Deep Neural-Network Bayesian
Optimization of Polaritonic Metasurfaces inContinuous SpaceZihan Zhang
Zhiyuan CollegeSchool of Mathematical
Science
Shanghai Jiao Tong
UniversityJintao ChenZhiyuan CollegeSchool of Electronic Informationand Electrical Engineering
Shanghai Jiao Tong
UniversityKehang Cui
School of Material Scienceand Engineering
Shanghai Jiao Tong
Universitycuikehang@sjtu.edu.cn

Abstract
Thermophotovoltaics (TPVs) rely on selective thermal emitters to tailor the blackbody radiation at high temperatures into band-matching emission for photovoltaiccells, resulting in power-conversion efﬁciencies surpassing the Shockley-Queisserlimit. The selectivity of the thermal emitter must cover three orders of magnitude range of wavelengths, spreading from visible to the far infrared, which requires the superposition of multiple transformation theories of optics and degrees
of freedom anisotropic geometries. It is extremely challenging to realize suchhigh-dimensional complex metasurface design using conventional computational
photonics. Here we develop a deep neural network-based Bayesian optimization(DeepBO) framework to screen a 16-dimensional design space of 1043 candidates,
and realize a record-high spectral efﬁciency of 69% for the TPV emitter. We showthat the neural network combined with Bayesian linear regression is an efﬁcientand robust surrogate model which scales linearly with the size of data. We also
reveal the underlying physical mechanisms of the geometric design of the TPVemitters using principal component analysis (PCA). We anticipate the DeepBO
framework is a useful tool for data-intensive complex geometric design for photonics research community.

1Introduction

Metasurfaces are sub-wavelength metal/dielectric structures that resonantly couple to the incoming electromagnetic ﬁelds [1]. The amplitudes, phases, and polarization states of the photons canbe tailored in a customizable manner, enabling applications in quantum-optical processors, energy
harvesting, sensing, communications, etc. The most critical feature of the metasurfaces is the highdegrees of freedom in geometric inhomogeneity. However, the high-dimensional design space posessigniﬁcant challenges for the structural design.
Integrating deep learning framework [2] with Maxwell solver is an effective method for the photonic
design of the high-dimension metasurfaces. Intensive research has been done in this ﬁeld. However,there are still two main issues. The ﬁrst issue is the discretization of the design space. The design
space is usually formulated into discretized unit pixels or voxels with ﬁxed geometries so that they
can be used as input (labels) for neural networks. This would degenerate the search space into a
slice of the free-form continuous design space. The second issue is the lack of underlying physicalmechanisms in deep neural networks. The generative neural network often produces fancy but unrealistic patterns which are not manufacturable. It is urgent to develop physics-informed frameworkfor photonic design in continuous space.
Fourth Workshop on Machine Learning and the Physical Sciences (NeurIPS 2021).

Herein we realize a Deep neural network-based Bayesian Optimization (DeepBO) framework todesign a thermal emitter in the continuous high-dimensional space to modulate the electromagnetic
radiation for thermophotovoltaic (TPV) energy conversion. More importantly, through primary component analysis (PCA), we identify the underlying physics of the spectral characteristic control ofthe complex metasurfaces.

2The DeepBO Framework

Figure 1: Design pipeline of the DeepBO. (A) The schematics of the thermal emitter and the 16dimensional feature space. (B) Framework of DeepBO. The deep neural network is connected toa Bayesian linear regression, which gives the predicted distribution of the TPV efﬁciency in thefeature space.
2.1

Design space

Our design of the thermal emitter, as illustrated in Figure 1A, is a three-layer periodic grating structure transferred from 3-d to 2-d and described by the feature vector [h, d, a1 , b1 , a2 , ...], which lists
all the parameters of each possible candidate in a 2-d unit cell. This simpliﬁes a three-dimensionalcase to a two-dimensional one by using the symmetries of two orthogonal polarizations. Parameterai (i = 1, 2, ..., 7), ranging from 0 µm to 0.5 µm continuously, denotes the width of each tungsten
strip, while bi (i = 1, 2, ..., 7) with the same range is the length of the gap separating the neighboringtungsten strips, forming the ﬁrst layer: a superposition of multi-fold periodic tungsten gratings, andthe upper limit of the superposition is 7. The height of the tungsten strip is denoted by h, whichranges from 0 µm to 0.3 µm continuously. Parameter d denotes the thickness of SiO2 layer ranging
from 0 µm to 3 µm continuously, which composes the second layer under the grating structure. The
bottom layer is an opaque tungsten substrate. If we regard 1 nm as the discrete unit of the ranges, adesign space with 50014 ∗ 300 ∗ 2000 ≈ 1043 candidates in total is formed.
2.2

Pipeline

The ﬁgure of merit (FOM) is deﬁned as the TPV efﬁciency:∫ λband gapbb(λ)ϵ(λ)λ/λband gap dλ∫∞,
η= 0bb(λ)dλ0
where λband gap is the wavelength corresponding to the band gap of the PV, ϵ(λ) is the emissivity ofthe design at the wavelength λ, and bb(λ) is the intensity of the blackbody radiation at the working
2temperature of the TPV. In our case, λband gap = 2.254 nm, ϵ(λ) is calculated by FDTD simulation,and the working temperature is 1673K.
The DeepBO framework (Figure 1B) combines a deep neural network (NN) with the Bayesian linear
regression (BLR) as the surrogate model. The NN, which takes the feature vector [a1 , b1 , a2 , ..., h, d]of a design as input and the predicted TPV efﬁciency as output, is trained by a randomly probed
set as a priori. The BLR performs a probabilistic linear regression based on the trained NN, andpredicts a posterior distribution of the efﬁciency to the design space [3]. The POI algorithm is usedto balance exploration (the highest uncertainty) and exploitation (the highest predicted value). We
probe the predicted optimal design using FDTD Maxwell solver. The probed mapping between thedesign feature vector and FOM is appended to the training set a priori. The procedure loops until asatisﬁed design is eventually found.

3

ResultFigure 2A shows the history of the maximum efﬁciency with respect to the number of optimization
loops. The spectral efﬁciency of the optimal structure is 0.69, which, to our best knowledge, is thehighest spectral efﬁciency for the TPV emitters [4] [5]. The whole optimization process takes ca.
50 hours on a moderate desktop workstation with 56-Core 3.2 GHz CPU and 128 GB memory. We
terminate the optimization process after 1400 loops as the maximum efﬁciency plateaus.
Figure 2B shows the comparison between the asymptotic time costs of DeepBO and Gaussian process (GP)-based BO from sklearn. It can be clearly observed that the time for each loop in DeepBO
increases linearly with the training loop, while the time for each loop in GP-based BO increases
cubically as GP has a computational complexity up to O(n3 ), where n is the number of known datapoints. The surrogate model in DeepBO consists of NN and BLR. The NN is used to perform the regression task, then the BLR is applied on the last hidden layer of the network to obtain a distributionof the regression result. As a reduced version of GP, the BLR is much less expressive than GP.
Moreover, the NN-BLR surrogate model in DeepBO offers higher accuracy and efﬁcacy than GP.
Figure 2C and 2D show the predicted mean value and distribution given by GP and NN-BLR along
SiO2 thickness, from the training set (Figure 2C) and the ﬁrst 5 probed points (Figure 2D). Thewell trained NN in DeepBO ﬁts the data more accurately, and even in the worst case the regression accuracy of NN in DeepBO will not lose out to GP. Compared with conventional deep neuralnetwork-based surrogate models, NN-BLR outputs the distribution of the regression results rather
than a ﬁxed value, which could provide more possibilities for exploration when sampling. The
speed, accuracy and efﬁcacy of NN-BLR makes DeepBO a powerful framwork for data-intensive
geometric design.

Figure 2: The performance of DeepBO. (A) Optimization history of the TPV efﬁciency. (B) Timeper loop for DeepBO and GP-based BO. (C) Regression comparison of the trained surrogate modelin GP-based BO and DeepBO. (D) The ﬁrst ﬁve predicted probing points and their distribution given
by GP-based BO and DeepBO.
3

4

Underlying Physical Mechanism

We further perform principal component analysis (PCA) on the 16-dimensional design space to reveal the physical mechanisms of the optimal structure which holds the record of the highest spectral
efﬁciency of TPV emitters so far.

Figure 3: The PCA results on the search data set.
Since our model is periodic, a cyclic permutation on {ai } and {bi } gives exactly the same design.
To make sure the parameters in PCA are physically meaningful, the feature vectors are cyclicallypermuted such that the largest ai is in the position of a1 before the PCA, while the positions of h, dremain the same. The regulated feature vector is denoted by [h, d, a′1 , b′1 , a′2 , . . .].
To explain the dynamic learning procedure, PCA is performed on the search set ﬁltered by different
thresholds of efﬁciency, as shown in Figure 3. The thresholds are: (A) efﬁciency > 0.50, 94% oftraining set, (B) efﬁciency > 0.60, 61% of training set, (C) efﬁciency > 0.66, 30% of training set.
The ﬁrst column in Figure 3 shows the projection of the ﬁltered sets on the plane spanned by the ﬁrst
(x-axis) and second (y-axis) principal components, while the second column shows the projection
on the plane spanned by the the ﬁrst (x-axis) and least (y-axis) components. The third column shows
the PCA components of each chosen data set, each row symbolizes a PCA component, the i-th row
corresponding to the i-th principal component.
When the PCA is performed on the whole search set (Figure 3A), the data points distribute in a
broad range, which demonstrates sufﬁcient exploration over the entire design space. According to
the range set of the parameters, if the data set is chosen uniformly, the variance in the d direction
will be the largest, while the variance in a′1 and b′1 should be the least. The result of PCA shows
the component with the largest variance is the direction of d, while the components with the leasttwo variances is the direction of a′1 and h. The order of variances calculated by the range setting
agrees with the order of the principal components. The data set is composed with a wide rangeof low-efﬁciency points and a cluster of high-efﬁciency points, demonstrating a balance between
exploration and exploitation.
When the efﬁciency threshold is higher, the variance in the d direction becomes lower. Meanwhile,
the variance along a′1 succeeds h to be the least component (Figure 3B). When the efﬁciency threshold is further increased to 0.66 (Figure 3C), the least three components are a′1 , h and d, i.e., thedistribution of high-efﬁciency points along this directions is narrow, indicating a′1 , h and d are key
parameters of our model. Therefore, the performance of the TPV emitter is the most sensitive to thewidth and height of the largest tungsten strip, which corresponds to the surface plasmon polaritons,
as well as the thickness of the SiO2 layer, which corresponds to magnetic polaritons.
45Conclusion

DeepBO innovatively combines the two state-of-the-art tools of black-box function ﬁtting and optimization together, to achieve both accuracy and calculation efﬁciency in ﬁnding an optimal design ofcomplicated material systems. The most suitable situation to use DeepBO is when the design spaceis extremely data-intensive. In this work, DeepBO proves its power by achieving a record design of
TPV emitter, providing a new way of thinking to the global optimization of physical devices.

6Acknowledgement

This work was ﬁnancially supported by the Zhiyuan Scholar Program of Shanghai Jiao Tong University (SJTU) and Shanghai Pujiang Program. This work made use of shared facilities of the Zhiyuan
Innovation Research Center (ZIRC) at SJTU.

References[1] Wei Ting Chen, Alexander Y. Zhu, and Federico Capasso. Flat optics with dispersion-engineered
metasurfaces. Nature Reviews Materials, 5:604–620, 2020.
[2] Wei Ma, Zhaocheng Liu, Zhaxylyk A. Kudyshev, Alexandra Boltasseva, Wenshan Cai, andYongmin Liu. Deep learning for the design of photonic structures. Nature Photonics, 15:77–90,2021.
[3] Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram,
Mostofa Patwary, Mr Prabhat, and Ryan Adams. Scalable bayesian optimization using deepneural networks. In International Conference on Machine Learning, pages 2171–2180. PMLR,2015.
[4] Tobias Burger, Caroline Sempere, Bosun Roy-Layinde, and Andrej Lenert. Present efﬁciencies
and future opportunities in thermophotovoltaics. Joule, 4(8):1660–1680, 8 2020.
[5] Reyu Sakakibara, Veronika Stelmakh, Walker R. Chan, Michael Ghebrebrhan, John D.
Joannopoulos, Marin Soljacic, and Ivan elanovi. Practical emitters for thermophotovoltaics:
a review. Journal of Photonics for Energy, 9(3):32713, 2019.

5