SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In addition to optical properties, our frameworks are also equipped to simulate and provide data on electric and magnetic fields (i.e., E- and H-fields), across various positions and time steps within a simulation cell. Figures 5 and 7 provide the visual representation of these fields, clearly showing how they are influenced and altered by the presence of nanophotonic structures.
(a) Anti-reflective nanocones
(b) Close-packed nanospheres
(c) Double-sided nanocones
Figure 6: Visualization of the target properties of anti-reflective nanocones, close-packed nanospheres, and three-layer film with nanocones, made of fused silica,  $\mathrm{cSi / TiO_2}$ , and  $\mathrm{TiO_2 / Ag / TiO_2 / TiO_2 / TiO_2}$  respectively. For the film with nanocones,  $t_2 = 3\mathrm{nm}$ ,  $r_1 = r_2 = 20\mathrm{nm}$ ,  $h_1 = h_2 = 50\mathrm{nm}$ .
(a) Three-layer
Figure 7: Examples of E-fields (left of each panel) and H-fields (right of each panel) out of plane for the structures studied in this work. The results for the combinatorial system are shown in Figure 5.
(b) Nanocones
(c) Nanowires
(d) Nanospheres
(e) Double-sided
4 Datasets and Benchmarks for Nanophotonic Structures and Their Designs
Following Section 3, we present our datasets and benchmarks defined with several aforementioned nanophotonic structures and their design problems in this section.
Our frameworks provide datasets and processes for generating datasets, tailored for nanophotonic structures and their associated parametric design simulations. To generate these datasets, it is necessary to define feasible search spaces for parametric structure designs. Building on Table 1, Table 2 presents the number of parameters associated with each structure, the increment value for each structure, and the number of possible configurations conducted for each structure.
Our frameworks provide datasets of two primary modalities:
- Reflection, absorption, and transmission spectra, accompanied by fidelity information: This includes measurements of the three key properties across various structural configurations, considering fidelity information. The fidelity level, which can be low, medium, or high, is determined by the simulation resolution as defined by the spacing of the Yee grid.
- Electric and magnetic fields: These E- and H-fields are measured across different positions and simulation time steps as a light source emits a Gaussian-pulse light until it decays sufficiently. The resolution of the simulation and the size of the simulation cell determine the number of grid points used for these positional and temporal measurements.
By leveraging the diverse components within our datasets, users can explore and test a variety of tasks, ranging from multi-fidelity and multi-objective optimization to direct property predictions. In
Table 2: Details of our discretization of the search spaces for the datasets and discretized search space optimization mode. Elaborate description on the number of configurations can be found in Section G.
Structure	#Parameters	Increment (nm)	#Configurations
Three-layer film	3	1	149,058
Anti-reflective nanocones	2	1	43,800
Vertical nanowires	3	1	39,200
Close-packed nanospheres	2	1	57,491
Three-layer film with double-sided nanocones	7	5	1,920,996
Combinatorial system with material blocks	-	-	\( 12^{80} \) or \( 12^{1600} \)
particular, our datasets can serve as a valuable testbed for evaluating a variety of machine learning models, such as training them to predict optical spectra or electromagnetic fields, based on material properties and geometric configurations.
To provide readily accessible datasets for practitioners, we select some material combinations from Table 1 and then generate datasets. Specifically, for the three-layer films with and without double-sided nanocones, we assume that the top and bottom layers, and potentially the top and bottom nanocones are made of the identical material. Due to the vast number of possible configurations for the combinatorial system with material blocks, we do not create datasets for this structure, but instead provide a specific feature to work with this structure; see Section 4.2 for further details.
4.2 Optimization Modes
Our benchmarks support two modes for fast prototyping of optimization methods as follows.
Discretized Search Space Mode. In this mode, a search space is discretized with the increment specified in Table 2. At each configuration a simulation is run and the outcome of the simulation is recorded. Access to these simulation results is available online.
Surrogate Model Mode. Based on the dataset collected from the discretized search space, we fit a surrogate model. This allows us to evaluate any structural configuration in the search space using the trained surrogate model. All the specifics of this process are detailed in Section J. It is crucial to note that the predictions made by the surrogate model are not actual evaluations and might contain errors. A comprehensive discussion of these limitations and their implications can be found in Section N.
Note that we currently support these two modes only for the selection of some material combinations. Also, we do not support these modes for the combinatorial system with material blocks because the search space of this system is extremely huge and therefore it is difficult to obtain its datasets and build an accurate surrogate model.
Simulation Mode. Users can also directly run these simulations using our frameworks based on Meep and perform an optimization algorithm by conducting a simulation at every iteration. This can be more time consuming to execute but is preferred if the user has sufficient compute available. This allows users to carry out an optimization procedure on the complete space specified in Table 1 including exploring different material choices.
4.3 Experiments with the Surrogate Model Mode
In this section, we evaluate the performance of various optimization methods applied to parametric structure optimization. We consider a range of approaches including random search, Powell's method [49], Py-BOBYQA [10], DIRECT [30], differential evolution [64], and Bayesian optimization [16]. For the experiments shown in Figure 8, we specifically focus on assessing the effectiveness of our benchmarks in the surrogate model mode that is defined on a continuous search space. Most of the algorithms are implemented using SciPy [69] and NumPy [25] and some algorithms are executed using the open-source versions of them; see Section K for their missing details. Each experiment is independently repeated 50 times, with consistent random seeds maintained across all methods. The detailed description of experimental setup and implementation can be found in Sections J and K.
(a) Three-layer film,  $\mathrm{TiO_2 / Ag / TiO_2}$
(d) CP nanospheres, cSi/TiO $_2$
Figure 8: Results of experiments on nanophotonic structure optimization. Each experiment is conducted 50 times and the mean and standard error are depicted. AR nanocones, CP nanospheres, DS cones indicate the anti-reflective nanocones, close-packed nanospheres, and three-layer film with double-sided nanocones, respectively. Also,  $\mathrm{TiO_2 / Ag / TiO_2 / - / - }$  stands for  $\mathrm{TiO_2 / Ag / TiO_2 / TiO_2 / TiO_2}$ .
(b) AR nanocones, fused silica
(c) Vertical nanowires, cSi
(e) DS cones,  $\mathrm{TiO_2 / Ag / TiO_2 / - / - }$
The experimental results presented in Figure 8 show that global optimization methods such as DIRECT, differential evolution, and Bayesian optimization generally perform well across different structures. DIRECT, in particular, consistently outperforms the other techniques. Bayesian optimization and differential evolution also exhibit strong performance, although they can be slower than or comparable to DIRECT. However, in the case of the three-layer film with nanocones, we find that DIRECT tends to be slower than both Bayesian optimization and differential evolution; see Figure 8e. Conversely, local optimization methods such as Powell's method and Py-BOBYQA generally take longer to converge and are prone to getting stuck in local optima. While Powell's method converges to the same solutions as DIRECT in some cases such as Figures 8b, 8c, and 8d, both local search methods struggle to escape local optima in the other instances. This is the expected behavior of local optimization methods. Interestingly, random search, despite its simplicity, demonstrates effectiveness and even outperforms the local optimization methods in particular cases. However, its performance drops in the higher-dimensional problem; see Figure 8e.
On the other hand, for the combinatorial system with material blocks, we conduct experiments with random search using the simulation mode to evaluate configurations directly. The results and the analysis on these results are described in Section L.
In this paper we introduced several nanophotonic structures such as the three-layer film, anti-reflective nanocones, vertical nanowires, close-packed nanospheres, three-layer film with double-sided nanocones, and combinatorial system with material blocks for real-world applications. To construct our datasets and benchmarks, we devised a generic simulation scheme and pipeline for nanophotonic structure and parametric design simulations. Finally, the datasets and benchmarks are proposed by modeling, simulating, and optimizing nanophotonic structures. Furthermore, the future directions, limitations, and societal impacts of our work are discussed in Sections M, N, and O.
This research was partly funded by the National Science Foundation (NSF) under grant ECCS 1552712. We also received support from the MDS-Rely Center, which is funded by the NSF's Industry-University Cooperative Research Center (IUCRC) program through awards EEC-2052662 and EEC-2052776. OH acknowledges support from the National Science Foundation and United States-Israel Binational Science Foundation (NSF-BSF) program under NSF grant 2239527 and from Air Force Office of Scientific Research (AFOSR) grant FA9550-23-1-0242. Additionally, the University of Pittsburgh Center for Research Computing provided essential resources for this work, particularly the H2P cluster. This cluster operates with support from the NSF award OAC-2117681.
[1] O. Aguilar, S. de Castro, M. P. F. Godoy, and M. R. S. Dias. Optoelectronic characterization of  $\mathrm{Zn_{1 - x}Cd_xO}$  thin films as an alternative to photonic crystals in organic solar cells. Optical Materials Express, 9(9):3638-3648, 2019.
[2] ASTM International. Tables for reference solar spectral irradiances: Direct normal and hemispherical on 37 tilted surface. https://doi.org/10.1520/g0173-03r20, 2020.
[3] S. Belakaria, A. Deshwal, and J. R. Doppa. Max-value entropy search for multi-objective Bayesian optimization. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, pages 7825-7835, Vancouver, British Columbia, Canada, 2019.
[4] S. Belakaria, A. Deshwal, and J. R. Doppa. Multi-fidelity multi-objective Bayesian optimization: An output space entropy search approach. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 10035-10043, New York, New York, USA, 2020.
[5] J.-P. Berenger. A perfectly matched layer for the absorption of electromagnetic waves. Journal of Computational Physics, 114(2):185-200, 1994.
[6] J. Billard. Contribution à l'étude de la propagation des ondes électromagnétiques planes dans certains millieux matériels. PhD thesis, Centre national de la recherche scientifique, 1967.
[7] M. Born and E. Wolf. Principles of optics: electromagnetic theory of propagation, interference and diffraction of light. Elsevier, 2013.
[8] N. Brown, M. Fiscato, M. H. S. Segler, and A. C. Vaucher. GuacaMol: benchmarking models for de novo molecular design. Journal of Chemical Information and Modeling, 59(3):1096-1108, 2019.
[9] D. M. Callahan, J. N. Munday, and H. A. Atwater. Solar cell light trapping beyond the ray optic limit. Nano Letters, 12(1):214-218, 2012.
[10] C. Cartis, J. Fiala, B. Marteau, and L. Roberts. Improving the flexibility and robustness of model-based derivative-free optimization solvers. ACM Transactions on Mathematical Software, 45(3):1-41, 2019.
[11] Y. Deng, J. Dong, S. Ren, O. Khatib, M. Soltani, V. Tarokh, W. Padilla, and J. Malof. Benchmarking data-driven surrogate simulators for artificial electromagnetic materials. In Advances in Neural Information Processing Systems (NeurIPS), Virtual, 2021. Datasets and Benchmarks Track.
[12] X. Dong and Y. Yang. NAS-Bench-201: Extending the scope of reproducible neural architecture search. In Proceedings of the International Conference on Learning Representations (ICLR), New Orleans, Louisiana, USA, 2019.
[13] X. Dong, L. Liu, K. Musial, and B. Gabrys. NATS-Bench: Benchmarking NAS algorithms for architecture topology and size. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3634-3646, 2021.
A Electromagnetic Wave of Light
Figure S1: Electromagnetic wave of light. Magenta and blue waves correspond to electric and magnetic fields, respectively.
B Reflection, Absorption, and Transmission of Light
Figure S1 shows the electromagnetic wave of light, which is composed of electric and magnetic fields. The oscillations of these two fields are perpendicular to each other and the direction of wave propagation.
Figure S2: Light interacting with some material.
Figure S2 depicts the properties of light, where light interacts with some material. As in (2), the sum of reflectance, absorbance, and transmittance is one.
C Solar Reflection
Solar reflection  $R_{\mathrm{solar}}$  over  $d$  parameters  $x_{1}, x_{2}, \ldots, x_{d}$  is defined as the following:
$$
R _ {\text {s o l a r}} \left(x _ {1}, x _ {2}, \dots , x _ {d}\right) = \frac {\int b _ {s} (\lambda) R \left(x _ {1} , x _ {2} , \dots , x _ {d} ; \lambda\right) \mathrm {d} \lambda}{\int b _ {s} (\lambda) \mathrm {d} \lambda}, \tag {S1}
$$
where  $R(x_{1},x_{2},\ldots ,x_{d};\lambda)$  is a reflection function of wavelength  $\lambda$  and  $b_{s}(\lambda)$  is the photon flux density of the AM1.5 spectrum [2] at wavelength  $\lambda$ .
D Solar Absorption and Ultimate Efficiency
Solar absorption  $A_{\mathrm{solar}}$  and ultimate efficiency  $\eta_{\mathrm{ue}}$  over  $d$  parameters  $x_{1}, x_{2}, \ldots, x_{d}$  are defined as the following:
$$
A _ {\text {s o l a r}} \left(x _ {1}, x _ {2}, \dots , x _ {d}\right) = \frac {\int_ {E _ {g}} ^ {\infty} b _ {s} (E) A \left(x _ {1} , x _ {2} , \dots , x _ {d} ; E\right) \mathrm {d} E}{\int_ {0} ^ {\infty} b _ {s} (E) \mathrm {d} E}, \tag {S2}
$$
$$
\eta_ {\mathrm {u e}} \left(x _ {1}, x _ {2}, \dots , x _ {d}\right) = \frac {\int_ {E _ {g}} ^ {\infty} I (E) A \left(x _ {1} , x _ {2} , \dots , x _ {d} ; E\right) \frac {E _ {g}}{E} \mathrm {d} E}{\int_ {0} ^ {\infty} I (E) \mathrm {d} E}, \tag {S3}
$$
where  $b_{s}(E)$  is a photon flux density,  $I(E)$  is the irradiance, and  $E_{g}$  is the materials' energy band gap. Note that ultimate efficiency assumes that the temperature of the solar cell is  $0\mathrm{K}$ , and  $\int_0^\infty I(E)\mathrm{d}E = 1,000\mathrm{W / m^2}$  for the AM1.5 global spectrum [2]. Each material can only absorb light with energy above its band gap  $E_{g}$  so the integration in the numerator is only for photons with energy greater than  $E_{g}$ .  $E_{g} = 1.12\mathrm{eV}$  ( $\lambda = 1107\mathrm{nm}$ ), for crystalline silicon (cSi),  $E_{g} = 1.43\mathrm{eV}$  ( $\lambda = 867\mathrm{nm}$ ) for gallium arsenide (GaAs), and  $E_{g} = 1.51\mathrm{eV}$  ( $\lambda = 821\mathrm{nm}$ ) for a perovskite structure of methylammonium lead iodide ( $\mathrm{CH}_3\mathrm{NH}_3\mathrm{PbI}_3$ ).
Consequently, maximizing  $A_{\mathrm{solar}}$  is equivalent to maximizing  $\eta_{\mathrm{ue}}$ :
$$
\max  A _ {\text {s o l a r}} \left(x _ {1}, x _ {2}, \dots , x _ {d}\right) = \max  \eta_ {\text {u e}} \left(x _ {1}, x _ {2}, \dots , x _ {d}\right). \tag {S4}
$$
E Material Properties
(a)  $\mathrm{Ag}$
(b) Au
(c) AZO
(f) Cu
(d) cSi
(e)  $\mathrm{CH}_3\mathrm{NH}_3\mathrm{PbI}_3$
(i) ITO
Figure S3: Relative permittivities of various materials versus different wavelengths. We utilize the raw values of complex permittivity and employ a gradient-based optimization technique to fit them to the Drude-Lorentz susceptibility model. The estimated complex permittivities presented here cover the solar spectrum range from  $280\mathrm{nm}$  to  $2500\mathrm{nm}$ .
(j) Ni
(g) Fused silica
(k)  $\mathrm{TiO}_2$
(h) GaAs
(1) ZnO
In Figure S3, we report relative permittivities across different wavelengths for the materials employed in this work. The range of the solar spectrum we focus on spans from  $280\mathrm{nm}$  to  $2500\mathrm{nm}$ . To derive permittivity values throughout this range, we apply a gradient-based optimization method to fit the raw complex permittivity data to the Drude-Lorentz susceptibility model, following the methodology outlined in the Meep documentation [44].
For AZO, we combine two different sets of raw permittivity data to create a comprehensive dataset. This approach aims to enhance the accuracy of the fitted model across the entire solar spectrum. The effectiveness of this approach and the results of the fitted model are illustrated in Figure S3c.
With regard to fused silica, a minor offset is intentionally introduced to the optical extinction coefficient of its complex refractive index. This adjustment is necessary to stabilize the simulations,
especially those at low and medium fidelity levels, which are otherwise susceptible to divergence or failure. In this research, we choose an offset value of 0.04, and the associated results and adjustments are detailed in Figure S3g.
The materials examined in this paper include silver (Ag) [51], gold (Au) [51], aluminum-doped zinc oxide (AZO) [67, 58], crystalline silicon (cSi) [55], a perovskite structure of methylammonium lead iodide  $\mathrm{(CH_3NH_3PbI_3)}$  [47], copper (Cu) [51], fused silica [41], gallium arsenide (GaAs) [50], indium tin oxide (ITO) [38], nickel (Ni) [51], titanium dioxide  $\mathrm{(TiO_2)}$  [61], and zinc oxide  $\mathrm{(ZnO)}$  [1]. Note that the respective references of the materials mentioned above indicate the sources of the raw data of the relative permittivities.
F Details of Simulation Cell Sizes
Table S1: Details of the simulation cell sizes declared for our simulations. A unit depth  $m$  determines the spacing of geometries. All values are in nanometers. For simplicity, denote that  $t^{\dagger} = t_{2} + \max(2t_{1}, 2t_{3}) + \max(2h_{1}, 2h_{2})$ .
Structure	Two-Dimensional Structure	Three-Dimensional Structure
Three-layer film	(10, t2 + max(2t1, 2t3) + 6m)	(10, 10, t2 + max(2t1, 2t3) + 6m)
Anti-reflective nanocones	(2r, 2h + 6m)	(2r, 2r, 2h + 6m)
Vertical nanowires	(p, h + 6m)	(p, p, h + 6m)
Close-packed nanospheres	(2r, max(4r, 2t) + 6m)	(2r, 2r√3, max(4r, 2t) + 6m)
Three-layer film with nanocones	(max(2r1, 2r2), t† + 6m)	(max(2r1, 2r2), max(2r1, 2r2), t† + 6m)
Combinatorial system	(p, t + 6m)	(p, p, t + 6m)
Table S1 represents the sizes of the simulation cells employed for nanophotonic structures. A unit depth  $m$  determines spacing between geometries such as the light source, transmission monitor, reflection monitor, and photonic structures. More precisely, supposing that we define the depth of PML as  $m$ , distance between the light source and the reflection monitor is set as  $m$  and the size of the simulation cell is also dependent on  $m$ ; see Table S1 for that dependency. Unless otherwise noted,  $m = 50 \mathrm{~nm}$  for our simulations.
G Details of Datasets
The number of possible configurations shown in Table 2 is determined by considering the search space presented in Table 1 and increment for the corresponding structure. For example, suppose that we are given two parameters where the first parameter can vary between 3 and 5, and the second between 10 and 14, increasing in steps of 1. In this example, the first parameter has three possible values \{3, 4, 5\}, and the second has five values \{10, 11, 12, 13, 14\}. This results in a total of 15 unique configurations that can be derived from these parameters.
H Additional Dataset Visualization
Along with the visualizations presented in Section 3.4, additional examples pertaining to various nanophotonic structures are provided in this section. For the three-layer film structure, we display simulation results with different material compositions in Figures S5 through S12. Specifically, these illustrate variations in the nanophotonic structure when different materials are used in its composition. Next, we present simulation results for the anti-reflective nanocones in Figure S13, showcasing variations in this particular structure type. This is followed by simulation results for the vertical nanowires, exhibited in Figure S14, and the close-packed nanospheres, shown in Figure S15. Furthermore, we provide the visualization of simulations results for the three-layer film with double-sided nanocones in Figures S16 through S23. Similar to other visualization, these demonstrate how the results change with different material compositions. It is worth noting that for nanophotonic structures characterized by more than two parameters, we vary two of the parameters while keeping the others constant. This approach allows us to effectively represent their parameter spaces in two-dimensional plots.
I Results on Elapsed Time
Table S2: Results on elapsed time for diverse nanophotonic structures and fidelity levels. Mean and standard deviation are reported. All values are in seconds.
Structure	Low Fidelity	Medium Fidelity	High Fidelity
Three-layer film, TiO2/Ag/TiO2	0.5927 ± 0.0810	1.1277 ± 0.1373	54.4757 ± 23.1198
Anti-reflective nanocones, fused silica	0.5999 ± 0.1357	0.6762 ± 0.1583	60.6909 ± 54.9886
Vertical nanowires, cSi	1.0668 ± 0.2669	1.5037 ± 0.4536	377.0777 ± 215.7626
Close-packed nanospheres, cSi/TiO2	1.5695 ± 3.2443	2.3399 ± 4.8280	673.3744 ± 848.0346
Film with nanocones, TiO2/Ag/TiO2/TiO2/TiO2	2.2194 ± 0.7021	2.3046 ± 0.7497	115.0109 ± 57.6097
We demonstrate results on elapsed time for the nanophotonic structures we study and three fidelity levels in Table S2. Since we run a large number of simulations through a job scheduler, i.e., the Slurm workload manager, the respective simulations can be run on distinct machines; see Section K. Although we assign the same number of threads for each job, the time elapsed for simulations might vary depending on the status of machines. Nevertheless, assuming that all simulation jobs are fairly distributed to the machines and machine states are also similar on average, the results on elapsed time are provided for analysis. As expected, simulations with low fidelity is faster than ones with medium fidelity and high fidelity, and simulations with high fidelity is slower than the others. Moreover, the time differences between low fidelity and medium fidelity are marginal compared to the differences between medium fidelity and high fidelity or between low fidelity and high fidelity.
J Details of Regression Models
To establish a regression model for predicting a target optical property, we begin by partitioning our dataset into subsets designated for training, validation, and testing. Specifically, these disjoint subsets are  $70\%$ ,  $10\%$ , and  $20\%$  of the entire dataset, respectively.
The regression model for the surrogate model mode is a multi-layer perceptron, detailed as follows:
First layer: (the number of parameters, 128, batch normalization, ReLU);
Second layer: (128, 64, batch normalization, ReLU);
Third layer: (64, 1, -, Logistic),