NeurOLight: A Physics-Agnostic Neural Operator Enabling Parametric Photonic Device Simulation
Authors: Jiaqi Gu
Zhengqi Gao
Chenghao Feng
Hanqing Zhu
Ray Chen
Duane S Boning
David Z. Pan
Venue: NeurIPS 2022 Accept (2022)
Keywords: Neural operator
optical simulation
machine learning for design automation

Abstract: Optical computing has become emerging technology in next-generation efficient artificial intelligence (AI) due to its ultra-high speed and efficiency. Electromagnetic field simulation is critical to the design, optimization, and validation of photonic devices and circuits.
However, costly numerical simulation significantly hinders the scalability and turn-around time in the photonic circuit design loop. Recently, physics-informed neural networks were proposed to predict the optical field solution of a single instance of a partial differential equation (PDE) with predefined parameters. Their complicated PDE formulation and lack of efficient parametrization mechanism limit their flexibility and generalization in practical simulation scenarios. In this work, for the first time, a physics-agnostic neural operator-based framework, dubbed NeurOLight, is proposed to learn a family of frequency-domain Maxwell PDEs for ultra-fast parametric photonic device simulation. Specifically, we discretize different devices into a unified domain, represent parametric PDEs with a compact wave prior, and encode the incident light via masked source modeling. We design our model to have parameter-efficient cross-shaped NeurOLight blocks and adopt superposition-based augmentation for data-efficient learning. With those synergistic approaches, NeurOLight demonstrates 2-orders-of-magnitude faster simulation speed than numerical solvers and outperforms prior NN-based models by ~54% lower prediction error using ~44% fewer parameters.

NeghOLight: A Physics-Agnostic Neural Operator Enabling Parametric Photonic Device Simulation
Jiaqi Gu', Zhengqi Gao?, Chenghao Feng', Hanqing Zhu', Ray T. Chen', Duane S. Boningg, David Z. Pan'
The University of Texas at Austin, MMassacuusetts Institute of Technology fjggu, fengchenghan 1996, hqzhu) @ utexas.edu, zhengqi mit.edu, moming@ml.mmrr.ede. [chen, dpan) @ ece. utexaa..eee.
Abstract
Optical computing is an emerging technology for next gextraenerttion efficient artifi- cial intelligence (AI) due to its ultra-high speed and efficiency. Electromagnetic field simulation is critical to the design, optimization, and validation of photonic devices and circuits. However, costly numerical simulation significantly hinders the scalability and turn-around time in the photonic circuit design loop. Recently, physics-informed neural networks have been proposed to predict the optical field solution of single instance of partial differential equation (PDE) with pre- defined parameters. Their complicated PDE formulation and lack of efficient parametrization mechanisms limit their flexibility and generalization in practical simulation scenarios. In this work, for the first time, a physics-agnostie neural -based framework, dubbed NeurOLight, is proposed to learn family of frequency-domain Maxwell PDEs for ultra-fast parametric photonic device simu- lation. We balance the efficiency and generalization of NeurOLight via several novel techniques. Specifically, we discretize different devices into a unified domain, represent parametric PDEs with compact wave prior, and encode the incident light via masked source modeling. We design our model with parameter-efficient cross-shaped NeurOLight blocks and adopt superposition--asse augmentation for data-efficient learning. With these synergistic approaches, NeurOLight generalizes a large space of unseen simulation settings, demonstrates corders-of-magninnn faster simulation speed than numerical solvers, and outperforms prior neural net- work models by ~54 lower prediction error with 444 fewer parameters. Our code is available at link.
Introduction
With recent advances in integrated photonics technology, optical deep learning represents a new paradigm in eexegeenerttinn efficient artificial intelligence (AI) [28, 27, An increasing number of co-design efforts have been made to enable synergistic light-A/ interaction. We see extensive developments for photonic AI with rapidly evolving optical neural network (ONN) hardware celerator designs [28, 8, 39, 5, 23, 29, 6] and various circuit-algorithm co-optimization methodolo- gies [8. 9, 7, 30, 10, 14]. However, applying AI for optics much less explored. Complementary prior Al-assisted architectural exploration [19, 11]. a natural question is whether Al can assist in the lower-level device simulation that requires deep understanding of the physical nature of optics.
Al-assisted photonic device simulation a critical step to closing the synergistic loop of light-Al interaction. Besides using standard devices that already have compact transfer matrix [28, 8, 32]. modern optical Al shows a trend to exploit customized photonic structures for scalable optical computing [7, 30, 40, 36). Unfortunately, because customized devices do not have analytical transfer
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

10
196x
W,
Solve PDE instance
FDFD Simulator: Solve Maxwell PDE (4, (4''Vxx) Mure
2 Slow runtime (>10
H(n w
NeurOLight H(r. 9,
Q J.
FOFD Simurator Hir o6, tr. Soive one
Runtime (o)
10-1
40x
Pirs (....
Probabilitt
Modeling
NenrOLight
A
Plysics-AAroodic Leaming
4
10
10
Simulator
10
( (,,,,,
NeurOLight Framework
(r(E(A))
Hir, (4, Conditiond food paramelers
Prior DNN
Domain Size (fm²)
G Parallel inference on parametric PDEs Fast runtime (<10 ms)
(a)
(b)
(c)
Figure (a) Compare FDFD simulation and our NeurOLight framework. (b) NeurOLight (1
Quadro RTX 6000 GPU) runs -200x faster than FDFD simulator (8-core 17-9700 CPUs) across different domain sizes (50 nm grid step). (c) Different methods cover different solution space.
functions, understanding their behavior heavily relies numerical simulators [16] to solve Maxwell partial differential equations (PDEs) to obtain the optical field distribution. Even solving a single dimensional (2-D) simulation instance 2 umn region can cost nearly on 8 CPUs, as shown Figure (a), which significantly hinders scalable circuit- simulation and optimization. Hence, target is to propose surrogate model that learns the light propagation principle and efficiently approximates the field solutions to new simulation instances while running orders- orders-of magniiid faster than the numerical solver, as shown in Figure 1(b).
Most prior work still uses conventional NNs predict several key properties based on a few design variables [31,35] which is an ad-hoe function approximator without learning the light propogation property. Several works attempt to leverage physics-informeet NNs (PINNs) [33, 2, 22] with physics- augmented residual loss to predict electromagnetic field solutions. However, previous methods have three major limitations. First, as illustrated in Figure I(c), they only model the field distribution conditioned pre-defined solving domains, input sources, and frequency. In other words, their models only learn the solution of a certain PDE instance with fixed parameters. Second, they all belong to the category of PINNs [25] that require an explicit description of the PDE as well as strict initial/boundary conditions. Constructing complicated Maxwell equation- based residual loss is no easier than re-implementing numerical solver [24, 20, 21J. Third, their CNN-based models show inadequate modeling capacity with too small receptive fields to learn important light propagation, scattering, and interference effects.
To learn a family of parametric Maxwell PDEs that models the joint probability of different design variables as shown in Figure I(c), we propose physics-agnostic light field prediction framework NeurOLight that consists joint PDE encoder and an efficient cross-shaped neural operator backbone. The main contributions of the work are as follows:
For the first time, an Al-based photonic device simulation framework is proposed to learn a family of parametric Maxwell PDEs for ultra-fast optical field prediction.
We propose a novel joint PDE encoder for compact PDE representation and an efficient cross- shaped Fourier neural operator backbone for end-to-end optical field prediction, over 2-order-of- magnitude faster than numerical simulators.
We propose a superposition-basse based mixup technique that dynamically boosts the data efficiency and generalization during the training of NeurOLight.
On different photonic device simulation benchmarks, our NLurOLight achieves the best prediction fidelity, generalizability, and transferability, outperforming UNet and -tate-of-theeart (SoTA) Fourier neural operators by average of 53.8% lower prediction error with 44.2% fewer parameters.
To our best knowledge, this is the first Al-based framework that can learn the terahertz light propagation inside photonic devices that generalizes to different domains. permittivities, input sources, and frequencies. We open-source our NeurOLight framework at link.
2 Related Work
Optical field simulation with machine learning. Finite difference frequency domain (FDFD) simulation is a widely adopted method to analyze silicon-photonic devices. Numerical simulators

---

are used to solve trequency-domicn Maxwell PDEs to obtain electromagnetic field distributions of an optical component with terahertz incident light sources. To accelerate this consuminn process, NNs have been utilized as surrogate models for fast optical simulation approximation. A multi-layer perceptron was used to map the design variables to scalar performance metric of a power splitter [31]. NNs were also utilized to predict intermediate values to accelerate the convergence of the numerical solver [35]. MaxwellNet [22] was proposed to train a UNet with physics-informed loss to predict the scattered field based on the material permittivity of the free-space lens. WaveYNe: [2] also adopted a 2-D UNet as the model trained with both data-driven supervision loss and Maxwell- equation- -based physics-augmentt augmented loss to predict the optical field of silicon meta-lens, However, prior NN-based methods require explicit physics knowledge of the Maxwell PDEs and only learn a small field solution space conditioned on fixed parameters.
Learning PDEs via neural operators. Recently, neural operators have been proposed as new NN models that learn family of parametric PDEs in the infnite-dimensional function space in mesh-free and purely data-driven fashion. The Fourier neural operator (FNO) [21] approximates the nonlinear mapping from PDE observations to solutions through Fourier-domain kernel integral operations, achieving record-breaking performance and efficiency on wide range of challenging applications. Several variants have been proposed to improve the performance and efficiency of the original FNO models, e.g., Factorized FNO [34], U-FNO [37], and multiwavelet- based neural operator [12].
3 Neural operator-based optical field simulation framework NeurOLight
3.1 Understanding optical field simulation for photonic devices
Waveguides can confine the incident laser beam and allow the optical signals to propagate and interfere with each other. Various optical components, e.g., couplers, shifters, and multi-mode interference (MMI) devices [4], can create phase shifts, magnitude modulation, and interference, especially useful for optical communication and neuromorphic computing. Analyzing how light wave propagates through those components are critical to device optimization and photonic integrated circuit design. Given a linear isotropic optical component, we will shine time-harmonic continuous- wave light beam on its input ports and analyze the steady-state electromagnetic field distributions X E + EEy + 2E, and H xH, HH + 2H, in it, each of which includes horizontal (x), vertical (y), and longitudinal (2) components. We can solve the steady-state optical field E(r) and Hrr) from the rrequency-domain curl-of-curl Maxwell PDE under absorptive boundary conditions [16] (details in Appendix A)),
E
((H5 ((V'Vx x Vx) - w² EnE, (()))(r) = (r) (V x (E, (F) w² paca) H(())))(rrr))))))))( = (r) (I)
where Vx is the curl operator of vector function, Mo is the vacuum magnetic permeability, and 6, are the vacuum and relative electric permittivity, and Jmm and J. are the magnetic and electric current sources. FDFD simulation discretizes the continuous domain into an M x N mesh grid and solves the above linear equation AX- = b to obtain the fields. Detailed forms of and b can be found in [16]. Solving for these optical fields exactly with a sparse matrix A E CMNXMM is prohibitively expensive and not scalable to large photonic structures. fast surrogate model that predicts optical fields with high fidelity is of tremendous interest.
3.2 The proposed NeurOLight framework
As shown in Figure 2, our NeurOLight framework models the optical field simulation problem as an diinite-ddmennional--ppe
mapping from Maxwell PDE observations A CO to the optical field solution U e Caxd, Here, 92 is the continuous 2-D physical solving domain, Ω = (2) typically in units of micrometers (μm), where the photonic device-of-interss can be tightly located. A and lI take values with d, and d, dimensions, respectively. To learn the ground truth nonlinear mapping T² A - U, we construct NeurOLight with PDE encoder E that produces compact PDE representations, followed by an efficient neural operator- beesed approximator V that minimizes the empirical error on discrete PDE observable samples a ~ A.
= min Ea-A ([(V)( V (a))).
(2)
3

---

A = (0, u,
E A' (4, Hy. Pi, Ps) E CAAMAN
V
U Hy(T, w) E CA
(1) Unified Domain
(4) Source
Wave Prior
Recovered Fields
0
(M, N, Al,, Al.)
NeurOLight Backbone W
H
P
Stem
Head
M
PDE Encoder 8
NeurOLight Block
x2
(2) Permittivity C
(3) Frequency
Masked Fields
W
2re E
Hy
C
Figure 2: NeurOLight framework for optical field simulation. Real part is plotted for complex fields.
3.2.1 Scale-adaptive domain discretization: Ω Ω
To generalize to PDEs in different physical do- mains and support batched parallel inference, we adopt an M N discrete unified domain = (M, N, Alz; Alg) with an adaptive mesh granularity, i.e., with grid steps A L./M and Al2 Lz/N. As shown in Figure. 3. multiple photonic devices with different phys- (120 px)
(1) (4): Mfg.Al, 50,50 (nmm)
(420
22 um (440 pxt
(140
pm
(1)
5 jm (100 px)
pm
(3)
4.5 190 px)
pm (520 px)
ical dimensions are normalized to the same f2. Their original physical dimensions can be ele- gantly encoded into the re-calculated mesh gran- ularities. This unified discrete domain gives NeurOLight the flexibility to handle parallel inference on different physical domain dimen- sions, unlike prior work [22, 2] that requires time-consuming model retraining once the phys- ical domain changes.
18 pm (360 px)
Scale-Adaplive Domain Discretization
AI,, Al, -5.7,87.5 (am)
(2): bl, = =573.62.5 (mm)
AI, 46.9,75.0 (sm)
(4): Mr 67.7,66.3 (rm)
384
Figure 3: Scale-adaptive domain discretization en- ables generalization to different solving domain dimensions and efficient batched processing.
3.2.2 Joint PDE representation: A + A
After we define unified solving domain, we need to construct effective PDE representations that describe the raw observations A = (Ωt, w,, The relative permittivity distribution can be simply represented by E, CM However, how to compactly encode other parameters, i.e., (й² w, J). remains non-trivial challenge. Let us first consider what makes good representation. First, it needs to be compatible with the model input, i.e., can be fused with the 2-D image representation in compact way. Second, it is preferred to reveal the physical essence of the parameters and inject useful prior knowledge that helps model generalization. Based on the above considerations, we propose a PDE encoder E : + At that converts the raw observations to joint PDE representation At (c. Pu,
Encoding (5t, w) via wave prior. The intuition behind
Slow wave SIO, (e, 2.1) (e, 12.1)
wave
Wave propagation behavior changes with different 6
Diverse high-order patterns leamt stem
the wave prior design is that vacuum angular frequency -
2ne and electric permittiv-
P.
ity 6, together decide the phys- ical light wavelength inside the material, i.c., X' = //y. The mesh granularity determines how many pixels can depict a wave period along both directions, i.e., (X/ Alz, X/ Al.). Therefore, as
P.
Wave Prior from PDE Encoder
Figure Wave prior as joint PDE representations.
shown in Figure 4, we construct artificial wave patterns, named wave prior, as P: e 2 12 At. and P. e 2m/er s1 Als where 10 M 1) and
(0, -. NN 1). The wave

---

prior jointly translates the (1), c, pair to unified representation with strong prior knowledge, significantly reducing the learning difficulty on overly-abstract raw observations. We note that the NeurOLight stem learns complex combinations of the wave prior and generates diverse high-order wave patterns for later feature transformation.
light source (J) encoding. In the optical simulation, light source J will be injected by shining light on the input waveguides of the photonic devices as stimuli to the system. J will have a vacuum angular frequency w and a polarization mode. For example, in the transverse magnetic (TM) mode, we have H₃ H O. Thus we focus on the prediction of Hy. However, as shown in Figure 5, is a combination of multiple length-w w 1-D vectors, where w is the imput port width, placed at the input waveguides, which is hard to be encoded the image prediction flow. Therefore, we borrow the idea of masked image modeling [1] to light source encoding. In the source representation H', we only maintain the fields in the imput waveguides before entering the key region the device, which easy to obtain and irrelevant to the structure it enters into, and mask out all the fields after. In this way, the field prediction task translates to a masked field restoration task conditioned on the input light source hint.
Masked image modeling-inspiree
Eigen Mode of Light Source
Hy
Masked
Masked Light Source Field
Figure 5: Masked light source modeling.
field
3.3 Efficient NeurOLight model architecture: V
Convolutional stem. The proposed NeurOLight architecture starts with convolutional stem S (r()) + (r)) Yr E S2 that encodes each complex- valued observation sample a'(r E C4x MXNN to real-valued representation (r E RCx N Lightweight blueprint convolutions [13] are used to perform local wave pattern transformation with a low hardware cost.
Cross-shaped NeurOLight block. In the projected C--dimensional space, we place K cascaded NeurOLight blocks to gradually restore the complex light field in the frequency domain as vo(r) (r) - - (r). Each NeurOLight block is formulated as
(k+1(r) FFN (FCu)))))()) vk, Vr Ω (Kox)(r) (2) Vri e Q
(3)
where K. is learnable kernel integral transform, and FFN(-) is feedforward network. When the
kernel satisfies r<(ri, x(r: r2). the above integral kernel operator is equivalent to a spatial- domain 2-D convolution, which can be efficiently computed by using Fourier transform F(-) [21].
clear downside of the original 2-D FNO is the huge parameter cost, i.e., (rx)(r) E Ck. xxx,kxxxxX XCXC, and the resultant severe overfitting issues. To improve the model ef- ficiency and generalization simultaneously, we introduce a shoss- Fourier neu-
Corizonial-FNO-T
FFN
C/2
C
Co 2
sC
Ra
C Go
IFFT
RR tu
C12
ral operator, shown in Figure 6. The input feature is first bi-partitioned along the channel dimension into two chunks en (r) = [uā();; ; vo (r)], representing hori- zontal and vertical patterns, and 1-D FNO is applied to both directions,
Linear
Conv1x1
DWConv3x3
BN
GELU
Figure 6: NeurOLight backbone model design.
(K'h = F. (F.()))) ( (.())) = F. ( (() (F((((( . (F) (Fr Vo E
(4)
(Ke))(r) = ((c)))) (KC*
We parametrize the Fourier kernels as lightweight complex- valued tensors (2) C
and Rd (x) x These orthogonal 1-D kernel operations intrinsically perform spatial and channel-wise feature aggregation in an interleaved way that are able to provide global receptive fields achieve long-distance modeling. Compared with k,,k,, 2 parameters in the 2-D FNO, our cross-shaped NeurOLight block only has (h. ky +8
parameters.
5

---

To increase nonlinearity and enhance local information interaction, we append an FFN block after the cross-shaped FNO. Inspired by the MixFFN designs SoTA vision transformers [38]. our FFN expands the channels by times, performs local information aggregation via 3 depth-wise convolution (DWConv), activates using the GELU function, and projects it back to C' channels.
Projection head. At the end, two point-wise convolutional layers are used to project ux (r) to the light field space u(r) = (r)) Dropout layer is inserted mitigate overfitting issues.
Loss function. Even with normalized light source power, optical fields tend to have distinet statistics. To balance the optimization effort among different fields, adopt the normalized mean absolute error (N-MAE) as the objective C W (E))), V (a)) = (I) (EE(a) - V 1
3.4 Toward better data efficiency and generalization via superposition- based mixup
The PDE observations can cover huge design space. Hence, the data efficiency and generalization of pure data-driven models raise concern. Simply drawing large numbers of random training examples with all pos- sible light sources has an intractable data acquisition cost. Standard augmentation techniques are effective in improving data efficiency and generalization on tasks with natural images; however, their direct application not compatible with PDE simulation. Since V (a;) is a highly nonlinear function of a; and closely related to the boundary conditions, simultaneously augment- ing c,, Ω and H, e.g., via cropping, distortion, or resizing, leads to invalid field solutions.
Single-Souras Source Training Examples (same W Ery !t Sources
Target Fields
H
Hy
HG
H1,s
Hy = Σ: H.'
H
Σ, H
H, 1/ Σ. H.
Σ 7:2 IT
Interestingly, we notice that the photonic system satis- fies the superposition property w.r.t. the light source,
Augmented Training Examples with Superposition based Mixup
Data augmentation with superposition based mixup. Only real part plotted for each field.
Figure 7:
J
A V'' (H')
m w (H) (5) Σ
Based on this, only involve single-input simulation data in the training set and dynamically mix multiple (IJ)) input sources via Superposition- based Mixup, as shown in Figure 7,
H H
H H ) =T
H H ". Te FFe x(JI [Tyl2 = 1.5()) 0, Vi [J]] (6)
At each iteration, we randomly generate the mixup coefficient matrix I and make it have unit row-wise f2-norm for light field power normalization. The global phase of the complex-valued optical field is normalized by forcing the first input port always to have phase that equals 0. In this way, NeurOLight learns how multiple incident light sources interfere with one another. Once NeurOLight can generalize to arbitrary light sources, multi-source simulation only needs an efficiens one-shot inference with superposed source fields instead of explicitly accumulating [J] single-source inference results.
4 Results
4.1 Experiment setup
Datasets. We focus on widely applied multi-mode interference (MMI) photonic devices. We select MMIs with rectangular tunable control pads (Tunable MMI). The permittivity of the tuning region can be programmed by external signals, so this family of devices exemplifies photomic structures with reconfigurable transmissions [18]. We also evaluate MMIs with rectangular etched cavities (Etched MMI) [31], which exemplifies another popular category of passive sub- wub-wavelength photonic devices with fixed yet highly discrete permittivity distributions. We use open-source CPU-based 2-D FDFD simulator angler [16] to simulate optical fields for randomly generated MMIs as our dataset. Details on dataset generation is in Appendix A2,

---

4.2 Main results
In Table I, we compare four models: (1) UNet [22, 2]. (2) 5-layer FNO-2d [21], (3) 12-layer factorized FNO (F-FNO) [34], and (4) our NeurOLight. Detailed training settings and model architectures can be found in Appendix A3 and Appendix A4, respectively. On these benchmarks, NeurOLight outperforms UNet and prior SoTA FNO variants by 53.8% lower test error with 44.2% % fewer parameters on average.
Results on tunable MMI. On tunable MMI, NeurOLight achieves the best prediction error with only half the parameter cost. Figure 8 visualizes the field prediction for one test MMI. UNet is significantly limited by its small receptive field and lack of long-distance modeling capability, thus failing to predict the full field even with the hint of wave prior. As representative neural operators, FNO-2d and factorized FNO (F-FNO) manifest the superior advantages of the Fourier-domain kernel integral operations, showing considerably lower prediction errors than their CNN counterparts. However, given the parameter budget (~3 M), the 5-layer FNO-2d only has 10 modes in the x-direction and 32-modes in the z-direction, which may not enough extract high-frequency waves. The 12-layer F-FNO adopts factorized I-D Fourier kernel to save parameters; however, its modeling capability is limited by the lack of local feature extractors. Our NeurOLight blocks benefit from the global view of the cross- cross-shaped 1-D kernel and local feature aggregation from convolutional FFN blocks. In the training curves Figure 10(a), NeurOLight achieves the fastest convergence and best generalization among all models. We animate the real-time prediction process of NeurOLight in Appendix A5.
Table I: Comparison of parameter count, train error, and test error on two benchmarks among four different models.
Benchmarks
Model
#Params (M)
Train
Test Err |
UNet [22, 2]
3.47
0.776
0,801
FNO-2d [21]
3.29
0.231
0.244
Tunable MMI
F-FNO [34]
3.16
0.272
0,292
KeurOLight
1.58
0.145
0.122
UNet [22, 2]
3.47
0.779
0.792
FNO-2d [21]
3.29
0.601
0.648
Etehed MMI
F-FNO [34]
3.16
0.411
0.525
NeurOLight
2.11
0.387
Average Improvement
-44.2%
.49.1 %
-53.8%
LiNNe
FNNo226
Vg ==FNNN
Vg NouroLLight
Pa P
Wave Prior
P
Wave Prior
P. P.P.
Wave Prior
P,
Wave Prior
R((,()(()))
((("(a))
(+(()))) (e))
Figure 8: Visualization on one test tunable MMI. (Alz 83.1 nm, Ale 70.8 nm, À 1.54 μmm).
Results etched MMI. Compared with tunable MMIs, predicting the field in etched MMIs, even with 2x more training examples, much harder task given the complicated scattering at the cavity-silicon interface and the considerably larger and highly discrete design space, shown in Figure 9. Hence, we increase the model capacity of NeurOLigh by using 16 layers. Among all prediction models, NeurOLight achieves the best results with 42% lower error while still saving 36% parameters on average.
VA
UNet
F-FNO
V
NeurOLight Wave Prior
P.,
Wave Prior
P
Wave Prior
P P.
Wave Prior
P.PP. P.
x((v(E(e)))
09
([(*(4))
(B(o)) [a)]
Figure 9: Visualization on one test etched MMI. (Al, = 91.3 nm, Al_ = -99.1 nm, À = 1.55 pm).
7

---

Wave
(Ry(8(a)) (P(a)
Teet NAVII
1.0
NeurOLight
0.100
#MMode-v (kc)
Log IestN-MAE
30
0.5
#1
12
0.178
00.80 At
0.84
0.9
0.200
20
0.2
0.152
0.88
0.1
Wave Prior
0.149
10
50
100
150
200
40
70
100
130
160
190
Epoch
Wave Prior
#Mode-h (Ku)
(a)
(b)
(c)
Figure 10: (a) Test N-MAE curves of four models. (b) Our PDE encoder achieves the lowest error.
(c) Normalized test eтror contour of a 8 8-layer NeurOLight with different # of Fourier modes.
4.3 Ablation studies
PDE encoding. In Figure 10(b), we extensively compare different combinations of PDE encoding methods. The first two methods only model the distribution over G, conditioned on fixed wavelength and domain sizes like prior work [22, 2]. which fail to generalize to examples in larger design space. With PDE parameters (e,, X, f2), the model finds it difficult to learn generalizable representation, thus showing large errors on the test dataset. The last three combinations validate that permittivity and wave prior compact and effective encodings in our joint PDE representation method, while extra raw wavelength and domain information are redundant and even harmful.
Fourier modes. As shown in Figure 10(c), we perform fine- fine-grained exploration in the Fourier mode space to find the most suitable configuration. Unlike the flow prediction tasks evaluated in FNO [21] that only require few modes, using inadequate Fourier modes fails to learn the terahertz high-frequency optical fields the photonic device simulation task. However, using all Fourier series is not necessary and makes the model prone overfitting issues. (kn, (70,, 40) is the best setting that balances expressiveness and efficiency in our NeurOLight.
Cross-shaped NeurOLight block. In Table 2, we independently change one technique in the full NeurOLight model to verify each individual con- tribution. Our proposed essential techniques synergistically boost the modeling capacity and generalization. Compared to the linear lifting in FNO- 2d that only performs point-wise pro- jection, our lightweight convolution stem extract complex high-order wave patterns with negligible runtime
Table 2: Ablation proposed techniques. Each entry changes technique independently. Runtime is averaged over multiple runs on NVIDIA Quadro RTX 6000 GPU.
Variants
#Parums (M))
Train Er
Test Eir |
Runtime
(ms)
NeurOLight
1.58
0.145
0.122
12.1
ConvStem Lifting
1.58
0.156
0.134
11.9
Extra Parallel Conv Path
1.64
0.149
0.129
14.5
FFN + BN- GELU
1.37
0.469
0.446
Remove DWConv FEN
1.57
0.164
0.144
10.6
Extra GELU After FNO
1.58
0,164
0,148
12.4
Remove DropPath
1.58
0.131
0.136
12.1
overhead. Similar to U-FNO [37]. append additional parallel convolution path alongside the cross-shaped FNO block; however, the extra 20% runtime penalty does not pay off. The proposed convolutional FFN significantly improves the nonlinearity and local feature extraction ability of NeurOLight Changing it to a simple Batch from the MI.--based FFN in F-FNO [34], our extra depthwise CONV is critical local feature extraction and can reduce the test error by 16%. Note that an extra GELU after the FNO block will distort the feature in the low-dimensional space and have a negative impact on the performance [26]. Besides the dropout in the head, stochastic network depth [15] in the residual NeurOLight block is also effective in mitigating overfitting.
causes significant degradation. Different
4.4 Discussion
Superposition-basee mixup.
As shown in Table 3, without augmentation, NeurOLight only sees single-source training examples, thus failing to generalize when multiple sources are fused as unified input source for fast one-shot prediction, named multi-source inference mode. simple arrk-a would to perform single- source prediction on JJ ports and superpose the resultant J fields, named single-source inference

---

mode. When training on a large enough training set, this method indeed works. How- ever, quickly deteriorates as the training set reduces with a JJ times higher runtime cost for a |port device. With our dynamic superposition-base mixup, NeurOLight works well both in single-source and multi-source inference modes
Table 3: Test N-MAE an 8-layer NeurOLight with different number of training examples. Multi-source inference mode has similar performance as the single-source method but shows 3x faster runtime on 3x3 MMIs.
#Train Examplee (K)
Train Augmentation
Inference Mode
Runtime (ms)
1.4
4.1
6.9
9.7
12.4
None
Single-Souree
0,346
0.257
0.202
0.198
0.194
23.8
Mulci-Souree
0.892
0.882
0.880
0.865
0.873
8.3
Superposition
Single-Source
Source
0.229
0.205
0.204
0.200
0.199
23.8
Mixup
Multi-Source
0.230
0.208
0.206
0.202
0.202
8.3
with superior generalizability even with only 10% training data.
Spectrum analysis. Spectroscopy is an important approach to understanding the broadband response of a photonic device. A tra- ditional FDFD simulator has to sweep the spectrum with multiple simulations. In contrast, NeurOLight models the joint probability over wavelengths, and thus only needs to perform parallel inference with different ب = 2 that, though NeurOLight is only trained with five selected wave- lengths, it can generalize to unseen devices with unseen wavelengths. Sweeping in the standard C-band (1550 rum- 1--5 nm) with a 2 nm granularity, NeurOLigh: can finish within 150 ms, achieving 450x speedup over the FDFD simulator.
FDFD: NeurOLight: <150
0.14
Generalize to frequency
Test NeMMN-
X values at one shot. Figure 11 demonstrates
0.10
covered training set
0.08
1.53
1.54
1.55
1.56
Wavelength (umn)
Figure 11: NeurOLight can generalize to unseen devices and wavelengths.
Device adaptation. We evaluate the transferability of NeurOLight via device adaptation. In Figure 12, we transfer NeurOLight trained on 3-port MMIs to larger MMIs with and 5 ports. Directly predicting new devices shows unsatisfying test error out of distribution (OOD). We adapt the model with 20-epoch fast linear probing and 30- epoch finetuning [17] on 3.7 K 4-port MMI examples and 4.6 K 5 5-port MMI examples. The model quickly transfers to new photonic devices with good prediction fidelity.
4x4 MMI: A=1553 pm
3x3 MMI 4x4 MMI Direct Test Eir OOD: 0,399
5x5 MMI: A=1.55 pm
3x3 MMI + 5x5 MMI
Al. Al.-62 -82.3 nmx81. 9 nm
0.4
(1)
Al. Al, -84 mmx 113. mm
(1) (1rei Test EiT COD: 0.773
NeurOLight
Test N-MAE
0.3
Probing
Linear
Pinatune
Test N-MAE
0.5
(2) Lineer
Probing
Finetune Test Em: 0.171
Simulator
0.2
0.130
0.3
0.1
Error
20
30
40
50
10
20
30
10
50
Epoch
Epoch
Figure 12: Device adaptation from 3-port to 4-/ 5-port MMI via linear probing and finetuning.
5 Conclusion
In this work. first time, a phhyiccs-gnnsttc neural operator, named NeurOLight, is proposed for ultra-fast parametric photonic device simulation. We propose a joint PDE encoder with wave prior and masked source modeling for compact PDE representation. Our lightweight cross-shaped NeurOLight backbone design achieves a superior balance between modeling capability and parameter efficiency. In addition, novel superposition-baat based mixup technique significantly boosts the data efficiency and model generalizability, Experiments show that NeurOLight outperforms prior DNN models with 53.8% better prediction fidelity and 44.2% less parameter cost, serving surrogate model to the numerical solvers in photonic device simulation. Currently, our model focuses on device-level simulation. As future direction, look forward to exploring the circuit-lewe simulation and utilizing our model to streamline the optimization loop for efficient Al-assisted optical circuit design automation.
over 100x faster
The authors acknowledge the Multidisciplinary University Research Initiative (MURI) program through the Air Foree Office of Scientific Research (AFOSR), contract No. FA 9550-17-1-0071, monitored by Dr. Gernot S. Pomrenke.
Acknowledgments

---

References
[1] Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In Proc. NeurIPS, 2020.
[2] Mingkun Chen, Robert Lupoiu, Chenkai Mao, Der-Han Huang, Jiaqi Jiang, Philippe Lalanne, and Jonathan deep learning for high-speed electromagnetic simulation and optimization. Nature,
Fan.
2021.
[3] Q. Cheng, Kwon, M. Glick, M. Bahadori, L.. P. Carloni, and Bergman. Silicon Photonics Codesign Deep Learning. Proceedings of the IEEE, 2020.
[4] Kicran Cooney and Frank H. Peters. Multimode Interference Couplers with Tunable Power Splitting Ratios. Optics Express, 2016.
[5] Johannes Feldmann, Nathan Youngblood, Maxim Karpov, Helge Gehring, Xuan Li, Maik Stappers, Manuel Le Gallo, Xin Fu, Anton Lukashchuk, Arslan Raja, Junqiu Liu, David Wright, Abu Sebastian, Tobias Kippenberg, Wolfram Pernice, and Harish Bhaskaran. Parallel convolutional processing using an integrated photonic tensor core. Nature, 2021.
[6] Chenghao Feng, Jiaqi Gu, Hanqing Zhu, Zhoufeng Ying, Zheng Zhao, David Z. Pan, and Ray T. Chen. A compact butterfly-style silicon photonic-clectronic neural chip for hardware-efficient deep learning, arXiv preprint arXiv:21 1 1..6005, 2021.
[7] Jiaqi Gu, Chenghao Feng, Zheng Zhao, Zhoufeng Ying, et al. Squeezelight: Towards scalable optical neural networks with multi-operand ring resonators. In Proc. DATE, 2021.
[8] Jiaqi Gu, Zheng Zhao, Chenghao Feng, ct al. Towards arca-efficient optical neural networks: an FFT-based architecture. In Proc. ASPDAC, 2020.
[9] Jiaqi Gu, Zheng Zhao, Chenghao Feng. Hanqing Zhu, Ray T. Chen, and David Z. Pan. ROQ: A noise-aware quantization scheme towards robust optical neural networks with low-bit controls. In Proc. DATE, 2020.
[10] Jiaqi Gu, Hanqing Zhu, Chenghao Feng. Zixuan Jiang, et al. L2ight: Enabling on-chip learning for optical neural networks via efficient in-situ subspace optimization. In Proc. NeurlPS, 2021.
[11] Jiaqi Gu, Hanqing Zhu, Chenghao Feng, Zixuan Jiang, et al. Adept: Automatic differentiable design of photonic tensor cores. In Proc. DAC, 2022.
[12] Gaurav Gupta, Xiongye Xiao, and Paul Bogdan, Multiwavelet-based operator learning for differential equations. In Proc. NeurlPS, 2021.
[13] Daniel Haase and Manuel Amthor. Rethinking Depthwise Separahle Convolutions: How Intra-Kerne Correlations Lead to Improved MobileNets. In Proc. CVPR, pages 15588-14997, 2020.
[14] Ryan Hamerly, Liane Bernstein, Alexander Sludds, et al. Large-scale optical neural networks based on photoclectric multiplication. Phys. Rev. 2019.
[15] Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q. Weinherger. Deep Networks with Stochastic Depth. In Proc ECCV, 206.
[16] Tyler W. Hughes, Momchil Minkov, Ian A. D. Williamson, and Shanhui Fan. Adjoint method and inverse design for nonlincar nanophotonic devices. ACS Photonics, 2018
[17] Ananya Kumar, Aditi Raghunathan, Robbie Matthew Jones, Tengyu Ma, and Perey Liang. Fine-tuning distort pretrained features and underperform out-of-distribution. In Proc. ICLR, 2022.
[18] Juerg Leuthold and Charles H. Joyner. Multimode Interference Couplers with Tunable Power Splitting Ratios. J. Lightwave Technol. 2001.
[19] Mengquan Li, Zhongzhi Yu, Yongan Zhang, Yonggan Fu, and Yingyan Lin. O-has: Optical hardware accelerator search for boosting both acceleration performance and development speed. In Proc. ICCAD, 2021.
[20] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differ- ential equations. arXiv preprint, arXiv:2003. 03485, 2020.
10

---

[21] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli. Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In Proc. ICLR, 2021.
[22] Joowon Lim and Demetri Psaltis. Maxwellnet: Physics- driven deep neural network training based on maxwell's equations. Appl. Phys. Lett. 2022.
[23] W. Liu, W. Liu, Y. Ye, Q. Lou, Y. Xie, and I.. Jiang. Holylight: A nanophotonic accelerator for deep learning in data centers. In Proc. DATE, 2019.
[24] Lu, Pengzhan Jin, and George Em Karniadakis. Deeponet: Learning nonlinear operators for identi- fying differential equations based on the universal approximation theorem of operators. arXiv preprint, 19Xi:39900001933 2019.
[25] M. Raissia, P. Perdikarish, and G. F. Karniadakisa. Physics-informed neural networks: A deep learning framework for solving forward and inverse prohlems involving nonlinear partial differential equations. J. Comp. Phys.. 2019.
[26] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chich Chen. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In Proc. CVPR, 2018.
[27] Bhavin J. Shastri, Alexander N. Tait, T. Ferreira de Lima, Wolfram H. P. Pernice, Harish Bhaskaran, C. D. Wright, and Paul R. Prucnal. Photonies for artificial intelligence and neuromorphic computing. Nature Photonics, 2021.
[28] Yichen Shen, Nicholas C. Harris, Scott Skirlo, et al. Deep learning with coherent nanophotonic circuits. Nature Photonics, 2017.
[29] Kyle Shiflett, Dylan Wright, Avinash Karanth, and Ahmed Louri. PIXEL: Photonic Neural Network Accelerator. In Proc. HPCA, pages 474487, 2020.
[30] Febin Sunny, Asif Mirza, Mahdi Nikdast, and Sudeep Pasricha. Crosslight: A cross-layer optimized silicon photonic neural network accelerator. In Proc. DAC, 2021.
[31] Mohammad H. Tahersima, Keisuke Kojima, Toshiaki Koike-Akino, Devesh Jha, BingnanWarg and Chungwei Lin. Deep neural network inverse design of integrated photonic power splitters. Sci. Rep., 2019.
[32] Alexander N. Tait, Thomas Ferreira de Lima, Ellen Zhou, al. Neuromorphic photonic networks using silicon photonic weight banks. Sei. Rep., 2017.
[33] Yingheng Tang. Jichao Fan, Xinwei Li, Jianzhu Ma, Minghao Qi. Cunxi Yu, and Weilu Gao. Physics- guided and thysics-explainabbe recurrent neural network for time dynamics in optical resonances. Nat. Compu. Sci., 2022.
[34] Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. In NeurlPS Workshop, 2021.
[35] Rahul Trivedi, Logan Su. Jesse Lu, Martin F. Schubert, and JelenaVuckovic. Data-driven acceleration of photonic simulations. Sei. Rep., 2019.
[36] Zi Wang, Lorry Chang, Feifan Wang, Tiantian Li, and Tingyi Gu. Integrated photonic metasystem for image classifications at telecommunication wavelength. preprint, 2021.
[37] Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M. Benson. U-fno: an enhanced fourier neural operator based-deep learning model for multiphase flow. arXiv preprint, 2021.
[38] Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo.. SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers. In Proc. NeurIPS, 2021.
[39] Xingyuan Xu, Mengxi Tan, Bill Corcoran, Jiayang Wu, Andreas Boes, Thach G. Nguyen, Sai T. Chu, Brent E. Little, Damien G. Hicks, Roberto Morandotti, Arnan Mitchell, and David J. Moss. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature, 2021.
[40] H. H. Zhu, J. Zou. H. Zhang. Y. Z. Shi, S. B. Luo, et al. Space-efficient optical computing with integrated chip diffractive neural network. Nature Communications, 2022.

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Please see Section
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] We open-source our codes at link
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Please see Appendix A3 and Appendix A4
(c) Did you report error bars (e.g., with respect to the random seed after running exper- iments multiple times)? [No] Since the training process is not sensitive to random seeds.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Please see Appendix A3
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either the supplemental material or as URL? [Yes]
The new dataset generation code is in the supplemental material.
(d) Did you discuss whether and how consent obtained from people whose data you're
using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
12

---

A1 Optical field simulation
Analyzing how light field propagate through those components critical to device optimization and photonic integrated circuit design. Given a linear isotropic optical component, we will shine time-harmonic continuous-wand wave light on its input ports and analyze the steady-state electromagnetic field distributions E = x + + + 2E and H = &H, + HH, + 2H, in it, each of which includes horizontal (z)), vertical (y), and longitudinal (2) components. The light field follows the Maxwell
PDE under certain absorptive boundary conditions [16],
V E(r, = po t + Jr,r,)) V x H(r,t) = (r) (E(r.t) (r. a1
(7)
at
where Vx is the curl operator of vector function, Mo is the vacuum magnetic permeability, 6o and G. are the vacuum and relative electric permittivity, Jn and Je are the magnetic and electric current sources, Since the input light is harmoncc a vacuum angular frequency w, the time-domain PDE can be transformed to the frequency domain for the steady state as follows,
V x (rr) (() + (r.,r), x H(r) = jjweee, + Je(r).
(8)
simple variable substitution gives us the curl-of-curl Maxwell PDE,
((15 x Vx) -2 (r) = jwJ,(r), (V (F, - -² poca) H(r) = jn (9)
To restrict a unique solution without boundary reflection, complicated boundary conditions will be inserted [16]. An artificial material, i.e., mooriinate-stretchen perfectly matched layer (SC-PML), will be padded around the solving domain. Such PML materials have large imaginary parts in the permittivitiies
introduce strong energy absorption and changes the derivative operator to 0 _ ) where is eecation-determing complex value. Solving the (a) dr sy(y) ōy
V
0
(a) above PDEs will give the steady-state frequency-domain complex magnitude of the optical fields.
A2 Dataset generation
We generate our customized MMI device simulation dataset using an open-source FDFD simulator angler [16]. The tunable MMI dataset has 5.5 K single-source training data, 614 validation data, and 1.5 K multi-source test data. The etched MMI dataset has 12.4 K single-source training data, 1.4 K validation data, and 1.5 K multi-source test data. We summarize how we generate random devices in Table A4. We randomly sample the physical dimension of the MMI, input/output waveguide width, the width of the perfectly matched layer (PML), device border width away from PML, controlling pad sizes, input light source frequencies, etched cavity sizes and ratio (determines the number of cavities in the MMIs), and permittivities in the controlling region.
Table A4: Summary of device design variable's sampling range, distribution, and unit. Valuc/Distribution
Variables
Unit
J Tunable MMI
JJ x [J] Etched MMI
Length
(4220, 30) u((.5, 7)
(200, 30)
jm
Width
(S.55
jm
Port Length
3
jm
Port Width
(0.8, 1.1)
(1088 1.I)
jm
Border Width
0.25
0.25
jm
PML Width
1.5
pm
Pad Length
(4077, (.9))Lenggh (0.4, 0.65)x Width/[J| (1.53, 1,565)
14(0.7, 0.9))LLnngth
jm
Pad Width
240044, 0.65) Width/JJ| 24(1.53, 1.565) (/.055 0.1) 0.027 Length x 0.114 Width [2.07, 12.11)
jm
Wavelengths
jm
Cavity Ratio
Cavity Size
pmn
Relative Permittivity c
2(11.9 12.3)
A3 Training settings
We implement all models and training logic in PyTorch 1.10.2. All experiments are conducted on a machine with Intel Core i7-9700 CPUs and an NVIDIA Quadro RTX 6000 GPU. For training from
A 13

---

scratch, we set the number of epochs to 200 with an initial learning rate of 0.002, cosine learning rate decay, and a mini-batch size of 12. For the tunable MMI dataset, split all 7.680 examples into 72% training data, 8% validation data, and 20% test data. For the etched MMI dataset, we split all 15,360 examples into 81% 1 training data, 9% validation data, and 10% test data. For device adaptation, we first perform linear probing for 20 epochs with an initial learning rate of 0.002 and cosine learning rate decay; then we perform finetuning for 30 epochs with an initial learning rate of 0.0002 and cosine learning rate decay. We apply stochastic network depth with linear scaling strategy and maximum drop rate of 0.1.
A4 Model architectures
UNet. We construct 4-level convolutional UNet with base channel number of 34. The total parameter count is 3.47 M.
FNO-2d. For Fourier neural operator (FNO), we use 5 2-D FNO layers with channel number of 32. The Fourier modes are set to (#Mode.=32, #Mode,=10). The final projection head is CONVI CONVIx1(256) GELU-CONVIx122 The total parameter count is 3.29 M.
F-FNO. For factorized Fourier neural operator (F-FNO), we use 12 F-FNO layers with channel number of 48. The Fourier modes are set to (#Mode;=70, #Mode,:=4), The final projection head is x 12). The total parameter count 3.16 M.
CONV1 105VIxX(256-GLLL--COOO
NeurOLight. For our proposed NeurOLight, we use 12 F-FNO layers for tunable MMIs and 16 layers for etched MMIs with base channel number C=64. The convolution stem BSConv3 3(32)- ON-ReL.U-BSCon' (64)) -U where BSConv is blueprint convolution [13]. The Fourier modes
(#Mode =70, #Mode,=400 The channel expansion ratio in the FFN is set to s=2.
The final projection head is CONVI ((36))GELUCCOONXXXX2) The total parameter count is
1.58 M.
A5
Animation of NeurOLight
Runtime 0.008 FPS 120.5
5.75 um
XX
1.500
11 0000
Mag.
E. 11.000
fol 11.000
4 = 31.73 pm
A14


Reviews:
[{"review_id": "jFfY0LcUH8l", "reviewer": "Reviewer_Fu4p", "summary": "This paper proposes a neural operator that jointly models multiple parameters of EM simulation. Given input light wavelength, permittivity, and domain/material properties, the goal here is to establish a parametric mapping from the inputs to the output EM field via a neural network. To this end, the input domains are first normalized in terms of spatial scales and resolutions. Then, the normalized inputs are encoded as wave prior similar to the Fourier positional encoding. Following the masked image modeling allows us to cast the challenging optics simulation problem into a synthesis problem for missing regions given the input light. The network architecture for doing that is inspired by the Fourier network with a separable modification applied to that: using two 1D FFT instead of having one 2D FFT. The authors also propose an augmentation method that uses a linear combination of input lights/output fields as additional inputs/outputs. ", "questions": "- The wave prior method reminds me of the positional encoding which has been extensively used in coordinate-baesd neural implicit functions. I wonder how these two could be related.\n\n- The authors compare the proposed method with FDFD solution that runs on CPUs instead of GPU. Is this because the FDFD on GPUs is challenging to be adopted for this problem?  Discussion on this part would be helpful for general readers to understand the computational challenges in EM simulation.\n\n ", "limitations": "The authors adequately addressed the limitations.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall I like this paper. It tackles an important, but a challenging problem with novel solutions. Execution is also excellent. I hope below comments could be helpful to refine the paper.\n \nStrength\n- The paper tackles an emerging problem of learning-based optics simulation. The proposed method achieves SoTA performance on the device level simulation in terms of both accuracy and speed. \n- The paper is very well written with clear descriptions and figures. \n- Execution of the paper is of high quality. \n\nWeaknesses\n- There is no evaluation on the overfitting which often occurs in neural optical operators.\n- Evaluation is limited on a device scale as mentioned by the authors.\n- The comparison of the proposed method running on a GPU is done against FDFD method running on CPUs as described in Fig1.\n- L104: It would be helpful to write detailed forms A,x, and b in the main paper  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9FNazN9PkZZ", "reviewer": "Reviewer_1Stt", "summary": "This paper proposed a physics-agnostic light field prediction framework, called NeurOLight, that consists of a joint PDE encoder and an efficient cross-shaped neural operator backbone. A superposition-based mixup technique is developed to dynamically boost the data efficiency and generalization during the training of NeurOLight. Evaluation results show that the proposed method significantly outperforms the existing methods. ", "questions": "I am wondering if you could introduce the model architecture of the PDE encoder in more detail? Shall we know a PDE formula?", "limitations": "Yes, they have.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n[1] The idea is novel and interesting. It proposed a physics-agnostic light field prediction framework that can improve the efficiency and accuracy of learning parametric Maxwell PDEs. \n[2] It is the first AI-based framework that can learn the terahertz light propagation inside photonic devices that generalizes to different domains.\n[3] The proposed framework significantly outperforms the state of the arts with an average of 53.8% lower prediction error.\n\nWeakness:\n[1] It is not clear how much prior knowledge we should have for PDE encoder. If we do not have any prior knowledge in some applications, how does the proposed method work?\n[2] It is better to report the averaged results with multiple random seeds in Table 1. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "1tQIrAzVcbs", "reviewer": "Reviewer_vy9K", "summary": "This paper proposes NeurOLight, a variant of neural operators that is suitable to emulate optical devices efficiently. While the core behavior of the proposed NeurOLight is similar to the Fourier neural operators (FNOs), the authors introduce several techniques for optical simulations, including the scale-adaptation for merging variant domains, wave priors for Maxwell PDE encoders, masked image modeling for the light source, cross-shaped FNO that separates the horizontal and vertical pattern predictions, and superposition-based data augmentations. These all techniques for NeurOLight can streamline the complexity of the model as well as generalize the predictive performance. The authors show the proposed NeurOLight outperforms other baselines such as U-Net and FNO. They also provide some ablation studies that can support the proposed techniques are indeed helpful.", "questions": "The proposed method seems to be sufficiently solid and complete for its purpose. What prevents me from recommending the clear acceptance of this paper is that, as I mentioned in Strengths and Weaknesses, the main limitation of this paper is that the targeted problem is niche.", "limitations": "The authors do not explicitly state the limitations of their proposed method.\n\nAs I mentioned in Strengths and Weaknesses, I think that the main limitation of this paper is that the targeted problem is less relevant to the NeurIPS audiences.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper has many merits: the introduced ad-hoc techniques are convincing for the targeted domain, the writing is clear, and the experimental validation is solid. The paper also demonstrates several interesting results including spectrum analysis and domain transfer learning. Without a doubt, it is a good application paper for optical device simulations.\n\nMy main concern is that the target domain seems to be a niche problem for NeurIPS audiences. Because the proposed techniques are generally applicable to such a niche domain, the relevance and significance of this paper are not entirely clear.\n\nOverall, I think this paper is worthy of publication at some venues (maybe a PDE-related workshop or computational physics journals), but the venue does not have to be NeurIPS main conference.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept