Learning to Solve Nonlinear Partial Differential Equation Systems To Accelerate MOSFET Simulation
Authors: Seungcheol Han
Jonghyun Choi
Sung-Min Hong
Venue:  (2021)
Keywords: Partial differential equation
nonlinear equation
Newton-Raphson method
convolutional neural network

Abstract: Semiconductor device simulation uses numerical analysis, where a set of coupled nonlinear partial differential equations is solved with the iterative Newton-Raphson method. Since an appropriate initial guess to start the Newton-Raphson method is not available, a solution of practical importance with desired boundary conditions cannot be trivially achieved. Instead, several solutions with intermediate boundary conditions should be calculated to address the nonlinearity and introducing intermediate boundary conditions significantly increases the computation time. In order to accelerate the semiconductor device simulation, we propose to use a neural network to learn an approximate solution for desired boundary conditions. With an initial solution sufficiently close to the final one by a trained neural network, computational cost to calculate several unnecessary solutions is significantly reduced. Specifically, a convolutional neural network for MOSFET (Metal-Oxide-Semiconductor Field-Effect Transistor), the most widely used semiconductor device, are trained in a supervised manner to compute the initial solution. Particularly, we propose to consider device grids with varying size and spacing and derive a compact expression of the solution based upon the electrostatic potential. We empirically show that the proposed method accelerates the simulation by more than 12 times. Results from the local linear regression and a fully-connected network are compared and extension to a complex two-dimensional domain is sketched.

Under review as a conference paper at ICLR 2021
LEARNING TO SOLVE NONLINEAR PARTIAL DIFFER- ENTIAL EQUATION SYSTEMS To ACCELERATE MOS- FET SIMULATION
Anonymous authors Paper under double-blee blind review
ABSTRACT
Semiconductor device simulation uses numerical analysis, where a set of cou- pled nonlinear partial differential equations is solved with the iterative Newton- Raphson method. Since an appropriate initial guess to start the Newton-Raphson method is not available, solution of practical importance with desired boundary conditions cannot be trivially achieved. Instead, several solutions with interme- diate boundary conditions should be calculated to address the nonlinearity and introducing intermediate boundary conditions significantly increases the compu- tation time. In order to accelerate the semiconductor device simulation, we pro- pose to use a neural network to learn an approximate solution for desired bound- ary conditions. With initial solution sufficiently close the final one by trained neural network, computational cost to calculate several unnecessary so- lutions is significantly reduced. Specifically, a convolutional neural network for MOSFET Sideall-xiide-SSmiiondeee
Field-Effect Transistor), the most widely used semiconductor device, are trained in supervised manner to compute the initial solution. Particularly, we propose to consider device grids with varying size and spacing and derive compact expression of the solution based upon the electrostatic potential. We empirically show that the proposed method accelerates the simulation by more than 12 times. Results from the local linear regression and collececdde network are compared and extension complex two- dimensional domain is sketched.
INTRODUCTION
Nonlinear partial differential equations (PDEs) appear fiequently in many science and engineer- ing problems including transport equations for certain quantities like heat, mass, momentum, and energy (Fischetti & Vandenberghe, 2016). The Maxwell equations for the electromagnetic fields (Jackson, 1999), which govern one of the fundamental forces in the physical world, is one of the examples. By calculating the solution of those equations, the status of pesstm--undr--considrratooo can be characterized. In the machine learning society, solving set of coupled partial differential equations has become an important emerging application field. (de Avila Belbute-Peres al., 2020; Sanchez-Gonzalez et al., 2020)
Among many nonlinear partial differential equations, we consider the semiconductor device simula- tion (Grasser et al., 2003). The simulation is pivotal application to foster next-generation semicon- scale. Since the technology development heavily relies on the device
ductor device technology simulation results, if the simulation time reduces, the turnaround time also significantly reduce. In order to reduce the simulation time, acceleration techniques based upon the multi-core computing have been successfully applied (Rupp et al., 2011; Sho & Odanaka, 2017). However, the number of cores cannot be exponentially increased and the cost also increases drastically as the number of cores involved increases. Moreover, engineers submit many simulation jobs for group of semicon- ductor devices, computing resources available to each simulation job is limited. As an alternative, propose to improve the efficiency of the simulation per se.
In the semiconductor device simulation, a solution of a system of partial differential equations is numerically calculated with certain boundary condition. Those differential equations coupled

---

Under review as a conference paper at ICLR 2021
together and the overall system is highly nonlinear. The Newton- method (Stoer & Bulirsch, 2002) is known to be one of the most robust methods to solve a set of coupled nonlinear equations. When the method converges to the solution, the error decreases rapidly the Newton iterations proceed. To achieve a rapid convergence, it is crucial that initial guess for the solution needs to be close enough the real solution; otherwise, the method converges very slowly or may even diverge. Although we are interested in obtaining a solution at a specific boundary condition which is determined by an applied voltage, even obtaining an approximated solution initiate the Newton- Raphson method successfully is challenging. In literature, in order to prepare initial guess for the target boundary condition, several intermediate boundary conditions are introduced and solutions with those boundary conditions are computed sequentially (Synopsys, 2014). It, however, increases the overall computation time significantly. If the initial solution that is sufficiently close to final one is provided by any means, we can save huge computational cost of calculating several unnecessary solutions.
Instead, we propose to learn approximate initial solution of a set of coupled PDE for target boundary condition by an artificial neural network. Specifically, when a set of labeled images is available, neural network can be trained to generate similar image for a given label. The trained model generates a numerical solution for a target boundary condition. We show that the proposed initial solution by our method speed up the device simulation significantly by providing better initial guess.
We summarize contributions follows:
We derive a compact solution for PDE systems based on the electrostatic potential. As a result, the network size is reduced by factor of three. Since the electrostatic potential is well bounded, the normalization issue can be avoided.
For addressing various semiconductor devices, we propose a device template that can ad- dress various device structures with set of hyper-parameters. Since the electrical charac- teristics of semiconductor devices are largely determined by the physical sizes of internal components, handling grids with varying size and spacing is particularly important.
We propose convolutional neural network (CNN) which generates the electrostatic poten- tial from the device parameters. It can be used to accelerate the device simulation.
Compared with the conventional method, the simulation time is significantly reduced (at
least 12 times) with the proposed method while the numerical stability not hampered.
Results from the convolutional neural network are compared with other methods such as alternative architecture (the fully-comeected network) and the local linear regression.
Our approach can be extended to the complex two-dimensional domains. Preliminary re- sults are shown.
RELATED WORK
2.1 NEURAL NETWORKS FOR SOLVING DIFFERENTIAL EQUATIONS
Recently, there have been many attempts to build a neural network to solve a differential equa- tion (Han et al., 2018; Long al., 2018; Piscopo et al., 2019; Raissi et al., 2019; Winovich et al., 2019; Zhu et al., 2019; Obiols-Sales et al., 2020; Lu et al., 2020; Xiao et al., 2020). Among them, the Poisson equation is particularly of importance in the semiconductor device simulation. The Poisson equation plays a fundamental role in the semiconductor device simulation, by connecting the electrostatic potential and other physical quantities. In Magill al. (2018), the Laplace equa- tion (the Poisson equation with a vanishing source term) is considered for nanofluidic device. A fully-connected neural network trained to minimize the loss function, which combines the residue vector and the boundary condition. The authors assume a specific two-dimensional structure and the mixed boundary condition is applied. Another attempt to solve the Poisson equation suggested in Özbay et al. (2019). The Poisson equation with the Dirichlet boundary condition is considered. is decomposed into two equations. One is the Poisson equation with the homogeneous Neumann boundary condition. The other one is the Laplace equation with the Dirichlet boundary condition. A convolutional neural network architecture is adopted and the entire source term and the grid spacing used as the input parameters. The network is trained with randomly generated source terms and
2

---

Under review as a conference paper at ICLR 2021
Gate
SIO,
Drvite cureett IIA
Source
Drain
Hour
Substrate
Dradn voltage (V)
Pecitio t1-adds, (00)
(a)
(b)
(c)
Figure 1: Components of the semiconductor device simulation. (a) Schematic diagram of device with multiple terminals (yellow bars). (Input) (b) furrent-voltage characteristics. (Output) (c) In- ternal quantity called the electrostatic potential. In order to calculate the terminal current, those internal quantities are needed.
a predicted solution shows a good agreement with the numerical solution. In order to consider complex two-dimensional geometry, rectangular domain covering the original geometry is intro- duced in Winovich et al. (2019) and a convolutional network is used. A neural network is used to solve the Poisson equation in L-shape domain in Lu et al. (2020).
Quite recently, has been independently proposed to accelcrate the computational fluid dynamics simulations by predicting appropriate initial guess from convolutional neural network (Obiols- Sales et al., 2020). The basic idea is quite similar with this work and such a coincidence strongly demonstrates that the proposed approach is not restricted relatively narrow application domain.
2.2 NEURAL NETWORKS SEMICONDUCTOR DEVICE SIMULATION
Researchers are much interested with application of machine learning technique to the semiconduc- tor device simulation (CarrilloNNuñez al., 2019; Bankapalli & Wong, 2019; Han & Hong, 2019; Souma & Ogawa, 2020). However, the actual research activities are quite diverse. In many cases, neural networks are merely used as an efficient descriptor the input-output relation without considering the internal physical quantities (Carrillo-Nußee et al., 2019; Bankapalli & Wong, 2019). On the other hand, in their quantum transport simulation, Souma & Ogawa (2020) try to predict the electron density profile from the electrostatic potential. Although application of neural networks the semiconductor device simulation in its infancy, has a huge potential.
3 PRELIMINARY: SEMICONDUCTOR DEVICE SIMULATION
We briefly introduce the semiconductor device simulation as a preliminary. As shown in Figure Ia, multi-dimensional device structure has multiple terminals (yellow bars in the figure) whose voltages are controllable. Since the planar MOSFET has the translational symmetry along the direction perpendicular the plane in Figure 1a, the dtree-dimensiona device structure two-dimensional one. The terminal currents (in Ampere) under the determined terminal voltages (in Volt) are the output of the simulator, shown in Figure 1b. As the motion of electrons yields the current, the terminal current itself can be only calculated from the internal physical quantities. (See Figure Ic.) Since the electrons are charged particles, the electric field due to the net charge density is also affected by the electronic motion. Therefore, we need to consider both effects of electronic motion and electric field by solving two equations, to calculate the terminal currents.
be treated
The first one is the continuity equation for electrons:
V JJ 0,
where the electron current density vector, Jn, is given by
J 9/4, n. 9D,,
where the elementary charge (q), the electron mobility (pn), and the electron diffusion constant (D,)) scalar parameters and the electron density (n) is dosition-dependent unknown variable. Note that the current density (J,)) also depends on the electric field vector (E), Similar relations hold for holes with minor modification.
3

---

Under review as a conference paper at ICLR 2021
The second equation is the Gauss law, also called as the Poisson equation:
(cE) q(p nt + N dop
where p is the hole density, is the permittivity (Jackson, 1999) and N is the positively charged impurity density. Under the electrostatic approximation, the electric field vector can be expressed in terms of the electrostatic potential (() as
VΦ.
With the electrostatic potential, the above set of equations has two unknown variables; and ø. Note that nonlinearity is originated from (2). The first term in the nighthhandside of (2) is proportional to E., which nonlinear with respect to unknown variables of n and Φ. Therefore, when ó is fixed, the continuity equation becomes linear.
For numerical analysis, we need to discretize the equations. In general, the i-th component of the residual vector can be written
fi(01, ON, T1,, ..., RN, PI..... PN) 5; 0,
where N is the number of grid points, fi(-) is nonlinear function, and 8; is constant. 8; becomes nonzero only at the boundary nodes and depends on the voltages applied to the device terminals. Since the system is nonlinar, the solution at each boundary condition must be computed. Detailed discussion can be found in Appendix A.
The set of discretized equations in (5) is highly nonlinear and the iterative Newton- Raphson method used to solve the system. Typically, a user wants to know the drain current at VG VD = VDD. where VDn is the maximum allowed voltage. However, Newton-Raphson method at that condition is not directly available. Only the solution at Va = VD = O V can be easily calculated. Starting from the solution with all zero terminal voltages, many inter- mediate steps toward the target terminal voltages are introduced. Based upon the converged solution the previous condition as an initial guess, the next step is solved sequentially. Detailed discussion can be found in Appendix B. Therefore, computing several intermediate boundary values is the main source of increase of computation. Let us denote the number of boundary values simulated during the solution procedure as N Nstep. To evaluate the acceleration quantitatively, we define the reduction factor of the simulation time, a, and well approximated as
appropriate initial guess start the
Toonv
Nstep Nnewton
ss N stepp
Nddrect neatonn
Tyn
where Teone is the simulation time with the conventional method while Tua is the one with the proposed method. Moreover, Nnewton is the average number of the Newton iterations per bias condition and N ddirect is the number of the Newton iterations at the target bias condition. Since Natep is typically larger than 10, we expect that the reduction factor can be larger than 10.
APPROACH
now clear that an approximate solution for given boundary condition is key to accelerate the device simulation. Instead of inventing yet another method to calculate the approximate solution efficiently, we propose data driven approach by using a neural network learn the numerical solution for desired boundary conditions.
4.1 COMPACT FORM OF A SOLUTION
In (5), the residual vector has 3N components, because we consider three internal quantities, Φ. n, and p. as solution variables. Typically, the number of grid points, N, ranges from thousands to tens of thousands. Therefore, is desirable to introduce compact form of solution. Among various physical quantities in the semiconductor device, the electrostatic potential, Ø, is a key quantity. As discussed earlier, the nonlinearity arises from nE Vo in (2). Under fixed electrostatic potential profile, the electron and hole continuity equations become linear. The continuity equation decoupled from (3) is easy to solve. In other words, the electron and hole densities, and p, can be
4

---

Under review as a conference paper at ICLR 2021
readily obtained long reasonably good potential profile is provided. Thus, we propose to a neural network to generate electrostatic potential profiles.
By the compact form of the solution, the number of output components reduces by a factor of three. Moreover, the electrostatic potential is well bounded in a small voltage range and varies smoothly over the device structure, normalization of the input data is required.
4.2 DEVICE TEMPLATE
The initial device design usually exhibits sub-optimal performance as the device parameters are not optimal the early stage of the technology development. During the technology development cycle, numerous structures are simulated for engineers to achieve better performance. Those devices are different in terms of the physical sizes of internal components and the doping densities. To en- able our method to address various device types, we pro- pose device template of MOSFETs. By determining each hyper-parameter of the template, we simulate vari- types of MOSFETs. We illustrate device template that we use in Figure 2.
00134mm
02591pm
11300pmm
Rue
Figure 2: A proposed device template. By changing the hyper-parameterss, one can readily simulate various types of MOSFETs.
Device parameters, such the gate length (Lo), the oxide thickness (loz), and the doping concentrations (Nad for the source/drain regions and Neub for the substrate region) are varied within predefined ranges. For that purpose, the x-directional grid spacing under the gate terminal is adjusted and the proposed method can be applied to a set of various device structures whose grids have different spacing. Since the electrical characteristics of semiconductor devices are largely determined by the physical sizes of internal components, handling grids with varying size and spacing is particularly important.
4.3 NETWORK ARCHITECTURES
We propose convolutional neural network to generate the two-dimensional potential profile for a given boundary condition. We design the architecture by adopting the generator part of the DCGAN (Radford et al., 2015) and illustrate it in Figure 3a. It takes the device parameters such as La, tox. Nad. and Naud and the applied terminal voltages such Va and VD input. The output layer generates a 64-by-64 matrix as input for the simulation. In addition to the CNN, fully-connected network (FCN) Figure 3b is tested. Its depth is the same with the CNN and sizes of layers are designed to have comparable level of computational complexity in the training phase.
Layt
Priecial
256-000
210
45:04
(a)
(b)
Figure 3: Layer structures adopted in the two-dimensional MOSFET. (a) CNN and (b) FCN.
4.3.1 OBJECTIVE FUNCTION
To learn such network, we use and minimize simple mean squared error objective function as
L Σ
c Φ1
where o. is the electrostatic potential at the i-th node, predicted by the neural network. After 150 epochs, the mean square error of the electrostatic potential is sufficiently reduced.
5

---

Under review as a conference paper at ICLR 2021
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
We conduct a set of experiments of MOSFET simulation (Taur & Ning, 1998). Device structures are shown in Figures la and 2. A two-dimensional grid with size of 64-by-64 is used for all devices. Since the number of grid points is fixed, when the physical length of the MOSFET changed, the grid spacing is also changed. For example, with large Le value, the distance between two neighboring points near the gate terminal becomes large. Since is 64 x 4,09,,,,, the solution vector has 3N 12,288 unknown variables. We apply voltages to the gate terminal (Vc) and the drain terminal (VD) to draw a current through the drain terminal (Tp). The room temperature is assumed to be 300 K throughout the experiments. We train our network in supervised fashion with the backpropagation algorithm on Pytorch 1.4.0 library. For the Adam optimizer, typer-parametee such the learning rate of 10-4, the L2 regularization coefficient of 10-7, and the batch size of 16, are used.
5.1.1 DATASET
We use a simulator by Han & Hong (2019) as it is free from the license and the source code is pub- licly available. Detailed description on the simulator can be found in Appendix C. Each data point is specified with parameters of La, tor. Ndd Nusb VG. and VD and their ranges are summarized in Appendix D. We curate the dataset with 10,112 instances that are randomly selected. We split the dataset by training, validation and test set by 70 % of the selected devices are the training set and the others in the validation (20 %) and test (10 %) sets. We will publicly release our splits for the future research in this avenue. The training and validation errors can be found in Appendix D.
5.1.2 COMPARISON WITH THE CONVENTIONAL METHOD
As our primary goal is to accelerate the semiconductor device simulation with help of the initial solution obtained by the neural network, we evaluate the speed up of the device simulation algorithm compared to the conventional method. Specifically, we report the number of the Newton iterations for the converged solution at the target bias condition (VG = VD = 1.1 V). When the maximum potential update is smaller than 10 V, the convergence criterion holds.
By the conventional method, starting from the equilibrium condition of VG = VD = 0.0 V, we first increase the drain voltage up to 1.1 V. After that, the gate voltage is raised up to 1.1 V, which is called bias ramping. During the bias ramping, a uniform voltage step is applied and the solution at the previous boundary condition is used as the initial guess. When the voltage step small enough, every simulation is successfully finished to get final solution. However, with a large voltage step, some simulation runs may fail. In other words, the simulation is numerically unstable with large voltage step.
We also report the ratio of the failed runs to the total simulation runs to check the numerical stability. In contrast, by our method, when the neural network provides the approximate solution, we start the simulation directly at the target bias condition. We compare the number of the Newton iterations for the converged solution with the one by the conventional bias ramping method with uniform voltage step.
5.2 RESULTS FROM THE CNN
In this subsection, the results from the CNN are presented. After the training phase is finished, the trained neural network can be used to generate an approximate potential in the inference phase. Figure 4a shows the electrostatic potential profile generated by the trained neural network. The parameters of the MOSFET are LG = 96 nm, for = 3.5 nm, Nad 4.8 x 1020 cm cm- and N.a5 1.8 x 10 cm -3 The bias condition is VG VD 0.24 V. The potential difference between the predicted potential profile and the numerical solution, shown in Figure 4b, reveals that the maximum difference is around 0. V. The device in Figure 4a has longer gate, LG = 164 nm. Other parameters are also changed to for 3.1 nm, N.d 2.1 x 10 cm -3 and N.ub 2.0 10 cm Regardless Lg. the neural network can predict the electrostatic potential at a given bias condition (Va = 0.05 and VD 0.66 V) accurately.

---

Under review as a conference paper at ICLR 2021
proatel))
Dkareezi
saur
8:15
8:18
Pesthm
B16
616
0.
0.4
86
Pesition Os azis, (um)
Pesilion Du axis pua)
(a)
(h)
Figure 4: (a) Electrostatic potential profile predicted by the CNN Vo VD 0.24 V. (b) Difference between (a) and the numerical solution. The maximum potential error is about 0.1 V, which is reasonably small.
Beerstalc
A15
A10
A10
Poaltion
Positio
636
amar
8.00
6:00
1.8
(a)
(b)
Figure 5: Same quantities with Figure for different MOSFET with Lo 164 nm. Va 0.05 V and VD= = 0.66 V.
Maximum patential error MS)
10
5x10
3.0x10²
10
Conventiona acthed
Number of total 1000
10"
Phoposed mebied
Failure (%)
4x10
2.5x10'
Number
19*
2.0x10'
10-2
3x10*
10
1.5x10²
Trained
10
2x10*
9.0-1.1 1.1
1.0x16
Umined
10
1x10
L.V
5.0x10
10"
10
0.0
5
10
15
20
25
35
40
1.2
1.4
1.6
1.8
2.0
22
(1)
bias (V)
Gate blas (V)
(a)
(b)
(c)
Figure 6: (a) Convergence behavior. (b) Failure rate for a drain voltage outside the training range.
VD 1.1 V. (c) Failure rate for gate voltage outside the training range. Va = =11 V.
We demonstrate that the trained neural network generates the electrostatic potential profile close to the numerical solution in Figures and 5. For 1,012 samples in the test set, average and variance of the maximum absolute potential error 92 mV and 0.0016 v2, respectively. The convergence red curve in Figure 6a. With only five
behavior of the Newton-Raphson method is shown Newton iterations, the maximum potential update becomes smaller than 10 initial update is quite small, about 0.1 V, the fast convergence is achieved. On the other hand, the conventional method with a uniform voltage step 0.275 V (green curves) takes 40 iterations reach the same result of our method. For each voltage condition, it takes only five iterations. However, after an intermediate boundary condition is solved, the next boundary condition should be solved again. Therefore, the green curves have peak values every five iteration. Here, we achieve significant computation reduction factor of 8.
Note that as the
5.2. NUMERICAL STABILITY
In order demonstrate the numerical stability of our method, the voltages beyond the training range are tested in Figures 6b and 6c. In Figure 6b, the drain voltage larger than 1.1 directly solved. For each drain voltage, 100 devices generated and tested. Even for high drain voltage of 2.2 V, the failure rate is just 4 %. We perform a similar test for the gate voltage larger than 1.1 in Figure 6c. Although the failure rate increases sharply above 1.9 V, it remains quite small up 1.8 V. These

---

Under review as a conference paper at ICLR 2021
Loct
1.410
Number of total cares: 1000
1.2110
Test case (1)
Coso
64.911
1a Faluree count
Kis10'
Fallure count
Mean
20xts
2.8x10'
10
60
50
160
129
140
0.677
Iteration (1)
Iteention
Ramping
(a)
(b)
(c)
Figure 7: Numerical stability and speed of the proposed approach. (a) Distribution of the Newton iterations out of 1000 test cases. (b) Distribution of the Newton iterations with the conventional method. (c) Trade-off between the numerical stability and the simulation speed.
examples clearly demonstrate that the trained neural network can provide a sufficiently good initial potential profile.
5.2.2 APPLICATIONS TO MANY DEVICES
We now investigate the applicability of our method to many devices. We evaluate our method on 1,000 different devices that are randomly generated. The simulation starts at VG = VD = 1.1 V. We show the distribution of the Newton iterations in Figure 7a. Among 1,000 test instances, only two samples fail to converge. For most cases (more than 99.7 %), four or five iterations are required for the converged solution. On the other hand, when the conventional method applied, the number of the Newton iteration increases significantly shown in Figure 7b.
In the conventional method, the voltage step is an important parameter to control the numerical stability and the speed. When we adopt small voltage step of 0.1 V, there occurs no failure case. In this case, about 90 Newton iterations are needed to have the converged solution and the simulation is almost 20 times slower than the proposed approach. We may try increase the voltage step for numerical efficiency. When the voltage step is 0.22 V, the number of the Newton iterations is reduced almost a factor 2. However, the failure rate is larger than 10 %, which is certainly not tolerable. Even with a very large voltage step of 0.367 V, the number is 21, which is much larger than that of the proposed approach.
These observations are summarized in Figure 7c and Table 1. For the conventional method, there exists trade-off between the numerical stability (related with the failure rate) and the simulation speed (related with the number of iterations). Our proposed approach exhibits both superior numeri- cal stability and short simulation time. When the failure rate is limited up to 1,, the reduction factor of the simulation time is larger than 12.
The speedup obtained with a cost of building the training dataset. the number of inference runs is not large enough, the speedup by adopting the neural network will not yield any performance merit. However, is noted that the semiconductor device simulator is typically developed by de- veloper group (a company experienced by the user is the most important metric to justify the additional cost of the developer group. Moreover, the total number of inference runs by the entire user group can easily exceed the number of data points the training set. Similar cases may be found in other research fields, where the developer and user groups of the numerical simulators are well separated.
research group) and distributed to several users. The simulation time
5.3 COMPARISON
As an alternative to the proposed neural network, we implement the local linear regression (Garimella) to predict an initial guess. For each prediction, we use the subset that has the clos- est dataset from input xa with ratio of B to the whole dataset. The weight function is the tri-cube weight function,
w (d) d3)3
(8)
where d is the normalized Euclidean distance. With cross- cross- validation, we find that the fraction coef- ficient, 3 is 0.2. We perform the same convergence test for many devices by using the local linear regression. As a result, the expected number of iterations is 6.5 and the failure rate is 2.1%. More
8

---

Under review as a conference paper at ICLR 2021
Table Failure rate and speed the conventional method and ours, a is the reduction factor defined in (6). The conventional method is tested with various voltage steps. For example, in case of 0.1 V, 22 boundary conditions (11 for the drain sweep and 11 for the gate sweep) are solved. When the failure rate is too high, the second simulation round should start to solve the failed cases with smaller voltage step, taking overall simulation time even longer. It implies that high failure rate not tolerable.
0.1 V
0.157 V
CNN (Ours)
FCN
0.22 V
0.367 V
Failure rate (%)
0
1.7
0.2
2.6
10.4
14.7
Number of iterations
91.6
65.0
5.4
5.91
46.6
21.5
(Reduction factor (6))
17.0
12.0
1.0
1.09
8.6
4.0
results can found in Appendix E. Clearly, the neural network outperforms the local linear regres- sion in terms of the numerical stability. Moreover, in order to apply the regression, we need the densely-sampled dataset at each inference run. Since there exist several users, it is not practical to deliver the censely-sampled dataset to each user. The trained neural network is a more efficient way to approximate the censely-sampled dataset. In addition, another comparison is made between the CNN (Figure 3a) and the FCN (Figure 3b) in Table 1. Discussions can be found in Appendix F.
5.4 EXTENSION TO COMPLEX TWO- INO-DIMENSIONAL DOMAIN
20 qpeed Convall
Paiatall
128 56-16
(a)
(b)
Figure 8: Complex two-dimensional domain. (a) An unstructured mesh for a MOSFET, (b) Layer structure of the adopted neural network. We have an internal CNN. (c) Predicted potential profile Va = Vp = 1.0 V. Colors represent the electrostatic potential.
Up now, have used device template describe various MOSFETs. In this subsection, pre- liminary result about extension of our approach to a complex two-dimensional domain is presented. In Figure 8a, unstructured mesh for MOSFET is shown. A network shown in Figure 8b learns the electrostatic potential in supervised fashion. In this example, the output of the internal CNN in Figure 8b is mapped into the original unstructured mesh. The electrostatic potential is generated by the trained neural network, as shown in Figure 8c. Since the maximum potential error is lower than 76 V. can be used as an approximate solution. More information can be found in Appendix G.
CONCLUSION
We train convolutional neural network to generate the electrostatic potential required for the semi- conductor device simulation. By using the generated electrostatic potential as an initial guess, the target bias conditions can be simulated directly without time-consuming ramping procedure. Our proposed approach can address set of various device structures whose grids have different spacing. Moreover, we suggest a compact expression for the solution based upon the electrostatic potential to reduce the computational complexity.
In the empirical validations with two-dimensional MOSFETs, the simulation time has been signifi- cantly reduced (more than 12 times) with the proposed method while the numerical stability is not hampered. As deep neural networks perform well when a sufficient amount data points are used in training, it is expected that the proposed approach would be accelerate the development cycle of semiconductors since massive simulation results are routinely generated during the technology optimization and are ready to be used for training such networks.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Y. S. Bankapalli and H. Y. Wong. Tead augmented machine learning for semiconductor device failure troubleshooting and reverse engineering. In 2019 International Conference on Simulation of Semiconductor Processes and Devices (SISPAD), pp. 1-4, 2019. 3
H. Carrillo-Nuñez, N. Dimitrova, A. Asenov, and V. Georgiev. Machine learning approach for
predicting the effect of statistical variability in si junctionless nanowire transistors. IEEE Electron Device Letters, 40(9): 1336-1666660 1369, 2019.
Timothy A. Davis. Algorithm 832: Umfpack v4.3- an unsymmetric-pateee patter multifrontal method. ACM Trans. Math. Softw., 30(2):196-199 June 2004. ISSN 0098-3500. 992206. URL https: doi. ott/ 1 1145/992200 992200. .1122200.992200 992206. 13
doi: 10.1145/992200.
Filipe de Avila Belbute-Peres, Thomas D. Economon, and J. Zico Kolter. Combining differentiable pde solvers and graph neural networks for fluid flow prediction, 2020.
M. V. Fischetti and W. G. Vandenberghe. Advanced physics of electron transport in semiconductors and nanostructures. Springer International Publishing Switzerland, 2016.
Rao Veerabhadra Garimella. A simple introduction to moving least squares and local regression
estimation. doi: 10.2172/1367799, 8
Tibor Grasser, Ting-Was Wei Tang. Hans Kosina, and Siegfried Selberherr. A review of hydrodynamic and energy-transport models for semiconductor device simulation. Proceedings of the IÉEE, (2):251-274, February 2003. 1
Jiequn Han, Arnulf Jentzen, and Weinan E. Solving high high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):55058851) 8510, 2018. ISSN 0027-8424. doi: 10.1073/pnas.171888 1718942115. URL http:::/t/wwtt pnas org/ content/115/34
2
Han and S. Hong. Deep neural network for generation of the initial electrostatic potential pro- file. In 2019 International Conference on Simulation of Semiconductor Processes and Devices (SISPAD), 2019. 3, 6, 13
Jackson. Classical Electrodynamics. John Wiley and Sons, third edition, 1999. 4,4, 12
C. Jungemann and R. Meinerzhagen. Hierarchical Device Simulation: The Monte-Carlo Perspec- tive. Springer-Verlet Wien, 2003. 13
S. E. Laux and B. M. Grossman. A general control-volume formulation for modeling impact ion- ization in semiconductor transport. IEEE Transactions Electron Devices, 32(10):077-22882 October 1985. 12
Zichao Long, Yiping Lu, Xianzhong Ma, and Bin Dong. PDE-net: Learning PDEs from data. volume 80 of Proceedings of Machine Learning Research, pp. 3208-3216, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR. URL
r press/
v8C/long18a html. 2
Lu Lu, Xuhui Meng, Zhiping Mao, and George E. Karniadakis. Deepxde: A deep learning library for solving differential equations, 2020. 2,
M. Magill, F. Z. Qureshi, and H. W. de Haan. Compact neural network solutions to Laplace's equation in nanofluidic device. In Neural Information Processing Systems Workshop CDNNRIA, 2018. 2
B. Meinerzhagen, K. H. Bach, I. Bork, and W. L. Engl. A new highly efficient nonlinear relaxation scheme for hydrodynamic MOS simulations. In NUPAD IV. Workshop on Numerical Modeling of Processes and Devices for Integrated Circuits, pp. 91-96, 1991. 12
Octavi Obiols-Sales, Abhinav Vishnu, Nicholas Malaya, and Aparna Chandramow/iswharran Cfd- net: deep learning based accelerator for fluid simulations. Proceedings of the 34th ACM In- ternational Conference on Supercomputing, Jun 2020. doi: 10.1145/392717.339222. URL 2,3
http://t.toooooo....11.1.11 http://des doi www.o///11... 11..1...1113319.1
10

---

Under review as a conference paper at ICLR 2021
Maria Laura Piscopo, Michael Spannowsky, and Philip Waite. Solving differential equations with neural networks: Applications to the calculation of cosmological phase transitions. Phys. Rev. D, 100:016002, Jul 2019. doi: 10.1103/PhysRew..1000 org/doi / /10. 1100 1011111///ssseeee wwt.//yyyseeb.... 100. 0.1 6002. 2
https::////i aps.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks, 2015. arXiv preprint, 1r.1v:1511.0644..
M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: deep learn- ing framework for solving forward and inverse problems involving nonlinear partial differen-
tial equations. Journal of Computational Physics, 378:686 - 707, 2019. ISSN 0021-9991. doi: http://...rrr0..1111o........... 1...0... URL http: www. sewenceeieeeeeeeeeeeeeeeeeeeeeee com/
2
K. Rupp. T. Grasser, and A. Jüngel. Parallel preconditioning for spherical harmonics expansions of the boltzmann transport equation. In 2011 International Conference Simulation of Semicon- ductor Processes and Devices, pp. 147-150, 2011.
Alvaro Sanchez- Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter W. Battaglia. Learning to simulate complex physics with graph networks, 2020.
Sho and S. Odanaka. A hybrid mpi/openmp parallclization method for a quantum drift-diffusion model. In 2017 International Conference Simulation of Semiconductor Processes and Devices (SISPAD), pp. 33-36, 2017.
Satofumi Souma and Matsuto Ogawa, Accelcration of nonequilibrium greenapos;s function sim- ulation for nanoscale fets by applying convolutional neural network model. IEICE Electronics Express, advpub, 2020. doi: 10. 11.1887/eex.17..019977 3
J. Stoer and R. Bulirsch. Introduction to Numerical Analysis. Springer-Verlay New York, third edition, 2002.
Synopsys. "Sentaurus Device User Guide". 2014. 2
Y. Taur and T. H. Ning. Fundamentals of Modern VLSI Devices. Cambridge University Press, 1998.
Nick Winovich, Karthik Ramani, and Guang Lin. Convpde-uq: Convolutional neural networks with quantified uncertainty for heterogeneous elliptic partial differential equations on varied do- mains. Journal of Computational Physics, 394:263 - 279, 2019. ISSN 0021-9991, doi: https://
URL http: /wwww sci ecieniiireett com/ science/
aaticclepp1/S0021911993333 2,
X. Xiao, Y. Zhou, H. Wang, and X. Yang. novel cnn-based poisson solver for fluid simulation.
IEEE Transactions on Visualization and Computer Graphics, 26(3): 144-114554 2020.
Yinhao Zhu, Nicholas Zabaras, Phacdon-Stehin Koutsourelakis, and Paris Perdikaris. Physics- constrained deep learning for high- surrogate modeling and uncertainty quantification without labeled data. Journal of Computational Physics, 394:56 81, 2019. ISSN 0021-9991.
201..0......11000022..4 URL http: / www.wwwwwoooooooooowwooo.. www.sseiienccieet............ com/ doi: occiaaeceaaaricceeaa.../00....1.
Ali Girayhan Özbay, Sylvain Laizet, Panagiotis Tzirakis, Georgios Rizos, and Björn Schuller. Pois- enn: Convolutional neural networks for the solution of the poisson equation with varying meshes and dirichlet boundary conditions, 2019. 2
11

---

Under review as a conference paper at ICLR 2021
A DISCRETIZATION OF THE SEMICONDUCTOR EQUATIONS
Typically, the finite volume method (or the box method) (Laux & Grossman, 1985) is applied to convert the differential operator into the algebraic form. Then, the equation is converted into collection of algebraic equations, which are evaluated at many sampling points (called vertex nodes). Since the original set nonlinear with respect to n, p. and Φ. converted algebraic equations are also nonlinear. The Newton-Re Newton-Raphson method is applied to the resultant set of equations. In this section, the discretization the semiconductor equations, (1) and (3), is sketched.
It is observed that both equations have common form of
F - F O,
where is the flux term and 8 is the source term. In order to get discretized form of the above equation a certain grid point, we integrate the equation over finite volume (or a box) surrounding the point. For example, in grid shown in Figure 9, the discretized equation for the center point can be obtained by the integration over the red pentagon. With help of the divergence theorem (Jackson, 1999), have
f Ysur face
{ Box
F. %.da-
sd x
(10)
2
0-
-1
-2
-3 -2
2
Figure 9: Grid points (Blue circles) and finite volume surrounding the center point (A red pen- tagon).
Every node except for the terminal nodes contributes the residual vector following (10). We must also consider the boundary conditions the terminal nodes. For the electron density (n) and the hole density (p), we have
n - nlg O,
(11)
p Peg = O,
(12)
where Thg and Peg are the carrier densities at the equilibrium condition. These densities do not depend the applied terminal voltages. However, the boundary condition for the electrostatic potential (Φ) reads
è- @eq Vierminel = O,
(13)
where deg is the equilibrium potential and Vierminat is the applied terminal voltage. When the applied terminal voltage changes, s,'s in (5) at the terminal nodes change accordingly.
B BIAS RAMPING
The set of discretized equations in (5) is highly nonlinear and the iterative Newton- Newton-Raphson method used to solve the system. Although nonlinear relaxation scheme avoid the full Newton- Raphson method (Meinerzhagen et al., 1991) has been proposed, the full Newton -aatonnRpphhon method still preferred due its robustness.
12

---

Under review as a conference paper at ICLR 2021
Let us consider typical case, where user wants to know the drain current at VG= VD VDD- VOD is the maximum voltage which can be applied to the MOSFET. For example, VOD is 1.1 V in example. The drain current this bias condition is called the ON current and the ON current represents the current-driving performance of the MOSFET. Since the ON current of interest, would be most desirable calculate the ON current immediately. However, appropriate initial guess to start the Newton-Rapie Raphson method at V2 VD = VDD is not available.
When Va = VD O V, the device is in its equilibrium condition. In this special case, an effective way to generate the initial guess is well known (Jungemann & Meinerzhagen, 2003). It is based upon the charge neutrality condition, where p - n + dop this method cannot be applied non-zero terminal voltages. No established method for directly solving the equation at high terminal voltages exists. In order to overcome such a difficulty, the bias ramping technique typically adopted. Starting from the equilibrium operating condition with all zero terminal voltages, many intermediate steps toward the target terminal voltages are introduced. Based upon the converged solution at the previous condition as an initial guess, the next step solved with the Newton-Raphson method.
is imposed locally. Unfortunately,
In this conventional solution method, the overall simulation time, Teonus can be written as
Natep
Natep
No newton Teingle
Natep X N Nnewton X Taingles (14)
Toonu
conv
i=1
2=1
where Nstep is the number of the entire bias conditions, Toome is the simulation time for the i-th bias condition, N pewtoon is the number of the Newton iterations for the i-th bias condition, Tsingle is the time spent for a single Newton iteration, and Nnewton is the average number of the Newton iterations. The bias ramping heavily increases the simulation time, because it introduces large Natep-
In this work, propose solve the equation set the target terminal voltages directly without the bias ramping. The overall simulation time with the proposed method based upon the neural network, Tyni can be written as
7on
newton
X Tsingle:
(15)
where Nddiret the number of the Newton iterations at the target bias condition. Two numbers, Nnewten and Andrrett may be comparable. It is noted that Netep in (14) does not appear any more in the above equation. The reduction factor of the simulation time, which is defined as Teone/ Tans can be approximated with N Nstep, as shown in (6).
C DEVICE SIMULATOR
In this work, the device simulator in Han & Hong (2019) has been used to obtain the simulation results. As it free from the license issue, several simulation runs can be performed simultaneously. The Poisson equation, (3), the electron continuity equation, (1), and the hole continuity equation coupled and solved self-consistently. A rectangular grid with varying spacing is adopted. Therefore, every finite volume for grid point has four faces. The device template is implemented to accept the device parameters (such as LGg loz, Nad, and N Naub) as input parameters. When the Netwon- Raphson method is used, we must solve the following matrix equation:
Ju r,
(16)
where J is the Jacobian matrix, u is the update vector, and r is the residual vector. It is noted that the Jacobian matrix is very sparse in the semiconductor device simulation. In order to solve the above equation, sparse matrix solver, the UMFPACK library (Davis, 2004), is used.
D MORE INFORMATION ON DATASET
More information the dataset is provided.
Each data point is specified with parameters of LGs tax, Nuts Nsubs VG, and Vp and their ranges are summarized in Table 2.
13

---

Under review as a conference paper at ICLR 2021
Table 2: Ranges of input parameters
Parameter
Range
Gate length, Le
90 170
Oxide thickness, for Source/drain doping density, Ned Substrate doping density, Neub Gate voltage, VG Drain voltage, VD
nm nm
10 cm
10 cm
5 x 10 cm ~ 5 X 10 cm
V ~ 1.V
0 V L.V
We curate the dataset with 10,1 112 instances that are randomly selected. shown in Figure 10a. The training and validation errors are measured as function of the learning epoch as shown in Figure 10b.
10
Traininig eruur Validation
Mean square error (V2)
Oxide thiekness (nm)
3.5
10
3.0
10a
2.5
10a
10
0.10
0.12
0.14
0.16
Gate length (um)
Epoch (1)
(a)
(b)
Figure 10: (a) Distribution the selected devices the Lo- plane. (b) Training and validation errors as functions of the learning epoch.
E LOCAL LINEAR REGRESSION
We compare the distribution of the Newton iterations in Figure 11. Two methods (the CNN and the local linear regression) are compared. The failure rate of the local linear regression (2.1%) is much higher than that of the CNN (0.2%). When the failure rate is limited up to 1%, the regression does not meet the criterion. Moreover, the expected number iterations (6.5) also higher than that of the CNN (5.4).
Local lincar regression
0.2%
Convolutional Neural Network
Case (I)
100.
2.1% failure
10
10
13
22
31
40
Iteration (1)
Figure 11: Distribution of the Newton iterations. Two methods (the CNN and the local linear gression) are compared.
14

---

Under review as a conference paper at ICLR 2021
F FULLY-CONNECTED GENERATIVE NETWORK
We compare the distribution of the Newton iterations in Figure 12. The expected number of iter- ations with the FCN (5.9) is larger than the one with the CNN (5.4). The failure rate of the FCN (2.6%) is much higher than that of the CNN (0.2%). Also the distribution in Figure 12 shows a long tail for the FCN, which indicates difficulties in the Newton Rarionn method. When the failure rate is limited up %, it is difficult to utilize the trained FCN, least in its present form.
Fully-connected Network
Convolutional Neural Network
10a
0.2% failure
Case
10a
2.6% failure
10
10"
10
20
30
Iteration (1)
Figure 12: Distribution of the Newton iterations. Two methods (the CNN and the FCN) are com- pared.
is noted that the FCN in Figure 3b designed to have a comparable level of computational complexity in the training phase. When an approximate solution predicted by the FCN fails, the error at the very first Newton iteration tends to be larger than that of the CNN. Better performance enjoyed by the CNN can be attributed to the nature of the problem hand. The neural network must generate the electrostatic potential profile in the two-dimensional domain. Since the potential profile smoothly varying function over the domain (as shown in Figure 4a and Figure 5a), it can be treated as an image, which is suitable for the CNN.
G MORE INFORMATION ON A MOSFET WITH AN UNSTRUCTURED MESH
In Figure 8a, magnified view of the mesh structure near the channel region is shown. The entire mesh structure is shown in Figure 13. In total, 3,505 mesh points are assigned. The rounded regions top describe the insulating layers, which separate different terminals.
41
-38
435
412
412
122
04
12.
Figure 13: Entire mesh structure of the MOSFET in Figure 8a. Numbers in the represent coordinates in the micrometer unit.
15

---

Under review as a conference paper at ICLR 2021
The training and validation emrors measured as function of the learning epoch as shown in Figure 14. After 100 epochs, the mean square error the electrostatic potential sufficiently reduced.
Mean square (V3)
10
Training loss Validation loss
10
10a
10-3
10
12
Epoch
Figure 14: Training and validation errors functions of the learning epoch.
We test the convergence behavior of the Newton-RRphson method 121 bias points. For all of these bias points, the converged solutions are obtained.
With uniform voltage step of 0.01 V, the total simulation time for typical condition of Va Vp =1 I V is 173 seconds. When the uniform step is increased to 0.05 V, takes 60 seconds. However, with uniform step of 0.1 V, it fails to converge. On the other hand, the simulation initiated by the trained neural network takes only 5 seconds. The speedup factor, a, is larger than 10.
Finally, is noted that the typical number of mesh points ranges from few thousands (this example) to several dozens of thousands. Since the simulation time increases super-linearly with the mesh points, for device with dense mesh, the simulation time for a single run typically expected to be several hours. Several simulation runs for optimization introduces a heavy computational burden.
16


Reviews:
[{"review_id": "eriGTsuime8", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The submission proposes speeding up MOSFET simulations by learning the electrostatic field by example.\n\nOn the positive side, the submission reports 12x speedup over running MOSFET simulation. The submission speeds up the simulation by reducing the problem to a lookup of a electrostatic field as a function of 6 parameters of a MOSFET (two doping levels, oxide thickness, the gate length, V_DD, and V_g).\n\nHowever, the application doesn't seem to require a convolutional network.. the input space is only 6-dimensional, so simple techniques ought to work well. When I run into low-dimensional problems like this, I use local linear regression. [1] Namely:\n\n1. Scale each input to lie in [0,1]\n2. Choose a distance h in the 6-dimensional scaled input space (via cross-validating the following procedure)\n3. For any test point, find N points that lie within distance h of the test point\n4. Do weighted linear regression on those N points, with the tricube weighting function scaled to have support h [2]\n5. Evaluate the linear regression once for each output position.\n\nNote that in step 4, you form a NxN correlation matrix once, and perform a Cholesky decomposition on it. Then, for each position in the electrostatic output, you only need to do backsubstitution, which is O(N^2). \n\nThis would be the standard way of solving such a problem. You'd have to show that the neural network was substantially better or cheaper than this technique.\n\nAnother issue is with exactly how you measure the 12x speedup. That speedup time ignores the ~10,000 times you have to run the simulator to generate the training data. If the simulator is run only on the 1,000 test samples, then the simulator has actually slowed down by 10x (i.e., it's cheaper to not generate the training set and only simulate on the test samples). How many times is the simulator expected to run? If the simulator will run 100,000 times (e.g.), then it could be that the 5-d space is sampled densely enough where you can simply perform a nearest neighbor computation and return the electrostatic field out of memory, and do absolutely no interpolation or simulaion.\n\nReferences.\n1. https://en.wikipedia.org/wiki/Local_regression\n2. https://en.wikipedia.org/wiki/Kernel_(statistics)\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "eR9pNNG2__M", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method for generating an initial guess for numerical simulations of semiconductor devices. Using CNNs to generate a better initial guess speeds up simulations by a significant constant factor.\n\nPros:\n(1) The paper addresses an important practical problem with industrial applications.\n(2) The exposition of the underlying theory and the presentation of results is very clear, the paper is well-written.\nEven a non-expert like myself is able to follow the paper.\n(3) The experimental results are promising.\n\nCons:\n(1) Even though this is a good applications paper, the theoretical contributions to machine\nlearning are minimal -- and may not be enough to match ICLR standards.\n(2) More justifications for the chosen architecture in Fig. 3, and better yet -- an experimental\ncomparison of alternative architectures would be interesting to see.\n\nOverall, despite the promising experimental results, I am concerned that the paper may not meet\nthe standards of ICLR, hence my rating.", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZHg9PaoU_YW", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposed to use a CNN as a low-accuracy solver to output a good initial guess from the device parameters and then the solution is refined by an expensive numerical solver to get a good accuracy. The experiment on the MOSFET problem shows that, compared with the conventional numerical solver, this hybrid strategy can accelerate the simulation by more than 12 times.\n\nPros:\n- The proposed hybrid method speeds up the simulation a lot without a sacrifice of accuracy.\n\nMajor comments:\n- This idea of using CNN to output a good initial guess is not new and has been proposed, e.g., in the paper https://arxiv.org/abs/2005.04485 . So there is little contribution in the method.\n- The problem considered in the paper is a 2D rectangle domain, which can be handled by CNN nicely. However, in practice this may not be true. The authors should consider more complex domains. For example, in https://doi.org/10.1016/j.jcp.2019.05.026 CNN is used for complex 2D geometry. In https://arxiv.org/abs/1907.04502 neural network is used to solve Poisson equation in a L-shape domain, which should also be discussed in Section 2.1.\n- This paper has no contribution in machine leaning, and instead it is an application for solving PDEs. Most part of this paper is not relevant to machine learning topics. It may be more suitable to publish this paper in an engineering journal instead of a machine learning conference.", "extra_scores": {}, "extra_text": {}}, {"review_id": "VtJZZHVBb0P", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Contribution: Authors propose using a neural network to learn an approximate solution for desired boundary conditions in order to accelerate the semiconductor device simulation. They significately reduce the computational cost to calculate several unnecessary solutions when considering an initial solution sufficiently close to the final one by a convolutional neural network (CNN). To compute this initial solution, authors authors train a MOSFET based convolutional neural network. They empirically show that their proposed method accelerates the simulation by more than 12 times.\n\nPros:\n\n1) Proposal is original.\n2) Proposal is fast.", "extra_scores": {}, "extra_text": {}}, {"review_id": "RyrtYRt36Zv", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes to use a CNN to compute an initial guess for the iterative Newton-Rhapson solution of a coupled PDE system used for semiconductor device simulation. To do so, the authors construct a \"device template\" which parametrizes the design space. The CNN then maps a device configuration in this 6-dim space to the predicted electrostatic potential in the form of a 64x64 grid. The authors provide an analysis showing why predicting the electrostatic potential alone is sufficient. Overall, this approach can provide a simulation speedup of 12x or more. The authors are also planning to publish the dataset generated for the paper.\n\nThe idea to take a data-driven approach to compute the initial approximate solution in the context of the semiconductor simulation domain is interesting. Most prior work utilizing ML to accelerate PDE solution tried to generate such solutions directly. The path taken here, while seemingly easier, is interesting from an application point of view in that the solver is still used as before, potentially alleviating concerns about the validity of ML-accelerated solutions. The claimed order of magnitude speedup is significant (but see below for some comments on this). The main downside of the paper is the very limited novelty on the ML front (off-the-shelf generator from the DC-GAN paper) and demonstrated applicability to a specific and relatively narrow domain. As such, I recommend against publication at ICLR, and would advise the authors to submit the paper to a venue where the presented advances are likely to be of wider interest.\n\nOne specific point that I believe should be addressed, is to clarify the cost of building the dataset (10k simulations) and training the network, and its impact on the effective simulation speedup. As is, the 12x speedup would seem to apply in the limit of a very large number of network inference runs, and it is unclear if this is a practically relevant setting.\n\nSuggestions for improvements:\n- Explain to the reader explicitly why this is a 2d problem while the device exists in 3d space.\n- What specifically was the validation set used for?\n- In the text it is stated that \"grid spacing is adjusted\". Please explain more how this is done.\n- The text states that the network was trained with backprop. Which specific optimizer was used, and with which hyperparameters?\n- Including parts of the explanations currently in appendix B could make the text clearer and easier to read for people from outside of your field (e.g. the importance of bias ramping, and its relation to the simulation cost).\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes using a neural network to learn an approximate solution for desired boundary conditions to accelerate the semiconductor device simulation. The work shows that speed-up simulation is increased significantly. However, the major concern about this work is the limited contribution to the machine learning community as exposed by the reviewers. 
Decision: Reject