AD-NEGF: An End-to-End Differentiable Quantum Transport Simulator for Sensitivity Analysis and Inverse Problems
Authors: Zhanghao Zhouyin
Xiang Chen
Peng Zhang
Jun Wang
Lei Wang
Hong Guo
Venue: Submitted to ICLR 2023 (2023)
Keywords: Quantum Transport
Non-Equilibrium Green Function
Automatic Differentiation
Differentiable Programming
Deep Learning
Sensitivity Analysis
Inverse Design

Abstract: Quantum transport theory describes transport phenomena from first principles, which is essential for domains such as semiconductor fabrication. As a representative, the Non-Equilibrium Green Function (NEGF) method achieves superiority in numerical accuracy. However, its tremendous computational cost makes it unbearable for high-throughput simulation tasks such as sensitivity analysis, inverse design, etc. In this work, we propose AD-NEGF, to the best of our knowledge the first Automatic Differentiation (AD) based quantum transport simulator. AD-NEGF calculates gradient information efficiently by utilizing automatic differentiation and implicit layer techniques, while guaranteeing the correctness of the forward simulation. Such gradient information enables accurate and efficient calculation of differential physical quantities and solving inverse problems that are intractable by traditional optimization methods.

Under review as a conference paper at ICLR 2023
AD-NEGF: AN END-TO-END DIFFERENTIABLE QUANTUM TRANSPORT SIMULATOR FOR SENSITIVITY ANALYSIS AND INVERSE PROBLEMS
Anonymous authors Paper under double-bling review
ABSTRACT
Quantum transport theory describes transport phenomena from first principles, which is essential for domains such as semiconductor fabrication. As representa- tive, the Non-Equilibrium Green Function (NEGF) method achieves superiority in numerical accuracy. However, its tremendous computational cost makes it un- bearable for high-throughput simulation tasks such as sensitivity analysis, inverse design, etc. In this work, propose AD-NEGF, to the best of our knowledge the first Automatic Differentiation (AD) based quantum transport simulator. AD-NEGF calculates gradient information efficiently by utilizing automatic differentiation and implicit layer techniques, while guaranteeing the correctness of the forward simulation. Such gradient information enables accurate and efficient calculation of differential physical quantities and solving inverse problems that are intractable by traditional optimization methods.
INTRODUCTION
The strong and lasting demand for higher computing power and lower energy consumption urges the downscale of semiconductor devices. Over the last 40 years, the microelectronics industry has successfully made the transistor feature size scale from 10um to near 20nm, of which size the quantum mechanical effect starts to dominate (Anantram et al., 2008 Wang et al.) 2008 Datta 1997). Therefore, device simulators facing the future need to take a quantum theory oriented formulation, while NEGF, as a representative, is one of the most rigorous approaches among existing quantum transport methods (Jacoboni 2010).
Although NEGF shows superiority in simulation accuracy, also extremely time and computation consuming. Recently, many works successfully integrate machine learning techniques resolve the accuracy-efficiency dilemma of scientific simulations. A typical paradigm is to build up learning- based surrogate models (e.g., neural network) (Li et al. 2020 Bürkle et al. 2021] Pimachev & Neogi 2021). By learning from data generated with highly accurate simulations beforehand, the surrogate model is expected maintain first- princciie accuracy while performing much faster in usage. A fatal problem of such methods is that there is no guarantee for prediction accuracy, especially for input out of the distribution of the training dataset. Such drawback limits the application of machine learning based surrogates in quantum transport scenarios.
An alternative is to utilize automatic differentiation make the computation process differentiable. In quantum transport simulations, practically useful information is often related to calculating derivatives. For instance, the thermoelectric property measured by the Seebeck coefficient; the obb-threshold swing of MOSFET that is related to the derivative of the drain current In with respect to the applied gate voltage Vg. etc. Compared traditional numerical differentiation, automatic differentiation can overcome the trade-off between the round-off error and the truncation error when choosing the step-size (Gautschi] 1997] Chap. 3), and also can be numerically more efficient when the input dimension is high. Moreover, in theoretical inverse problems, end-to-end differentiable solver also extremely useful and in fact, critical. The availability of gradients makes it possible to conduct efficient gradient-bag based optimization, which outperform black-box optimization methods such as Bayesian optimization, genetic algorithm, etc., and can conduct optimization on a scale that blo methods cannot. Recent advances have also shown the value to apply differentiable

---

Under review as a conference paper at ICLR 2023
programming in scientific computation scenarios, such fluid dynamics (Holl et al. 2019). quantum
chemistry (Kasim & Vinko, 2021). molecular dynamics (Schoenholz Cubuk 2020). photonic
crystal optimization (Minkov et al. 2020), etc.
In this work, we propose AD-NEGF, to the best of our knowledge the first end-to-end diffcrentiable quantum transport simulator. The entire numerical process of NEGF and TB modeling is implemented in PyTorch, including the computation of the self-energy term, the Green function, the electrostatic potential, the transport properties, as well as an optional Klater-Koster Tight-Binding (SKTB) module to generate the block tri-diagonal Tight-Bindin
(TB) Hamiltonian (Klymenko et al. 202T), which we will introduce in detail in Section 3 The backward pass to compute the gradients improved by utilizing the implicit gradient techniques and the adjoint sensitivity method for Partial Differential Equations (PDE). To efficiently hackpropagate through Poisson's equation transport, we propose and implement the image charge gradient method, which can utilize the Fast Multi-pole Method (FMM) to reduce the backpropagation complexity of Poisson's 's equation from O(N$) ( to O(N4/3) We demonstrate the capability of AD-NEGF to efficiently and accurately compute differential physical properties by comparing with numerical differentiation. Also, it is shown that by cooperating AD- NEGF with the gradient-based optimizer, can perform high-dimensional optimization that is not affordable with conventional optimization approaches. Furthermore, in a more practical scenario of material doping optimization where we optimize the empirical SK parameters of injected atoms, method shows significant advances in convergence speed and optimization solution, compared with traditional black- optimization methods.
scale
Our contributions can be summarized as follows:
We propose and implement AD-NEGF, as far as we know the first end-to-end differentiable quantum transport simulator, including the NEGF method, the Poisson's 's equation module for self-consistent electrostatic potential computation, and the SKTB module to generate the
Hamiltonian from the coordinates and properties of the system atoms.
The efficiency of the backward gradient computation is improved by applying the implicit gradient method, the adjoint method for PDEs, well newly proposed gradient computation for the image charge method.
We validate the advantages of AD-NEGF in calculating differential transport quantities, high-dimensional parameter fitting, and device optimization, where AD-NEGF outperforms numerical differentiation and black-box optimization methods.
2 RELATED WORKS
NEGF.
Originating from Keldysh (1944); Kadanofi 20118) NEGF has been a well-received method in the quantum transport theory, which describes a system with finite bias voltage and contact interactions under consideration. Recently, NEGF-based computation methods gain increasing popularity for the simplicity of the formulation, and the easy implementation in programming (Ferry
Goodnick 1999 Taylor et al., 2001 Brandbyge et al., 2002, Fetter Walecka 2012). which makes NEGF one of the most widely applied methods in transport calculation. Several methods dedicated to improving its numerical stability and computational efficiency are proposed (Sancho et al. 1985 Krstič et al. 2002 Rungger& & Sanvito, 2008), some of which are widely implemented in modern quantum transport simulation software, including but not limited to Papior et al. (2017); Smidstrup et al. (2019); 9 Steiger al. (2011). On the other hand, despite its advantages, the NEGF method suffers from heavy computational burdens.
AI for Quantum Transport. There have been prior works to apply machine learning techniques in quantum transport, mostly by training a neural network with data generated from first-principle simulations, so that the neural network can serve as an efficient surrogate model to predict transport properties, such conductance (Bürkle et al.) 2021 Pimachev & Neogi 2021 Li et al. 2020), trans- port coefficients (Lopez-Bezanilla Bezanilla & von Lilienfeld 2014), etc. Most existing methods use relatively simple deep learning models such as mulli-layen perceptrons (Zupančćć et al.) and convolutional networks (Han et al. 1 Souma & Ogawa 2021 2020). while in some cases more advanced and Section dataset generated with ab-initio simulation is required, which is expensive to obtain. Moreover,
specially designed models are utilized (Bürkle et al. (202T). However, as mentioned
2

---

Under review as a conference paper at ICLR 2023
Left Electrode
Device
Right Electrode
Ho. No Ne.//o..Na
Compute Electrode Self-Energy
Compute Function
El
Haje
(E) M Man/wo D. (EI Hinyoo)
Gy(B) = (BI - Hy 23
D, Mo Haod,
Compute Transport Properties
Compute Electrostatic Potential
T(E) (pp
(M))M
-=
T(EXs- Lo)d
yuy Ps
Figure 1: Workflow of AD-NEGF. Solid lines indicate the forward simulation flow, where loops denote self-consistent iterations. Dashed lines indicate the gradient backpropagation flow.
limited by the fundamental drawbacks of statistical learning, there no guarantee for prediction accuracy, which will limit its application scenarios.
Differentiable Programming. Deep learning has been applied to more and more diverse scenarios, which requires the network structure to be more and more flexible. One emerging direction is to embed physical models numerical computation processes into the model, in order improve data efficiency, generalization capability, and interpretability. This is sometimes referred to as differentiable programming. It requires the automatic differentiation framework support implicit numerical operations, such as fixed-point iterations (Bai et al. 2019), optimization (Amos & Kolter 2017), initial value problems (Chen al. 2018), etc. Differentiable programming has been widely applied to physical simulations (Hu et al. [2019; Innes et al. 2019), such as rigid body dynamics (de Avila Belbute-Peree et al.) 2
Freeman et al. 2021). computational fluid dynamics (Kochkov et al. [021]] Holl et al.) 2019 Schenek & Fox] 2018). ray tracing (L.i et al. (018). etc. More specifically ab-initio simulations, there have been works for density functional theory (I.i et al.) 2021 Kasim& & Vinko 2021). Hartree-Fock (Tamayo- Mendoza al. 2018), coupled cluster methods (Pavošević & Hammes- Schifferr 2220),, and molecular dynamics (Schoenholz & Cubuk 2020).
However, have not found any previous works to apply differentiable programming techniques in the quantum transport domain.
PRELIMINARIES FOR THE NON-EQUILIBRIUM GREEN FUNCTION METHOD
In this section, we give brief introduction to the NEGF method, while more details can be referred in Appendix A Consider transport system containing device region and two semi-infinite contacts that attach to the left and right sides of the device, as shown in Figure [ The contacts can also be referred to as leads or electrodes interchangeably. According to the theory of quantum mechanics, whole system, including the device and the contacts, can fully described by Hamiltonian (TB) model (Slater & Koster (1954)), which makes
H. In this paper, we consider the Tight-Bindin И block tri-diagonal. We assume a set of basis has been selected so that the full NEGF process can be expressed in the matrix form. The stationary Schrödinger equation of this open system is H = F.V, where stands for the wave function of electrons, and E is a scalar value corresponding the system energy. The characteristics of the system contained in its Green function
G G=|EI- H1
where / the identity matrix. However, the Hamiltonian H is infinitely large and hence intractable. This is resolved by computing the Green function only for the device part, and modeling the effect of two semi-infinite contacts a term referred to as self-energy. The device Green function Go will then be used describe the non-equilibrium charge transport process by solving Poisson's equation
3

---

Under review as a conference paper at ICLR 2023
in self-consistent iteration. The output self-consistent potential field V and GD be used to compute transport properties, such as transmission, current, etc.
Device Green Function. The device Green function expressed as function of the device Hamiltonian HD and the self-energy Σ:
G HD -.
(2)
Directly computing the matrix inversion is with complexity O(N)). which is unbearable the matrix size is proportional to the vast amount of atoms. By utilizing the block tri-diagonal form of the Hamiltonian matrix, efficient recursive algorithm (Anantram et al. 2008) can be implemented, which scales linearly with the system size.
Electrode Self-Energy. Self-energy of electrodes computed from the surface green function ys of the electrode layer coupled with devices. Under the half-infinite hypothesis, 9s is approximated identically, expressed as a self-consistent equation:
9,
[A A.1-19.4
where A4,-1 is blocks of EI H of coupling between I and layer. To speed up, we implement the Lopez-Sancho algorithm (Sancho et al. 1985), as illustrated in Algorithm [ which converges exponentially faster than the conventional self-consistent iteration. Details of the algorithm are illustrated in Appendix. We also implement modern method based the generalized eigenvalue problem WWng et al. 2008) as an alternative.
Electrostatic Potential. NEGF, charge transfer due to the applied bias voltage is modeled as an external potential, which is attained self-consistently by solving Poisson's equation for the electrostatic field:
V ((r))(Av(() [p(r;; AV) Po(r)], (r(
(4)
(Vz. VR).
where Vy, and VR represent the voltage boundary conditions at electrodes 21, and ZR. AV = is the difference between real potential energy with equilibrium one. This equation is solved self- consistently with updated Hneq- Poisson's 's equation can be solved using numerical PDE solvers with spherical charges. Mcanwhile, computationally more efficient image charge method using the Fast Multipole Method (FMM) is preferred (Svizhenko & Anantram 2005; Zahn 1976). After the procedure converges stable solution, transport properties be computed accordingly. Once the convergence achieved, the Green function computed with Hneg will be used to compute various transport properties. We refer Appendix for details of equation solving, and expression of transport properties.
Vo
4 METHOD OF DIFFERENTIATING THE NEGF PROCESS
The differentiable NEGF model is implemented with PyTorch (Paszke et al. 2019). We extend the autograd function with implicit gradient techniques for backpropagation through self-consistent iterations, with the adjoint sensitivity method for caleulating gradients through Poisson's 's equation (Pontryagin] 1987). Moreover, the efficient gradient formula for the image charge method (Svizhenko) & Anantram 2005). accelerated by the Fast Multipole Method (FMM) method, is proposed to speed up the gradient calculation of Poisson's 's equation. The derived formula can be regarded as a summation of point charges produced by the gradients and thus also be computed with FMM. Details of the customized backward propagation modules are explained as follows.
4.1 IMPLICIT GRADIENT
The implicit gradient method is implemented when the direct automatic differentiation through function y fx) is unavailable expensive compute. Instances often arise when wants to calculate gradients through numerical solvers of equilibrium problems or complicated iterative algorithms. Based the implicit function theorem (Krantz Parks 2002), if there exists such
4

---

Under review as a conference paper at ICLR 2023
constrained function h(y. x)) where y is taken as the converged output of function f. the gradient ( Dy
can be given as dir
dy
dy
ph(g..)
di
d2
We use the implicit gradient techniques to derive the gradient of the surface Green function (Sancho ct al. 1985). where according to the ideal definition of the two semi-infimite leads, the converged surface Green function g=( must satisfy the self-consistent Equation (3). Hence h(gs, 0) gs O, where Au stands for [ESy Hul, and denotes the input
[An - Au-19 variables to compute Gs. Thus we could write down the gradient of ys with respect to 0 explicitly by 0h(g..0) (l( do dy.
dg.
09
Another scenario that the implicit gradient method can be applied to is to compute gradients through the self-consistent Poisson's 's equation loop, where the system electrostatic potential updated until consistent with the bias voltage of contacts and other boundary conditions.
4.2 ADIOINT METHOD FOR PDE
In order to perform backpropagation through the solver for Poisson's equation, adjoint sensitivity method (Plessix 2006 Pontryagin 1987) for PDE-constrained optimization problems is adopted, which is widely applied in constrained optimization of inverse problems. Here, the forward process of the numerical PDE solver is unaltered, which is often denoted as the state equation that links the controlled parameter and the state of the constrained system. Meanwhile, an adjoint state equation that connects the perturbation of variables and states is solved by using the same numeral solver. Then gradients can be evaluated with the adjoint state, and join in the gradient chain of backward propagation. Since the adjoint state equation is independent of the number of controlled variables, the total complexity is only proportional to the forward process, which makes it suitable for control problems with scalar output and high-dimensional input. Recently, the adjoint sensitivity method has also been applied in designing neural network structures with physics intuitions, including the Neural ODE (Chen et al. 2018)) and Deep Equilibrium Models (Bai et al. 2019), which can be considered as examples of cooperations of automatic differentiation and adjoint methods.
4.3 GRADIENT OF FMM IMAGE CHARGE METHOD
An alternative approach to solve Poisson's 's equation raised in Equation (4. is to apply the point charge approximation, where the charge density considered the linear combination of series of point charges as (r()) Σ; áq:ō(r - r;). Then by employing the linearity of Poisson's 's equation, the original form can be further decomposed into Laplace's equation with Dirichlet boundary conditions and Poisson's 's equation with zero Dirichlet boundary conditions:
- (AV) 0, AVi(r)) (V,, VR).
( V2 (AVV(()) AApr)) AV 0.
(5)
The first Laplace's 's equation can be easily solved by linear drop potential. The second equation can be solved by assuming the charge density as a combination of point charges of each atom site. The closed-form solution can be obtained using the image charge method (Svizhenko & Anantram 2005 Harb 2019), and the second potential can be written as:
Vi(ri)
Σ
4
1
4TC
(22 1j
+ (2, 2,)2
JENofi
Σ JEN
(6)
4TC
+
Vi + +
22g 43
V V + A7
==1
where 12) (z; )2 (1: 3, and A stands for the distance in the transport direction between central charges and charges from two electrodes. Therefore, the first term here describes the interactions inside the device, while all the remaining terms simulate the effect of its coupling to charges outside. The summation the second term is computed until achieving certain accuracy, which is empirically hundreds of site numbers. Hence direct summation is also too expensive
5

---

Under review as a conference paper at ICLR 2023
550
25 59
(a) Structure of an AGNR with (b) Structure of a 7-4 graphene (c) Structure a 5-2 graphene width and length 5. nano-junction.
nano-junction.
Figure 2: Device structures used the experiments.
Seepack dentilienttyyy
Transmission of AGNR(7)
T(E)
DOS AGNR17)
14-
ASE AD NEGF
(ev)
Vievi
Transmission and DOS calculated AD-NEGF Seebeck coefficient and differential conductance calculated by AD-NEGR.
and confirmed with ASE.
Figure 3: Transmission Quantity Computation with AD-NEGF.
compute. In this case, the Fast Multipole Method (Engheta al. 1992) is employed to reduce the computational complexity from O(N3) to O(N4/3).
To perform backward propagation through the fast multipole layer, the gradient of the output potential to the charges is required. By taking the derivative of target objective T. Cd - R, the derivative of I. with respect to charge {j can expanded as the image summation form of accumulated gradients from the last layer, which is:
al(v) 0qj
aL OV,
Σ
OV, 0qj
aL/ov,
4hT
v 12 + (2j )
ieN
aL/ov,
+
(8)
4TC
Pi A
A3
eig
43
V
A
SEN
Similarly, computing gradients of this form can be accelerated by the Fast Multipole Method, which also with complexity ((N4/3) and much faster than solving adjoint Poisson's 's equation.
5 APPLICATIONS
In this section, the advantages of AD-NEGF for sensitivity analysis and inverse problems demonstrated with three applications. For all experiments, we take graphene as the transport system, including the Armchair Graphene NanoRibbon (AGNR) and the graphene nano-junction, the basic device structures which are displayed in Figure More details of the experimental setup can be found in Appendix B

---

Under review as a conference paper at ICLR 2023
5.1 DIFFERENTIAL TRANSMISSION QUANTITY COMPUTATION
A direct and major application to perform differentiation on physical models is to evaluate differential physical quantities. In most cases, the analytical form is difficult to obtain. For numerical differenti- ation, the trade-off will be encountered between the round-off error and the truncation error when choosing the step-size (Gautschi] 1997 Chap. 3), and the computation will be unacceptable when the input dimension is high. On the contrary, automatic differentiation can achieve machine precision while maintaining (1) complexity when the output dimension is low and the input dimension is high (Baydin et al. 2018).
In this experiment, we first validate the correctness of the forward computation of AD-NEGF. As shown in Figure 3(a) the transmission coefficient and the density of states (DOS) of an AGNR system with width 7 are computed by AD-NEGF, which perfectly match the results of ASE (Larsen et al., 2017). an atomistic simulation package including electron transport modules. Based on it, we compute two differential transmission guantities, the Seebeck coefficient and the differential conductance. which are shown in Figure 3(b The Scebeck coefficient is measure of the magnitude of an induced thermoelectric voltage in response to a temperature difference across an atomic structure, mathematically expressed as the derivative of transmission T T(E) concerning the chemical potential E (Reddy et al., 2007):
n2 R.t.TTTT aln (T(E)(
Sjunction
3e
OE
where 7' stands for the temperature and Kg is the Boltzmann constant. The differential conductance is the gradient of electronic current to voltage: ID = av.
di
The singularity of the transmission function leads to peaks in the Seebeck coefficient curve, which highly sensitive thus challenging for derivative calculation, as illustrated in Figure 4 To amplify the phenomenon for clearer demonstration, the output transmission coefficient T(E) of the forward computation is transformed into half preecpreciioon foattin-pooii format for both automatic and numerical differentiation, before it is used to compute the Seebeck coefficient. It can be seen that, with AD- NEGF, we can still generate high-quality results. However, for numerical differentiation, the trade-off between the truncation error and the round-off error is observed by selecting different step-sizes from le-2 to le-5, With a large step-size, peaks may be skipped or mistakenly generated due to truncation error. With a small step-size, lacking machine precision causes noises on the curve. Specifically for step-size le-5, the calculated curve becomes totally meaningless. Moreover, even though this is not a high-dimensional input situation, evaluating the Seebeck coefficient with AD-NEGF can still be faster than numerical differentiation, since in AD-NEGF the backward pass is improved. According to our experiments, for a smaller system with 70 carbon atoms, computing the Seebeck coefficient for 400 energy samples costs 71.1 seconds with AD-NEGF and 98.3 seconds with numerical differentiation. For a larger system with 240 carbon atoms, computing the Scebeck coefficient for 400 energy samples costs 363.1 seconds with AD-NEGF and 512.6 seconds with numerical differentiation.
To summarize, by conducting the above experiments, the correctness and effectiveness of AD-NEGF validated. With AD-NEGF, differential transport quantities can be calculated simply by calling one backward step. Moreover, the process of computing derivatives is itself differentiable, permitting the computation of higher-order derivatives, which remains for further discovery.
5.2
TRANSMISSION FITTING
Inverse problems. which require inferring input parameters reversely from the output objectives, are in general difficult in first fisst-principle simulations. Black-box optimization methods require sampling large number of input combinations, the cost of which grows exponentially with the number of parameters. Based on the efficient and accurate gradient computation capability of AD-NEGE, performing gradient-brat based optimization holds the potential to outperform black-box optimization methods for high dimensional inverse problems.
We conduct a 10a dimensional optimization experiment to fit the transmission curve of one graphene nano-junction to another. The target system is a 7-4 nano-junction, with 7 graphene rings on the left and on the right. The fitting system is 5-2 nano-junction, and the fitting variables are the elements of its Hamiltonian, including the device, leads, and the corresponding couplings. The dimension of the optimizing variables is at the level of 10a. The transmission curve, shown in Figure
7

---

Under review as a conference paper at ICLR 2023
20
12.
FD 1step SBC-1e
-20
750 Iter
7-4 stargeti
5-2 (mibal)
5-2 (titte)
-20
FD (ele
20
FD Ista 10
0 E (eV)
(ev)
Figure 4: Comparison of Automatic Dif- ferentiation and Numerical Differentia- tion with different step-sizes.
The fitting loss and the fitted transmis- sion curve of a 5-2 graphene nano-junction.
Figure 5:
consists of 2000 energy points sampled from (-5eV, 5eV). Since directly computing the gradients of all 2000 points is inefficient for iterations, apply the stochastic gradient descent algorithm to conduct mini-batch optimization, which has shown supremacy of efficiency and performance in high dimensional optimization problems. More specifically speaking, the fitting parameters are optimized with the Adam optimizer (Kingma & Ba 2014) built in PyTorch, making the procedure highly similar training neural network.
The results are displayed in Figure where the loss is reduced to considerably low level, which means the converged parameters of the 5-2 nano-junction
nicely the larger 7-4 nano-junction. The fitted curve is akin to a smoothed version of the curve of the 7-4 junction, which agrees with the intuition since a graphene junction of 5-2 is of less freedom than that of 7-4 nano-junction. the other hand, we have also tried traditional black-box methods, such as Bayesian optimization, the genetic algorithm, and gradient-bar optimization with numerical differentiation, none of them can even work for this problem because of the curse of dimensionality.
5.3 ON-SITE DOPING OPTIMIZATION
Modern material engineering is capable of manipulating at the atomic level. More specifically speaking, by performing processes such as deformation, doping, etc., microscopic physical quantities such as atomic spatial coordinates, bond lengths and doping positions further modify the macroscopic material properties. The doping process is one of the most common techniques in material development, which can dramatically change the properties of the original material, by injecting foreign atoms into specific positions. In this experiment, we further explore the possibility to solve practical inverse problems with AD-NEGF by performing an end-to-end doping optimization cooperated with established material models.
be changed, which
In this experiment, try to reduce the average transmission of an AGNR system in a specified energy range of (-leV, leV), by injecting other atoms into the center of the AGNR system along the transmission direction. reduction of transmission coefficient near zero Energy point would indicate an increase of the truncation voltage, which changes the cemi-conductive properties of the device (Wu et al. 2013). Doping can be modeled as an effective change in the site and the hopping terms in the tight-binding Hamiltonian, i.e., the diagonal and off-diagonal elements of the Hamiltonian matrix. This on-site approximation allows us to treat doping optimization as tuning local terms the Hamiltonian influenced by the injected atoms. However, although the process above is applicable, the tuning terms in the TB Hamiltonian need to be distinguished carefully from those invariant ones. It will be more convenient to cooperate with SKTB model, which constructs the TB Hamiltonian based on strict rules of local dependence of atom identities and their semi- semi-empirical SK parameters. Besides convenience, it has more concrete physical interpretation than directly optimizing elements
8

---

Under review as a conference paper at ICLR 2023
Before Doping
14
12-
Doping Based AD-NEGF
24-
1
0.0
202
450
$20 time (second)
1002
Doping Bayesian Optimization
7.4
2.5
1.2
Bapealnn de, intuation Geneds:
Doping Based on Genetic Algarithm
6.-
0.0
-2
step
(eV)
(a) Loss against running time and iteration steps respec- (b) Original and optimized transmission curves. tively.
Figure 6: Comparison between AD-NEGF and conventional black-box optimization methods in the doping optimization task.
of the Hamiltonian, since it provides guidelines for practitioners find the possible atom satisfying the SKTB parameters from the optimization result. In this way. doping optimization is modeled as an optimization of the SKTB parameters of the doped atoms, which include the orbital energy and parameters for two center integrals. The total number of optimization variables is 13.
For comparison, we also apply black-box optimization methods including the genetic algorithm and Bayesian optimization. The results are displayed in Figure 6 In the loss diagram, the gradient-basee method converges significantly faster and better than the other approaches, especially the beginning of the training. The loss curves of the genetic optimization and Bayesian optimization are also dropping, but much slower and less effective, in terms of either the running time or the iteration step. Moreover, the performances of the genetic / Bayesian optimization are sensitive to preset nyper-parameters. Corresponding to the loss curves, the results of optimized transmission curves demonstrate the advantages of AD-NEGF in more straightforward way, where the gradient-based optimization gives a much cleaner band with low transmission in the target interval (-1eV, 1eV) compared to other methods. These results validate the effectiveness of the AD-NEGF method in conducting practical atomic-level inverse design to optimize transport properties by cooperating with material models.
CONCLUSION
In this paper, we have proposed AD-NEGF, the first end-to-end differentiable quantum transport simulator to the best of our knowledge. It guarantees the correctness of the forward simulation without the need for data or training, while providing gradient information based on differentiable programming. Compared with numerical differentiation, gradients can be computed more efficiently and accurately. Moreover, it accelerates parameter fitting and parameter optimization with gradient- based optimization. The results validated in applications such differential physical quantity computation, transmission curve fitting, and device doping optimization.
REFERENCES
Brandon Amos and J Zico Kolter. Optnet: Differentiable optimization layer in neural networks.
In International Conference on Machine Learning, pp. 136-145. PMLR, 2017.
MP Anantram, Mark Lundstrom, and Dmitri Nikonov. Modeling of nanoscale devices. Proceed- ings of the IEEF, 9619); 2008.

---

Under review as a conference paper at ICLR 2023
Denis A Areshkin and Branislav K Nikolić. Electron density and transport in top-gated graphene nanoribbon devices: First-principles green function algorithms for systems containing a large number of atoms. Physical Review B, 81(15)155450, 2010.
Shaojie Bai, Zico Kolter, and Vladlen Koltun. Deep equilibrium models. Advances in Neural Information Processing Systems, 32, 2019.
Atilim Gunes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. Automatic differentiation machine learning: a survey, Journal of machine learning research, 18, 2018.
Mads Brandbyge, José-Luis Mozos, Pablo Ordejón, Jeremy Taylor, and Kurt Stokbro. Density- functional method for nonequilibrium electron transport. Physical Review B, 65(16),165401, 2002.
Marius Bürkle, Umesha Perera, Florian Gimbert, Hisao Nakamura, Masaaki Kawata, and Yoshihiro Asai. Deep-learning approach to first-principle transport simulations. Physical Review Letters, 126(17):177701, 2021.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Supriyo Datta. Electronic transport mesoscopic systems. Cambridge university press, 1997.
Filipe de Avila Belbute-Peres, Kevin Smith, Kelsey Allen, Josh Tenenbaum, and Zico Kolter. End- to-end differentiable physics for learning and control. Advances in neural information processing systems, 31:7178-7189, 2018.
Nader Engheta, William D Murphy, Vladimir Rokhlin, and Marius S Vassiliou. The fast multipole method (fmm) for electromagnetic scattering problems. IEEE Transactions on Antennas and Propagation, 40(6):634-641, 1992.
David Ferry and Stephen Marshall Goodnick. Transport in nanostructures. Number 6. Cambridge university press, 1999.
Alexander L Fetter and John Dirk Walecka. Quantum theory of mmay--aatiinn systems. Courier Corporation, 2012.
Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. Brax-a differentiablec physics engine for large scale rigid body simulation. arXiv preprint arXiv:2106.15 1020006.181,... 2021.
Walter Gautschi. Numerical analysis. Springer Science & Business Media, 1997.
Seung-Cheod Han, Jonghyun Choi, and Sung-Min Hong. Acceleration of three-dimensional device simulation with the 3d convolutional neural network. In 2021 International Conference Simulation of Semiconductor Processes and Devices (SISPAD), pp. 52-55. IEEE, 2021.
Mohammed Aziz Harb. Scattering Effects in Atomistic Quantum Transport Simulations. McGill University (Canada), 2019.
Philipp Holl, Nils Thuerey, and Vladlen Koltun. Learning to control pdes with differentiable physics. In International Conference on Learning Representations, 2019.
Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan- and Frédo Durand. Difftaichi: Differentiable programming for physical simulation. arXiv preprint arXiv:19I0. 00935, 2019.
Mike Innes, Alan Fdelman, Keno Fischer, Chris Rackauckas, Elliot Saba, Viral B Shah, and Will Tebbutt. A differentiable programming system to bridge machine learning and scientific computing. arXiv preprint arXiv:
2019.
Carlo Jacoboni. Theory of electron transport in semiconductors: a pathway from elementary physics to nonequilibrium Green functions, volume 165. Springer Science & Business Media, 2010.
10

---

Under review as a conference paper at ICLR 2023
Leo P Kadanoff. Quantum statistical mechanics. CRC Press, 2018.
Muhammad F Kasim and Sam M Vinko. Learning the exchange-correlation functional from nature with fully differentiable density functional theory. Physical Review Letters, 127(12):126403, 2021.
L. V. Keldysh. Diagram technique for nonequilibrium processes. Zh. Eksp. Teor. Fiz., 47:1515-1527,, 1964.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
MV Klymenko, JA Vaitkus, JS Smith, and JH Cole. Nanonet: an extendable python framework for semi-empirical tight-binding models. Computer Physics Communications, 259:107676, 2021.
Dmitrii Kochkov, Jamie A Smith, Ayya Alieva, Qing Wang, Michael P Brenner, and Stephan Hoyer.
Machine learning-acced acceneeaecceeeeeedddddeeeeeeedde computational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21), 2021.
Steven George Krantz and Harold R Parks. The implicit function theorem: history, theory, and applications. Springer Science Business Media, 2002.
PS Krstić, X-G Zhang, and WH Butler. Generalized conductance formula for the multiband tight- binding model. Physical Review B, 66(20):20531, 2002.
Ask Hjorth Larsen, Jens Jørgen Mortensen, Jakob Blomqvist, Ivano E Castelli, Rune Christensen, Marcin Dulak, Jesper Friss, Michael N Groves, Bjørk Hammer, Cory Hargus, et al. The atomic simulation enviromment- python library for working with atoms. Journal of Physics: Condensed Matter, 27227)2773002, 2017.
Kangyuan Li, Junqiang Lu, and Feng Zhai. Neural networks for modeling electron transport properties of mesoscopic systems. Physical Review B, 102(6):06425,, 2020.
Li Li, Stephan Hoyer, Ryan Pederson, Ruoxi Sun, Ekin Cubuk, Patrick Riley, Kieron Burke, et al. Kohn-sham equations as regularizer: Building prior knowledge into machine-learned physics. Physical review letters, 126(3)033401, 2021.
Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. Differentiable monte carlo ray tracing through edge sampling. ACM Transactions on Graphics (TOG), 37(6);1-11, 2018.
Alejandro Lopez-Bezanila and O Anatole von Lilienfeld. Modeling electronic quantum transport with machine learning. Physical Review (9(23)23554 2014.
Momchil Minkov, Ian AD Williamson, Lucio C Andreani, Dario Gerace, Beicheng Lou, Alex Song, Tyler W Hughes, and Shanhui Fan. Inverse design of photonic crystals through automatic differentiation. ACS Photonics, 7(7):1729-1744) 2020.
Fernando Nogueira. Bayesian Optimization: Open source constrained global optimization tool for Python, 2014- /RLhtttss://iithub... com/ fmfn www.f///aaesiian00pp
ion
Taisuke Ozaki. Continued fraction representation of the fermi-dirac function for large-scale electronic structure calculations. Physical Review B, 75(3)0335 2007.
Nick Papior, Nicolás Lorente, Thomas Frederiksen, Alberto García, and Mads Brandbyge. Improve- ments coulequilir and transport green function techniques: The next-generation transiesta. Computer Physics Communications, 212:8-24 24, 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary VeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-BB Buc. E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024 8035, Curran Associates, Inc., 2019.
11

---

Under review as a conference paper at ICLR 2023
Fabijan Pavošević and Sharon Schiffe. Automatic differentiation for coupled cluster methods. arXiv preprint arXiv:2011.11600 2020.
Artem K Pimachev and Sanghamitra Neogi. First-principles prediction of electronic transport in fabricated semiconductor heterostructures via physics-aware machine learning. npj Computational Materials, 7(1):1-12, 2021.
R-E Plessix. A review of the adjoint-state method for computing the gradient of a functional with geophysical applications. Geophysical Journal International, 167(2):495-503 2006.
Lev Semenovich Pontryagin. Mathematical theory of optimal processes. CRC press, 1987.
Pramod Reddy, Sung- Yeon Jang, Rachel A Segalman, and Arun Majumdar. Thermoelectricity in 2007.
molecular junctions. Science, 18)
Ivan Rungger and Stefano Sanvito. Algorithm for the construction of self-energies for electronic transport calculations based on singularity elimination and singular value decomposition. Physical Review B, 78(3):034407, 2008.
MP Lopez Sancho, JM Lopez Sancho, JM Lopez Sancho, and Rubio. Highly convergent schemes for the calculation of bulk and surface green functions. Journal of Physics F: Metal Physics, 15(4): 851, 1985.
Connor Schenck and Dieter Fox. Spnets: Differentiable fluid dynamics for deep neural networks. In Conference on Robot Learning, pp. 317-335. PMLR, 2018
Samuel Schoenholz and Ekin D. Cubuk. Jax m.d. a framework for differentiable physics. In Advances in Neural Information Processing Systems, volume 33. Curran Associates, Inc., 2020.
John C Slater and George F Koster. Simplified Icao method for the periodic potential problem. Physical Review, 94(6))1498, 1954.
Søren Smidstrup. Troels Markussen, Pieter Vancraeyveld, Jess Wellendorff, Julian Schneider, Tue Gunst, Brecht Verstichel, Daniele Stradi, Petr Khomyakov, Ulrik Vej-Hansen, er Quantu- matk: integrated platform of electronic and atomic-scale modelling tools. Journal of Physics: Condensed Matter, (2(1):0159,,, 2019.
Ryan Mohammad Solgi. Genetic algorithm package for python, 2020- URL httts::/tttthttii com/ rtt.//mmllll i g ca g thm
Satofumi Souma and Matsuto Ogawa. Acceleration of nonequilibrium green's function simulation for nanoscale fets by applying convolutional neural network model. IEICE Electronics Express, pp. 17-20190739, 2020.
Satofumi Souma and Matsuto Ogawa. Neural network model for implementation of dlectron-phone scattering in nanoscale device simulations based negf method. In 2021 International Conference on Simulation of Semiconductor Processes and Devices (SISPAD), pp. 56-59. IEEE, 2021.
Sebastian Steiger, Michael Povolotskyi, Hlong-Hyun Park, Tillmann Kubis, and Gerhard Klimeck. Nemo5: A parallel multiscale nanoelectronics modeling tool. IEEE Transactions on Nanotechnol- ogy, (006))1464 1474, 2011.
A Svizhenko and MP Anantram. Effect of scattering and contacts current and electrostatics in carbon nanotubes. Physical Review B, 7288)0085430, 2005.
Teresa Humayo-Mendozze Christoph Kreisbeck, Roland Lindh, and Alán Aspuru-Guzik. Automatic differentiation in quantum chemistry with applications fully variational hartree-fock. ACS central science, 4(5):559-566, 2018.
Jeremy Taylor, Hong Guo, and Jian Wang. Ab initio modeling of quantum transport properties of molecular electronic devices. Physical Review B, 63(24))245407, 2001.
J-S Wang, Jian Wang, and JT Lü. Quantum thermal transport nanostructures. The European Physical Journal 62(4))381 404, 2008.
12

---

Under review as a conference paper at ICLR 2023
Guo-xun Wu, Zhen-qing Wang. Yu-hang Jing, and Chao-ying Wang. I- curves of graphene nanoribbons under uniaxial compressive and tensile strain. Chemical Physics Letters, 559:82-87 -7- 2013.
Markus Zahn. Point charge between two parallel grounded planes. American Journal of Physics, 44 (11):1132-1134 134, 1976.
Zupančić, I Stresec, and M Poljak. Predicting the transport properties of silicene nanoribbons using neural network. In 2020 43rd International Convention Information, Communication and Electronic Technology (MIPRO), pp. 44-48. IEEE.
A ADDITIONAL DETAILS ON THE NEGF METHOD
A.1 COMPUTATION OF THE SELF-CONSISTE SURFACE GREEN FUNCTION
Since the system is made up of device and two semi-infinite contacts on the side, Equation be expanded in the following form:
can
A
ALD
0
Gr.
G
GL.²
ApL
An
ApR
GpL.
Gn
GDR
I,
(10)
0
ARD
AR
[GRl
GRD
Gr
where A= [EI - H]. and the subscripts are used to distinguish the matrix elements corresponding to the left lead (L), the device (D), the right lead (R), and their interactions. Thanks to its block tri-diagonal form, the device Green function GD satisfies
[Ap ADL.A ALD ADnAR AnD|Gn A =I.
(11)
Since Ap = [EI- HD). compared with Equation 2 we have
f = ADLA ALD.
(12)
R ADRAR ARD.
(13)
Σ= ΣーΣ R.
(14)
Here we assume only the neighbouring layers have interactions with each other, and denote the left lead layer connected to the device by L. Then the left self-energy can be simplified as Σ AD Ayp. The coupling matrix App is given as input of NEGF. What remains unclear is A the bottom-right block of A7- This is known as the surface green function, denoted as Js- By utilizing the ideal lead assumption that removing one layer of the lead will not change ga. we obtain a self-consistent form as 3 where Lopez-Sancho algorithm (Sancho et al. 1985) can be applied to accelerate the convergence, here we display the detailed algorithm below:
Algorithm Lopez- algorithm for surface Green function
EÚ repeat
h0,0: 60 h0 hg, ESo, 30 h1,0 ESI.0
<² ci-1 10-11 4-1) 1 1
-
G 4-1 + 3 EES (-11) -1 a--1 + 04-1 11-1-11:-
(FS
= (ES G-1)
-
8 = 8 (ES G-1) -18,11
until converge
9o.0 = (ES-.. e -1
A.2 COMPUTATION OF THE SELF-CONSISTENT ELECTROSTATIC POTENTIAL
Denote the charge densities in the equilibrium and non-equilibrium states as Po and p. and the potential fields from the original neutral and redistributed charges Vo and V. The equilibrium and non-equilibrium Hamiltonian can be expressed as Ho T + Vo. Hneg = + V. where T is the kinetic energy. Poisson's s equation relates potentials to the corresponding charge densities:
13

---

Under review as a conference paper at ICLR 2023
. v(r))V(()) = -p(r), V e(r)VVo(r) - -Po(r)
(15)
Therefore we have V e V(rVV)()(rr)) = -
-o(r)), where AV V Vo is used to correct
the Hamiltonian by Hneg = Ho + AV. The updated Hneg will again used to update AV. Hence self-consistent iteration is constructed:
V (())V(VV(()) = -l() - Po(r)],
(16)
(V2. Va).
Charge densities are necessary input for the above equation. Denote potentials in left and right electrodes as iy and u, (assume i < uy), then the charge density p(r) 전े -- dEG(E). which can be decomposed into equilibrium and non-equilibrium terms:
p(r) = (r) + Pree(T)
(17)
mm [ dEGn(E)
- l dEG 2T
+
(18)
The first integration up to infinity can be computed efficiently using contour integration with the residue theorem. achieved by expanding the Fermi-Dirac function (Ozaki) 2007 Areshkin &
Nikolić 2010). On the other hand, the non-equilibrum charge density Pneg is computed directly by
numerical integration. The density of neutral charges Pa can be computed by setting ii u, 0.
A.3 EXPRESSIONS OF TRANSPORT PROPERTIES
With the NEGF theory, electronic transport properties be derived, such as the transmission probability (T))),, the density states (DOS), the electronic current (7), the equilibrium and
non-equilibrium electronic densities (Pan and Prog); etc. Here list some of the expressions:
T(E) = Trace DOS(E)
(19)
1 TTaae[[m(G((E)).
(20)
π
2e
+00
d.
I =
T(E)[f(E - ui) - f(E - u,),,
(21)
h
2x
1
2T / dFG
1
p(r) = Tm
dFG
(22)
For Equation (21). the integral range of the current is decided by the subtraction of the Fermi-Dirac function, which a little wider than (w, u.))
B ADDITIONAL DETAILS ON EXPERIMENTAL SETUP
The experiments are run on an Intel(() Xeon(R) CPU E5-2650 @ 2.20GHz CPU, and an NVIDIA Tesla P40 GPU. We implemented simulation results by comparing with ASE version 3.22.0.
method in PyTorch 1.9.1. We validated the correctness of our
In the experiments, set the learning rate of the Adam optimizer as 0.001, and the batch size as 64. Bayesian optimization is implemented based on NNgueira (2014-). and the genetic algorithm is implemented based Solgi (2020-). The bounds of the optimization variables for the black-box optimizers are (fg - 0.3, 6g 0.3), where On is the initial value, namely the original 5-2 nano-junction TB Hamiltonian for the transmission curve fitting experiment, and undoped SKTB parameters for the device doping optimization experiment.
The parameeess of the genetic algorithm are:
max iteration" ": None,
14

---

Under review as a conference paper at ICLR 2023
Runtime Simulation
Tracking
tme
15
0-
atomms
(a) Forward simulation runtime cost against the system size.
Runtime of Backpropogation of 1D Soabock Coefficient
Runtime Bockpropagation for high-Dimentions. Hamiltonian:
x
120
30
x
80-
X
10
atoms
atooms
(b) Backward ID derivative computation runtime cost (c) Backward multi-dimensional derivative computa- tion runtume cost against the system size.
against the system size.
Figure Runtime cost of forward and backward computation.
"population size", : "; 20,
"mutation_probabiliiy 0.1, "elit_ratio ": 0.01,
"crossoeer.rrobbblliy""" 0.5,
pprttion 0.3.
"crossover.type". 'uniform
"max mmax_iteration. iipootttttttorrrrr""""" ": None
The hyper-parameterr parameters of the Bayesian Optimization algorithm are:
handom_state "; 3. " "verbose": ": 2,
"kind": "ucb".
"kappa"; 2.5
"xi": 0.0
We have uploaded source code in the supplementary materials for cross-checking. The code will also be released and maintained as an open-source repository in the future.
C SCALABILITY AND RUNTIME ANALYSIS OF AD AND FD
In this section, we present the test results and analysis the scalability and runtime for both the forward simulation and the backward gradient calculation of AD-NEGF, compared against traditional numerical differentiation. The test system we use is AGNR. By controlling the length of AGNR, the number of atoms contained in the system changes accordingly, that test the performances
15

---

Under review as a conference paper at ICLR 2023
with respect to the system scale. The time cost is measured by the CPU time with functions provided by the python standard library.
In the forward simulation test, compute the transmission function T(E) the same in Section 5 Since a differentiable solver requires tracking each conducted operation in the automatic differenti- ation tool, which brings extra computational cost compared to conventional simulation programs, and turning off such operation tracking functionality will make the differentiable solver behave just like traditional solver in forward simulation, we compare the CPU time cost for AGNR systems in different sizes, with both the operation tracking functionality turned on, and with the functionality turned off. The results are displayed in Figure ((a) from which we can conclude that, the CPU time cost scales linearly with the system size, and the additional computational cost brought by operation tracking is almost negligible.
In the backpropagation test, we compare the runtime cost of calculating derivatives for both 1- dimensional variables (the Seebeck coefficient) and multi-dimensional variables (device Hamilto- nians). The results for both cases are shown in Figure 7(b) and [(c) respectively. In the Seebeck coefficient calculation experiment, the derivative is computed as TT(E) / where we sample 100 energy points in each run. We can see the computational cost of both FD and AD scales linearly with the system size. Since the backpropagation computation is more complex than the forward computation, the slope of AD is bit larger than FD. However, in such cases, efficiency will not be the bottleneck anyway, while as demonstrated Section 5 AD can achieve higher precision than FD. For the high-dimensional variable gradient computation, as shown in Figure [7(c]
AD significantly outperforms FD in computing gradients for the device Hamiltonians of dimension N where NA the number of atom orbitals. This is because, FD can only compute the derivative for one dimension of the input variable each time, making its complexity grow in O(N)). On the other hand, the complexity to compute gradients by backward mode AD depends only on the complexity of the forward simulation, regardless of the increasing input variable dimensions.
In conclusion, by utilizing differentiable programming techniques, AD-NEGF, as an end-to-end differ- entiable quantum transport simulator, computes gradients for high-dimensional variables significantly faster than FD, while merely increasing the forward simulation cost.
16


Reviews:
[{"review_id": "ue0lSWcJIf", "reviewer": "Reviewer_uefF", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to use auto-diff to compute the gradients of the Non-Equilibrium Green Function (NEGF) to solve the quantum transport problem. Numerical differentiation methods are more expensive and can result in numerical errors that could lead to wrong estimations of the physical properties of the system. Moreover, while using Neural Networks as surrogate models is a possible path to achive faster solutions, but because they are trained from data, there's no guarantee that the derived results will be accurate. The paper moves on to briefly describe the NEFG method and specific techniques they employ to compute its derivatives. Finally, the AD-NEGF method is compared against numerical differentiation, showing the correctness of AD-NEGF on one side, while demonstrating the possible pitfalls of relying on numerical differentiation. In addition to establishing the basic benefits of the proposed method, additional experiments are carried out on other settings within this domain.", "summary_of_the_review": "My main concern is that the paper might not be a good fit for the ICLR audience unless a major revision is provided to make it more accessible. While the proposed method belongs to the intersection of ML and Physics and so it could in principle find an audience in ICLR, more space should be spent on introducing the problem to those new to it.", "strength_and_weaknesses": "Strengths:\n* The first use of autodiff for solving the quantum transport problem that can alleviate many of the problems of earlier approaches.\n* Introducing a new topic for the ML community.\n\nWeaknesses:\n* The problem this paper aims to solve is too obscure for the ML audience. It would not be a problem if it were introduced more clearly in order to make it more accessible. As a reader with a bit of background in quantum physics but no familiarity with the quantum transport problem, I cannot say that I have a clear understanding of what exactly the authors are trying to solve.\n* The comparison to prior works is also a bit unclear. What exactly are black-box methods in this context? It would be useful to provide specific citations to what you are comparing against. Also, What is considered today to be the best performing method and how it compares? Finally, considering that one of the claimed benefits of your method is efficiency, comparing FLOPs or runtime would be crucial. (Beyond the time plot in Figure 6)\n* A more minor note is the overuse of initials. It's already hard to follow a new subject, but having so many unfamiliar intials makes reading the paper even harder. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Oj4jtu6jv2C", "reviewer": "Reviewer_FLBL", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper study the problem of quantum transport simulator. It implement an automatic differentiable quantum transport simulator which can provide gradients for sensitivity analysis and solving inverse problems.", "summary_of_the_review": "A differentiable simulator for quantum transport.\n\nThanks for the response. I agree that the work is under the category of AI4Science and is a good infrastructure. So I would like to increase score to 6.", "strength_and_weaknesses": "Strength\n1. A automatic differentiable simulator is useful for scientific study of the quantum transport.\n\nWeakness:\n1. There is no theoretical novelty of the proposed simulator. It is expected that the gradient based optimization outperforms the BO and Genetic algorithm. ICLR may not be a suitable venue for the work.\n2. More discussions on the scalability and runtime is needed to understand the efficiency of the simulator.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "NFkrFsJ-hC", "reviewer": "Reviewer_4fBj", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors proposed a novel quantum transport simulator, AD-NEGF, that leverages automatic differentiation with Non-Equilibrium Green Function method. The proposed model uses implicit gradient method to accelerate backward gradient computation and is validated via different application scenarios against numerical differentiation and black-box optimization.", "summary_of_the_review": "The paper presents a novel quantum transport simulator which is end-to-end differentiable. The proposed model is validated via experiments on three different applications and the results seem to support the claims. Despite that the direction of this paper does not entirely fit in the topics of ICLR, I think the quality of the paper meet the bar for this venue.", "strength_and_weaknesses": "Strength\n\n* The proposed method is novel and is validated on three different applications.\n* The discussion of related works seems extensive.\n\nWeaknesses\n\n* Only qualitative analysis of the empirical result are presented.\n* More ablation study is needed to support some of the claims (e.x. the complexity of numerical differentiation and automatic differentiation w.r.t. input and output dimensions). ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "6iICLUEWYrI", "reviewer": "Reviewer_5rbi", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "****This is an emergency review for the paper 2820****\n\nThe paper is about Automatic Differentiation (AD) based quantum transport simulator, and the paper proposes the AD-NEGF algorithm. Key ingredients of the new algorithm efficient gradient calculation by utilizing automatic differentiation and implicit layer techniques and guarantees of correctness of the forward simulation.  The algorithm is used to calculate differential physical quantities and solving inverse problems that are intractable by traditional optimization methods.", "summary_of_the_review": "Out of scope for ICLR. While this should be used sparely (and I'm not comfortable either using it), I find it hard to place this paper in a session in ICLR. Further, the emergency of this review definitely harms the review quality, but I feel strongly that the same outcome would be even if I had a couple more days to review this paper. The idea of automated differentiation for quantum optimal transport could be interesting in other communities, but does not feel a good fit for ICLR. This justifies this score.", "strength_and_weaknesses": "Strength:\n- (I could not identify the strengths of the paper ---other than the contributions stated--- since it is not easy to parse the paper. See below).\n\n\nWeaknesses:\n- The paper does not read as a great fit for the ICLR conference. While the conference is inclusive to new ideas and techniques, the paper focuses on the quantum transport simulation problem, and studies again the non-equilibrium Green function. The reviewer strongly believes that the paper would make a great fit into a more quantum-related conference. It is not clear to the reviewer how this paper would be of relevance to the audience of ICLR (which is mostly focusing on ML/AI topics). This definitely is a negative point for this paper (which might be unfair for its content).\n- To the best of the reviewer's understanding, the introduction uses keywords and language that are not known to the average ICLR reviewer: E.g., a sentence that is hard to parse is \"The entire numerical process of the quantum transport simulation is implemented in PyTorch, including the computation of the self-energy, the Green function, the electrostatic potential, the transport properties, as well as an optional Slater-Koster Tight-Binding (SKTB) module to generate the block tri-diagonal Tight-Binding (TB) Hamiltonian (Klymenko et al., 2021).\" It is not easy for most of the reviewers in ICLR to follow this language in order to appreciate the contributions of this work. That's why the reviewer strongly believes that this paper should resubmitted to a more appropriately selected venue. See also \"We propose and implement AD-NEGF, as far as we know the first end-to-end differentiable quantum transport simulator, including the NEGF method, the Poisson’s equation module for self-consistent electrostatic potential computation, and the SKTB module to generate the tight-binding Hamiltonian from the coordinates and properties of the system atoms\".\n- While the reviewer believes that the math is correct, he/she could not follow the math steps in equations eq. (1)-(4), what they mean, how they are derived and how these are computationally difficult to compute, etc. While introducing new optimization problems in ML/AI venues is more than welcome, that should also come in the form that is easily understandable by the corresponding audience. In other words, it could be the case that the paper could be written in a way that introduces an abstract problem and its efficient solution, and then connect it with the problems considered from the physics perspective. \n- All experiments and results are better suited to a physics or materials science or systems/hardware electrical engineering venue, rather than ICLR. They are not easily appreciated for the problems considered in the current draft.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject