# Research on Device Modeling Technique Based on MLP Neural Network for Model Parameter Extraction
## 3 Training and Testing the MLP Neural Network  
This MLP neural network leverages the backpropagation (BP) algorithm for training. The training process is divided into forward propagation and backpropagation. Forward propagation is a series of linear and activation operations using the hyperparameters with the inputs, and the output results are calculated backward from the input layer. Backpropagation uses the output layer error to optimize the hyperparameters [26,27]. The NN employs the adaptive momentum estimation (Adam) algorithm to accelerate the convergence [28], and the initial learning rate of the Adam algorithm is set to 0.01. The mean square root of the relative error (RR) is defined as the performance metrics, which is defined as follows:  
\[RR\ =\ \sqrt{\frac{1}{N}\sum_{i\ =\ 1}^{N}\big{|}\frac{I_{in}^{i}-I_{sim}^{i}}{I _{sim}^{i}}\big{|}^{2}} \tag{3}\]  
where \(I_{sim}^{i}\) denotes the current value of SPICE simulation and \(I_{in}^{i}\) represents the predicted value of the NN. The IV curves can be divided into subthreshold and non-subthreshold regions according to the relationship between \(V_{gs}\) and the threshold voltage (\(V_{th}\)). The relationship between the current and voltage in the subthreshold region is exponential, and the equation is provided as follows:  
\[I_{ds}\ =\ \mu_{0}\times\frac{W_{eff}}{L_{eff}}\times\sqrt{\frac{q\varepsilon_{ si}N_{ch}}{2\phi_{s}}}\times v_{t}^{2}\times\left(1-\exp\!\left(-\frac{V_{ds}}{v_{ t}}\right)\right)\times\exp\left(\frac{V_{gs}-V_{th}-V_{off}}{nv_{t}}\right) \tag{4}\]  
where \(\mu_{0}\) represents the mobility, \(W_{eff}\) is the effective channel width, \(L_{off}\) is the effective channel length, \(q\) is the electronic charge, \(\varepsilon_{si}\) is the dielectric constant of silicon, \(N_{ch}\) is the substrate doping concentration, \(\phi_{s}\) is the surface potential, \(v_{t}\) is the thermal voltage, \(V_{off}\) is the offset voltage, and \(n\) is the subthreshold swing parameter. The relationship between the current and voltage in the non-subthreshold region is linear, and the equation is provided as follows:\[I_{ds}\ =\ W_{eff}v_{sat}C_{ox}\times\left(V_{gs}-V_{th}-A_{bulk}\times V_{ dest}\right)\times\left(1+\frac{V_{ds}-V_{dest}}{V_{A}}\right)\times(1+\frac{V_{ds}-V_{ dest}}{V_{ASCBE}}) \tag{5}\]  
where \(v_{sat}\) is the saturation velocity, \(C_{ox}\) is the gate oxide capacitance, \(A_{bulk}\) the body charge effect parameter, \(V_{dest}\) the drain-source saturation voltage, \(V_{A}\) the Early voltage, and \(V_{ASCBE}\) is the substrate current-induced body effect (SCBE) corresponding to the Early voltage [29]. Among the common NN activation functions, sigmoid types are used in preference, as they have smooth derivatives to meet the requirements for high-order derivatives in circuit simulations. Among the sigmoid-type functions, the commonly used ones are log-sigmoid and tanh-sigmoid functions, which can map (\(-\infty\), \(+\infty\)) inputs to the steps (0, 1) and (\(-\)1, \(+\)1), respectively. The equations of log-sigmoid and tanh-sigmoid functions are shown in Equations (6) and (7), respectively. \[log-sigmoid\ =\ \frac{1}{1+e^{-x}} \tag{6}\]  
\[tanh-sigmoid\ =\ \frac{e^{x}-e^{-x}}{e^{x}+e^{-x}} \tag{7}\]  
Two activation functions are used to model the device separately. From Figure 3, we can observe that log-sigmoid fits better than tanh-sigmoid. Therefore, the activation function of this NN eventually uses the log-sigmoid. During the forward propagation of the NN, log-sigmoid controls the data within 0 and 1, and the data change smoothly without data diffusion, so log-sigmoid obtains a higher model accuracy. Some criteria were used in this study to better estimate the trained NN models and avoid overfitting. Those criteria include the Akaike Information Criterion (AIC), the corrected Akaike Information Criterion (_AICc_), and the Bayesian Information Criterion (_BIC_). The _AIC_ method attempts to find the model that explains the data best with the smallest number of parameters.