—Machine learning (ML) compact device models (CM) h ave emerged as an alternative to physics-based CMs. ML CMs can find a mathematical model close to the device chara cteristics without much prior knowledge, which save s the time of model formation. Additionally, versatile capabilities such as proces s-awareness, model merging, and fitting new technol ogies, promote the usage of ML CMs. While ML CMs draw great attention in CAD, thei r convergence in SPICE has not been carefully studi ed. Here different activation functions are used to create ML CMs, and then the c ircuit convergence is tested. We found that inverse square root unit (ISRU) activation has the best convergence. Besides, gate-to-source a nd gate-to-drain capacitance is founded to benefit the convergence in transient analysis. The circuit convergence rate is 100% for ISRU, sigmoid, and tanh when the capacitor is prese nt. On the other hand, ISRU significantly outperforms other activation function s in DC sweep, achieving 81% convergence. If quasi- static transient analysis is employed to replace DC sweep, 100% convergence is a chieved by ISRU. Due to its superior convergence, ISRU is the most p romising for future ML CMs in SPICE. Index Terms —semiconductor device simulation, machine learning, CMOS, device models I. INTRODUCTION With the evolution of the times, many people pay at tention to machine learning. In the early days, hum ans were not interested in machine learning because machine learning models co nsume a lot of computing power reference to physica l models. Nowadays, as more data or more complex phenomena need to be p rocessed, it is more difficult for physical models to process it in a reasonable manner or in a short time. Metal oxide semiconducto r field(effect transistor (MOSFET) is the most basi c semiconductor device in circuits. Semiconductor device becomes smaller with Moore's Law, and many characteristics are not well described in emerging areas, which leads to a complex physical model. Bes ides, the physical model is increasing complexity, so the formation of the physical model takes a lot of time and labor costs[ 1]. This aspect can cause circuit design companies to be unable to use the most advanced devices to design the circuit in time. As the size of MOSFETs shrinks, the BSIM model has bec ome more complex. When the oxide layer is thicker on the MOSFETs, we do not need to consider the gate leakage current an d gate(induced drain leakage (GIDL) effect. At this time, the BSIM3 mode l is relatively simple. When the oxide layer become s thinner on the MOSFETs, the device has a gate leakage current and GIDL effect. The BSIM model is expanded with the ga te leakage current model and GIDL model [2]. The complexity of the BSI M model also increases with the advanced process. T o prevent the short channel effect, halo implantation is available in t he process, which makes the electric field and para sitic resistance more complicated [3, 4]. At the same time, the halo impl antation is also added to the BSIM model[5]. Afterw ard, since the shallow trench isolation (STI) process affects threshold vo ltage and mobility, so the stress effect model is i ntroduced after the BSIM4 model [6]. It can be seen that the device model wil l become complicated in nano(devices, which leads t o a lot of time to compose a physical model. Therefore, researchers are lookin g after using machine learning to construct a devic e CM. The machine learning model is a neural network trai ned by forwarding propagation and backward propagat ion [7]. The nonlinearity is formed by activation functions. Spe cifically, multilayer perceptron (MLP) is a mathema tical model which uses a matrix to create an equation. We can adjust the num ber of neurons and hidden layers, input/output para meters, and activation function. The number of neurons, the number of hidd en layers, and the activation function can determin e the accuracy and complexity of the model. While physical models requ ire repeated modifications, which cost engineers a lot of time to build physical models. In contrast, engineers only need a database , and then machine learning models can be composed. Among the architecture parameters in ML models, the activation functions p lay an important role in convergence. Thus, in this work, we study the effect of different activations on circuit convergence. In literature, there have been some works using ML CMs in semiconductor technologies[1]. Chea(Wei Teo et al. uses the parameters of TCAD and the random forest method to predict the location of the defect[8]. Himmel et al . uses machine learning to complete the etching model [9]. Sidi Bel Abbes e xtracts the critical parameters of MOSFET and then uses MLP to predict the current value [10]. Creech et al. construct a ML C M for microave devices [11] Chen et. al. use ANN to form a compact model of TFTs, and then V gs and V ds , gate width ( W), and gate length( L) are used to predict the current value [12]. Gao et. al. also use NN to form a device model[13]. Huang et. al. proposed dimension(reduced ANN to red uce the problem of overfitting[14]. Kim uses physics(based loss function and then uses backpropa gation to train the device model, and this practice can make the model prediction more accurate [15]. Habal uses voltage and gate length to predict the c urrent value and g m accurately[16]. While ML has been used extensively in semiconductor process and used in several cases in semiconductor device CMs construction, the application of ML device CMs in SPICE has not preva iled in literature, and few people discuss the appl ication of machine learning models to circuits simulation [17(22]. Samsung and Sandia's paper discusses the accuracy of the ML CMs [18, 19]. V gs and V ds are 0V to 1V with adaptive steps in Sandia's paper. It can be known from the result that when the high (density step value is extracted in the subthreshold, this can make the mo del prediction more accurate. Samsung's paper propo sed that the simulation speed of the ML model is slower than the simulation speed of the traditional model. Hammouda also used ANN and Taylor series to compose a device model, and the device model was put into the circuit simulation. Nevertheless, Hammouda did not mention the convergence issue of ML models in circuit simul ation [20]. We choose HSPICE to study the circuit convergence u sing ML CMs because HSPICE is the most popular SPIC E used by industry or academia. We use Verilog(A to build device CMs [ 23]. Because Verilog(A has defined the fundamental laws, it can save a lot of time for model developers and allow model develo pers to focus on the device models [24]. While many papers use machine learning to form device models , there are only a f ew papers that really use machine learning models i n circuit simulation s [17( 22]. Here the focus is on the application of ML CMs to circuits and proposing an activation function f or high convergence. II. METHOD A. Data collection BSIM(CMG 110.0.0 is used as the baseline to compare the convergence with ML(based CMs[25]. The ML CMs are fitted to BSIM(CMG IV. V gs , V ds, and the number of fins are the parameters in BSIM(C MG. When collecting the parameters of BSIM( CMG 110.0.0, the gate length is 30 nm, and then we can adjust the number of fins. There are 20 kinds o f number of fins from 10 to 200. Subsequently, the fitting is carried in pyt hon and Tensorflow. To test the convergence of the circuit, we choose different kinds of activation functions. There are three diff erent types of activation functions [26]. There are ReLu and ISRLU for the discontinuous type and unrestricted output value ac tivation functions. Softplus and Swish possess cont inuous unrestricted output values. Softsign, ISRU, Sigmoid and hyperbolic tang ent possess continuous and restricted output values . Finally, the model is incorportaed into HSPICE using Verilog(A syntax. Vgs and V ds of BSIM(CMG 110.0.0 are set from 0V to 1V with 50m V step for n(MOSFET. On the contrary, V gs and V ds of BSIM(CMG 110.0.0 are set from 0V to (1V with 50mV s tep for p(MOSFET. We only consider the electrical c haracteristics caused by the gate and drain and source, and the base shou ld be connected to the source in circuits. Each dev ice has 441 data, so there are 8,820 data for n(MOSFET and p(MOSFET, respectively. B. Machine learning model formation After collecting the parameters, the machine learni ng model is formed in python 3.7.6 [27]. The traini ng model uses MLP to train the device model. Function of MLP is referenc ed from the Tensorflow 2.0.0[28], Numpy 1.21.2[29], Scikit(Learn 0.24.2 [30], Matplotlib 3.3.1[31], Pandas 1.1.1. First, the data can be shuffled. 90% of data is used as the traini ng set, and10% is used as the test set. When forming the BSIM(CMG model, the inpu t parameters are V gs , V ds, and the number of fins. The number of hidden layers is three, and the number of neurons is fifte en in each hidden layer. The activation functions u se ReLu, softsign, hyperbolic tangent, sigmoid, softplus, ISRU, ISRLU, Swish, as expressed by (1a)((1h) [32]. ( ) max(0, ) f x x = (1a) ( ) 1xf x x=+ (1b) ( ) Tanh( ) x x x x e e f x x e e − −−= = + (1c) 1( ) 1xf x e−= + (1d) ( ) ln(1 ) xf x e = + (1e) 2( ) 1xf x x= + (1f) 2(0 ) ( ) 1 (0 ) xx f x x x x >=+ ≤  (1g) ( ) 1xxf x e−= + (1h) i i i 1(Activation( )) n iy v w x b c == + + ∑ (2) When the device model is trained, the data should b e normalized first. The machine repeats gradient de scent and weight update until the gradient of the loss function does not decrease , early stopping can be set. After the model is trained, the training data and t esting data are fed to the models, and root mean sq uare error (RMSE) between the estimated and actual values. This can determine the accuracy of the model. 2 i(real) i(predict) 1( ) RMSE n iy y n=− =∑ (3) C. Verilog-A expression First, the device name, and the device nodes, and t he device parameters are defined [23, 33, 34]. Then the weights of the ML model are introduced into the verilog file. MOSFET has three nodes. Device parameters are V ds , V gs , and the number of fins for CMG(MOSFETs, and L is uniformly 30nm in this work Loop expression can form a layer of neuron calculat ion, and the number cycles of the loop are determin ed by the number of neurons in a layer. Since the ML model of output va lue is standardized, it needs to be returned to the original value after the model calculation is completed. Finally, we use the curre nt expression to complete the device model. The out put value of the ML model is current in unit of ampere. While we only fit the characteristics of the first quadrant of V ds (Vgs , which can cause the circuit simulation to be distorted to the third quadrant. T herefore, we add the third quadrant current in Veri log(A by flipping the current values for negative V ds since the device is symmetric in structures. D. HSPICE simulation We use CPU Intel core i7(4790 3.6GHz to test the si mulation time of the circuit. The circuit simulator uses the Newton(Raphson method to find the convergence value. If roots of t he system of non(linear equations can be found quic kly, the function needs to be continuous and smooth. Machine learning has many activation functions with different properties, an d each activation function has different smoothness and continuity. As a resul t, the selection of activation functions has critic al effect on the circuit convergence. After constructing Verilog(A models, t he device CMs are introduced into HSPICE. We chose a logic gate with a more complicated configuration to test the converge nce. As shown in Fig. 1 XOR, instead of common NAND , NOR, or NOT, gate is used. The speed of the ML models are also compar ed in addition to the convergence. When we simulate the circuit, the high potential op erates at 1V, and the low potential operates at 0V. First, Matlab is used to generate four hundred circuit files with different proportions of device sizes. Then we use batch file s to automatically run circuit tests with 400 different ratios of device sizes. Af terward simulation, Matlab R2021a is used to read t he waveforms from Cscope. Hspice Toolbox for Matlab and Octave is used so tha t Matlab can read HSPICE output files[35]. Fig. 1. XOR circuit and Truth table. We use transient analysis or DC sweep to test the convergence and simulation speed. In transient simu lation, the total transient time is 10ns using sinusoidal stimuli. The amplitud e is 0.5V with 0.5V offset. The frequency is set to 2GHz/1GHz for two inputs. We use 400 different n(MOSFET and p(MOSFET ratios t o test the convergence of circuits in transient ana lysis. Then the circuit convergence ratio of each activation function can b e displayed together with the simulation time. Afte rward, we can choose three activation functions with better convergence, and t hen using these activation functions to test the su bsequent analysis in HSPICE. VDD A B A BA B Output 0 0 0 0 1 1 1 0 1 1 1 0Output AVDD BVDD The gate to source and gate to drain capacitance, C gs and C gd , is added to the device model, and then the circui t convergence can be tested again. After transient simulation, the DC sweep is then te sted with the scanning range of the DC sweep from 0 V to 1V at two input terminals with 0.1V step. Because the convergence i n DC sweep is not 100% for any of the activation fu nctions, we use quasi( static transient analysis to replace the DC sweep a s illustrated in Fig. 2. The maximum step size of t ransient analysis is set to 0.016sec. To improve the convergence, the input wav eform is carefully ramped. Input A is a triangular wave, and input B will slowly rise from 0V to 1V. When the voltage of inpu t A is from 0V to 1V, the voltage of input B is fix ed to a value. When the voltage of input A changes from 1V to 0V, the volta ge of Input B is slowly raised by 0.05V. When the v oltage of Input B reaches. The calculation is repeated until 400 different W/L ratios are simulated. Fig. 2. Quasi(static transient analysis is used to replace the dc sweep. E. Capacitance model In our ML CMs, increased capacitance of C gs and C gd in general improves the convergence of the circuit . This is a positive, desired phenomenon, and the problem is that what is the range of C gs and C gd values in realistic devices. C gs and C gd values can be reference from literature [36(43]. The reasonable c apacitance values, based on our literature review, fall into the orders of magnitude in 1 ×10 (8F/m ( 1×10 (12F/m. In fact, the C gs and C gd consists of overlap capacitance (C ov ) and channel capacitance (C ch ). Cov is a fixed value while C ch is bias dependent. To estimate the worst cases and to see the effect of capacitance orders of magnitu de, we ignore C ch , and only retain a fixed(value C ov . Different orders of magnitude of C gs = C gd =C ov are tested, and convergence performance is recorded. III. RESULTS AND DISCUSSION A. Fitting ability of ML model It can be known from Fig. 3 and Fig. 4 that the ML model can accurately predict the current values of the MOSFETs. Different activation functions can also lead to different acc uracy in ML model predictions. It can be known from Table I. that the ML model using Swish, the fitting capability is the best for n(MOSFET. On the other hand, ISRLU fits p(MOSFET t he best. It can be found from MAPE that the activation functions with larger output ranges has a better fitting ability. On the contrary, the activation functions with limited output ranges can be less ef fective in fitting. When the input of Swish and ISR LU is >>0, the output value has a linear rise. This can make the output values relatively large, and the fitting ability is better . In addition, it is observed that the prediction accuracy of n(MOSFETs is better than tha t of p(MOSFETs. To test the predictive ability of t he ML model for p( MOSFET, we have tried different loss functions in ( 4). 2 10 Re 10 Pr 1 2 Real Pr 1(log ( ) log ( )) ( ) n al edict i n edict iI I n I I Wn= =− −−∑ ∑ (4) When W is 0, (11) is equivalently log(I) fitting. W can also be adjusted, from 1 to 10 10 , to form a balanced fitting between linear and log scales. Based on our trial the results, it can be found that different loss functions are stil l difficult to improve the MAPE of the ML model for p(MOSFET. Using W from 1 to 10 10 improves the train set MAPE but does not have much effect on the test set data MAPE. It is seen that the predictive ability o f the ML model is relatively poor in the vicinity o f 0V. We have also tried to use all the data into training, and then the accuracy o f the ML model prediction can be improved. Neverthe less, when all of the data is put into training, overfitting can exist. Thus, we finally use the original loss function ( W=0), and 90% of the data is used as training sets. Input Voltage A Input Voltage B 0V 0.2V 1V 0V 0.16 0.32 0.48 0.64 1.12 1.28 0.8 0.96 Time(sec) Input voltage 0.05V 0.1V 0.15V Table I MAPE OF ML MODELS Activation function n(MOSFET p(MOSFET Sigmoid 2.76% 10.69% ISRU 1.38% 10.52% ReLu 3.05% 7.04% ISRLU 1.02% 6.78% Swish 0.73% 9.47% Softsign 1.58% 9.66% Tanh 1.38% 9.33% Softplus 1.78% 9.39% Fig. 3. The scatter plots for different activations (a)((b ) Sigmoid (c)((d) ISRU (e)((f) Softsign (g)((h) ReL u (i)((j) ISRLU (k)((l) swish (m)((n) Tanh (o)((p) Softplus. -7.88 0 Target(mA) -7.88 0 Predict Ideal-7.88 0 Target(mA) -7.88 0 Predict Ideal -7.88 0 Target(mA) -7.88 0 Predict Ideal -7.88 0 Target(mA) -7.88 0 Predict Ideal Prediction(mA)0 11.7 Target(mA) 011.7 Predict Ideal 0 11.7 Target(mA) 011.7 Predict Ideal (a) (b) (c) (d) (e) (g) (i) (f) (h) (j) p-MOSFET p-MOSFET p-MOSFET p-MOSFET p-MOSFET n-MOSFET n-MOSFET n-MOSFET n-MOSFET n-MOSFET n-MOSFET p-MOSFET (l) (k) n-MOSFET n-MOSFET p-MOSFET p-MOSFET (o) (p) (n) (m) (d) (c) (a) (b) (h) (g) (e) (f) Vgs=0.2V,0.4V,0.6V,0.8V,1V Vgs=0.2V,0.4V,0.6V,0.8V,1V Vds=0.2V,0.4V,0.6V,0.8V,1V Vds=0.2V,0.4V,0.6V,0.8V,1V Vds=-0.2V,-0.4V,-0.6V,-0.8V,-1V Vds=-0.2V,-0.4V,-0.6V,-0.8V,-1V Vgs=-0.2V,-0.4V,-0.6V,-0.8V,-1V Vgs=-0.2V,-0.4V,-0.6V,-0.8V,-1V Fig. 4. The ML CMs fitting to BSIM(CMG. (a)((b) n(MOSFET, L=30nm, NFIN= 200. (c)((b) p(MOSFET, L=30nm, NFIN= 200. (e)((f) n(MOSFET, L=30nm, NFIN= 100. (g)((h)p( MOSFET, L=30nm, NFIN= 100. Activation function is I SRU. Fig. 5. The contours of simulation time (a) Sigmoid, (b) I SRU, (c) ISRLU, (d) Softsign, (e)Tanh, and (f) BSIM . Fig. 6. The scatter plots of the cases w/o convergence, fo r (a) Sigmoid, (b) ISRU, (c) ISRLU, (d) ReLu, (e) S oftsign, (f) Swish, (g) Tanh, and (h) Softplus. B. Convergence of ML model for transient analysis Before we test the convergence of the circuit, we c heck whether the current is minimal to V gs less than 0 and whether the current has an explosive current to V gs greater than 1V. The inspection result shows that the current is smaller than 1 ×10 (10A for V gs less than 0V, and the current is not explosive for V gs greater than 1V. Therefore, in case the node volta ge during New(Raphson iterations n-MOSFET fins(a) (e) (b) (d) (f) (c) (h) (g) n-MOSFET fins 1020 40 60 80 100 120 140 160 180 200 p-MOSFET fins 10 20 40 60 80 100 120 140 160 180 200 1020 40 60 80 100 120 140 160 180 200 p-MOSFET fins 10 20 40 60 80 100 120 140 160 180 200 is accidentally entering a voltage range outside tr aining, there should not be exploding value problem s. As a result, the divergent circuit simulations are more likely due to the ML C M itself. Before we consider the third quadrant current, we h ave tested XOR, NAND, NOR, and Inverter. The test r esult is that the circuit with XOR has the worst convergence, because the XOR circuit is more complicated. The circuit convergen ce of NAND, NOR, and Inverter is much improved, but it still follows different activation functions, leading to differe nt circuit convergence. In general, activation functions with smooth co(domain, smooth derivatives, and bounded co(domain can lead to bett er convergence. As far as XOR is concerned, circuit simulation is unable t o converge if the third quadrant current in transis tor is not specified in ML CMs. Therefore, we added the current in the third quadra nt to the ML CMs assuming symmetric device structur es, and then the circuit convergence was tested again. The test results show that ISRU, Sigmoid, Softsign, and hyperbolic tange nt SPICE convergence are satisfactory in transient analysis, as shown in Tab le II. The correlation between the average speed in SPICE and convergence, for different activation functions, is actually not ver y pronounced. The results in Table II show that the average time of circuit simulation of ReLu and ISRLU is relatively fast, bu t their circuit convergence is rather poor. When we look for a suitable activation function fo r HSPICE, we expect to find an activation function that possesses a continuous function values and derivatives. If an activation f unction has a discontinuity, which can cause the ci rcuit to diverge at the moment of potential transition, such as ReLu. The ReLu fun ction has a discontinuous first derivative at zero, making it impossible to be differentiable. ISRLU function is continuous until the second derivatives, but carefully processed con tinuity does not improve the SPICE convergence[44]. Another problem of activati on functions in SPICE is an unbounded function valu es, which can cause convergence problems. This is seen in RELU, ISRLU, Swish, and Softplus. Because the value changes dram atically, and errors can accumulate and propagate, which can cause diver gence. ISRU, Sigmoid, and hyperbolic tangent are co ntinuous and differentiable functions without piecewise definiti ons. Besides, the function values are bounded, so t he convergence of ISRU, Sigmoid, Softsign, and hyperbolic tangent is better than ReLu, ISRLU, Swish, and Softplus. The derivat ives of ISRU, Sigmoid, Softsign, and hyperbolic tangent are in (5a)((5d), respectively. Table II ML MODEL WITHOUT CAPACITANCE (TRANSIENT TIME :10 NS , SINUSOID WAVE , THE AMPLITUDE OF 0.5V, THE FREQUENCY OF AC VOLTAGE IS 2GH Z AND 1GH Z). Different activation function of ML model and BSIM Convergence ratio Total convergence /Total test Simulation time BSIM 100% 400/400 0.52sec Sigmoid 96.25% 385/400 3.66sec ISRU 96.75% 387/400 3.38sec ReLu 0.75% 3/400 2.78sec ISRLU 56% 224/400 2.78sec Swish 0% 0/400 failed Softsign 74.75% 299/400 4.54sec Tanh 99.50% 398/400 3.79sec Softplus 0% 0/400 failed ' 3 21( ) ( ) 1f x x= + (5a) ' 2( ) (1 ) x xef x e− −= + (5b) ' 21( ) (1 ) f x x= + (5c) ' 2 ( ) 1 ( ) x x x x e e f x e e − −−= − + (5d) From Table II, it can be found that the ML models o f all activation functions are slower than the BSIM model in the circuit simulation [18, 19]. Nevertheless, the BSIM model has many capacitors. The capacitors have differenti al expressions so that the variation in current and voltage is slowed down, wh ich is beneficial for circuit convergence. Fig. 5 s hows the circuit simulation speed of different ratios between n(MOSFET and p(MO SFET. Circuit simulation speed does not have to spe cific correlation with respect to the n(MOSFET to p(MOSFET W/L dimension ratio, i.e., n(MOSFET to p(MOSFET dimens ion ratio has no significant relevance to the circuit convergence. Even if the n (MOSFET to p(MOSFET dimension ratio does not match, the ML models convergence is not deteriorated. On the other hand, the BSIM model does not have any problem in conver gence in all n(MOSFET to p(MOSFET dimension ratios. The scatter plots for divergent cases in terms of the n(MOSFET to p(MOSF ET W/L dimension ratio is shown in Fig. 6. C. Convergence for capacitors of ML model Capacitors are added to the ML CMs in this section to promote the transient analysis convergence. Only practical capacitance values in C gs and C gd can be added to prevent overly large, unrealistic scenarios. From Table III, we can find that the con vergence of the circuit has been significantly improved. Sin ce the capacitor can help the voltage to change mor e slowly, improved circuit convergence is achieved. The capacitance value also deserves some discussion. Certainly, we prefer lar ge capacitors, since small capacitance is regard as an open circuit, which has no effect on the device model and SPICE. The capac itance values, however, needs to be realistic to make the calculation here meaningful. From literature, we found that the rea sonable range of C gs and C gd is 1 ×10 (8F/m (1×10 (12 F/m, while most of the reported MOSFET values falls around 1 ×10 (10 F/m. It can be seen from Table III that when the capacitance is 1 ×10 (12 F/m, ML models does not converged in all cases. In addition, the results show that the larger the capacitance values are, the more pronounced the imp roved circuit convergence is. In the capacitance ca lculations, I=C×dV /dt . dV /dt does not directly affect the continuity of the current, whi le C can affect current continuity if the capacitance v alues have discontinuities. Fortunately, although discontinuou s capacitance values can cause current value discon tinuities, the effect is not very pronounced for C gs and C gd in the range of 1 ×10 (8F/m (1×10 (12 F/m. Therefore the continuity of the capacitance v alues are less critical than the capacitance value in the dev ice model. In addition to a single XOR gate, we also connect t he XOR circuit in series, i.e. two XOR at the first stage connected to one XOR at the second stage, and then we use ISRU ML models to test the convergence of the circuit. The test r esult is that the 100% convergence is seen when the capacitance value is 1 ×10 (10 F/m. Besides, the deterioration of circuit converg ence occurs when the device model does not include C gs and C gd . This results in only 87% circuit convergence. Table III ML MODEL WITH CAPACITANCE (TRANSIENT TIME :10 NS , SINUSOID WAVE , AMPLITUDE OF 0.5V, THE FREQUENCY OF AC VOLTAGE IS 2GH Z AND 1GH Z). Capacitance Activation function Convergence ratio Total convergence /Total test 1×10 (12 F/m Sigmoid 99.75% 399/400 ISRU 99.75% 399/400 Tanh 99.75% 399/400 1×10 (11 F/m Sigmoid 100% 400/400 ISRU 100% 400/400 Tanh 100% 400/400 1×10 (10 F/m Sigmoid 100% 400/400 ISRU 100% 400/400 Tanh 100% 400/400 1×10 (9 F/m Sigmoid 100% 400/400 ISRU 100% 400/400 Tanh 100% 400/400 1×10 (8 F/m Sigmoid 100% 400/400 ISRU 100% 400/400 Tanh 100% 400/400 D. ML model for DC sweep When we do dc sweep simulation, we can find that IS RU has the best circuit convergence and Tanh and Si gmoid have worse circuit convergence, as shown in Table IV. The D.C . sweep in Table IV is not 100% even for ISRU, and thus we manage to use transient simulation to replace the D.C. sweep to p romote the usage of ML CMs. In general, the dc swee p ramping can be discontinuous if we have nested sweep at two nodes. To circumvent this problem and utilize the previou s SPICE solution for the next step, we design the input waveform for quasi(s tatic transient. The results of using quasi(static transient analysis to replace dc sweep are shown in Table V. It can be seen from the results that using transient analysis can signific antly improve the convergence. Because the input voltage gradually ramped and the discontinuity in nested sweep is resolved, the conv ergence of the circuit is significantly improved. Similar to Table III in the previous section, here we have tested different pr actical capacitor values in C gs and C gd to show the feasibility of the proposed methodolog y. Since quasi(static transient analysis is operate d at low frequencies, capacitance has less effect compared to the results in Table III. Nonetheless, we can still see ISRU h as 100% convergence for all capacitance values while sigmoid and tanh fails at a small proportion at some capacitance values. Table IV ML MODEL WITHOUT CAPACITANCE (DC SWEEP ). Activation function Convergence ratio Total convergence /Total test Sigmoid 54.25% 217/400 ISRU 81.00% 324/400 Tanh 11.25% 45/400 Table V ML MODELS WITH CAPACITANCE (U SING QUASI (STATIC TRANSIENT FOR DC SWEEP ) Capacitance Activation function Convergence ratio Total convergence /Total test 1×10 (12 F/m Sigmoid 97.25% 389/400 ISRU 100% 400/400 Tanh 100% 400/400 1×10 (11 F/m Sigmoid 100% 400/400 ISRU 100% 400/400 Tanh 100% 400/400 1×10 (10 F/m Sigmoid 100% 400/400 ISRU 100% 400/400 Tanh 99.75% 399/400 1×10 (9 F/m Sigmoid 100% 400/400 ISRU 100% 400/400 Tanh 100% 400/400 1×10 (8 F/m Sigmoid 100% 400/400 ISRU 100% 400/400 Tanh 100% 400/400 III. CONCLUSION The problems in convergence and simulation speed ne ed to be overcome when machine learning is applied to the model of semiconductor devices. Specifically, the convergenc e of circuit simulations using ML CMs still cannot be compared with the convergence in conventional physical models. Even w ith the best activation function, i.e. ISRU, the ci rcuit still cannot fully converge. Here we found that to improve the SPICE c onvergence, the activation functions should be cont inuous and possess smooth derivatives with bounded function values. Si gmoid, ISRU, and hyperbolic tangent have continuity , smooth derivatives and a bounded co(domain, and thus the circuit converges easier. The circuit convergence ratio of hyperboli c tangent is 99.50% for transient analysis without C gs and C gd . The circuit convergence of sigmoid and ISRU is 95 %. Adding C gs and C gd . in transient simulation further improves the convergence. When a realistic capacitance values for C gs and C gd is added to the MOSFET device model, the circuit convergence of tanh, ISRU, and s igmoid is 100% if C gs and C gd is >10 (11 F/m. Additionally, it is found that the circuit convergence of dc sweep is worse than the c ircuit convergence of transient analysis. In DC swe ep, ISRU significantly outperforms other activation functions. If a quasi( static transient analysis is used to replace DC swe ep, and realistic capacitance values are specified in ML MOSFET CMs, ISRU can pro vide convergent results for all purposes in circuit simulation, including DC and transient analysis. Therefore, in terms of our numerical study, ISRU has the best overall conv ergence and can potentially be more suitable for future machine learning based CMs in circuit simulation.