## 2 Overview of the Proposed Neural Networks and Theoretical Background
### Deep Neural Networks  
Feedforward neural networks are one of the most common networks and offer good predictive performance. A single artificial neuron consists of input nodes and a single output node, which is connected to the input nodes. The output nodes of a hidden layer are connected to the nodes of the following layer with a weight \(w_{ij}\)[12]. The total input to a hidden neuron i is computed using the weights and biases of each layer as follows [68]:  
\[i_{j}=\sum_{i=1}^{n}x_{i}w_{ij}+b_{j} \tag{13}\]  
where \(i_{j}\) represents the total input of the neuron of the hidden layer \(j\), \(x_{i}\) presents the input of the neuron of the hidden layer \(i\) and the neuron of the hidden layer \(j\)[12], and \(b_{j}\) corresponds to the bias of the neuron of the hidden layer \(j\). Several activation functions can be used on the hidden layer. It is applied to the input and provides the output from the hidden neuron to the following layer as follows:  
\[H_{j}=F\big{(}i_{j}\big{)}=\frac{1}{1+\exp\big{(}-i_{j}\big{)}} \tag{14}\]  
The MLP model is a feedforward network, which indicates that all layers are connected in a single direction, starting at the input and ending at the output. Figure 4 illustrates a DNN architecture used to estimate the SoC for batteries [51]. The inputs are the temperature, current, and voltage of the battery, while the output is the SoC of the battery. The DNN can accurately and successfully estimate the SoC of a lithium-ion battery [69].  
Matlab was used to perform this estimation. This software has toolboxes that use neural networks to train the input data and generate an estimation model capable of predicting the SoC of a cell. An application was developed with the main purpose of creating a graphic interface for estimating the SoC with more accuracy. The reason for the development of this application was that the available toolboxes did not incorporate all the  
Figure 4: Flowchart of the deep learning process for estimating the state of charge of a Li-ion battery.  
available activation functions. Among these toolboxes, nprtool can be mentioned, which enables to select the number of neurons to be used and to build of a single hidden layer of the feedforward neural network with a sigmoid activation function in the hidden layer and softmax in the output. When this toolbox was tested, it appeared that it did not perform well because it had only one hidden layer and limited activation functions. Therefore, it was necessary to find another one that could be able to simulate multiple hidden layers and various activation functions. For this purpose, the nntool toolbox was useful to simulate what was necessary; however, it was only capable of simulating three activation functions: purelin, tansig, and logsig. The function softmax was not included. Also, it is important to notice that the available toolboxes did not provide a visualization of the training, testing, and validation data plots. Therefore, it was necessary to obtain the indices of data and plot them manually with a script. To be efficient, the proposed application has integrated buttons to perform this operation with a single click. The activation functions tested and added in the application were: poslin, purelin, radbas, logsig, netinv, satlins, softmax, radbasn, satin, compet, elliostig, hardlin, hardlims, tribas, and tansig. The proposed multilayer network used the transfer function tansig:  
\[\text{tansig}(x)=\frac{2}{1+\exp(-2x)}-1 \tag{15}\]