## 2 Related Works
### A Design of an Optimized Deep Learning Module (ODLM)
#### 3.3.1 A Design of the VPDS
\[\frac{1}{2}(d-y)^{2} \tag{1}\]  
The VPDS uses the ReLU function and the Sigmoid function as an activation function, solves the vanishing gradient problem from an input layer to the last hidden layer by using an ReLU function, and diagnoses the result between the last hidden layer and an output layer by using a Sigmoid function. The VPDS is processed in the following phases. * The 1 st phase: the weight W-Z are initialized as a small value, not 0 and Training Datasets are used. * The 2 nd phase: The initial learning rate and a critical value of an error, Emax are decided. * The 3 rd phase: The nodes' values of the 1 st Hidden Layer and the weight sum of the \(h_{1j}\) of the 1 st Hidden Layer by using Formula (2), \(NET_{h_{1j}}\) are obtained in the VPDS. At this time, the \(x_{i}\) of Formula (2) means the value of the \(i\)th input node, and \(W_{ij}\) means a weight between the \(i\)th input node and the \(j\)th Hidden Layer. The VPDS computes the \(h_{1j}\), which is the node value of the 1 st Hidden Layer by applying the \(NET_{h_{1j}}\) of the expression (3). \[NET_{h_{1j}}=\sum_{i=1}^{n}x_{i}W_{ij}\] (2) \[h_{1j}=\max\Big{(}0,\ NET_{h_{1j}}\Big{)}\] (3)  
In this way, the VPDS computes \(NET\) and the Output value from the 1 st hidden layer to the rest of the hidden layer. * The 4 th Phase: The VPDS computes the \(NET_{y_{k}}\) between the 5 th hidden layer and an Output layer by using Formula (4) and computes the output value, \(y_{k}\) by using Formula (5). The \(NET_{y_{k}}\) is applied to the Sigmoid function of Formula (5). \[NET_{y_{k}}=\sum_{i=1}^{l}h_{5i}Z_{ik}\] (4) \[y_{k}=\frac{1}{1+e^{-NET_{y_{k}}}}\] (5)* The 5 th Phase: The VPDS computes an error E in the Formula (6) and compares it with the Emax of the 2 nd Phase. The VPDS closes the learning if the error E is less than the Emax. The VPDS goes to the 5 th Phase if the error E is greater than the Emax. \[\text{E}\ =\frac{1}{2}\sum_{i=1}^{k}(d_{i}-y_{i})^{2}\] (6) Here, \(d_{i}\) means a target value and, \(y_{i}\) is the result value of the Output Layer computed in the 3 rd phase. * The 6 th Phase: The VPDS computes an error signal, \(\delta_{h}\) between the Output Layer and the last Hidden Layer and between the Hidden Layers. The process works as follows. First, the VPDS computes the error signal, \(\delta_{y}\) between the Output Layer and the last Hidden Layer by using the Formula (7). \[\delta_{y}=(d-y)y(1-y)\] (7) Second, It computes the error signal, \(\delta_{h}\) between the Hidden Layers by using the Formula (8). \[\delta_{h_{5}}=h_{5}(1-h_{5})\sum_{i=1}^{k}\delta_{y}Z_{li}\] (8) Here, \(h_{5}\) means the 5 th Hidden Layer, \(\delta_{y}\) means the error signal between Output Layer and the last Hidden Layer and \(Z_{li}\) means the weight between the Output Layer and the last Hidden Layer. The VPDS computes the \(\delta_{h_{4}}\)-\(\delta_{x}\) in the same way as Formula (8). * The 7 th Phase: The VPDS modifies the weight, Z between the last Hidden Layer and Output Layer by using the Formula (9). \[Z_{k+1}=Z_{k}+\alpha\delta_{y}h_{5}.\] (9) Here, \(Z_{k}\) means the weight before being modified and \(Z_{k+1}\) means the weight after being modified. \(\alpha\) means a learning rate, \(\delta_{y}\) means the error signal computed by the Formula (8), and \(h_{5}\) means the 5 th Hidden Layer. The VPDS modifies the weight from the 5 th Hidden Layer to an input layer in the same way as Formula (8). For example, If the weight between the 1 st Hidden Layer and an Input Layer is \(W\), the \(W\) is modified in Formula (9). \[\frac{1}{2}(d-y)^{2} \tag{1}\]0  
Here, \(W_{k}\) means the weight before being modified and \(W_{k+1}\) means the weight after being modified. And \(\alpha\) means a learning rate, \(\delta_{h_{1}}\) means the error signal of 1 st Hidden Layer. And \(x\) means nodes value of the Input Layer. * The 8 th Phase: the VPDS repeats the learning process (training) from the 3 rd phase to the 7 th phase till an error E becomes less than the Emax.