## 4 Proposed Model Design
### Parallel Deep Neural Network  
This section introduces using the Spark Framework to create parallel deep neural networks. The proposed parallel model is illustrated in Figure 3, in which the Spark framework splits massive training data in RDDs samples labeled _D1_, _D2_, _D3_, _..._, _Dn_ and shares them over a nodes cluster (i.e., utilizing data parallelization). Additionally, a DNN model copy is distributed across several workers by the Spark framework (i.e., using \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Data-Balancing Techniques** & **SN (\%)** & **SP (\%)** & **ACC (\%)** & **MCC** \\ \hline Simple data & 46.617 & 67.403 & 59.24 & 14.234 \\ Under-sampling & 68.595 & 51.538 & 60.09 & 20.394 \\ SmoteTomek & 79.29 & 61.257 & 68.51 & 46.968 \\ SmoteENN & 89.011 & 82.352 & 83.40 & 71.066 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of different data balancing techniques. parallelization of the model). The training begins with the DNN model being executed across all worker nodes simultaneously (i.e., at the same time), and several models have been given parallel training using the provided hyperparameters indicated in Table 2. After the training process (shown in Figure 3 with an orange line), each worker node in the trained models updates the master node after receiving a trained model (i.e., in globally trained mode, every worker assigns a different test dataset as an RDD sample). Worker nodes use a globally trained model to be applied to specific test datasets and produce local output (i.e., classification metrics). Finally, the average parameter function combines \(n\) local outcomes via the master node (i.e., parameter server) to produce the final result. To improve the proposed model, the back-propagation mechanism is used on every model and is distributed over the worker nodes. The DNN model iterates to a lower loss function in each iteration. Every single worker node calculates Stochastic Gradient Descent (SGD) on a provided subset to reduce prediction error. Every worker node reports the computed SGD to a centralized parameter server. The worker node and parameter server communicate using a synchronous communication mechanism. The parameter server gathers partial gradients of the worker nodes to calculate the new weights set. After this, the parameter server distributes updated weights to the worker nodes that recalculate the gradients (illustrated with the blue line in Figure 3). The server parameter in the proposed model handles the distributed scaling model and functions as a coordinating agent among the worker nodes and central server [60]. Compared to asynchronous SGD optimization, synchronous SGD optimization achieves better scalability and efficiency [61; 62]. However, because of the occasional slowdowns of a worker node, the synchronous SGD process can decrease model performance. We suppose that every worker nodes in the proposed model are homogeneous and there are no slowdown worker nodes. The proposed model, at a higher level, trains and tests in the following ways:  
1. The Spark framework breaks down many sequences into smaller parts and divides them among all worker nodes. 2. The DNN model is replicated on each worker node. 3. Each worker node runs a model on a particular partition for local model training. 4. The worker node communicates the local model parameters to the parameter server. The parameter server uses the parameter average overall local models to integrate all parameters. 5. The parameter server updates the weights and distributes them to the worker nodes. Figure 3: Proposed Parallel Deep Model Framework. Steps 3, 4, and 5 are repeated several times in the proposed model to generate a globally trained model with the best parameters. 7. The final average classification metrics are created by applying the globally trained model to the testing dataset. Using a parallel approach, the proposed method dramatically reduces computing time while maintaining high scalability.