# An Efficient Dropout for Robust Deep Neural Networks
## 4 Dataset for Comparison
### Implementation Details  
The CNN architecture was first trained for 20 epochs for the test. The trained network was recorded and compared by continuing training in 3 stages. As shown in Figure 1, the recorded network was trained separately in 3 stages as \"No Dropout\", \"Proposed Dropout\", and \"Standard Dropout\". The naming in the architecture  
* No Dropout: The network continues its training in the same way without any additions until it reaches 40 epochs. * Standard Dropout: The network is trained with random dropout (standard random dropout code) and continues its training until it reaches 40 epochs. * Proposed Dropout: The network is trained with the recommended method (Table 2) and continues its training until it reaches 40 epochs. The performance of the models was evaluated using validation accuracy and validation loss. Validation accuracy measures the proportion of correctly classified samples,  
Figure 1: A graphic representation of comparison. while validation loss quantifies the divergence between predicted and true distributions using categorical cross-entropy. \[\text{Validation Accuracy}=\frac{1}{N}\sum_{i=1}^{N}\mathds{1}(\text{argmax}( \hat{y_{i}})=\text{argmax}(y_{i})) \tag{15}\]  
\[\text{Validation Loss}=-\frac{1}{N}\sum_{i=1}^{N}\sum_{c=1}^{C}y_{i,c}\text{log}( \hat{y_{i,c}}+\varepsilon) \tag{16}\]  
where \(N\) is the number of validation samples, \(c\) is the number of classes, \(y_{i}\) is the one-hot encoded true label  
In Figure 2, the \"No Dropout\" setup rapidly attained reasonable accuracy (~78-79%) but did not progress beyond the initial epochs. The validation loss exhibited significant variations and an upward trend, indicating overfitting. The \"Standard Dropout\" enhanced generalization by stabilizing accuracy and diminishing loss values (~1.05), while slight variations persisting. The \"Proposed\" technique outperformed both options, with a steady validation accuracy of 80% and a modest validation loss (between 0.98 and 1.03), indicating more stability and reduced overfitting. In Figure 3, the proposed method achieved the highest validation accuracy of 58.59%, outperforming both the no dropout (57.75%) and standard dropout (55.91%) configurations. In terms of validation loss, the proposed technique demonstrated a slow and sustained fall, reaching 2.37, compared to 2.54 for standard dropout and a rapid increase to 4.43 for the no-dropout baseline. These results indicate that the proposed dropout technique delivers greater generalization and training stability, notably in managing overfitting during later training phases. In Figure 4, which used a deeper CNN architecture, the suggested technique obtained a high validation accuracy of 54.4% with a continuously low and steady validation loss of 2.47. The no-dropout baseline obtained a comparable accuracy of 53.75% but suffered from severe overfitting, with validation loss growing to 4.66 by the last epoch. The standard dropout model performed moderately better, reaching 53.87% accuracy and ending with a loss of 2.54. Compared to both alternatives, the proposed method demonstrated better convergence stability, lower loss, and less performance fluctuation, confirming its effectiveness in deeper architectures and its ability to reduce overfitting under complex training conditions. Figure 4: Validation Accuracy and Validation Loss for Model 3 CIFAR-100 dataset. Figure 5: Validation Accuracy and Validation Loss for Model 4 CIFAR-100 dataset. In Figure 5, a ResNet-inspired architecture with residual blocks is used to test dropout scalability in deeper networks. The proposed method achieved the highest validation accuracy of 55.22%, with a consistently low validation loss of 2.40. In contrast, the baseline model without dropout peaked at 53.36% accuracy and suffered from unstable training, ending with a much higher loss of 3.38. The standard dropout design attained 54.18% accuracy with a final loss of 2.47.