# An Efficient Dropout for Robust Deep Neural Networks
## 4 Dataset for Comparison  
Four different convolutional neural network (CNN) architectures were implemented to thoroughly evaluate the effectiveness of the proposed technique, as shown in Table 3. These models vary in both depth and architectural complexity to examine the generalizability and robustness of the method. The first three are custom-designed CNNs with unique layer configurations, while the fourth is based on the well-established ResNet architecture, which uses residual connections to enable training of deeper networks. By applying the proposed dropout method to a variety of architectures, including a standard deep residual model, the study shows its adaptability and performance advantages in both simple and complex learning scenarios. To evaluate the performance of the proposed dropout method, experiments are conducted on CIFAR-10, and CIFAR-100 datasets using convolutional neural network (CNN) architecture. All models were implemented and trained using TensorFlow (version: 2.18.0) \begin{table}
\begin{tabular}{l l} \hline Step 1: Load and preprocess the dataset \\ Step 2: Define model hyperparameters \\ Step 3: Load the previously saved model \\ Step 4: Define helper functions \\ - & Calculates the mean of low magnitude weights using Gaussian distribution principles. \\ - & Computes the temperature parameter based on the standard deviation of the model’s weights. \\ - & Applies the Proposed Dropout Function, \\ - & Amplifies the top percentage of the model’s weights. \\ - & Adjusts the learning rate based on the current epoch. \\ - & Track neuron activity and weight evolution over recent epochs. \\ - & Returns max connections based on the current epoch. \\ Step 5: Training loop \\ Step 6: Save the final model \\ \hline \end{tabular}
\end{table} 
Table 2: Example Applying the Proposed Method. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model Name** & **Dataset** & **Architectural Summary (Flow and Key Components)** \\ \hline \multirow{3}{*}{Model 1} & \multirow{3}{*}{CIFAR10} & Input (32 \(\times\) 32 \(\times\) 3) \(\rightarrow\) [5 \(\times\) (Conv2D: 128 F, 3 \(\times\) 3 K, ReLU, Same Padding + Batch Norm + MaxPooling: 2 \(\times\) 2 P)] \(\rightarrow\) Flatten \(\rightarrow\) Dense (512 U, ReLU) \(\rightarrow\) Output \\  & & (num\_classes, Softmax) \\ \hline \multirow{3}{*}{Model 2} & \multirow{3}{*}{CIFAR100} & Input (32 \(\times\) 32 \(\times\) 3) \(\rightarrow\) [5 \(\times\) (Conv2D: 512 F, 3 \(\times\) 3 K, ReLU, Same Padding + Batch Norm + (MaxPooling: 2 \(\times\) 2 P after every 2 nd Conv Layer))] \(\rightarrow\) Global Average Pooling 2D \(\rightarrow\) Dense (1024 U, ReLU) \(\rightarrow\) Output (100, Softmax) \\ \hline \multirow{3}{*}{Model 3} & \multirow{3}{*}{CIFAR100} & Input (32 \(\times\) 32 \(\times\) 3) \(\rightarrow\) Conv2D (64 F) \(\rightarrow\) Batch Norm \(\rightarrow\) Conv2D (128 F) \(\rightarrow\) Batch Norm \(\rightarrow\) MaxPooling (2 \(\times\) 2 P) \(\rightarrow\) Conv2D (256 F) \(\rightarrow\) Batch Norm \(\rightarrow\) Conv2D (512 F) \(\rightarrow\) Batch Norm \(\rightarrow\) MaxPooling (2 \(\times\) 2 P) \(\rightarrow\) Global Average Pooling 2D \(\rightarrow\) Dense (512 U, ReLU) \(\rightarrow\) Output (100, Softmax) (All Conv2D layers are 3 \(\times\) 3 K, ReLU, Same Padding) \\ \hline \multirow{3}{*}{Model 4} & \multirow{3}{*}{CIFAR100} & Input (32 \(\times\) 32 \(\times\) 3) \(\rightarrow\) Conv2D (64 F, 3 \(\times\) 3 K, Same, ReLU) \(\rightarrow\) Batch Norm \(\rightarrow\) 8 \(\times\) Residual Block: [2 \(\times\) (64 F), 2 \(\times\) (128 F, Downsample = 1), 2 \(\times\) (256 F, Downsample = 1), 2 \(\times\) (256 F, Downsample = 1), 2 \(\times\) (512 F, Downsample = 1)] \(\rightarrow\) Global Average Pooling 2D \(\rightarrow\) Dense (512 U, ReLU) \(\rightarrow\) Dropout (0.5) \(\rightarrow\) Output (100, Softmax) (Each block has 2 \(\times\) Conv2D (3 \(\times\) 3 K) and skip connection; downsampled blocks use Conv2D (1 \(\times\) 1 K) and stride 2)  
\end{table} 
Table 3: Architectural Summary of Network Models.