## 6 Experiments
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Standard CNN** & **Flattened CNN** & **Proposed CNN** \\ \hline \multirow{3}{*}{\(1\times 3\times 3\) conv} & Conv1: 64 filters, each filter constituted as: \\  & \(1\times 1\times 1\times\) conv, \(1\times 3\times 1\) conv, \(\mathbf{w}_{1}\doteq\mathbf{p}_{1}(1\times 3\times 1)\otimes\mathbf{q}_{1}(1 \times 1\times 3)\otimes\mathbf{t}_{1}(1\times 1\times 1)\) \\  & \(1\times 1\times 3\) conv & \(1\times 3\times 3\) conv \\ \hline \multirow{3}{*}{\(64\times 3\times 3\) conv} & Conv2: 64 filters, each filter constituted as: \\  & \(64\times 1\times 1\) conv, \(1\times 3\times 1\) conv, \(\mathbf{w}_{2}\doteq\mathbf{p}_{2}(1\times 3\times 1)\otimes\mathbf{q}_{2}(1 \times 1\times 3)\otimes\mathbf{t}_{2}(64\times 1\times 1)\) \\  & \(1\times 1\times 3\) conv & \(64\times 3\times 3\) conv \\ \hline \multicolumn{3}{c}{Max Pool (\(\frac{1}{2}\))} \\ \hline \multirow{3}{*}{\(64\times 3\times 3\) conv} & Conv3: 144 filters, each filter constituted as: \\  & \(64\times 1\times 1\) conv, \(1\times 3\times 1\) conv & \(\mathbf{w}_{3}\doteq\mathbf{p}_{3}(1\times 3\times 1)\otimes\mathbf{q}_{3}(1 \times 1\times 3)\otimes\mathbf{t}_{3}(64\times 1\times 1)\) \\  & \(1\times 1\times 3\) conv & \(64\times 3\times 3\) conv \\ \hline \multicolumn{3}{c}{Conv4: 144 filters, each filter constituted as:} \\ \hline \multirow{3}{*}{\(144\times 3\times 3\) conv} & Conv4: 144 filters, each filter constituted as: \\  & \(144\times 1\times 1\) conv, \(1\times 3\times 1\) conv & \(\mathbf{w}_{4}\doteq\mathbf{p}_{4}(1\times 3\times 1)\otimes\mathbf{q}_{4}(1 \times 1\times 3)\otimes\mathbf{t}_{4}(144\times 1\times 1)\) \\  & \(1\times 1\times 3\) conv & \(144\times 3\times 3\) conv \\ \hline \multicolumn{3}{c}{Max Pool (\(\frac{1}{2}\))} \\ \hline \multirow{3}{*}{\(144\times 3\times 3\) conv} & Conv5: 144 filters, each filter constituted as: \\  & \(144\times 1\times 1\) conv, \(1\times 3\times 1\) conv & \(\mathbf{w}_{5}\doteq\mathbf{p}_{5}(1\times 3\times 1)\otimes\mathbf{q}_{5}(1 \times 1\times 3)\otimes\mathbf{t}_{5}(144\times 1\times 1)\) \\  & \(1\times 1\times 3\) conv & \(144\times 3\times 3\) conv \\ \hline \multicolumn{3}{c}{Conv6: 256 filters, each filter constituted as:} \\ \hline \multirow{3}{*}{\(144\times 3\times 3\) conv} & Conv6: 256 filters, each filter constituted as: \\  & \(144\times 1\times 1\) conv, \(1\times 3\times 1\) conv & \(\mathbf{w}_{6}\doteq\mathbf{p}_{6}(1\times 3\times 1)\otimes\mathbf{q}_{6}(1\times 1\times 3)\otimes\mathbf{t}_{6}(1 44\times 1\times 1)\) \\  & \(1\times 1\times 3\) conv & \(144\times 3\times 3\) conv \\ \hline \multicolumn{3}{c}{Conv7: 256 filters, each filter constituted as:} \\ \hline \multirow{3}{*}{\(256\times 3\times 3\) conv} & Conv256 \(1\times 1\) conv, \(1\times 3\times 1\) conv & \(\mathbf{w}_{7}\doteq\mathbf{p}_{7}(1\times 3\times 1)\otimes\mathbf{q}_{7}(1 \times 1\times 3)\otimes\mathbf{t}_{7}(256\times 1\times 1)\) \\  & \(1\times 1\times 3\) conv & \(256\times 3\times 3\) conv \\ \hline \multicolumn{3}{c}{FC 2048 + Batch Normalization + ReLU + Drop Out (P = 0.5)} \\ \hline \multicolumn{3}{c}{FC 1024 + Batch Normalization + ReLU + Drop Out (P = 0.5)} \\ \hline \multicolumn{3}{c}{FC 10 + ReLU + Drop Out (P = 0.5)} \\ \hline \multicolumn{3}{c}{Soft-Max} \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Structure of convolutional neural network (CNN)1 for the MNIST dataset. the final testing. We plotted for every training epoch the validation accuracy values in Figures 5-9. The number of epochs in the figures were determined to be greater than the epochs for which the validation accuracies with the proposed model sufficiently converged, and which resulted in graphs from which it became possible to visually compare the results of the different methods. Figures 5-9 show the validation accuracies for every epoch along the training process for a single training for each model.