# Model Parallelism Optimization for CNN FPGA Accelerator
## 5 Experiment
### Performance  
This section presents the evaluation of the performance of the model in terms of time and energy consumption. Table 3 shows the experimental results for both versions of _ShuffleNet_. In Table 3, _ShuffleNet_ is the original model, while _I-ShuffleNet_ is the model with the inside shuffle algorithm. The _optimizations_ are divided into _channel_ and _others_. Note that _others_ represents three optimization methods: pipelined computation, aggregate read, and kernel vectorization. _Number_ is the number of FPGA devices. _Time_ is the average inference time in milliseconds for many experiments, and _Power_ is the average power consumption in watts for each device during inference. The power consumption of the FPGA includes not only the FPGA chip, but also the power consumption of the off-chip memory and other devices on the board. On Inspur F10A, compared to _ShuffleNet_, _I-ShuffleNet_ reduced the inference time by almost two-fold for the same number of devices. This was due to the introduction of the channel, which reduced the number of data interactions between the FPGA and the CPU. When there was only one device, the channel achieved the highest time reduction. In the case of a single device, the number of interactions was only two: one write and one write out. In the case of multiple devices, due to the _ShuffleNet_ network structure, data synchronization was required after each block was calculated, so the number of interactions was equal to the number of blocks, i.e., 16. This is why the degree of time reduction was not very large. As presented in Figure 6, the time comparison showed that the model parallelism had a good acceleration effect when using two FPGAs. The running time required for each layer basically demonstrated a linearly proportional reduction, achieving a speed-up ratio of 1.9\(\times\). However, in the _I-ShuffleNet_ version, the model with two devices was 1.42\(\times\) faster than the model with one device. Because model parallelism requires synchronization time, it is not linearly proportional. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**CNN Model**} & \multirow{2}{*}{**Channel Shuffle**} & \multirow{2}{*}{**Group**} & \multicolumn{2}{c}{**Top-1 Accuracy**} \\ \cline{3-5}  & & & **MNIST** & **CIFAR-10** \\ \hline \multirow{5}{*}{_ShuffleNet_} & Original Shuffle & 2 & 98.90\% & 84.44\% \\  & Inside Shuffle & 2 & 99.00\% & 84.75\% \\ \cline{1-1}  & Original Shuffle & 3 & 98.70\% & 83.52\% \\ \cline{1-1}  & Inside Shuffle & 3 & 98.83\% & 84.11\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Accuracy of experimental results. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Device**} & \multirow{2}{*}{**Version**} & \multicolumn{3}{c}{**Optimizations**} & \multirow{2}{*}{**Number**} & \multirow{2}{*}{**Time (ms)**} & \multirow{2}{*}{**Power (W)**} \\ \cline{3-5}  & & **Channel** & & & & \\ \hline \multirow{5}{*}{Inspur F10A} & X & X & 1 & 1045.198 & 23.5 \\  & \multirow{2}{*}{_ShuffleNet_} & X & \(\surd\) & 1 & 329.869 & 23.4 \\  & \multirow{2}{*}{_I-ShuffleNet_} & X & \(\surd\) & 2 & 228.358 & 20.5 \\  & \multirow{2}{*}{_I-ShuffleNet_} & \(\surd\) & \(\surd\) & 1 & 186.635 & 23.5 \\  & \multirow{2}{*}{_I-ShuffleNet_} & \(\surd\) & \(\surd\) & 2 & 130.521 & 21.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Experimental results. The resources used for the entire network inference process are shown in Table 4. The largest change due to the number of devices was in the use of RAMs. The number of RAMs required ranged from 1707 to 1265, with a 34% reduction in memory utilization. This reduction in memory utilization opens up the possibility of implementing neural network inference using devices with lower amounts of resources. This paper demonstrated the experimental effect of CNN model parallelism when using two FPGAs. Of course, this method could be used to add more devices and act on multiple devices. However, it is not the case that more devices bring better results.