# Model Parallelism Optimization for CNN FPGA Accelerator
## 3 Approach
### Convolution  
In general, the convolutional layer (CL) is the most resource-intensive layer in a CNN. In a CNN model, a CL contains hundreds of convolutional kernels (y). The convolutional operation aims to generate the output by sliding the convolutional kernel over the input. Since the convolutional kernel is a three-dimensional (width, height, depth) tensor, the core of convolution is a three-dimensional multiplicative accumulation operation. This is shown in Equation (1). \[C_{OUT}(f_{o},y,x)=\sum_{f_{i}=0}^{C_{n}}\sum_{k_{y}=0}^{K}\sum_{k_{x}=0}^{K}W_{ l}\big{(}f_{o},f_{i},k_{y},k_{x}\big{)}\times C_{IN}\big{(}f_{i},y+k_{y},x+k_{x} \big{)} \tag{1}\]  
where \(C_{IN}\big{(}f_{i},y+k_{y},x+k_{x}\big{)}\) and \(C_{OUT}(f_{o},y,x)\) represent the neurons at location \((x,y)\) in the input feature map \(f_{i}\) and the output feature map \(f_{o}\), respectively; and \(W_{l}\big{(}f_{o},f_{i},k_{y},k_{x}\big{)}\) represent the weight of the corresponding position \((k_{x},k_{y})\) in the nth layer of \(f_{o}\) obtained by convolution with \(f_{i}\). Previous works have parallelized the convolution layer through data partition. A common partitioning method for convolutional layers used in research is shown in Figure 2. Each device processes a portion of data from each input channel and generates the corresponding output. When the convolutional kernel is \(1\times 1\), the partitioning method in Figure 2 is reasonable. However, when the convolutional kernel exceeds \(1\times 1\), the data of each input channel require adjacent input data from neighboring devices, and thus communication between devices occurs. This kind of communication negatively affects model parallelism. A larger region of input data than the partitioned region itself, as shown by the red dashed box in Figure 2, has to be allocated to each device, because its computation depends on the neighboring data. To make matters worse, in this partitioning approach the kernel weights are not partitioned, and each device needs to keep all the weights, which has a slight impact on the memory footprint. If each device fetches the data of the region in the red dashed box in advance, the communication between devices can be eliminated. Unfortunately, this approach lacks universality, since the size of the excess depends on the size of the convolutional kernel. To deal with the problems of the above partitioning methods, we proposed the replacement of this tightly coupled convolutional structure with group convolution in order to apply model parallelism. For convolutional neural networks, group convolution is more suitable for model parallelism than convolutional layers. This is because group convolution first appeared in AlexNet to solve the problem of insufficient memory. It can increase the diagonal correlation between convolutional kernels and reduce the training parameters. Previous research works have indicated that group convolution can make a model less prone to overfitting, which is similar to the effect of regularization. As shown in Figure 3, with the group convolution structure, each device acquires a part of the data of the input channel, while the weights are uniformly distributed on each device and are applicable to any convolutional kernel size. Assume that there are X feature maps (channel = X) and the number of group convolutions is M (group = M). The algorithm aims to divide the channel into M parts. Each group corresponds to X/M channels, which are connected independently. The size of the convolution kernel also changes from X*W*H  
Figure 2: Partitioning method for convolutional layer (number of devices = 3). to X*W*H/M. This change does not affect the dimensionality of the input and output feature maps, but significantly reduces the computational complexity and the number of model parameters and greatly reduces the memory occupation resources of each device, which is beneficial for hardware devices with limited resources.