## 2 Background and Motivation
### Data Partition  
Based on the dependency relationship between the output features and input features of a network layer, we can divide the CNN network layers into two categories: partial dependency layers and full dependency layers. The convolution layers and pooling layers belong to partial dependency layers. We use convolution layers as an example to illustrate the partial dependency. As shown in Figure 1(a), we demonstrated the dependency performance of the convolution layer. We pay attention to the blue area, input features and convolution kernels perform convolution operations to obtain output features, so the relationship between the input feature map and the output feature map is partially dependent. As shown in Figure 1(b), we described the dependency behavior of full connection layers. The full connection layer in Figure 1(b) has parameters containing three sets of vectors, each containing ten elements. Taking a vector containing ten elements as the input, the full connection layer will dot product each set of vector parameters with the input to obtain a three-dimensional vector.  
The data partitioning of a network layer involves dividing its original input into several sub-inputs. These sub-inputs are arranged to be calculated on different devices to obtain outputs, and all outputs belonging to these devices can be seamlessly concatenated into the original output without redundancy. Each network layer has three dimensions for input or output: height (H), width (W), and number of channels (C). In this article, we divide data into network layers along the height dimension. As shown in Figure 3, we give a convolution layer data partition sample, and the number of input and output channels is simplified to 1. Without data partitioning, the operation of this convolution layer is shown in Figure 1(a), which executes in a single device. In Figure 3, the convolution kernel is deployed on two devices: the original input is divided into two parts, which are used as inputs for two devices to obtain the convolution result. We record the two output results as \(output_{1}\) and \(output_{2}\), respectively. Additionally, we record the original output  
Figure 1: Model partition.  
Figure 2: Convolution layer and full connection layer.  
result as \(output_{0}\). So, the relationship between the original output and the output after data partitioning can be expressed as follows.  
\[output_{1}\cap output_{2}=\varnothing \tag{1}\]  
\[output_{1}\cup output_{2}=output_{0} \tag{2}\]  
Of course, partitioning the input data can also be transformed into partitioning the output data, as there is a correspondence between them. Taking convolution operation as an example, the kernel size, padding, and stride of convolution operation are denoted as \(k\), \(p\), and \(s\), respectively. If we record the input height range as \([1,H_{1}]\) and the output height range as \([1,H_{2}]\), then the input \([in_{s},in_{e}]\) and output \([out_{s},out_{e}]\) of convolution operation in a device satisfy the following relationships.  
\[1\leq in_{s}\leq in_{e}\leq H_{1} \tag{3}\]  
\[1\leq out_{s}\leq out_{e}\leq H_{2} \tag{4}\]  
\[in_{s}=(out_{s}-1)*s-p+1 \tag{5}\]  
\[in_{e}=(out_{e}-1)*s+k-p \tag{6}\]  
In this article, our data partitioning acts on the output data, which can better ensure the relationships explained in Formulas (1) and (2) and avoid unnecessary duplicate calculations, saving computing resources.  
Data partitioning in partial dependency layers can achieve parallel execution, effectively reducing the execution time. However, if data partitioning is used in the full dependency layer, similar parallel computing effects cannot be achieved because even if partial output is obtained, the full dependency layer must use the complete original input. The volume range of the full dependency layer depends on the parameter volume and is independent of the input volume.