# High-Throughput Line Buffer Microarchitecture for Arbitrary Sized Streaming Image Processing
## 5 Evaluation
### Case Study of Image Processing with D-SWIM
#### 5.4.2 Harris Corner (HC) Detector  
The Harris Corner (HC) detector [15] is commonly used in computer vision applications that detects the corner position for feature matching. The HC operator swaps the window (e.g., \(3\times 3\) pixels in the benchmark) on the image and determines if the window contains a corner pattern. The HC algorithm on each window is listed in Equation (6). Firstly, HC obtains the gradient matrix \(\mathbf{M}\), where \(I(i,j)\) is the intensity value of the pixel inside the window; \(\frac{\partial I(i,j)}{\partial x}\) and \(\frac{\partial I(i,j)}{\partial y}\) are the intensity derivative in the horizontal and vertical axes, respectively. Secondly, \(R\) was calculated to estimate the eigenvalue of \(\mathbf{M}\), where \(k\) is a constant of 0.04-0.06, and det and trace calculates the determinant and trace of the matrix, respectively. If the value of \(R\) is larger than the threshold, the current window contains the corner pattern.  
\[\mathbf{M} =\begin{bmatrix}\sum_{i,j}^{W}\frac{\partial I(i,j)}{\partial x} \end{bmatrix}^{2}\quad\quad\quad\sum_{i,j}^{W}\frac{\partial I(i,j)}{\partial x }(\frac{\partial I(i,j)}{\partial y})\\ \sum_{i,j}^{W}\frac{\partial I(i,j)}{\partial x}(\frac{\partial I(i,j)}{ \partial y})\quad\quad\quad\sum_{i,j}^{W}\frac{\partial I(i,j)}{\partial y} \end{bmatrix}^{2} \tag{6}\] \[R =\det(\mathbf{M})-k\times\text{trace}(\mathbf{M})^{2}\]  
Figure 14 demonstrates the D-SWIM-based architecture for HC. This streaming architecture is composed of a buffer (rectangular shape), operator (circular shape), and interconnections. Note that the derivative calculation in different axes can be realized by a Conv2D operation with the Sobel kernels. We used a \(3\times 3\) Sobel kernel in the example, and the operators are denoted as dx and dy for the two axes. The derivative results (\(\frac{\partial I(i,j)}{\partial x}\), \(\frac{\partial I(i,j)}{\partial y}\)) were stored in Buf2 and Buf3 because they were accessed with a 2D window pattern in the subsequent operations. The sx, sy, and sxy operators perform the element-wise multiplication and accumulate the values in the window. After \(\mathbf{M}\) is obtained, operator rc calculates the \(R\) in Equation (6) and compares it with the threshold to determine whether the window contains a corner or not.  
The evaluation method of HC is the same as the Conv2D case. Table 6 shows the implementation results of D-SWIM and Design1-2. In the HC case, the D-SWIM-based design achieves both the highest throughput and the best hardware efficiency. Comparing with the superior design (Design1), D-SWIM increases the throughput to 3.5\(\times\). Furthermore, with D-SWIM, the efficiency of LUT and REG is 25\(\times\) and 30\(\times\) that in prior studies.  
Figure 14: D-SWIM-based architecture for HC detector (\(3\times 3\) window).  
\begin{table}
\begin{tabular}{c|c c c|c c c|c c|c c} \hline \hline \multirow{2}{*}{**Work**} & \multirow{2}{*}{**Device**} & \multirow{2}{*}{\begin{tabular}{c} \(N_{blk}\) \\ **(pixel)** \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} **Precision** \\ **(bit/pixel)** \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} **Hardware Consumption** \\ **LUT** \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} \(f_{max}\) \\ **REG** \\ \end{tabular} } & \multirow{2}{*}{\
\begin{tabular}{c} **Throughput** \\ **(MHz)** \\ \end{tabular} } & \multirow{2}{*}{
\begin{tabular}{c} **Efficiency** \\ **(GPPS)** \\ \end{tabular} } \\ \hline Design1[10] & Intel-5 SGXEA7 & 32 & 8 & 47,045 & 73,584 & 363 & 0 & 303.6 & 9.71 & 20.7 & 1.3 \\ Design2[7] & Xilinx-XC7Z045 & 1 & 8 & 288 & 521 & 2 & 0 & 349.9 & 0.35 & 121.5 & 6.7 \\ D-SWIM & Xilinx-XC7 VX690 & 16 & 8 & 4514 & 4232 & 9 & 76 & 283 & 4.5 & 100.3 & 10.7 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Hardware resource consumption and throughput in a Conv2D implementation.