## 5 Hardware System Design
### Temporal Parallelization  
In proposed modified algorithm, selection of a pair for the stereo-matching depends on the baseline distance \(b\). The different \(b\) with neighboring target micro-images \(I_{ti}\) from reference micro-image \(I_{r}\) in an MLA is shown in Figure 9. During the stereo-matching process, to find the correspondence, \(I_{r}\) needs to be correlated with several \(I_{ti}\) which are placed at \(b\) distance from the respective \(I_{r}\). Whereby \(b\) is calculated by Equation (10). The values inscribed in the micro-lenses refer to the \(l_{n}\) of the respective baseline distance. The angles represent the arrangement of \(I_{ti}\) with respect to \(I_{r}\) in the MLA. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Sub-Module** & \(\mathrm{BS}^{\mathrm{P}}_{\mathrm{n}}\) & **Description** \\ \hline Pre-Processing & \(\mathrm{BS}^{\mathrm{PP}}_{1}\) & Consists of recursive operations to calculate the grid lines of micro-lenses array, such as lens-type. \\  & \(\mathrm{BS}^{\mathrm{PP}}_{2}\) & Arranges and assigns the calculated micro-lens grid data to the corresponding micro-lenses. \\ Census Transform & \(\mathrm{BS}^{\mathrm{ct}}_{1}\) & Recursive operations to convert the input image to the transformed image with respect to the window size. \\ Best-Match & \(\mathrm{BS}^{\mathrm{bc}}_{1}\) & Selects reference micro-image and fetches the respective input data to find the correspondence. \\  & \(\mathrm{BS}^{\mathrm{bc}}_{2}\) & Recursive operations to select the target micro-images for matching. \\  & \(\mathrm{BS}^{\mathrm{bc}}_{3}\) & Iterative search process to calculate error to find the correspondence across the target micro-image \\ Depth-Evaluation & \(\mathrm{BS}^{\mathrm{de}}_{1}\) & Transferring the calculated depth estimations to the memory \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Selected \(\mathrm{BS}^{\mathrm{P}}_{\mathrm{n}}\) from sub-modules of the algorithm. Figure 8: UML structure diagram showing the dependencies between different components of the algorithm. Search process starts with target micro-images that are located at the shortest \(b=1\cdot d_{L}\) in all directions from \(I_{r}\). Subsequently, \(I_{ti}\) that is placed at higher \(b\), is searched for a match. This way, baseline distance keeps increasing until a match is found. It is a computation intensive task due to its recursive nature. The modified algorithm reduces this effort by restricting the matching process. It is because the dimensions of the micro-images are quite small, e.g., \(d_{L}\) of Raytrix R5 camera is almost 23 pixels. It only allows each micro-image to capture a little fraction of information. Therefore, it can be established that the probability of finding the match is higher in the micro-images which are located in close proximity to the \(I_{r}\). By taking it into account, it is determined from experiments that a good match is found with \(I_{ti}\), which is located within range of \(l_{max}=4\) from \(I_{r}\). Thus the maximum baseline distance is restricted to \(b=4\cdot d_{L}\). From Figure 9, it is apparent that the matching occurs in all directions with respect to the \(I_{r}\). However, it causes redundant matching computations to take place, because in subsequent stereo-matching the role of same pair gets swapped, i.e., reference with target micro-image. For this reason, the proposed design allows only those micro-images to be correlated during the stereo-matching, which are located between \(0\leq\theta\leq-90\) from the reference micro-image. It ensures matching to be carried out only once for each pair, and therefore it significantly reduces the redundant computations. In this formation, maximum number of target micro-images that can be correlated with \(I_{r}\) is \(n_{T}=18\). Selection of micro-image pairs to find the best match is pipelined, as depicted in Figure 10. Whereby \(N\) corresponds to the latency of one matching task. The temporal parallelization ensures the extensive search for the match to be carried out concurrently.