## 3 Diffusive Markers Based Compensation
A normal, single-shot, picture (shown in Figure 4a) is not sufficient to recognize the markers. Instead, we use a dedicated procedure where the camera is configured to have a long shutter-time (30 s) and a low light sensitivity (ISO 100). Furthermore, a small aperture is used to reduce the incoming light further. While the camera is taking the picture (shutter open), we move a light source around in front of the screen. This way, we integrate the light hitting the diffusers from a different direction, over time. The light reflected from the mirrors is spread over multiple locations on the mirror, which ensures that these reflections do not result in an overly bright image. A crop-out of such an image is shown in Figure 4b. In this image, the markers are clearly visible as small dots. To detect these dots, we first apply an adaptive thresholding algorithm to determine the pixels that received a notable amount of light reflected by the screen. The thresholding algorithm is based on the work of [15] and applies binary thresholding on the image, using a dynamic threshold that is determined on the neighborhood of the pixel to threshold. After thresholding, we apply connected components labeling to determine pixels that belong to the same blob. The connected components are filtered by the area and aspect ratios of the components: the components that represent a diffuser are usually a few pixels wide and high, and have an aspect ratio close to 1. This filter removes the light paths formed by the moving light source being reflected by the mirrors, as the corresponding components tend to be wider than they are high. The remaining components are considered to be the artificially introduced markers. After detecting the markers, we apply the homography based identification that we described in [13] on the positions of the markers to identify to which mirror a marker belongs. This method utilizes the grid-structure combined with a homography to identify the markers, and filters out possible outliers. Since we know the actual positions of the markers on the screen, contrary to an assumed center, we can use this identification to build a homography \(H\), which is a \(3\times 3\) matrix that transforms pixels in the camera image plane to their corresponding location on the DDHOE plane. When visualizing the homography, by drawing the lines corresponding to the elemental images--their borders--the intersections of these lines will coincide with the diffusive markers, as can be seen in Figure 4c. Finally, we apply our structured light approach from [13] to find correspondences between the projector \((p_{x},p_{y})\) and camera \((c_{x},c_{y})\). Several binary patterns are projected onto the DDHOE, allowing us to identify which projector pixels are reflected to which camera pixel. Figure 4: (**a**) camera image of the mirror array screen as seen by the naked eye when completely lit by the projector (diffuse areas are not visible); (**b**) camera image of the mirror array screen, captured using a dedicated procedure to detect diffusing markers (diffuse areas are visible as dots); (**c**) borders of the elemental images overlaid on top of the image using the estimated homography. Intersections coincide with the diffusers, which are located at the corners of the diffusers. The homography \(H\) is used to transform the observed camera pixels \((c_{x},c_{y})\), back to the DDHOE plane. Doing so, we obtain the correct position \((X,Y)\) on the DDHOE screen, where the projector pixel is being projected to:  
\[ \begin{bmatrix}M_{x}\\ M_{y}\\ M_{z}\end{bmatrix} =H\cdot \begin{bmatrix}c_{x}\\ c_{y}\\ 1\end{bmatrix}  \tag{1}\]  
\[(s_{x},s_{y})=(M_{x}/M_{z},M_{y}/M_{z}); \tag{2}\]  
this is the part where we get rid of the assumption that the observed correspondences are located at the centers of the micromirrors, since the previous approach selected \(s\) to be the center of the corresponding micromirror. Instead, we now use the calculated location \((s_{x},s_{y})\) of the correspondence on the DDHOE plane.