# Real Time Shadow Mapping for Augmented Reality Photorealistic Rendering
## 3 Proposed Solution
### Image Processing  
OpenCV computer vision library is necessary for the image processing pipeline. Since it was written in C++, the program for the recovery of the light source position was first developed and tested in a standalone mode and then integrated as a dynamic-link library (.dll) into Unity. It was chosen to create a specific.dll instead of using a wrapper, such as \"Emgu CV\" or \"OpenCV for Unity\" because they are not free for Unity. Moreover, our purpose was to create a standalone package including only the necessary functions and referring to the major installation of OpenCV on the computer, hence the multi-platform capability of adding OpenCV to a specific path of the system. The light detection routine applies a sequence of filters, thresholds and algorithms. Firstly, the 360 degree spherical image is converted from RGB to B/W, since the final scope is to recognize light sources that tend to be white and to have a high level of brightness (Figure 4). Moreover, the conversion of the 360-degree spherical image from RGB to B/W makes it easier to apply a threshold value to the pixels color values since they vary only from 0 to 255. After the greyscale conversion, it is possible to apply a threshold value to the image in order to delete all the pixels with color value below the given threshold. This threshold is chosen by analyzing the image histogram (Figure 5). In the first step, the maximum value of pixels was found. It is important to notice that not every photography covers the entire range of 0-255 such as too dark or too bright images. After the definition of the spectrum, it was possible to calculate the average value of pixels. The imposition we made was the definition of the ratio of these two values:  
\[\frac{Max(PixelValue)}{Average(PixelValue)}\geq 1.5 \tag{1}\]  
If this condition is true, it means that the distance between the maximum value and the average value of pixels is enough to state that there is a visible difference between the ambient light and a possible point light.  
Therefore, a ratio of 1.5-1 makes the photography useless, because in this condition it is possible to misidentify ambient light with point light, resulting, firstly, in a too high density of white pixels and, secondly, in an inaccurate positioning of the light in the method second step. (Figure 6). Contrariwise, if the ratio is over 1.5, as shown in Figure 5, where the value is 2.197, the correct positioning of the light source is achieved. In the second step. the estimation of threshold value was made. If 93% of the spectrum (starting from 0) covers at least 98% of the pixels, then it is possible to threshold the remaining 7% (Figure 7). This step allows cutting out the image and considering only the areas of the image corresponding to the actual light sources (Figure 8). At this point, only the brightest areas remain and they correspond to the room chandelier, the window and PC monitors. For this fist example, we decided to identify the area with the brightest intensity and to assign to this area the task of being the only light source of the scene. A median blur filter (Figure 9) was applied as noise reduction technique since it is possible that there are some areas with pixels value above the threshold value due to reflections of the objects in the environment. Using other types of filters, for example erode and dilate filters, provides the same result but with two or more sequential operations, instead of one single operation as in the case of median blur implementation.  
Figure 3: 360-degree spherical image.  
Figure 4: Greyscale conversion.  
Figure 5: Histogram of the office room sphere map case study.  
Figure 6: Example of an overexposed image not suitable for the proposed method.