# Intelligent Fusion Imaging Photonics for Real-Time Lighting Obstructions
###### Abstract  
Dynamic detection in challenging lighting environments is essential for advancing intelligent robots and autonomous vehicles. Traditional vision systems are prone to severe lighting conditions in which rapid increases or decreases in contrast or saturation obscures objects, resulting in a loss of visibility. By incorporating intelligent optimization of polarization into vision systems using the iNC (integrated nanoscopic correction), we introduce an intelligent real-time fusion algorithm to address challenging and changing lighting conditions. Through real-time iterative feedback, we rapidly select polarizations, which is difficult to achieve with traditional methods. Fusion images were also dynamically reconstructed using pixel-based weights calculated in the intelligent polarization selection process. We showed that fused images by intelligent polarization selection reduced the mean-square error by two orders of magnitude to uncover subtle features of occluded objects. Our intelligent real-time fusion algorithm also achieved two orders of magnitude increase in time performance without compromising image quality. We expect intelligent fusion imaging photonics to play increasingly vital roles in the fields of next generation intelligent robots and autonomous vehicles.  
intelligent robotics; intelligent autonomous vehicles; intelligent drones 2022  
[PERSON] 1  
[PERSON] 1  
[PERSON] 1  
[PERSON] 1  
[PERSON] 2  
[PERSON] 1  
and spatial errors [21]. Simple approaches to rotate the polarization by predetermined increments also cannot readily adapt to dynamic scenarios. Furthermore, approaches identifying a single optimal polarization do not take into account the fact that different locations within an image may have different optimal polarizations.  
In this work, we incorporate intelligent optimization of polarization into vision systems using the iNC (integrated nanoscopic correction) (Figure 1). We introduce an intelligent real-time fusion algorithm to address challenging and changing lighting conditions. Through real-time iterative feedback, we rapidly assess the optimal polarization which is difficult to achieve with traditional methods. Image quality is quantitatively scored using peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and root mean squared error (RMSE). A fused image with minimal of lighting obstructions can be dynamically reconstructed using a pixel-based weighting mechanism. We demonstrate that fused images created by intelligent polarization selection uncover subtle features of occluded objects. We show that intelligent real-time fusion algorithm can significantly increase time performance without compromising image quality. We anticipate intelligent fusion imaging photonics will advance the ability of intelligent robots and autonomous vehicles to dynamically detect and respond to dynamic scenarios.  
Figure 1: Intelligent fusion imaging photonics for real-time lighting obstructions. (**a**) Conceptual schematic of dynamic detection in challenging lighting conditions. Rapid increase and decrease in contrast and saturation obscures objects, resulting in loss of visibility. (**b**) Conceptual schematic of experimental setup consisting of iNC incorporated on top of a vision system (CMOS camera). FR1, fixed retarder; LC1, liquid crystal retarder; FP, fixed polarizer; LC2, liquid crystal retarder; FR2, fixed retarder. (**c**) Intelligent polarization selection process by iNC: Voltage to the iNC was modulated. Images were downsampled and segmented based on optimal polarization by analyzing each pixel such that the relation of the intensity of the pixel to the segmentation was not in the spatial domain but in the polarization domain.