Abstract:
A method using Long Wave Infrared Imaging Polarimetry for improved mapping and perception of a roadway or path and for perceiving or detecting obstacles comprises recording raw image data using a polarimeter to obtain polarized images of the roadway or area. The images are then corrected for non-uniformity, optical distortion, and registration. IR and polarization data products are computed, and the resultant data products are converted to a multi-dimensional data set for exploitation. Contrast enhancement algorithms are applied to the multi-dimensional imagery to form enhanced object images. The enhanced object images may then be displayed to a user, and/or an annunciator may announce the presence of an object. Further, the vehicle may take evasive action based upon the presence of an object in the roadway.

Description:
REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application claims the benefit of and priority to Provisional Patent Application U.S. Ser. No. 62/041,778, entitled “Polarization-based Mapping and Perception System and Method,” and filed on Aug. 26, 2014, which is fully incorporated herein by reference. This application further is a continuation-in-part of U.S. Non-Provisional application Ser. No. 14/602,823, entitled “Polarization Imaging for Facial Recognition Enhancement System and Method,” and filed on Jan. 22, 2015, which claims the benefit of U.S. Provisional Application No. 61/930,272, entitled “Polarization Imaging for Facial Recognition Enhancement,” and filed on Jan. 22, 2014, both of which are fully incorporated herein by reference. 
     
    
     GOVERNMENT LICENSE RIGHTS 
       [0002]    This invention was made with government support under Contract Number N00014-13-C-0290 awarded by the U.S. Navy. The government has certain rights in the invention. 
     
    
     BACKGROUND AND SUMMARY 
       [0003]    As used herein, Long Wave Infrared is referred to as “LWIR” or “thermal.” As used herein, “mapping” refers to placing objects in a scene relative to other objects or elements in the scene. As an example, “that little rock is in the road next to that big rock just off the road.” As used herein, “roadway” refers to any path along which a person, animal, or vehicle may traverse. 
         [0004]    A method using Long Wave Infrared Imaging Polarimetry for improved mapping and perception of a roadway or path and for perceiving or detecting objects is disclosed herein. The described method is not tied to any one specific polarimeter sensor architecture, and thus the method described pertains to all LWIR sensors capable of detecting the critical polarimetric signature. The method comprises recording raw image data of an area using a polarimeter to obtain polarized images of the area. The images are then corrected for non-uniformity, optical distortion, and registration in accordance with the procedure necessitated by the sensor&#39;s architecture. IR and polarization data products are computed, and the resultant data products are converted to a multi-dimensional data set for exploitation. Contrast enhancement algorithms are applied to the multi-dimensional imagery to form enhanced object images. The enhanced object images may then be displayed to a user, and/or an annunciator may announce the presence of an object. Further, the vehicle may take evasive action based upon the presence of an object in the roadway. 
         [0005]    A standard IR camera gives information about an IR signature (i.e., how bright a target looks), spatial information (i.e., where a target is in the scene), and temporal information (i.e., how the target changes in the scene from frame to frame). A polarimetric system and method as disclosed herein provides all of this information and also a polarimetric signature. 
     
    
     
       DESCRIPTION OF THE DRAWINGS 
         [0006]    The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawings will be provided by the Office upon request and payment of the necessary fee. 
           [0007]      FIG. 1  is a block diagram illustrating a system in accordance with an exemplary embodiment of the present disclosure. 
           [0008]      FIG. 2  depicts an exemplary polarimeter and signal processing unit as depicted in  FIG. 1 . 
           [0009]      FIG. 3  is a flowchart depicting exemplary architecture and functionality of the system logic in accordance with an exemplary embodiment of the disclosure. 
           [0010]      FIG. 4 a    depicts a visible image of a roadway at night. 
           [0011]      FIG. 4 b    is a thermal image of the same roadway at night. 
           [0012]      FIG. 4 c    depicts a polarization image of the roadway obtained with the system and method according to an exemplary embodiment of the present disclosure. 
           [0013]      FIG. 5 a    is a visible image of a roadway during the daytime. 
           [0014]      FIG. 5 b    is a thermal image of the roadway of  FIG. 5   a.    
           [0015]      FIG. 5 c    is a polarization image of the roadway of  FIG. 5 a    obtained with the system and method according to an exemplary embodiment of the present disclosure. 
           [0016]      FIG. 6 a    is a visible image of a roadway that has a median and sidewalk, shown at night. 
           [0017]      FIG. 6 b    is a thermal image of the roadway of  FIG. 6 a   , also at night. 
           [0018]      FIG. 6 c    is a polarization image of the roadway of  FIG. 6 a    obtained with the system and method according to an exemplary embodiment of the present disclosure 
           [0019]      FIG. 7 a    is a visible image of a dirt road depicting exemplary obstacles on the road. 
           [0020]      FIG. 7 b    is a thermal image of the road of  FIG. 7   a.    
           [0021]      FIG. 7 c    is a contrast enhanced thermal image of the road of  FIG. 7   a.    
           [0022]      FIG. 7 d    is a polarization image of the road of  FIG. 7   a.    
           [0023]      FIG. 7 e    is a ColorFuse image of the road of  FIG. 7   d.    
           [0024]      FIG. 8 a    is a visible image of a dirt road depicting exemplary obstacles the road at night. 
           [0025]      FIG. 8 b    is a thermal image of the road  FIG. 8   a.    
           [0026]      FIG. 8 c    is a contrast enhanced thermal image of the road of  FIG. 8   a.    
           [0027]      FIG. 8 d    is a polarization image of the road of  FIG. 8   a.    
           [0028]      FIG. 8 e    is a ColorFuse image of the roadway of  FIG. 8   d.    
           [0029]      FIG. 9 a    depicts a visible image of train tracks. 
           [0030]      FIG. 9 b    is a thermal image of the train tracks of  FIG. 9   a.    
           [0031]      FIG. 9 c    is a polarization image of the train tracks of  FIG. 9   c.    
           [0032]      FIG. 10 a    is a Stokes vector image S 0  of a maritime scene showing fishermen in a boat. 
           [0033]      FIG. 10 b    is a ColorFuse image of the scene of  FIG. 10   a.    
           [0034]      FIG. 10 c    is a Stokes vector images S 1  of the maritime of  FIG. 10   a.    
           [0035]      FIG. 10 d    is a Stokes vector images S 2  of the maritime of  FIG. 10   a.    
           [0036]      FIG. 10 e    is a DoLP image of the maritime scene of  FIG. 10   a.    
       
    
    
     DETAILED DESCRIPTION 
       [0037]      FIG. 1  illustrates a system  100  in accordance with an exemplary embodiment of the present disclosure. The system  100  comprises a polarimeter  101  mounted on a vehicle  103  and a signal processing unit  107 , which collect and analyze images of a surface  111  for detection and annunciation of an object  102 . Exemplary objects  102  shown in  FIG. 1  include an obstacle  104 , water or mud puddle  105 , and a roadway edge  106 . As used in this disclosure, the term “object” may refer to any object, pathway defect or area of interest, including in some embodiments humans or other animals. In  FIG. 1 , the obstacle  104  and the puddle  105  are objects the vehicle would want to avoid. The roadway edge  106  is an object that the vehicle would want to know the location of, in order to stay on a roadway. Thus in some embodiments, the objects  102  are objects to be avoided or located in order to safely navigate the vehicle  103 . In other embodiments, the objects  102  are items in need of location, for example, humans during search and rescue operations, as further discussed herein. 
         [0038]    The polarimeter  101  comprises a polarizing imaging device for recording polarized images, such as a digital camera or thermal imager that collects images. The vehicle  103  may be an automobile, watercraft, aircraft, or any navigable vehicle, or a human on foot. The polarimeter  101  collects raw image data of the roadway environment consisting of the surface  111  (a roadway, for example), and objects  102  such as the obstacle  104 , the water or mud puddle  105 , and the roadway edge  106 . 
         [0039]    The polarimeter  101  transmits raw image data to the signal processing unit  107 , which processes the data as further discussed herein. The processed data is then displayed to the operator on display  108  or detection is annunciated on an annunciator  110 , as further discussed herein. Although  FIG. 1  shows the polarimeter  101 , the signal processing unit  107 , the display  109 , and annunciator  110  as separate items, the polarimeter  101  and signal processing unit  107  are packaged into one device in certain embodiments and placed on the vehicle  103  such that the polarimeter has a view of the roadway, and with the display  109  and annunciator  110  packaged together and placed inside the vehicle. 
         [0040]    In the illustrated embodiment, the polarimeter  101  sends raw image data (not shown) to the signal processing unit  107  over a network or communication channel  108  and processed data sent to the display  109  and annunciator  110 . The signal processing unit  107  may be any suitable computer known in the art or future-developed. The signal processing unit  107  receives the raw image data, filters the data, and analyzes the data as discussed further herein to provide enhanced imagery and detections and annunciations. The network  108  may be of any type network or networks known in the art or future-developed, such as a simple communications cable, the internet backbone, Ethernet, Wifi, WiMax, broadband over power line, coaxial cable, and the like. The network  108  may be any combination of hardware, software, or both. Further, the network  108  could be resident in a sensor (not shown) housing both the polarimeter  101  and the signal processing unit  107 . 
         [0041]    In another exemplary embodiment (not shown), the vehicle  103  comprises manned or unmanned (autonomous) agricultural equipment in a farming environment and the objects  102  include obstacles along farm roads or in fields. In another embodiment, the vehicle  103  comprises manned or unmanned (autonomous) vessels that operate on waterways or oceans and the objects  102  are floating in the water. In another exemplary embodiment (not shown), the vehicle  103  comprises a person or vessel conducting search and rescue activities and objects  102  are victims of an incident involving bodies of water. In another exemplary embodiment (not shown), the vehicle  103  comprises manned or unmanned (autonomous) aircraft and objects  102  are those found in an airfield environment, including runways and the grassy areas in and around runways. In another exemplary embodiment (not shown), the vehicle  103  comprises railroad equipment and the objects  102  are those found in the environment around railroad tracks and switches. 
         [0042]      FIG. 2  depicts an exemplary polarimeter  101  and signal processing unit  107  according to an embodiment of the present disclosure. The polarimeter  101  comprises an objective imaging lens  1201 , a filter array  1203 , and a focal plane array  1202 . The objective imaging lens  1201  comprises a lens pointed at the surface  111  ( FIG. 1 ). The filter array  1203  filters the images received from the objective imaging lens system  1201 . The focal plane array  1202  comprises an array of light sensing pixels. 
         [0043]    The signal processing unit  107  comprises image processing logic  120  and system data  121 . In the exemplary signal processing unit  107  image processing logic  120  and system data  121  are shown as stored in memory  1123 . The image processing logic  120  and system data  121  may be implemented in hardware, software, or a combination of hardware and software. 
         [0044]    The signal processing unit  107  also comprises a processor  130 , which comprises a digital processor or other type of circuitry configured to run the image processing logic  120  by processing the image processing logic  120 , as applicable. The processor  130  communicates to and drives the other elements within the signal processing unit  107  via a local interface  1124 , which can include one or more buses. When stored in memory  1123 , the image processing logic  120  and the system data  121  can be stored and transported on any computer-readable medium for use by or in connection with logic circuitry, a processor, an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a “computer-readable medium” can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. Note that the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory. 
         [0045]    Exemplary system data  121  is depicted comprises:
       a. Raw image data (not pictured) from the polarimeter  101  ( FIG. 2 ) obtained from step  1001  of the method  100  ( FIG. 3 ).   b. Corrected image data (not pictured), which is the data that has been corrected for non-uniformity, optical distortion, and registration per step  1002  of the method  1000  ( FIG. 3 ).   c. Thermal (IR) and Polarization images obtained from step  1003  of the method  1000  ( FIG. 3 ).   d. Conversion of polarization and thermal data to multi-dimensional image data applied in step  1004  of the method  1000  ( FIG. 3 ).   e. Contrast enhancing algorithms applied to image data in step  1005  of the method  1000  ( FIG. 3 ).   f. Object detection algorithms applied to contrast enhanced image data in step  1006  of the method  1000  ( FIG. 3 ).   g. Image data applied to the display  109  and annunciator  110  in step  1007  of the method  1000  ( FIG. 3 ).   h. Thermal image data as described herein.   i. Hybrid thermal/polarization images as described herein.       
 
         [0055]    The image processing logic  120  executes the processes described herein with respect to  FIG. 3 . 
         [0056]    Referring to  FIG. 2 , an external interface device  126  connects to and communicates with the display  109  and annunciator  110 . The external interface device  126  may also communicate with or comprise an input device, for example, a keyboard, a switch, a mouse, a touchscreen, and/or other type of interface, which can be used to input data from a user of the system  100 . The external interface device  126  may also or alternatively communicate with or comprise a personal digital assistant (PDA), computer tablet device, laptop, portable or non-portable computer, cellular or mobile phone, or the like. The external interface device  126  may also or alternatively communicate with or comprise a non-personal computer, e.g., a server, embedded computer, field programmable gate array (FPGA), microprocessor, or the like. 
         [0057]    The external interface device  126  is shown as part of the signal processing unit  107  in the exemplary embodiment of  FIG. 2 . In other embodiments, the external interface device  126  may be outside of the signal processing unit  107 . 
         [0058]    The display device  109  may consist of a tv, lcd screen, monitor or any electronic device that conveys image data resulting from the method  1000  or is attached to a personal digital assistant (PDA), computer tablet device, laptop, portable or non-portable computer, cellular or mobile phone, or the like. The annunciator device  110  can consist of a warning buzzer, bell, flashing light, or any other auditory or visual or tactile means to warn the operator of the detection of an object or obstacle. 
         [0059]    In some embodiments, autonomous action may be taken based upon the objects  102  ( FIG. 1 ) detected. For example, the vehicle  103  ( FIG. 1 ) may automatically be directed to avoid objects  102 . In this regard, the external interface device  126  may interface with the vehicle  103  such that the processor  130  may direct the vehicle to swerve around an object  102 . In some cases where automatic action is taken, the annunciator  110  may not be required. 
         [0060]    In other embodiments, a Global Positioning System (“GPS”) device (not shown) may interface with the external interface device  126  to provide a position of the objects  102  detected. 
         [0061]    In the illustrated embodiment, the display  109  and annunciator  110  are shown as separate, but the annunciator  110  may be combined with the display  109 , and in another embodiments, annunciation could take the form of highlighted boxes or regions or another means used to highlight the object as part of the image data display. For example, an indicator box (e.g., a red box (not shown)), can provides a visual indication of an object  102  detected. 
         [0062]      FIG. 3  is a flowchart depicting exemplary architecture and functionality of the image processing logic  120  ( FIG. 2 ) in accordance with a method  1000 . In step  1001  of the method  1000 , the polarimeter  101  captures an image of a roadway scene from a vehicle on a roadway  111  ( FIG. 1 ) and sends raw image data to the signal processing unit  107  ( FIG. 1 ). 
         [0063]    In step  1002 , the signal processing unit  107  ( FIG. 1 ) corrects imager non-uniformity of the images received from the polarimeter  101 . Examples of imager non-uniformity include fixed pattern lines in the image, noisy pixels, bad pixels, bright spots, and the like. Algorithms that are known in the art may be used for correcting the imager non-uniformity. In some embodiments, step  1002  is not performed because the imager non-uniformity does not require correction. 
         [0064]    Additionally in step  1002 , the signal processing unit  107  removes image distortion from the image data. An example of image distortion is warping at the edges of the image caused by the objective imaging lens system. Algorithms that are known in the art may be used for correcting image distortion. Registration corrections may also be performed in step  1002 , using methods known in the art. 
         [0065]    In step  1003 , IR and polarization data products are computed. In this step, Stokes parameters (S 0 , S 1 , S 2 ) are calculated by weighted subtraction of the polarized image obtained in step  1002 . The LWIR imaging polarimeter measures both a radiance image and a polarization image. A radiance image is a standard image whereby each pixel in the image is a measure of the radiance, typically expressed in Watts/cm2-sr, reflected or emitted from that corresponding pixel area of the scene. Standard photographs and thermal images are radiance images, simply mappings of the radiance distribution emitted or reflected from the scene. A polarization image is a mapping of the polarization state distribution across the image. The polarization state distribution is typically expressed in terms of a Stokes image. 
         [0066]    Of the Stokes parameters, S 0  represents the conventional LWIR thermal image with no polarization information. S 1  and S 2  display orthogonal polarimetric information. Thus the Stokes vector, first introduced by G. G. Stokes in 1852, is useful for describing partially polarized light and is defined as 
         [0000]    
       
         
           
             
               
                 
                   
                     S 
                     → 
                   
                   = 
                   
                     
                       [ 
                       
                         
                           
                             
                               s 
                               o 
                             
                           
                         
                         
                           
                             
                               s 
                               1 
                             
                           
                         
                         
                           
                             
                               s 
                               2 
                             
                           
                         
                         
                           
                             
                               s 
                               3 
                             
                           
                         
                       
                       ] 
                     
                     = 
                     
                       [ 
                       
                         
                           
                             
                               
                                 I 
                                 0 
                               
                               + 
                               
                                 I 
                                 90 
                               
                             
                           
                         
                         
                           
                             
                               
                                 I 
                                 0 
                               
                               - 
                               
                                 I 
                                 90 
                               
                             
                           
                         
                         
                           
                             
                               
                                 I 
                                 45 
                               
                               - 
                               
                                 I 
                                 135 
                               
                             
                           
                         
                         
                           
                             
                               
                                 I 
                                 R 
                               
                               - 
                               
                                 I 
                                 L 
                               
                             
                           
                         
                       
                       ] 
                     
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
         [0000]    Where I 0  is the radiance that is linearly polarized in a direction making an angle of 0 degrees with the horizontal plane, I 90  is radiance linearly polarized in a direction making an angle of 90 degrees with the horizontal plane. Similarly I 45  and I 135  are radiance values of linearly polarized light making an angle of 45° and 135° with respect to the horizontal plane. Finally I R  and I L  are radiance values for right and left circularly polarized light. For this invention, right and left circularly polarized light is not necessary and the imaging polarimeter does not need to measure these states of polarization. For this reason, the Stokes vectors that we consider will be limited to the first 3 elements which express linearly polarized light only, 
         [0000]    
       
         
           
             
               
                 
                   
                     S 
                     → 
                   
                   = 
                   
                     
                       [ 
                       
                         
                           
                             
                               s 
                               o 
                             
                           
                         
                         
                           
                             
                               s 
                               1 
                             
                           
                         
                         
                           
                             
                               s 
                               2 
                             
                           
                         
                       
                       ] 
                     
                     = 
                     
                       [ 
                       
                         
                           
                             
                               
                                 I 
                                 0 
                               
                               + 
                               
                                 I 
                                 90 
                               
                             
                           
                         
                         
                           
                             
                               
                                 I 
                                 0 
                               
                               - 
                               
                                 I 
                                 90 
                               
                             
                           
                         
                         
                           
                             
                               
                                 I 
                                 45 
                               
                               - 
                               
                                 I 
                                 135 
                               
                             
                           
                         
                       
                       ] 
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
         [0067]    Another useful form of equation (2) is a normalized form of the equation given by 
         [0000]    
       
         
           
             
               
                 
                   
                     s 
                     → 
                   
                   = 
                   
                     
                       
                         s 
                         o 
                       
                        
                       
                         [ 
                         
                           
                             
                               1 
                             
                           
                           
                             
                               
                                 
                                   s 
                                   1 
                                 
                                 / 
                                 
                                   s 
                                   o 
                                 
                               
                             
                           
                           
                             
                               
                                 
                                   s 
                                   2 
                                 
                                 / 
                                 
                                   s 
                                   o 
                                 
                               
                             
                           
                         
                         ] 
                       
                     
                     = 
                     
                       
                         ( 
                         
                           
                             I 
                             0 
                           
                           + 
                           
                             I 
                             90 
                           
                         
                         ) 
                       
                        
                       
                         [ 
                         
                           
                             
                               1 
                             
                           
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       I 
                                       0 
                                     
                                     - 
                                     
                                       I 
                                       90 
                                     
                                   
                                   ) 
                                 
                                 / 
                                 
                                   ( 
                                   
                                     
                                       I 
                                       0 
                                     
                                     + 
                                     
                                       I 
                                       90 
                                     
                                   
                                   ) 
                                 
                               
                             
                           
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       I 
                                       45 
                                     
                                     = 
                                     
                                       I 
                                       135 
                                     
                                   
                                   ) 
                                 
                                 / 
                                 
                                   ( 
                                   
                                     
                                       I 
                                       0 
                                     
                                     + 
                                     
                                       I 
                                       90 
                                     
                                   
                                   ) 
                                 
                               
                             
                           
                         
                         ] 
                       
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0068]    The polarization state emitted or reflected from the roadway surface, surfaces to the side of the road, and objects or surfaces in the road depends on a number of factors including the angle of emission, the surface temperature of the surface, the micro-roughness of the surface (texture), the complex refractive index of the surface and the background temperature of the surrounding environment. The invention here primarily makes use of the fact that the polarization state of light emitted and reflected from the surfaces and objects is a function of angle of emission and different surface texture. 
         [0069]    The emissivity of an object is determined from Kirchoff&#39;s radiation law. The most familiar form of Kirchoff&#39;s law is gives the emissivity of a surface E in terms of the reflectance r, given by 
         [0000]      ε(θ,φ)=1− r (θ)  (4)
 
         [0000]    where θ is the angle between the surface normal and the camera&#39;s line of sight. The more general equations for Kirchoff&#39;s law are given by 
         [0000]      ε p (θ)=1− r   p (θ)  (5)
 
         [0000]      and 
         [0000]      ε s (θ)=1− r   s (θ)  (6)
 
         [0000]    where the subscripts p and s denote the emissivity and reflectance of particular polarization states. The p-state indicates the plane of emission for light that is linearly polarized in a plane that contains the surface normal and the line of sight to the camera. For example, if the camera is looking down at a horizontal surface, the p-state of polarization would appear vertically polarized. The s-state of polarization is perpendicular to the p-state. Note that we have suppressed the temperature and wavelength dependence in equations 4-6. 
         [0070]    Substituting equations (5) and (6) into equation (3) gives 
         [0000]    
       
         
           
             
               
                 
                   
                     s 
                     → 
                   
                   = 
                   
                     
                       s 
                       0 
                     
                      
                     
                       [ 
                       
                         
                           
                             1 
                           
                         
                         
                           
                             
                               
                                 P 
                                  
                                 
                                   ( 
                                   θ 
                                   ) 
                                 
                               
                                
                               
                                 cos 
                                  
                                 
                                   ( 
                                   φ 
                                   ) 
                                 
                               
                             
                           
                         
                         
                           
                             
                               
                                 P 
                                  
                                 
                                   ( 
                                   θ 
                                   ) 
                                 
                               
                                
                               
                                 sin 
                                  
                                 
                                   ( 
                                   φ 
                                   ) 
                                 
                               
                             
                           
                         
                       
                       ] 
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where φ is the angle that the plane of incidence makes with the horizontal plane and 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                      
                     
                       ( 
                       θ 
                       ) 
                     
                   
                   = 
                   
                     ( 
                     
                       
                         
                           
                             ɛ 
                             s 
                           
                            
                           
                             ( 
                             θ 
                             ) 
                           
                         
                         - 
                         
                           
                             ɛ 
                             p 
                           
                            
                           
                             ( 
                             θ 
                             ) 
                           
                         
                       
                       
                         
                           
                             ɛ 
                             s 
                           
                            
                           
                             ( 
                             θ 
                             ) 
                           
                         
                         + 
                         
                           
                             ɛ 
                             p 
                           
                            
                           
                             ( 
                             θ 
                             ) 
                           
                         
                       
                     
                     ) 
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
         [0071]    Equation 8 can be written out more explicitly as 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                      
                     
                       ( 
                       θ 
                       ) 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         
                           1 
                           - 
                           
                             
                               r 
                               s 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                           - 
                           
                             ( 
                             
                               1 
                               - 
                               
                                 
                                   r 
                                   p 
                                 
                                  
                                 
                                   ( 
                                   θ 
                                   ) 
                                 
                               
                             
                             ) 
                           
                         
                         
                           1 
                           + 
                           
                             
                               r 
                               s 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                           + 
                           1 
                           + 
                           
                             
                               r 
                               p 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                         
                       
                       ) 
                     
                     = 
                     
                       ( 
                       
                         
                           
                             
                               r 
                               p 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                           - 
                           
                             
                               r 
                               s 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                         
                         
                           2 
                           + 
                           
                             
                               r 
                               p 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                           + 
                           
                             
                               r 
                               s 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                         
                       
                       ) 
                     
                   
                 
               
               
                 
                   ( 
                   9 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where r p  and r s  are given by the Fresnel equations for reflection 
         [0000]    
       
         
           
             
               
                 
                   
                     r 
                     p 
                   
                   = 
                   
                     
                       
                         
                           
                             n 
                             2 
                           
                            
                           
                             cos 
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                         
                         - 
                         
                           
                             
                               n 
                               2 
                             
                             - 
                             
                               
                                 sin 
                                 2 
                               
                                
                               
                                 ( 
                                 θ 
                                 ) 
                               
                             
                           
                         
                       
                       
                         
                           
                             n 
                             2 
                           
                            
                           
                             cos 
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                         
                         + 
                         
                           
                             
                               n 
                               2 
                             
                             - 
                             
                               
                                 sin 
                                 2 
                               
                                
                               
                                 ( 
                                 θ 
                                 ) 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   
                     9 
                      
                     a 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     r 
                     s 
                   
                   = 
                   
                     
                       
                         
                           cos 
                            
                           
                             ( 
                             θ 
                             ) 
                           
                         
                         - 
                         
                           
                             
                               n 
                               2 
                             
                             - 
                             
                               
                                 sin 
                                 2 
                               
                                
                               
                                 ( 
                                 θ 
                                 ) 
                               
                             
                           
                         
                       
                       
                         
                           cos 
                            
                           
                             ( 
                             θ 
                             ) 
                           
                         
                         + 
                         
                           
                             
                               n 
                               2 
                             
                             - 
                             
                               
                                 sin 
                                 2 
                               
                                
                               
                                 ( 
                                 θ 
                                 ) 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   
                     9 
                      
                     b 
                   
                   ) 
                 
               
             
           
         
       
     
         [0072]    Note that P(θ) does not explicitly depend on the angle φ that the plane of incidence makes with the horizontal plane. The angle φ is critical to determine the orientation of plane of incidence and ultimately the azimuthal angle of the surface normal. The angle φ can be determined from the following angle, 
         [0000]    
       
         
           
             
               
                 
                   φ 
                   = 
                   
                     arctan 
                      
                     
                       ( 
                       
                         
                           s 
                           2 
                         
                         
                           s 
                           1 
                         
                       
                       ) 
                     
                   
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
     
         [0073]    The angle θ can be determined a number of ways. A method for determining θ and φ from a normalized Stokes image (Equation 3) are known in the art. 
         [0074]    Also in step  1003 , a degree of linear polarization (DoLP) image is computed from the Stokes images. A DoLP image is useful for providing contrast for roadway surface and objects in the road, and can be calculated as follows: 
         [0000]    
       
         
           
             
               
                 
                   DoLP 
                   = 
                   
                     
                       
                         
                           ( 
                           
                             
                               s 
                               1 
                             
                             / 
                             
                               s 
                               o 
                             
                           
                           ) 
                         
                         2 
                       
                       + 
                       
                         
                           ( 
                           
                             
                               s 
                               2 
                             
                             / 
                             
                               s 
                               o 
                             
                           
                           ) 
                         
                         2 
                       
                     
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
             
               
                 or 
               
               
                 
                     
                 
               
             
             
               
                 
                   DoLP 
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               ɛ 
                               s 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                           - 
                           
                             
                               ɛ 
                               p 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                         
                         
                           
                             
                               ɛ 
                               s 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                           + 
                           
                             
                               ɛ 
                               p 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                         
                       
                       ) 
                     
                     = 
                     
                       ( 
                       
                         
                           
                             
                               r 
                               p 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                           - 
                           
                             
                               r 
                               s 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                         
                         
                           2 
                           + 
                           
                             
                               r 
                               p 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                           + 
                           
                             
                               r 
                               s 
                             
                              
                             
                               ( 
                               θ 
                               ) 
                             
                           
                         
                       
                       ) 
                     
                   
                 
               
               
                 
                   ( 
                   12 
                   ) 
                 
               
             
           
         
       
     
         [0075]    Note that DoLP is linear polarization. As one with skill in the art would know, in some situations polarization that is not linear (e.g., circular) may be desired. Thus in other embodiments, step  1004  may use polarization images derived from any combination of S 0 , S 1 , S 2 , or S 3  and is not limited to DoLP. 
         [0076]    The DoLP image is one available image used to view polarization contrast in an image. Another alternative image to view polarization content is a “ColorFuse” image that is generated by mapping the radiance, DoLP, and orientation images to a color map. Persons with skill in the art makes the following mapping of polarization data to a hue-saturation-value representation for color: 
         [0077]    S 0 =value 
         [0078]    DoLP=saturation 
         [0079]    Orientation=hue 
         [0080]    This representation enables display of all optical information (radiance and polarization) in a single image and provides a means to show both radiometric and polarization contrast enhancing understanding of the scene. In many cases where polarization contrast is strong, this representation provides scene context for the surfaces or objects that are polarized. Those experienced in the art can imagine other ways of doing this. 
         [0081]    The ColorFuse is one embodiment of multidimensional representation that can be produced in step  1004 . Those knowledgeable in the art can conceive similar mappings. For one example, the DoLP information may be emphasized when radiance values are low. 
         [0082]    As mentioned above, the polarization state emitted or reflected from the surface of objects or surfaces in the imaged scene depends on a number of factors including the angle of emission, the surface temperature of the surface, the micro-roughness or texture of the surface, and the complex refractive index of the surface. Generally speaking then, the contrast of surfaces and objects in the scene due to polarization are dependent on the geometry and the material or surface properties of the objects in the scene. While surface temperature contributes to polarization signature contrast, temperature differences of objects in the scene are not necessary in order for there to be polarization contrast. This is important because frequently many objects in an imaged scene can be at the same or very similar temperatures and hence show little contrast. 
         [0083]    Because the underlying optical radiation depends on emission, no additional light sources, illumination, or ambient light is required for polarization imaging. This is a key point and differentiates this approach from all of the prior art. Further, the approach works equally well during the night time as it does during the day. 
         [0084]    In step  1005 , contrast enhancing algorithms that are known in the art are applied to the multidimensional image from step  1004 . The multi-dimensional data exploits the polarization data to significantly enhance the information content in a scene. Non-restrictive examples include global mean, variance, and higher order moment analysis, Principal Component Analysis, or Linear Discriminate Analysis, computation of the statistics of the multidimensional data as a whole and then computation of local values based on a kernel convolved with the image as a whole and then normalized by global statistics of the scene. 
         [0085]    In step  1006 , object detection algorithms that are known in the art are applied to the contrast enhanced data from step  1005 . Non-restrictive examples of object detection algorithms include setting manually or automatically a threshold value based on the image statistics, segmenting portions of the image based on the contrast enhancements, edge detection, and morphological properties. 
         [0086]    In step  1007 , detected objects may then be annunciated to the user through visual or auditory means. Non-restrictive examples includes bells, buzzers or lights to draw the operator&#39;s attention to the display, or indications on the display such as distinctive colors or boxes in the region of the obstacle or surface. In addition or alternatively, in step  1007  enhanced contrast images may be displayed to the user (not shown). 
         [0087]    In other embodiments, steps  1003 ,  1004 ,  1005 , and  1006  are used in combinations that omit one or more of the steps. In other embodiments, the polarization image data, or the multi-dimensional (ColorFuse) data, may be viewed by humans for object detection, and no algorithms are applied. 
         [0088]    Algorithms that exploit a combination of image features extracted from a LWIR imaging polarimeter can be used to detect potential obstacles or roadway edges. In the case of train tracks, algorithms could be used to confirm continuity of the tracks automatically. Once potential noteworthy features are detected, they can be automatically highlighted for the operator, and a warning can be given through some annunciation mechanism (buzzer or light). Algorithms could also potentially be used to exploit the orientation information to help improve understanding of the image such as segmentation or shape recognition. 
         [0089]    For the purposes of operating a vehicle, the enhanced contrast enables the mapping of features in the imaged scene that, through operator perception or automated detection and warning, improves the safety of the operator, or in the case of autonomously operated equipment such as agricultural equipment, provides autonomous obstacle avoidance to the steering or navigation systems. Specifically, improved detection and recognition of obstacles will allow the operator to maneuver the vehicle (or vessel) to avoid obstacles. Improved detection and perception of roadway edges will reduce chances of inadvertently leaving the roadway. This is especially true at night when the operator&#39;s vision is limited by darkness. 
         [0090]    As discussed herein, the system and method of the present disclosure adds a polarimetric signature to the information that was previously attainable by an IR camera, i.e., temporal, special and IR signature. These four categories of information can be used simultaneously to classify/categorize objects detected. Further, the classification/categorization of the detected objects can influence evasive action to be taken by a vehicle. For example, a detected object in a roadway may be classified as an obstacle that needs to be avoided, rather than a pothole the vehicle is capable of driving over. Further, multiple objects may be independently and simultaneously classified into separate groups or sub-groups based on their temporal, spatial, IR, and/or polarimetric signatures in accordance with given criteria. 
         [0091]      FIG. 4 a    depicts a visible image of a roadway  400  at night.  FIG. 4 b    is a thermal image of the same roadway  400  at night. Note that in  FIG. 4 b   , the roadway  400  and surrounding terrain have nearly the same temperature and hence there is little contrast between the roadway  400  and a shoulder  401  of the road in the thermal image.  FIG. 4 c    depicts a polarization image of the roadway  400  obtained with the system and method according to an exemplary embodiment of the present disclosure. The polarization image in  4   c  shows strong contrast of the road  400  and the shoulder  401  is easily discernable. A white stripe  402  that parallels the roadway  400  on the left hand side is a sidewalk. The polarization image in  4   c  was obtained with no external light source. 
         [0092]      FIG. 5 a    is a visible image of a roadway  500  during the daytime.  FIG. 5 b    is a thermal image of the roadway  500  of  FIG. 5 a   . The roadway  500  and other scene elements show confusing contrast in the thermal image of  FIG. 5 b   .  FIG. 5 c    is a polarization image of the roadway  500  of FIG.  5   a  obtained with the system and method according to an exemplary embodiment of the present disclosure. The polarization image of  FIG. 5 c    shows strong contrast of only the roadway  500 . A sidewalk  501  that parallels the road on the left hand side and a driveway  502  on the right are easily discernable in the polarization image of  FIG. 5 c   . The sidewalk  501  and the driveway  502  are not easily perceptible in the thermal image of  FIG. 5   b.    
         [0093]      FIG. 6 a    is visible image of a roadway  600  that has a median  601  and sidewalk  602 , shown at night.  FIG. 6 b    is a thermal image of the roadway  600  of  FIG. 6 a   , also at night. The roadway and surrounding terrain have similar temperatures and hence there is weak contrast between the roadway and the media of the road in the thermal image. 
         [0094]      FIG. 6 c    is a polarization image of the roadway  600  of  FIG. 6 a    obtained with the system and method according to an exemplary embodiment of the present disclosure. The polarization image of  FIG. 6 c    shows strong contrast of the roadway  600 . The sidewalk  602  that parallels the road on the right hand side and the median  601  are easily discernable in the polarization image of  FIG. 6 c   . The sidewalk  602  and the median  601  are not easily perceptible in the thermal image of  FIG. 6   b.    
         [0095]      FIG. 7 a    is a visible image of a dirt road  700  depicting exemplary obstacles  701  on a road  700 . The obstacles  701  comprise wood planks in the image.  FIG. 7 b    is a thermal image of the road  700  of  FIG. 7 a   . In the image of  FIG. 7 b   , the obstacles  701  are easier to discern than in the visible image of  FIG. 7 a   .  FIG. 7 c    is a contrast enhanced thermal image of the road  700  of  FIG. 7   a.    
         [0096]      FIG. 7 d    is a polarization image of the road  700  of  FIG. 7 a   . The obstacle  701  in this image is easily discerned, though the polarization image does not provide much context to the obstacle in relation to the road  700 .  FIG. 7 e    is a ColorFuse image of the roadway of  FIG. 7 d   . The ColorFuse image shows both thermal and polarimetric data in a single image, and provides the greatest contrast. 
         [0097]      FIG. 8 a    is a visible image of a dirt road  800  depicting exemplary obstacles  801  on a road  800  at night. The obstacles  801  comprise wet dirt and mud in the image. These are potential hazards which might immobilize some ground vehicles.  FIG. 8 b    is a thermal image of the road  800  of  FIG. 8 a   .  FIG. 8 c    is a contrast enhanced thermal image of the road  800  of  FIG. 8   a.    
         [0098]      FIG. 8 d    is a polarization image of the road  800  of  FIG. 8 a   . The obstacles  801  in this image are easily discerned, though the polarization image does not provide much context to the obstacles in relation to the road  800 .  FIG. 8 e    is a ColorFuse image of the roadway of  FIG. 8 d   . The ColorFuse image shows both thermal and polarimetric data in a single image, and provides the greatest contrast. The ColorFuse image of  FIG. 8 e    shows how the combination of thermal and polarization data products can be used to provide a good representation of road surfaces. 
         [0099]      FIG. 9 a    (inset photo) is a visible image of train tracks  900  depicting exemplary segmentation of the rails in a railroad environment.  FIG. 9 b    is a thermal image, in which identification of the tracks is difficult due to different temperatures of the objects in areas adjacent to the tracks.  FIG. 9 c    is a polarimetric image of the train tracks  900  of  FIG. 9 a   , and show good delineation of the tracks. 
         [0100]    Similarly, for vessels navigating a body of water that need to avoid obstacles in the water, an LWIR polarimeter can be used to enhance contrast between obstacles that break the surface of the water and the water background. This can be particularly effective since objects floating in water tend to have the same temperature as the water they&#39;re floating in and can hence be difficult to detect through the radiometric (thermal) image. 
         [0101]      FIGS. 10 a , 10 c , and 10 d    are Stokes vector images S 0 , S 1  and S 2 , respectively, of a maritime scene showing fishermen in a boat.  FIG. 10 e    is a DoLP image of the same scene.  FIG. 10 b    is a ColorFuse image of the scene. The ColorFuse image shows improvement in contrast for obstacle avoidance for vessels or advantage for search and rescue.