Abstract:
This invention fuses spectral information from multiple imagers of an unmanned ground vehicle (UGV). Since the imagers contain different spectral information for each pixel location, the invention provides highly accurate targeting and guidance information to the UGV. The invention applies a robust 2-step image registration process for alignment of images captured by each of the multiple imagers to automatically identify targets of interest, so that the UGV can move toward the targets. This two-step image registration process can achieve sub-pixel accuracy. After registration, a new multispectral image is formed with pixels containing spectral information from all imagers. The invention further incorporates an accurate anomaly detection algorithm to help detect new targets in the scene, and incorporates advanced composition estimation algorithms to determine the composition of targets. Finally, the invention allows users to interact with the target detection results through a user friendly graphical user interface.

Description:
BACKGROUND OF THE INVENTION 
     At present, devices and methods for waypoint target generation and mission spooling for mobile ground robots or Unmanned Ground Vehicle (UGV) require an operator to manually enter waypoints. In order to implement automated waypoint generation, methods of image registration, parallax compensation and change detection would be required. However, it is well known that image registration may not be perfect. In addition, parallax is an important practical issue during data collection. Hence, a robust change detection algorithm such as CKRX is needed. 
     Support Vector Machines (SVM) and non-deep neural networks (NN) have been used in many pattern classification applications. However, it is believed that there is a lot of room for further improvement. This is because SVM and non-deep NN have only one or two layers of tunable parameters. Since pattern recognition and concentration estimation are complex and involve sophisticated features, SVM and non-deep NN may be restricted in achieving high classification rate. 
     BRIEF SUMMARY OF THE INVENTION 
     The present invention utilizes left and right multispectral imagers to apply a novel robust 2-step image registration process for image alignment that improves downstream identification of interesting targets automatically, so that the UGV can move toward the targets. 
     One embodiment of the present invention is to provide a method and system, which utilizes multiple multi-spectral imagers for UGV guidance and target acquisition. Since the different imagers of a UGV may contain different spectral information for each pixel location, this invention provides several novel and high performance sub-systems to fuse spectral information from the multiple imagers to provide highly accurate targeting and guidance information to the UGV. The invention presents a method for use with a UGV utilizing multiple multispectral imagers and an onboard PC. 
     An embodiment of the present invention incorporates a novel two-step image registration process that can achieve sub-pixel accuracy. After registration, a new multispectral image is formed with each pixel containing spectral information from all imagers. 
     Another embodiment of the present invention incorporates an accurate anomaly detection process to help detect new targets in the scene. 
     Another embodiment of the present invention is to incorporate Advanced Composition Estimation algorithms to determine the composition of targets. 
     Another embodiment of the present invention is to allow users to interact with the target detection results through a user friendly graphical user interface. 
    
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
         FIG. 1  shows a block diagram of multispectral imagers using anomaly detection and composition estimation to select interesting locations for automatic UGV guidance and targeting. 
         FIG. 2  shows the two-step image registration approach. 
         FIGS. 3 a   )- 3   d ) show alignment results with the two-step registration approach. 
         FIGS. 4 a   ) and  4   b ) show error difference images with the two-step registration process. 
         FIGS. 5 a   )- 5   c ) show the alignment accuracy with a pixel-distance based measure in the two-step registration process. 
         FIGS. 6 a   )- 6   d ) show alignment results with the two-step registration process using a partial image section from one of the Middlebury stereo pair images. 
         FIG. 7  shows computed pixel distances in the two-step registration process showing the second step reduces the registration errors to smaller subpixel levels. 
         FIGS. 8 a   )- 8   c ) show the impact of precise registration on anomaly detection performance. 
         FIG. 9  shows resultant ROC curves for each step of the alignment in the two-step registration process. 
         FIG. 10  shows NASA-KSC image and tagged pixels with ground truth information. 
         FIGS. 11 a   )- 11   c ) show results of different algorithms. 
         FIGS. 12 a   )- 12   b ) show comparison of kernel RX with background subsampling (2×2) options and CKRX. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       FIG. 1  shows the block diagram of the present invention. There is a preprocessing step to first convert the 8-bit numbers in the image data to decompanded numbers of a range with greater than 8 bits. The conversion is optionally done through a look-up table as in the example shown below. 
     
       
         
               
             
               
               
               
             
               
               
               
             
           
               
                 TABLE 1 
               
             
             
               
                   
               
               
                 Example decompanding table between 8-bit and 11-bit numbers. 
               
             
          
           
               
                   
                 8-bit 
                 12-bit 
               
               
                   
                   
               
             
          
           
               
                   
                 0 
                 0 
               
               
                   
                 1 
                 2 
               
               
                   
                 2 
                 3 
               
               
                   
                 3 
                 3 
               
               
                   
                 4 
                 4 
               
               
                   
                 5 
                 5 
               
               
                   
                 6 
                 5 
               
               
                   
                 7 
                 6 
               
               
                   
                 8 
                 7 
               
               
                   
                 9 
                 8 
               
               
                   
                 10 
                 9 
               
               
                   
                 11 
                 10 
               
               
                   
                 12 
                 11 
               
               
                   
                 13 
                 12 
               
               
                   
                 14 
                 14 
               
               
                   
                 15 
                 15 
               
               
                   
                 16 
                 16 
               
               
                   
                 17 
                 18 
               
               
                   
                 18 
                 19 
               
               
                   
                 19 
                 20 
               
               
                   
                 20 
                 22 
               
               
                   
                 21 
                 24 
               
               
                   
                 22 
                 25 
               
               
                   
                 23 
                 27 
               
               
                   
                 24 
                 29 
               
               
                   
                 25 
                 31 
               
               
                   
                 26 
                 33 
               
               
                   
                 27 
                 35 
               
               
                   
                 28 
                 37 
               
               
                   
                 29 
                 39 
               
               
                   
                 30 
                 41 
               
               
                   
                 31 
                 43 
               
               
                   
                 32 
                 46 
               
               
                   
                 33 
                 48 
               
               
                   
                 34 
                 50 
               
               
                   
                 35 
                 53 
               
               
                   
                 36 
                 55 
               
               
                   
                 37 
                 58 
               
               
                   
                 38 
                 61 
               
               
                   
                 39 
                 63 
               
               
                   
                 40 
                 66 
               
               
                   
                 41 
                 69 
               
               
                   
                 42 
                 72 
               
               
                   
                 43 
                 75 
               
               
                   
                 44 
                 78 
               
               
                   
                 45 
                 81 
               
               
                   
                 46 
                 84 
               
               
                   
                 47 
                 87 
               
               
                   
                 48 
                 90 
               
               
                   
                 49 
                 94 
               
               
                   
                 50 
                 97 
               
               
                   
                 51 
                 100 
               
               
                   
                 52 
                 104 
               
               
                   
                 53 
                 107 
               
               
                   
                 54 
                 111 
               
               
                   
                 55 
                 115 
               
               
                   
                 56 
                 118 
               
               
                   
                 57 
                 122 
               
               
                   
                 58 
                 126 
               
               
                   
                 59 
                 130 
               
               
                   
                 60 
                 134 
               
               
                   
                 61 
                 138 
               
               
                   
                 62 
                 142 
               
               
                   
                 63 
                 146 
               
               
                   
                 64 
                 150 
               
               
                   
                 65 
                 154 
               
               
                   
                 66 
                 159 
               
               
                   
                 67 
                 163 
               
               
                   
                 68 
                 168 
               
               
                   
                 69 
                 172 
               
               
                   
                 70 
                 177 
               
               
                   
                 71 
                 181 
               
               
                   
                 72 
                 186 
               
               
                   
                 73 
                 191 
               
               
                   
                 74 
                 196 
               
               
                   
                 75 
                 201 
               
               
                   
                 76 
                 206 
               
               
                   
                 77 
                 211 
               
               
                   
                 78 
                 216 
               
               
                   
                 79 
                 221 
               
               
                   
                 80 
                 226 
               
               
                   
                 81 
                 231 
               
               
                   
                 82 
                 236 
               
               
                   
                 83 
                 241 
               
               
                   
                 84 
                 247 
               
               
                   
                 85 
                 252 
               
               
                   
                 86 
                 258 
               
               
                   
                 87 
                 263 
               
               
                   
                 88 
                 269 
               
               
                   
                 89 
                 274 
               
               
                   
                 90 
                 280 
               
               
                   
                 91 
                 286 
               
               
                   
                 92 
                 292 
               
               
                   
                 93 
                 298 
               
               
                   
                 94 
                 304 
               
               
                   
                 95 
                 310 
               
               
                   
                 96 
                 316 
               
               
                   
                 97 
                 322 
               
               
                   
                 98 
                 328 
               
               
                   
                 99 
                 334 
               
               
                   
                 100 
                 341 
               
               
                   
                 101 
                 347 
               
               
                   
                 102 
                 354 
               
               
                   
                 103 
                 360 
               
               
                   
                 104 
                 367 
               
               
                   
                 105 
                 373 
               
               
                   
                 106 
                 380 
               
               
                   
                 107 
                 387 
               
               
                   
                 108 
                 394 
               
               
                   
                 109 
                 401 
               
               
                   
                 110 
                 408 
               
               
                   
                 111 
                 415 
               
               
                   
                 112 
                 422 
               
               
                   
                 113 
                 429 
               
               
                   
                 114 
                 436 
               
               
                   
                 115 
                 443 
               
               
                   
                 116 
                 450 
               
               
                   
                 117 
                 458 
               
               
                   
                 118 
                 465 
               
               
                   
                 119 
                 472 
               
               
                   
                 120 
                 480 
               
               
                   
                 121 
                 487 
               
               
                   
                 122 
                 495 
               
               
                   
                 123 
                 503 
               
               
                   
                 124 
                 510 
               
               
                   
                 125 
                 518 
               
               
                   
                 126 
                 526 
               
               
                   
                 127 
                 534 
               
               
                   
                 128 
                 542 
               
               
                   
                 129 
                 550 
               
               
                   
                 130 
                 558 
               
               
                   
                 131 
                 566 
               
               
                   
                 132 
                 575 
               
               
                   
                 133 
                 583 
               
               
                   
                 134 
                 591 
               
               
                   
                 135 
                 600 
               
               
                   
                 136 
                 608 
               
               
                   
                 137 
                 617 
               
               
                   
                 138 
                 626 
               
               
                   
                 139 
                 634 
               
               
                   
                 140 
                 643 
               
               
                   
                 141 
                 652 
               
               
                   
                 142 
                 661 
               
               
                   
                 143 
                 670 
               
               
                   
                 144 
                 679 
               
               
                   
                 145 
                 688 
               
               
                   
                 146 
                 697 
               
               
                   
                 147 
                 706 
               
               
                   
                 148 
                 715 
               
               
                   
                 149 
                 724 
               
               
                   
                 150 
                 733 
               
               
                   
                 151 
                 743 
               
               
                   
                 152 
                 752 
               
               
                   
                 153 
                 761 
               
               
                   
                 154 
                 771 
               
               
                   
                 155 
                 781 
               
               
                   
                 156 
                 790 
               
               
                   
                 157 
                 800 
               
               
                   
                 158 
                 810 
               
               
                   
                 159 
                 819 
               
               
                   
                 160 
                 829 
               
               
                   
                 161 
                 839 
               
               
                   
                 162 
                 849 
               
               
                   
                 163 
                 859 
               
               
                   
                 164 
                 869 
               
               
                   
                 165 
                 880 
               
               
                   
                 166 
                 890 
               
               
                   
                 167 
                 900 
               
               
                   
                 168 
                 911 
               
               
                   
                 169 
                 921 
               
               
                   
                 170 
                 932 
               
               
                   
                 171 
                 942 
               
               
                   
                 172 
                 953 
               
               
                   
                 173 
                 964 
               
               
                   
                 174 
                 974 
               
               
                   
                 175 
                 985 
               
               
                   
                 176 
                 996 
               
               
                   
                 177 
                 1007 
               
               
                   
                 178 
                 1018 
               
               
                   
                 179 
                 1029 
               
               
                   
                 180 
                 1040 
               
               
                   
                 181 
                 1051 
               
               
                   
                 182 
                 1062 
               
               
                   
                 183 
                 1074 
               
               
                   
                 184 
                 1085 
               
               
                   
                 185 
                 1096 
               
               
                   
                 186 
                 1108 
               
               
                   
                 187 
                 1119 
               
               
                   
                 188 
                 1131 
               
               
                   
                 189 
                 1142 
               
               
                   
                 190 
                 1154 
               
               
                   
                 191 
                 1166 
               
               
                   
                 192 
                 1177 
               
               
                   
                 193 
                 1189 
               
               
                   
                 194 
                 1201 
               
               
                   
                 195 
                 1213 
               
               
                   
                 196 
                 1225 
               
               
                   
                 197 
                 1237 
               
               
                   
                 198 
                 1249 
               
               
                   
                 199 
                 1262 
               
               
                   
                 200 
                 1274 
               
               
                   
                 201 
                 1286 
               
               
                   
                 202 
                 1299 
               
               
                   
                 203 
                 1311 
               
               
                   
                 204 
                 1324 
               
               
                   
                 205 
                 1336 
               
               
                   
                 206 
                 1349 
               
               
                   
                 207 
                 1362 
               
               
                   
                 208 
                 1374 
               
               
                   
                 209 
                 1387 
               
               
                   
                 210 
                 1400 
               
               
                   
                 211 
                 1413 
               
               
                   
                 212 
                 1426 
               
               
                   
                 213 
                 1439 
               
               
                   
                 214 
                 1452 
               
               
                   
                 215 
                 1465 
               
               
                   
                 216 
                 1479 
               
               
                   
                 217 
                 1492 
               
               
                   
                 218 
                 1505 
               
               
                   
                 219 
                 1519 
               
               
                   
                 220 
                 1532 
               
               
                   
                 221 
                 1545 
               
               
                   
                 222 
                 1559 
               
               
                   
                 223 
                 1573 
               
               
                   
                 224 
                 1586 
               
               
                   
                 225 
                 1600 
               
               
                   
                 226 
                 1614 
               
               
                   
                 227 
                 1628 
               
               
                   
                 228 
                 1642 
               
               
                   
                 229 
                 1656 
               
               
                   
                 230 
                 1670 
               
               
                   
                 231 
                 1684 
               
               
                   
                 232 
                 1698 
               
               
                   
                 233 
                 1712 
               
               
                   
                 234 
                 1727 
               
               
                   
                 235 
                 1741 
               
               
                   
                 236 
                 1755 
               
               
                   
                 237 
                 1770 
               
               
                   
                 238 
                 1784 
               
               
                   
                 239 
                 1799 
               
               
                   
                 240 
                 1814 
               
               
                   
                 241 
                 1828 
               
               
                   
                 242 
                 1843 
               
               
                   
                 243 
                 1858 
               
               
                   
                 244 
                 1873 
               
               
                   
                 245 
                 1888 
               
               
                   
                 246 
                 1903 
               
               
                   
                 247 
                 1918 
               
               
                   
                 248 
                 1933 
               
               
                   
                 249 
                 1948 
               
               
                   
                 250 
                 1963 
               
               
                   
                 251 
                 1979 
               
               
                   
                 252 
                 1994 
               
               
                   
                 253 
                 2009 
               
               
                   
                 254 
                 2025 
               
               
                   
                 255 
                 2033 
               
               
                   
                   
               
             
          
         
       
     
     Then, the resultant radiance factor (I/F) values is obtained by multiplying the decompanded numbers with the corresponding Radiance Scaling Factor (RSF) values, which are used to linearly map the 16-bit values to a radiance factor (I/F) value that should be between 0 and 1. These values are found in the label (LBL) file of each image. For estimating the image registration parameters, the left and right camera RGB images are used. After transforming them to radiance factor (I/F) values, a RGB to gray transformation is applied. Then, the two-step registration approach is applied to align these two RGB images, and the registration parameters for each step are obtained. All other stereo band images are then aligned using the registration parameters and a multispectral image cube is formed. Then, a robust anomaly detection algorithm is applied to locate interesting targets in the scene. Moreover, composition estimation algorithm is applied to determine the composition of each anomaly. With the provided set of anomaly detection and composition estimation tools, interesting locations are selected which can be used to guide the UGV to these locations. 
     The multiple imagers can be optical cameras or hyperspectral imagers. 
     Accurate image registration is important in aligning the two multispectral images. After image registration/alignment, anomaly detection can then be performed.  FIG. 2  shows the block diagram of the two-step image registration approach. Given two images, the first step corresponds to initialization with random sample consensus (RANSAC). In this first step, Speeded Up Robust Features (SURF) are detected in both images; these features are then matched; followed by applying RANSAC to estimate the geometric transformation. Assuming one image is the reference image; the other image content is then projected to a new image that is aligned with the reference image using the estimated geometric transformation with RANSAC. The second step uses the RANSAC-aligned image and the reference image and applies diffeomorphic registration, as described below. 
     The diffeomorphic registration algorithm solves the following problem: Given two images S and T, defined over Ω⊂R 2 , find the function pair (m(ξ) g(ξ), ξεΩ that optimizes a similarity measure E Sim (S,T,φ m,g ) between S and T subject to the constraints: 
                       ∫   Ω     ⁢   m     =        Ω                (     1   ⁢   a     )                   th   high     &gt;     m   ⁡     (   ξ   )       &gt;       th     low   ,       ⁢   ξ       ⁢           ∈     Ω   ′     ⋐   Ω           (     1   ⁢   b     )               
where th low &gt;0 ensures that φ m,g  is a diffeomorphism. Here, Ω denotes the image domain, m(ξ) corresponds to transformation Jacobian and g(ξ) corresponds to curl where ξεΩ. The transformation is parametrized by m(ξ) and g(ξ) and is denoted by φ m,g . E Sim  corresponds to a similarity measure which in this case SSD (Sum of Squared Differences) is used.
 
     It should be noted that when th low ≈th high  inequality constraint (1b) effectively imposes the incompressibility constraint in a subregion Ω of the image domain Ω. 
     Given an image pair S (study) and T (template), and th low  and th high , the diffeomorphic registration algorithm consists of the following implementation steps: Step 1. Compute unconstrained gradients, ∇ m E Sim (S,T,φ m     i     ,g     i   ) and ∇ g E Sim (S,T,φ m     i     ,g     i   ) Step 2. 
     a. Terminate if step size δ&lt;δ th , or the maximum iteration is reached. 
     b. Update (m,g) by 
               m     i   +   1       =       m   i     +       δ   ·         ∇   m     ⁢     E   Sim         max   ⁢            ∇   m     ⁢     E   Sim                  ⁢           ⁢   and                     g     i   +   1       =       g   i     +     δ   ·         ∇   g     ⁢     E   Sim         max   ⁢            ∇   g     ⁢     E   Sim                          
Step 3.
 
a. For each pixel location ξεΩ′⊂Ω impose constraint (1b) by m i+1 (ξ)←max(m i+1 (ξ)th low ) and m i+1 (ξ)←min(m i+1 (ξ),th high ).
 
b. For each pixel location ξεΩ impose constraint (1a) by
 
                 m     i   +   1       ⁡     (   ξ   )       ←         m     i   +   1       ⁡     (   ξ   )       ·          Ω            ∑     ξ   ∈   Ω       ⁢       m     i   +   1       ⁡     (   ξ   )                   
Step 4.
 
Compute φ m     i+1     ,g     i+1    and update E Sim . If it improves, i←i+1, go to Step 1, otherwise, decrease δ and go to Step 2.
 
     The alignment method can be applied to more than two images through serial or parallel application of the method involving subsequent alignment of an aligned image with a third image. Further, it can be applied to the output of multiple video images to create a series of aligned images along a time dimension. The time points can then be optionally aligned and fused into a mosaicked image for visualization. 
     The present invention further utilizes a novel algorithm called CKRX based on Kernel RX, which is a generalization of the Reed-Xiaoli (RX) algorithm. For instance, when the kernel distance function is defined as the dot product of the two vectors, Kernel RX is the same as RX. Its advantage lies in its flexibility over RX; however, it is significantly slower than RX. CKRX is a generalization of Kernel RX, i.e. CKRX is reduced to Kernel RX under some particular settings. 
     The basic idea in creating a CKRX is to first determine the clusters of the background points. Then, replace each point in the cluster with its cluster&#39;s center. After replacement, the number of unique points is the number of clusters, which can be very small comparing to the original point set. Although the total number of points does not change, the computation of the anomaly value can be simplified using only the unique cluster centers, which improves the speed by several orders of magnitude. 
     The present invention proposes to apply Deep Neural Network (DNN) techniques to further improve the chemical element classification and composition estimation performance in targets or anomalies. Two of the DNN techniques adapt to the element classification and chemical composition estimation problem are the Deep Belief Network (DBN) and Convolutional Neural Network (CNN). DNN techniques have the following advantages:
         Better capture of hierarchical feature representations   Ability to learn more complex behaviors   Better performance than conventional methods   Use distributed representations to learn the interactions of many different factors on different levels   Can learn from unlabeled data such as using the RBM pretraining method   Performance can scale up with the number of hidden layers and hidden nodes on fast GPUs       

     The present invention also allows operators to interact with target detection results via a user friendly graphical user interface. 
     Example 1 
     This is a demonstration of subpixel level registration errors with the two-step registration approach using actual Mars MASTCAM images (SOLDAY 188). 
     As shown in  FIGS. 3 a   )- 3   d ), using one of the MASTCAM stereo image pair (RGB images) to demonstrate the effectiveness of the two-step image registration approach. Referring to  FIG. 3 a   ), this stereo image is a partial image from the SOLDAY 188 data. It shows the left MASTCAM image which will be used as the reference image.  FIG. 3 b   ) shows the right MASTCAM image which is going to be aligned to the left camera image.  FIG. 3 c   ) shows the aligned image after the first step with RANSAC.  FIG. 3 d   ) shows final aligned image after the second step with diffeomorphic registration. 
     In order to show the effectiveness of the registration approach, first, the difference image between the aligned image and the left camera image in each of the two steps of the two-step registration approach is used. The difference images can be seen in  FIGS. 4 a   ) and  4   b ), respectively. It is realized that the registration errors can be easily noticed in the first step of registration with RANSAC whereas after the second step with diffeomorphic registration. The errors in the difference image can be hardly noticed. 
     In order to assess the performance of the two-step registration accuracy, a “pixel-distance” type measure is designed. In this measure, first, the SURF features in the reference and the aligned images in each step are found; then, the matching SURF features in the reference image and aligned image are found. The process is repeated for the pair of “reference image and RANSAC aligned image” and “reference image and final aligned image”. Finally, the norm values for each matching SURF feature pair are found. The average of the norm values is considered as a quantitative indicator that provides information about the registration performance. 
       FIGS. 5 a   ) and  5   b ) show the matching features in each step of the two-step registration approach.  FIG. 5 c   ) shows the resultant pixel distances in the matched SURF features in each step of the two-step registration approach. It can be clearly noticed that the second step of the two-step registration process reduces the registration errors to subpixel levels. 
     Example 2 
     A partial image section from one of the Middlebury stereo pair images, as described in the article mentioned above, “Representation Learning: A Review and New Perspectives,” by Yoshua Bengio, Aaron Courville, and Pascal Vincent, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, to examine the alignment improvement and assess its impact on the anomaly detection in each step of the two-step registration approach is used. 
       FIG. 6 a   ) shows the partial image from one of the cameras (left camera), and  FIG. 6 b   ) shows the right camera which is to be aligned to the reference image.  FIG. 6 c   ) shows the initial alignment with RANSAC which is the aligned image after the first step of the two-step image registration approach.  FIG. 6 d   ) shows the final aligned image with the diffeomorphic registration, which is the second step. Because it is hard to assess the alignment accuracy, the same pixel distance based measure to examine the registration errors is applied. 
       FIG. 7  shows the computed pixel distances in each step of the two-step registration approach. It can be noticed that the second step of the two-step registration approach further reduces the registration errors. 
     In  FIGS. 8 a   )- 8   c ), an example that precise registration has a considerable impact on the anomaly detection results yielding more accurate anomaly detections is shown. Again, a section from the Middlebury image used earlier is utilized. A section from this image where there are a few pixels that are quite different from the background pixels is selected. The selected image section that is used in the anomaly detection can be seen in  FIG. 8 a   ). It is the back left foot of the toy-alligator. The selected image section that will be used in the anomaly detection is shown in  FIG. 8 b   ). The pixels that correspond to the foot in the reference camera image are manually extracted. These pixels are considered the anomaly. 
       FIG. 8 c   ) shows the manually extracted ground truth map. In order to see the impact of the registration performance on the anomaly detection, the reference camera RGB image and the aligned images in each step of the two-step registration process separately are stacked into a 6-band multispectral image cube. That is by stacking the reference camera RGB image and the RANSAC-aligned image of the first step to obtain one multispectral image cube. By stacking the reference camera RGB image and the diffeomorphic aligned image of the second step, another multispectral image cube is obtained. Then, a RXD anomaly detection technique to both these multispectral image cubes separately is applied. Following the generation of anomaly score images, the extracted ground truth map to get the ROC curves to compare the detection performances in each of the two steps is used. 
     The resultant ROC curves can be seen in  FIG. 9 . From the ROC curves, it can be noticed that anomaly detection performance considerably improves with the second step of the two-step registration process which justifies our anticipation that improving the alignment of the stereo images with the proposed two-step image registration approach will have significant impacts on the anomaly detection or any analysis that involves the constructed multispectral image cube from the stereo images. 
     In the present invention, neural networks are used for target classification. 
     In a preliminary investigation, one of the DNN techniques known as Deep Belief Network (DBN) is applied for target classification in hyperspectral data. The hyperspectral image used in this example is called “NASA-KSC” image. The image corresponds to the mixed vegetation site over the Kennedy Space Center (KSC) in Florida. The image data was acquired by the National Aeronautics and Space Administration (NASA) Airborne Visible/Infrared Imaging Spectrometer instrument, on Mar. 23, 1996, as described in the article “Deep Learning-Based Classification of Hyperspectral Data,” Y. Chen, et. al.,  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , Vol. 7, No. 6, June 2014. AVIRIS acquires data in a range of 224 bands with wavelengths ranging from 0.4 μm to 2.5 μm. The KSC data has a spatial resolution of 18 m. After excluding water absorption and low signal-to noise ratio (SNR) bands, there are 176 spectral bands for classification. In the NASA-KSC image, there are 13 different land-cover classes available. It should be noted that only a small portion of the image has been tagged with the ground truth information and these pixels with the tagged ground truth information have been used in the classification study. The tagged pixels with ground truth information are shown in  FIG. 8 . 
     For the benchmark techniques, SVM (Support Vector Machine) and SAM (Spectral Angle Mapper) are applied. In SVM, LIBSVM toolbox is used with a kernel type of Radial Basis Function and automatically regularized support vector classification SVM method type (nu-SVC). In addition to using spectral information, local spatial information for each pixel (RGB bands of a local window of size 7×7) is extracted and transformed into a vector, then added to the end of the spectral information. The correct classification rates for the test data set are shown in Table 1 below. It can be seen that DBN and SVM results are very close to each other and both perform. 
     
       
         
               
             
               
               
               
             
               
               
               
               
             
           
               
                 TABLE 1 
               
             
             
               
                   
               
               
                 Classification performance for NASA-KSC 
               
             
          
           
               
                   
                   
                 Test set 
               
               
                   
                 Input data type 
                 (correct classification rate) 
               
               
                   
                   
               
             
          
           
               
                   
                 SAM 
                 Spectral 
                 0.7847 
               
               
                   
                 SVM 
                 Spectral 
                 0.9340 
               
               
                   
                 DBN 
                 Spectral 
                 0.9389 
               
               
                   
                 SVM 
                 Spectral + Spatial 
                 0.9709 
               
               
                   
                 DBN 
                 Spectral + Spatial 
                 0.9631 
               
               
                   
                   
               
             
          
         
       
     
     As mentioned above, Kernel RX, is a generalization of the Reed-Xiaoli (RX) algorithm. When the kernel distance function is defined as the dot product of two vectors, kernel RX is the same as RX. Its advantage lies in its flexibility over RX. However, it is significantly slower than RX. The present invention therefore utilizes a novel cluster kernel RX (CKRX) algorithm, which can perform fast approximation of kernel RX. CKRX is a generalization of kernel RX, meaning CKRX is reduced to kernel RX under some particular settings. The CKRX algorithm is below: 
     
       
         
               
             
               
               
             
           
               
                   
               
               
                 Algorithm CKRX 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                   
                 Input: Background x b  = [x 1 , x 2 , . . . , x M  ] , a testing pixel r 
               
               
                   
                 Output: The anomaly value v 
               
               
                   
                 Algorithm: 
               
               
                   
                  1. Perform clustering on X b  and get a set of clusters 
               
               
                   
                    C = {(z 1 ,s 2 ),(z 2 ,s 2 ), . . . ,(z m ,s m )} where z i   
               
               
                   
                    and s i  are center and size of i th  cluster. 
               
               
                   
                  2. Set v = WKRX (C,r). 
               
               
                   
                   
               
             
          
         
       
     
     WKRX is the weighted KRX algorithm: 
     
       
         
               
             
               
               
             
           
               
                   
               
             
             
               
                 Algorithm WKRX 
               
               
                 Input: Weighted points C = {(z 1 , s 1 ), (z 2 , s 2 ), . . . , (z m , s m )}, a testing point r 
               
               
                 Output: The anomaly value v 
               
               
                 Algorithm: 
               
             
          
           
               
                  1. 
                 Construct kernel matrix K, where K ij  = k(z i , z j ) and k is the kernel function. A 
               
               
                   
                 commonly used kernel is the Gaussian radial basis function (RBF) kernel 
               
               
                   
                 k(x, y) = exp ((−||x − y|| 2 )/c) 
               
               
                  2. 
                 
                   
                     
                       
                         
                           Set 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           
                             μ 
                             ^ 
                           
                         
                         , 
                         
                           = 
                           
                             μ 
                             = 
                             
                               
                                 Kw 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 where 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   w 
                                   i 
                                 
                               
                               = 
                               
                                 
                                   s 
                                   i 
                                 
                                 / 
                                 
                                   
                                     ∑ 
                                     
                                       i 
                                       = 
                                       1 
                                     
                                     m 
                                   
                                   ⁢ 
                                   
                                       
                                   
                                   ⁢ 
                                   
                                     s 
                                     i 
                                   
                                 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                  3. 
                 Set {circumflex over (K)} = K − μe T  − eμ T  + ew T  μe T  where e is an m × 1 vector. 
               
               
                  4. 
                 Perform eigen-decomposition. {circumflex over (K)}SV = VD where S is a diagonal matrix with S ii  = s i . 
               
               
                  5. 
                 Cut D and V to a length of t.  D  = D(1:t, 1:t),  V  = V (:, 1:t) where D(t + 1, t + 1) &lt; D(1, 1) × 10 −8   
               
               
                  6. 
                 Set  μ  = μ − ew T μ 
               
               
                  7. 
                 Set  γ  = γ − ew T γ where γ i  = k (z i , r) 
               
               
                  8. 
                 Set v = || D   −1   V   T  ( γ  −  μ )|| 2 . 
               
               
                   
               
             
          
         
       
     
       FIGS. 11 a   )- 11   c ) show an example of CKRX applied to a synthetic data set. The data model is a mixture of Gaussian functions, and there are 1000 data points. The kernel is a Gaussian kernel. The color in the image corresponds to the log of the anomaly value. The results using KRX, KRX with sub-sampling, and CKRX are shown in  FIGS. 11 a   ),  11   b ) and  11   c ), respectively. The number of the original data points is 1000, and the data point number in both sub-sampled KRX and CKRX is 50. From the image, the CKRX provides better approximation than sub-sampled KRX is observed. Also, the speed of these three algorithms is compared, and the result is shown in Table 2 below. The eigen-decomposition of the kernel matrix in CKRX is about 1/2000 of that in original KRX, which is a huge speed improvement. 
     
       
         
               
             
               
               
             
               
               
               
               
               
             
               
               
               
               
               
             
           
               
                 TABLE 2 
               
             
             
               
                   
               
               
                 Comparison of the speed of KRX, KRX with sub-sampling and CKRX 
               
             
          
           
               
                   
                 Algorithm 
               
             
          
           
               
                   
                   
                 KRX 
                 KRX 
                 CKRX 
               
               
                   
                 Time (s) 
                 (1000 points) 
                 (50 points) 
                 (50 clusters) 
               
               
                   
                   
               
             
          
           
               
                   
                 Construct Kernel 
                 0.1590 
                 0.0038 
                 0.0030 
               
               
                   
                 Eigen 
                 4.6511 
                 0.0018 
                 0.0023 
               
               
                   
                 Decomposition 
               
               
                   
                 Image Anomaly 
                 6.82 
                 0.62 
                 0.61 
               
               
                   
                   
               
             
          
         
       
     
     Another experiment was to use the Air Force hyperspectral image with PCA transformation and only 10 bands are kept.  FIGS. 12 a   ) and  12   b ) show the comparison of kernel RX with background sub-sampling (2×2), and CKRX. From  FIG. 12 b   ), we can see that CKRX performs better than kernel RX while the speed of CKRX is at least 2 times faster. The performance CKRX is better than KRX, but the speed improvement factor is more than 2 times. It should be noted that the CKRX of the present invention can be implemented in multi-core PCs. If a QuadCore PC is used, 4 times of the speed can be further improved, which is about 6 seconds, quite close to real-time.